[ncl-talk] Memory leak with grib2 files

[Dave Allured - NOAA Affiliate]

Kevin and Dave Brown,

Thanks so much for the fix and the development build.  We activated the
ncl_dev environment.  The NCL version string is what you said it should be.

The grib2 test program shows that the memory leak is fixed.  Have a good
weekend!

--Dave


On Fri, Jun 30, 2017 at 3:43 PM, Kevin Hallock <hallock at ucar.edu> wrote:

> Hi Dave A.,
>
> I just uploaded a new development build of NCL to the “ncar" conda
> channel. If you do not have Anaconda/Miniconda/“conda” installed already,
> please refer to our conda installation instructions
> <http://ncl.ucar.edu/Download/conda.shtml>.
>
> Once conda is installed (if it wasn’t already), could you please try
> running the following commands to create a new conda environment, activate
> it, and check the version of NCL:
> conda create -n ncl_dev -c ncar -c conda-forge ncl=dev
> source activate ncl_dev
> ncl -V
>
> The version string reported by NCL should be "dev_20170629-9b3b2ff”.
>
> Also, while the NCL package itself is on the “ncar” channel, all of its
> dependencies (NetCDF, HDF, GRIB, etc) are on “conda-forge”, which is why
> “-c conda-forge” is included in the “conda create …” command.
>
> Thanks,
> Kevin
>
> On Jun 29, 2017, at 6:23 PM, David Brown <dbrown at ucar.edu> wrote:
>
> Hopefully very soon. We will announce it.
> -dave
>
> On Thu, Jun 29, 2017 at 5:11 PM, Dave Allured - NOAA Affiliate
> <dave.allured at noaa.gov> wrote:
>
> Dave B,
>
> That's great.  Thank you for the quick response.
>
> Do you have a rough estimate for when the conda development branch access
> might be available?
>
> --Dave A.
>
>
> On Thu, Jun 29, 2017 at 4:08 PM, David Brown <dbrown at ucar.edu> wrote:
>
>
> Dave ,
> This leak has been fixed. If you need the new version, let us know. We
> are in the process of creating a system where a new build will be
> available the next day whenever a change has been made to the
> development branch of the code. This will be via the ncar conda
> channel.
> -dave
>
> On Thu, Jun 29, 2017 at 10:21 AM, David Brown <dbrown at ucar.edu> wrote:
>
> Hi Dave,
> We will check this out.
> -dave
>
> On Thu, Jun 29, 2017 at 10:09 AM, Dave Allured - NOAA Affiliate
> <dave.allured at noaa.gov> wrote:
>
> NCL team,
>
> When reading a series of grib2 files, I get a serious memory leak.  For
> each
> grib2 file of about 60 Mb size, it seems that about 15 Mb of user
> memory is
> consumed by addfile, and never released.
>
> Is this an NCL bug?  Is there a quick fix or a patch available?
>
> We tested similar scenarios with Netcdf file sets, and never saw a
> memory
> accumulation like this.
>
> I tested this with NCL 6.4.0 on two systems, and got identical
> behavior:
>
> * Mac OS El Capitan (10.11.6)
> * Linux, Centos 6.9.
>
> Following is a simple test case.  I suggest using any available grib2
> data
> set for your own testing.  I can upload some of these grib2 files if
> you
> can't duplicate the behavior with your own files.
>
> begin
>  base = "/Projects/Reforecast2/1985/198501/"
>  wildcard = "198501??00/[cp]??/latlon/hgt_pres_198501??00_???.grib2"
>
>  files  = systemfunc ("ls -1 " + base + wildcard)
>  nfiles = dimsizes (files)
>  print ("Number of files = " + nfiles)
>
>  do i = 0, nfiles-1
>    f = addfile (files(i), "r")
>    print (i + "  " + files(i))
>    delete (f)
>  end do
> end
>
> Thanks for looking into this.
>
> --Dave
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170630/fd4ca4a5/attachment.html