[ncl-talk] Memory leak with grib2 files

[Kevin Hallock]

Hi Dave A.,

I just uploaded a new development build of NCL to the “ncar" conda channel. If you do not have Anaconda/Miniconda/“conda” installed already, please refer to our conda installation instructions <http://ncl.ucar.edu/Download/conda.shtml>.

Once conda is installed (if it wasn’t already), could you please try running the following commands to create a new conda environment, activate it, and check the version of NCL:
conda create -n ncl_dev -c ncar -c conda-forge ncl=dev
source activate ncl_dev
ncl -V

The version string reported by NCL should be "dev_20170629-9b3b2ff”.

Also, while the NCL package itself is on the “ncar” channel, all of its dependencies (NetCDF, HDF, GRIB, etc) are on “conda-forge”, which is why “-c conda-forge” is included in the “conda create …” command.

Thanks,
Kevin

> On Jun 29, 2017, at 6:23 PM, David Brown <dbrown at ucar.edu> wrote:
> 
> Hopefully very soon. We will announce it.
> -dave
> 
> On Thu, Jun 29, 2017 at 5:11 PM, Dave Allured - NOAA Affiliate
> <dave.allured at noaa.gov> wrote:
>> Dave B,
>> 
>> That's great.  Thank you for the quick response.
>> 
>> Do you have a rough estimate for when the conda development branch access
>> might be available?
>> 
>> --Dave A.
>> 
>> 
>> On Thu, Jun 29, 2017 at 4:08 PM, David Brown <dbrown at ucar.edu> wrote:
>>> 
>>> Dave ,
>>> This leak has been fixed. If you need the new version, let us know. We
>>> are in the process of creating a system where a new build will be
>>> available the next day whenever a change has been made to the
>>> development branch of the code. This will be via the ncar conda
>>> channel.
>>> -dave
>>> 
>>> On Thu, Jun 29, 2017 at 10:21 AM, David Brown <dbrown at ucar.edu> wrote:
>>>> Hi Dave,
>>>> We will check this out.
>>>> -dave
>>>> 
>>>> On Thu, Jun 29, 2017 at 10:09 AM, Dave Allured - NOAA Affiliate
>>>> <dave.allured at noaa.gov> wrote:
>>>>> NCL team,
>>>>> 
>>>>> When reading a series of grib2 files, I get a serious memory leak.  For
>>>>> each
>>>>> grib2 file of about 60 Mb size, it seems that about 15 Mb of user
>>>>> memory is
>>>>> consumed by addfile, and never released.
>>>>> 
>>>>> Is this an NCL bug?  Is there a quick fix or a patch available?
>>>>> 
>>>>> We tested similar scenarios with Netcdf file sets, and never saw a
>>>>> memory
>>>>> accumulation like this.
>>>>> 
>>>>> I tested this with NCL 6.4.0 on two systems, and got identical
>>>>> behavior:
>>>>> 
>>>>> * Mac OS El Capitan (10.11.6)
>>>>> * Linux, Centos 6.9.
>>>>> 
>>>>> Following is a simple test case.  I suggest using any available grib2
>>>>> data
>>>>> set for your own testing.  I can upload some of these grib2 files if
>>>>> you
>>>>> can't duplicate the behavior with your own files.
>>>>> 
>>>>> begin
>>>>>  base = "/Projects/Reforecast2/1985/198501/"
>>>>>  wildcard = "198501??00/[cp]??/latlon/hgt_pres_198501??00_???.grib2"
>>>>> 
>>>>>  files  = systemfunc ("ls -1 " + base + wildcard)
>>>>>  nfiles = dimsizes (files)
>>>>>  print ("Number of files = " + nfiles)
>>>>> 
>>>>>  do i = 0, nfiles-1
>>>>>    f = addfile (files(i), "r")
>>>>>    print (i + "  " + files(i))
>>>>>    delete (f)
>>>>>  end do
>>>>> end
>>>>> 
>>>>> Thanks for looking into this.
>>>>> 
>>>>> --Dave
>> 
>> 
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> 
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170630/ad982fe2/attachment.html