[ncl-talk] Reading WRF compressed files -- how to?

Tue Apr 9 12:12:03 MDT 2019

Yikes!!  Rick and Alan are much quicker than me!!!  Probably smarter too!!
Not nearly as good-looking though!

Still, I'll add what I was writing:
============================================
Hi Barry,

For reading a compressed file, the user need not set any file information.
NCL detects the file type and if compression is being used. 'knows' how to
uncompress.
This holds for 6.3.0 (maybe earlier) ===> 6.5.0

======
*%> ncl*
 NCAR Command Language Version* 6.3.0*

ncl 0> f = *addfile*("wrfout_d03_2018-01-05_00_00_00*.nc*","r")   ; or
*.nc4*
ncl 1> print(f)    ; <=== see attached
=========

%> *ncdump -k* wrfout_d03_2018-01-05_00_00_00
netCDF-4

%> *ncl -V*
NCL 6.5.0                   [ Also: NCL 6.3.0 ]

*%> ncl_filedump* wrfout_d03_2018-01-05_00_00_00.nc >&!
out.wrfout_nc4_compressed
*%> less* out.wrfout_nc4_compressed

See attached

Note:*  Chunking Info*:

Variable: XLONG
[ 1 <Time> x 237 <south_north> x 150 <west_east> ]

Variable: ZNW
Chunking Info:      [ 1 <Time> x 44 <bottom_top> ]

Variable: ZS
Chunking Info:      [ 1 <Time> x 4 <soil_layers_stag> ]

On Tue, Apr 9, 2019 at 11:50 AM Rick Brownrigg <brownrig at ucar.edu> wrote:

> Yes, to add to what Alan just said, I've been able to open that file just
> fine with a simple addfile().  However, if I say
>
>    f = addfile(.....)
>   print(f)
>
> and then quit out of the pager before all the output has been written to
> screen, the code hands indefinitely.  Very odd behavior, and I've not seen
> that with other files...Still looking into this one.
>
>
>
> On Tue, Apr 9, 2019 at 11:46 AM Brammer, Alan P <abrammer at albany.edu>
> wrote:
>
>> To read a compressed NetCDF you shouldn't need to do anything special.
>> Just use the addfile() as normal, no need to specify the setfileoption
>> parameters, they will only work on creating (maybe writing) to a file.
>>
>> In your wrf namelist you can tell wrf to put the ".nc" suffix on the file
>> names, so it's more obvious what file it is.  Otherwise just add it as a
>> suffix in the ncl addfile command so NCL knows what kind of file it's
>> dealing with.
>>
>> ncl 0> filename = "wrfout_d03_2018-01-05_00_00_00"
>> ncl 1> print(addfile(filename+".nc", "r"))
>>
>>
>>
>> Alan
>>
>>
>> On Tue, Apr 9, 2019 at 11:28 AM Barry Lynn <barry.h.lynn at gmail.com>
>> wrote:
>>
>>>  wrfout_d03_2018-01-05_00:00:00
>>> <https://drive.google.com/file/d/1_C3tTQj3rwnfiCM4TBUYPdD8jOwUynt8/view?usp=drive_web>
>>> Please try this one. It is definitely 1 time step of output from WRF.
>>>
>>> On Tue, Apr 9, 2019 at 8:11 PM Rick Brownrigg <brownrig at ucar.edu> wrote:
>>>
>>>> Hi Barry,
>>>>
>>>> This looks like a truncated file?  Its only 336K, and the name is
>>>> suspect.?
>>>>
>>>> RB
>>>>
>>>> On Tue, Apr 9, 2019 at 10:55 AM Barry Lynn <barry.h.lynn at gmail.com>
>>>> wrote:
>>>>
>>>>> begin
>>>>>
>>>>>
>>>>> ; filename="wrfout_d03_2018-01-05_00:00:00"
>>>>>
>>>>>   filename="wrfout_compresed.one_time_step.nc.pid19740.ncks.tmp"
>>>>>
>>>>>
>>>>> ;
>>>>>
>>>>> ;   mpdatabaseversion= "Earth..4"
>>>>>
>>>>>  setfileoption("nc","format","netcdf4")
>>>>>
>>>>>  print("got to here 1")
>>>>>
>>>>> setfileoption("nc","CompressionLevel",5)
>>>>>
>>>>>  print("got to here 2")
>>>>>
>>>>> a = addfile(filename,"r")
>>>>>
>>>>>
>>>>> IT DOES NOT PASS THE LINE ABOVE (ncl 6.5)
>>>>>
>>>>>  print("got to here 3")
>>>>>
>>>>>  end
>>>>>
>>>>> On Tue, Apr 9, 2019 at 7:12 PM Barry Lynn <barry.h.lynn at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi:
>>>>>>
>>>>>> I am working on it.  It will take a bit of time, as I am missing a
>>>>>> library.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> On Tue, Apr 9, 2019 at 7:09 PM Rick Brownrigg <brownrig at ucar.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Barry,
>>>>>>>
>>>>>>> Are you able to send me that file?
>>>>>>>
>>>>>>> Rick
>>>>>>>
>>>>>>> On Tue, Apr 9, 2019 at 12:40 AM Barry Lynn <barry.h.lynn at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello:
>>>>>>>>
>>>>>>>> I am at a loss trying to find an example to read a WRF file defined
>>>>>>>> this way.
>>>>>>>>
>>>>>>>> This build of WRF will use NETCDF4 with HDF5 compression
>>>>>>>>
>>>>>>>>
>>>>>>>> I tried the following, but the ncl program just hangs up after
>>>>>>>> passing "here 2"
>>>>>>>>
>>>>>>>>
>>>>>>>>  setfileoption("nc","format","netcdf4")
>>>>>>>>
>>>>>>>>  print("got to here 1")
>>>>>>>>
>>>>>>>> setfileoption("nc","CompressionLevel",5)
>>>>>>>>
>>>>>>>>  print("got to here 2")
>>>>>>>>
>>>>>>>> a = addfile(filename,"r")
>>>>>>>>
>>>>>>>> --
>>>>>>>> Barry H. Lynn, Ph.D
>>>>>>>> Senior Associate Scientist, Lecturer,
>>>>>>>> The Institute of the Earth Science,
>>>>>>>> The Hebrew University of Jerusalem,
>>>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>>>> Tel: 972 547 231 170
>>>>>>>> Fax: (972)-25662581
>>>>>>>>
>>>>>>>> C.E.O, Weather It Is, LTD
>>>>>>>> Weather and Climate Focus
>>>>>>>> http://weather-it-is.com
>>>>>>>> Jerusalem, Israel
>>>>>>>> Local: 02 930 9525
>>>>>>>> Cell: 054 7 231 170
>>>>>>>> Int-IS: x972 2 930 9525
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ncl-talk mailing list
>>>>>>>> ncl-talk at ucar.edu
>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Barry H. Lynn, Ph.D
>>>>>> Senior Associate Scientist, Lecturer,
>>>>>> The Institute of the Earth Science,
>>>>>> The Hebrew University of Jerusalem,
>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>> Tel: 972 547 231 170
>>>>>> Fax: (972)-25662581
>>>>>>
>>>>>> C.E.O, Weather It Is, LTD
>>>>>> Weather and Climate Focus
>>>>>> http://weather-it-is.com
>>>>>> Jerusalem, Israel
>>>>>> Local: 02 930 9525
>>>>>> Cell: 054 7 231 170
>>>>>> Int-IS: x972 2 930 9525
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Barry H. Lynn, Ph.D
>>>>> Senior Associate Scientist, Lecturer,
>>>>> The Institute of the Earth Science,
>>>>> The Hebrew University of Jerusalem,
>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>> Tel: 972 547 231 170
>>>>> Fax: (972)-25662581
>>>>>
>>>>> C.E.O, Weather It Is, LTD
>>>>> Weather and Climate Focus
>>>>> http://weather-it-is.com
>>>>> Jerusalem, Israel
>>>>> Local: 02 930 9525
>>>>> Cell: 054 7 231 170
>>>>> Int-IS: x972 2 930 9525
>>>>>
>>>>>
>>>
>>> --
>>> Barry H. Lynn, Ph.D
>>> Senior Associate Scientist, Lecturer,
>>> The Institute of the Earth Science,
>>> The Hebrew University of Jerusalem,
>>> Givat Ram, Jerusalem 91904, Israel
>>> Tel: 972 547 231 170
>>> Fax: (972)-25662581
>>>
>>> C.E.O, Weather It Is, LTD
>>> Weather and Climate Focus
>>> http://weather-it-is.com
>>> Jerusalem, Israel
>>> Local: 02 930 9525
>>> Cell: 054 7 231 170
>>> Int-IS: x972 2 930 9525
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190409/a78595cf/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out.wrfout_nc4_compressed
Type: application/octet-stream
Size: 186857 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190409/a78595cf/attachment.obj>