[ncl-talk] Reading WRF compressed files -- how to?

Barry Lynn barry.h.lynn at gmail.com
Wed Apr 10 01:06:16 MDT 2019 

Hello:

I renamed that the file I previous attached with *nc, but it still hangs up.

It did so with and without these two lines commented out.

;setfileoption("nc","format","netcdf4")

 print("got to here 1")

;setfileoption("nc","CompressionLevel",5)

Barry

On Wed, Apr 10, 2019 at 6:16 AM Dennis Shea <shea at ucar.edu> wrote:

> No.
>
> Sent from my iPhone
>
> On Apr 9, 2019, at 8:49 PM, Barry Lynn <barry.h.lynn at gmail.com> wrote:
>
> Hi Dennis, Rick, Alan.
>
> Thank you for looking into this.
>
> It was noted that NCL knows how to uncompress the file when reading it.
> Does this imply that the read takes a lot longer?
>
> Barry
>
> On Tue, Apr 9, 2019 at 9:12 PM Dennis Shea <shea at ucar.edu> wrote:
>
>> Yikes!!  Rick and Alan are much quicker than me!!!  Probably smarter
>> too!! Not nearly as good-looking though!
>>
>> Still, I'll add what I was writing:
>> ============================================
>> Hi Barry,
>>
>> For reading a compressed file, the user need not set any file
>> information. NCL detects the file type and if compression is being used.
>> 'knows' how to uncompress.
>> This holds for 6.3.0 (maybe earlier) ===> 6.5.0
>>
>> ======
>> *%> ncl*
>>  NCAR Command Language Version* 6.3.0*
>>
>> ncl 0> f = *addfile*("wrfout_d03_2018-01-05_00_00_00*.nc*","r")   ; or
>> *.nc4*
>> ncl 1> print(f)    ; <=== see attached
>> =========
>>
>>
>> %> *ncdump -k* wrfout_d03_2018-01-05_00_00_00
>> netCDF-4
>>
>> %> *ncl -V*
>> NCL 6.5.0                   [ Also: NCL 6.3.0 ]
>>
>> *%> ncl_filedump* wrfout_d03_2018-01-05_00_00_00.nc >&!
>> out.wrfout_nc4_compressed
>> *%> less* out.wrfout_nc4_compressed
>>
>> See attached
>>
>> Note:*  Chunking Info*:
>>
>> Variable: XLONG
>> [ 1 <Time> x 237 <south_north> x 150 <west_east> ]
>>
>> Variable: ZNW
>> Chunking Info:      [ 1 <Time> x 44 <bottom_top> ]
>>
>> Variable: ZS
>> Chunking Info:      [ 1 <Time> x 4 <soil_layers_stag> ]
>>
>>
>>
>> On Tue, Apr 9, 2019 at 11:50 AM Rick Brownrigg <brownrig at ucar.edu> wrote:
>>
>>> Yes, to add to what Alan just said, I've been able to open that file
>>> just fine with a simple addfile().  However, if I say
>>>
>>>    f = addfile(.....)
>>>   print(f)
>>>
>>> and then quit out of the pager before all the output has been written to
>>> screen, the code hands indefinitely.  Very odd behavior, and I've not seen
>>> that with other files...Still looking into this one.
>>>
>>>
>>>
>>> On Tue, Apr 9, 2019 at 11:46 AM Brammer, Alan P <abrammer at albany.edu>
>>> wrote:
>>>
>>>> To read a compressed NetCDF you shouldn't need to do anything special.
>>>> Just use the addfile() as normal, no need to specify the setfileoption
>>>> parameters, they will only work on creating (maybe writing) to a file.
>>>>
>>>> In your wrf namelist you can tell wrf to put the ".nc" suffix on the
>>>> file names, so it's more obvious what file it is.  Otherwise just add it as
>>>> a suffix in the ncl addfile command so NCL knows what kind of file it's
>>>> dealing with.
>>>>
>>>> ncl 0> filename = "wrfout_d03_2018-01-05_00_00_00"
>>>> ncl 1> print(addfile(filename+".nc", "r"))
>>>>
>>>>
>>>>
>>>> Alan
>>>>
>>>>
>>>> On Tue, Apr 9, 2019 at 11:28 AM Barry Lynn <barry.h.lynn at gmail.com>
>>>> wrote:
>>>>
>>>>>  wrfout_d03_2018-01-05_00:00:00
>>>>> <https://drive.google.com/file/d/1_C3tTQj3rwnfiCM4TBUYPdD8jOwUynt8/view?usp=drive_web>
>>>>> Please try this one. It is definitely 1 time step of output from WRF.
>>>>>
>>>>> On Tue, Apr 9, 2019 at 8:11 PM Rick Brownrigg <brownrig at ucar.edu>
>>>>> wrote:
>>>>>
>>>>>> Hi Barry,
>>>>>>
>>>>>> This looks like a truncated file?  Its only 336K, and the name is
>>>>>> suspect.?
>>>>>>
>>>>>> RB
>>>>>>
>>>>>> On Tue, Apr 9, 2019 at 10:55 AM Barry Lynn <barry.h.lynn at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> begin
>>>>>>>
>>>>>>>
>>>>>>> ; filename="wrfout_d03_2018-01-05_00:00:00"
>>>>>>>
>>>>>>>   filename="wrfout_compresed.one_time_step.nc.pid19740.ncks.tmp"
>>>>>>>
>>>>>>>
>>>>>>> ;
>>>>>>>
>>>>>>> ;   mpdatabaseversion= "Earth..4"
>>>>>>>
>>>>>>>  setfileoption("nc","format","netcdf4")
>>>>>>>
>>>>>>>  print("got to here 1")
>>>>>>>
>>>>>>> setfileoption("nc","CompressionLevel",5)
>>>>>>>
>>>>>>>  print("got to here 2")
>>>>>>>
>>>>>>> a = addfile(filename,"r")
>>>>>>>
>>>>>>>
>>>>>>> IT DOES NOT PASS THE LINE ABOVE (ncl 6.5)
>>>>>>>
>>>>>>>  print("got to here 3")
>>>>>>>
>>>>>>>  end
>>>>>>>
>>>>>>> On Tue, Apr 9, 2019 at 7:12 PM Barry Lynn <barry.h.lynn at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi:
>>>>>>>>
>>>>>>>> I am working on it.  It will take a bit of time, as I am missing a
>>>>>>>> library.
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>> On Tue, Apr 9, 2019 at 7:09 PM Rick Brownrigg <brownrig at ucar.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Barry,
>>>>>>>>>
>>>>>>>>> Are you able to send me that file?
>>>>>>>>>
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>> On Tue, Apr 9, 2019 at 12:40 AM Barry Lynn <barry.h.lynn at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello:
>>>>>>>>>>
>>>>>>>>>> I am at a loss trying to find an example to read a WRF file
>>>>>>>>>> defined this way.
>>>>>>>>>>
>>>>>>>>>> This build of WRF will use NETCDF4 with HDF5 compression
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I tried the following, but the ncl program just hangs up after
>>>>>>>>>> passing "here 2"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  setfileoption("nc","format","netcdf4")
>>>>>>>>>>
>>>>>>>>>>  print("got to here 1")
>>>>>>>>>>
>>>>>>>>>> setfileoption("nc","CompressionLevel",5)
>>>>>>>>>>
>>>>>>>>>>  print("got to here 2")
>>>>>>>>>>
>>>>>>>>>> a = addfile(filename,"r")
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Barry H. Lynn, Ph.D
>>>>>>>>>> Senior Associate Scientist, Lecturer,
>>>>>>>>>> The Institute of the Earth Science,
>>>>>>>>>> The Hebrew University of Jerusalem,
>>>>>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>>>>>> Tel: 972 547 231 170
>>>>>>>>>> Fax: (972)-25662581
>>>>>>>>>>
>>>>>>>>>> C.E.O, Weather It Is, LTD
>>>>>>>>>> Weather and Climate Focus
>>>>>>>>>> http://weather-it-is.com
>>>>>>>>>> Jerusalem, Israel
>>>>>>>>>> Local: 02 930 9525
>>>>>>>>>> Cell: 054 7 231 170
>>>>>>>>>> Int-IS: x972 2 930 9525
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> ncl-talk mailing list
>>>>>>>>>> ncl-talk at ucar.edu
>>>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Barry H. Lynn, Ph.D
>>>>>>>> Senior Associate Scientist, Lecturer,
>>>>>>>> The Institute of the Earth Science,
>>>>>>>> The Hebrew University of Jerusalem,
>>>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>>>> Tel: 972 547 231 170
>>>>>>>> Fax: (972)-25662581
>>>>>>>>
>>>>>>>> C.E.O, Weather It Is, LTD
>>>>>>>> Weather and Climate Focus
>>>>>>>> http://weather-it-is.com
>>>>>>>> Jerusalem, Israel
>>>>>>>> Local: 02 930 9525
>>>>>>>> Cell: 054 7 231 170
>>>>>>>> Int-IS: x972 2 930 9525
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Barry H. Lynn, Ph.D
>>>>>>> Senior Associate Scientist, Lecturer,
>>>>>>> The Institute of the Earth Science,
>>>>>>> The Hebrew University of Jerusalem,
>>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>>> Tel: 972 547 231 170
>>>>>>> Fax: (972)-25662581
>>>>>>>
>>>>>>> C.E.O, Weather It Is, LTD
>>>>>>> Weather and Climate Focus
>>>>>>> http://weather-it-is.com
>>>>>>> Jerusalem, Israel
>>>>>>> Local: 02 930 9525
>>>>>>> Cell: 054 7 231 170
>>>>>>> Int-IS: x972 2 930 9525
>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> Barry H. Lynn, Ph.D
>>>>> Senior Associate Scientist, Lecturer,
>>>>> The Institute of the Earth Science,
>>>>> The Hebrew University of Jerusalem,
>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>> Tel: 972 547 231 170
>>>>> Fax: (972)-25662581
>>>>>
>>>>> C.E.O, Weather It Is, LTD
>>>>> Weather and Climate Focus
>>>>> http://weather-it-is.com
>>>>> Jerusalem, Israel
>>>>> Local: 02 930 9525
>>>>> Cell: 054 7 231 170
>>>>> Int-IS: x972 2 930 9525
>>>>>
>>>>> _______________________________________________
>>>>> ncl-talk mailing list
>>>>> ncl-talk at ucar.edu
>>>>> List instructions, subscriber options, unsubscribe:
>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>
>>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
>
> --
> Barry H. Lynn, Ph.D
> Senior Associate Scientist, Lecturer,
> The Institute of the Earth Science,
> The Hebrew University of Jerusalem,
> Givat Ram, Jerusalem 91904, Israel
> Tel: 972 547 231 170
> Fax: (972)-25662581
>
> C.E.O, Weather It Is, LTD
> Weather and Climate Focus
> http://weather-it-is.com
> Jerusalem, Israel
> Local: 02 930 9525
> Cell: 054 7 231 170
> Int-IS: x972 2 930 9525
>
>

-- 
Barry H. Lynn, Ph.D
Senior Associate Scientist, Lecturer,
The Institute of the Earth Science,
The Hebrew University of Jerusalem,
Givat Ram, Jerusalem 91904, Israel
Tel: 972 547 231 170
Fax: (972)-25662581

C.E.O, Weather It Is, LTD
Weather and Climate Focus
http://weather-it-is.com
Jerusalem, Israel
Local: 02 930 9525
Cell: 054 7 231 170
Int-IS: x972 2 930 9525
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190410/97373bac/attachment.html>

More information about the ncl-talk mailing list