[ncl-talk] Problem running ncl_convert2nc (Error unpacking GRIB record section 4)
Ligia Bernardet - NOAA Affiliate
ligia.bernardet at noaa.gov
Wed Dec 28 13:15:58 MST 2016
Hi Dennis,
I also had the problem with the CDO not being built with GRIB support, but
the NOAA HPC sys adm rebuilt it yesterday and it started working.
The variable name is indeed nonsensical, and it is weird that CDO
interprets the data as 1 variable at 18 times (should be 18 variables at 1
time). This is undesirable but tolerable, since I know what the variables
are. This file was produced by the NCEP Unified Postprocessor (UPP) and I
have the control file that was used to generate it, so I can see which
variables are selected for output. They are indeed Brightness Temperatures
(K) produced by a forecast model to simulated various channels of various
satellites. Offline metadata is dangerous, but I guess I have to resort to
it for now.
Thanks,
Ligia
On Wed, Dec 28, 2016 at 1:06 PM, Dennis Shea <shea at ucar.edu> wrote:
> Initially, I did try the CDO on my local linux box but I got
>
> %> cdo -f nc copy gonzalo08l.2014101412 <(201)%20410-1412>.hwrfsat.global.0p25.f024.grb2
> gon_CDO.nc
>
> Error (gribapiScanTimestep1) : GRIB_API support not compiled in!
>
> ---
> I guess my local version needs an upgrade.
>
> ---
> I did try the above on another machine and got
>
> /glade/p/ncldev/data/grib2>ncdump -h gon_CDO.nc
>
> netcdf gon_CDO {
> dimensions:
> lon = 1440 ;
> lat = 721 ;
> time = UNLIMITED ; // (18 currently)
> variables:
> double lon(lon) ;
> lon:standard_name = "longitude" ;
> lon:long_name = "longitude" ;
> lon:units = "degrees_east" ;
> lon:axis = "X" ;
> double lat(lat) ;
> lat:standard_name = "latitude" ;
> lat:long_name = "latitude" ;
> lat:units = "degrees_north" ;
> lat:axis = "Y" ;
> double time(time) ;
> time:standard_name = "time" ;
> time:units = "hours since 2014-10-14 12:00:00" ;
> time:calendar = "proleptic_gregorian" ;
> float param7.5.0(time, lat, lon) ;
> param7.5.0:_FillValue = -9.e+33f ;
> param7.5.0:missing_value = -9.e+33f ;
>
> // global attributes:
> :CDI = "Climate Data Interface version 1.6.3 (
> http://code.zmaw.de/projects/cdi)" ;
> :Conventions = "CF-1.4" ;
> :history = "Wed Dec 28 12:04:14 2016: cdo -f nc copy gonzalo08l.
> 2014101412 <(201)%20410-1412>.hwrfsat.global.0p25.f024.grb2 gon_CDO.nc" ;
> :institution = "National Centers for Environmental Prediction" ;
> :CDO = "Climate Data Operators version 1.6.3 (
> http://code.zmaw.de/projects/cdo)" ;
> }
>
>
> The CDO returns the variable with a nonsensical name:
>
> float param7.5.0(time, lat, lon) ;
>
> but no units or long_name .... How do you know what it is?
> I am sure the numbers are unpacked correctly ... but ... without using
> some external information, how do you know to what 'param7.5.0' refers?
>
> ===
> Combining the CDO file with NCL naming (via an appropriate internal lookup
> table) you would get:
>
> float BRTEMP_P32_GLL0 ( lat_0, lon_0 )
> long_name : Brightness temperature
> units : K
>
> param7.5.0 ===> BRTEMP_P32_GLL0
>
> and the attributes should be:
>
> long_name : Brightness temperature
> units : K
>
> D
>
> On Wed, Dec 28, 2016 at 11:04 AM, Ligia Bernardet - NOAA Affiliate <
> ligia.bernardet at noaa.gov> wrote:
>
>> Dennis,
>>
>> Thanks for your willingness to help.
>>
>> I was able to use CDO to convert the GRIB2 file to NetCDF. All 18 fields
>> are in the output file (interestingly, it appears as one variable at 18
>> time levels, which is definitely not the case in the GRIB file ). I was
>> able to open the resulting NetCDF file in NCL and manipulate it. So, I am
>> on my way to get the results I need.
>>
>> But if you guys get this working in NCL directly, let me know.
>>
>> Thanks again,
>> Ligia
>>
>> On Wed, Dec 28, 2016 at 8:07 AM, Dennis Shea <shea at ucar.edu> wrote:
>>
>>> Hi Ligea
>>>
>>> FYI ... a tool unrelated to NCL also fails.
>>>
>>> ------
>>> There is another tool available from NCEP: wgrib2
>>>
>>> http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/
>>>
>>> There is an option to create netCDF
>>>
>>> http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/netcdf.html
>>>
>>> ===
>>> When I use this tool on the grib2 file you sent, it also fails:
>>>
>>> %> wgrib2 gonzalo08l.2014101412 <(201)%20410-1412>.hwrfsat.global.0p25.f024.grb2
>>> -netcdf gon.nc
>>>
>>> ---
>>> The wgrib2 tool prints many warning lines like:
>>>
>>> code_table_4.5a: product definition template #32 not supported
>>> code_table_4.5b: product definition template #32 not supported
>>> ....
>>> code_table_4.5b: product definition template #32 not supported
>>> 1:0:d=2014101412 <(201)%20410-1412>:var discipline=0 master_table=8
>>> parmcat=5 parm=7:no_level:24 hour fcst:
>>> code_table_4.5a: product definition template #32 not supported
>>> ...
>>> code_table_4.5b: product definition template #32 not supported
>>> 2:279767:d=2014101412 <(201)%20410-1412>:var discipline=0
>>> master_table=8 parmcat=5 parm=7:no_level:24 hour fcst:
>>> code_table_4.5a: product definition template #32 not supported
>>> ...
>>>
>>> etc, etc
>>>
>>> Like 'ncl_convert2nc', wgrib2 produces just one variable within the
>>> output netCDF. Specifically,
>>>
>>>
>>> %> ncddump -h gon.nc
>>>
>>> netcdf gon {
>>> dimensions:
>>> latitude = 721 ;
>>> longitude = 1440 ;
>>> time = UNLIMITED ; // (1 currently)
>>> variables:
>>> double latitude(latitude) ;
>>> latitude:units = "degrees_north" ;
>>> latitude:long_name = "latitude" ;
>>> double longitude(longitude) ;
>>> longitude:units = "degrees_east" ;
>>> longitude:long_name = "longitude" ;
>>> double time(time) ;
>>> time:units = "seconds since 1970-01-01 00:00:00.0 0:00" ;
>>> time:long_name = "verification time generated by wgrib2 function
>>> verftime()" ;
>>> time:reference_time = 1413288000. ;
>>> time:reference_time_type = 3 ;
>>> time:reference_date = "2014.10.14 12:00:00 UTC" ;
>>> time:reference_time_description = "forecast or accumulated,
>>> reference date is fixed" ;
>>> time:time_step_setting = "auto" ;
>>> time:time_step = 0. ;
>>> float var0_5_7_no_level(time, latitude, longitude) ;
>>> var0_5_7_no_level:_FillValue = 9.999e+20f ;
>>> var0_5_7_no_level:short_name = "var0_5_7_no_level" ;
>>> var0_5_7_no_level:long_name = "desc" ;
>>> var0_5_7_no_level:level = "no_level" ;
>>> var0_5_7_no_level:units = "unit" ;
>>>
>>> // global attributes:
>>> :Conventions = "COARDS" ;
>>> :History = "created by wgrib2" ;
>>> :GRIB2_grid_template = 0 ;
>>> ==========================================
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Dec 27, 2016 at 5:01 PM, Dennis Shea <shea at ucar.edu> wrote:
>>>
>>>> 6.3.0 will unpack one variable also. The name of the variable will be
>>>> different because (! wild guess !) the needed table was not complete or
>>>> available.
>>>>
>>>> GRIB is a bit of a 'moving target' :-(
>>>>
>>>> I have opened a JIRA ticket: NCL-2533
>>>>
>>>> Unfortunately, this is not a great week to get things done.
>>>>
>>>> Maybe, an IO developer will look?
>>>>
>>>> D
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Dec 27, 2016 at 4:48 PM, Ligia Bernardet - NOAA Affiliate <
>>>> ligia.bernardet at noaa.gov> wrote:
>>>>
>>>>> Hi Dennis,
>>>>>
>>>>> There are 18 fields in the GRIB2 file. In the NetCDF file you sent
>>>>> back to me there is 1.
>>>>> But more importantly, I need to find a way to do this myself, so I can
>>>>> process the multiple files I have.
>>>>>
>>>>> The machine I am using, a NOAA's R&D platform, I was able to find NCL
>>>>> v6.3. Would this version be usable to read this GRIB2 file, until a newer
>>>>> version is available?
>>>>>
>>>>> Thanks,
>>>>> Ligia
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Dec 27, 2016 at 4:11 PM, Dennis Shea <shea at ucar.edu> wrote:
>>>>>
>>>>>> Attached is a gzip'd version of the netCDF file.
>>>>>>
>>>>>> The 'soon-to-be-released' beta 6.4.0 returns a warning message. I
>>>>>> will open a JIRA ticket.
>>>>>> Note: I believe the numbers are correct.
>>>>>>
>>>>>> =========
>>>>>>
>>>>>> NCAR Command Language Version 6.4.0 -27Dec2016_0148
>>>>>>
>>>>>> warning:NclGRIB2: BRTEMP_P32 contains records that NCL cannot
>>>>>> currently differentiate. One or more records will be ignored.
>>>>>>
>>>>>> Variable: f
>>>>>> Type: file
>>>>>> filename: gonzalo08l.2014101412 <(201)%20410-1412>.hwrfsat.
>>>>>> global.0p25.f024
>>>>>> path: /Users/shea/Data/GRIB/gonzalo08l.2014101412
>>>>>> <(201)%20410-1412>.hwrfsat.global.0p25.f024.grb2
>>>>>> file global attributes:
>>>>>> dimensions:
>>>>>> lat_0 = 721
>>>>>> lon_0 = 1440
>>>>>> variables:
>>>>>> float BRTEMP_P32_GLL0 ( lat_0, lon_0 )
>>>>>> center : US National Weather Service - NCEP (WMC)
>>>>>> production_status : Operational products
>>>>>> long_name : Brightness temperature
>>>>>> units : K
>>>>>> _FillValue : 1e+20
>>>>>> grid_type : Latitude/longitude
>>>>>> parameter_discipline_and_category : Meteorological
>>>>>> products, Long wave radiation
>>>>>> parameter_template_discipline_category_number : (
>>>>>> 32, 0, 5, 7 )
>>>>>> forecast_time : 0
>>>>>> forecast_time_units : hours
>>>>>> initial_time : 10/14/2014 (12:00)
>>>>>>
>>>>>> float lat_0 ( lat_0 )
>>>>>> long_name : latitude
>>>>>> grid_type : Latitude/Longitude
>>>>>> units : degrees_north
>>>>>> Dj : 0.25
>>>>>> Di : 0.25
>>>>>> Lo2 : 359.75
>>>>>> La2 : -90
>>>>>> Lo1 : 0
>>>>>> La1 : 90
>>>>>>
>>>>>> float lon_0 ( lon_0 )
>>>>>> long_name : longitude
>>>>>> grid_type : Latitude/Longitude
>>>>>> units : degrees_east
>>>>>> Dj : 0.25
>>>>>> Di : 0.25
>>>>>> Lo2 : 359.75
>>>>>> La2 : -90
>>>>>> Lo1 : 0
>>>>>> La1 : 90
>>>>>>
>>>>>> =========
>>>>>> The min and max are:
>>>>>>
>>>>>> (0) Brightness temperature (K) : min=184.67 max=319.49
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Dec 27, 2016 at 1:13 PM, Ligia Bernardet - NOAA Affiliate <
>>>>>> ligia.bernardet at noaa.gov> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have transferred file gonzalo08l.2014101412 <(201)%20410-1412>
>>>>>>> .hwrfsat.global.0p25.f024.grb2 to /ftp/incoming.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ligia
>>>>>>>
>>>>>>> On Tue, Dec 27, 2016 at 1:06 PM, Dennis Shea <shea at ucar.edu> wrote:
>>>>>>>
>>>>>>>> [1]
>>>>>>>> 6.1.2 (Feb. 2013) is almost 4 years old. Much has happened since.
>>>>>>>>
>>>>>>>> [2]
>>>>>>>> To be able to determine the problem or if the current version of
>>>>>>>> NCL has addressed the issue, we would need the file.
>>>>>>>>
>>>>>>>> ftp ftp.cgd.ucar.edu
>>>>>>>> anonymous
>>>>>>>> your_email
>>>>>>>> cd incoming
>>>>>>>> prompt
>>>>>>>> put gonzalo08l.2014101412 <%28201%29%20410-1412>.hwrfsat.
>>>>>>>> global.0p25.f024.grb2
>>>>>>>> quit
>>>>>>>>
>>>>>>>> We can not look at /ftp/incoming, so ....
>>>>>>>>
>>>>>>>> Let ncl-talk know when the file has been transferred.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Dec 27, 2016 at 12:56 PM, Ligia Bernardet - NOAA Affiliate
>>>>>>>> <ligia.bernardet at noaa.gov> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I would like to use convert a GRIB2 file generated by NCEP's
>>>>>>>>> Hurricane WRF (HWRF) model to a NetCDF file. However, I am encountering the
>>>>>>>>> error below. Any suggestions?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ligia
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *ncl_convert2nc gonzalo08l.2014101412
>>>>>>>>> <(201)%20410-1412>.hwrfsat.global.0p25.f024.grb2*
>>>>>>>>> Processing file: gonzalo08l.2014101412 <(201)%20410-1412>.hwrfsat.
>>>>>>>>> global.0p25.f024.grb2...
>>>>>>>>> Copyright (C) 1995-2013 - All Rights Reserved
>>>>>>>>> University Corporation for Atmospheric Research
>>>>>>>>> NCAR Command Language Version 6.1.2
>>>>>>>>> The use of this software is governed by a License Agreement.
>>>>>>>>> See http://www.ncl.ucar.edu/ for more details.
>>>>>>>>> getpdstemplate: PDS Template 4.32 not defined.
>>>>>>>>> warning:Error unpacking GRIB record section 4.
>>>>>>>>> fatal:Could not open (./gonzalo08l.2014101412 <(201)%20410-1412>
>>>>>>>>> .hwrfsat.global.0p25.f024.grb2)
>>>>>>>>> Segmentation fault
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ncl-talk mailing list
>>>>>>>>> ncl-talk at ucar.edu
>>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20161228/2e964f8b/attachment.html
More information about the ncl-talk
mailing list