[ncl-talk] Problem running ncl_convert2nc (Error unpacking GRIB record section 4)
Dennis Shea
shea at ucar.edu
Wed Dec 28 14:22:22 MST 2016
Dave ... I have the control file.
I will attach to NCL-2533
D
On Wed, Dec 28, 2016 at 2:06 PM, David Brown <dbrown at ucar.edu> wrote:
> Hi Ligia,
> I took a look at this file both with NCL and wgrib2. My version of
> wgrib2 is likely newer than Dennis version and does not produce the
> errors that he mentions. It produces 12 records that all have the name
> BRTEMP (Brightness temperature). Unfortunately it does not output any
> metadata that would distinguish these records. I believe that it does
> not completely decode the PDS template 32, and therefore does not put
> out any information about the simulated satellite channels.
>
> NCL depends on the NCEP library g2clib to do the low level decoding of
> GRIB2 records. This library does not yet support PDS template 32 at
> all. That is the reason NCL is having a problem with this file.
>
> Presumably CDO also does not know this template, since it tries to
> make all 12 records into various timesteps of the the single BRTEMP
> variable -- even though the time stamp on all the records is clearly
> the same.
>
> It would help our efforts to add support for this template if you
> could send the control file that was used to generate this GRIB2 file.
> Thank you.
> -dave
>
>
>
>
>
> On Wed, Dec 28, 2016 at 1:15 PM, Ligia Bernardet - NOAA Affiliate
> <ligia.bernardet at noaa.gov> wrote:
> > Hi Dennis,
> >
> > I also had the problem with the CDO not being built with GRIB support,
> but
> > the NOAA HPC sys adm rebuilt it yesterday and it started working.
> >
> > The variable name is indeed nonsensical, and it is weird that CDO
> interprets
> > the data as 1 variable at 18 times (should be 18 variables at 1 time).
> This
> > is undesirable but tolerable, since I know what the variables are. This
> file
> > was produced by the NCEP Unified Postprocessor (UPP) and I have the
> control
> > file that was used to generate it, so I can see which variables are
> selected
> > for output. They are indeed Brightness Temperatures (K) produced by a
> > forecast model to simulated various channels of various satellites.
> Offline
> > metadata is dangerous, but I guess I have to resort to it for now.
> >
> > Thanks,
> > Ligia
> >
> > On Wed, Dec 28, 2016 at 1:06 PM, Dennis Shea <shea at ucar.edu> wrote:
> >>
> >> Initially, I did try the CDO on my local linux box but I got
> >>
> >> %> cdo -f nc copy gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2
> >> gon_CDO.nc
> >>
> >> Error (gribapiScanTimestep1) : GRIB_API support not compiled in!
> >>
> >> ---
> >> I guess my local version needs an upgrade.
> >>
> >> ---
> >> I did try the above on another machine and got
> >>
> >> /glade/p/ncldev/data/grib2>ncdump -h gon_CDO.nc
> >>
> >> netcdf gon_CDO {
> >> dimensions:
> >> lon = 1440 ;
> >> lat = 721 ;
> >> time = UNLIMITED ; // (18 currently)
> >> variables:
> >> double lon(lon) ;
> >> lon:standard_name = "longitude" ;
> >> lon:long_name = "longitude" ;
> >> lon:units = "degrees_east" ;
> >> lon:axis = "X" ;
> >> double lat(lat) ;
> >> lat:standard_name = "latitude" ;
> >> lat:long_name = "latitude" ;
> >> lat:units = "degrees_north" ;
> >> lat:axis = "Y" ;
> >> double time(time) ;
> >> time:standard_name = "time" ;
> >> time:units = "hours since 2014-10-14 12:00:00" ;
> >> time:calendar = "proleptic_gregorian" ;
> >> float param7.5.0(time, lat, lon) ;
> >> param7.5.0:_FillValue = -9.e+33f ;
> >> param7.5.0:missing_value = -9.e+33f ;
> >>
> >> // global attributes:
> >> :CDI = "Climate Data Interface version 1.6.3
> >> (http://code.zmaw.de/projects/cdi)" ;
> >> :Conventions = "CF-1.4" ;
> >> :history = "Wed Dec 28 12:04:14 2016: cdo -f nc copy
> >> gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2 gon_CDO.nc" ;
> >> :institution = "National Centers for Environmental Prediction" ;
> >> :CDO = "Climate Data Operators version 1.6.3
> >> (http://code.zmaw.de/projects/cdo)" ;
> >> }
> >>
> >>
> >> The CDO returns the variable with a nonsensical name:
> >>
> >> float param7.5.0(time, lat, lon) ;
> >>
> >> but no units or long_name .... How do you know what it is?
> >> I am sure the numbers are unpacked correctly ... but ... without using
> >> some external information, how do you know to what 'param7.5.0' refers?
> >>
> >> ===
> >> Combining the CDO file with NCL naming (via an appropriate internal
> lookup
> >> table) you would get:
> >>
> >> float BRTEMP_P32_GLL0 ( lat_0, lon_0 )
> >> long_name : Brightness temperature
> >> units : K
> >>
> >> param7.5.0 ===> BRTEMP_P32_GLL0
> >>
> >> and the attributes should be:
> >>
> >> long_name : Brightness temperature
> >> units : K
> >>
> >> D
> >>
> >> On Wed, Dec 28, 2016 at 11:04 AM, Ligia Bernardet - NOAA Affiliate
> >> <ligia.bernardet at noaa.gov> wrote:
> >>>
> >>> Dennis,
> >>>
> >>> Thanks for your willingness to help.
> >>>
> >>> I was able to use CDO to convert the GRIB2 file to NetCDF. All 18
> fields
> >>> are in the output file (interestingly, it appears as one variable at
> 18 time
> >>> levels, which is definitely not the case in the GRIB file ). I was
> able to
> >>> open the resulting NetCDF file in NCL and manipulate it. So, I am on
> my way
> >>> to get the results I need.
> >>>
> >>> But if you guys get this working in NCL directly, let me know.
> >>>
> >>> Thanks again,
> >>> Ligia
> >>>
> >>> On Wed, Dec 28, 2016 at 8:07 AM, Dennis Shea <shea at ucar.edu> wrote:
> >>>>
> >>>> Hi Ligea
> >>>>
> >>>> FYI ... a tool unrelated to NCL also fails.
> >>>>
> >>>> ------
> >>>> There is another tool available from NCEP: wgrib2
> >>>>
> >>>> http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/
> >>>>
> >>>> There is an option to create netCDF
> >>>>
> >>>> http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/netcdf.html
> >>>>
> >>>> ===
> >>>> When I use this tool on the grib2 file you sent, it also fails:
> >>>>
> >>>> %> wgrib2 gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2 -netcdf
> >>>> gon.nc
> >>>>
> >>>> ---
> >>>> The wgrib2 tool prints many warning lines like:
> >>>>
> >>>> code_table_4.5a: product definition template #32 not supported
> >>>> code_table_4.5b: product definition template #32 not supported
> >>>> ....
> >>>> code_table_4.5b: product definition template #32 not supported
> >>>> 1:0:d=2014101412:var discipline=0 master_table=8 parmcat=5
> >>>> parm=7:no_level:24 hour fcst:
> >>>> code_table_4.5a: product definition template #32 not supported
> >>>> ...
> >>>> code_table_4.5b: product definition template #32 not supported
> >>>> 2:279767:d=2014101412:var discipline=0 master_table=8 parmcat=5
> >>>> parm=7:no_level:24 hour fcst:
> >>>> code_table_4.5a: product definition template #32 not supported
> >>>> ...
> >>>>
> >>>> etc, etc
> >>>>
> >>>> Like 'ncl_convert2nc', wgrib2 produces just one variable within the
> >>>> output netCDF. Specifically,
> >>>>
> >>>>
> >>>> %> ncddump -h gon.nc
> >>>>
> >>>> netcdf gon {
> >>>> dimensions:
> >>>> latitude = 721 ;
> >>>> longitude = 1440 ;
> >>>> time = UNLIMITED ; // (1 currently)
> >>>> variables:
> >>>> double latitude(latitude) ;
> >>>> latitude:units = "degrees_north" ;
> >>>> latitude:long_name = "latitude" ;
> >>>> double longitude(longitude) ;
> >>>> longitude:units = "degrees_east" ;
> >>>> longitude:long_name = "longitude" ;
> >>>> double time(time) ;
> >>>> time:units = "seconds since 1970-01-01 00:00:00.0 0:00" ;
> >>>> time:long_name = "verification time generated by wgrib2
> function
> >>>> verftime()" ;
> >>>> time:reference_time = 1413288000. ;
> >>>> time:reference_time_type = 3 ;
> >>>> time:reference_date = "2014.10.14 12:00:00 UTC" ;
> >>>> time:reference_time_description = "forecast or accumulated,
> >>>> reference date is fixed" ;
> >>>> time:time_step_setting = "auto" ;
> >>>> time:time_step = 0. ;
> >>>> float var0_5_7_no_level(time, latitude, longitude) ;
> >>>> var0_5_7_no_level:_FillValue = 9.999e+20f ;
> >>>> var0_5_7_no_level:short_name = "var0_5_7_no_level" ;
> >>>> var0_5_7_no_level:long_name = "desc" ;
> >>>> var0_5_7_no_level:level = "no_level" ;
> >>>> var0_5_7_no_level:units = "unit" ;
> >>>>
> >>>> // global attributes:
> >>>> :Conventions = "COARDS" ;
> >>>> :History = "created by wgrib2" ;
> >>>> :GRIB2_grid_template = 0 ;
> >>>> ==========================================
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Dec 27, 2016 at 5:01 PM, Dennis Shea <shea at ucar.edu> wrote:
> >>>>>
> >>>>> 6.3.0 will unpack one variable also. The name of the variable will be
> >>>>> different because (! wild guess !) the needed table was not complete
> or
> >>>>> available.
> >>>>>
> >>>>> GRIB is a bit of a 'moving target' :-(
> >>>>>
> >>>>> I have opened a JIRA ticket: NCL-2533
> >>>>>
> >>>>> Unfortunately, this is not a great week to get things done.
> >>>>>
> >>>>> Maybe, an IO developer will look?
> >>>>>
> >>>>> D
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Dec 27, 2016 at 4:48 PM, Ligia Bernardet - NOAA Affiliate
> >>>>> <ligia.bernardet at noaa.gov> wrote:
> >>>>>>
> >>>>>> Hi Dennis,
> >>>>>>
> >>>>>> There are 18 fields in the GRIB2 file. In the NetCDF file you sent
> >>>>>> back to me there is 1.
> >>>>>> But more importantly, I need to find a way to do this myself, so I
> can
> >>>>>> process the multiple files I have.
> >>>>>>
> >>>>>> The machine I am using, a NOAA's R&D platform, I was able to find
> NCL
> >>>>>> v6.3. Would this version be usable to read this GRIB2 file, until a
> newer
> >>>>>> version is available?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Ligia
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Dec 27, 2016 at 4:11 PM, Dennis Shea <shea at ucar.edu> wrote:
> >>>>>>>
> >>>>>>> Attached is a gzip'd version of the netCDF file.
> >>>>>>>
> >>>>>>> The 'soon-to-be-released' beta 6.4.0 returns a warning message. I
> >>>>>>> will open a JIRA ticket.
> >>>>>>> Note: I believe the numbers are correct.
> >>>>>>>
> >>>>>>> =========
> >>>>>>>
> >>>>>>> NCAR Command Language Version 6.4.0 -27Dec2016_0148
> >>>>>>>
> >>>>>>> warning:NclGRIB2: BRTEMP_P32 contains records that NCL cannot
> >>>>>>> currently differentiate. One or more records will be ignored.
> >>>>>>>
> >>>>>>> Variable: f
> >>>>>>> Type: file
> >>>>>>> filename: gonzalo08l.2014101412.hwrfsat.global.0p25.f024
> >>>>>>> path:
> >>>>>>> /Users/shea/Data/GRIB/gonzalo08l.2014101412.hwrfsat.
> global.0p25.f024.grb2
> >>>>>>> file global attributes:
> >>>>>>> dimensions:
> >>>>>>> lat_0 = 721
> >>>>>>> lon_0 = 1440
> >>>>>>> variables:
> >>>>>>> float BRTEMP_P32_GLL0 ( lat_0, lon_0 )
> >>>>>>> center : US National Weather Service - NCEP (WMC)
> >>>>>>> production_status : Operational products
> >>>>>>> long_name : Brightness temperature
> >>>>>>> units : K
> >>>>>>> _FillValue : 1e+20
> >>>>>>> grid_type : Latitude/longitude
> >>>>>>> parameter_discipline_and_category : Meteorological
> >>>>>>> products, Long wave radiation
> >>>>>>> parameter_template_discipline_category_number : (
> 32,
> >>>>>>> 0, 5, 7 )
> >>>>>>> forecast_time : 0
> >>>>>>> forecast_time_units : hours
> >>>>>>> initial_time : 10/14/2014 (12:00)
> >>>>>>>
> >>>>>>> float lat_0 ( lat_0 )
> >>>>>>> long_name : latitude
> >>>>>>> grid_type : Latitude/Longitude
> >>>>>>> units : degrees_north
> >>>>>>> Dj : 0.25
> >>>>>>> Di : 0.25
> >>>>>>> Lo2 : 359.75
> >>>>>>> La2 : -90
> >>>>>>> Lo1 : 0
> >>>>>>> La1 : 90
> >>>>>>>
> >>>>>>> float lon_0 ( lon_0 )
> >>>>>>> long_name : longitude
> >>>>>>> grid_type : Latitude/Longitude
> >>>>>>> units : degrees_east
> >>>>>>> Dj : 0.25
> >>>>>>> Di : 0.25
> >>>>>>> Lo2 : 359.75
> >>>>>>> La2 : -90
> >>>>>>> Lo1 : 0
> >>>>>>> La1 : 90
> >>>>>>>
> >>>>>>> =========
> >>>>>>> The min and max are:
> >>>>>>>
> >>>>>>> (0) Brightness temperature (K) : min=184.67 max=319.49
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Dec 27, 2016 at 1:13 PM, Ligia Bernardet - NOAA Affiliate
> >>>>>>> <ligia.bernardet at noaa.gov> wrote:
> >>>>>>>>
> >>>>>>>> Hello,
> >>>>>>>>
> >>>>>>>> I have transferred file
> >>>>>>>> gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2 to
> /ftp/incoming.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Ligia
> >>>>>>>>
> >>>>>>>> On Tue, Dec 27, 2016 at 1:06 PM, Dennis Shea <shea at ucar.edu>
> wrote:
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>> 6.1.2 (Feb. 2013) is almost 4 years old. Much has happened since.
> >>>>>>>>>
> >>>>>>>>> [2]
> >>>>>>>>> To be able to determine the problem or if the current version of
> >>>>>>>>> NCL has addressed the issue, we would need the file.
> >>>>>>>>>
> >>>>>>>>> ftp ftp.cgd.ucar.edu
> >>>>>>>>> anonymous
> >>>>>>>>> your_email
> >>>>>>>>> cd incoming
> >>>>>>>>> prompt
> >>>>>>>>> put gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2
> >>>>>>>>> quit
> >>>>>>>>>
> >>>>>>>>> We can not look at /ftp/incoming, so ....
> >>>>>>>>>
> >>>>>>>>> Let ncl-talk know when the file has been transferred.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Dec 27, 2016 at 12:56 PM, Ligia Bernardet - NOAA
> Affiliate
> >>>>>>>>> <ligia.bernardet at noaa.gov> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hello,
> >>>>>>>>>>
> >>>>>>>>>> I would like to use convert a GRIB2 file generated by NCEP's
> >>>>>>>>>> Hurricane WRF (HWRF) model to a NetCDF file. However, I am
> encountering the
> >>>>>>>>>> error below. Any suggestions?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Ligia
> >>>>>>>>>>
> >>>>>>>>>> ncl_convert2nc gonzalo08l.2014101412.hwrfsat.global.
> 0p25.f024.grb2
> >>>>>>>>>>
> >>>>>>>>>> Processing file:
> >>>>>>>>>> gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2...
> >>>>>>>>>> Copyright (C) 1995-2013 - All Rights Reserved
> >>>>>>>>>> University Corporation for Atmospheric Research
> >>>>>>>>>> NCAR Command Language Version 6.1.2
> >>>>>>>>>> The use of this software is governed by a License Agreement.
> >>>>>>>>>> See http://www.ncl.ucar.edu/ for more details.
> >>>>>>>>>> getpdstemplate: PDS Template 4.32 not defined.
> >>>>>>>>>> warning:Error unpacking GRIB record section 4.
> >>>>>>>>>> fatal:Could not open
> >>>>>>>>>> (./gonzalo08l.2014101412.hwrfsat.global.0p25.f024.grb2)
> >>>>>>>>>> Segmentation fault
> >>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> ncl-talk mailing list
> >>>>>>>>>> ncl-talk at ucar.edu
> >>>>>>>>>> List instructions, subscriber options, unsubscribe:
> >>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
> >
> > _______________________________________________
> > ncl-talk mailing list
> > ncl-talk at ucar.edu
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20161228/2ad0bace/attachment.html
More information about the ncl-talk
mailing list