<div dir="ltr"><div><div>Dave ... I have the control file.<br><br></div>I will attach to NCL-2533<br><br></div>D<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 28, 2016 at 2:06 PM, David Brown <span dir="ltr"><<a href="mailto:dbrown@ucar.edu" target="_blank">dbrown@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Ligia,<br>
I took a look at this file both with NCL and wgrib2. My version of<br>
wgrib2 is likely newer than Dennis version and does not produce the<br>
errors that he mentions. It produces 12 records that all have the name<br>
BRTEMP (Brightness temperature). Unfortunately it does not output any<br>
metadata that would distinguish these records. I believe that it does<br>
not completely decode the PDS template 32, and therefore does not put<br>
out any information about the simulated satellite channels.<br>
<br>
NCL depends on the NCEP library g2clib to do the low level decoding of<br>
GRIB2 records. This library does not yet support PDS template 32 at<br>
all. That is the reason NCL is having a problem with this file.<br>
<br>
Presumably CDO also does not know this template, since it tries to<br>
make all 12 records into various timesteps of the the single BRTEMP<br>
variable -- even though the time stamp on all the records is clearly<br>
the same.<br>
<br>
It would help our efforts to add support for this template if you<br>
could send the control file that was used to generate this GRIB2 file.<br>
Thank you.<br>
-dave<br>
<br>
<br>
<br>
<br>
<br>
On Wed, Dec 28, 2016 at 1:15 PM, Ligia Bernardet - NOAA Affiliate<br>
<div class="HOEnZb"><div class="h5"><<a href="mailto:ligia.bernardet@noaa.gov">ligia.bernardet@noaa.gov</a>> wrote:<br>
> Hi Dennis,<br>
><br>
> I also had the problem with the CDO not being built with GRIB support, but<br>
> the NOAA HPC sys adm rebuilt it yesterday and it started working.<br>
><br>
> The variable name is indeed nonsensical, and it is weird that CDO interprets<br>
> the data as 1 variable at 18 times (should be 18 variables at 1 time). This<br>
> is undesirable but tolerable, since I know what the variables are. This file<br>
> was produced by the NCEP Unified Postprocessor (UPP) and I have the control<br>
> file that was used to generate it, so I can see which variables are selected<br>
> for output. They are indeed Brightness Temperatures (K) produced by a<br>
> forecast model to simulated various channels of various satellites. Offline<br>
> metadata is dangerous, but I guess I have to resort to it for now.<br>
><br>
> Thanks,<br>
> Ligia<br>
><br>
> On Wed, Dec 28, 2016 at 1:06 PM, Dennis Shea <<a href="mailto:shea@ucar.edu">shea@ucar.edu</a>> wrote:<br>
>><br>
>> Initially, I did try the CDO on my local linux box but I got<br>
>><br>
>> %> cdo -f nc copy gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2<br>
>> gon_CDO.nc<br>
>><br>
>> Error (gribapiScanTimestep1) : GRIB_API support not compiled in!<br>
>><br>
>> ---<br>
>> I guess my local version needs an upgrade.<br>
>><br>
>> ---<br>
>> I did try the above on another machine and got<br>
>><br>
>> /glade/p/ncldev/data/grib2><wbr>ncdump -h gon_CDO.nc<br>
>><br>
>> netcdf gon_CDO {<br>
>> dimensions:<br>
>> lon = 1440 ;<br>
>> lat = 721 ;<br>
>> time = UNLIMITED ; // (18 currently)<br>
>> variables:<br>
>> double lon(lon) ;<br>
>> lon:standard_name = "longitude" ;<br>
>> lon:long_name = "longitude" ;<br>
>> lon:units = "degrees_east" ;<br>
>> lon:axis = "X" ;<br>
>> double lat(lat) ;<br>
>> lat:standard_name = "latitude" ;<br>
>> lat:long_name = "latitude" ;<br>
>> lat:units = "degrees_north" ;<br>
>> lat:axis = "Y" ;<br>
>> double time(time) ;<br>
>> time:standard_name = "time" ;<br>
>> time:units = "hours since 2014-10-14 12:00:00" ;<br>
>> time:calendar = "proleptic_gregorian" ;<br>
>> float param7.5.0(time, lat, lon) ;<br>
>> param7.5.0:_FillValue = -9.e+33f ;<br>
>> param7.5.0:missing_value = -9.e+33f ;<br>
>><br>
>> // global attributes:<br>
>> :CDI = "Climate Data Interface version 1.6.3<br>
>> (<a href="http://code.zmaw.de/projects/cdi" rel="noreferrer" target="_blank">http://code.zmaw.de/projects/<wbr>cdi</a>)" ;<br>
>> :Conventions = "CF-1.4" ;<br>
>> :history = "Wed Dec 28 12:04:14 2016: cdo -f nc copy<br>
>> gonzalo08l.2014101412.hwrfsat.<wbr>global.0p25.f024.grb2 gon_CDO.nc" ;<br>
>> :institution = "National Centers for Environmental Prediction" ;<br>
>> :CDO = "Climate Data Operators version 1.6.3<br>
>> (<a href="http://code.zmaw.de/projects/cdo" rel="noreferrer" target="_blank">http://code.zmaw.de/projects/<wbr>cdo</a>)" ;<br>
>> }<br>
>><br>
>><br>
>> The CDO returns the variable with a nonsensical name:<br>
>><br>
>> float param7.5.0(time, lat, lon) ;<br>
>><br>
>> but no units or long_name .... How do you know what it is?<br>
>> I am sure the numbers are unpacked correctly ... but ... without using<br>
>> some external information, how do you know to what 'param7.5.0' refers?<br>
>><br>
>> ===<br>
>> Combining the CDO file with NCL naming (via an appropriate internal lookup<br>
>> table) you would get:<br>
>><br>
>> float BRTEMP_P32_GLL0 ( lat_0, lon_0 )<br>
>> long_name : Brightness temperature<br>
>> units : K<br>
>><br>
>> param7.5.0 ===> BRTEMP_P32_GLL0<br>
>><br>
>> and the attributes should be:<br>
>><br>
>> long_name : Brightness temperature<br>
>> units : K<br>
>><br>
>> D<br>
>><br>
>> On Wed, Dec 28, 2016 at 11:04 AM, Ligia Bernardet - NOAA Affiliate<br>
>> <<a href="mailto:ligia.bernardet@noaa.gov">ligia.bernardet@noaa.gov</a>> wrote:<br>
>>><br>
>>> Dennis,<br>
>>><br>
>>> Thanks for your willingness to help.<br>
>>><br>
>>> I was able to use CDO to convert the GRIB2 file to NetCDF. All 18 fields<br>
>>> are in the output file (interestingly, it appears as one variable at 18 time<br>
>>> levels, which is definitely not the case in the GRIB file ). I was able to<br>
>>> open the resulting NetCDF file in NCL and manipulate it. So, I am on my way<br>
>>> to get the results I need.<br>
>>><br>
>>> But if you guys get this working in NCL directly, let me know.<br>
>>><br>
>>> Thanks again,<br>
>>> Ligia<br>
>>><br>
>>> On Wed, Dec 28, 2016 at 8:07 AM, Dennis Shea <<a href="mailto:shea@ucar.edu">shea@ucar.edu</a>> wrote:<br>
>>>><br>
>>>> Hi Ligea<br>
>>>><br>
>>>> FYI ... a tool unrelated to NCL also fails.<br>
>>>><br>
>>>> ------<br>
>>>> There is another tool available from NCEP: wgrib2<br>
>>>><br>
>>>> <a href="http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/" rel="noreferrer" target="_blank">http://www.cpc.ncep.noaa.gov/<wbr>products/wesley/wgrib2/</a><br>
>>>><br>
>>>> There is an option to create netCDF<br>
>>>><br>
>>>> <a href="http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/netcdf.html" rel="noreferrer" target="_blank">http://www.cpc.ncep.noaa.gov/<wbr>products/wesley/wgrib2/netcdf.<wbr>html</a><br>
>>>><br>
>>>> ===<br>
>>>> When I use this tool on the grib2 file you sent, it also fails:<br>
>>>><br>
>>>> %> wgrib2 gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2 -netcdf<br>
>>>> <a href="http://gon.nc" rel="noreferrer" target="_blank">gon.nc</a><br>
>>>><br>
>>>> ---<br>
>>>> The wgrib2 tool prints many warning lines like:<br>
>>>><br>
>>>> code_table_4.5a: product definition template #32 not supported<br>
>>>> code_table_4.5b: product definition template #32 not supported<br>
>>>> ....<br>
>>>> code_table_4.5b: product definition template #32 not supported<br>
>>>> 1:0:d=2014101412:var discipline=0 master_table=8 parmcat=5<br>
>>>> parm=7:no_level:24 hour fcst:<br>
>>>> code_table_4.5a: product definition template #32 not supported<br>
>>>> ...<br>
>>>> code_table_4.5b: product definition template #32 not supported<br>
>>>> 2:279767:d=2014101412:var discipline=0 master_table=8 parmcat=5<br>
>>>> parm=7:no_level:24 hour fcst:<br>
>>>> code_table_4.5a: product definition template #32 not supported<br>
>>>> ...<br>
>>>><br>
>>>> etc, etc<br>
>>>><br>
>>>> Like 'ncl_convert2nc', wgrib2 produces just one variable within the<br>
>>>> output netCDF. Specifically,<br>
>>>><br>
>>>><br>
>>>> %> ncddump -h <a href="http://gon.nc" rel="noreferrer" target="_blank">gon.nc</a><br>
>>>><br>
>>>> netcdf gon {<br>
>>>> dimensions:<br>
>>>> latitude = 721 ;<br>
>>>> longitude = 1440 ;<br>
>>>> time = UNLIMITED ; // (1 currently)<br>
>>>> variables:<br>
>>>> double latitude(latitude) ;<br>
>>>> latitude:units = "degrees_north" ;<br>
>>>> latitude:long_name = "latitude" ;<br>
>>>> double longitude(longitude) ;<br>
>>>> longitude:units = "degrees_east" ;<br>
>>>> longitude:long_name = "longitude" ;<br>
>>>> double time(time) ;<br>
>>>> time:units = "seconds since 1970-01-01 00:00:00.0 0:00" ;<br>
>>>> time:long_name = "verification time generated by wgrib2 function<br>
>>>> verftime()" ;<br>
>>>> time:reference_time = 1413288000. ;<br>
>>>> time:reference_time_type = 3 ;<br>
>>>> time:reference_date = "2014.10.14 12:00:00 UTC" ;<br>
>>>> time:reference_time_<wbr>description = "forecast or accumulated,<br>
>>>> reference date is fixed" ;<br>
>>>> time:time_step_setting = "auto" ;<br>
>>>> time:time_step = 0. ;<br>
>>>> float var0_5_7_no_level(time, latitude, longitude) ;<br>
>>>> var0_5_7_no_level:_FillValue = 9.999e+20f ;<br>
>>>> var0_5_7_no_level:short_name = "var0_5_7_no_level" ;<br>
>>>> var0_5_7_no_level:long_name = "desc" ;<br>
>>>> var0_5_7_no_level:level = "no_level" ;<br>
>>>> var0_5_7_no_level:units = "unit" ;<br>
>>>><br>
>>>> // global attributes:<br>
>>>> :Conventions = "COARDS" ;<br>
>>>> :History = "created by wgrib2" ;<br>
>>>> :GRIB2_grid_template = 0 ;<br>
>>>> ==============================<wbr>============<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Tue, Dec 27, 2016 at 5:01 PM, Dennis Shea <<a href="mailto:shea@ucar.edu">shea@ucar.edu</a>> wrote:<br>
>>>>><br>
>>>>> 6.3.0 will unpack one variable also. The name of the variable will be<br>
>>>>> different because (! wild guess !) the needed table was not complete or<br>
>>>>> available.<br>
>>>>><br>
>>>>> GRIB is a bit of a 'moving target' :-(<br>
>>>>><br>
>>>>> I have opened a JIRA ticket: NCL-2533<br>
>>>>><br>
>>>>> Unfortunately, this is not a great week to get things done.<br>
>>>>><br>
>>>>> Maybe, an IO developer will look?<br>
>>>>><br>
>>>>> D<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> On Tue, Dec 27, 2016 at 4:48 PM, Ligia Bernardet - NOAA Affiliate<br>
>>>>> <<a href="mailto:ligia.bernardet@noaa.gov">ligia.bernardet@noaa.gov</a>> wrote:<br>
>>>>>><br>
>>>>>> Hi Dennis,<br>
>>>>>><br>
>>>>>> There are 18 fields in the GRIB2 file. In the NetCDF file you sent<br>
>>>>>> back to me there is 1.<br>
>>>>>> But more importantly, I need to find a way to do this myself, so I can<br>
>>>>>> process the multiple files I have.<br>
>>>>>><br>
>>>>>> The machine I am using, a NOAA's R&D platform, I was able to find NCL<br>
>>>>>> v6.3. Would this version be usable to read this GRIB2 file, until a newer<br>
>>>>>> version is available?<br>
>>>>>><br>
>>>>>> Thanks,<br>
>>>>>> Ligia<br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> On Tue, Dec 27, 2016 at 4:11 PM, Dennis Shea <<a href="mailto:shea@ucar.edu">shea@ucar.edu</a>> wrote:<br>
>>>>>>><br>
>>>>>>> Attached is a gzip'd version of the netCDF file.<br>
>>>>>>><br>
>>>>>>> The 'soon-to-be-released' beta 6.4.0 returns a warning message. I<br>
>>>>>>> will open a JIRA ticket.<br>
>>>>>>> Note: I believe the numbers are correct.<br>
>>>>>>><br>
>>>>>>> =========<br>
>>>>>>><br>
>>>>>>> NCAR Command Language Version 6.4.0 -27Dec2016_0148<br>
>>>>>>><br>
>>>>>>> warning:NclGRIB2: BRTEMP_P32 contains records that NCL cannot<br>
>>>>>>> currently differentiate. One or more records will be ignored.<br>
>>>>>>><br>
>>>>>>> Variable: f<br>
>>>>>>> Type: file<br>
>>>>>>> filename: gonzalo08l.2014101412.hwrfsat.<wbr>global.0p25.f024<br>
>>>>>>> path:<br>
>>>>>>> /Users/shea/Data/GRIB/<wbr>gonzalo08l.2014101412.hwrfsat.<wbr>global.0p25.f024.grb2<br>
>>>>>>> file global attributes:<br>
>>>>>>> dimensions:<br>
>>>>>>> lat_0 = 721<br>
>>>>>>> lon_0 = 1440<br>
>>>>>>> variables:<br>
>>>>>>> float BRTEMP_P32_GLL0 ( lat_0, lon_0 )<br>
>>>>>>> center : US National Weather Service - NCEP (WMC)<br>
>>>>>>> production_status : Operational products<br>
>>>>>>> long_name : Brightness temperature<br>
>>>>>>> units : K<br>
>>>>>>> _FillValue : 1e+20<br>
>>>>>>> grid_type : Latitude/longitude<br>
>>>>>>> parameter_discipline_and_<wbr>category : Meteorological<br>
>>>>>>> products, Long wave radiation<br>
>>>>>>> parameter_template_discipline_<wbr>category_number : ( 32,<br>
>>>>>>> 0, 5, 7 )<br>
>>>>>>> forecast_time : 0<br>
>>>>>>> forecast_time_units : hours<br>
>>>>>>> initial_time : 10/14/2014 (12:00)<br>
>>>>>>><br>
>>>>>>> float lat_0 ( lat_0 )<br>
>>>>>>> long_name : latitude<br>
>>>>>>> grid_type : Latitude/Longitude<br>
>>>>>>> units : degrees_north<br>
>>>>>>> Dj : 0.25<br>
>>>>>>> Di : 0.25<br>
>>>>>>> Lo2 : 359.75<br>
>>>>>>> La2 : -90<br>
>>>>>>> Lo1 : 0<br>
>>>>>>> La1 : 90<br>
>>>>>>><br>
>>>>>>> float lon_0 ( lon_0 )<br>
>>>>>>> long_name : longitude<br>
>>>>>>> grid_type : Latitude/Longitude<br>
>>>>>>> units : degrees_east<br>
>>>>>>> Dj : 0.25<br>
>>>>>>> Di : 0.25<br>
>>>>>>> Lo2 : 359.75<br>
>>>>>>> La2 : -90<br>
>>>>>>> Lo1 : 0<br>
>>>>>>> La1 : 90<br>
>>>>>>><br>
>>>>>>> =========<br>
>>>>>>> The min and max are:<br>
>>>>>>><br>
>>>>>>> (0) Brightness temperature (K) : min=184.67 max=319.49<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> On Tue, Dec 27, 2016 at 1:13 PM, Ligia Bernardet - NOAA Affiliate<br>
>>>>>>> <<a href="mailto:ligia.bernardet@noaa.gov">ligia.bernardet@noaa.gov</a>> wrote:<br>
>>>>>>>><br>
>>>>>>>> Hello,<br>
>>>>>>>><br>
>>>>>>>> I have transferred file<br>
>>>>>>>> gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2 to /ftp/incoming.<br>
>>>>>>>><br>
>>>>>>>> Thanks,<br>
>>>>>>>> Ligia<br>
>>>>>>>><br>
>>>>>>>> On Tue, Dec 27, 2016 at 1:06 PM, Dennis Shea <<a href="mailto:shea@ucar.edu">shea@ucar.edu</a>> wrote:<br>
>>>>>>>>><br>
>>>>>>>>> [1]<br>
>>>>>>>>> 6.1.2 (Feb. 2013) is almost 4 years old. Much has happened since.<br>
>>>>>>>>><br>
>>>>>>>>> [2]<br>
>>>>>>>>> To be able to determine the problem or if the current version of<br>
>>>>>>>>> NCL has addressed the issue, we would need the file.<br>
>>>>>>>>><br>
>>>>>>>>> ftp <a href="http://ftp.cgd.ucar.edu" rel="noreferrer" target="_blank">ftp.cgd.ucar.edu</a><br>
>>>>>>>>> anonymous<br>
>>>>>>>>> your_email<br>
>>>>>>>>> cd incoming<br>
>>>>>>>>> prompt<br>
>>>>>>>>> put gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2<br>
>>>>>>>>> quit<br>
>>>>>>>>><br>
>>>>>>>>> We can not look at /ftp/incoming, so ....<br>
>>>>>>>>><br>
>>>>>>>>> Let ncl-talk know when the file has been transferred.<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> On Tue, Dec 27, 2016 at 12:56 PM, Ligia Bernardet - NOAA Affiliate<br>
>>>>>>>>> <<a href="mailto:ligia.bernardet@noaa.gov">ligia.bernardet@noaa.gov</a>> wrote:<br>
>>>>>>>>>><br>
>>>>>>>>>> Hello,<br>
>>>>>>>>>><br>
>>>>>>>>>> I would like to use convert a GRIB2 file generated by NCEP's<br>
>>>>>>>>>> Hurricane WRF (HWRF) model to a NetCDF file. However, I am encountering the<br>
>>>>>>>>>> error below. Any suggestions?<br>
>>>>>>>>>><br>
>>>>>>>>>> Thanks,<br>
>>>>>>>>>> Ligia<br>
>>>>>>>>>><br>
>>>>>>>>>> ncl_convert2nc gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2<br>
>>>>>>>>>><br>
>>>>>>>>>> Processing file:<br>
>>>>>>>>>> gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2...<br>
>>>>>>>>>> Copyright (C) 1995-2013 - All Rights Reserved<br>
>>>>>>>>>> University Corporation for Atmospheric Research<br>
>>>>>>>>>> NCAR Command Language Version 6.1.2<br>
>>>>>>>>>> The use of this software is governed by a License Agreement.<br>
>>>>>>>>>> See <a href="http://www.ncl.ucar.edu/" rel="noreferrer" target="_blank">http://www.ncl.ucar.edu/</a> for more details.<br>
>>>>>>>>>> getpdstemplate: PDS Template 4.32 not defined.<br>
>>>>>>>>>> warning:Error unpacking GRIB record section 4.<br>
>>>>>>>>>> fatal:Could not open<br>
>>>>>>>>>> (./gonzalo08l.<a href="tel:2014101412" value="+12014101412">2014101412</a>.hwrfsat.global.<wbr>0p25.f024.grb2)<br>
>>>>>>>>>> Segmentation fault<br>
>>>>>>>>>><br>
>>>>>>>>>> ______________________________<wbr>_________________<br>
>>>>>>>>>> ncl-talk mailing list<br>
>>>>>>>>>> <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
>>>>>>>>>> List instructions, subscriber options, unsubscribe:<br>
>>>>>>>>>> <a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
>>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>><br>
>>>>>>><br>
>>>>>><br>
>>>>><br>
>>>><br>
>>><br>
>><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> ncl-talk mailing list<br>
> <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
> List instructions, subscriber options, unsubscribe:<br>
> <a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
><br>
</div></div></blockquote></div><br></div>