[ncl-talk] grib2 file read
David Brown
dbrown at ucar.edu
Tue Jul 26 19:40:09 MDT 2016
I don't know how this file is organized. I do know that the 'strings'
tool outputs some stuff that looks like variable names at the
beginning:
strings ngac.t00z.aerf24 |more
0NEMSIO GFS bin4
bccmass bcdp001 bcdp002 bcem001
bcem002 bceman bcembb bcembf
bchyphil bcsd001 bcsd002 bcsmass
bcsv001 bcsv002 bcwt001 bcwt002
dmscmass dmssmass ducmass25 ducmass
dudp001 dudp002 dudp003 dudp004
dudp005 duem001 duem002 duem003
duem004 duem005 dusd001 dusd002
dusd003 dusd004 dusd005 dusmass25
dusmass dusv001 dusv002 dusv003
dusv004 dusv005 duwt001 duwt002
duwt003 duwt004 duwt005 occmass
ocdp001 ocdp002 ocem001 ocem002 oceman
ocembb ocembf ocembg ochyphil
ocsd001 ocsd002 ocsmass ocsv001
ocsv002 ocwt001 ocwt002 so2cmass
so2eman so2embb so2emve so2emvn
so2smass so4cmass so4eman so4smass
sscmass25 sscmass ssdp001 ssdp002
ssdp003 ssdp004 ssdp005 ssem001
ssem002 ssem003 ssem004 ssem005
sssd001 sssd002 sssd003 sssd004
sssd005 sssmass25 sssmass sssv001
sssv002 sssv003 sssv004 sssv005
sswt001 sswt002 sswt003 sswt004
sswt005 sudp001 sudp002 sudp003
sudp004 suem001 suem002 suem003
suem004 supmsa supso2 supso4aq
supso4g supso4wt susd001 susd002
susd003 susd004 susv001 susv002
susv003 susv004 suwt001 suwt002
suwt003 suwt004
atmos col atmos col atmos col atmos col atmos
col atmos col atmos col atmos col atmos col
atmos col atmos col atmos col atmos col
atmos col atmos col atmos col atmos col atmos
col atmos col atmos col
.....
The "bin4" at the beginning suggests maybe that it is 4 byte floats,
but other than than I have no idea.
I think you will need to find some explanation of the contents of this
file from the people who created it.
-dave
On Tue, Jul 26, 2016 at 7:16 PM, Debasish Hazra
<debasish.hazra5 at gmail.com> wrote:
> Thanks David. Do I need to read the file as "binary" ?
>
> Debasish
>
> On Tue, Jul 26, 2016 at 6:12 PM, David Brown <dbrown at ucar.edu> wrote:
>>
>> I am afraid there have been some misunderstandings.
>> First of all the file "ngac.t00z.aerf24.grib2" (actually called
>> ngac.t00z.aerf24 on the ftp site if anyone else wants to download it)
>> is
>> not a GRIB file at all. A simple test if you suspect something is not
>> GRIB:
>>
>> strings <filename> | grep GRIB
>>
>> If this produces nothing then it's definitely not a GRIB file (of
>> course the presence of "GRIB" does not prove it is a GRIB file).
>> To digress for a moment, if it is a GRIB file then the command,
>>
>> strings <filename> | egrep 'GRIB|7777'
>>
>> will produce alternating instances of 'GRIB' and '7777'. These strings
>> mark the beginning and end of each GRIB record. This is true for both
>> GRIB 1 and 2.
>>
>> strings ngac.t00z.aerf24 |grep GRIB
>> produces nothing so it is not GRIB.
>>
>> Second, the warning messages that are emitted when an ngac file that
>> is valid GRIB is read by NCL such as:
>>
>> warning: Entry (62006) not found in code table file
>> /Users/shea/devel/lib/ncarg/grib2_codetables/ncep/4/4.3.table
>>
>> refer to entries not found in table 4.3 which is "type of generating
>> process". This is not the same as table 4.233 "Aerosol Type"
>> and so the number 62006 in this context has nothing to do with 62006
>> ("Sulphate Dry") in the Aerosol Type table
>>
>> 62006 as a type of generating processing is not documented in the 4.3
>> table and hence the message. Since type of generating process is
>> usually of less concern to users I am considering eliminating the
>> error message in this case.
>>
>> It is true that NCL does not yet make use of the 4.233 table. That is
>> because it is an auxiliary table that needs to be used in the context
>> of certain GRIB2 product templates that address aerosols. It does not
>> contain actual parameters. Rather a parameter might be named, e.g.
>> "Aerosol optical thickness" and based on the template there is another
>> piece of metadata called "Aerosol Type" which is as documented in the
>> "Aerosol Types" table. A small extension to our GRIB2 data model is
>> required to handle these templates. We hope to implement it in time
>> for 6.4.0 but that is not yet certain.
>> -dave
>>
>>
>> On Tue, Jul 26, 2016 at 12:23 PM, Debasish Hazra
>> <debasish.hazra5 at gmail.com> wrote:
>> > Hi Dennis,
>> >
>> > The variables in the range 62000 to 62010 in the updated grib2 table
>> > "http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-233.shtml" are
>> > "optical depth" and has no units associated with them.
>> >
>> > 62000 Total Aerosol
>> > 62001 Dust Dry
>> > 62006 Sulphate Dry
>> > 62008 Sea Salt Dry
>> > 62009 Black Carbon Dry
>> > 62010 Particulate Organic Matter Dry
>> >
>> > 62010-62016 are mixing ratios in kg/kg
>> >
>> > Debasish
>> >
>> >
>> >
>> > On Tue, Jul 26, 2016 at 2:13 PM, Dennis Shea <shea at ucar.edu> wrote:
>> >>
>> >> Again, I will forward this. please send to ncl-talk. Otherwise, I am
>> >> just
>> >> a middleman forwarding the emails.
>> >>
>> >> Usually the GRIB tables have units:
>> >>
>> >> Examples:
>> >> http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/ncep_141.htm
>> >> http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/ecmwf_132.htm
>> >>
>> >> NCL ... see GRIB section
>> >> See:
>> >>
>> >> http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclFormatSupport.shtml
>> >>
>> >> On Tue, Jul 26, 2016 at 12:05 PM, Debasish Hazra
>> >> <debasish.hazra5 at gmail.com> wrote:
>> >>>
>> >>> The link updated grib2 table is here :
>> >>>
>> >>> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_table4-233.shtml
>> >>>
>> >>> Thanks
>> >>> Debasish
>> >>>
>> >>> On Tue, Jul 26, 2016 at 1:33 PM, Dennis Shea <shea at ucar.edu> wrote:
>> >>>>
>> >>>> The issue is that the NGAC grib tables are not publicly available.
>> >>>> If you know of where they are located, they can be included within
>> >>>> NCL
>> >>>>
>> >>>> In fact, there is a JIRA ticket on this:
>> >>>>
>> >>>> [JIRA] (NCL-2442) GRIB-2: warning: Entry (not found in code table
>> >>>> file
>> >>>> .../grib2_codetables/ncep/4/4.3.table
>> >>>>
>> >>>> Just yesterday, a comment was made on this ticket:
>> >>>>
>> >>>> ---------
>> >>>>
>> >>>> Unfortunately the "not found in code table file" errors
>> >>>> that Dennis reported in this ticket cannot be corrected
>> >>>> because the entries for that were not found are not documented
>> >>>> by the NCEP GRIB2 web pages. Table 4.3 contains the generating
>> >>>> process
>> >>>> mapping. The values found in the file, all in the range 62000 to
>> >>>> 62016,
>> >>>> probably refer to local "generating processes" related to the NGAC
>> >>>> model.
>> >>>> If documentation can be found to describe these I would be happy to
>> >>>> add
>> >>>> them to the tables.
>> >>>>
>> >>>> The other option to get rid of these admittedly annoying
>> >>>> messages might be to just not print the warnings for this piece of
>> >>>> metadata
>> >>>> since it is not usually of that much interest to the data consumer. I
>> >>>> would
>> >>>> be happy to consider this.
>> >>>>
>> >>>> ---------
>> >>>> Note: the *values* returned by NCL are correct. The meta data is not
>> >>>> because the
>> >>>> NCEP GRIB2 web pages are not available.
>> >>>>
>> >>>> --------
>> >>>>
>> >>>> Again, if you know where the appropriate tables are located, please
>> >>>> send
>> >>>> to ncl-talk.
>> >>>>
>> >>>> THX
>> >>>>
>> >>>> On Tue, Jul 26, 2016 at 11:21 AM, Debasish Hazra
>> >>>> <debasish.hazra5 at gmail.com> wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> I am trying to read grib2 file with ncl (v6.3) and it shows the
>> >>>>> following error while trying to open it:
>> >>>>>
>> >>>>> fatal:An internal error has occurred. The file format requested does
>> >>>>> not appear to be supported, could not open (ngac.t00z.aerf24.grib2)
>> >>>>> Variable: f
>> >>>>> Type: file
>> >>>>> (0) File Missing Value : -1
>> >>>>>
>> >>>>> I kept the data file in ftp. Any help on this is appreciated.
>> >>>>>
>> >>>>> Thanks.
>> >>>>> Debasish
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> ncl-talk mailing list
>> >>>>> ncl-talk at ucar.edu
>> >>>>> List instructions, subscriber options, unsubscribe:
>> >>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >>>>>
>> >>>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> ncl-talk mailing list
>> >>> ncl-talk at ucar.edu
>> >>> List instructions, subscriber options, unsubscribe:
>> >>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >>>
>> >>
>> >
>> >
>> > _______________________________________________
>> > ncl-talk mailing list
>> > ncl-talk at ucar.edu
>> > List instructions, subscriber options, unsubscribe:
>> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >
>
>
More information about the ncl-talk
mailing list