[Met_help] [rt.rap.ucar.edu #97769] History for urgent question for reading ECMWF ensmeble data
John Halley Gotway via RT
met_help at ucar.edu
Mon Dec 14 13:05:22 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I have an urgent question for how to read ECMWF ensemble grib2 data.
The GRIB2 tables by EC are quite different from NCEP/WMO.
In VSDB we can read it by using getgb2 function. But In MET, I have no idea
to set up in the ensemble_stat and grid-stat config files.
If using wgrib2 for one member file, the output is following:
1:0:d=2020120100:var discipline=255 master_table=2 parmcat=3 parm=5:500 mb:12 hour fcst:ENS=-25
2:29547:d=2020120100:var discipline=255 master_table=2 parmcat=3 parm=5:1000 mb:12 hour fcst:ENS=-25
3:61542:d=2020120100:var discipline=255 master_table=2 parmcat=0 parm=0:850 mb:12 hour fcst:ENS=-25
4:78706:d=2020120100:var discipline=255 master_table=2 parmcat=2 parm=2:850 mb:12 hour fcst:ENS=-25
5:100088:d=2020120100:var discipline=255 master_table=2 parmcat=2 parm=3:850 mb:12 hour fcst:ENS=-25
6:123772:d=2020120100:var discipline=255 master_table=2 parmcat=2 parm=2:250 mb:12 hour fcst:ENS=-25
7:146448:d=2020120100:var discipline=255 master_table=2 parmcat=2 parm=3:250 mb:12 hour fcst:ENS=-25
8:171304:d=2020120100:var discipline=255 master_table=2 parmcat=0 parm=0:2 m above ground:12 hour fcst:ENS=-25
9:193892:d=2020120100:var discipline=255 master_table=2 parmcat=2 parm=2:10 m above ground:12 hour fcst:ENS=-25
10:217559:d=2020120100:var discipline=255 master_table=2 parmcat=2 parm=3:10 m above ground:12 hour fcst:ENS=-25
These 10 records are fixed, 500mb HGT, 1000mb HGT, 850 T, 850mb U and V, 250mb U and V, 2m T and 10m U and V.
The record number is not changed for all members and not change at all times.
The problem is there is no field name string in this file. Only the record number is known to match each field. In VSDB,
getgb2 subroutine uses 3 product IDs: parmcat=3 parm=5:1000 mb to get them correctly. But I don't know how to set
up fcst entry in config files. I look at the MET instruction, saying can use level="Rxxx", but name has to be specified?
or can use white card "*" ?
Thanks!
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Fri Dec 04 09:57:10 2020
Binbin,
You can do something similar with MET. Please go to this
link:https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
And then search for "GRIB2_mtab", and you'll find the content
copied/pasted below. You can specify these INSTEAD OF specifying a
requested name.
// - The "GRIB2_mtab" is an integer specifying the master table
number.
// - The "GRIB2_ltab" is an integer specifying the local table
number.
// - The "GRIB2_disc" is an integer specifying the GRIB2
discipline code.
// - The "GRIB2_parm_cat" is an integer specifying the parameter
category
// code.
// - The "GRIB2_parm" is an integer specifying the parameter
code.
// - The "GRIB2_pdt" is an integer specifying the product
definition
// template (Table 4.0).
// - The "GRIB2_process" is an integer specifying the generating
process
// (Table 4.3).
// - The "GRIB2_cntr" is an integer specifying the originating
center.
// - The "GRIB2_ens_type" is an integer specifying the ensemble
type
// (Table 4.6).
// - The "GRIB2_der_type" is an integer specifying the derived
product
// type (Table 4.7).
// - The "GRIB2_stat_type" is an integer specifying the
statistical
// processing type (Table 4.10).
// - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val" entries
are arrays
// of integers which specify the product description template
values to
// be used. The indices are 0-based. For example, use the
following to
// request a GRIB2 record whose 9-th and 27-th product
description
// template values are 1 and 2, respectively:
// GRIB2_ipdtmpl_index=[8, 26]; GRIB2_ipdtmpl_val=[1, 2];
While I don't have any ECMWF data on hand for testing, I do have a
file from the Air Force from which wgrib2 prints a similar result:
549:687031550:d=2017061000:var discipline=0 center=57 local_table=1
parmcat=1 parm=192:surface:18 hour fcst:
So I used that for testing. I ran the following plot_data_plane
command WITHOUT specifying a requested name:
plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps 'GRIB2_disc=0;
GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1; GRIB2_parm=192;
level="Z0";' -v 3
And then I looked closely at the log message to confirm that it used
record 549, as expected:
DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
So MET used these GRIB2 numbers I defined to do a table lookup and
find a match in the MET GRIB
tables:https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
And it determined that that field should be called "CTSTM". Hopefully,
the ECMWF GRIB2 tables are already defined there in MET. If not, we
can always add them.
Please try running plot_data_plane at verbosity level 3 on one of
these files first. And make sure it runs without error and the log
message prints out the "name" it found for that data. If so, you can
just skip using all those integers and revert back to using the name
MET knows. If not, then we may need to define a custom GRIB table for
this ECMWF GRIB2 data.
Thanks,
John
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Fri Dec 04 10:07:41 2020
John,
You can go to WCOSS/Venus my directory
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
ecme.ens01.t00z.grd3.f12.grib2 is one sample file
Binbin
On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
> You can do something similar with MET. Please go to this link:
> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>
> And then search for "GRIB2_mtab", and you'll find the content
> copied/pasted below. You can specify these INSTEAD OF specifying a
> requested name.
>
> // - The "GRIB2_mtab" is an integer specifying the master
table
> number.
> // - The "GRIB2_ltab" is an integer specifying the local table
> number.
> // - The "GRIB2_disc" is an integer specifying the GRIB2
discipline
> code.
> // - The "GRIB2_parm_cat" is an integer specifying the
parameter
> category
> // code.
> // - The "GRIB2_parm" is an integer specifying the parameter
code.
> // - The "GRIB2_pdt" is an integer specifying the product
definition
> // template (Table 4.0).
> // - The "GRIB2_process" is an integer specifying the
generating
> process
> // (Table 4.3).
> // - The "GRIB2_cntr" is an integer specifying the originating
> center.
> // - The "GRIB2_ens_type" is an integer specifying the
ensemble type
> // (Table 4.6).
> // - The "GRIB2_der_type" is an integer specifying the derived
> product
> // type (Table 4.7).
> // - The "GRIB2_stat_type" is an integer specifying the
statistical
> // processing type (Table 4.10).
> // - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val" entries
are
> arrays
> // of integers which specify the product description
template
> values to
> // be used. The indices are 0-based. For example, use the
> following to
> // request a GRIB2 record whose 9-th and 27-th product
description
> // template values are 1 and 2, respectively:
> // GRIB2_ipdtmpl_index=[8, 26]; GRIB2_ipdtmpl_val=[1, 2];
>
> While I don't have any ECMWF data on hand for testing, I do have a
file
> from the Air Force from which wgrib2 prints a similar result:
>
> 549:687031550:d=2017061000:var discipline=0 center=57 local_table=1
> parmcat=1 parm=192:surface:18 hour fcst:
>
> So I used that for testing. I ran the following plot_data_plane
command
> WITHOUT specifying a requested name:
>
>
> plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps 'GRIB2_disc=0;
> GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1; GRIB2_parm=192;
level="Z0";'
> -v 3
>
> And then I looked closely at the log message to confirm that it used
> record 549, as expected:
>
> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>
> So MET used these GRIB2 numbers I defined to do a table lookup and
find a
> match in the MET GRIB tables:
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>
> And it determined that that field should be called "CTSTM".
Hopefully, the
> ECMWF GRIB2 tables are already defined there in MET. If not, we can
always
> add them.
>
> Please try running plot_data_plane at verbosity level 3 on one of
these
> files first. And make sure it runs without error and the log message
prints
> out the "name" it found for that data. If so, you can just skip
using all
> those integers and revert back to using the name MET knows. If not,
then we
> may need to define a custom GRIB table for this ECMWF GRIB2 data.
>
> Thanks,
> John
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Fri Dec 04 11:05:32 2020
John,
I tested using following command:
plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";' -v3
ERROR :
ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
GRIB2_parm_cat and GRIB2_parm must be specified in field information
plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v
3
Still got error
ERROR :
ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
GRIB2_parm_cat and GRIB2_parm must be specified in field information
Looks like discipline=255 is not recognized
Binbin
On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
> You can do something similar with MET. Please go to this link:
> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>
> And then search for "GRIB2_mtab", and you'll find the content
> copied/pasted below. You can specify these INSTEAD OF specifying a
> requested name.
>
> // - The "GRIB2_mtab" is an integer specifying the master
table
> number.
> // - The "GRIB2_ltab" is an integer specifying the local table
> number.
> // - The "GRIB2_disc" is an integer specifying the GRIB2
discipline
> code.
> // - The "GRIB2_parm_cat" is an integer specifying the
parameter
> category
> // code.
> // - The "GRIB2_parm" is an integer specifying the parameter
code.
> // - The "GRIB2_pdt" is an integer specifying the product
definition
> // template (Table 4.0).
> // - The "GRIB2_process" is an integer specifying the
generating
> process
> // (Table 4.3).
> // - The "GRIB2_cntr" is an integer specifying the originating
> center.
> // - The "GRIB2_ens_type" is an integer specifying the
ensemble type
> // (Table 4.6).
> // - The "GRIB2_der_type" is an integer specifying the derived
> product
> // type (Table 4.7).
> // - The "GRIB2_stat_type" is an integer specifying the
statistical
> // processing type (Table 4.10).
> // - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val" entries
are
> arrays
> // of integers which specify the product description
template
> values to
> // be used. The indices are 0-based. For example, use the
> following to
> // request a GRIB2 record whose 9-th and 27-th product
description
> // template values are 1 and 2, respectively:
> // GRIB2_ipdtmpl_index=[8, 26]; GRIB2_ipdtmpl_val=[1, 2];
>
> While I don't have any ECMWF data on hand for testing, I do have a
file
> from the Air Force from which wgrib2 prints a similar result:
>
> 549:687031550:d=2017061000:var discipline=0 center=57 local_table=1
> parmcat=1 parm=192:surface:18 hour fcst:
>
> So I used that for testing. I ran the following plot_data_plane
command
> WITHOUT specifying a requested name:
>
>
> plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps 'GRIB2_disc=0;
> GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1; GRIB2_parm=192;
level="Z0";'
> -v 3
>
> And then I looked closely at the log message to confirm that it used
> record 549, as expected:
>
> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>
> So MET used these GRIB2 numbers I defined to do a table lookup and
find a
> match in the MET GRIB tables:
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>
> And it determined that that field should be called "CTSTM".
Hopefully, the
> ECMWF GRIB2 tables are already defined there in MET. If not, we can
always
> add them.
>
> Please try running plot_data_plane at verbosity level 3 on one of
these
> files first. And make sure it runs without error and the log message
prints
> out the "name" it found for that data. If so, you can just skip
using all
> those integers and revert back to using the name MET knows. If not,
then we
> may need to define a custom GRIB table for this ECMWF GRIB2 data.
>
> Thanks,
> John
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Fri Dec 04 15:31:01 2020
We might already have them. I'm not sure.
But there's an additional required piece of the puzzle. Binbin runs
"wgrib2" to figure out what "name" he should use in the MET config
files.
If wgrib2 doesn't tell him the right "name", the problem will remain.
wgrib2 can be configured to define additional tables as well, but for
this
to be easy for Binbin, that'd mean updating the script that's run for
"module load grib_utils" on WCOSS. And I'm guessing that's not very
easy.
Yes, I do agree that the best solution is getting the ECMWF GRIB
tables and
adding them for both wgrib2 and MET.
John
On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <jensen at ucar.edu> wrote:
> John,
>
> It seems to me that it would be worthwhile to just ask ECMWF for
their
> Grib2 Table for their Ensemble files. And other operational centers
for
> that matter. Do you agree this would be a good solution?
>
> Thanks, Tara
>
> ---------- Forwarded message ---------
> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> Date: Fri, Dec 4, 2020 at 11:06 AM
> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for reading
ECMWF
> ensmeble data
> To:
>
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> I tested using following command:
>
> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";' -v3
>
> ERROR :
> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
> GRIB2_parm_cat and GRIB2_parm must be specified in field information
>
> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v
> 3
>
> Still got error
> ERROR :
> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
> GRIB2_parm_cat and GRIB2_parm must be specified in field information
>
> Looks like discipline=255 is not recognized
>
> Binbin
>
>
> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binbin,
> > You can do something similar with MET. Please go to this link:
> > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> >
> > And then search for "GRIB2_mtab", and you'll find the content
> > copied/pasted below. You can specify these INSTEAD OF specifying a
> > requested name.
> >
> > // - The "GRIB2_mtab" is an integer specifying the master
table
> > number.
> > // - The "GRIB2_ltab" is an integer specifying the local
table
> > number.
> > // - The "GRIB2_disc" is an integer specifying the GRIB2
discipline
> > code.
> > // - The "GRIB2_parm_cat" is an integer specifying the
parameter
> > category
> > // code.
> > // - The "GRIB2_parm" is an integer specifying the parameter
code.
> > // - The "GRIB2_pdt" is an integer specifying the product
> definition
> > // template (Table 4.0).
> > // - The "GRIB2_process" is an integer specifying the
generating
> > process
> > // (Table 4.3).
> > // - The "GRIB2_cntr" is an integer specifying the
originating
> > center.
> > // - The "GRIB2_ens_type" is an integer specifying the
ensemble
> type
> > // (Table 4.6).
> > // - The "GRIB2_der_type" is an integer specifying the
derived
> > product
> > // type (Table 4.7).
> > // - The "GRIB2_stat_type" is an integer specifying the
statistical
> > // processing type (Table 4.10).
> > // - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val"
entries are
> > arrays
> > // of integers which specify the product description
template
> > values to
> > // be used. The indices are 0-based. For example, use the
> > following to
> > // request a GRIB2 record whose 9-th and 27-th product
> description
> > // template values are 1 and 2, respectively:
> > // GRIB2_ipdtmpl_index=[8, 26]; GRIB2_ipdtmpl_val=[1,
2];
> >
> > While I don't have any ECMWF data on hand for testing, I do have a
file
> > from the Air Force from which wgrib2 prints a similar result:
> >
> > 549:687031550:d=2017061000:var discipline=0 center=57
local_table=1
> > parmcat=1 parm=192:surface:18 hour fcst:
> >
> > So I used that for testing. I ran the following plot_data_plane
command
> > WITHOUT specifying a requested name:
> >
> >
> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
'GRIB2_disc=0;
> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1; GRIB2_parm=192;
> level="Z0";'
> > -v 3
> >
> > And then I looked closely at the log message to confirm that it
used
> > record 549, as expected:
> >
> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> >
> > So MET used these GRIB2 numbers I defined to do a table lookup and
find a
> > match in the MET GRIB tables:
> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> >
> > And it determined that that field should be called "CTSTM".
Hopefully,
> the
> > ECMWF GRIB2 tables are already defined there in MET. If not, we
can
> always
> > add them.
> >
> > Please try running plot_data_plane at verbosity level 3 on one of
these
> > files first. And make sure it runs without error and the log
message
> prints
> > out the "name" it found for that data. If so, you can just skip
using all
> > those integers and revert back to using the name MET knows. If
not, then
> we
> > may need to define a custom GRIB table for this ECMWF GRIB2 data.
> >
> > Thanks,
> > John
> >
>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Tara Jensen
> Project Manager II
> NCAR RAL and DTC
> PO Box 3000, Boulder, Colorado 80307 USA
> +1 303-497-8479 jensen at ucar.edu
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Fri Dec 04 15:48:11 2020
Binbin,
OK, working on Venus.
You have a typo in the command you sent.
plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v
3
You have "RIB2_parm_cat" that should be "GRIB2_parm_cat". After
correcting
that I did still get an error but it's a very useful one!
module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
module load met/9.1
module load grib_util/1.1.1
cd
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
$WGRIB2 ensposte.t00z.z500.grd3.grib2
plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
'GRIB2_disc=255;
GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5; level="P500";'
*ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter found with
matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm (5). Use the
MET_GRIB_TABLES environment variable to define custom GRIB
tables.ERROR : *
So this table is not yet defined in MET. I checked their website:
https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
But I don't see an entry defined for GRIB2_parm=5;
3 things...
(1) As a crude temporary solution, you could refer to it by it's
record
number. Let's assume you know that record 1 has temperature data:
plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps 'name="TMP";
level="R1"; set_attr_level="P500";' -v 5
Here I'm reading record 1, naming it TMP and the using set_attr_level
write
the output level as P500 instead of R1.
(2) You could create a custom GRIB table yourself locally, and then
set the
MET_GRIB_TABLES environment variable to point MET to it.
(3) The best solution is contacting ECMWF and requesting they send
over the
GRIB2 table we should be using to decode this data.
Thanks,
John
On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> We might already have them. I'm not sure.
>
> But there's an additional required piece of the puzzle. Binbin runs
> "wgrib2" to figure out what "name" he should use in the MET config
files.
> If wgrib2 doesn't tell him the right "name", the problem will
remain.
> wgrib2 can be configured to define additional tables as well, but
for this
> to be easy for Binbin, that'd mean updating the script that's run
for
> "module load grib_utils" on WCOSS. And I'm guessing that's not very
easy.
>
> Yes, I do agree that the best solution is getting the ECMWF GRIB
tables
> and adding them for both wgrib2 and MET.
>
> John
>
> On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <jensen at ucar.edu> wrote:
>
>> John,
>>
>> It seems to me that it would be worthwhile to just ask ECMWF for
their
>> Grib2 Table for their Ensemble files. And other operational
centers for
>> that matter. Do you agree this would be a good solution?
>>
>> Thanks, Tara
>>
>> ---------- Forwarded message ---------
>> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
>> Date: Fri, Dec 4, 2020 at 11:06 AM
>> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for reading
ECMWF
>> ensmeble data
>> To:
>>
>>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>>
>> John,
>>
>> I tested using following command:
>>
>> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";' -v3
>>
>> ERROR :
>> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
>> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
>>
>> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> -v
>> 3
>>
>> Still got error
>> ERROR :
>> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
>> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
>>
>> Looks like discipline=255 is not recognized
>>
>> Binbin
>>
>>
>> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> > Binbin,
>> > You can do something similar with MET. Please go to this link:
>> > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>> >
>> > And then search for "GRIB2_mtab", and you'll find the content
>> > copied/pasted below. You can specify these INSTEAD OF specifying
a
>> > requested name.
>> >
>> > // - The "GRIB2_mtab" is an integer specifying the master
table
>> > number.
>> > // - The "GRIB2_ltab" is an integer specifying the local
table
>> > number.
>> > // - The "GRIB2_disc" is an integer specifying the GRIB2
>> discipline
>> > code.
>> > // - The "GRIB2_parm_cat" is an integer specifying the
parameter
>> > category
>> > // code.
>> > // - The "GRIB2_parm" is an integer specifying the
parameter code.
>> > // - The "GRIB2_pdt" is an integer specifying the product
>> definition
>> > // template (Table 4.0).
>> > // - The "GRIB2_process" is an integer specifying the
generating
>> > process
>> > // (Table 4.3).
>> > // - The "GRIB2_cntr" is an integer specifying the
originating
>> > center.
>> > // - The "GRIB2_ens_type" is an integer specifying the
ensemble
>> type
>> > // (Table 4.6).
>> > // - The "GRIB2_der_type" is an integer specifying the
derived
>> > product
>> > // type (Table 4.7).
>> > // - The "GRIB2_stat_type" is an integer specifying the
>> statistical
>> > // processing type (Table 4.10).
>> > // - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val"
entries are
>> > arrays
>> > // of integers which specify the product description
template
>> > values to
>> > // be used. The indices are 0-based. For example, use the
>> > following to
>> > // request a GRIB2 record whose 9-th and 27-th product
>> description
>> > // template values are 1 and 2, respectively:
>> > // GRIB2_ipdtmpl_index=[8, 26]; GRIB2_ipdtmpl_val=[1,
2];
>> >
>> > While I don't have any ECMWF data on hand for testing, I do have
a file
>> > from the Air Force from which wgrib2 prints a similar result:
>> >
>> > 549:687031550:d=2017061000:var discipline=0 center=57
local_table=1
>> > parmcat=1 parm=192:surface:18 hour fcst:
>> >
>> > So I used that for testing. I ran the following plot_data_plane
command
>> > WITHOUT specifying a requested name:
>> >
>> >
>> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
'GRIB2_disc=0;
>> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1; GRIB2_parm=192;
>> level="Z0";'
>> > -v 3
>> >
>> > And then I looked closely at the log message to confirm that it
used
>> > record 549, as expected:
>> >
>> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
>> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
>> >
>>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>> >
>> > So MET used these GRIB2 numbers I defined to do a table lookup
and find
>> a
>> > match in the MET GRIB tables:
>> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>> >
>> > And it determined that that field should be called "CTSTM".
Hopefully,
>> the
>> > ECMWF GRIB2 tables are already defined there in MET. If not, we
can
>> always
>> > add them.
>> >
>> > Please try running plot_data_plane at verbosity level 3 on one of
these
>> > files first. And make sure it runs without error and the log
message
>> prints
>> > out the "name" it found for that data. If so, you can just skip
using
>> all
>> > those integers and revert back to using the name MET knows. If
not,
>> then we
>> > may need to define a custom GRIB table for this ECMWF GRIB2 data.
>> >
>> > Thanks,
>> > John
>> >
>>
>>
>> --
>>
>> Binbin Zhou
>>
>> IMSG at NOAA/NWS/NCEP/EMC
>>
>> 5830 University Research Ct.
>>
>> College Park, MD 20740
>>
>> Binbin.Zhou at noaa.gov
>>
>> 301-683-3683
>>
>>
>>
>> --
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Tara Jensen
>> Project Manager II
>> NCAR RAL and DTC
>> PO Box 3000, Boulder, Colorado 80307 USA
>> +1 303-497-8479 jensen at ucar.edu
>>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Mon Dec 07 14:49:00 2020
John,
Our ECMWF grib2 ensemble data files are converted from its grib1
format
data.
Before conversion, the parameter names are there, but after
conversion, the
names
are gone. It is because of unknown tables. The grib1 data is still
original. The original
grib1 data files are very strange data format we can not read. So we
have
to use
a Frotran code to re-arrange the fields and save them as NCEP-like
grib1
data files.
So after several processes, some important info like tables could be
lost.
I'll test your first suggestion since the field record order is
fixed.
Currently we still
obtained grib1 data from ECWMF, eventually, we will get grib2 data
directly. So before
we can get original grib2 files from ECMWF, we just use this temporal
solution.
'
Thanks!
On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> OK, working on Venus.
>
> You have a typo in the command you sent.
> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v
> 3
>
> You have "RIB2_parm_cat" that should be "GRIB2_parm_cat". After
correcting
> that I did still get an error but it's a very useful one!
>
> module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> module load met/9.1
> module load grib_util/1.1.1
> cd
>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> $WGRIB2 ensposte.t00z.z500.grd3.grib2
> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
'GRIB2_disc=255;
> GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5; level="P500";'
>
>
>
> *ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter found
with
> matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm (5). Use the
> MET_GRIB_TABLES environment variable to define custom GRIB
tables.ERROR :
> *
>
> So this table is not yet defined in MET. I checked their website:
>
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
>
> But I don't see an entry defined for GRIB2_parm=5;
>
> 3 things...
>
> (1) As a crude temporary solution, you could refer to it by it's
record
> number. Let's assume you know that record 1 has temperature data:
>
> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
'name="TMP";
> level="R1"; set_attr_level="P500";' -v 5
>
> Here I'm reading record 1, naming it TMP and the using
set_attr_level write
> the output level as P500 instead of R1.
>
> (2) You could create a custom GRIB table yourself locally, and then
set the
> MET_GRIB_TABLES environment variable to point MET to it.
>
> (3) The best solution is contacting ECMWF and requesting they send
over the
> GRIB2 table we should be using to decode this data.
>
> Thanks,
> John
>
> On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> > We might already have them. I'm not sure.
> >
> > But there's an additional required piece of the puzzle. Binbin
runs
> > "wgrib2" to figure out what "name" he should use in the MET config
files.
> > If wgrib2 doesn't tell him the right "name", the problem will
remain.
> > wgrib2 can be configured to define additional tables as well, but
for
> this
> > to be easy for Binbin, that'd mean updating the script that's run
for
> > "module load grib_utils" on WCOSS. And I'm guessing that's not
very easy.
> >
> > Yes, I do agree that the best solution is getting the ECMWF GRIB
tables
> > and adding them for both wgrib2 and MET.
> >
> > John
> >
> > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <jensen at ucar.edu>
wrote:
> >
> >> John,
> >>
> >> It seems to me that it would be worthwhile to just ask ECMWF for
their
> >> Grib2 Table for their Ensemble files. And other operational
centers for
> >> that matter. Do you agree this would be a good solution?
> >>
> >> Thanks, Tara
> >>
> >> ---------- Forwarded message ---------
> >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> >> Date: Fri, Dec 4, 2020 at 11:06 AM
> >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for reading
ECMWF
> >> ensmeble data
> >> To:
> >>
> >>
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >>
> >> John,
> >>
> >> I tested using following command:
> >>
> >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";' -v3
> >>
> >> ERROR :
> >> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
> >> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
> >>
> >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> -v
> >> 3
> >>
> >> Still got error
> >> ERROR :
> >> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
> >> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
> >>
> >> Looks like discipline=255 is not recognized
> >>
> >> Binbin
> >>
> >>
> >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> > Binbin,
> >> > You can do something similar with MET. Please go to this link:
> >> > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> >> >
> >> > And then search for "GRIB2_mtab", and you'll find the content
> >> > copied/pasted below. You can specify these INSTEAD OF
specifying a
> >> > requested name.
> >> >
> >> > // - The "GRIB2_mtab" is an integer specifying the master
table
> >> > number.
> >> > // - The "GRIB2_ltab" is an integer specifying the local
table
> >> > number.
> >> > // - The "GRIB2_disc" is an integer specifying the GRIB2
> >> discipline
> >> > code.
> >> > // - The "GRIB2_parm_cat" is an integer specifying the
parameter
> >> > category
> >> > // code.
> >> > // - The "GRIB2_parm" is an integer specifying the
parameter
> code.
> >> > // - The "GRIB2_pdt" is an integer specifying the product
> >> definition
> >> > // template (Table 4.0).
> >> > // - The "GRIB2_process" is an integer specifying the
generating
> >> > process
> >> > // (Table 4.3).
> >> > // - The "GRIB2_cntr" is an integer specifying the
originating
> >> > center.
> >> > // - The "GRIB2_ens_type" is an integer specifying the
ensemble
> >> type
> >> > // (Table 4.6).
> >> > // - The "GRIB2_der_type" is an integer specifying the
derived
> >> > product
> >> > // type (Table 4.7).
> >> > // - The "GRIB2_stat_type" is an integer specifying the
> >> statistical
> >> > // processing type (Table 4.10).
> >> > // - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val"
entries
> are
> >> > arrays
> >> > // of integers which specify the product description
template
> >> > values to
> >> > // be used. The indices are 0-based. For example, use
the
> >> > following to
> >> > // request a GRIB2 record whose 9-th and 27-th product
> >> description
> >> > // template values are 1 and 2, respectively:
> >> > // GRIB2_ipdtmpl_index=[8, 26];
GRIB2_ipdtmpl_val=[1, 2];
> >> >
> >> > While I don't have any ECMWF data on hand for testing, I do
have a
> file
> >> > from the Air Force from which wgrib2 prints a similar result:
> >> >
> >> > 549:687031550:d=2017061000:var discipline=0 center=57
local_table=1
> >> > parmcat=1 parm=192:surface:18 hour fcst:
> >> >
> >> > So I used that for testing. I ran the following plot_data_plane
> command
> >> > WITHOUT specifying a requested name:
> >> >
> >> >
> >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
'GRIB2_disc=0;
> >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1; GRIB2_parm=192;
> >> level="Z0";'
> >> > -v 3
> >> >
> >> > And then I looked closely at the log message to confirm that it
used
> >> > record 549, as expected:
> >> >
> >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match
for
> >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
> >> >
> >>
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> >> >
> >> > So MET used these GRIB2 numbers I defined to do a table lookup
and
> find
> >> a
> >> > match in the MET GRIB tables:
> >> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> >> >
> >> > And it determined that that field should be called "CTSTM".
Hopefully,
> >> the
> >> > ECMWF GRIB2 tables are already defined there in MET. If not, we
can
> >> always
> >> > add them.
> >> >
> >> > Please try running plot_data_plane at verbosity level 3 on one
of
> these
> >> > files first. And make sure it runs without error and the log
message
> >> prints
> >> > out the "name" it found for that data. If so, you can just skip
using
> >> all
> >> > those integers and revert back to using the name MET knows. If
not,
> >> then we
> >> > may need to define a custom GRIB table for this ECMWF GRIB2
data.
> >> >
> >> > Thanks,
> >> > John
> >> >
> >>
> >>
> >> --
> >>
> >> Binbin Zhou
> >>
> >> IMSG at NOAA/NWS/NCEP/EMC
> >>
> >> 5830 University Research Ct.
> >>
> >> College Park, MD 20740
> >>
> >> Binbin.Zhou at noaa.gov
> >>
> >> 301-683-3683
> >>
> >>
> >>
> >> --
> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> Tara Jensen
> >> Project Manager II
> >> NCAR RAL and DTC
> >> PO Box 3000, Boulder, Colorado 80307 USA
> >> +1 303-497-8479 jensen at ucar.edu
> >>
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Tue Dec 08 13:56:51 2020
Binbin,
Sounds like there's a lot of moving parts there. How would you like to
proceed?
Should we check to see if MET can read those ECMWF GRIB1 files
directly
without needing to reformat?
Or do you just want to create a custom GRIB table for the result of
your
pre-processing and conversion steps?
John
On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> Our ECMWF grib2 ensemble data files are converted from its grib1
format
> data.
> Before conversion, the parameter names are there, but after
conversion, the
> names
> are gone. It is because of unknown tables. The grib1 data is still
> original. The original
> grib1 data files are very strange data format we can not read. So we
have
> to use
> a Frotran code to re-arrange the fields and save them as NCEP-like
grib1
> data files.
> So after several processes, some important info like tables could be
lost.
>
> I'll test your first suggestion since the field record order is
fixed.
> Currently we still
> obtained grib1 data from ECWMF, eventually, we will get grib2 data
> directly. So before
> we can get original grib2 files from ECMWF, we just use this
temporal
> solution.
> '
> Thanks!
>
>
> On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binbin,
> >
> > OK, working on Venus.
> >
> > You have a typo in the command you sent.
> > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> -v
> > 3
> >
> > You have "RIB2_parm_cat" that should be "GRIB2_parm_cat". After
> correcting
> > that I did still get an error but it's a very useful one!
> >
> > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > module load met/9.1
> > module load grib_util/1.1.1
> > cd
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> 'GRIB2_disc=255;
> > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5; level="P500";'
> >
> >
> >
> > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter found
with
> > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm (5). Use
the
> > MET_GRIB_TABLES environment variable to define custom GRIB
tables.ERROR
> :
> > *
> >
> > So this table is not yet defined in MET. I checked their website:
> >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> >
> > But I don't see an entry defined for GRIB2_parm=5;
> >
> > 3 things...
> >
> > (1) As a crude temporary solution, you could refer to it by it's
record
> > number. Let's assume you know that record 1 has temperature data:
> >
> > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
'name="TMP";
> > level="R1"; set_attr_level="P500";' -v 5
> >
> > Here I'm reading record 1, naming it TMP and the using
set_attr_level
> write
> > the output level as P500 instead of R1.
> >
> > (2) You could create a custom GRIB table yourself locally, and
then set
> the
> > MET_GRIB_TABLES environment variable to point MET to it.
> >
> > (3) The best solution is contacting ECMWF and requesting they send
over
> the
> > GRIB2 table we should be using to decode this data.
> >
> > Thanks,
> > John
> >
> > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
> >
> > > We might already have them. I'm not sure.
> > >
> > > But there's an additional required piece of the puzzle. Binbin
runs
> > > "wgrib2" to figure out what "name" he should use in the MET
config
> files.
> > > If wgrib2 doesn't tell him the right "name", the problem will
remain.
> > > wgrib2 can be configured to define additional tables as well,
but for
> > this
> > > to be easy for Binbin, that'd mean updating the script that's
run for
> > > "module load grib_utils" on WCOSS. And I'm guessing that's not
very
> easy.
> > >
> > > Yes, I do agree that the best solution is getting the ECMWF GRIB
tables
> > > and adding them for both wgrib2 and MET.
> > >
> > > John
> > >
> > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <jensen at ucar.edu>
wrote:
> > >
> > >> John,
> > >>
> > >> It seems to me that it would be worthwhile to just ask ECMWF
for their
> > >> Grib2 Table for their Ensemble files. And other operational
centers
> for
> > >> that matter. Do you agree this would be a good solution?
> > >>
> > >> Thanks, Tara
> > >>
> > >> ---------- Forwarded message ---------
> > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for
reading
> ECMWF
> > >> ensmeble data
> > >> To:
> > >>
> > >>
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >>
> > >> John,
> > >>
> > >> I tested using following command:
> > >>
> > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";' -v3
> > >>
> > >> ERROR :
> > >> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
> > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
> > >>
> > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >>
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> -v
> > >> 3
> > >>
> > >> Still got error
> > >> ERROR :
> > >> ERROR : VarInfoGrib2::set_dict() -> either name or GRIB2_disc,
> > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
> > >>
> > >> Looks like discipline=255 is not recognized
> > >>
> > >> Binbin
> > >>
> > >>
> > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT <
> > >> met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> > Binbin,
> > >> > You can do something similar with MET. Please go to this
link:
> > >> >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > >> >
> > >> > And then search for "GRIB2_mtab", and you'll find the content
> > >> > copied/pasted below. You can specify these INSTEAD OF
specifying a
> > >> > requested name.
> > >> >
> > >> > // - The "GRIB2_mtab" is an integer specifying the
master
> table
> > >> > number.
> > >> > // - The "GRIB2_ltab" is an integer specifying the
local table
> > >> > number.
> > >> > // - The "GRIB2_disc" is an integer specifying the
GRIB2
> > >> discipline
> > >> > code.
> > >> > // - The "GRIB2_parm_cat" is an integer specifying the
> parameter
> > >> > category
> > >> > // code.
> > >> > // - The "GRIB2_parm" is an integer specifying the
parameter
> > code.
> > >> > // - The "GRIB2_pdt" is an integer specifying the
product
> > >> definition
> > >> > // template (Table 4.0).
> > >> > // - The "GRIB2_process" is an integer specifying the
> generating
> > >> > process
> > >> > // (Table 4.3).
> > >> > // - The "GRIB2_cntr" is an integer specifying the
originating
> > >> > center.
> > >> > // - The "GRIB2_ens_type" is an integer specifying the
> ensemble
> > >> type
> > >> > // (Table 4.6).
> > >> > // - The "GRIB2_der_type" is an integer specifying the
derived
> > >> > product
> > >> > // type (Table 4.7).
> > >> > // - The "GRIB2_stat_type" is an integer specifying the
> > >> statistical
> > >> > // processing type (Table 4.10).
> > >> > // - The "GRIB2_ipdtmpl_index" and "GRIB2_ipdtmpl_val"
entries
> > are
> > >> > arrays
> > >> > // of integers which specify the product description
> template
> > >> > values to
> > >> > // be used. The indices are 0-based. For example, use
the
> > >> > following to
> > >> > // request a GRIB2 record whose 9-th and 27-th
product
> > >> description
> > >> > // template values are 1 and 2, respectively:
> > >> > // GRIB2_ipdtmpl_index=[8, 26];
GRIB2_ipdtmpl_val=[1, 2];
> > >> >
> > >> > While I don't have any ECMWF data on hand for testing, I do
have a
> > file
> > >> > from the Air Force from which wgrib2 prints a similar result:
> > >> >
> > >> > 549:687031550:d=2017061000:var discipline=0 center=57
local_table=1
> > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > >> >
> > >> > So I used that for testing. I ran the following
plot_data_plane
> > command
> > >> > WITHOUT specifying a requested name:
> > >> >
> > >> >
> > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
> 'GRIB2_disc=0;
> > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
GRIB2_parm=192;
> > >> level="Z0";'
> > >> > -v 3
> > >> >
> > >> > And then I looked closely at the log message to confirm that
it used
> > >> > record 549, as expected:
> > >> >
> > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match
for
> > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
> > >> >
> > >>
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > >> >
> > >> > So MET used these GRIB2 numbers I defined to do a table
lookup and
> > find
> > >> a
> > >> > match in the MET GRIB tables:
> > >> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > >> >
> > >> > And it determined that that field should be called "CTSTM".
> Hopefully,
> > >> the
> > >> > ECMWF GRIB2 tables are already defined there in MET. If not,
we can
> > >> always
> > >> > add them.
> > >> >
> > >> > Please try running plot_data_plane at verbosity level 3 on
one of
> > these
> > >> > files first. And make sure it runs without error and the log
message
> > >> prints
> > >> > out the "name" it found for that data. If so, you can just
skip
> using
> > >> all
> > >> > those integers and revert back to using the name MET knows.
If not,
> > >> then we
> > >> > may need to define a custom GRIB table for this ECMWF GRIB2
data.
> > >> >
> > >> > Thanks,
> > >> > John
> > >> >
> > >>
> > >>
> > >> --
> > >>
> > >> Binbin Zhou
> > >>
> > >> IMSG at NOAA/NWS/NCEP/EMC
> > >>
> > >> 5830 University Research Ct.
> > >>
> > >> College Park, MD 20740
> > >>
> > >> Binbin.Zhou at noaa.gov
> > >>
> > >> 301-683-3683
> > >>
> > >>
> > >>
> > >> --
> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> Tara Jensen
> > >> Project Manager II
> > >> NCAR RAL and DTC
> > >> PO Box 3000, Boulder, Colorado 80307 USA
> > >> +1 303-497-8479 jensen at ucar.edu
> > >>
> > >
> >
> >
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Tue Dec 08 14:37:07 2020
John,
Grib1 data is too large, another issue is the low-res control
analysis
data can not be retrieved
from the grib1 ecmwf data files. If using wgrib, we can see 3 analysis
fields, but don't know
which is low-res control. After convert to grib2, wgrib2 can show
which
record is low-res control.
Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
Binbin
On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> Sounds like there's a lot of moving parts there. How would you like
to
> proceed?
>
> Should we check to see if MET can read those ECMWF GRIB1 files
directly
> without needing to reformat?
> Or do you just want to create a custom GRIB table for the result of
your
> pre-processing and conversion steps?
>
> John
>
> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > Our ECMWF grib2 ensemble data files are converted from its grib1
format
> > data.
> > Before conversion, the parameter names are there, but after
conversion,
> the
> > names
> > are gone. It is because of unknown tables. The grib1 data is still
> > original. The original
> > grib1 data files are very strange data format we can not read. So
we have
> > to use
> > a Frotran code to re-arrange the fields and save them as NCEP-like
grib1
> > data files.
> > So after several processes, some important info like tables could
be
> lost.
> >
> > I'll test your first suggestion since the field record order is
fixed.
> > Currently we still
> > obtained grib1 data from ECWMF, eventually, we will get grib2 data
> > directly. So before
> > we can get original grib2 files from ECMWF, we just use this
temporal
> > solution.
> > '
> > Thanks!
> >
> >
> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binbin,
> > >
> > > OK, working on Venus.
> > >
> > > You have a typo in the command you sent.
> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > -v
> > > 3
> > >
> > > You have "RIB2_parm_cat" that should be "GRIB2_parm_cat". After
> > correcting
> > > that I did still get an error but it's a very useful one!
> > >
> > > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > module load met/9.1
> > > module load grib_util/1.1.1
> > > cd
> > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> > 'GRIB2_disc=255;
> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5; level="P500";'
> > >
> > >
> > >
> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter
found with
> > > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm (5). Use
the
> > > MET_GRIB_TABLES environment variable to define custom GRIB
tables.ERROR
> > :
> > > *
> > >
> > > So this table is not yet defined in MET. I checked their
website:
> > >
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > >
> > > But I don't see an entry defined for GRIB2_parm=5;
> > >
> > > 3 things...
> > >
> > > (1) As a crude temporary solution, you could refer to it by it's
record
> > > number. Let's assume you know that record 1 has temperature
data:
> > >
> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
'name="TMP";
> > > level="R1"; set_attr_level="P500";' -v 5
> > >
> > > Here I'm reading record 1, naming it TMP and the using
set_attr_level
> > write
> > > the output level as P500 instead of R1.
> > >
> > > (2) You could create a custom GRIB table yourself locally, and
then set
> > the
> > > MET_GRIB_TABLES environment variable to point MET to it.
> > >
> > > (3) The best solution is contacting ECMWF and requesting they
send over
> > the
> > > GRIB2 table we should be using to decode this data.
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> > >
> > > > We might already have them. I'm not sure.
> > > >
> > > > But there's an additional required piece of the puzzle. Binbin
runs
> > > > "wgrib2" to figure out what "name" he should use in the MET
config
> > files.
> > > > If wgrib2 doesn't tell him the right "name", the problem will
remain.
> > > > wgrib2 can be configured to define additional tables as well,
but for
> > > this
> > > > to be easy for Binbin, that'd mean updating the script that's
run for
> > > > "module load grib_utils" on WCOSS. And I'm guessing that's not
very
> > easy.
> > > >
> > > > Yes, I do agree that the best solution is getting the ECMWF
GRIB
> tables
> > > > and adding them for both wgrib2 and MET.
> > > >
> > > > John
> > > >
> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <jensen at ucar.edu>
wrote:
> > > >
> > > >> John,
> > > >>
> > > >> It seems to me that it would be worthwhile to just ask ECMWF
for
> their
> > > >> Grib2 Table for their Ensemble files. And other operational
centers
> > for
> > > >> that matter. Do you agree this would be a good solution?
> > > >>
> > > >> Thanks, Tara
> > > >>
> > > >> ---------- Forwarded message ---------
> > > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for
reading
> > ECMWF
> > > >> ensmeble data
> > > >> To:
> > > >>
> > > >>
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > > >>
> > > >> John,
> > > >>
> > > >> I tested using following command:
> > > >>
> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > > >> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v3
> > > >>
> > > >> ERROR :
> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
> > > >>
> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > > >>
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> -v
> > > >> 3
> > > >>
> > > >> Still got error
> > > >> ERROR :
> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
information
> > > >>
> > > >> Looks like discipline=255 is not recognized
> > > >>
> > > >> Binbin
> > > >>
> > > >>
> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT <
> > > >> met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> > Binbin,
> > > >> > You can do something similar with MET. Please go to this
link:
> > > >> >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > >> >
> > > >> > And then search for "GRIB2_mtab", and you'll find the
content
> > > >> > copied/pasted below. You can specify these INSTEAD OF
specifying a
> > > >> > requested name.
> > > >> >
> > > >> > // - The "GRIB2_mtab" is an integer specifying the
master
> > table
> > > >> > number.
> > > >> > // - The "GRIB2_ltab" is an integer specifying the
local
> table
> > > >> > number.
> > > >> > // - The "GRIB2_disc" is an integer specifying the
GRIB2
> > > >> discipline
> > > >> > code.
> > > >> > // - The "GRIB2_parm_cat" is an integer specifying
the
> > parameter
> > > >> > category
> > > >> > // code.
> > > >> > // - The "GRIB2_parm" is an integer specifying the
parameter
> > > code.
> > > >> > // - The "GRIB2_pdt" is an integer specifying the
product
> > > >> definition
> > > >> > // template (Table 4.0).
> > > >> > // - The "GRIB2_process" is an integer specifying the
> > generating
> > > >> > process
> > > >> > // (Table 4.3).
> > > >> > // - The "GRIB2_cntr" is an integer specifying the
> originating
> > > >> > center.
> > > >> > // - The "GRIB2_ens_type" is an integer specifying
the
> > ensemble
> > > >> type
> > > >> > // (Table 4.6).
> > > >> > // - The "GRIB2_der_type" is an integer specifying
the
> derived
> > > >> > product
> > > >> > // type (Table 4.7).
> > > >> > // - The "GRIB2_stat_type" is an integer specifying
the
> > > >> statistical
> > > >> > // processing type (Table 4.10).
> > > >> > // - The "GRIB2_ipdtmpl_index" and
"GRIB2_ipdtmpl_val"
> entries
> > > are
> > > >> > arrays
> > > >> > // of integers which specify the product
description
> > template
> > > >> > values to
> > > >> > // be used. The indices are 0-based. For example,
use the
> > > >> > following to
> > > >> > // request a GRIB2 record whose 9-th and 27-th
product
> > > >> description
> > > >> > // template values are 1 and 2, respectively:
> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
GRIB2_ipdtmpl_val=[1,
> 2];
> > > >> >
> > > >> > While I don't have any ECMWF data on hand for testing, I do
have a
> > > file
> > > >> > from the Air Force from which wgrib2 prints a similar
result:
> > > >> >
> > > >> > 549:687031550:d=2017061000:var discipline=0 center=57
> local_table=1
> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > > >> >
> > > >> > So I used that for testing. I ran the following
plot_data_plane
> > > command
> > > >> > WITHOUT specifying a requested name:
> > > >> >
> > > >> >
> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
> > 'GRIB2_disc=0;
> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
GRIB2_parm=192;
> > > >> level="Z0";'
> > > >> > -v 3
> > > >> >
> > > >> > And then I looked closely at the log message to confirm
that it
> used
> > > >> > record 549, as expected:
> > > >> >
> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match for
> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
> > > >> >
> > > >>
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > >> >
> > > >> > So MET used these GRIB2 numbers I defined to do a table
lookup and
> > > find
> > > >> a
> > > >> > match in the MET GRIB tables:
> > > >> >
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > >> >
> > > >> > And it determined that that field should be called "CTSTM".
> > Hopefully,
> > > >> the
> > > >> > ECMWF GRIB2 tables are already defined there in MET. If
not, we
> can
> > > >> always
> > > >> > add them.
> > > >> >
> > > >> > Please try running plot_data_plane at verbosity level 3 on
one of
> > > these
> > > >> > files first. And make sure it runs without error and the
log
> message
> > > >> prints
> > > >> > out the "name" it found for that data. If so, you can just
skip
> > using
> > > >> all
> > > >> > those integers and revert back to using the name MET knows.
If
> not,
> > > >> then we
> > > >> > may need to define a custom GRIB table for this ECMWF GRIB2
data.
> > > >> >
> > > >> > Thanks,
> > > >> > John
> > > >> >
> > > >>
> > > >>
> > > >> --
> > > >>
> > > >> Binbin Zhou
> > > >>
> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > >>
> > > >> 5830 University Research Ct.
> > > >>
> > > >> College Park, MD 20740
> > > >>
> > > >> Binbin.Zhou at noaa.gov
> > > >>
> > > >> 301-683-3683
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >> Tara Jensen
> > > >> Project Manager II
> > > >> NCAR RAL and DTC
> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > > >> +1 303-497-8479 jensen at ucar.edu
> > > >>
> > > >
> > >
> > >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Tue Dec 08 14:51:26 2020
John,
Look at my
directory /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
The grib1 file is ensposte.t00z.z500hr, this is only for 500 HGT.
wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
I get 3 records:
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
mb:anl:type=analysis:NAve=0
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
mb:anl:type=analysis:NAve=0
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
mb:anl:type=analysis:NAve=0
>From its grib2 file, we know only record#26 is analysis low-res
control.
If you have some way to determine which record is analysis low-res
control
from the grib1 file,
then we can use grib1 directly (don't consider the size issue).
Binbin
On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate <
binbin.zhou at noaa.gov> wrote:
> John,
>
> Grib1 data is too large, another issue is the low-res control
analysis
> data can not be retrieved
> from the grib1 ecmwf data files. If using wgrib, we can see 3
analysis
> fields, but don't know
> which is low-res control. After convert to grib2, wgrib2 can show
which
> record is low-res control.
> Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
>
> Binbin
>
> On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Binbin,
>>
>> Sounds like there's a lot of moving parts there. How would you like
to
>> proceed?
>>
>> Should we check to see if MET can read those ECMWF GRIB1 files
directly
>> without needing to reformat?
>> Or do you just want to create a custom GRIB table for the result of
your
>> pre-processing and conversion steps?
>>
>> John
>>
>> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> >
>> > John,
>> >
>> > Our ECMWF grib2 ensemble data files are converted from its
grib1
>> format
>> > data.
>> > Before conversion, the parameter names are there, but after
conversion,
>> the
>> > names
>> > are gone. It is because of unknown tables. The grib1 data is
still
>> > original. The original
>> > grib1 data files are very strange data format we can not read. So
we
>> have
>> > to use
>> > a Frotran code to re-arrange the fields and save them as NCEP-
like grib1
>> > data files.
>> > So after several processes, some important info like tables could
be
>> lost.
>> >
>> > I'll test your first suggestion since the field record order
is
>> fixed.
>> > Currently we still
>> > obtained grib1 data from ECWMF, eventually, we will get grib2
data
>> > directly. So before
>> > we can get original grib2 files from ECMWF, we just use this
temporal
>> > solution.
>> > '
>> > Thanks!
>> >
>> >
>> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Binbin,
>> > >
>> > > OK, working on Venus.
>> > >
>> > > You have a typo in the command you sent.
>> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> > >
>>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > -v
>> > > 3
>> > >
>> > > You have "RIB2_parm_cat" that should be "GRIB2_parm_cat". After
>> > correcting
>> > > that I did still get an error but it's a very useful one!
>> > >
>> > > module use
>> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
>> > > module load met/9.1
>> > > module load grib_util/1.1.1
>> > > cd
>> > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
>> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
>> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
>> > 'GRIB2_disc=255;
>> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5; level="P500";'
>> > >
>> > >
>> > >
>> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter
found with
>> > > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm (5).
Use the
>> > > MET_GRIB_TABLES environment variable to define custom GRIB
>> tables.ERROR
>> > :
>> > > *
>> > >
>> > > So this table is not yet defined in MET. I checked their
website:
>> > >
>> > >
>> >
>> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
>> > >
>> > > But I don't see an entry defined for GRIB2_parm=5;
>> > >
>> > > 3 things...
>> > >
>> > > (1) As a crude temporary solution, you could refer to it by
it's
>> record
>> > > number. Let's assume you know that record 1 has temperature
data:
>> > >
>> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
'name="TMP";
>> > > level="R1"; set_attr_level="P500";' -v 5
>> > >
>> > > Here I'm reading record 1, naming it TMP and the using
set_attr_level
>> > write
>> > > the output level as P500 instead of R1.
>> > >
>> > > (2) You could create a custom GRIB table yourself locally, and
then
>> set
>> > the
>> > > MET_GRIB_TABLES environment variable to point MET to it.
>> > >
>> > > (3) The best solution is contacting ECMWF and requesting they
send
>> over
>> > the
>> > > GRIB2 table we should be using to decode this data.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway
<johnhg at ucar.edu>
>> > wrote:
>> > >
>> > > > We might already have them. I'm not sure.
>> > > >
>> > > > But there's an additional required piece of the puzzle.
Binbin runs
>> > > > "wgrib2" to figure out what "name" he should use in the MET
config
>> > files.
>> > > > If wgrib2 doesn't tell him the right "name", the problem will
>> remain.
>> > > > wgrib2 can be configured to define additional tables as well,
but
>> for
>> > > this
>> > > > to be easy for Binbin, that'd mean updating the script that's
run
>> for
>> > > > "module load grib_utils" on WCOSS. And I'm guessing that's
not very
>> > easy.
>> > > >
>> > > > Yes, I do agree that the best solution is getting the ECMWF
GRIB
>> tables
>> > > > and adding them for both wgrib2 and MET.
>> > > >
>> > > > John
>> > > >
>> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <jensen at ucar.edu>
wrote:
>> > > >
>> > > >> John,
>> > > >>
>> > > >> It seems to me that it would be worthwhile to just ask ECMWF
for
>> their
>> > > >> Grib2 Table for their Ensemble files. And other operational
>> centers
>> > for
>> > > >> that matter. Do you agree this would be a good solution?
>> > > >>
>> > > >> Thanks, Tara
>> > > >>
>> > > >> ---------- Forwarded message ---------
>> > > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
>> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
>> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for
reading
>> > ECMWF
>> > > >> ensmeble data
>> > > >> To:
>> > > >>
>> > > >>
>> > > >>
>> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> > > >>
>> > > >> John,
>> > > >>
>> > > >> I tested using following command:
>> > > >>
>> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> > > >> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v3
>> > > >>
>> > > >> ERROR :
>> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
>> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
>> information
>> > > >>
>> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> > > >>
>> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > >> -v
>> > > >> 3
>> > > >>
>> > > >> Still got error
>> > > >> ERROR :
>> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
>> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
>> information
>> > > >>
>> > > >> Looks like discipline=255 is not recognized
>> > > >>
>> > > >> Binbin
>> > > >>
>> > > >>
>> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT <
>> > > >> met_help at ucar.edu>
>> > > >> wrote:
>> > > >>
>> > > >> > Binbin,
>> > > >> > You can do something similar with MET. Please go to this
link:
>> > > >> >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>> > > >> >
>> > > >> > And then search for "GRIB2_mtab", and you'll find the
content
>> > > >> > copied/pasted below. You can specify these INSTEAD OF
specifying
>> a
>> > > >> > requested name.
>> > > >> >
>> > > >> > // - The "GRIB2_mtab" is an integer specifying the
master
>> > table
>> > > >> > number.
>> > > >> > // - The "GRIB2_ltab" is an integer specifying the
local
>> table
>> > > >> > number.
>> > > >> > // - The "GRIB2_disc" is an integer specifying the
GRIB2
>> > > >> discipline
>> > > >> > code.
>> > > >> > // - The "GRIB2_parm_cat" is an integer specifying
the
>> > parameter
>> > > >> > category
>> > > >> > // code.
>> > > >> > // - The "GRIB2_parm" is an integer specifying the
>> parameter
>> > > code.
>> > > >> > // - The "GRIB2_pdt" is an integer specifying the
product
>> > > >> definition
>> > > >> > // template (Table 4.0).
>> > > >> > // - The "GRIB2_process" is an integer specifying
the
>> > generating
>> > > >> > process
>> > > >> > // (Table 4.3).
>> > > >> > // - The "GRIB2_cntr" is an integer specifying the
>> originating
>> > > >> > center.
>> > > >> > // - The "GRIB2_ens_type" is an integer specifying
the
>> > ensemble
>> > > >> type
>> > > >> > // (Table 4.6).
>> > > >> > // - The "GRIB2_der_type" is an integer specifying
the
>> derived
>> > > >> > product
>> > > >> > // type (Table 4.7).
>> > > >> > // - The "GRIB2_stat_type" is an integer specifying
the
>> > > >> statistical
>> > > >> > // processing type (Table 4.10).
>> > > >> > // - The "GRIB2_ipdtmpl_index" and
"GRIB2_ipdtmpl_val"
>> entries
>> > > are
>> > > >> > arrays
>> > > >> > // of integers which specify the product
description
>> > template
>> > > >> > values to
>> > > >> > // be used. The indices are 0-based. For example,
use the
>> > > >> > following to
>> > > >> > // request a GRIB2 record whose 9-th and 27-th
product
>> > > >> description
>> > > >> > // template values are 1 and 2, respectively:
>> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
GRIB2_ipdtmpl_val=[1,
>> 2];
>> > > >> >
>> > > >> > While I don't have any ECMWF data on hand for testing, I
do have
>> a
>> > > file
>> > > >> > from the Air Force from which wgrib2 prints a similar
result:
>> > > >> >
>> > > >> > 549:687031550:d=2017061000:var discipline=0 center=57
>> local_table=1
>> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
>> > > >> >
>> > > >> > So I used that for testing. I ran the following
plot_data_plane
>> > > command
>> > > >> > WITHOUT specifying a requested name:
>> > > >> >
>> > > >> >
>> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
>> > 'GRIB2_disc=0;
>> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
GRIB2_parm=192;
>> > > >> level="Z0";'
>> > > >> > -v 3
>> > > >> >
>> > > >> > And then I looked closely at the log message to confirm
that it
>> used
>> > > >> > record 549, as expected:
>> > > >> >
>> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match for
>> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
>> > > >> >
>> > > >>
>> > >
>> >
>>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>> > > >> >
>> > > >> > So MET used these GRIB2 numbers I defined to do a table
lookup
>> and
>> > > find
>> > > >> a
>> > > >> > match in the MET GRIB tables:
>> > > >> >
>> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>> > > >> >
>> > > >> > And it determined that that field should be called
"CTSTM".
>> > Hopefully,
>> > > >> the
>> > > >> > ECMWF GRIB2 tables are already defined there in MET. If
not, we
>> can
>> > > >> always
>> > > >> > add them.
>> > > >> >
>> > > >> > Please try running plot_data_plane at verbosity level 3 on
one of
>> > > these
>> > > >> > files first. And make sure it runs without error and the
log
>> message
>> > > >> prints
>> > > >> > out the "name" it found for that data. If so, you can just
skip
>> > using
>> > > >> all
>> > > >> > those integers and revert back to using the name MET
knows. If
>> not,
>> > > >> then we
>> > > >> > may need to define a custom GRIB table for this ECMWF
GRIB2 data.
>> > > >> >
>> > > >> > Thanks,
>> > > >> > John
>> > > >> >
>> > > >>
>> > > >>
>> > > >> --
>> > > >>
>> > > >> Binbin Zhou
>> > > >>
>> > > >> IMSG at NOAA/NWS/NCEP/EMC
>> > > >>
>> > > >> 5830 University Research Ct.
>> > > >>
>> > > >> College Park, MD 20740
>> > > >>
>> > > >> Binbin.Zhou at noaa.gov
>> > > >>
>> > > >> 301-683-3683
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > > >> Tara Jensen
>> > > >> Project Manager II
>> > > >> NCAR RAL and DTC
>> > > >> PO Box 3000, Boulder, Colorado 80307 USA
>> > > >> +1 303-497-8479 jensen at ucar.edu
>> > > >>
>> > > >
>> > >
>> > >
>> >
>> > --
>> >
>> > Binbin Zhou
>> >
>> > IMSG at NOAA/NWS/NCEP/EMC
>> >
>> > 5830 University Research Ct.
>> >
>> > College Park, MD 20740
>> >
>> > Binbin.Zhou at noaa.gov
>> >
>> > 301-683-3683
>> >
>> >
>>
>>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Tue Dec 08 15:43:19 2020
Binbin,
I used the "-PDS10" option for wgrib to dump those records:
[John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
| egrep "^11:|^26:|^43:"
11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0 0
0 21
2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0 0
0 21
2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0 0
0 21
2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
Looking VERY closely, there is a difference in the 42-nd column which
is
the 4-th from the last... records 11, 26, and 43 have values of 3, 1,
and 2
respectively.
On this page
(https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html),
byte 42 = 1 means "unperturbed control forecast".
So we *should* be able to use the "GRIB_ens" config option to
differentiate
between those records. BUT when I try, it doesn't work. And that's
because
of this line:
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
We only apply those filtering options to GRIB files created by NCEP.
If
you'd like, I could try removing that condition to see if we can pick
out
the single record of interest.
John
On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> Look at my
> directory /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> The grib1 file is ensposte.t00z.z500hr, this is only for 500 HGT.
> wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
> I get 3 records:
>
>
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> mb:anl:type=analysis:NAve=0
>
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> mb:anl:type=analysis:NAve=0
>
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> mb:anl:type=analysis:NAve=0
>
> From its grib2 file, we know only record#26 is analysis low-res
control.
> If you have some way to determine which record is analysis low-res
control
> from the grib1 file,
> then we can use grib1 directly (don't consider the size issue).
>
> Binbin
>
> On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate <
> binbin.zhou at noaa.gov> wrote:
>
> > John,
> >
> > Grib1 data is too large, another issue is the low-res control
analysis
> > data can not be retrieved
> > from the grib1 ecmwf data files. If using wgrib, we can see 3
analysis
> > fields, but don't know
> > which is low-res control. After convert to grib2, wgrib2 can show
which
> > record is low-res control.
> > Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
> >
> > Binbin
> >
> > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Binbin,
> >>
> >> Sounds like there's a lot of moving parts there. How would you
like to
> >> proceed?
> >>
> >> Should we check to see if MET can read those ECMWF GRIB1 files
directly
> >> without needing to reformat?
> >> Or do you just want to create a custom GRIB table for the result
of your
> >> pre-processing and conversion steps?
> >>
> >> John
> >>
> >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >> >
> >> > John,
> >> >
> >> > Our ECMWF grib2 ensemble data files are converted from its
grib1
> >> format
> >> > data.
> >> > Before conversion, the parameter names are there, but after
> conversion,
> >> the
> >> > names
> >> > are gone. It is because of unknown tables. The grib1 data is
still
> >> > original. The original
> >> > grib1 data files are very strange data format we can not read.
So we
> >> have
> >> > to use
> >> > a Frotran code to re-arrange the fields and save them as NCEP-
like
> grib1
> >> > data files.
> >> > So after several processes, some important info like tables
could be
> >> lost.
> >> >
> >> > I'll test your first suggestion since the field record order
is
> >> fixed.
> >> > Currently we still
> >> > obtained grib1 data from ECWMF, eventually, we will get grib2
data
> >> > directly. So before
> >> > we can get original grib2 files from ECMWF, we just use this
temporal
> >> > solution.
> >> > '
> >> > Thanks!
> >> >
> >> >
> >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
> >> > met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Binbin,
> >> > >
> >> > > OK, working on Venus.
> >> > >
> >> > > You have a typo in the command you sent.
> >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> > >
> >>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > -v
> >> > > 3
> >> > >
> >> > > You have "RIB2_parm_cat" that should be "GRIB2_parm_cat".
After
> >> > correcting
> >> > > that I did still get an error but it's a very useful one!
> >> > >
> >> > > module use
> >> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> >> > > module load met/9.1
> >> > > module load grib_util/1.1.1
> >> > > cd
> >> > >
> >> > >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> >> > 'GRIB2_disc=255;
> >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5; level="P500";'
> >> > >
> >> > >
> >> > >
> >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter
found
> with
> >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm (5).
Use the
> >> > > MET_GRIB_TABLES environment variable to define custom GRIB
> >> tables.ERROR
> >> > :
> >> > > *
> >> > >
> >> > > So this table is not yet defined in MET. I checked their
website:
> >> > >
> >> > >
> >> >
> >>
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> >> > >
> >> > > But I don't see an entry defined for GRIB2_parm=5;
> >> > >
> >> > > 3 things...
> >> > >
> >> > > (1) As a crude temporary solution, you could refer to it by
it's
> >> record
> >> > > number. Let's assume you know that record 1 has temperature
data:
> >> > >
> >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> 'name="TMP";
> >> > > level="R1"; set_attr_level="P500";' -v 5
> >> > >
> >> > > Here I'm reading record 1, naming it TMP and the using
> set_attr_level
> >> > write
> >> > > the output level as P500 instead of R1.
> >> > >
> >> > > (2) You could create a custom GRIB table yourself locally,
and then
> >> set
> >> > the
> >> > > MET_GRIB_TABLES environment variable to point MET to it.
> >> > >
> >> > > (3) The best solution is contacting ECMWF and requesting they
send
> >> over
> >> > the
> >> > > GRIB2 table we should be using to decode this data.
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway
<johnhg at ucar.edu>
> >> > wrote:
> >> > >
> >> > > > We might already have them. I'm not sure.
> >> > > >
> >> > > > But there's an additional required piece of the puzzle.
Binbin
> runs
> >> > > > "wgrib2" to figure out what "name" he should use in the MET
config
> >> > files.
> >> > > > If wgrib2 doesn't tell him the right "name", the problem
will
> >> remain.
> >> > > > wgrib2 can be configured to define additional tables as
well, but
> >> for
> >> > > this
> >> > > > to be easy for Binbin, that'd mean updating the script
that's run
> >> for
> >> > > > "module load grib_utils" on WCOSS. And I'm guessing that's
not
> very
> >> > easy.
> >> > > >
> >> > > > Yes, I do agree that the best solution is getting the ECMWF
GRIB
> >> tables
> >> > > > and adding them for both wgrib2 and MET.
> >> > > >
> >> > > > John
> >> > > >
> >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen
<jensen at ucar.edu>
> wrote:
> >> > > >
> >> > > >> John,
> >> > > >>
> >> > > >> It seems to me that it would be worthwhile to just ask
ECMWF for
> >> their
> >> > > >> Grib2 Table for their Ensemble files. And other
operational
> >> centers
> >> > for
> >> > > >> that matter. Do you agree this would be a good solution?
> >> > > >>
> >> > > >> Thanks, Tara
> >> > > >>
> >> > > >> ---------- Forwarded message ---------
> >> > > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question for
> reading
> >> > ECMWF
> >> > > >> ensmeble data
> >> > > >> To:
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >> > > >>
> >> > > >> John,
> >> > > >>
> >> > > >> I tested using following command:
> >> > > >>
> >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> > > >> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
-v3
> >> > > >>
> >> > > >> ERROR :
> >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
> >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
> >> information
> >> > > >>
> >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> > > >>
> >> >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > > >> -v
> >> > > >> 3
> >> > > >>
> >> > > >> Still got error
> >> > > >> ERROR :
> >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
> >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
> >> information
> >> > > >>
> >> > > >> Looks like discipline=255 is not recognized
> >> > > >>
> >> > > >> Binbin
> >> > > >>
> >> > > >>
> >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via RT
<
> >> > > >> met_help at ucar.edu>
> >> > > >> wrote:
> >> > > >>
> >> > > >> > Binbin,
> >> > > >> > You can do something similar with MET. Please go to this
link:
> >> > > >> >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> >> > > >> >
> >> > > >> > And then search for "GRIB2_mtab", and you'll find the
content
> >> > > >> > copied/pasted below. You can specify these INSTEAD OF
> specifying
> >> a
> >> > > >> > requested name.
> >> > > >> >
> >> > > >> > // - The "GRIB2_mtab" is an integer specifying the
master
> >> > table
> >> > > >> > number.
> >> > > >> > // - The "GRIB2_ltab" is an integer specifying the
local
> >> table
> >> > > >> > number.
> >> > > >> > // - The "GRIB2_disc" is an integer specifying the
GRIB2
> >> > > >> discipline
> >> > > >> > code.
> >> > > >> > // - The "GRIB2_parm_cat" is an integer specifying
the
> >> > parameter
> >> > > >> > category
> >> > > >> > // code.
> >> > > >> > // - The "GRIB2_parm" is an integer specifying the
> >> parameter
> >> > > code.
> >> > > >> > // - The "GRIB2_pdt" is an integer specifying the
product
> >> > > >> definition
> >> > > >> > // template (Table 4.0).
> >> > > >> > // - The "GRIB2_process" is an integer specifying
the
> >> > generating
> >> > > >> > process
> >> > > >> > // (Table 4.3).
> >> > > >> > // - The "GRIB2_cntr" is an integer specifying the
> >> originating
> >> > > >> > center.
> >> > > >> > // - The "GRIB2_ens_type" is an integer specifying
the
> >> > ensemble
> >> > > >> type
> >> > > >> > // (Table 4.6).
> >> > > >> > // - The "GRIB2_der_type" is an integer specifying
the
> >> derived
> >> > > >> > product
> >> > > >> > // type (Table 4.7).
> >> > > >> > // - The "GRIB2_stat_type" is an integer
specifying the
> >> > > >> statistical
> >> > > >> > // processing type (Table 4.10).
> >> > > >> > // - The "GRIB2_ipdtmpl_index" and
"GRIB2_ipdtmpl_val"
> >> entries
> >> > > are
> >> > > >> > arrays
> >> > > >> > // of integers which specify the product
description
> >> > template
> >> > > >> > values to
> >> > > >> > // be used. The indices are 0-based. For
example, use
> the
> >> > > >> > following to
> >> > > >> > // request a GRIB2 record whose 9-th and 27-th
product
> >> > > >> description
> >> > > >> > // template values are 1 and 2, respectively:
> >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> GRIB2_ipdtmpl_val=[1,
> >> 2];
> >> > > >> >
> >> > > >> > While I don't have any ECMWF data on hand for testing, I
do
> have
> >> a
> >> > > file
> >> > > >> > from the Air Force from which wgrib2 prints a similar
result:
> >> > > >> >
> >> > > >> > 549:687031550:d=2017061000:var discipline=0 center=57
> >> local_table=1
> >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> >> > > >> >
> >> > > >> > So I used that for testing. I ran the following
plot_data_plane
> >> > > command
> >> > > >> > WITHOUT specifying a requested name:
> >> > > >> >
> >> > > >> >
> >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
> >> > 'GRIB2_disc=0;
> >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
GRIB2_parm=192;
> >> > > >> level="Z0";'
> >> > > >> > -v 3
> >> > > >> >
> >> > > >> > And then I looked closely at the log message to confirm
that it
> >> used
> >> > > >> > record 549, as expected:
> >> > > >> >
> >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match
> for
> >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
> >> > > >> >
> >> > > >>
> >> > >
> >> >
> >>
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> >> > > >> >
> >> > > >> > So MET used these GRIB2 numbers I defined to do a table
lookup
> >> and
> >> > > find
> >> > > >> a
> >> > > >> > match in the MET GRIB tables:
> >> > > >> >
> >>
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> >> > > >> >
> >> > > >> > And it determined that that field should be called
"CTSTM".
> >> > Hopefully,
> >> > > >> the
> >> > > >> > ECMWF GRIB2 tables are already defined there in MET. If
not, we
> >> can
> >> > > >> always
> >> > > >> > add them.
> >> > > >> >
> >> > > >> > Please try running plot_data_plane at verbosity level 3
on one
> of
> >> > > these
> >> > > >> > files first. And make sure it runs without error and the
log
> >> message
> >> > > >> prints
> >> > > >> > out the "name" it found for that data. If so, you can
just skip
> >> > using
> >> > > >> all
> >> > > >> > those integers and revert back to using the name MET
knows. If
> >> not,
> >> > > >> then we
> >> > > >> > may need to define a custom GRIB table for this ECMWF
GRIB2
> data.
> >> > > >> >
> >> > > >> > Thanks,
> >> > > >> > John
> >> > > >> >
> >> > > >>
> >> > > >>
> >> > > >> --
> >> > > >>
> >> > > >> Binbin Zhou
> >> > > >>
> >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> >> > > >>
> >> > > >> 5830 University Research Ct.
> >> > > >>
> >> > > >> College Park, MD 20740
> >> > > >>
> >> > > >> Binbin.Zhou at noaa.gov
> >> > > >>
> >> > > >> 301-683-3683
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> --
> >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> > > >> Tara Jensen
> >> > > >> Project Manager II
> >> > > >> NCAR RAL and DTC
> >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> >> > > >> +1 303-497-8479 jensen at ucar.edu
> >> > > >>
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> >
> >> > Binbin Zhou
> >> >
> >> > IMSG at NOAA/NWS/NCEP/EMC
> >> >
> >> > 5830 University Research Ct.
> >> >
> >> > College Park, MD 20740
> >> >
> >> > Binbin.Zhou at noaa.gov
> >> >
> >> > 301-683-3683
> >> >
> >> >
> >>
> >>
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Tue Dec 08 16:46:20 2020
John,
Yes, please try to use this approach.
Thanks!
Binbin
Binbin
On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> I used the "-PDS10" option for wgrib to dump those records:
>
> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> | egrep "^11:|^26:|^43:"
> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0
0 0 21
> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0
0 0 21
> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0
0 0 21
> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
>
> Looking VERY closely, there is a difference in the 42-nd column
which is
> the 4-th from the last... records 11, 26, and 43 have values of 3,
1, and 2
> respectively.
>
> On this page
(https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> ),
> byte 42 = 1 means "unperturbed control forecast".
>
> So we *should* be able to use the "GRIB_ens" config option to
differentiate
> between those records. BUT when I try, it doesn't work. And that's
because
> of this line:
>
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
>
> We only apply those filtering options to GRIB files created by NCEP.
If
> you'd like, I could try removing that condition to see if we can
pick out
> the single record of interest.
>
> John
>
> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > Look at my
> > directory /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > The grib1 file is ensposte.t00z.z500hr, this is only for 500 HGT.
> > wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
> > I get 3 records:
> >
> >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > mb:anl:type=analysis:NAve=0
> >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > mb:anl:type=analysis:NAve=0
> >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > mb:anl:type=analysis:NAve=0
> >
> > From its grib2 file, we know only record#26 is analysis low-res
control.
> > If you have some way to determine which record is analysis low-res
> control
> > from the grib1 file,
> > then we can use grib1 directly (don't consider the size issue).
> >
> > Binbin
> >
> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate <
> > binbin.zhou at noaa.gov> wrote:
> >
> > > John,
> > >
> > > Grib1 data is too large, another issue is the low-res control
> analysis
> > > data can not be retrieved
> > > from the grib1 ecmwf data files. If using wgrib, we can see 3
analysis
> > > fields, but don't know
> > > which is low-res control. After convert to grib2, wgrib2 can
show which
> > > record is low-res control.
> > > Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
> > >
> > > Binbin
> > >
> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Binbin,
> > >>
> > >> Sounds like there's a lot of moving parts there. How would you
like to
> > >> proceed?
> > >>
> > >> Should we check to see if MET can read those ECMWF GRIB1 files
> directly
> > >> without needing to reformat?
> > >> Or do you just want to create a custom GRIB table for the
result of
> your
> > >> pre-processing and conversion steps?
> > >>
> > >> John
> > >>
> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > >> >
> > >> > John,
> > >> >
> > >> > Our ECMWF grib2 ensemble data files are converted from its
grib1
> > >> format
> > >> > data.
> > >> > Before conversion, the parameter names are there, but after
> > conversion,
> > >> the
> > >> > names
> > >> > are gone. It is because of unknown tables. The grib1 data is
still
> > >> > original. The original
> > >> > grib1 data files are very strange data format we can not
read. So we
> > >> have
> > >> > to use
> > >> > a Frotran code to re-arrange the fields and save them as
NCEP-like
> > grib1
> > >> > data files.
> > >> > So after several processes, some important info like tables
could be
> > >> lost.
> > >> >
> > >> > I'll test your first suggestion since the field record
order is
> > >> fixed.
> > >> > Currently we still
> > >> > obtained grib1 data from ECWMF, eventually, we will get grib2
data
> > >> > directly. So before
> > >> > we can get original grib2 files from ECMWF, we just use this
> temporal
> > >> > solution.
> > >> > '
> > >> > Thanks!
> > >> >
> > >> >
> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
> > >> > met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > Binbin,
> > >> > >
> > >> > > OK, working on Venus.
> > >> > >
> > >> > > You have a typo in the command you sent.
> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >> > >
> > >>
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > -v
> > >> > > 3
> > >> > >
> > >> > > You have "RIB2_parm_cat" that should be "GRIB2_parm_cat".
After
> > >> > correcting
> > >> > > that I did still get an error but it's a very useful one!
> > >> > >
> > >> > > module use
> > >> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > >> > > module load met/9.1
> > >> > > module load grib_util/1.1.1
> > >> > > cd
> > >> > >
> > >> > >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> > >> > 'GRIB2_disc=255;
> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
level="P500";'
> > >> > >
> > >> > >
> > >> > >
> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no parameter
found
> > with
> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm
(5). Use
> the
> > >> > > MET_GRIB_TABLES environment variable to define custom GRIB
> > >> tables.ERROR
> > >> > :
> > >> > > *
> > >> > >
> > >> > > So this table is not yet defined in MET. I checked their
website:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > >> > >
> > >> > > But I don't see an entry defined for GRIB2_parm=5;
> > >> > >
> > >> > > 3 things...
> > >> > >
> > >> > > (1) As a crude temporary solution, you could refer to it by
it's
> > >> record
> > >> > > number. Let's assume you know that record 1 has temperature
data:
> > >> > >
> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> > 'name="TMP";
> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > >> > >
> > >> > > Here I'm reading record 1, naming it TMP and the using
> > set_attr_level
> > >> > write
> > >> > > the output level as P500 instead of R1.
> > >> > >
> > >> > > (2) You could create a custom GRIB table yourself locally,
and
> then
> > >> set
> > >> > the
> > >> > > MET_GRIB_TABLES environment variable to point MET to it.
> > >> > >
> > >> > > (3) The best solution is contacting ECMWF and requesting
they send
> > >> over
> > >> > the
> > >> > > GRIB2 table we should be using to decode this data.
> > >> > >
> > >> > > Thanks,
> > >> > > John
> > >> > >
> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <
> johnhg at ucar.edu>
> > >> > wrote:
> > >> > >
> > >> > > > We might already have them. I'm not sure.
> > >> > > >
> > >> > > > But there's an additional required piece of the puzzle.
Binbin
> > runs
> > >> > > > "wgrib2" to figure out what "name" he should use in the
MET
> config
> > >> > files.
> > >> > > > If wgrib2 doesn't tell him the right "name", the problem
will
> > >> remain.
> > >> > > > wgrib2 can be configured to define additional tables as
well,
> but
> > >> for
> > >> > > this
> > >> > > > to be easy for Binbin, that'd mean updating the script
that's
> run
> > >> for
> > >> > > > "module load grib_utils" on WCOSS. And I'm guessing
that's not
> > very
> > >> > easy.
> > >> > > >
> > >> > > > Yes, I do agree that the best solution is getting the
ECMWF GRIB
> > >> tables
> > >> > > > and adding them for both wgrib2 and MET.
> > >> > > >
> > >> > > > John
> > >> > > >
> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen
<jensen at ucar.edu>
> > wrote:
> > >> > > >
> > >> > > >> John,
> > >> > > >>
> > >> > > >> It seems to me that it would be worthwhile to just ask
ECMWF
> for
> > >> their
> > >> > > >> Grib2 Table for their Ensemble files. And other
operational
> > >> centers
> > >> > for
> > >> > > >> that matter. Do you agree this would be a good
solution?
> > >> > > >>
> > >> > > >> Thanks, Tara
> > >> > > >>
> > >> > > >> ---------- Forwarded message ---------
> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question
for
> > reading
> > >> > ECMWF
> > >> > > >> ensmeble data
> > >> > > >> To:
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > >> > > >>
> > >> > > >> John,
> > >> > > >>
> > >> > > >> I tested using following command:
> > >> > > >>
> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >> > > >>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";' -v3
> > >> > > >>
> > >> > > >> ERROR :
> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
> > >> information
> > >> > > >>
> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >> > > >>
> > >> >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > > >> -v
> > >> > > >> 3
> > >> > > >>
> > >> > > >> Still got error
> > >> > > >> ERROR :
> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
GRIB2_disc,
> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in field
> > >> information
> > >> > > >>
> > >> > > >> Looks like discipline=255 is not recognized
> > >> > > >>
> > >> > > >> Binbin
> > >> > > >>
> > >> > > >>
> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via
RT <
> > >> > > >> met_help at ucar.edu>
> > >> > > >> wrote:
> > >> > > >>
> > >> > > >> > Binbin,
> > >> > > >> > You can do something similar with MET. Please go to
this
> link:
> > >> > > >> >
> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > >> > > >> >
> > >> > > >> > And then search for "GRIB2_mtab", and you'll find the
content
> > >> > > >> > copied/pasted below. You can specify these INSTEAD OF
> > specifying
> > >> a
> > >> > > >> > requested name.
> > >> > > >> >
> > >> > > >> > // - The "GRIB2_mtab" is an integer specifying
the
> master
> > >> > table
> > >> > > >> > number.
> > >> > > >> > // - The "GRIB2_ltab" is an integer specifying
the
> local
> > >> table
> > >> > > >> > number.
> > >> > > >> > // - The "GRIB2_disc" is an integer specifying
the
> GRIB2
> > >> > > >> discipline
> > >> > > >> > code.
> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
specifying the
> > >> > parameter
> > >> > > >> > category
> > >> > > >> > // code.
> > >> > > >> > // - The "GRIB2_parm" is an integer specifying
the
> > >> parameter
> > >> > > code.
> > >> > > >> > // - The "GRIB2_pdt" is an integer specifying
the
> product
> > >> > > >> definition
> > >> > > >> > // template (Table 4.0).
> > >> > > >> > // - The "GRIB2_process" is an integer
specifying the
> > >> > generating
> > >> > > >> > process
> > >> > > >> > // (Table 4.3).
> > >> > > >> > // - The "GRIB2_cntr" is an integer specifying
the
> > >> originating
> > >> > > >> > center.
> > >> > > >> > // - The "GRIB2_ens_type" is an integer
specifying the
> > >> > ensemble
> > >> > > >> type
> > >> > > >> > // (Table 4.6).
> > >> > > >> > // - The "GRIB2_der_type" is an integer
specifying the
> > >> derived
> > >> > > >> > product
> > >> > > >> > // type (Table 4.7).
> > >> > > >> > // - The "GRIB2_stat_type" is an integer
specifying the
> > >> > > >> statistical
> > >> > > >> > // processing type (Table 4.10).
> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
"GRIB2_ipdtmpl_val"
> > >> entries
> > >> > > are
> > >> > > >> > arrays
> > >> > > >> > // of integers which specify the product
description
> > >> > template
> > >> > > >> > values to
> > >> > > >> > // be used. The indices are 0-based. For
example, use
> > the
> > >> > > >> > following to
> > >> > > >> > // request a GRIB2 record whose 9-th and 27-th
> product
> > >> > > >> description
> > >> > > >> > // template values are 1 and 2, respectively:
> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> > GRIB2_ipdtmpl_val=[1,
> > >> 2];
> > >> > > >> >
> > >> > > >> > While I don't have any ECMWF data on hand for testing,
I do
> > have
> > >> a
> > >> > > file
> > >> > > >> > from the Air Force from which wgrib2 prints a similar
result:
> > >> > > >> >
> > >> > > >> > 549:687031550:d=2017061000:var discipline=0 center=57
> > >> local_table=1
> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > >> > > >> >
> > >> > > >> > So I used that for testing. I ran the following
> plot_data_plane
> > >> > > command
> > >> > > >> > WITHOUT specifying a requested name:
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
> > >> > 'GRIB2_disc=0;
> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
> GRIB2_parm=192;
> > >> > > >> level="Z0";'
> > >> > > >> > -v 3
> > >> > > >> >
> > >> > > >> > And then I looked closely at the log message to
confirm that
> it
> > >> used
> > >> > > >> > record 549, as expected:
> > >> > > >> >
> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact
match
> > for
> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
> > >> > > >> >
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > >> > > >> >
> > >> > > >> > So MET used these GRIB2 numbers I defined to do a
table
> lookup
> > >> and
> > >> > > find
> > >> > > >> a
> > >> > > >> > match in the MET GRIB tables:
> > >> > > >> >
> > >>
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > >> > > >> >
> > >> > > >> > And it determined that that field should be called
"CTSTM".
> > >> > Hopefully,
> > >> > > >> the
> > >> > > >> > ECMWF GRIB2 tables are already defined there in MET.
If not,
> we
> > >> can
> > >> > > >> always
> > >> > > >> > add them.
> > >> > > >> >
> > >> > > >> > Please try running plot_data_plane at verbosity level
3 on
> one
> > of
> > >> > > these
> > >> > > >> > files first. And make sure it runs without error and
the log
> > >> message
> > >> > > >> prints
> > >> > > >> > out the "name" it found for that data. If so, you can
just
> skip
> > >> > using
> > >> > > >> all
> > >> > > >> > those integers and revert back to using the name MET
knows.
> If
> > >> not,
> > >> > > >> then we
> > >> > > >> > may need to define a custom GRIB table for this ECMWF
GRIB2
> > data.
> > >> > > >> >
> > >> > > >> > Thanks,
> > >> > > >> > John
> > >> > > >> >
> > >> > > >>
> > >> > > >>
> > >> > > >> --
> > >> > > >>
> > >> > > >> Binbin Zhou
> > >> > > >>
> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > >> > > >>
> > >> > > >> 5830 University Research Ct.
> > >> > > >>
> > >> > > >> College Park, MD 20740
> > >> > > >>
> > >> > > >> Binbin.Zhou at noaa.gov
> > >> > > >>
> > >> > > >> 301-683-3683
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> --
> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > > >> Tara Jensen
> > >> > > >> Project Manager II
> > >> > > >> NCAR RAL and DTC
> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > >> > > >>
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> > --
> > >> >
> > >> > Binbin Zhou
> > >> >
> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> >
> > >> > 5830 University Research Ct.
> > >> >
> > >> > College Park, MD 20740
> > >> >
> > >> > Binbin.Zhou at noaa.gov
> > >> >
> > >> > 301-683-3683
> > >> >
> > >> >
> > >>
> > >>
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Tue Dec 08 17:27:23 2020
John,
If this removing impacts other applications, or takes some time
to be effective, then won't try. I will use another approach, such as
borrow GEFS's grib2 data structure as tank to save EC ensemble
data.
Binbin
On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
binbin.zhou at noaa.gov> wrote:
> John,
>
> Yes, please try to use this approach.
>
> Thanks!
>
> Binbin
>
> Binbin
>
> On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Binbin,
>>
>> I used the "-PDS10" option for wgrib to dump those records:
>>
>> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
>>
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
>> | egrep "^11:|^26:|^43:"
>> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0
0 0 21
>> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
>> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0
0 0 21
>> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
>> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1 0
0 0 21
>> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
>>
>> Looking VERY closely, there is a difference in the 42-nd column
which is
>> the 4-th from the last... records 11, 26, and 43 have values of 3,
1, and
>> 2
>> respectively.
>>
>> On this page
(https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
>> ),
>> byte 42 = 1 means "unperturbed control forecast".
>>
>> So we *should* be able to use the "GRIB_ens" config option to
>> differentiate
>> between those records. BUT when I try, it doesn't work. And that's
because
>> of this line:
>>
>>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
>>
>> We only apply those filtering options to GRIB files created by
NCEP. If
>> you'd like, I could try removing that condition to see if we can
pick out
>> the single record of interest.
>>
>> John
>>
>> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> >
>> > John,
>> >
>> > Look at my
>> > directory
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
>> > The grib1 file is ensposte.t00z.z500hr, this is only for 500 HGT.
>> > wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
>> > I get 3 records:
>> >
>> >
>> >
>>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > mb:anl:type=analysis:NAve=0
>> >
>> >
>>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > mb:anl:type=analysis:NAve=0
>> >
>> >
>>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > mb:anl:type=analysis:NAve=0
>> >
>> > From its grib2 file, we know only record#26 is analysis low-res
>> control.
>> > If you have some way to determine which record is analysis low-
res
>> control
>> > from the grib1 file,
>> > then we can use grib1 directly (don't consider the size issue).
>> >
>> > Binbin
>> >
>> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate <
>> > binbin.zhou at noaa.gov> wrote:
>> >
>> > > John,
>> > >
>> > > Grib1 data is too large, another issue is the low-res control
>> analysis
>> > > data can not be retrieved
>> > > from the grib1 ecmwf data files. If using wgrib, we can see 3
analysis
>> > > fields, but don't know
>> > > which is low-res control. After convert to grib2, wgrib2 can
show
>> which
>> > > record is low-res control.
>> > > Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
>> > >
>> > > Binbin
>> > >
>> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > >> Binbin,
>> > >>
>> > >> Sounds like there's a lot of moving parts there. How would you
like
>> to
>> > >> proceed?
>> > >>
>> > >> Should we check to see if MET can read those ECMWF GRIB1 files
>> directly
>> > >> without needing to reformat?
>> > >> Or do you just want to create a custom GRIB table for the
result of
>> your
>> > >> pre-processing and conversion steps?
>> > >>
>> > >> John
>> > >>
>> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
>> > >> met_help at ucar.edu> wrote:
>> > >>
>> > >> >
>> > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> > >> >
>> > >> > John,
>> > >> >
>> > >> > Our ECMWF grib2 ensemble data files are converted from its
grib1
>> > >> format
>> > >> > data.
>> > >> > Before conversion, the parameter names are there, but after
>> > conversion,
>> > >> the
>> > >> > names
>> > >> > are gone. It is because of unknown tables. The grib1 data is
still
>> > >> > original. The original
>> > >> > grib1 data files are very strange data format we can not
read. So
>> we
>> > >> have
>> > >> > to use
>> > >> > a Frotran code to re-arrange the fields and save them as
NCEP-like
>> > grib1
>> > >> > data files.
>> > >> > So after several processes, some important info like tables
could
>> be
>> > >> lost.
>> > >> >
>> > >> > I'll test your first suggestion since the field record
order is
>> > >> fixed.
>> > >> > Currently we still
>> > >> > obtained grib1 data from ECWMF, eventually, we will get
grib2 data
>> > >> > directly. So before
>> > >> > we can get original grib2 files from ECMWF, we just use this
>> temporal
>> > >> > solution.
>> > >> > '
>> > >> > Thanks!
>> > >> >
>> > >> >
>> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
>> > >> > met_help at ucar.edu>
>> > >> > wrote:
>> > >> >
>> > >> > > Binbin,
>> > >> > >
>> > >> > > OK, working on Venus.
>> > >> > >
>> > >> > > You have a typo in the command you sent.
>> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> > >> > >
>> > >>
>>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > >> > -v
>> > >> > > 3
>> > >> > >
>> > >> > > You have "RIB2_parm_cat" that should be "GRIB2_parm_cat".
After
>> > >> > correcting
>> > >> > > that I did still get an error but it's a very useful one!
>> > >> > >
>> > >> > > module use
>> > >> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
>> > >> > > module load met/9.1
>> > >> > > module load grib_util/1.1.1
>> > >> > > cd
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
>> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
>> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
>> > >> > 'GRIB2_disc=255;
>> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
level="P500";'
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
parameter found
>> > with
>> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm
(5). Use
>> the
>> > >> > > MET_GRIB_TABLES environment variable to define custom GRIB
>> > >> tables.ERROR
>> > >> > :
>> > >> > > *
>> > >> > >
>> > >> > > So this table is not yet defined in MET. I checked their
website:
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
>> > >> > >
>> > >> > > But I don't see an entry defined for GRIB2_parm=5;
>> > >> > >
>> > >> > > 3 things...
>> > >> > >
>> > >> > > (1) As a crude temporary solution, you could refer to it
by it's
>> > >> record
>> > >> > > number. Let's assume you know that record 1 has
temperature data:
>> > >> > >
>> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
>> > 'name="TMP";
>> > >> > > level="R1"; set_attr_level="P500";' -v 5
>> > >> > >
>> > >> > > Here I'm reading record 1, naming it TMP and the using
>> > set_attr_level
>> > >> > write
>> > >> > > the output level as P500 instead of R1.
>> > >> > >
>> > >> > > (2) You could create a custom GRIB table yourself locally,
and
>> then
>> > >> set
>> > >> > the
>> > >> > > MET_GRIB_TABLES environment variable to point MET to it.
>> > >> > >
>> > >> > > (3) The best solution is contacting ECMWF and requesting
they
>> send
>> > >> over
>> > >> > the
>> > >> > > GRIB2 table we should be using to decode this data.
>> > >> > >
>> > >> > > Thanks,
>> > >> > > John
>> > >> > >
>> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <
>> johnhg at ucar.edu>
>> > >> > wrote:
>> > >> > >
>> > >> > > > We might already have them. I'm not sure.
>> > >> > > >
>> > >> > > > But there's an additional required piece of the puzzle.
Binbin
>> > runs
>> > >> > > > "wgrib2" to figure out what "name" he should use in the
MET
>> config
>> > >> > files.
>> > >> > > > If wgrib2 doesn't tell him the right "name", the problem
will
>> > >> remain.
>> > >> > > > wgrib2 can be configured to define additional tables as
well,
>> but
>> > >> for
>> > >> > > this
>> > >> > > > to be easy for Binbin, that'd mean updating the script
that's
>> run
>> > >> for
>> > >> > > > "module load grib_utils" on WCOSS. And I'm guessing
that's not
>> > very
>> > >> > easy.
>> > >> > > >
>> > >> > > > Yes, I do agree that the best solution is getting the
ECMWF
>> GRIB
>> > >> tables
>> > >> > > > and adding them for both wgrib2 and MET.
>> > >> > > >
>> > >> > > > John
>> > >> > > >
>> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen
<jensen at ucar.edu>
>> > wrote:
>> > >> > > >
>> > >> > > >> John,
>> > >> > > >>
>> > >> > > >> It seems to me that it would be worthwhile to just ask
ECMWF
>> for
>> > >> their
>> > >> > > >> Grib2 Table for their Ensemble files. And other
operational
>> > >> centers
>> > >> > for
>> > >> > > >> that matter. Do you agree this would be a good
solution?
>> > >> > > >>
>> > >> > > >> Thanks, Tara
>> > >> > > >>
>> > >> > > >> ---------- Forwarded message ---------
>> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
>> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
>> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question
for
>> > reading
>> > >> > ECMWF
>> > >> > > >> ensmeble data
>> > >> > > >> To:
>> > >> > > >>
>> > >> > > >>
>> > >> > > >>
>> > >> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> >
>> > >> > > >>
>> > >> > > >> John,
>> > >> > > >>
>> > >> > > >> I tested using following command:
>> > >> > > >>
>> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> > >> > > >>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> -v3
>> > >> > > >>
>> > >> > > >> ERROR :
>> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
>> GRIB2_disc,
>> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in
field
>> > >> information
>> > >> > > >>
>> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
>> > >> > > >>
>> > >> >
>> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > >> > > >> -v
>> > >> > > >> 3
>> > >> > > >>
>> > >> > > >> Still got error
>> > >> > > >> ERROR :
>> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
>> GRIB2_disc,
>> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in
field
>> > >> information
>> > >> > > >>
>> > >> > > >> Looks like discipline=255 is not recognized
>> > >> > > >>
>> > >> > > >> Binbin
>> > >> > > >>
>> > >> > > >>
>> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway via
RT <
>> > >> > > >> met_help at ucar.edu>
>> > >> > > >> wrote:
>> > >> > > >>
>> > >> > > >> > Binbin,
>> > >> > > >> > You can do something similar with MET. Please go to
this
>> link:
>> > >> > > >> >
>> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>> > >> > > >> >
>> > >> > > >> > And then search for "GRIB2_mtab", and you'll find the
>> content
>> > >> > > >> > copied/pasted below. You can specify these INSTEAD OF
>> > specifying
>> > >> a
>> > >> > > >> > requested name.
>> > >> > > >> >
>> > >> > > >> > // - The "GRIB2_mtab" is an integer specifying
the
>> master
>> > >> > table
>> > >> > > >> > number.
>> > >> > > >> > // - The "GRIB2_ltab" is an integer specifying
the
>> local
>> > >> table
>> > >> > > >> > number.
>> > >> > > >> > // - The "GRIB2_disc" is an integer specifying
the
>> GRIB2
>> > >> > > >> discipline
>> > >> > > >> > code.
>> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
specifying the
>> > >> > parameter
>> > >> > > >> > category
>> > >> > > >> > // code.
>> > >> > > >> > // - The "GRIB2_parm" is an integer specifying
the
>> > >> parameter
>> > >> > > code.
>> > >> > > >> > // - The "GRIB2_pdt" is an integer specifying
the
>> product
>> > >> > > >> definition
>> > >> > > >> > // template (Table 4.0).
>> > >> > > >> > // - The "GRIB2_process" is an integer
specifying the
>> > >> > generating
>> > >> > > >> > process
>> > >> > > >> > // (Table 4.3).
>> > >> > > >> > // - The "GRIB2_cntr" is an integer specifying
the
>> > >> originating
>> > >> > > >> > center.
>> > >> > > >> > // - The "GRIB2_ens_type" is an integer
specifying the
>> > >> > ensemble
>> > >> > > >> type
>> > >> > > >> > // (Table 4.6).
>> > >> > > >> > // - The "GRIB2_der_type" is an integer
specifying the
>> > >> derived
>> > >> > > >> > product
>> > >> > > >> > // type (Table 4.7).
>> > >> > > >> > // - The "GRIB2_stat_type" is an integer
specifying
>> the
>> > >> > > >> statistical
>> > >> > > >> > // processing type (Table 4.10).
>> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
"GRIB2_ipdtmpl_val"
>> > >> entries
>> > >> > > are
>> > >> > > >> > arrays
>> > >> > > >> > // of integers which specify the product
description
>> > >> > template
>> > >> > > >> > values to
>> > >> > > >> > // be used. The indices are 0-based. For
example,
>> use
>> > the
>> > >> > > >> > following to
>> > >> > > >> > // request a GRIB2 record whose 9-th and 27-
th
>> product
>> > >> > > >> description
>> > >> > > >> > // template values are 1 and 2, respectively:
>> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
>> > GRIB2_ipdtmpl_val=[1,
>> > >> 2];
>> > >> > > >> >
>> > >> > > >> > While I don't have any ECMWF data on hand for
testing, I do
>> > have
>> > >> a
>> > >> > > file
>> > >> > > >> > from the Air Force from which wgrib2 prints a similar
>> result:
>> > >> > > >> >
>> > >> > > >> > 549:687031550:d=2017061000:var discipline=0 center=57
>> > >> local_table=1
>> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
>> > >> > > >> >
>> > >> > > >> > So I used that for testing. I ran the following
>> plot_data_plane
>> > >> > > command
>> > >> > > >> > WITHOUT specifying a requested name:
>> > >> > > >> >
>> > >> > > >> >
>> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2 plot.ps
>> > >> > 'GRIB2_disc=0;
>> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
>> GRIB2_parm=192;
>> > >> > > >> level="Z0";'
>> > >> > > >> > -v 3
>> > >> > > >> >
>> > >> > > >> > And then I looked closely at the log message to
confirm
>> that it
>> > >> used
>> > >> > > >> > record 549, as expected:
>> > >> > > >> >
>> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found
exact match
>> > for
>> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2 file
>> > >> > > >> >
>> > >> > > >>
>> > >> > >
>> > >> >
>> > >>
>> >
>>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>> > >> > > >> >
>> > >> > > >> > So MET used these GRIB2 numbers I defined to do a
table
>> lookup
>> > >> and
>> > >> > > find
>> > >> > > >> a
>> > >> > > >> > match in the MET GRIB tables:
>> > >> > > >> >
>> > >>
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>> > >> > > >> >
>> > >> > > >> > And it determined that that field should be called
"CTSTM".
>> > >> > Hopefully,
>> > >> > > >> the
>> > >> > > >> > ECMWF GRIB2 tables are already defined there in MET.
If
>> not, we
>> > >> can
>> > >> > > >> always
>> > >> > > >> > add them.
>> > >> > > >> >
>> > >> > > >> > Please try running plot_data_plane at verbosity level
3 on
>> one
>> > of
>> > >> > > these
>> > >> > > >> > files first. And make sure it runs without error and
the log
>> > >> message
>> > >> > > >> prints
>> > >> > > >> > out the "name" it found for that data. If so, you can
just
>> skip
>> > >> > using
>> > >> > > >> all
>> > >> > > >> > those integers and revert back to using the name MET
knows.
>> If
>> > >> not,
>> > >> > > >> then we
>> > >> > > >> > may need to define a custom GRIB table for this ECMWF
GRIB2
>> > data.
>> > >> > > >> >
>> > >> > > >> > Thanks,
>> > >> > > >> > John
>> > >> > > >> >
>> > >> > > >>
>> > >> > > >>
>> > >> > > >> --
>> > >> > > >>
>> > >> > > >> Binbin Zhou
>> > >> > > >>
>> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
>> > >> > > >>
>> > >> > > >> 5830 University Research Ct.
>> > >> > > >>
>> > >> > > >> College Park, MD 20740
>> > >> > > >>
>> > >> > > >> Binbin.Zhou at noaa.gov
>> > >> > > >>
>> > >> > > >> 301-683-3683
>> > >> > > >>
>> > >> > > >>
>> > >> > > >>
>> > >> > > >> --
>> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > >> > > >> Tara Jensen
>> > >> > > >> Project Manager II
>> > >> > > >> NCAR RAL and DTC
>> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
>> > >> > > >> +1 303-497-8479 jensen at ucar.edu
>> > >> > > >>
>> > >> > > >
>> > >> > >
>> > >> > >
>> > >> >
>> > >> > --
>> > >> >
>> > >> > Binbin Zhou
>> > >> >
>> > >> > IMSG at NOAA/NWS/NCEP/EMC
>> > >> >
>> > >> > 5830 University Research Ct.
>> > >> >
>> > >> > College Park, MD 20740
>> > >> >
>> > >> > Binbin.Zhou at noaa.gov
>> > >> >
>> > >> > 301-683-3683
>> > >> >
>> > >> >
>> > >>
>> > >>
>> > >
>> > > --
>> > >
>> > > Binbin Zhou
>> > >
>> > > IMSG at NOAA/NWS/NCEP/EMC
>> > >
>> > > 5830 University Research Ct.
>> > >
>> > > College Park, MD 20740
>> > >
>> > > Binbin.Zhou at noaa.gov
>> > >
>> > > 301-683-3683
>> > >
>> >
>> >
>> > --
>> >
>> > Binbin Zhou
>> >
>> > IMSG at NOAA/NWS/NCEP/EMC
>> >
>> > 5830 University Research Ct.
>> >
>> > College Park, MD 20740
>> >
>> > Binbin.Zhou at noaa.gov
>> >
>> > 301-683-3683
>> >
>> >
>>
>>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Wed Dec 09 14:14:08 2020
Binbin,
Success! When I recompile with that one line commented out,
plot_data_plane
is able to filter down all those records to the single one that we
want.
Note in the example below that I used the "lead_time" and "GRIB_ens"
options to filter the input records down.
plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
level="P500";
lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
DEBUG 1: Opening data file: ensposte.t00z.z500hr
DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact match
for
VarInfo "HGT/P500" in GRIB record 26 of GRIB file
"ensposte.t00z.z500hr".*
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "HGT/P500" in GRIB file "ensposte.t00z.z500hr".
DEBUG 1: Creating postscript file: plot.ps
Also note that this selects out record number 26.
Shall I make this change as a "bugfix" for met-9.1...which could be
included in a met-9.1.2 bugfix release? Or shall I just include it in
development for the next release (planned for March 2021)?
Thanks,
John
On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> If this removing impacts other applications, or takes some time
> to be effective, then won't try. I will use another approach, such
as
> borrow GEFS's grib2 data structure as tank to save EC ensemble
> data.
>
> Binbin
>
>
> On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
> binbin.zhou at noaa.gov> wrote:
>
> > John,
> >
> > Yes, please try to use this approach.
> >
> > Thanks!
> >
> > Binbin
> >
> > Binbin
> >
> > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Binbin,
> >>
> >> I used the "-PDS10" option for wgrib to dump those records:
> >>
> >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> >>
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> >> | egrep "^11:|^26:|^43:"
> >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1
0 0 0
> 21
> >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1
0 0 0
> 21
> >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0 1
0 0 0
> 21
> >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> >>
> >> Looking VERY closely, there is a difference in the 42-nd column
which is
> >> the 4-th from the last... records 11, 26, and 43 have values of
3, 1,
> and
> >> 2
> >> respectively.
> >>
> >> On this page (
> https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> >> ),
> >> byte 42 = 1 means "unperturbed control forecast".
> >>
> >> So we *should* be able to use the "GRIB_ens" config option to
> >> differentiate
> >> between those records. BUT when I try, it doesn't work. And
that's
> because
> >> of this line:
> >>
> >>
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> >>
> >> We only apply those filtering options to GRIB files created by
NCEP. If
> >> you'd like, I could try removing that condition to see if we can
pick
> out
> >> the single record of interest.
> >>
> >> John
> >>
> >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >> >
> >> > John,
> >> >
> >> > Look at my
> >> > directory
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> >> > The grib1 file is ensposte.t00z.z500hr, this is only for 500
HGT.
> >> > wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
> >> > I get 3 records:
> >> >
> >> >
> >> >
> >>
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >> > mb:anl:type=analysis:NAve=0
> >> >
> >> >
> >>
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >> > mb:anl:type=analysis:NAve=0
> >> >
> >> >
> >>
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >> > mb:anl:type=analysis:NAve=0
> >> >
> >> > From its grib2 file, we know only record#26 is analysis low-
res
> >> control.
> >> > If you have some way to determine which record is analysis low-
res
> >> control
> >> > from the grib1 file,
> >> > then we can use grib1 directly (don't consider the size issue).
> >> >
> >> > Binbin
> >> >
> >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate <
> >> > binbin.zhou at noaa.gov> wrote:
> >> >
> >> > > John,
> >> > >
> >> > > Grib1 data is too large, another issue is the low-res
control
> >> analysis
> >> > > data can not be retrieved
> >> > > from the grib1 ecmwf data files. If using wgrib, we can see 3
> analysis
> >> > > fields, but don't know
> >> > > which is low-res control. After convert to grib2, wgrib2 can
show
> >> which
> >> > > record is low-res control.
> >> > > Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
> >> > >
> >> > > Binbin
> >> > >
> >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > >> Binbin,
> >> > >>
> >> > >> Sounds like there's a lot of moving parts there. How would
you like
> >> to
> >> > >> proceed?
> >> > >>
> >> > >> Should we check to see if MET can read those ECMWF GRIB1
files
> >> directly
> >> > >> without needing to reformat?
> >> > >> Or do you just want to create a custom GRIB table for the
result of
> >> your
> >> > >> pre-processing and conversion steps?
> >> > >>
> >> > >> John
> >> > >>
> >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT <
> >> > >> met_help at ucar.edu> wrote:
> >> > >>
> >> > >> >
> >> > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >> > >> >
> >> > >> > John,
> >> > >> >
> >> > >> > Our ECMWF grib2 ensemble data files are converted from
its
> grib1
> >> > >> format
> >> > >> > data.
> >> > >> > Before conversion, the parameter names are there, but
after
> >> > conversion,
> >> > >> the
> >> > >> > names
> >> > >> > are gone. It is because of unknown tables. The grib1 data
is
> still
> >> > >> > original. The original
> >> > >> > grib1 data files are very strange data format we can not
read. So
> >> we
> >> > >> have
> >> > >> > to use
> >> > >> > a Frotran code to re-arrange the fields and save them as
> NCEP-like
> >> > grib1
> >> > >> > data files.
> >> > >> > So after several processes, some important info like
tables could
> >> be
> >> > >> lost.
> >> > >> >
> >> > >> > I'll test your first suggestion since the field record
order
> is
> >> > >> fixed.
> >> > >> > Currently we still
> >> > >> > obtained grib1 data from ECWMF, eventually, we will get
grib2
> data
> >> > >> > directly. So before
> >> > >> > we can get original grib2 files from ECMWF, we just use
this
> >> temporal
> >> > >> > solution.
> >> > >> > '
> >> > >> > Thanks!
> >> > >> >
> >> > >> >
> >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT <
> >> > >> > met_help at ucar.edu>
> >> > >> > wrote:
> >> > >> >
> >> > >> > > Binbin,
> >> > >> > >
> >> > >> > > OK, working on Venus.
> >> > >> > >
> >> > >> > > You have a typo in the command you sent.
> >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> > >> > >
> >> > >>
> >>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > >> > -v
> >> > >> > > 3
> >> > >> > >
> >> > >> > > You have "RIB2_parm_cat" that should be
"GRIB2_parm_cat". After
> >> > >> > correcting
> >> > >> > > that I did still get an error but it's a very useful
one!
> >> > >> > >
> >> > >> > > module use
> >> > >> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> >> > >> > > module load met/9.1
> >> > >> > > module load grib_util/1.1.1
> >> > >> > > cd
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> >> > >> > 'GRIB2_disc=255;
> >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
level="P500";'
> >> > >> > >
> >> > >> > >
> >> > >> > >
> >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
parameter
> found
> >> > with
> >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3) GRIB2_parm
(5).
> Use
> >> the
> >> > >> > > MET_GRIB_TABLES environment variable to define custom
GRIB
> >> > >> tables.ERROR
> >> > >> > :
> >> > >> > > *
> >> > >> > >
> >> > >> > > So this table is not yet defined in MET. I checked their
> website:
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> >> > >> > >
> >> > >> > > But I don't see an entry defined for GRIB2_parm=5;
> >> > >> > >
> >> > >> > > 3 things...
> >> > >> > >
> >> > >> > > (1) As a crude temporary solution, you could refer to it
by
> it's
> >> > >> record
> >> > >> > > number. Let's assume you know that record 1 has
temperature
> data:
> >> > >> > >
> >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 ~/plot.ps
> >> > 'name="TMP";
> >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> >> > >> > >
> >> > >> > > Here I'm reading record 1, naming it TMP and the using
> >> > set_attr_level
> >> > >> > write
> >> > >> > > the output level as P500 instead of R1.
> >> > >> > >
> >> > >> > > (2) You could create a custom GRIB table yourself
locally, and
> >> then
> >> > >> set
> >> > >> > the
> >> > >> > > MET_GRIB_TABLES environment variable to point MET to it.
> >> > >> > >
> >> > >> > > (3) The best solution is contacting ECMWF and requesting
they
> >> send
> >> > >> over
> >> > >> > the
> >> > >> > > GRIB2 table we should be using to decode this data.
> >> > >> > >
> >> > >> > > Thanks,
> >> > >> > > John
> >> > >> > >
> >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <
> >> johnhg at ucar.edu>
> >> > >> > wrote:
> >> > >> > >
> >> > >> > > > We might already have them. I'm not sure.
> >> > >> > > >
> >> > >> > > > But there's an additional required piece of the
puzzle.
> Binbin
> >> > runs
> >> > >> > > > "wgrib2" to figure out what "name" he should use in
the MET
> >> config
> >> > >> > files.
> >> > >> > > > If wgrib2 doesn't tell him the right "name", the
problem will
> >> > >> remain.
> >> > >> > > > wgrib2 can be configured to define additional tables
as well,
> >> but
> >> > >> for
> >> > >> > > this
> >> > >> > > > to be easy for Binbin, that'd mean updating the script
that's
> >> run
> >> > >> for
> >> > >> > > > "module load grib_utils" on WCOSS. And I'm guessing
that's
> not
> >> > very
> >> > >> > easy.
> >> > >> > > >
> >> > >> > > > Yes, I do agree that the best solution is getting the
ECMWF
> >> GRIB
> >> > >> tables
> >> > >> > > > and adding them for both wgrib2 and MET.
> >> > >> > > >
> >> > >> > > > John
> >> > >> > > >
> >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen
<jensen at ucar.edu>
> >> > wrote:
> >> > >> > > >
> >> > >> > > >> John,
> >> > >> > > >>
> >> > >> > > >> It seems to me that it would be worthwhile to just
ask ECMWF
> >> for
> >> > >> their
> >> > >> > > >> Grib2 Table for their Ensemble files. And other
operational
> >> > >> centers
> >> > >> > for
> >> > >> > > >> that matter. Do you agree this would be a good
solution?
> >> > >> > > >>
> >> > >> > > >> Thanks, Tara
> >> > >> > > >>
> >> > >> > > >> ---------- Forwarded message ---------
> >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <met_help at ucar.edu>
> >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent question
for
> >> > reading
> >> > >> > ECMWF
> >> > >> > > >> ensmeble data
> >> > >> > > >> To:
> >> > >> > > >>
> >> > >> > > >>
> >> > >> > > >>
> >> > >> > > >> <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >> >
> >> > >> > > >>
> >> > >> > > >> John,
> >> > >> > > >>
> >> > >> > > >> I tested using following command:
> >> > >> > > >>
> >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> > >> > > >>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> -v3
> >> > >> > > >>
> >> > >> > > >> ERROR :
> >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
> >> GRIB2_disc,
> >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in
field
> >> > >> information
> >> > >> > > >>
> >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> >> > >> > > >>
> >> > >> >
> >> >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > >> > > >> -v
> >> > >> > > >> 3
> >> > >> > > >>
> >> > >> > > >> Still got error
> >> > >> > > >> ERROR :
> >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
> >> GRIB2_disc,
> >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in
field
> >> > >> information
> >> > >> > > >>
> >> > >> > > >> Looks like discipline=255 is not recognized
> >> > >> > > >>
> >> > >> > > >> Binbin
> >> > >> > > >>
> >> > >> > > >>
> >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway
via RT <
> >> > >> > > >> met_help at ucar.edu>
> >> > >> > > >> wrote:
> >> > >> > > >>
> >> > >> > > >> > Binbin,
> >> > >> > > >> > You can do something similar with MET. Please go to
this
> >> link:
> >> > >> > > >> >
> >> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> >> > >> > > >> >
> >> > >> > > >> > And then search for "GRIB2_mtab", and you'll find
the
> >> content
> >> > >> > > >> > copied/pasted below. You can specify these INSTEAD
OF
> >> > specifying
> >> > >> a
> >> > >> > > >> > requested name.
> >> > >> > > >> >
> >> > >> > > >> > // - The "GRIB2_mtab" is an integer
specifying the
> >> master
> >> > >> > table
> >> > >> > > >> > number.
> >> > >> > > >> > // - The "GRIB2_ltab" is an integer
specifying the
> >> local
> >> > >> table
> >> > >> > > >> > number.
> >> > >> > > >> > // - The "GRIB2_disc" is an integer
specifying the
> >> GRIB2
> >> > >> > > >> discipline
> >> > >> > > >> > code.
> >> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
specifying
> the
> >> > >> > parameter
> >> > >> > > >> > category
> >> > >> > > >> > // code.
> >> > >> > > >> > // - The "GRIB2_parm" is an integer
specifying the
> >> > >> parameter
> >> > >> > > code.
> >> > >> > > >> > // - The "GRIB2_pdt" is an integer specifying
the
> >> product
> >> > >> > > >> definition
> >> > >> > > >> > // template (Table 4.0).
> >> > >> > > >> > // - The "GRIB2_process" is an integer
specifying
> the
> >> > >> > generating
> >> > >> > > >> > process
> >> > >> > > >> > // (Table 4.3).
> >> > >> > > >> > // - The "GRIB2_cntr" is an integer
specifying the
> >> > >> originating
> >> > >> > > >> > center.
> >> > >> > > >> > // - The "GRIB2_ens_type" is an integer
specifying
> the
> >> > >> > ensemble
> >> > >> > > >> type
> >> > >> > > >> > // (Table 4.6).
> >> > >> > > >> > // - The "GRIB2_der_type" is an integer
specifying
> the
> >> > >> derived
> >> > >> > > >> > product
> >> > >> > > >> > // type (Table 4.7).
> >> > >> > > >> > // - The "GRIB2_stat_type" is an integer
specifying
> >> the
> >> > >> > > >> statistical
> >> > >> > > >> > // processing type (Table 4.10).
> >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
> "GRIB2_ipdtmpl_val"
> >> > >> entries
> >> > >> > > are
> >> > >> > > >> > arrays
> >> > >> > > >> > // of integers which specify the product
> description
> >> > >> > template
> >> > >> > > >> > values to
> >> > >> > > >> > // be used. The indices are 0-based. For
example,
> >> use
> >> > the
> >> > >> > > >> > following to
> >> > >> > > >> > // request a GRIB2 record whose 9-th and
27-th
> >> product
> >> > >> > > >> description
> >> > >> > > >> > // template values are 1 and 2,
respectively:
> >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> >> > GRIB2_ipdtmpl_val=[1,
> >> > >> 2];
> >> > >> > > >> >
> >> > >> > > >> > While I don't have any ECMWF data on hand for
testing, I
> do
> >> > have
> >> > >> a
> >> > >> > > file
> >> > >> > > >> > from the Air Force from which wgrib2 prints a
similar
> >> result:
> >> > >> > > >> >
> >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
center=57
> >> > >> local_table=1
> >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> >> > >> > > >> >
> >> > >> > > >> > So I used that for testing. I ran the following
> >> plot_data_plane
> >> > >> > > command
> >> > >> > > >> > WITHOUT specifying a requested name:
> >> > >> > > >> >
> >> > >> > > >> >
> >> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2
plot.ps
> >> > >> > 'GRIB2_disc=0;
> >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
> >> GRIB2_parm=192;
> >> > >> > > >> level="Z0";'
> >> > >> > > >> > -v 3
> >> > >> > > >> >
> >> > >> > > >> > And then I looked closely at the log message to
confirm
> >> that it
> >> > >> used
> >> > >> > > >> > record 549, as expected:
> >> > >> > > >> >
> >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found
exact
> match
> >> > for
> >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2
file
> >> > >> > > >> >
> >> > >> > > >>
> >> > >> > >
> >> > >> >
> >> > >>
> >> >
> >>
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> >> > >> > > >> >
> >> > >> > > >> > So MET used these GRIB2 numbers I defined to do a
table
> >> lookup
> >> > >> and
> >> > >> > > find
> >> > >> > > >> a
> >> > >> > > >> > match in the MET GRIB tables:
> >> > >> > > >> >
> >> > >>
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> >> > >> > > >> >
> >> > >> > > >> > And it determined that that field should be called
> "CTSTM".
> >> > >> > Hopefully,
> >> > >> > > >> the
> >> > >> > > >> > ECMWF GRIB2 tables are already defined there in
MET. If
> >> not, we
> >> > >> can
> >> > >> > > >> always
> >> > >> > > >> > add them.
> >> > >> > > >> >
> >> > >> > > >> > Please try running plot_data_plane at verbosity
level 3 on
> >> one
> >> > of
> >> > >> > > these
> >> > >> > > >> > files first. And make sure it runs without error
and the
> log
> >> > >> message
> >> > >> > > >> prints
> >> > >> > > >> > out the "name" it found for that data. If so, you
can just
> >> skip
> >> > >> > using
> >> > >> > > >> all
> >> > >> > > >> > those integers and revert back to using the name
MET
> knows.
> >> If
> >> > >> not,
> >> > >> > > >> then we
> >> > >> > > >> > may need to define a custom GRIB table for this
ECMWF
> GRIB2
> >> > data.
> >> > >> > > >> >
> >> > >> > > >> > Thanks,
> >> > >> > > >> > John
> >> > >> > > >> >
> >> > >> > > >>
> >> > >> > > >>
> >> > >> > > >> --
> >> > >> > > >>
> >> > >> > > >> Binbin Zhou
> >> > >> > > >>
> >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> >> > >> > > >>
> >> > >> > > >> 5830 University Research Ct.
> >> > >> > > >>
> >> > >> > > >> College Park, MD 20740
> >> > >> > > >>
> >> > >> > > >> Binbin.Zhou at noaa.gov
> >> > >> > > >>
> >> > >> > > >> 301-683-3683
> >> > >> > > >>
> >> > >> > > >>
> >> > >> > > >>
> >> > >> > > >> --
> >> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> > >> > > >> Tara Jensen
> >> > >> > > >> Project Manager II
> >> > >> > > >> NCAR RAL and DTC
> >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> >> > >> > > >>
> >> > >> > > >
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >> > --
> >> > >> >
> >> > >> > Binbin Zhou
> >> > >> >
> >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> >> > >> >
> >> > >> > 5830 University Research Ct.
> >> > >> >
> >> > >> > College Park, MD 20740
> >> > >> >
> >> > >> > Binbin.Zhou at noaa.gov
> >> > >> >
> >> > >> > 301-683-3683
> >> > >> >
> >> > >> >
> >> > >>
> >> > >>
> >> > >
> >> > > --
> >> > >
> >> > > Binbin Zhou
> >> > >
> >> > > IMSG at NOAA/NWS/NCEP/EMC
> >> > >
> >> > > 5830 University Research Ct.
> >> > >
> >> > > College Park, MD 20740
> >> > >
> >> > > Binbin.Zhou at noaa.gov
> >> > >
> >> > > 301-683-3683
> >> > >
> >> >
> >> >
> >> > --
> >> >
> >> > Binbin Zhou
> >> >
> >> > IMSG at NOAA/NWS/NCEP/EMC
> >> >
> >> > 5830 University Research Ct.
> >> >
> >> > College Park, MD 20740
> >> >
> >> > Binbin.Zhou at noaa.gov
> >> >
> >> > 301-683-3683
> >> >
> >> >
> >>
> >>
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Wed Dec 09 14:28:52 2020
John,
Great! Let me test it at first before make change in MET9.1
Another question:
Since climatology data is still in NCEP's grib2 format, In the
ensemble_stat
config file, climo entry settings should be different from those of
fcst, obs entries?
How to set climo (name, level, etc.)?
Binbin
On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> Success! When I recompile with that one line commented out,
plot_data_plane
> is able to filter down all those records to the single one that we
want.
> Note in the example below that I used the "lead_time" and "GRIB_ens"
> options to filter the input records down.
>
> plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
level="P500";
> lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> DEBUG 1: Opening data file: ensposte.t00z.z500hr
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact match
for
> VarInfo "HGT/P500" in GRIB record 26 of GRIB file
"ensposte.t00z.z500hr".*
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "HGT/P500" in GRIB file "ensposte.t00z.z500hr".
> DEBUG 1: Creating postscript file: plot.ps
>
> Also note that this selects out record number 26.
>
> Shall I make this change as a "bugfix" for met-9.1...which could be
> included in a met-9.1.2 bugfix release? Or shall I just include it
in
> development for the next release (planned for March 2021)?
>
> Thanks,
> John
>
> On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > If this removing impacts other applications, or takes some time
> > to be effective, then won't try. I will use another approach,
such as
> > borrow GEFS's grib2 data structure as tank to save EC ensemble
> > data.
> >
> > Binbin
> >
> >
> > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
> > binbin.zhou at noaa.gov> wrote:
> >
> > > John,
> > >
> > > Yes, please try to use this approach.
> > >
> > > Thanks!
> > >
> > > Binbin
> > >
> > > Binbin
> > >
> > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Binbin,
> > >>
> > >> I used the "-PDS10" option for wgrib to dump those records:
> > >>
> > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > >>
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > >> | egrep "^11:|^26:|^43:"
> > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0
1 0 0
> 0
> > 21
> > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0
1 0 0
> 0
> > 21
> > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0 0
1 0 0
> 0
> > 21
> > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > >>
> > >> Looking VERY closely, there is a difference in the 42-nd column
which
> is
> > >> the 4-th from the last... records 11, 26, and 43 have values of
3, 1,
> > and
> > >> 2
> > >> respectively.
> > >>
> > >> On this page (
> > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > >> ),
> > >> byte 42 = 1 means "unperturbed control forecast".
> > >>
> > >> So we *should* be able to use the "GRIB_ens" config option to
> > >> differentiate
> > >> between those records. BUT when I try, it doesn't work. And
that's
> > because
> > >> of this line:
> > >>
> > >>
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > >>
> > >> We only apply those filtering options to GRIB files created by
NCEP.
> If
> > >> you'd like, I could try removing that condition to see if we
can pick
> > out
> > >> the single record of interest.
> > >>
> > >> John
> > >>
> > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > >> >
> > >> > John,
> > >> >
> > >> > Look at my
> > >> > directory
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > >> > The grib1 file is ensposte.t00z.z500hr, this is only for 500
HGT.
> > >> > wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
> > >> > I get 3 records:
> > >> >
> > >> >
> > >> >
> > >>
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >> > mb:anl:type=analysis:NAve=0
> > >> >
> > >> >
> > >>
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >> > mb:anl:type=analysis:NAve=0
> > >> >
> > >> >
> > >>
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >> > mb:anl:type=analysis:NAve=0
> > >> >
> > >> > From its grib2 file, we know only record#26 is analysis low-
res
> > >> control.
> > >> > If you have some way to determine which record is analysis
low-res
> > >> control
> > >> > from the grib1 file,
> > >> > then we can use grib1 directly (don't consider the size
issue).
> > >> >
> > >> > Binbin
> > >> >
> > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate <
> > >> > binbin.zhou at noaa.gov> wrote:
> > >> >
> > >> > > John,
> > >> > >
> > >> > > Grib1 data is too large, another issue is the low-res
control
> > >> analysis
> > >> > > data can not be retrieved
> > >> > > from the grib1 ecmwf data files. If using wgrib, we can see
3
> > analysis
> > >> > > fields, but don't know
> > >> > > which is low-res control. After convert to grib2, wgrib2
can show
> > >> which
> > >> > > record is low-res control.
> > >> > > Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
> > >> > >
> > >> > > Binbin
> > >> > >
> > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT <
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > >> Binbin,
> > >> > >>
> > >> > >> Sounds like there's a lot of moving parts there. How would
you
> like
> > >> to
> > >> > >> proceed?
> > >> > >>
> > >> > >> Should we check to see if MET can read those ECMWF GRIB1
files
> > >> directly
> > >> > >> without needing to reformat?
> > >> > >> Or do you just want to create a custom GRIB table for the
result
> of
> > >> your
> > >> > >> pre-processing and conversion steps?
> > >> > >>
> > >> > >> John
> > >> > >>
> > >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via RT
<
> > >> > >> met_help at ucar.edu> wrote:
> > >> > >>
> > >> > >> >
> > >> > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > >> > >> >
> > >> > >> > John,
> > >> > >> >
> > >> > >> > Our ECMWF grib2 ensemble data files are converted from
its
> > grib1
> > >> > >> format
> > >> > >> > data.
> > >> > >> > Before conversion, the parameter names are there, but
after
> > >> > conversion,
> > >> > >> the
> > >> > >> > names
> > >> > >> > are gone. It is because of unknown tables. The grib1
data is
> > still
> > >> > >> > original. The original
> > >> > >> > grib1 data files are very strange data format we can not
read.
> So
> > >> we
> > >> > >> have
> > >> > >> > to use
> > >> > >> > a Frotran code to re-arrange the fields and save them as
> > NCEP-like
> > >> > grib1
> > >> > >> > data files.
> > >> > >> > So after several processes, some important info like
tables
> could
> > >> be
> > >> > >> lost.
> > >> > >> >
> > >> > >> > I'll test your first suggestion since the field
record order
> > is
> > >> > >> fixed.
> > >> > >> > Currently we still
> > >> > >> > obtained grib1 data from ECWMF, eventually, we will get
grib2
> > data
> > >> > >> > directly. So before
> > >> > >> > we can get original grib2 files from ECMWF, we just use
this
> > >> temporal
> > >> > >> > solution.
> > >> > >> > '
> > >> > >> > Thanks!
> > >> > >> >
> > >> > >> >
> > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via RT
<
> > >> > >> > met_help at ucar.edu>
> > >> > >> > wrote:
> > >> > >> >
> > >> > >> > > Binbin,
> > >> > >> > >
> > >> > >> > > OK, working on Venus.
> > >> > >> > >
> > >> > >> > > You have a typo in the command you sent.
> > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >> > >> > >
> > >> > >>
> > >>
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > >> > -v
> > >> > >> > > 3
> > >> > >> > >
> > >> > >> > > You have "RIB2_parm_cat" that should be
"GRIB2_parm_cat".
> After
> > >> > >> > correcting
> > >> > >> > > that I did still get an error but it's a very useful
one!
> > >> > >> > >
> > >> > >> > > module use
> > >> > >>
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > >> > >> > > module load met/9.1
> > >> > >> > > module load grib_util/1.1.1
> > >> > >> > > cd
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
> > >> > >> > 'GRIB2_disc=255;
> > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
level="P500";'
> > >> > >> > >
> > >> > >> > >
> > >> > >> > >
> > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
parameter
> > found
> > >> > with
> > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
GRIB2_parm (5).
> > Use
> > >> the
> > >> > >> > > MET_GRIB_TABLES environment variable to define custom
GRIB
> > >> > >> tables.ERROR
> > >> > >> > :
> > >> > >> > > *
> > >> > >> > >
> > >> > >> > > So this table is not yet defined in MET. I checked
their
> > website:
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > >> > >> > >
> > >> > >> > > But I don't see an entry defined for GRIB2_parm=5;
> > >> > >> > >
> > >> > >> > > 3 things...
> > >> > >> > >
> > >> > >> > > (1) As a crude temporary solution, you could refer to
it by
> > it's
> > >> > >> record
> > >> > >> > > number. Let's assume you know that record 1 has
temperature
> > data:
> > >> > >> > >
> > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
> > >> > 'name="TMP";
> > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > >> > >> > >
> > >> > >> > > Here I'm reading record 1, naming it TMP and the using
> > >> > set_attr_level
> > >> > >> > write
> > >> > >> > > the output level as P500 instead of R1.
> > >> > >> > >
> > >> > >> > > (2) You could create a custom GRIB table yourself
locally,
> and
> > >> then
> > >> > >> set
> > >> > >> > the
> > >> > >> > > MET_GRIB_TABLES environment variable to point MET to
it.
> > >> > >> > >
> > >> > >> > > (3) The best solution is contacting ECMWF and
requesting they
> > >> send
> > >> > >> over
> > >> > >> > the
> > >> > >> > > GRIB2 table we should be using to decode this data.
> > >> > >> > >
> > >> > >> > > Thanks,
> > >> > >> > > John
> > >> > >> > >
> > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <
> > >> johnhg at ucar.edu>
> > >> > >> > wrote:
> > >> > >> > >
> > >> > >> > > > We might already have them. I'm not sure.
> > >> > >> > > >
> > >> > >> > > > But there's an additional required piece of the
puzzle.
> > Binbin
> > >> > runs
> > >> > >> > > > "wgrib2" to figure out what "name" he should use in
the MET
> > >> config
> > >> > >> > files.
> > >> > >> > > > If wgrib2 doesn't tell him the right "name", the
problem
> will
> > >> > >> remain.
> > >> > >> > > > wgrib2 can be configured to define additional tables
as
> well,
> > >> but
> > >> > >> for
> > >> > >> > > this
> > >> > >> > > > to be easy for Binbin, that'd mean updating the
script
> that's
> > >> run
> > >> > >> for
> > >> > >> > > > "module load grib_utils" on WCOSS. And I'm guessing
that's
> > not
> > >> > very
> > >> > >> > easy.
> > >> > >> > > >
> > >> > >> > > > Yes, I do agree that the best solution is getting
the ECMWF
> > >> GRIB
> > >> > >> tables
> > >> > >> > > > and adding them for both wgrib2 and MET.
> > >> > >> > > >
> > >> > >> > > > John
> > >> > >> > > >
> > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
> jensen at ucar.edu>
> > >> > wrote:
> > >> > >> > > >
> > >> > >> > > >> John,
> > >> > >> > > >>
> > >> > >> > > >> It seems to me that it would be worthwhile to just
ask
> ECMWF
> > >> for
> > >> > >> their
> > >> > >> > > >> Grib2 Table for their Ensemble files. And other
> operational
> > >> > >> centers
> > >> > >> > for
> > >> > >> > > >> that matter. Do you agree this would be a good
solution?
> > >> > >> > > >>
> > >> > >> > > >> Thanks, Tara
> > >> > >> > > >>
> > >> > >> > > >> ---------- Forwarded message ---------
> > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT
<met_help at ucar.edu>
> > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent
question for
> > >> > reading
> > >> > >> > ECMWF
> > >> > >> > > >> ensmeble data
> > >> > >> > > >> To:
> > >> > >> > > >>
> > >> > >> > > >>
> > >> > >> > > >>
> > >> > >> > > >> <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >> >
> > >> > >> > > >>
> > >> > >> > > >> John,
> > >> > >> > > >>
> > >> > >> > > >> I tested using following command:
> > >> > >> > > >>
> > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > >> > >> > > >>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> -v3
> > >> > >> > > >>
> > >> > >> > > >> ERROR :
> > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
> > >> GRIB2_disc,
> > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in
field
> > >> > >> information
> > >> > >> > > >>
> > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > >> > >> > > >>
> > >> > >> >
> > >> >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > >> > > >> -v
> > >> > >> > > >> 3
> > >> > >> > > >>
> > >> > >> > > >> Still got error
> > >> > >> > > >> ERROR :
> > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name or
> > >> GRIB2_disc,
> > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified in
field
> > >> > >> information
> > >> > >> > > >>
> > >> > >> > > >> Looks like discipline=255 is not recognized
> > >> > >> > > >>
> > >> > >> > > >> Binbin
> > >> > >> > > >>
> > >> > >> > > >>
> > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley Gotway
via RT
> <
> > >> > >> > > >> met_help at ucar.edu>
> > >> > >> > > >> wrote:
> > >> > >> > > >>
> > >> > >> > > >> > Binbin,
> > >> > >> > > >> > You can do something similar with MET. Please go
to this
> > >> link:
> > >> > >> > > >> >
> > >> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > >> > >> > > >> >
> > >> > >> > > >> > And then search for "GRIB2_mtab", and you'll find
the
> > >> content
> > >> > >> > > >> > copied/pasted below. You can specify these
INSTEAD OF
> > >> > specifying
> > >> > >> a
> > >> > >> > > >> > requested name.
> > >> > >> > > >> >
> > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
specifying the
> > >> master
> > >> > >> > table
> > >> > >> > > >> > number.
> > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
specifying the
> > >> local
> > >> > >> table
> > >> > >> > > >> > number.
> > >> > >> > > >> > // - The "GRIB2_disc" is an integer
specifying the
> > >> GRIB2
> > >> > >> > > >> discipline
> > >> > >> > > >> > code.
> > >> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
specifying
> > the
> > >> > >> > parameter
> > >> > >> > > >> > category
> > >> > >> > > >> > // code.
> > >> > >> > > >> > // - The "GRIB2_parm" is an integer
specifying the
> > >> > >> parameter
> > >> > >> > > code.
> > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
specifying the
> > >> product
> > >> > >> > > >> definition
> > >> > >> > > >> > // template (Table 4.0).
> > >> > >> > > >> > // - The "GRIB2_process" is an integer
specifying
> > the
> > >> > >> > generating
> > >> > >> > > >> > process
> > >> > >> > > >> > // (Table 4.3).
> > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
specifying the
> > >> > >> originating
> > >> > >> > > >> > center.
> > >> > >> > > >> > // - The "GRIB2_ens_type" is an integer
specifying
> > the
> > >> > >> > ensemble
> > >> > >> > > >> type
> > >> > >> > > >> > // (Table 4.6).
> > >> > >> > > >> > // - The "GRIB2_der_type" is an integer
specifying
> > the
> > >> > >> derived
> > >> > >> > > >> > product
> > >> > >> > > >> > // type (Table 4.7).
> > >> > >> > > >> > // - The "GRIB2_stat_type" is an integer
> specifying
> > >> the
> > >> > >> > > >> statistical
> > >> > >> > > >> > // processing type (Table 4.10).
> > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
> > "GRIB2_ipdtmpl_val"
> > >> > >> entries
> > >> > >> > > are
> > >> > >> > > >> > arrays
> > >> > >> > > >> > // of integers which specify the product
> > description
> > >> > >> > template
> > >> > >> > > >> > values to
> > >> > >> > > >> > // be used. The indices are 0-based. For
> example,
> > >> use
> > >> > the
> > >> > >> > > >> > following to
> > >> > >> > > >> > // request a GRIB2 record whose 9-th and
27-th
> > >> product
> > >> > >> > > >> description
> > >> > >> > > >> > // template values are 1 and 2,
respectively:
> > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> > >> > GRIB2_ipdtmpl_val=[1,
> > >> > >> 2];
> > >> > >> > > >> >
> > >> > >> > > >> > While I don't have any ECMWF data on hand for
testing, I
> > do
> > >> > have
> > >> > >> a
> > >> > >> > > file
> > >> > >> > > >> > from the Air Force from which wgrib2 prints a
similar
> > >> result:
> > >> > >> > > >> >
> > >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
center=57
> > >> > >> local_table=1
> > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > >> > >> > > >> >
> > >> > >> > > >> > So I used that for testing. I ran the following
> > >> plot_data_plane
> > >> > >> > > command
> > >> > >> > > >> > WITHOUT specifying a requested name:
> > >> > >> > > >> >
> > >> > >> > > >> >
> > >> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2
plot.ps
> > >> > >> > 'GRIB2_disc=0;
> > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
> > >> GRIB2_parm=192;
> > >> > >> > > >> level="Z0";'
> > >> > >> > > >> > -v 3
> > >> > >> > > >> >
> > >> > >> > > >> > And then I looked closely at the log message to
confirm
> > >> that it
> > >> > >> used
> > >> > >> > > >> > record 549, as expected:
> > >> > >> > > >> >
> > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found
exact
> > match
> > >> > for
> > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2
file
> > >> > >> > > >> >
> > >> > >> > > >>
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> >
> > >>
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > >> > >> > > >> >
> > >> > >> > > >> > So MET used these GRIB2 numbers I defined to do a
table
> > >> lookup
> > >> > >> and
> > >> > >> > > find
> > >> > >> > > >> a
> > >> > >> > > >> > match in the MET GRIB tables:
> > >> > >> > > >> >
> > >> > >>
> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > >> > >> > > >> >
> > >> > >> > > >> > And it determined that that field should be
called
> > "CTSTM".
> > >> > >> > Hopefully,
> > >> > >> > > >> the
> > >> > >> > > >> > ECMWF GRIB2 tables are already defined there in
MET. If
> > >> not, we
> > >> > >> can
> > >> > >> > > >> always
> > >> > >> > > >> > add them.
> > >> > >> > > >> >
> > >> > >> > > >> > Please try running plot_data_plane at verbosity
level 3
> on
> > >> one
> > >> > of
> > >> > >> > > these
> > >> > >> > > >> > files first. And make sure it runs without error
and the
> > log
> > >> > >> message
> > >> > >> > > >> prints
> > >> > >> > > >> > out the "name" it found for that data. If so, you
can
> just
> > >> skip
> > >> > >> > using
> > >> > >> > > >> all
> > >> > >> > > >> > those integers and revert back to using the name
MET
> > knows.
> > >> If
> > >> > >> not,
> > >> > >> > > >> then we
> > >> > >> > > >> > may need to define a custom GRIB table for this
ECMWF
> > GRIB2
> > >> > data.
> > >> > >> > > >> >
> > >> > >> > > >> > Thanks,
> > >> > >> > > >> > John
> > >> > >> > > >> >
> > >> > >> > > >>
> > >> > >> > > >>
> > >> > >> > > >> --
> > >> > >> > > >>
> > >> > >> > > >> Binbin Zhou
> > >> > >> > > >>
> > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > >> > >> > > >>
> > >> > >> > > >> 5830 University Research Ct.
> > >> > >> > > >>
> > >> > >> > > >> College Park, MD 20740
> > >> > >> > > >>
> > >> > >> > > >> Binbin.Zhou at noaa.gov
> > >> > >> > > >>
> > >> > >> > > >> 301-683-3683
> > >> > >> > > >>
> > >> > >> > > >>
> > >> > >> > > >>
> > >> > >> > > >> --
> > >> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > >> > > >> Tara Jensen
> > >> > >> > > >> Project Manager II
> > >> > >> > > >> NCAR RAL and DTC
> > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > >> > >> > > >>
> > >> > >> > > >
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >> > --
> > >> > >> >
> > >> > >> > Binbin Zhou
> > >> > >> >
> > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> > >> >
> > >> > >> > 5830 University Research Ct.
> > >> > >> >
> > >> > >> > College Park, MD 20740
> > >> > >> >
> > >> > >> > Binbin.Zhou at noaa.gov
> > >> > >> >
> > >> > >> > 301-683-3683
> > >> > >> >
> > >> > >> >
> > >> > >>
> > >> > >>
> > >> > >
> > >> > > --
> > >> > >
> > >> > > Binbin Zhou
> > >> > >
> > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > >> > >
> > >> > > 5830 University Research Ct.
> > >> > >
> > >> > > College Park, MD 20740
> > >> > >
> > >> > > Binbin.Zhou at noaa.gov
> > >> > >
> > >> > > 301-683-3683
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> >
> > >> > Binbin Zhou
> > >> >
> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> >
> > >> > 5830 University Research Ct.
> > >> >
> > >> > College Park, MD 20740
> > >> >
> > >> > Binbin.Zhou at noaa.gov
> > >> >
> > >> > 301-683-3683
> > >> >
> > >> >
> > >>
> > >>
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Wed Dec 09 16:53:13 2020
Binbin,
In order for you to test this change out, we'd need to recompile a
version
of the code on WCOSS. I wrote this up as a GitHub issue:
https://github.com/dtcenter/MET/issues/1604
And I'll create a branch from main_v9.1 named "bugfix_1604_ens_config"
and
commit the change there. Then we can checkout that code on WCOSS,
compile
it, and you can test it.
Yes, you're right, if the fcst fields differ from the climo fields,
you'd
need to specify them separately. That's pretty easy in the Ensemble-
Stat
config file... but are you asking how to control that via METplus?
John
On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> Great! Let me test it at first before make change in MET9.1
> Another question:
> Since climatology data is still in NCEP's grib2 format, In the
> ensemble_stat
> config file, climo entry settings should be different from those
of
> fcst, obs entries?
> How to set climo (name, level, etc.)?
>
> Binbin
>
>
>
> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binbin,
> >
> > Success! When I recompile with that one line commented out,
> plot_data_plane
> > is able to filter down all those records to the single one that we
want.
> > Note in the example below that I used the "lead_time" and
"GRIB_ens"
> > options to filter the input records down.
> >
> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
level="P500";
> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact
match for
> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> "ensposte.t00z.z500hr".*
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "HGT/P500" in GRIB file "ensposte.t00z.z500hr".
> > DEBUG 1: Creating postscript file: plot.ps
> >
> > Also note that this selects out record number 26.
> >
> > Shall I make this change as a "bugfix" for met-9.1...which could
be
> > included in a met-9.1.2 bugfix release? Or shall I just include it
in
> > development for the next release (planned for March 2021)?
> >
> > Thanks,
> > John
> >
> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >
> > > John,
> > >
> > > If this removing impacts other applications, or takes some
time
> > > to be effective, then won't try. I will use another approach,
such as
> > > borrow GEFS's grib2 data structure as tank to save EC ensemble
> > > data.
> > >
> > > Binbin
> > >
> > >
> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
> > > binbin.zhou at noaa.gov> wrote:
> > >
> > > > John,
> > > >
> > > > Yes, please try to use this approach.
> > > >
> > > > Thanks!
> > > >
> > > > Binbin
> > > >
> > > > Binbin
> > > >
> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Binbin,
> > > >>
> > > >> I used the "-PDS10" option for wgrib to dump those records:
> > > >>
> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > >>
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > >> | egrep "^11:|^26:|^43:"
> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0
0 1 0
> 0
> > 0
> > > 21
> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0
0 1 0
> 0
> > 0
> > > 21
> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1 0
0 1 0
> 0
> > 0
> > > 21
> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > >>
> > > >> Looking VERY closely, there is a difference in the 42-nd
column
> which
> > is
> > > >> the 4-th from the last... records 11, 26, and 43 have values
of 3,
> 1,
> > > and
> > > >> 2
> > > >> respectively.
> > > >>
> > > >> On this page (
> > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > >> ),
> > > >> byte 42 = 1 means "unperturbed control forecast".
> > > >>
> > > >> So we *should* be able to use the "GRIB_ens" config option to
> > > >> differentiate
> > > >> between those records. BUT when I try, it doesn't work. And
that's
> > > because
> > > >> of this line:
> > > >>
> > > >>
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > >>
> > > >> We only apply those filtering options to GRIB files created
by NCEP.
> > If
> > > >> you'd like, I could try removing that condition to see if we
can
> pick
> > > out
> > > >> the single record of interest.
> > > >>
> > > >> John
> > > >>
> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > >> >
> > > >> > John,
> > > >> >
> > > >> > Look at my
> > > >> > directory
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > >> > The grib1 file is ensposte.t00z.z500hr, this is only for
500 HGT.
> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
> > > >> > I get 3 records:
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >> > mb:anl:type=analysis:NAve=0
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >> > mb:anl:type=analysis:NAve=0
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >> > mb:anl:type=analysis:NAve=0
> > > >> >
> > > >> > From its grib2 file, we know only record#26 is analysis
low-res
> > > >> control.
> > > >> > If you have some way to determine which record is analysis
low-res
> > > >> control
> > > >> > from the grib1 file,
> > > >> > then we can use grib1 directly (don't consider the size
issue).
> > > >> >
> > > >> > Binbin
> > > >> >
> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA Affiliate
<
> > > >> > binbin.zhou at noaa.gov> wrote:
> > > >> >
> > > >> > > John,
> > > >> > >
> > > >> > > Grib1 data is too large, another issue is the low-res
control
> > > >> analysis
> > > >> > > data can not be retrieved
> > > >> > > from the grib1 ecmwf data files. If using wgrib, we can
see 3
> > > analysis
> > > >> > > fields, but don't know
> > > >> > > which is low-res control. After convert to grib2, wgrib2
can
> show
> > > >> which
> > > >> > > record is low-res control.
> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set on WCOSS.
> > > >> > >
> > > >> > > Binbin
> > > >> > >
> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT
<
> > > >> > > met_help at ucar.edu> wrote:
> > > >> > >
> > > >> > >> Binbin,
> > > >> > >>
> > > >> > >> Sounds like there's a lot of moving parts there. How
would you
> > like
> > > >> to
> > > >> > >> proceed?
> > > >> > >>
> > > >> > >> Should we check to see if MET can read those ECMWF GRIB1
files
> > > >> directly
> > > >> > >> without needing to reformat?
> > > >> > >> Or do you just want to create a custom GRIB table for
the
> result
> > of
> > > >> your
> > > >> > >> pre-processing and conversion steps?
> > > >> > >>
> > > >> > >> John
> > > >> > >>
> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via
RT <
> > > >> > >> met_help at ucar.edu> wrote:
> > > >> > >>
> > > >> > >> >
> > > >> > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > > >> > >> >
> > > >> > >> > John,
> > > >> > >> >
> > > >> > >> > Our ECMWF grib2 ensemble data files are converted
from its
> > > grib1
> > > >> > >> format
> > > >> > >> > data.
> > > >> > >> > Before conversion, the parameter names are there, but
after
> > > >> > conversion,
> > > >> > >> the
> > > >> > >> > names
> > > >> > >> > are gone. It is because of unknown tables. The grib1
data is
> > > still
> > > >> > >> > original. The original
> > > >> > >> > grib1 data files are very strange data format we can
not
> read.
> > So
> > > >> we
> > > >> > >> have
> > > >> > >> > to use
> > > >> > >> > a Frotran code to re-arrange the fields and save them
as
> > > NCEP-like
> > > >> > grib1
> > > >> > >> > data files.
> > > >> > >> > So after several processes, some important info like
tables
> > could
> > > >> be
> > > >> > >> lost.
> > > >> > >> >
> > > >> > >> > I'll test your first suggestion since the field
record
> order
> > > is
> > > >> > >> fixed.
> > > >> > >> > Currently we still
> > > >> > >> > obtained grib1 data from ECWMF, eventually, we will
get grib2
> > > data
> > > >> > >> > directly. So before
> > > >> > >> > we can get original grib2 files from ECMWF, we just
use this
> > > >> temporal
> > > >> > >> > solution.
> > > >> > >> > '
> > > >> > >> > Thanks!
> > > >> > >> >
> > > >> > >> >
> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via
RT <
> > > >> > >> > met_help at ucar.edu>
> > > >> > >> > wrote:
> > > >> > >> >
> > > >> > >> > > Binbin,
> > > >> > >> > >
> > > >> > >> > > OK, working on Venus.
> > > >> > >> > >
> > > >> > >> > > You have a typo in the command you sent.
> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > > >> > >> > >
> > > >> > >>
> > > >>
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> > >> > -v
> > > >> > >> > > 3
> > > >> > >> > >
> > > >> > >> > > You have "RIB2_parm_cat" that should be
"GRIB2_parm_cat".
> > After
> > > >> > >> > correcting
> > > >> > >> > > that I did still get an error but it's a very useful
one!
> > > >> > >> > >
> > > >> > >> > > module use
> > > >> > >>
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > >> > >> > > module load met/9.1
> > > >> > >> > > module load grib_util/1.1.1
> > > >> > >> > > cd
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
> > > >> > >> > 'GRIB2_disc=255;
> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
> level="P500";'
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
parameter
> > > found
> > > >> > with
> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
GRIB2_parm
> (5).
> > > Use
> > > >> the
> > > >> > >> > > MET_GRIB_TABLES environment variable to define
custom GRIB
> > > >> > >> tables.ERROR
> > > >> > >> > :
> > > >> > >> > > *
> > > >> > >> > >
> > > >> > >> > > So this table is not yet defined in MET. I checked
their
> > > website:
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > >> > >> > >
> > > >> > >> > > But I don't see an entry defined for GRIB2_parm=5;
> > > >> > >> > >
> > > >> > >> > > 3 things...
> > > >> > >> > >
> > > >> > >> > > (1) As a crude temporary solution, you could refer
to it by
> > > it's
> > > >> > >> record
> > > >> > >> > > number. Let's assume you know that record 1 has
temperature
> > > data:
> > > >> > >> > >
> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
> > > >> > 'name="TMP";
> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > > >> > >> > >
> > > >> > >> > > Here I'm reading record 1, naming it TMP and the
using
> > > >> > set_attr_level
> > > >> > >> > write
> > > >> > >> > > the output level as P500 instead of R1.
> > > >> > >> > >
> > > >> > >> > > (2) You could create a custom GRIB table yourself
locally,
> > and
> > > >> then
> > > >> > >> set
> > > >> > >> > the
> > > >> > >> > > MET_GRIB_TABLES environment variable to point MET to
it.
> > > >> > >> > >
> > > >> > >> > > (3) The best solution is contacting ECMWF and
requesting
> they
> > > >> send
> > > >> > >> over
> > > >> > >> > the
> > > >> > >> > > GRIB2 table we should be using to decode this data.
> > > >> > >> > >
> > > >> > >> > > Thanks,
> > > >> > >> > > John
> > > >> > >> > >
> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <
> > > >> johnhg at ucar.edu>
> > > >> > >> > wrote:
> > > >> > >> > >
> > > >> > >> > > > We might already have them. I'm not sure.
> > > >> > >> > > >
> > > >> > >> > > > But there's an additional required piece of the
puzzle.
> > > Binbin
> > > >> > runs
> > > >> > >> > > > "wgrib2" to figure out what "name" he should use
in the
> MET
> > > >> config
> > > >> > >> > files.
> > > >> > >> > > > If wgrib2 doesn't tell him the right "name", the
problem
> > will
> > > >> > >> remain.
> > > >> > >> > > > wgrib2 can be configured to define additional
tables as
> > well,
> > > >> but
> > > >> > >> for
> > > >> > >> > > this
> > > >> > >> > > > to be easy for Binbin, that'd mean updating the
script
> > that's
> > > >> run
> > > >> > >> for
> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
guessing
> that's
> > > not
> > > >> > very
> > > >> > >> > easy.
> > > >> > >> > > >
> > > >> > >> > > > Yes, I do agree that the best solution is getting
the
> ECMWF
> > > >> GRIB
> > > >> > >> tables
> > > >> > >> > > > and adding them for both wgrib2 and MET.
> > > >> > >> > > >
> > > >> > >> > > > John
> > > >> > >> > > >
> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
> > jensen at ucar.edu>
> > > >> > wrote:
> > > >> > >> > > >
> > > >> > >> > > >> John,
> > > >> > >> > > >>
> > > >> > >> > > >> It seems to me that it would be worthwhile to
just ask
> > ECMWF
> > > >> for
> > > >> > >> their
> > > >> > >> > > >> Grib2 Table for their Ensemble files. And other
> > operational
> > > >> > >> centers
> > > >> > >> > for
> > > >> > >> > > >> that matter. Do you agree this would be a good
> solution?
> > > >> > >> > > >>
> > > >> > >> > > >> Thanks, Tara
> > > >> > >> > > >>
> > > >> > >> > > >> ---------- Forwarded message ---------
> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT
<met_help at ucar.edu>
> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent
question
> for
> > > >> > reading
> > > >> > >> > ECMWF
> > > >> > >> > > >> ensmeble data
> > > >> > >> > > >> To:
> > > >> > >> > > >>
> > > >> > >> > > >>
> > > >> > >> > > >>
> > > >> > >> > > >> <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >> >
> > > >> > >> > > >>
> > > >> > >> > > >> John,
> > > >> > >> > > >>
> > > >> > >> > > >> I tested using following command:
> > > >> > >> > > >>
> > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > > >> > >> > > >>
> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> -v3
> > > >> > >> > > >>
> > > >> > >> > > >> ERROR :
> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name
or
> > > >> GRIB2_disc,
> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in field
> > > >> > >> information
> > > >> > >> > > >>
> > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > > >> > >> > > >>
> > > >> > >> >
> > > >> >
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> > >> > > >> -v
> > > >> > >> > > >> 3
> > > >> > >> > > >>
> > > >> > >> > > >> Still got error
> > > >> > >> > > >> ERROR :
> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name
or
> > > >> GRIB2_disc,
> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in field
> > > >> > >> information
> > > >> > >> > > >>
> > > >> > >> > > >> Looks like discipline=255 is not recognized
> > > >> > >> > > >>
> > > >> > >> > > >> Binbin
> > > >> > >> > > >>
> > > >> > >> > > >>
> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley
Gotway via
> RT
> > <
> > > >> > >> > > >> met_help at ucar.edu>
> > > >> > >> > > >> wrote:
> > > >> > >> > > >>
> > > >> > >> > > >> > Binbin,
> > > >> > >> > > >> > You can do something similar with MET. Please
go to
> this
> > > >> link:
> > > >> > >> > > >> >
> > > >>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > >> > >> > > >> >
> > > >> > >> > > >> > And then search for "GRIB2_mtab", and you'll
find the
> > > >> content
> > > >> > >> > > >> > copied/pasted below. You can specify these
INSTEAD OF
> > > >> > specifying
> > > >> > >> a
> > > >> > >> > > >> > requested name.
> > > >> > >> > > >> >
> > > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
specifying
> the
> > > >> master
> > > >> > >> > table
> > > >> > >> > > >> > number.
> > > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
specifying
> the
> > > >> local
> > > >> > >> table
> > > >> > >> > > >> > number.
> > > >> > >> > > >> > // - The "GRIB2_disc" is an integer
specifying
> the
> > > >> GRIB2
> > > >> > >> > > >> discipline
> > > >> > >> > > >> > code.
> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
> specifying
> > > the
> > > >> > >> > parameter
> > > >> > >> > > >> > category
> > > >> > >> > > >> > // code.
> > > >> > >> > > >> > // - The "GRIB2_parm" is an integer
specifying
> the
> > > >> > >> parameter
> > > >> > >> > > code.
> > > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
specifying
> the
> > > >> product
> > > >> > >> > > >> definition
> > > >> > >> > > >> > // template (Table 4.0).
> > > >> > >> > > >> > // - The "GRIB2_process" is an integer
> specifying
> > > the
> > > >> > >> > generating
> > > >> > >> > > >> > process
> > > >> > >> > > >> > // (Table 4.3).
> > > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
specifying
> the
> > > >> > >> originating
> > > >> > >> > > >> > center.
> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an integer
> specifying
> > > the
> > > >> > >> > ensemble
> > > >> > >> > > >> type
> > > >> > >> > > >> > // (Table 4.6).
> > > >> > >> > > >> > // - The "GRIB2_der_type" is an integer
> specifying
> > > the
> > > >> > >> derived
> > > >> > >> > > >> > product
> > > >> > >> > > >> > // type (Table 4.7).
> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an integer
> > specifying
> > > >> the
> > > >> > >> > > >> statistical
> > > >> > >> > > >> > // processing type (Table 4.10).
> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
> > > "GRIB2_ipdtmpl_val"
> > > >> > >> entries
> > > >> > >> > > are
> > > >> > >> > > >> > arrays
> > > >> > >> > > >> > // of integers which specify the
product
> > > description
> > > >> > >> > template
> > > >> > >> > > >> > values to
> > > >> > >> > > >> > // be used. The indices are 0-based.
For
> > example,
> > > >> use
> > > >> > the
> > > >> > >> > > >> > following to
> > > >> > >> > > >> > // request a GRIB2 record whose 9-th
and 27-th
> > > >> product
> > > >> > >> > > >> description
> > > >> > >> > > >> > // template values are 1 and 2,
respectively:
> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> > > >> > GRIB2_ipdtmpl_val=[1,
> > > >> > >> 2];
> > > >> > >> > > >> >
> > > >> > >> > > >> > While I don't have any ECMWF data on hand for
> testing, I
> > > do
> > > >> > have
> > > >> > >> a
> > > >> > >> > > file
> > > >> > >> > > >> > from the Air Force from which wgrib2 prints a
similar
> > > >> result:
> > > >> > >> > > >> >
> > > >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
center=57
> > > >> > >> local_table=1
> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > > >> > >> > > >> >
> > > >> > >> > > >> > So I used that for testing. I ran the following
> > > >> plot_data_plane
> > > >> > >> > > command
> > > >> > >> > > >> > WITHOUT specifying a requested name:
> > > >> > >> > > >> >
> > > >> > >> > > >> >
> > > >> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2
plot.ps
> > > >> > >> > 'GRIB2_disc=0;
> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
> > > >> GRIB2_parm=192;
> > > >> > >> > > >> level="Z0";'
> > > >> > >> > > >> > -v 3
> > > >> > >> > > >> >
> > > >> > >> > > >> > And then I looked closely at the log message to
> confirm
> > > >> that it
> > > >> > >> used
> > > >> > >> > > >> > record 549, as expected:
> > > >> > >> > > >> >
> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() ->
Found exact
> > > match
> > > >> > for
> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of GRIB2
file
> > > >> > >> > > >> >
> > > >> > >> > > >>
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > >> > >> > > >> >
> > > >> > >> > > >> > So MET used these GRIB2 numbers I defined to do
a
> table
> > > >> lookup
> > > >> > >> and
> > > >> > >> > > find
> > > >> > >> > > >> a
> > > >> > >> > > >> > match in the MET GRIB tables:
> > > >> > >> > > >> >
> > > >> > >>
> > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > >> > >> > > >> >
> > > >> > >> > > >> > And it determined that that field should be
called
> > > "CTSTM".
> > > >> > >> > Hopefully,
> > > >> > >> > > >> the
> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined there in
MET.
> If
> > > >> not, we
> > > >> > >> can
> > > >> > >> > > >> always
> > > >> > >> > > >> > add them.
> > > >> > >> > > >> >
> > > >> > >> > > >> > Please try running plot_data_plane at verbosity
level
> 3
> > on
> > > >> one
> > > >> > of
> > > >> > >> > > these
> > > >> > >> > > >> > files first. And make sure it runs without
error and
> the
> > > log
> > > >> > >> message
> > > >> > >> > > >> prints
> > > >> > >> > > >> > out the "name" it found for that data. If so,
you can
> > just
> > > >> skip
> > > >> > >> > using
> > > >> > >> > > >> all
> > > >> > >> > > >> > those integers and revert back to using the
name MET
> > > knows.
> > > >> If
> > > >> > >> not,
> > > >> > >> > > >> then we
> > > >> > >> > > >> > may need to define a custom GRIB table for this
ECMWF
> > > GRIB2
> > > >> > data.
> > > >> > >> > > >> >
> > > >> > >> > > >> > Thanks,
> > > >> > >> > > >> > John
> > > >> > >> > > >> >
> > > >> > >> > > >>
> > > >> > >> > > >>
> > > >> > >> > > >> --
> > > >> > >> > > >>
> > > >> > >> > > >> Binbin Zhou
> > > >> > >> > > >>
> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > >> > >> > > >>
> > > >> > >> > > >> 5830 University Research Ct.
> > > >> > >> > > >>
> > > >> > >> > > >> College Park, MD 20740
> > > >> > >> > > >>
> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > >> > >> > > >>
> > > >> > >> > > >> 301-683-3683
> > > >> > >> > > >>
> > > >> > >> > > >>
> > > >> > >> > > >>
> > > >> > >> > > >> --
> > > >> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >> > >> > > >> Tara Jensen
> > > >> > >> > > >> Project Manager II
> > > >> > >> > > >> NCAR RAL and DTC
> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > > >> > >> > > >>
> > > >> > >> > > >
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >> > --
> > > >> > >> >
> > > >> > >> > Binbin Zhou
> > > >> > >> >
> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > >> >
> > > >> > >> > 5830 University Research Ct.
> > > >> > >> >
> > > >> > >> > College Park, MD 20740
> > > >> > >> >
> > > >> > >> > Binbin.Zhou at noaa.gov
> > > >> > >> >
> > > >> > >> > 301-683-3683
> > > >> > >> >
> > > >> > >> >
> > > >> > >>
> > > >> > >>
> > > >> > >
> > > >> > > --
> > > >> > >
> > > >> > > Binbin Zhou
> > > >> > >
> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > >
> > > >> > > 5830 University Research Ct.
> > > >> > >
> > > >> > > College Park, MD 20740
> > > >> > >
> > > >> > > Binbin.Zhou at noaa.gov
> > > >> > >
> > > >> > > 301-683-3683
> > > >> > >
> > > >> >
> > > >> >
> > > >> > --
> > > >> >
> > > >> > Binbin Zhou
> > > >> >
> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >> >
> > > >> > 5830 University Research Ct.
> > > >> >
> > > >> > College Park, MD 20740
> > > >> >
> > > >> > Binbin.Zhou at noaa.gov
> > > >> >
> > > >> > 301-683-3683
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > >
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> > >
> >
> >
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Wed Dec 09 17:18:11 2020
Julie,
I have created a bugfix branch for main_v9.1 named
"feature_1604_ens_config" and it's up on GitHub.
Only a single file has changed in it. But I need to get this version
compiled on WCOSS with which Binbin can test.
Are you to help me with this? Would it be easier to use git to
checkout the
repo on WCOSS? Or would it be easier if I built a tarfile for this
version
of the code?
Thanks,
John
On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Binbin,
>
> In order for you to test this change out, we'd need to recompile a
version
> of the code on WCOSS. I wrote this up as a GitHub issue:
> https://github.com/dtcenter/MET/issues/1604
>
> And I'll create a branch from main_v9.1 named
"bugfix_1604_ens_config" and
> commit the change there. Then we can checkout that code on WCOSS,
compile
> it, and you can test it.
>
> Yes, you're right, if the fcst fields differ from the climo fields,
you'd
> need to specify them separately. That's pretty easy in the Ensemble-
Stat
> config file... but are you asking how to control that via METplus?
>
> John
>
> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>>
>> John,
>>
>> Great! Let me test it at first before make change in MET9.1
>> Another question:
>> Since climatology data is still in NCEP's grib2 format, In the
>> ensemble_stat
>> config file, climo entry settings should be different from those
of
>> fcst, obs entries?
>> How to set climo (name, level, etc.)?
>>
>> Binbin
>>
>>
>>
>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> > Binbin,
>> >
>> > Success! When I recompile with that one line commented out,
>> plot_data_plane
>> > is able to filter down all those records to the single one that
we want.
>> > Note in the example below that I used the "lead_time" and
"GRIB_ens"
>> > options to filter the input records down.
>> >
>> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
>> level="P500";
>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact
match for
>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
>> "ensposte.t00z.z500hr".*
>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> > matching VarInfo "HGT/P500" in GRIB file "ensposte.t00z.z500hr".
>> > DEBUG 1: Creating postscript file: plot.ps
>> >
>> > Also note that this selects out record number 26.
>> >
>> > Shall I make this change as a "bugfix" for met-9.1...which could
be
>> > included in a met-9.1.2 bugfix release? Or shall I just include
it in
>> > development for the next release (planned for March 2021)?
>> >
>> > Thanks,
>> > John
>> >
>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> > >
>> > > John,
>> > >
>> > > If this removing impacts other applications, or takes some
time
>> > > to be effective, then won't try. I will use another approach,
such as
>> > > borrow GEFS's grib2 data structure as tank to save EC ensemble
>> > > data.
>> > >
>> > > Binbin
>> > >
>> > >
>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
>> > > binbin.zhou at noaa.gov> wrote:
>> > >
>> > > > John,
>> > > >
>> > > > Yes, please try to use this approach.
>> > > >
>> > > > Thanks!
>> > > >
>> > > > Binbin
>> > > >
>> > > > Binbin
>> > > >
>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
>> > > > met_help at ucar.edu> wrote:
>> > > >
>> > > >> Binbin,
>> > > >>
>> > > >> I used the "-PDS10" option for wgrib to dump those records:
>> > > >>
>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
>> > > >>
>> > > >>
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
>> > > >> | egrep "^11:|^26:|^43:"
>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
>> 0 0
>> > 0
>> > > 21
>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
>> 0 0
>> > 0
>> > > 21
>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
>> 0 0
>> > 0
>> > > 21
>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
>> > > >>
>> > > >> Looking VERY closely, there is a difference in the 42-nd
column
>> which
>> > is
>> > > >> the 4-th from the last... records 11, 26, and 43 have values
of 3,
>> 1,
>> > > and
>> > > >> 2
>> > > >> respectively.
>> > > >>
>> > > >> On this page (
>> > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
>> > > >> ),
>> > > >> byte 42 = 1 means "unperturbed control forecast".
>> > > >>
>> > > >> So we *should* be able to use the "GRIB_ens" config option
to
>> > > >> differentiate
>> > > >> between those records. BUT when I try, it doesn't work. And
that's
>> > > because
>> > > >> of this line:
>> > > >>
>> > > >>
>> > >
>> >
>>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
>> > > >>
>> > > >> We only apply those filtering options to GRIB files created
by
>> NCEP.
>> > If
>> > > >> you'd like, I could try removing that condition to see if we
can
>> pick
>> > > out
>> > > >> the single record of interest.
>> > > >>
>> > > >> John
>> > > >>
>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT <
>> > > >> met_help at ucar.edu> wrote:
>> > > >>
>> > > >> >
>> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> > > >> >
>> > > >> > John,
>> > > >> >
>> > > >> > Look at my
>> > > >> > directory
>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is only for
500 HGT.
>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep anl:type=analysis
>> > > >> > I get 3 records:
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > > >> > mb:anl:type=analysis:NAve=0
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > > >> > mb:anl:type=analysis:NAve=0
>> > > >> >
>> > > >> >
>> > > >>
>> > >
>> >
>>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > > >> > mb:anl:type=analysis:NAve=0
>> > > >> >
>> > > >> > From its grib2 file, we know only record#26 is analysis
low-res
>> > > >> control.
>> > > >> > If you have some way to determine which record is analysis
>> low-res
>> > > >> control
>> > > >> > from the grib1 file,
>> > > >> > then we can use grib1 directly (don't consider the size
issue).
>> > > >> >
>> > > >> > Binbin
>> > > >> >
>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
Affiliate <
>> > > >> > binbin.zhou at noaa.gov> wrote:
>> > > >> >
>> > > >> > > John,
>> > > >> > >
>> > > >> > > Grib1 data is too large, another issue is the low-res
control
>> > > >> analysis
>> > > >> > > data can not be retrieved
>> > > >> > > from the grib1 ecmwf data files. If using wgrib, we can
see 3
>> > > analysis
>> > > >> > > fields, but don't know
>> > > >> > > which is low-res control. After convert to grib2, wgrib2
can
>> show
>> > > >> which
>> > > >> > > record is low-res control.
>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set on
WCOSS.
>> > > >> > >
>> > > >> > > Binbin
>> > > >> > >
>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via RT
<
>> > > >> > > met_help at ucar.edu> wrote:
>> > > >> > >
>> > > >> > >> Binbin,
>> > > >> > >>
>> > > >> > >> Sounds like there's a lot of moving parts there. How
would you
>> > like
>> > > >> to
>> > > >> > >> proceed?
>> > > >> > >>
>> > > >> > >> Should we check to see if MET can read those ECMWF
GRIB1 files
>> > > >> directly
>> > > >> > >> without needing to reformat?
>> > > >> > >> Or do you just want to create a custom GRIB table for
the
>> result
>> > of
>> > > >> your
>> > > >> > >> pre-processing and conversion steps?
>> > > >> > >>
>> > > >> > >> John
>> > > >> > >>
>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov via
RT <
>> > > >> > >> met_help at ucar.edu> wrote:
>> > > >> > >>
>> > > >> > >> >
>> > > >> > >> > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> > >
>> > > >> > >> >
>> > > >> > >> > John,
>> > > >> > >> >
>> > > >> > >> > Our ECMWF grib2 ensemble data files are converted
from its
>> > > grib1
>> > > >> > >> format
>> > > >> > >> > data.
>> > > >> > >> > Before conversion, the parameter names are there, but
after
>> > > >> > conversion,
>> > > >> > >> the
>> > > >> > >> > names
>> > > >> > >> > are gone. It is because of unknown tables. The grib1
data is
>> > > still
>> > > >> > >> > original. The original
>> > > >> > >> > grib1 data files are very strange data format we can
not
>> read.
>> > So
>> > > >> we
>> > > >> > >> have
>> > > >> > >> > to use
>> > > >> > >> > a Frotran code to re-arrange the fields and save them
as
>> > > NCEP-like
>> > > >> > grib1
>> > > >> > >> > data files.
>> > > >> > >> > So after several processes, some important info like
tables
>> > could
>> > > >> be
>> > > >> > >> lost.
>> > > >> > >> >
>> > > >> > >> > I'll test your first suggestion since the field
record
>> order
>> > > is
>> > > >> > >> fixed.
>> > > >> > >> > Currently we still
>> > > >> > >> > obtained grib1 data from ECWMF, eventually, we will
get
>> grib2
>> > > data
>> > > >> > >> > directly. So before
>> > > >> > >> > we can get original grib2 files from ECMWF, we just
use this
>> > > >> temporal
>> > > >> > >> > solution.
>> > > >> > >> > '
>> > > >> > >> > Thanks!
>> > > >> > >> >
>> > > >> > >> >
>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway via
RT <
>> > > >> > >> > met_help at ucar.edu>
>> > > >> > >> > wrote:
>> > > >> > >> >
>> > > >> > >> > > Binbin,
>> > > >> > >> > >
>> > > >> > >> > > OK, working on Venus.
>> > > >> > >> > >
>> > > >> > >> > > You have a typo in the command you sent.
>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
>> > > >> > >> > >
>> > > >> > >>
>> > > >>
>> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > >> > >> > -v
>> > > >> > >> > > 3
>> > > >> > >> > >
>> > > >> > >> > > You have "RIB2_parm_cat" that should be
"GRIB2_parm_cat".
>> > After
>> > > >> > >> > correcting
>> > > >> > >> > > that I did still get an error but it's a very
useful one!
>> > > >> > >> > >
>> > > >> > >> > > module use
>> > > >> > >>
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
>> > > >> > >> > > module load met/9.1
>> > > >> > >> > > module load grib_util/1.1.1
>> > > >> > >> > > cd
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> >
>> > > >> > >>
>> > > >> >
>> > > >>
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
>> > > >> > >> > 'GRIB2_disc=255;
>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
>> level="P500";'
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
>> parameter
>> > > found
>> > > >> > with
>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
GRIB2_parm
>> (5).
>> > > Use
>> > > >> the
>> > > >> > >> > > MET_GRIB_TABLES environment variable to define
custom GRIB
>> > > >> > >> tables.ERROR
>> > > >> > >> > :
>> > > >> > >> > > *
>> > > >> > >> > >
>> > > >> > >> > > So this table is not yet defined in MET. I checked
their
>> > > website:
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> >
>> > > >> > >>
>> > > >> >
>> > > >>
>> > >
>> >
>> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
>> > > >> > >> > >
>> > > >> > >> > > But I don't see an entry defined for GRIB2_parm=5;
>> > > >> > >> > >
>> > > >> > >> > > 3 things...
>> > > >> > >> > >
>> > > >> > >> > > (1) As a crude temporary solution, you could refer
to it
>> by
>> > > it's
>> > > >> > >> record
>> > > >> > >> > > number. Let's assume you know that record 1 has
>> temperature
>> > > data:
>> > > >> > >> > >
>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
>> > > >> > 'name="TMP";
>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
>> > > >> > >> > >
>> > > >> > >> > > Here I'm reading record 1, naming it TMP and the
using
>> > > >> > set_attr_level
>> > > >> > >> > write
>> > > >> > >> > > the output level as P500 instead of R1.
>> > > >> > >> > >
>> > > >> > >> > > (2) You could create a custom GRIB table yourself
locally,
>> > and
>> > > >> then
>> > > >> > >> set
>> > > >> > >> > the
>> > > >> > >> > > MET_GRIB_TABLES environment variable to point MET
to it.
>> > > >> > >> > >
>> > > >> > >> > > (3) The best solution is contacting ECMWF and
requesting
>> they
>> > > >> send
>> > > >> > >> over
>> > > >> > >> > the
>> > > >> > >> > > GRIB2 table we should be using to decode this data.
>> > > >> > >> > >
>> > > >> > >> > > Thanks,
>> > > >> > >> > > John
>> > > >> > >> > >
>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway <
>> > > >> johnhg at ucar.edu>
>> > > >> > >> > wrote:
>> > > >> > >> > >
>> > > >> > >> > > > We might already have them. I'm not sure.
>> > > >> > >> > > >
>> > > >> > >> > > > But there's an additional required piece of the
puzzle.
>> > > Binbin
>> > > >> > runs
>> > > >> > >> > > > "wgrib2" to figure out what "name" he should use
in the
>> MET
>> > > >> config
>> > > >> > >> > files.
>> > > >> > >> > > > If wgrib2 doesn't tell him the right "name", the
problem
>> > will
>> > > >> > >> remain.
>> > > >> > >> > > > wgrib2 can be configured to define additional
tables as
>> > well,
>> > > >> but
>> > > >> > >> for
>> > > >> > >> > > this
>> > > >> > >> > > > to be easy for Binbin, that'd mean updating the
script
>> > that's
>> > > >> run
>> > > >> > >> for
>> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
guessing
>> that's
>> > > not
>> > > >> > very
>> > > >> > >> > easy.
>> > > >> > >> > > >
>> > > >> > >> > > > Yes, I do agree that the best solution is getting
the
>> ECMWF
>> > > >> GRIB
>> > > >> > >> tables
>> > > >> > >> > > > and adding them for both wgrib2 and MET.
>> > > >> > >> > > >
>> > > >> > >> > > > John
>> > > >> > >> > > >
>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
>> > jensen at ucar.edu>
>> > > >> > wrote:
>> > > >> > >> > > >
>> > > >> > >> > > >> John,
>> > > >> > >> > > >>
>> > > >> > >> > > >> It seems to me that it would be worthwhile to
just ask
>> > ECMWF
>> > > >> for
>> > > >> > >> their
>> > > >> > >> > > >> Grib2 Table for their Ensemble files. And other
>> > operational
>> > > >> > >> centers
>> > > >> > >> > for
>> > > >> > >> > > >> that matter. Do you agree this would be a good
>> solution?
>> > > >> > >> > > >>
>> > > >> > >> > > >> Thanks, Tara
>> > > >> > >> > > >>
>> > > >> > >> > > >> ---------- Forwarded message ---------
>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT
<met_help at ucar.edu>
>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent
question
>> for
>> > > >> > reading
>> > > >> > >> > ECMWF
>> > > >> > >> > > >> ensmeble data
>> > > >> > >> > > >> To:
>> > > >> > >> > > >>
>> > > >> > >> > > >>
>> > > >> > >> > > >>
>> > > >> > >> > > >> <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> > > >> >
>> > > >> > >> > > >>
>> > > >> > >> > > >> John,
>> > > >> > >> > > >>
>> > > >> > >> > > >> I tested using following command:
>> > > >> > >> > > >>
>> > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
>> > > >> > >> > > >>
>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > >> -v3
>> > > >> > >> > > >>
>> > > >> > >> > > >> ERROR :
>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name
or
>> > > >> GRIB2_disc,
>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in
>> field
>> > > >> > >> information
>> > > >> > >> > > >>
>> > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
>> > > >> > >> > > >>
>> > > >> > >> >
>> > > >> >
>> > >
>>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > >> > >> > > >> -v
>> > > >> > >> > > >> 3
>> > > >> > >> > > >>
>> > > >> > >> > > >> Still got error
>> > > >> > >> > > >> ERROR :
>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either name
or
>> > > >> GRIB2_disc,
>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in
>> field
>> > > >> > >> information
>> > > >> > >> > > >>
>> > > >> > >> > > >> Looks like discipline=255 is not recognized
>> > > >> > >> > > >>
>> > > >> > >> > > >> Binbin
>> > > >> > >> > > >>
>> > > >> > >> > > >>
>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley
Gotway via
>> RT
>> > <
>> > > >> > >> > > >> met_help at ucar.edu>
>> > > >> > >> > > >> wrote:
>> > > >> > >> > > >>
>> > > >> > >> > > >> > Binbin,
>> > > >> > >> > > >> > You can do something similar with MET. Please
go to
>> this
>> > > >> link:
>> > > >> > >> > > >> >
>> > > >>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and you'll
find the
>> > > >> content
>> > > >> > >> > > >> > copied/pasted below. You can specify these
INSTEAD OF
>> > > >> > specifying
>> > > >> > >> a
>> > > >> > >> > > >> > requested name.
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
specifying
>> the
>> > > >> master
>> > > >> > >> > table
>> > > >> > >> > > >> > number.
>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
specifying
>> the
>> > > >> local
>> > > >> > >> table
>> > > >> > >> > > >> > number.
>> > > >> > >> > > >> > // - The "GRIB2_disc" is an integer
specifying
>> the
>> > > >> GRIB2
>> > > >> > >> > > >> discipline
>> > > >> > >> > > >> > code.
>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
>> specifying
>> > > the
>> > > >> > >> > parameter
>> > > >> > >> > > >> > category
>> > > >> > >> > > >> > // code.
>> > > >> > >> > > >> > // - The "GRIB2_parm" is an integer
specifying
>> the
>> > > >> > >> parameter
>> > > >> > >> > > code.
>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
specifying
>> the
>> > > >> product
>> > > >> > >> > > >> definition
>> > > >> > >> > > >> > // template (Table 4.0).
>> > > >> > >> > > >> > // - The "GRIB2_process" is an integer
>> specifying
>> > > the
>> > > >> > >> > generating
>> > > >> > >> > > >> > process
>> > > >> > >> > > >> > // (Table 4.3).
>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
specifying
>> the
>> > > >> > >> originating
>> > > >> > >> > > >> > center.
>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an integer
>> specifying
>> > > the
>> > > >> > >> > ensemble
>> > > >> > >> > > >> type
>> > > >> > >> > > >> > // (Table 4.6).
>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an integer
>> specifying
>> > > the
>> > > >> > >> derived
>> > > >> > >> > > >> > product
>> > > >> > >> > > >> > // type (Table 4.7).
>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an integer
>> > specifying
>> > > >> the
>> > > >> > >> > > >> statistical
>> > > >> > >> > > >> > // processing type (Table 4.10).
>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
>> > > "GRIB2_ipdtmpl_val"
>> > > >> > >> entries
>> > > >> > >> > > are
>> > > >> > >> > > >> > arrays
>> > > >> > >> > > >> > // of integers which specify the
product
>> > > description
>> > > >> > >> > template
>> > > >> > >> > > >> > values to
>> > > >> > >> > > >> > // be used. The indices are 0-based.
For
>> > example,
>> > > >> use
>> > > >> > the
>> > > >> > >> > > >> > following to
>> > > >> > >> > > >> > // request a GRIB2 record whose 9-th
and
>> 27-th
>> > > >> product
>> > > >> > >> > > >> description
>> > > >> > >> > > >> > // template values are 1 and 2,
respectively:
>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
>> > > >> > GRIB2_ipdtmpl_val=[1,
>> > > >> > >> 2];
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > While I don't have any ECMWF data on hand for
>> testing, I
>> > > do
>> > > >> > have
>> > > >> > >> a
>> > > >> > >> > > file
>> > > >> > >> > > >> > from the Air Force from which wgrib2 prints a
similar
>> > > >> result:
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
center=57
>> > > >> > >> local_table=1
>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > So I used that for testing. I ran the
following
>> > > >> plot_data_plane
>> > > >> > >> > > command
>> > > >> > >> > > >> > WITHOUT specifying a requested name:
>> > > >> > >> > > >> >
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2
>> plot.ps
>> > > >> > >> > 'GRIB2_disc=0;
>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1; GRIB2_parm_cat=1;
>> > > >> GRIB2_parm=192;
>> > > >> > >> > > >> level="Z0";'
>> > > >> > >> > > >> > -v 3
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > And then I looked closely at the log message
to
>> confirm
>> > > >> that it
>> > > >> > >> used
>> > > >> > >> > > >> > record 549, as expected:
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() ->
Found
>> exact
>> > > match
>> > > >> > for
>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of
GRIB2 file
>> > > >> > >> > > >> >
>> > > >> > >> > > >>
>> > > >> > >> > >
>> > > >> > >> >
>> > > >> > >>
>> > > >> >
>> > > >>
>> > >
>> >
>>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > So MET used these GRIB2 numbers I defined to
do a
>> table
>> > > >> lookup
>> > > >> > >> and
>> > > >> > >> > > find
>> > > >> > >> > > >> a
>> > > >> > >> > > >> > match in the MET GRIB tables:
>> > > >> > >> > > >> >
>> > > >> > >>
>> > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > And it determined that that field should be
called
>> > > "CTSTM".
>> > > >> > >> > Hopefully,
>> > > >> > >> > > >> the
>> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined there
in MET.
>> If
>> > > >> not, we
>> > > >> > >> can
>> > > >> > >> > > >> always
>> > > >> > >> > > >> > add them.
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > Please try running plot_data_plane at
verbosity
>> level 3
>> > on
>> > > >> one
>> > > >> > of
>> > > >> > >> > > these
>> > > >> > >> > > >> > files first. And make sure it runs without
error and
>> the
>> > > log
>> > > >> > >> message
>> > > >> > >> > > >> prints
>> > > >> > >> > > >> > out the "name" it found for that data. If so,
you can
>> > just
>> > > >> skip
>> > > >> > >> > using
>> > > >> > >> > > >> all
>> > > >> > >> > > >> > those integers and revert back to using the
name MET
>> > > knows.
>> > > >> If
>> > > >> > >> not,
>> > > >> > >> > > >> then we
>> > > >> > >> > > >> > may need to define a custom GRIB table for
this ECMWF
>> > > GRIB2
>> > > >> > data.
>> > > >> > >> > > >> >
>> > > >> > >> > > >> > Thanks,
>> > > >> > >> > > >> > John
>> > > >> > >> > > >> >
>> > > >> > >> > > >>
>> > > >> > >> > > >>
>> > > >> > >> > > >> --
>> > > >> > >> > > >>
>> > > >> > >> > > >> Binbin Zhou
>> > > >> > >> > > >>
>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
>> > > >> > >> > > >>
>> > > >> > >> > > >> 5830 University Research Ct.
>> > > >> > >> > > >>
>> > > >> > >> > > >> College Park, MD 20740
>> > > >> > >> > > >>
>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
>> > > >> > >> > > >>
>> > > >> > >> > > >> 301-683-3683
>> > > >> > >> > > >>
>> > > >> > >> > > >>
>> > > >> > >> > > >>
>> > > >> > >> > > >> --
>> > > >> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > > >> > >> > > >> Tara Jensen
>> > > >> > >> > > >> Project Manager II
>> > > >> > >> > > >> NCAR RAL and DTC
>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
>> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
>> > > >> > >> > > >>
>> > > >> > >> > > >
>> > > >> > >> > >
>> > > >> > >> > >
>> > > >> > >> >
>> > > >> > >> > --
>> > > >> > >> >
>> > > >> > >> > Binbin Zhou
>> > > >> > >> >
>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
>> > > >> > >> >
>> > > >> > >> > 5830 University Research Ct.
>> > > >> > >> >
>> > > >> > >> > College Park, MD 20740
>> > > >> > >> >
>> > > >> > >> > Binbin.Zhou at noaa.gov
>> > > >> > >> >
>> > > >> > >> > 301-683-3683
>> > > >> > >> >
>> > > >> > >> >
>> > > >> > >>
>> > > >> > >>
>> > > >> > >
>> > > >> > > --
>> > > >> > >
>> > > >> > > Binbin Zhou
>> > > >> > >
>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
>> > > >> > >
>> > > >> > > 5830 University Research Ct.
>> > > >> > >
>> > > >> > > College Park, MD 20740
>> > > >> > >
>> > > >> > > Binbin.Zhou at noaa.gov
>> > > >> > >
>> > > >> > > 301-683-3683
>> > > >> > >
>> > > >> >
>> > > >> >
>> > > >> > --
>> > > >> >
>> > > >> > Binbin Zhou
>> > > >> >
>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
>> > > >> >
>> > > >> > 5830 University Research Ct.
>> > > >> >
>> > > >> > College Park, MD 20740
>> > > >> >
>> > > >> > Binbin.Zhou at noaa.gov
>> > > >> >
>> > > >> > 301-683-3683
>> > > >> >
>> > > >> >
>> > > >>
>> > > >>
>> > > >
>> > > > --
>> > > >
>> > > > Binbin Zhou
>> > > >
>> > > > IMSG at NOAA/NWS/NCEP/EMC
>> > > >
>> > > > 5830 University Research Ct.
>> > > >
>> > > > College Park, MD 20740
>> > > >
>> > > > Binbin.Zhou at noaa.gov
>> > > >
>> > > > 301-683-3683
>> > > >
>> > >
>> > >
>> > > --
>> > >
>> > > Binbin Zhou
>> > >
>> > > IMSG at NOAA/NWS/NCEP/EMC
>> > >
>> > > 5830 University Research Ct.
>> > >
>> > > College Park, MD 20740
>> > >
>> > > Binbin.Zhou at noaa.gov
>> > >
>> > > 301-683-3683
>> > >
>> > >
>> >
>> >
>>
>> --
>>
>> Binbin Zhou
>>
>> IMSG at NOAA/NWS/NCEP/EMC
>>
>> 5830 University Research Ct.
>>
>> College Park, MD 20740
>>
>> Binbin.Zhou at noaa.gov
>>
>> 301-683-3683
>>
>>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Thu Dec 10 06:25:18 2020
John,
Sure, I'll test the testing version of MET_9.1 after it is
available.
First step is just MET, not METplus. Could you please provide
an example of how to specify the climo entry in ensemble_stat
config file?
Thanks!
On Wed, Dec 9, 2020 at 6:53 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> In order for you to test this change out, we'd need to recompile a
version
> of the code on WCOSS. I wrote this up as a GitHub issue:
> https://github.com/dtcenter/MET/issues/1604
>
> And I'll create a branch from main_v9.1 named
"bugfix_1604_ens_config" and
> commit the change there. Then we can checkout that code on WCOSS,
compile
> it, and you can test it.
>
> Yes, you're right, if the fcst fields differ from the climo fields,
you'd
> need to specify them separately. That's pretty easy in the Ensemble-
Stat
> config file... but are you asking how to control that via METplus?
>
> John
>
> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > Great! Let me test it at first before make change in MET9.1
> > Another question:
> > Since climatology data is still in NCEP's grib2 format, In the
> > ensemble_stat
> > config file, climo entry settings should be different from
those of
> > fcst, obs entries?
> > How to set climo (name, level, etc.)?
> >
> > Binbin
> >
> >
> >
> > On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binbin,
> > >
> > > Success! When I recompile with that one line commented out,
> > plot_data_plane
> > > is able to filter down all those records to the single one that
we
> want.
> > > Note in the example below that I used the "lead_time" and
"GRIB_ens"
> > > options to filter the input records down.
> > >
> > > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
> level="P500";
> > > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact
match for
> > > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > "ensposte.t00z.z500hr".*
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > > matching VarInfo "HGT/P500" in GRIB file "ensposte.t00z.z500hr".
> > > DEBUG 1: Creating postscript file: plot.ps
> > >
> > > Also note that this selects out record number 26.
> > >
> > > Shall I make this change as a "bugfix" for met-9.1...which could
be
> > > included in a met-9.1.2 bugfix release? Or shall I just include
it in
> > > development for the next release (planned for March 2021)?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > > >
> > > > John,
> > > >
> > > > If this removing impacts other applications, or takes some
time
> > > > to be effective, then won't try. I will use another approach,
such
> as
> > > > borrow GEFS's grib2 data structure as tank to save EC ensemble
> > > > data.
> > > >
> > > > Binbin
> > > >
> > > >
> > > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
> > > > binbin.zhou at noaa.gov> wrote:
> > > >
> > > > > John,
> > > > >
> > > > > Yes, please try to use this approach.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Binbin
> > > > >
> > > > > Binbin
> > > > >
> > > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Binbin,
> > > > >>
> > > > >> I used the "-PDS10" option for wgrib to dump those records:
> > > > >>
> > > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > > >>
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > > >> | egrep "^11:|^26:|^43:"
> > > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
> 0
> > 0
> > > 0
> > > > 21
> > > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
> 0
> > 0
> > > 0
> > > > 21
> > > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
> 0
> > 0
> > > 0
> > > > 21
> > > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > > >>
> > > > >> Looking VERY closely, there is a difference in the 42-nd
column
> > which
> > > is
> > > > >> the 4-th from the last... records 11, 26, and 43 have
values of 3,
> > 1,
> > > > and
> > > > >> 2
> > > > >> respectively.
> > > > >>
> > > > >> On this page (
> > > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > > >> ),
> > > > >> byte 42 = 1 means "unperturbed control forecast".
> > > > >>
> > > > >> So we *should* be able to use the "GRIB_ens" config option
to
> > > > >> differentiate
> > > > >> between those records. BUT when I try, it doesn't work. And
that's
> > > > because
> > > > >> of this line:
> > > > >>
> > > > >>
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > > >>
> > > > >> We only apply those filtering options to GRIB files created
by
> NCEP.
> > > If
> > > > >> you'd like, I could try removing that condition to see if
we can
> > pick
> > > > out
> > > > >> the single record of interest.
> > > > >>
> > > > >> John
> > > > >>
> > > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT
<
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > > >> >
> > > > >> > John,
> > > > >> >
> > > > >> > Look at my
> > > > >> > directory
> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > > >> > The grib1 file is ensposte.t00z.z500hr, this is only for
500
> HGT.
> > > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
anl:type=analysis
> > > > >> > I get 3 records:
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >> > mb:anl:type=analysis:NAve=0
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >> > mb:anl:type=analysis:NAve=0
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >> > mb:anl:type=analysis:NAve=0
> > > > >> >
> > > > >> > From its grib2 file, we know only record#26 is analysis
low-res
> > > > >> control.
> > > > >> > If you have some way to determine which record is
analysis
> low-res
> > > > >> control
> > > > >> > from the grib1 file,
> > > > >> > then we can use grib1 directly (don't consider the size
issue).
> > > > >> >
> > > > >> > Binbin
> > > > >> >
> > > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
Affiliate <
> > > > >> > binbin.zhou at noaa.gov> wrote:
> > > > >> >
> > > > >> > > John,
> > > > >> > >
> > > > >> > > Grib1 data is too large, another issue is the low-res
> control
> > > > >> analysis
> > > > >> > > data can not be retrieved
> > > > >> > > from the grib1 ecmwf data files. If using wgrib, we can
see 3
> > > > analysis
> > > > >> > > fields, but don't know
> > > > >> > > which is low-res control. After convert to grib2,
wgrib2 can
> > show
> > > > >> which
> > > > >> > > record is low-res control.
> > > > >> > > Tomorrow I'll show you the ecmwf grib1 data set on
WCOSS.
> > > > >> > >
> > > > >> > > Binbin
> > > > >> > >
> > > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via
RT <
> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > >
> > > > >> > >> Binbin,
> > > > >> > >>
> > > > >> > >> Sounds like there's a lot of moving parts there. How
would
> you
> > > like
> > > > >> to
> > > > >> > >> proceed?
> > > > >> > >>
> > > > >> > >> Should we check to see if MET can read those ECMWF
GRIB1
> files
> > > > >> directly
> > > > >> > >> without needing to reformat?
> > > > >> > >> Or do you just want to create a custom GRIB table for
the
> > result
> > > of
> > > > >> your
> > > > >> > >> pre-processing and conversion steps?
> > > > >> > >>
> > > > >> > >> John
> > > > >> > >>
> > > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov
via RT <
> > > > >> > >> met_help at ucar.edu> wrote:
> > > > >> > >>
> > > > >> > >> >
> > > > >> > >> > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >
> > > > >> > >> >
> > > > >> > >> > John,
> > > > >> > >> >
> > > > >> > >> > Our ECMWF grib2 ensemble data files are converted
from
> its
> > > > grib1
> > > > >> > >> format
> > > > >> > >> > data.
> > > > >> > >> > Before conversion, the parameter names are there,
but after
> > > > >> > conversion,
> > > > >> > >> the
> > > > >> > >> > names
> > > > >> > >> > are gone. It is because of unknown tables. The grib1
data
> is
> > > > still
> > > > >> > >> > original. The original
> > > > >> > >> > grib1 data files are very strange data format we can
not
> > read.
> > > So
> > > > >> we
> > > > >> > >> have
> > > > >> > >> > to use
> > > > >> > >> > a Frotran code to re-arrange the fields and save
them as
> > > > NCEP-like
> > > > >> > grib1
> > > > >> > >> > data files.
> > > > >> > >> > So after several processes, some important info like
tables
> > > could
> > > > >> be
> > > > >> > >> lost.
> > > > >> > >> >
> > > > >> > >> > I'll test your first suggestion since the field
record
> > order
> > > > is
> > > > >> > >> fixed.
> > > > >> > >> > Currently we still
> > > > >> > >> > obtained grib1 data from ECWMF, eventually, we will
get
> grib2
> > > > data
> > > > >> > >> > directly. So before
> > > > >> > >> > we can get original grib2 files from ECMWF, we just
use
> this
> > > > >> temporal
> > > > >> > >> > solution.
> > > > >> > >> > '
> > > > >> > >> > Thanks!
> > > > >> > >> >
> > > > >> > >> >
> > > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway
via RT <
> > > > >> > >> > met_help at ucar.edu>
> > > > >> > >> > wrote:
> > > > >> > >> >
> > > > >> > >> > > Binbin,
> > > > >> > >> > >
> > > > >> > >> > > OK, working on Venus.
> > > > >> > >> > >
> > > > >> > >> > > You have a typo in the command you sent.
> > > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > > > >> > >> > >
> > > > >> > >>
> > > > >>
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >> > >> > -v
> > > > >> > >> > > 3
> > > > >> > >> > >
> > > > >> > >> > > You have "RIB2_parm_cat" that should be
"GRIB2_parm_cat".
> > > After
> > > > >> > >> > correcting
> > > > >> > >> > > that I did still get an error but it's a very
useful one!
> > > > >> > >> > >
> > > > >> > >> > > module use
> > > > >> > >>
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > >> > >> > > module load met/9.1
> > > > >> > >> > > module load grib_util/1.1.1
> > > > >> > >> > > cd
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> >
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
> > > > >> > >> > 'GRIB2_disc=255;
> > > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
> > level="P500";'
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
> parameter
> > > > found
> > > > >> > with
> > > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
GRIB2_parm
> > (5).
> > > > Use
> > > > >> the
> > > > >> > >> > > MET_GRIB_TABLES environment variable to define
custom
> GRIB
> > > > >> > >> tables.ERROR
> > > > >> > >> > :
> > > > >> > >> > > *
> > > > >> > >> > >
> > > > >> > >> > > So this table is not yet defined in MET. I checked
their
> > > > website:
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> >
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > > >> > >> > >
> > > > >> > >> > > But I don't see an entry defined for GRIB2_parm=5;
> > > > >> > >> > >
> > > > >> > >> > > 3 things...
> > > > >> > >> > >
> > > > >> > >> > > (1) As a crude temporary solution, you could refer
to it
> by
> > > > it's
> > > > >> > >> record
> > > > >> > >> > > number. Let's assume you know that record 1 has
> temperature
> > > > data:
> > > > >> > >> > >
> > > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
> > > > >> > 'name="TMP";
> > > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > > > >> > >> > >
> > > > >> > >> > > Here I'm reading record 1, naming it TMP and the
using
> > > > >> > set_attr_level
> > > > >> > >> > write
> > > > >> > >> > > the output level as P500 instead of R1.
> > > > >> > >> > >
> > > > >> > >> > > (2) You could create a custom GRIB table yourself
> locally,
> > > and
> > > > >> then
> > > > >> > >> set
> > > > >> > >> > the
> > > > >> > >> > > MET_GRIB_TABLES environment variable to point MET
to it.
> > > > >> > >> > >
> > > > >> > >> > > (3) The best solution is contacting ECMWF and
requesting
> > they
> > > > >> send
> > > > >> > >> over
> > > > >> > >> > the
> > > > >> > >> > > GRIB2 table we should be using to decode this
data.
> > > > >> > >> > >
> > > > >> > >> > > Thanks,
> > > > >> > >> > > John
> > > > >> > >> > >
> > > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway
<
> > > > >> johnhg at ucar.edu>
> > > > >> > >> > wrote:
> > > > >> > >> > >
> > > > >> > >> > > > We might already have them. I'm not sure.
> > > > >> > >> > > >
> > > > >> > >> > > > But there's an additional required piece of the
puzzle.
> > > > Binbin
> > > > >> > runs
> > > > >> > >> > > > "wgrib2" to figure out what "name" he should use
in the
> > MET
> > > > >> config
> > > > >> > >> > files.
> > > > >> > >> > > > If wgrib2 doesn't tell him the right "name", the
> problem
> > > will
> > > > >> > >> remain.
> > > > >> > >> > > > wgrib2 can be configured to define additional
tables as
> > > well,
> > > > >> but
> > > > >> > >> for
> > > > >> > >> > > this
> > > > >> > >> > > > to be easy for Binbin, that'd mean updating the
script
> > > that's
> > > > >> run
> > > > >> > >> for
> > > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
guessing
> > that's
> > > > not
> > > > >> > very
> > > > >> > >> > easy.
> > > > >> > >> > > >
> > > > >> > >> > > > Yes, I do agree that the best solution is
getting the
> > ECMWF
> > > > >> GRIB
> > > > >> > >> tables
> > > > >> > >> > > > and adding them for both wgrib2 and MET.
> > > > >> > >> > > >
> > > > >> > >> > > > John
> > > > >> > >> > > >
> > > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
> > > jensen at ucar.edu>
> > > > >> > wrote:
> > > > >> > >> > > >
> > > > >> > >> > > >> John,
> > > > >> > >> > > >>
> > > > >> > >> > > >> It seems to me that it would be worthwhile to
just ask
> > > ECMWF
> > > > >> for
> > > > >> > >> their
> > > > >> > >> > > >> Grib2 Table for their Ensemble files. And
other
> > > operational
> > > > >> > >> centers
> > > > >> > >> > for
> > > > >> > >> > > >> that matter. Do you agree this would be a good
> > solution?
> > > > >> > >> > > >>
> > > > >> > >> > > >> Thanks, Tara
> > > > >> > >> > > >>
> > > > >> > >> > > >> ---------- Forwarded message ---------
> > > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT
<met_help at ucar.edu>
> > > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent
question
> > for
> > > > >> > reading
> > > > >> > >> > ECMWF
> > > > >> > >> > > >> ensmeble data
> > > > >> > >> > > >> To:
> > > > >> > >> > > >>
> > > > >> > >> > > >>
> > > > >> > >> > > >>
> > > > >> > >> > > >> <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >> >
> > > > >> > >> > > >>
> > > > >> > >> > > >> John,
> > > > >> > >> > > >>
> > > > >> > >> > > >> I tested using following command:
> > > > >> > >> > > >>
> > > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > > > >> > >> > > >>
> > 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >> -v3
> > > > >> > >> > > >>
> > > > >> > >> > > >> ERROR :
> > > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name or
> > > > >> GRIB2_disc,
> > > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in
> field
> > > > >> > >> information
> > > > >> > >> > > >>
> > > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> > > > >> > >> > > >>
> > > > >> > >> >
> > > > >> >
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >> > >> > > >> -v
> > > > >> > >> > > >> 3
> > > > >> > >> > > >>
> > > > >> > >> > > >> Still got error
> > > > >> > >> > > >> ERROR :
> > > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name or
> > > > >> GRIB2_disc,
> > > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in
> field
> > > > >> > >> information
> > > > >> > >> > > >>
> > > > >> > >> > > >> Looks like discipline=255 is not recognized
> > > > >> > >> > > >>
> > > > >> > >> > > >> Binbin
> > > > >> > >> > > >>
> > > > >> > >> > > >>
> > > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley
Gotway via
> > RT
> > > <
> > > > >> > >> > > >> met_help at ucar.edu>
> > > > >> > >> > > >> wrote:
> > > > >> > >> > > >>
> > > > >> > >> > > >> > Binbin,
> > > > >> > >> > > >> > You can do something similar with MET. Please
go to
> > this
> > > > >> link:
> > > > >> > >> > > >> >
> > > > >>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > And then search for "GRIB2_mtab", and you'll
find
> the
> > > > >> content
> > > > >> > >> > > >> > copied/pasted below. You can specify these
INSTEAD
> OF
> > > > >> > specifying
> > > > >> > >> a
> > > > >> > >> > > >> > requested name.
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
specifying
> > the
> > > > >> master
> > > > >> > >> > table
> > > > >> > >> > > >> > number.
> > > > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
specifying
> > the
> > > > >> local
> > > > >> > >> table
> > > > >> > >> > > >> > number.
> > > > >> > >> > > >> > // - The "GRIB2_disc" is an integer
specifying
> > the
> > > > >> GRIB2
> > > > >> > >> > > >> discipline
> > > > >> > >> > > >> > code.
> > > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
> > specifying
> > > > the
> > > > >> > >> > parameter
> > > > >> > >> > > >> > category
> > > > >> > >> > > >> > // code.
> > > > >> > >> > > >> > // - The "GRIB2_parm" is an integer
specifying
> > the
> > > > >> > >> parameter
> > > > >> > >> > > code.
> > > > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
specifying
> > the
> > > > >> product
> > > > >> > >> > > >> definition
> > > > >> > >> > > >> > // template (Table 4.0).
> > > > >> > >> > > >> > // - The "GRIB2_process" is an integer
> > specifying
> > > > the
> > > > >> > >> > generating
> > > > >> > >> > > >> > process
> > > > >> > >> > > >> > // (Table 4.3).
> > > > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
specifying
> > the
> > > > >> > >> originating
> > > > >> > >> > > >> > center.
> > > > >> > >> > > >> > // - The "GRIB2_ens_type" is an integer
> > specifying
> > > > the
> > > > >> > >> > ensemble
> > > > >> > >> > > >> type
> > > > >> > >> > > >> > // (Table 4.6).
> > > > >> > >> > > >> > // - The "GRIB2_der_type" is an integer
> > specifying
> > > > the
> > > > >> > >> derived
> > > > >> > >> > > >> > product
> > > > >> > >> > > >> > // type (Table 4.7).
> > > > >> > >> > > >> > // - The "GRIB2_stat_type" is an
integer
> > > specifying
> > > > >> the
> > > > >> > >> > > >> statistical
> > > > >> > >> > > >> > // processing type (Table 4.10).
> > > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
> > > > "GRIB2_ipdtmpl_val"
> > > > >> > >> entries
> > > > >> > >> > > are
> > > > >> > >> > > >> > arrays
> > > > >> > >> > > >> > // of integers which specify the
product
> > > > description
> > > > >> > >> > template
> > > > >> > >> > > >> > values to
> > > > >> > >> > > >> > // be used. The indices are 0-based.
For
> > > example,
> > > > >> use
> > > > >> > the
> > > > >> > >> > > >> > following to
> > > > >> > >> > > >> > // request a GRIB2 record whose 9-th
and
> 27-th
> > > > >> product
> > > > >> > >> > > >> description
> > > > >> > >> > > >> > // template values are 1 and 2,
> respectively:
> > > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> > > > >> > GRIB2_ipdtmpl_val=[1,
> > > > >> > >> 2];
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > While I don't have any ECMWF data on hand for
> > testing, I
> > > > do
> > > > >> > have
> > > > >> > >> a
> > > > >> > >> > > file
> > > > >> > >> > > >> > from the Air Force from which wgrib2 prints a
> similar
> > > > >> result:
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
> center=57
> > > > >> > >> local_table=1
> > > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > So I used that for testing. I ran the
following
> > > > >> plot_data_plane
> > > > >> > >> > > command
> > > > >> > >> > > >> > WITHOUT specifying a requested name:
> > > > >> > >> > > >> >
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2
> plot.ps
> > > > >> > >> > 'GRIB2_disc=0;
> > > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
GRIB2_parm_cat=1;
> > > > >> GRIB2_parm=192;
> > > > >> > >> > > >> level="Z0";'
> > > > >> > >> > > >> > -v 3
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > And then I looked closely at the log message
to
> > confirm
> > > > >> that it
> > > > >> > >> used
> > > > >> > >> > > >> > record 549, as expected:
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() ->
Found
> exact
> > > > match
> > > > >> > for
> > > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of
GRIB2 file
> > > > >> > >> > > >> >
> > > > >> > >> > > >>
> > > > >> > >> > >
> > > > >> > >> >
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > So MET used these GRIB2 numbers I defined to
do a
> > table
> > > > >> lookup
> > > > >> > >> and
> > > > >> > >> > > find
> > > > >> > >> > > >> a
> > > > >> > >> > > >> > match in the MET GRIB tables:
> > > > >> > >> > > >> >
> > > > >> > >>
> > > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > And it determined that that field should be
called
> > > > "CTSTM".
> > > > >> > >> > Hopefully,
> > > > >> > >> > > >> the
> > > > >> > >> > > >> > ECMWF GRIB2 tables are already defined there
in MET.
> > If
> > > > >> not, we
> > > > >> > >> can
> > > > >> > >> > > >> always
> > > > >> > >> > > >> > add them.
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > Please try running plot_data_plane at
verbosity
> level
> > 3
> > > on
> > > > >> one
> > > > >> > of
> > > > >> > >> > > these
> > > > >> > >> > > >> > files first. And make sure it runs without
error and
> > the
> > > > log
> > > > >> > >> message
> > > > >> > >> > > >> prints
> > > > >> > >> > > >> > out the "name" it found for that data. If so,
you
> can
> > > just
> > > > >> skip
> > > > >> > >> > using
> > > > >> > >> > > >> all
> > > > >> > >> > > >> > those integers and revert back to using the
name MET
> > > > knows.
> > > > >> If
> > > > >> > >> not,
> > > > >> > >> > > >> then we
> > > > >> > >> > > >> > may need to define a custom GRIB table for
this
> ECMWF
> > > > GRIB2
> > > > >> > data.
> > > > >> > >> > > >> >
> > > > >> > >> > > >> > Thanks,
> > > > >> > >> > > >> > John
> > > > >> > >> > > >> >
> > > > >> > >> > > >>
> > > > >> > >> > > >>
> > > > >> > >> > > >> --
> > > > >> > >> > > >>
> > > > >> > >> > > >> Binbin Zhou
> > > > >> > >> > > >>
> > > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > >> > > >>
> > > > >> > >> > > >> 5830 University Research Ct.
> > > > >> > >> > > >>
> > > > >> > >> > > >> College Park, MD 20740
> > > > >> > >> > > >>
> > > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > > >> > >> > > >>
> > > > >> > >> > > >> 301-683-3683
> > > > >> > >> > > >>
> > > > >> > >> > > >>
> > > > >> > >> > > >>
> > > > >> > >> > > >> --
> > > > >> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > >> > >> > > >> Tara Jensen
> > > > >> > >> > > >> Project Manager II
> > > > >> > >> > > >> NCAR RAL and DTC
> > > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > > > >> > >> > > >>
> > > > >> > >> > > >
> > > > >> > >> > >
> > > > >> > >> > >
> > > > >> > >> >
> > > > >> > >> > --
> > > > >> > >> >
> > > > >> > >> > Binbin Zhou
> > > > >> > >> >
> > > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > >> >
> > > > >> > >> > 5830 University Research Ct.
> > > > >> > >> >
> > > > >> > >> > College Park, MD 20740
> > > > >> > >> >
> > > > >> > >> > Binbin.Zhou at noaa.gov
> > > > >> > >> >
> > > > >> > >> > 301-683-3683
> > > > >> > >> >
> > > > >> > >> >
> > > > >> > >>
> > > > >> > >>
> > > > >> > >
> > > > >> > > --
> > > > >> > >
> > > > >> > > Binbin Zhou
> > > > >> > >
> > > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > >
> > > > >> > > 5830 University Research Ct.
> > > > >> > >
> > > > >> > > College Park, MD 20740
> > > > >> > >
> > > > >> > > Binbin.Zhou at noaa.gov
> > > > >> > >
> > > > >> > > 301-683-3683
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> >
> > > > >> > Binbin Zhou
> > > > >> >
> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> >
> > > > >> > 5830 University Research Ct.
> > > > >> >
> > > > >> > College Park, MD 20740
> > > > >> >
> > > > >> > Binbin.Zhou at noaa.gov
> > > > >> >
> > > > >> > 301-683-3683
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > >
> > > > > Binbin Zhou
> > > > >
> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >
> > > > > 5830 University Research Ct.
> > > > >
> > > > > College Park, MD 20740
> > > > >
> > > > > Binbin.Zhou at noaa.gov
> > > > >
> > > > > 301-683-3683
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Julie Prestopnik
Time: Thu Dec 10 09:24:29 2020
John,
I am able to help with this install on WCOSS.
Would it be easier to use git to checkout the repo on WCOSS? Or would
it be
> easier if I built a tarfile for this version of the code?
>
The last time I tried to build from git on WCOSS (a while ago), I
couldn't. I don't remember all of the details, but I think I couldn't
run
bootstrap and maybe it had to do with the version of autoconf being
too
old? I'm happy to try it again if you'd like though. Otherwise, I
know a
tarfile would work, but I know that is extra work for you.
Julie
On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Julie,
>
> I have created a bugfix branch for main_v9.1 named
> "feature_1604_ens_config" and it's up on GitHub.
>
> Only a single file has changed in it. But I need to get this version
> compiled on WCOSS with which Binbin can test.
>
> Are you to help me with this? Would it be easier to use git to
checkout
> the repo on WCOSS? Or would it be easier if I built a tarfile for
this
> version of the code?
>
> Thanks,
> John
>
> On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
>> Binbin,
>>
>> In order for you to test this change out, we'd need to recompile a
>> version of the code on WCOSS. I wrote this up as a GitHub issue:
>> https://github.com/dtcenter/MET/issues/1604
>>
>> And I'll create a branch from main_v9.1 named
"bugfix_1604_ens_config"
>> and commit the change there. Then we can checkout that code on
WCOSS,
>> compile it, and you can test it.
>>
>> Yes, you're right, if the fcst fields differ from the climo fields,
you'd
>> need to specify them separately. That's pretty easy in the
Ensemble-Stat
>> config file... but are you asking how to control that via METplus?
>>
>> John
>>
>> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>>>
>>> John,
>>>
>>> Great! Let me test it at first before make change in MET9.1
>>> Another question:
>>> Since climatology data is still in NCEP's grib2 format, In the
>>> ensemble_stat
>>> config file, climo entry settings should be different from
those of
>>> fcst, obs entries?
>>> How to set climo (name, level, etc.)?
>>>
>>> Binbin
>>>
>>>
>>>
>>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
>>> met_help at ucar.edu>
>>> wrote:
>>>
>>> > Binbin,
>>> >
>>> > Success! When I recompile with that one line commented out,
>>> plot_data_plane
>>> > is able to filter down all those records to the single one that
we
>>> want.
>>> > Note in the example below that I used the "lead_time" and
"GRIB_ens"
>>> > options to filter the input records down.
>>> >
>>> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
>>> level="P500";
>>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
>>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
>>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact
match for
>>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
>>> "ensposte.t00z.z500hr".*
>>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> > matching VarInfo "HGT/P500" in GRIB file "ensposte.t00z.z500hr".
>>> > DEBUG 1: Creating postscript file: plot.ps
>>> >
>>> > Also note that this selects out record number 26.
>>> >
>>> > Shall I make this change as a "bugfix" for met-9.1...which could
be
>>> > included in a met-9.1.2 bugfix release? Or shall I just include
it in
>>> > development for the next release (planned for March 2021)?
>>> >
>>> > Thanks,
>>> > John
>>> >
>>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
>>> > met_help at ucar.edu> wrote:
>>> >
>>> > >
>>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
>>> > >
>>> > > John,
>>> > >
>>> > > If this removing impacts other applications, or takes some
time
>>> > > to be effective, then won't try. I will use another approach,
such
>>> as
>>> > > borrow GEFS's grib2 data structure as tank to save EC ensemble
>>> > > data.
>>> > >
>>> > > Binbin
>>> > >
>>> > >
>>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate <
>>> > > binbin.zhou at noaa.gov> wrote:
>>> > >
>>> > > > John,
>>> > > >
>>> > > > Yes, please try to use this approach.
>>> > > >
>>> > > > Thanks!
>>> > > >
>>> > > > Binbin
>>> > > >
>>> > > > Binbin
>>> > > >
>>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
>>> > > > met_help at ucar.edu> wrote:
>>> > > >
>>> > > >> Binbin,
>>> > > >>
>>> > > >> I used the "-PDS10" option for wgrib to dump those records:
>>> > > >>
>>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
>>> > > >> | egrep "^11:|^26:|^43:"
>>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
>>> 0 0
>>> > 0
>>> > > 21
>>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
>>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
>>> 0 0
>>> > 0
>>> > > 21
>>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
>>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0 1
0 0 1
>>> 0 0
>>> > 0
>>> > > 21
>>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
>>> > > >>
>>> > > >> Looking VERY closely, there is a difference in the 42-nd
column
>>> which
>>> > is
>>> > > >> the 4-th from the last... records 11, 26, and 43 have
values of
>>> 3, 1,
>>> > > and
>>> > > >> 2
>>> > > >> respectively.
>>> > > >>
>>> > > >> On this page (
>>> > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
>>> > > >> ),
>>> > > >> byte 42 = 1 means "unperturbed control forecast".
>>> > > >>
>>> > > >> So we *should* be able to use the "GRIB_ens" config option
to
>>> > > >> differentiate
>>> > > >> between those records. BUT when I try, it doesn't work. And
that's
>>> > > because
>>> > > >> of this line:
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
>>> > > >>
>>> > > >> We only apply those filtering options to GRIB files created
by
>>> NCEP.
>>> > If
>>> > > >> you'd like, I could try removing that condition to see if
we can
>>> pick
>>> > > out
>>> > > >> the single record of interest.
>>> > > >>
>>> > > >> John
>>> > > >>
>>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via RT
<
>>> > > >> met_help at ucar.edu> wrote:
>>> > > >>
>>> > > >> >
>>> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>>> > > >> >
>>> > > >> > John,
>>> > > >> >
>>> > > >> > Look at my
>>> > > >> > directory
>>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
>>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is only for
500
>>> HGT.
>>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
anl:type=analysis
>>> > > >> > I get 3 records:
>>> > > >> >
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>>> > > >> > mb:anl:type=analysis:NAve=0
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>>> > > >> > mb:anl:type=analysis:NAve=0
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>>> > > >> > mb:anl:type=analysis:NAve=0
>>> > > >> >
>>> > > >> > From its grib2 file, we know only record#26 is analysis
low-res
>>> > > >> control.
>>> > > >> > If you have some way to determine which record is
analysis
>>> low-res
>>> > > >> control
>>> > > >> > from the grib1 file,
>>> > > >> > then we can use grib1 directly (don't consider the size
issue).
>>> > > >> >
>>> > > >> > Binbin
>>> > > >> >
>>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
Affiliate <
>>> > > >> > binbin.zhou at noaa.gov> wrote:
>>> > > >> >
>>> > > >> > > John,
>>> > > >> > >
>>> > > >> > > Grib1 data is too large, another issue is the low-res
>>> control
>>> > > >> analysis
>>> > > >> > > data can not be retrieved
>>> > > >> > > from the grib1 ecmwf data files. If using wgrib, we can
see 3
>>> > > analysis
>>> > > >> > > fields, but don't know
>>> > > >> > > which is low-res control. After convert to grib2,
wgrib2 can
>>> show
>>> > > >> which
>>> > > >> > > record is low-res control.
>>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set on
WCOSS.
>>> > > >> > >
>>> > > >> > > Binbin
>>> > > >> > >
>>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via
RT <
>>> > > >> > > met_help at ucar.edu> wrote:
>>> > > >> > >
>>> > > >> > >> Binbin,
>>> > > >> > >>
>>> > > >> > >> Sounds like there's a lot of moving parts there. How
would
>>> you
>>> > like
>>> > > >> to
>>> > > >> > >> proceed?
>>> > > >> > >>
>>> > > >> > >> Should we check to see if MET can read those ECMWF
GRIB1
>>> files
>>> > > >> directly
>>> > > >> > >> without needing to reformat?
>>> > > >> > >> Or do you just want to create a custom GRIB table for
the
>>> result
>>> > of
>>> > > >> your
>>> > > >> > >> pre-processing and conversion steps?
>>> > > >> > >>
>>> > > >> > >> John
>>> > > >> > >>
>>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov
via RT <
>>> > > >> > >> met_help at ucar.edu> wrote:
>>> > > >> > >>
>>> > > >> > >> >
>>> > > >> > >> > <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>>> > >
>>> > > >> > >> >
>>> > > >> > >> > John,
>>> > > >> > >> >
>>> > > >> > >> > Our ECMWF grib2 ensemble data files are converted
from
>>> its
>>> > > grib1
>>> > > >> > >> format
>>> > > >> > >> > data.
>>> > > >> > >> > Before conversion, the parameter names are there,
but after
>>> > > >> > conversion,
>>> > > >> > >> the
>>> > > >> > >> > names
>>> > > >> > >> > are gone. It is because of unknown tables. The grib1
data
>>> is
>>> > > still
>>> > > >> > >> > original. The original
>>> > > >> > >> > grib1 data files are very strange data format we can
not
>>> read.
>>> > So
>>> > > >> we
>>> > > >> > >> have
>>> > > >> > >> > to use
>>> > > >> > >> > a Frotran code to re-arrange the fields and save
them as
>>> > > NCEP-like
>>> > > >> > grib1
>>> > > >> > >> > data files.
>>> > > >> > >> > So after several processes, some important info like
tables
>>> > could
>>> > > >> be
>>> > > >> > >> lost.
>>> > > >> > >> >
>>> > > >> > >> > I'll test your first suggestion since the field
record
>>> order
>>> > > is
>>> > > >> > >> fixed.
>>> > > >> > >> > Currently we still
>>> > > >> > >> > obtained grib1 data from ECWMF, eventually, we will
get
>>> grib2
>>> > > data
>>> > > >> > >> > directly. So before
>>> > > >> > >> > we can get original grib2 files from ECMWF, we just
use
>>> this
>>> > > >> temporal
>>> > > >> > >> > solution.
>>> > > >> > >> > '
>>> > > >> > >> > Thanks!
>>> > > >> > >> >
>>> > > >> > >> >
>>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway
via RT <
>>> > > >> > >> > met_help at ucar.edu>
>>> > > >> > >> > wrote:
>>> > > >> > >> >
>>> > > >> > >> > > Binbin,
>>> > > >> > >> > >
>>> > > >> > >> > > OK, working on Venus.
>>> > > >> > >> > >
>>> > > >> > >> > > You have a typo in the command you sent.
>>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
>>> > > >> > >> > >
>>> > > >> > >>
>>> > > >>
>>> >
>>>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>>> > > >> > >> > -v
>>> > > >> > >> > > 3
>>> > > >> > >> > >
>>> > > >> > >> > > You have "RIB2_parm_cat" that should be
"GRIB2_parm_cat".
>>> > After
>>> > > >> > >> > correcting
>>> > > >> > >> > > that I did still get an error but it's a very
useful one!
>>> > > >> > >> > >
>>> > > >> > >> > > module use
>>> > > >> > >>
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
>>> > > >> > >> > > module load met/9.1
>>> > > >> > >> > > module load grib_util/1.1.1
>>> > > >> > >> > > cd
>>> > > >> > >> > >
>>> > > >> > >> > >
>>> > > >> > >> >
>>> > > >> > >>
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
>>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
>>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
>>> > > >> > >> > 'GRIB2_disc=255;
>>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
>>> level="P500";'
>>> > > >> > >> > >
>>> > > >> > >> > >
>>> > > >> > >> > >
>>> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() -> no
>>> parameter
>>> > > found
>>> > > >> > with
>>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
GRIB2_parm
>>> (5).
>>> > > Use
>>> > > >> the
>>> > > >> > >> > > MET_GRIB_TABLES environment variable to define
custom
>>> GRIB
>>> > > >> > >> tables.ERROR
>>> > > >> > >> > :
>>> > > >> > >> > > *
>>> > > >> > >> > >
>>> > > >> > >> > > So this table is not yet defined in MET. I checked
their
>>> > > website:
>>> > > >> > >> > >
>>> > > >> > >> > >
>>> > > >> > >> >
>>> > > >> > >>
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
>>> > > >> > >> > >
>>> > > >> > >> > > But I don't see an entry defined for GRIB2_parm=5;
>>> > > >> > >> > >
>>> > > >> > >> > > 3 things...
>>> > > >> > >> > >
>>> > > >> > >> > > (1) As a crude temporary solution, you could refer
to it
>>> by
>>> > > it's
>>> > > >> > >> record
>>> > > >> > >> > > number. Let's assume you know that record 1 has
>>> temperature
>>> > > data:
>>> > > >> > >> > >
>>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/plot.ps
>>> > > >> > 'name="TMP";
>>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
>>> > > >> > >> > >
>>> > > >> > >> > > Here I'm reading record 1, naming it TMP and the
using
>>> > > >> > set_attr_level
>>> > > >> > >> > write
>>> > > >> > >> > > the output level as P500 instead of R1.
>>> > > >> > >> > >
>>> > > >> > >> > > (2) You could create a custom GRIB table yourself
>>> locally,
>>> > and
>>> > > >> then
>>> > > >> > >> set
>>> > > >> > >> > the
>>> > > >> > >> > > MET_GRIB_TABLES environment variable to point MET
to it.
>>> > > >> > >> > >
>>> > > >> > >> > > (3) The best solution is contacting ECMWF and
requesting
>>> they
>>> > > >> send
>>> > > >> > >> over
>>> > > >> > >> > the
>>> > > >> > >> > > GRIB2 table we should be using to decode this
data.
>>> > > >> > >> > >
>>> > > >> > >> > > Thanks,
>>> > > >> > >> > > John
>>> > > >> > >> > >
>>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley Gotway
<
>>> > > >> johnhg at ucar.edu>
>>> > > >> > >> > wrote:
>>> > > >> > >> > >
>>> > > >> > >> > > > We might already have them. I'm not sure.
>>> > > >> > >> > > >
>>> > > >> > >> > > > But there's an additional required piece of the
puzzle.
>>> > > Binbin
>>> > > >> > runs
>>> > > >> > >> > > > "wgrib2" to figure out what "name" he should use
in
>>> the MET
>>> > > >> config
>>> > > >> > >> > files.
>>> > > >> > >> > > > If wgrib2 doesn't tell him the right "name", the
>>> problem
>>> > will
>>> > > >> > >> remain.
>>> > > >> > >> > > > wgrib2 can be configured to define additional
tables as
>>> > well,
>>> > > >> but
>>> > > >> > >> for
>>> > > >> > >> > > this
>>> > > >> > >> > > > to be easy for Binbin, that'd mean updating the
script
>>> > that's
>>> > > >> run
>>> > > >> > >> for
>>> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
guessing
>>> that's
>>> > > not
>>> > > >> > very
>>> > > >> > >> > easy.
>>> > > >> > >> > > >
>>> > > >> > >> > > > Yes, I do agree that the best solution is
getting the
>>> ECMWF
>>> > > >> GRIB
>>> > > >> > >> tables
>>> > > >> > >> > > > and adding them for both wgrib2 and MET.
>>> > > >> > >> > > >
>>> > > >> > >> > > > John
>>> > > >> > >> > > >
>>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
>>> > jensen at ucar.edu>
>>> > > >> > wrote:
>>> > > >> > >> > > >
>>> > > >> > >> > > >> John,
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> It seems to me that it would be worthwhile to
just ask
>>> > ECMWF
>>> > > >> for
>>> > > >> > >> their
>>> > > >> > >> > > >> Grib2 Table for their Ensemble files. And
other
>>> > operational
>>> > > >> > >> centers
>>> > > >> > >> > for
>>> > > >> > >> > > >> that matter. Do you agree this would be a good
>>> solution?
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> Thanks, Tara
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> ---------- Forwarded message ---------
>>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT
<met_help at ucar.edu>
>>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
>>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent
>>> question for
>>> > > >> > reading
>>> > > >> > >> > ECMWF
>>> > > >> > >> > > >> ensmeble data
>>> > > >> > >> > > >> To:
>>> > > >> > >> > > >>
>>> > > >> > >> > > >>
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> <URL:
>>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>>> > > >> >
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> John,
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> I tested using following command:
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
>>> > > >> > >> > > >>
>>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>>> > > >> -v3
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> ERROR :
>>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name or
>>> > > >> GRIB2_disc,
>>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in
>>> field
>>> > > >> > >> information
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
>>> > > >> > >> > > >>
>>> > > >> > >> >
>>> > > >> >
>>> > >
>>>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>>> > > >> > >> > > >> -v
>>> > > >> > >> > > >> 3
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> Still got error
>>> > > >> > >> > > >> ERROR :
>>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name or
>>> > > >> GRIB2_disc,
>>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be specified
in
>>> field
>>> > > >> > >> information
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> Looks like discipline=255 is not recognized
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> Binbin
>>> > > >> > >> > > >>
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley
Gotway
>>> via RT
>>> > <
>>> > > >> > >> > > >> met_help at ucar.edu>
>>> > > >> > >> > > >> wrote:
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> > Binbin,
>>> > > >> > >> > > >> > You can do something similar with MET. Please
go to
>>> this
>>> > > >> link:
>>> > > >> > >> > > >> >
>>> > > >>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and you'll
find
>>> the
>>> > > >> content
>>> > > >> > >> > > >> > copied/pasted below. You can specify these
INSTEAD
>>> OF
>>> > > >> > specifying
>>> > > >> > >> a
>>> > > >> > >> > > >> > requested name.
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
>>> specifying the
>>> > > >> master
>>> > > >> > >> > table
>>> > > >> > >> > > >> > number.
>>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
>>> specifying the
>>> > > >> local
>>> > > >> > >> table
>>> > > >> > >> > > >> > number.
>>> > > >> > >> > > >> > // - The "GRIB2_disc" is an integer
>>> specifying the
>>> > > >> GRIB2
>>> > > >> > >> > > >> discipline
>>> > > >> > >> > > >> > code.
>>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an integer
>>> specifying
>>> > > the
>>> > > >> > >> > parameter
>>> > > >> > >> > > >> > category
>>> > > >> > >> > > >> > // code.
>>> > > >> > >> > > >> > // - The "GRIB2_parm" is an integer
>>> specifying the
>>> > > >> > >> parameter
>>> > > >> > >> > > code.
>>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
specifying
>>> the
>>> > > >> product
>>> > > >> > >> > > >> definition
>>> > > >> > >> > > >> > // template (Table 4.0).
>>> > > >> > >> > > >> > // - The "GRIB2_process" is an integer
>>> specifying
>>> > > the
>>> > > >> > >> > generating
>>> > > >> > >> > > >> > process
>>> > > >> > >> > > >> > // (Table 4.3).
>>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
>>> specifying the
>>> > > >> > >> originating
>>> > > >> > >> > > >> > center.
>>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an integer
>>> specifying
>>> > > the
>>> > > >> > >> > ensemble
>>> > > >> > >> > > >> type
>>> > > >> > >> > > >> > // (Table 4.6).
>>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an integer
>>> specifying
>>> > > the
>>> > > >> > >> derived
>>> > > >> > >> > > >> > product
>>> > > >> > >> > > >> > // type (Table 4.7).
>>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an
integer
>>> > specifying
>>> > > >> the
>>> > > >> > >> > > >> statistical
>>> > > >> > >> > > >> > // processing type (Table 4.10).
>>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
>>> > > "GRIB2_ipdtmpl_val"
>>> > > >> > >> entries
>>> > > >> > >> > > are
>>> > > >> > >> > > >> > arrays
>>> > > >> > >> > > >> > // of integers which specify the
product
>>> > > description
>>> > > >> > >> > template
>>> > > >> > >> > > >> > values to
>>> > > >> > >> > > >> > // be used. The indices are 0-based.
For
>>> > example,
>>> > > >> use
>>> > > >> > the
>>> > > >> > >> > > >> > following to
>>> > > >> > >> > > >> > // request a GRIB2 record whose 9-th
and
>>> 27-th
>>> > > >> product
>>> > > >> > >> > > >> description
>>> > > >> > >> > > >> > // template values are 1 and 2,
>>> respectively:
>>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
>>> > > >> > GRIB2_ipdtmpl_val=[1,
>>> > > >> > >> 2];
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > While I don't have any ECMWF data on hand for
>>> testing, I
>>> > > do
>>> > > >> > have
>>> > > >> > >> a
>>> > > >> > >> > > file
>>> > > >> > >> > > >> > from the Air Force from which wgrib2 prints a
>>> similar
>>> > > >> result:
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
>>> center=57
>>> > > >> > >> local_table=1
>>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > So I used that for testing. I ran the
following
>>> > > >> plot_data_plane
>>> > > >> > >> > > command
>>> > > >> > >> > > >> > WITHOUT specifying a requested name:
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > plot_data_plane GALWEM_20170610_CY00_FH18.GR2
>>> plot.ps
>>> > > >> > >> > 'GRIB2_disc=0;
>>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
GRIB2_parm_cat=1;
>>> > > >> GRIB2_parm=192;
>>> > > >> > >> > > >> level="Z0";'
>>> > > >> > >> > > >> > -v 3
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > And then I looked closely at the log message
to
>>> confirm
>>> > > >> that it
>>> > > >> > >> used
>>> > > >> > >> > > >> > record 549, as expected:
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() ->
Found
>>> exact
>>> > > match
>>> > > >> > for
>>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of
GRIB2 file
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >>
>>> > > >> > >> > >
>>> > > >> > >> >
>>> > > >> > >>
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > So MET used these GRIB2 numbers I defined to
do a
>>> table
>>> > > >> lookup
>>> > > >> > >> and
>>> > > >> > >> > > find
>>> > > >> > >> > > >> a
>>> > > >> > >> > > >> > match in the MET GRIB tables:
>>> > > >> > >> > > >> >
>>> > > >> > >>
>>> > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > And it determined that that field should be
called
>>> > > "CTSTM".
>>> > > >> > >> > Hopefully,
>>> > > >> > >> > > >> the
>>> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined there
in
>>> MET. If
>>> > > >> not, we
>>> > > >> > >> can
>>> > > >> > >> > > >> always
>>> > > >> > >> > > >> > add them.
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > Please try running plot_data_plane at
verbosity
>>> level 3
>>> > on
>>> > > >> one
>>> > > >> > of
>>> > > >> > >> > > these
>>> > > >> > >> > > >> > files first. And make sure it runs without
error
>>> and the
>>> > > log
>>> > > >> > >> message
>>> > > >> > >> > > >> prints
>>> > > >> > >> > > >> > out the "name" it found for that data. If so,
you
>>> can
>>> > just
>>> > > >> skip
>>> > > >> > >> > using
>>> > > >> > >> > > >> all
>>> > > >> > >> > > >> > those integers and revert back to using the
name MET
>>> > > knows.
>>> > > >> If
>>> > > >> > >> not,
>>> > > >> > >> > > >> then we
>>> > > >> > >> > > >> > may need to define a custom GRIB table for
this
>>> ECMWF
>>> > > GRIB2
>>> > > >> > data.
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >> > Thanks,
>>> > > >> > >> > > >> > John
>>> > > >> > >> > > >> >
>>> > > >> > >> > > >>
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> --
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> Binbin Zhou
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> 5830 University Research Ct.
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> College Park, MD 20740
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> 301-683-3683
>>> > > >> > >> > > >>
>>> > > >> > >> > > >>
>>> > > >> > >> > > >>
>>> > > >> > >> > > >> --
>>> > > >> > >> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> > > >> > >> > > >> Tara Jensen
>>> > > >> > >> > > >> Project Manager II
>>> > > >> > >> > > >> NCAR RAL and DTC
>>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
>>> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
>>> > > >> > >> > > >>
>>> > > >> > >> > > >
>>> > > >> > >> > >
>>> > > >> > >> > >
>>> > > >> > >> >
>>> > > >> > >> > --
>>> > > >> > >> >
>>> > > >> > >> > Binbin Zhou
>>> > > >> > >> >
>>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
>>> > > >> > >> >
>>> > > >> > >> > 5830 University Research Ct.
>>> > > >> > >> >
>>> > > >> > >> > College Park, MD 20740
>>> > > >> > >> >
>>> > > >> > >> > Binbin.Zhou at noaa.gov
>>> > > >> > >> >
>>> > > >> > >> > 301-683-3683
>>> > > >> > >> >
>>> > > >> > >> >
>>> > > >> > >>
>>> > > >> > >>
>>> > > >> > >
>>> > > >> > > --
>>> > > >> > >
>>> > > >> > > Binbin Zhou
>>> > > >> > >
>>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
>>> > > >> > >
>>> > > >> > > 5830 University Research Ct.
>>> > > >> > >
>>> > > >> > > College Park, MD 20740
>>> > > >> > >
>>> > > >> > > Binbin.Zhou at noaa.gov
>>> > > >> > >
>>> > > >> > > 301-683-3683
>>> > > >> > >
>>> > > >> >
>>> > > >> >
>>> > > >> > --
>>> > > >> >
>>> > > >> > Binbin Zhou
>>> > > >> >
>>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
>>> > > >> >
>>> > > >> > 5830 University Research Ct.
>>> > > >> >
>>> > > >> > College Park, MD 20740
>>> > > >> >
>>> > > >> > Binbin.Zhou at noaa.gov
>>> > > >> >
>>> > > >> > 301-683-3683
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > > >>
>>> > > >
>>> > > > --
>>> > > >
>>> > > > Binbin Zhou
>>> > > >
>>> > > > IMSG at NOAA/NWS/NCEP/EMC
>>> > > >
>>> > > > 5830 University Research Ct.
>>> > > >
>>> > > > College Park, MD 20740
>>> > > >
>>> > > > Binbin.Zhou at noaa.gov
>>> > > >
>>> > > > 301-683-3683
>>> > > >
>>> > >
>>> > >
>>> > > --
>>> > >
>>> > > Binbin Zhou
>>> > >
>>> > > IMSG at NOAA/NWS/NCEP/EMC
>>> > >
>>> > > 5830 University Research Ct.
>>> > >
>>> > > College Park, MD 20740
>>> > >
>>> > > Binbin.Zhou at noaa.gov
>>> > >
>>> > > 301-683-3683
>>> > >
>>> > >
>>> >
>>> >
>>>
>>> --
>>>
>>> Binbin Zhou
>>>
>>> IMSG at NOAA/NWS/NCEP/EMC
>>>
>>> 5830 University Research Ct.
>>>
>>> College Park, MD 20740
>>>
>>> Binbin.Zhou at noaa.gov
>>>
>>> 301-683-3683
>>>
>>>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Thu Dec 10 10:05:37 2020
Julie,
Building the tarfile is very easy. I just ran this script from the MET
repo
and specified the branch to be built:
MET/scripts/met_checkout_and_build.sh branch
feature_1604_ens_config
And that created a tarfile which I posted to RAL's anonymous ftp
server:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
I assume that'll work for you.
Thanks,
John
On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> I am able to help with this install on WCOSS.
>
> Would it be easier to use git to checkout the repo on WCOSS? Or
would it be
> > easier if I built a tarfile for this version of the code?
> >
> The last time I tried to build from git on WCOSS (a while ago), I
> couldn't. I don't remember all of the details, but I think I
couldn't run
> bootstrap and maybe it had to do with the version of autoconf being
too
> old? I'm happy to try it again if you'd like though. Otherwise, I
know a
> tarfile would work, but I know that is extra work for you.
>
> Julie
>
> On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> > Julie,
> >
> > I have created a bugfix branch for main_v9.1 named
> > "feature_1604_ens_config" and it's up on GitHub.
> >
> > Only a single file has changed in it. But I need to get this
version
> > compiled on WCOSS with which Binbin can test.
> >
> > Are you to help me with this? Would it be easier to use git to
checkout
> > the repo on WCOSS? Or would it be easier if I built a tarfile for
this
> > version of the code?
> >
> > Thanks,
> > John
> >
> > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
> >
> >> Binbin,
> >>
> >> In order for you to test this change out, we'd need to recompile
a
> >> version of the code on WCOSS. I wrote this up as a GitHub issue:
> >> https://github.com/dtcenter/MET/issues/1604
> >>
> >> And I'll create a branch from main_v9.1 named
"bugfix_1604_ens_config"
> >> and commit the change there. Then we can checkout that code on
WCOSS,
> >> compile it, and you can test it.
> >>
> >> Yes, you're right, if the fcst fields differ from the climo
fields,
> you'd
> >> need to specify them separately. That's pretty easy in the
Ensemble-Stat
> >> config file... but are you asking how to control that via
METplus?
> >>
> >> John
> >>
> >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >>>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >>>
> >>> John,
> >>>
> >>> Great! Let me test it at first before make change in MET9.1
> >>> Another question:
> >>> Since climatology data is still in NCEP's grib2 format, In
the
> >>> ensemble_stat
> >>> config file, climo entry settings should be different from
those of
> >>> fcst, obs entries?
> >>> How to set climo (name, level, etc.)?
> >>>
> >>> Binbin
> >>>
> >>>
> >>>
> >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> >>> met_help at ucar.edu>
> >>> wrote:
> >>>
> >>> > Binbin,
> >>> >
> >>> > Success! When I recompile with that one line commented out,
> >>> plot_data_plane
> >>> > is able to filter down all those records to the single one
that we
> >>> want.
> >>> > Note in the example below that I used the "lead_time" and
"GRIB_ens"
> >>> > options to filter the input records down.
> >>> >
> >>> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
> >>> level="P500";
> >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found exact
match
> for
> >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> >>> "ensposte.t00z.z500hr".*
> >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> >>> > matching VarInfo "HGT/P500" in GRIB file
"ensposte.t00z.z500hr".
> >>> > DEBUG 1: Creating postscript file: plot.ps
> >>> >
> >>> > Also note that this selects out record number 26.
> >>> >
> >>> > Shall I make this change as a "bugfix" for met-9.1...which
could be
> >>> > included in a met-9.1.2 bugfix release? Or shall I just
include it in
> >>> > development for the next release (planned for March 2021)?
> >>> >
> >>> > Thanks,
> >>> > John
> >>> >
> >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
> >>> > met_help at ucar.edu> wrote:
> >>> >
> >>> > >
> >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >>> > >
> >>> > > John,
> >>> > >
> >>> > > If this removing impacts other applications, or takes some
time
> >>> > > to be effective, then won't try. I will use another
approach, such
> >>> as
> >>> > > borrow GEFS's grib2 data structure as tank to save EC
ensemble
> >>> > > data.
> >>> > >
> >>> > > Binbin
> >>> > >
> >>> > >
> >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA Affiliate
<
> >>> > > binbin.zhou at noaa.gov> wrote:
> >>> > >
> >>> > > > John,
> >>> > > >
> >>> > > > Yes, please try to use this approach.
> >>> > > >
> >>> > > > Thanks!
> >>> > > >
> >>> > > > Binbin
> >>> > > >
> >>> > > > Binbin
> >>> > > >
> >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT <
> >>> > > > met_help at ucar.edu> wrote:
> >>> > > >
> >>> > > >> Binbin,
> >>> > > >>
> >>> > > >> I used the "-PDS10" option for wgrib to dump those
records:
> >>> > > >>
> >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> >>> > > >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> >>> > > >> | egrep "^11:|^26:|^43:"
> >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0
1 0 0
> 1
> >>> 0 0
> >>> > 0
> >>> > > 21
> >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0
1 0 0
> 1
> >>> 0 0
> >>> > 0
> >>> > > 21
> >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0 0
1 0 0
> 1
> >>> 0 0
> >>> > 0
> >>> > > 21
> >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> >>> > > >>
> >>> > > >> Looking VERY closely, there is a difference in the 42-nd
column
> >>> which
> >>> > is
> >>> > > >> the 4-th from the last... records 11, 26, and 43 have
values of
> >>> 3, 1,
> >>> > > and
> >>> > > >> 2
> >>> > > >> respectively.
> >>> > > >>
> >>> > > >> On this page (
> >>> > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> >>> > > >> ),
> >>> > > >> byte 42 = 1 means "unperturbed control forecast".
> >>> > > >>
> >>> > > >> So we *should* be able to use the "GRIB_ens" config
option to
> >>> > > >> differentiate
> >>> > > >> between those records. BUT when I try, it doesn't work.
And
> that's
> >>> > > because
> >>> > > >> of this line:
> >>> > > >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> >>> > > >>
> >>> > > >> We only apply those filtering options to GRIB files
created by
> >>> NCEP.
> >>> > If
> >>> > > >> you'd like, I could try removing that condition to see if
we can
> >>> pick
> >>> > > out
> >>> > > >> the single record of interest.
> >>> > > >>
> >>> > > >> John
> >>> > > >>
> >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via
RT <
> >>> > > >> met_help at ucar.edu> wrote:
> >>> > > >>
> >>> > > >> >
> >>> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> >>> > > >> >
> >>> > > >> > John,
> >>> > > >> >
> >>> > > >> > Look at my
> >>> > > >> > directory
> >>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is only
for 500
> >>> HGT.
> >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
anl:type=analysis
> >>> > > >> > I get 3 records:
> >>> > > >> >
> >>> > > >> >
> >>> > > >> >
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >>> > > >> > mb:anl:type=analysis:NAve=0
> >>> > > >> >
> >>> > > >> >
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >>> > > >> > mb:anl:type=analysis:NAve=0
> >>> > > >> >
> >>> > > >> >
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >>> > > >> > mb:anl:type=analysis:NAve=0
> >>> > > >> >
> >>> > > >> > From its grib2 file, we know only record#26 is
analysis
> low-res
> >>> > > >> control.
> >>> > > >> > If you have some way to determine which record is
analysis
> >>> low-res
> >>> > > >> control
> >>> > > >> > from the grib1 file,
> >>> > > >> > then we can use grib1 directly (don't consider the size
> issue).
> >>> > > >> >
> >>> > > >> > Binbin
> >>> > > >> >
> >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
Affiliate <
> >>> > > >> > binbin.zhou at noaa.gov> wrote:
> >>> > > >> >
> >>> > > >> > > John,
> >>> > > >> > >
> >>> > > >> > > Grib1 data is too large, another issue is the low-
res
> >>> control
> >>> > > >> analysis
> >>> > > >> > > data can not be retrieved
> >>> > > >> > > from the grib1 ecmwf data files. If using wgrib, we
can see
> 3
> >>> > > analysis
> >>> > > >> > > fields, but don't know
> >>> > > >> > > which is low-res control. After convert to grib2,
wgrib2 can
> >>> show
> >>> > > >> which
> >>> > > >> > > record is low-res control.
> >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set on
WCOSS.
> >>> > > >> > >
> >>> > > >> > > Binbin
> >>> > > >> > >
> >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway via
RT <
> >>> > > >> > > met_help at ucar.edu> wrote:
> >>> > > >> > >
> >>> > > >> > >> Binbin,
> >>> > > >> > >>
> >>> > > >> > >> Sounds like there's a lot of moving parts there. How
would
> >>> you
> >>> > like
> >>> > > >> to
> >>> > > >> > >> proceed?
> >>> > > >> > >>
> >>> > > >> > >> Should we check to see if MET can read those ECMWF
GRIB1
> >>> files
> >>> > > >> directly
> >>> > > >> > >> without needing to reformat?
> >>> > > >> > >> Or do you just want to create a custom GRIB table
for the
> >>> result
> >>> > of
> >>> > > >> your
> >>> > > >> > >> pre-processing and conversion steps?
> >>> > > >> > >>
> >>> > > >> > >> John
> >>> > > >> > >>
> >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM Binbin.Zhou at noaa.gov
via
> RT <
> >>> > > >> > >> met_help at ucar.edu> wrote:
> >>> > > >> > >>
> >>> > > >> > >> >
> >>> > > >> > >> > <URL:
> >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >>> > >
> >>> > > >> > >> >
> >>> > > >> > >> > John,
> >>> > > >> > >> >
> >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
converted from
> >>> its
> >>> > > grib1
> >>> > > >> > >> format
> >>> > > >> > >> > data.
> >>> > > >> > >> > Before conversion, the parameter names are there,
but
> after
> >>> > > >> > conversion,
> >>> > > >> > >> the
> >>> > > >> > >> > names
> >>> > > >> > >> > are gone. It is because of unknown tables. The
grib1 data
> >>> is
> >>> > > still
> >>> > > >> > >> > original. The original
> >>> > > >> > >> > grib1 data files are very strange data format we
can not
> >>> read.
> >>> > So
> >>> > > >> we
> >>> > > >> > >> have
> >>> > > >> > >> > to use
> >>> > > >> > >> > a Frotran code to re-arrange the fields and save
them as
> >>> > > NCEP-like
> >>> > > >> > grib1
> >>> > > >> > >> > data files.
> >>> > > >> > >> > So after several processes, some important info
like
> tables
> >>> > could
> >>> > > >> be
> >>> > > >> > >> lost.
> >>> > > >> > >> >
> >>> > > >> > >> > I'll test your first suggestion since the field
record
> >>> order
> >>> > > is
> >>> > > >> > >> fixed.
> >>> > > >> > >> > Currently we still
> >>> > > >> > >> > obtained grib1 data from ECWMF, eventually, we
will get
> >>> grib2
> >>> > > data
> >>> > > >> > >> > directly. So before
> >>> > > >> > >> > we can get original grib2 files from ECMWF, we
just use
> >>> this
> >>> > > >> temporal
> >>> > > >> > >> > solution.
> >>> > > >> > >> > '
> >>> > > >> > >> > Thanks!
> >>> > > >> > >> >
> >>> > > >> > >> >
> >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley Gotway
via RT
> <
> >>> > > >> > >> > met_help at ucar.edu>
> >>> > > >> > >> > wrote:
> >>> > > >> > >> >
> >>> > > >> > >> > > Binbin,
> >>> > > >> > >> > >
> >>> > > >> > >> > > OK, working on Venus.
> >>> > > >> > >> > >
> >>> > > >> > >> > > You have a typo in the command you sent.
> >>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
a.ps
> >>> > > >> > >> > >
> >>> > > >> > >>
> >>> > > >>
> >>> >
> >>>
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >>> > > >> > >> > -v
> >>> > > >> > >> > > 3
> >>> > > >> > >> > >
> >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
> "GRIB2_parm_cat".
> >>> > After
> >>> > > >> > >> > correcting
> >>> > > >> > >> > > that I did still get an error but it's a very
useful
> one!
> >>> > > >> > >> > >
> >>> > > >> > >> > > module use
> >>> > > >> > >>
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> >>> > > >> > >> > > module load met/9.1
> >>> > > >> > >> > > module load grib_util/1.1.1
> >>> > > >> > >> > > cd
> >>> > > >> > >> > >
> >>> > > >> > >> > >
> >>> > > >> > >> >
> >>> > > >> > >>
> >>> > > >> >
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> >>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/
> plot.ps
> >>> > > >> > >> > 'GRIB2_disc=255;
> >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
> >>> level="P500";'
> >>> > > >> > >> > >
> >>> > > >> > >> > >
> >>> > > >> > >> > >
> >>> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() ->
no
> >>> parameter
> >>> > > found
> >>> > > >> > with
> >>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
GRIB2_parm
> >>> (5).
> >>> > > Use
> >>> > > >> the
> >>> > > >> > >> > > MET_GRIB_TABLES environment variable to define
custom
> >>> GRIB
> >>> > > >> > >> tables.ERROR
> >>> > > >> > >> > :
> >>> > > >> > >> > > *
> >>> > > >> > >> > >
> >>> > > >> > >> > > So this table is not yet defined in MET. I
checked
> their
> >>> > > website:
> >>> > > >> > >> > >
> >>> > > >> > >> > >
> >>> > > >> > >> >
> >>> > > >> > >>
> >>> > > >> >
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> >>> > > >> > >> > >
> >>> > > >> > >> > > But I don't see an entry defined for
GRIB2_parm=5;
> >>> > > >> > >> > >
> >>> > > >> > >> > > 3 things...
> >>> > > >> > >> > >
> >>> > > >> > >> > > (1) As a crude temporary solution, you could
refer to
> it
> >>> by
> >>> > > it's
> >>> > > >> > >> record
> >>> > > >> > >> > > number. Let's assume you know that record 1 has
> >>> temperature
> >>> > > data:
> >>> > > >> > >> > >
> >>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/
> plot.ps
> >>> > > >> > 'name="TMP";
> >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> >>> > > >> > >> > >
> >>> > > >> > >> > > Here I'm reading record 1, naming it TMP and the
using
> >>> > > >> > set_attr_level
> >>> > > >> > >> > write
> >>> > > >> > >> > > the output level as P500 instead of R1.
> >>> > > >> > >> > >
> >>> > > >> > >> > > (2) You could create a custom GRIB table
yourself
> >>> locally,
> >>> > and
> >>> > > >> then
> >>> > > >> > >> set
> >>> > > >> > >> > the
> >>> > > >> > >> > > MET_GRIB_TABLES environment variable to point
MET to
> it.
> >>> > > >> > >> > >
> >>> > > >> > >> > > (3) The best solution is contacting ECMWF and
> requesting
> >>> they
> >>> > > >> send
> >>> > > >> > >> over
> >>> > > >> > >> > the
> >>> > > >> > >> > > GRIB2 table we should be using to decode this
data.
> >>> > > >> > >> > >
> >>> > > >> > >> > > Thanks,
> >>> > > >> > >> > > John
> >>> > > >> > >> > >
> >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley
Gotway <
> >>> > > >> johnhg at ucar.edu>
> >>> > > >> > >> > wrote:
> >>> > > >> > >> > >
> >>> > > >> > >> > > > We might already have them. I'm not sure.
> >>> > > >> > >> > > >
> >>> > > >> > >> > > > But there's an additional required piece of
the
> puzzle.
> >>> > > Binbin
> >>> > > >> > runs
> >>> > > >> > >> > > > "wgrib2" to figure out what "name" he should
use in
> >>> the MET
> >>> > > >> config
> >>> > > >> > >> > files.
> >>> > > >> > >> > > > If wgrib2 doesn't tell him the right "name",
the
> >>> problem
> >>> > will
> >>> > > >> > >> remain.
> >>> > > >> > >> > > > wgrib2 can be configured to define additional
tables
> as
> >>> > well,
> >>> > > >> but
> >>> > > >> > >> for
> >>> > > >> > >> > > this
> >>> > > >> > >> > > > to be easy for Binbin, that'd mean updating
the
> script
> >>> > that's
> >>> > > >> run
> >>> > > >> > >> for
> >>> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
guessing
> >>> that's
> >>> > > not
> >>> > > >> > very
> >>> > > >> > >> > easy.
> >>> > > >> > >> > > >
> >>> > > >> > >> > > > Yes, I do agree that the best solution is
getting the
> >>> ECMWF
> >>> > > >> GRIB
> >>> > > >> > >> tables
> >>> > > >> > >> > > > and adding them for both wgrib2 and MET.
> >>> > > >> > >> > > >
> >>> > > >> > >> > > > John
> >>> > > >> > >> > > >
> >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
> >>> > jensen at ucar.edu>
> >>> > > >> > wrote:
> >>> > > >> > >> > > >
> >>> > > >> > >> > > >> John,
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> It seems to me that it would be worthwhile to
just
> ask
> >>> > ECMWF
> >>> > > >> for
> >>> > > >> > >> their
> >>> > > >> > >> > > >> Grib2 Table for their Ensemble files. And
other
> >>> > operational
> >>> > > >> > >> centers
> >>> > > >> > >> > for
> >>> > > >> > >> > > >> that matter. Do you agree this would be a
good
> >>> solution?
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> Thanks, Tara
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> ---------- Forwarded message ---------
> >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu>
> >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769] urgent
> >>> question for
> >>> > > >> > reading
> >>> > > >> > >> > ECMWF
> >>> > > >> > >> > > >> ensmeble data
> >>> > > >> > >> > > >> To:
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> <URL:
> >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >>> > > >> >
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> John,
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> I tested using following command:
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> a.ps
> >>> > > >> > >> > > >>
> >>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >>> > > >> -v3
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> ERROR :
> >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name or
> >>> > > >> GRIB2_disc,
> >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified in
> >>> field
> >>> > > >> > >> information
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> a.ps
> >>> > > >> > >> > > >>
> >>> > > >> > >> >
> >>> > > >> >
> >>> > >
> >>>
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >>> > > >> > >> > > >> -v
> >>> > > >> > >> > > >> 3
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> Still got error
> >>> > > >> > >> > > >> ERROR :
> >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name or
> >>> > > >> GRIB2_disc,
> >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified in
> >>> field
> >>> > > >> > >> information
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> Looks like discipline=255 is not recognized
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> Binbin
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley
Gotway
> >>> via RT
> >>> > <
> >>> > > >> > >> > > >> met_help at ucar.edu>
> >>> > > >> > >> > > >> wrote:
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> > Binbin,
> >>> > > >> > >> > > >> > You can do something similar with MET.
Please go
> to
> >>> this
> >>> > > >> link:
> >>> > > >> > >> > > >> >
> >>> > > >>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and
you'll find
> >>> the
> >>> > > >> content
> >>> > > >> > >> > > >> > copied/pasted below. You can specify these
INSTEAD
> >>> OF
> >>> > > >> > specifying
> >>> > > >> > >> a
> >>> > > >> > >> > > >> > requested name.
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
> >>> specifying the
> >>> > > >> master
> >>> > > >> > >> > table
> >>> > > >> > >> > > >> > number.
> >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
> >>> specifying the
> >>> > > >> local
> >>> > > >> > >> table
> >>> > > >> > >> > > >> > number.
> >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an integer
> >>> specifying the
> >>> > > >> GRIB2
> >>> > > >> > >> > > >> discipline
> >>> > > >> > >> > > >> > code.
> >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an
integer
> >>> specifying
> >>> > > the
> >>> > > >> > >> > parameter
> >>> > > >> > >> > > >> > category
> >>> > > >> > >> > > >> > // code.
> >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an integer
> >>> specifying the
> >>> > > >> > >> parameter
> >>> > > >> > >> > > code.
> >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
> specifying
> >>> the
> >>> > > >> product
> >>> > > >> > >> > > >> definition
> >>> > > >> > >> > > >> > // template (Table 4.0).
> >>> > > >> > >> > > >> > // - The "GRIB2_process" is an
integer
> >>> specifying
> >>> > > the
> >>> > > >> > >> > generating
> >>> > > >> > >> > > >> > process
> >>> > > >> > >> > > >> > // (Table 4.3).
> >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
> >>> specifying the
> >>> > > >> > >> originating
> >>> > > >> > >> > > >> > center.
> >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an
integer
> >>> specifying
> >>> > > the
> >>> > > >> > >> > ensemble
> >>> > > >> > >> > > >> type
> >>> > > >> > >> > > >> > // (Table 4.6).
> >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an
integer
> >>> specifying
> >>> > > the
> >>> > > >> > >> derived
> >>> > > >> > >> > > >> > product
> >>> > > >> > >> > > >> > // type (Table 4.7).
> >>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an
integer
> >>> > specifying
> >>> > > >> the
> >>> > > >> > >> > > >> statistical
> >>> > > >> > >> > > >> > // processing type (Table 4.10).
> >>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
> >>> > > "GRIB2_ipdtmpl_val"
> >>> > > >> > >> entries
> >>> > > >> > >> > > are
> >>> > > >> > >> > > >> > arrays
> >>> > > >> > >> > > >> > // of integers which specify the
product
> >>> > > description
> >>> > > >> > >> > template
> >>> > > >> > >> > > >> > values to
> >>> > > >> > >> > > >> > // be used. The indices are 0-
based. For
> >>> > example,
> >>> > > >> use
> >>> > > >> > the
> >>> > > >> > >> > > >> > following to
> >>> > > >> > >> > > >> > // request a GRIB2 record whose 9-
th and
> >>> 27-th
> >>> > > >> product
> >>> > > >> > >> > > >> description
> >>> > > >> > >> > > >> > // template values are 1 and 2,
> >>> respectively:
> >>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8, 26];
> >>> > > >> > GRIB2_ipdtmpl_val=[1,
> >>> > > >> > >> 2];
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > While I don't have any ECMWF data on hand
for
> >>> testing, I
> >>> > > do
> >>> > > >> > have
> >>> > > >> > >> a
> >>> > > >> > >> > > file
> >>> > > >> > >> > > >> > from the Air Force from which wgrib2 prints
a
> >>> similar
> >>> > > >> result:
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var discipline=0
> >>> center=57
> >>> > > >> > >> local_table=1
> >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > So I used that for testing. I ran the
following
> >>> > > >> plot_data_plane
> >>> > > >> > >> > > command
> >>> > > >> > >> > > >> > WITHOUT specifying a requested name:
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > plot_data_plane
GALWEM_20170610_CY00_FH18.GR2
> >>> plot.ps
> >>> > > >> > >> > 'GRIB2_disc=0;
> >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
GRIB2_parm_cat=1;
> >>> > > >> GRIB2_parm=192;
> >>> > > >> > >> > > >> level="Z0";'
> >>> > > >> > >> > > >> > -v 3
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > And then I looked closely at the log
message to
> >>> confirm
> >>> > > >> that it
> >>> > > >> > >> used
> >>> > > >> > >> > > >> > record 549, as expected:
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane() ->
Found
> >>> exact
> >>> > > match
> >>> > > >> > for
> >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of
GRIB2
> file
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >>
> >>> > > >> > >> > >
> >>> > > >> > >> >
> >>> > > >> > >>
> >>> > > >> >
> >>> > > >>
> >>> > >
> >>> >
> >>>
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I defined
to do a
> >>> table
> >>> > > >> lookup
> >>> > > >> > >> and
> >>> > > >> > >> > > find
> >>> > > >> > >> > > >> a
> >>> > > >> > >> > > >> > match in the MET GRIB tables:
> >>> > > >> > >> > > >> >
> >>> > > >> > >>
> >>> > >
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > And it determined that that field should be
called
> >>> > > "CTSTM".
> >>> > > >> > >> > Hopefully,
> >>> > > >> > >> > > >> the
> >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined
there in
> >>> MET. If
> >>> > > >> not, we
> >>> > > >> > >> can
> >>> > > >> > >> > > >> always
> >>> > > >> > >> > > >> > add them.
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > Please try running plot_data_plane at
verbosity
> >>> level 3
> >>> > on
> >>> > > >> one
> >>> > > >> > of
> >>> > > >> > >> > > these
> >>> > > >> > >> > > >> > files first. And make sure it runs without
error
> >>> and the
> >>> > > log
> >>> > > >> > >> message
> >>> > > >> > >> > > >> prints
> >>> > > >> > >> > > >> > out the "name" it found for that data. If
so, you
> >>> can
> >>> > just
> >>> > > >> skip
> >>> > > >> > >> > using
> >>> > > >> > >> > > >> all
> >>> > > >> > >> > > >> > those integers and revert back to using the
name
> MET
> >>> > > knows.
> >>> > > >> If
> >>> > > >> > >> not,
> >>> > > >> > >> > > >> then we
> >>> > > >> > >> > > >> > may need to define a custom GRIB table for
this
> >>> ECMWF
> >>> > > GRIB2
> >>> > > >> > data.
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >> > Thanks,
> >>> > > >> > >> > > >> > John
> >>> > > >> > >> > > >> >
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> --
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> Binbin Zhou
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> 5830 University Research Ct.
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> College Park, MD 20740
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> 301-683-3683
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >> --
> >>> > > >> > >> > > >>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>> > > >> > >> > > >> Tara Jensen
> >>> > > >> > >> > > >> Project Manager II
> >>> > > >> > >> > > >> NCAR RAL and DTC
> >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> >>> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> >>> > > >> > >> > > >>
> >>> > > >> > >> > > >
> >>> > > >> > >> > >
> >>> > > >> > >> > >
> >>> > > >> > >> >
> >>> > > >> > >> > --
> >>> > > >> > >> >
> >>> > > >> > >> > Binbin Zhou
> >>> > > >> > >> >
> >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> >>> > > >> > >> >
> >>> > > >> > >> > 5830 University Research Ct.
> >>> > > >> > >> >
> >>> > > >> > >> > College Park, MD 20740
> >>> > > >> > >> >
> >>> > > >> > >> > Binbin.Zhou at noaa.gov
> >>> > > >> > >> >
> >>> > > >> > >> > 301-683-3683
> >>> > > >> > >> >
> >>> > > >> > >> >
> >>> > > >> > >>
> >>> > > >> > >>
> >>> > > >> > >
> >>> > > >> > > --
> >>> > > >> > >
> >>> > > >> > > Binbin Zhou
> >>> > > >> > >
> >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> >>> > > >> > >
> >>> > > >> > > 5830 University Research Ct.
> >>> > > >> > >
> >>> > > >> > > College Park, MD 20740
> >>> > > >> > >
> >>> > > >> > > Binbin.Zhou at noaa.gov
> >>> > > >> > >
> >>> > > >> > > 301-683-3683
> >>> > > >> > >
> >>> > > >> >
> >>> > > >> >
> >>> > > >> > --
> >>> > > >> >
> >>> > > >> > Binbin Zhou
> >>> > > >> >
> >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> >>> > > >> >
> >>> > > >> > 5830 University Research Ct.
> >>> > > >> >
> >>> > > >> > College Park, MD 20740
> >>> > > >> >
> >>> > > >> > Binbin.Zhou at noaa.gov
> >>> > > >> >
> >>> > > >> > 301-683-3683
> >>> > > >> >
> >>> > > >> >
> >>> > > >>
> >>> > > >>
> >>> > > >
> >>> > > > --
> >>> > > >
> >>> > > > Binbin Zhou
> >>> > > >
> >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> >>> > > >
> >>> > > > 5830 University Research Ct.
> >>> > > >
> >>> > > > College Park, MD 20740
> >>> > > >
> >>> > > > Binbin.Zhou at noaa.gov
> >>> > > >
> >>> > > > 301-683-3683
> >>> > > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > >
> >>> > > Binbin Zhou
> >>> > >
> >>> > > IMSG at NOAA/NWS/NCEP/EMC
> >>> > >
> >>> > > 5830 University Research Ct.
> >>> > >
> >>> > > College Park, MD 20740
> >>> > >
> >>> > > Binbin.Zhou at noaa.gov
> >>> > >
> >>> > > 301-683-3683
> >>> > >
> >>> > >
> >>> >
> >>> >
> >>>
> >>> --
> >>>
> >>> Binbin Zhou
> >>>
> >>> IMSG at NOAA/NWS/NCEP/EMC
> >>>
> >>> 5830 University Research Ct.
> >>>
> >>> College Park, MD 20740
> >>>
> >>> Binbin.Zhou at noaa.gov
> >>>
> >>> 301-683-3683
> >>>
> >>>
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Julie Prestopnik
Time: Thu Dec 10 11:59:29 2020
Thanks, John. That worked great.
Binbin, this version of MET is now accessible to you by running the
following on venus:
module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
module load met/ens_config_test
Please let us know if you encounter any problems.
Julie
On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> Julie,
>
> Building the tarfile is very easy. I just ran this script from the
MET repo
> and specified the branch to be built:
> MET/scripts/met_checkout_and_build.sh branch
feature_1604_ens_config
>
> And that created a tarfile which I posted to RAL's anonymous ftp
server:
>
>
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
>
> I assume that'll work for you.
>
> Thanks,
> John
>
> On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > I am able to help with this install on WCOSS.
> >
> > Would it be easier to use git to checkout the repo on WCOSS? Or
would it
> be
> > > easier if I built a tarfile for this version of the code?
> > >
> > The last time I tried to build from git on WCOSS (a while ago), I
> > couldn't. I don't remember all of the details, but I think I
couldn't
> run
> > bootstrap and maybe it had to do with the version of autoconf
being too
> > old? I'm happy to try it again if you'd like though. Otherwise,
I know
> a
> > tarfile would work, but I know that is extra work for you.
> >
> > Julie
> >
> > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
> >
> > > Julie,
> > >
> > > I have created a bugfix branch for main_v9.1 named
> > > "feature_1604_ens_config" and it's up on GitHub.
> > >
> > > Only a single file has changed in it. But I need to get this
version
> > > compiled on WCOSS with which Binbin can test.
> > >
> > > Are you to help me with this? Would it be easier to use git to
checkout
> > > the repo on WCOSS? Or would it be easier if I built a tarfile
for this
> > > version of the code?
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> > >
> > >> Binbin,
> > >>
> > >> In order for you to test this change out, we'd need to
recompile a
> > >> version of the code on WCOSS. I wrote this up as a GitHub
issue:
> > >> https://github.com/dtcenter/MET/issues/1604
> > >>
> > >> And I'll create a branch from main_v9.1 named
"bugfix_1604_ens_config"
> > >> and commit the change there. Then we can checkout that code on
WCOSS,
> > >> compile it, and you can test it.
> > >>
> > >> Yes, you're right, if the fcst fields differ from the climo
fields,
> > you'd
> > >> need to specify them separately. That's pretty easy in the
> Ensemble-Stat
> > >> config file... but are you asking how to control that via
METplus?
> > >>
> > >> John
> > >>
> > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >>>
> > >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > >>>
> > >>> John,
> > >>>
> > >>> Great! Let me test it at first before make change in MET9.1
> > >>> Another question:
> > >>> Since climatology data is still in NCEP's grib2 format, In
the
> > >>> ensemble_stat
> > >>> config file, climo entry settings should be different from
those
> of
> > >>> fcst, obs entries?
> > >>> How to set climo (name, level, etc.)?
> > >>>
> > >>> Binbin
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> > >>> met_help at ucar.edu>
> > >>> wrote:
> > >>>
> > >>> > Binbin,
> > >>> >
> > >>> > Success! When I recompile with that one line commented out,
> > >>> plot_data_plane
> > >>> > is able to filter down all those records to the single one
that we
> > >>> want.
> > >>> > Note in the example below that I used the "lead_time" and
> "GRIB_ens"
> > >>> > options to filter the input records down.
> > >>> >
> > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
> > >>> level="P500";
> > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found
exact match
> > for
> > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > >>> "ensposte.t00z.z500hr".*
> > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1
GRIB
> records
> > >>> > matching VarInfo "HGT/P500" in GRIB file
"ensposte.t00z.z500hr".
> > >>> > DEBUG 1: Creating postscript file: plot.ps
> > >>> >
> > >>> > Also note that this selects out record number 26.
> > >>> >
> > >>> > Shall I make this change as a "bugfix" for met-9.1...which
could be
> > >>> > included in a met-9.1.2 bugfix release? Or shall I just
include it
> in
> > >>> > development for the next release (planned for March 2021)?
> > >>> >
> > >>> > Thanks,
> > >>> > John
> > >>> >
> > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT <
> > >>> > met_help at ucar.edu> wrote:
> > >>> >
> > >>> > >
> > >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >>> > >
> > >>> > > John,
> > >>> > >
> > >>> > > If this removing impacts other applications, or takes
some time
> > >>> > > to be effective, then won't try. I will use another
approach,
> such
> > >>> as
> > >>> > > borrow GEFS's grib2 data structure as tank to save EC
ensemble
> > >>> > > data.
> > >>> > >
> > >>> > > Binbin
> > >>> > >
> > >>> > >
> > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
Affiliate <
> > >>> > > binbin.zhou at noaa.gov> wrote:
> > >>> > >
> > >>> > > > John,
> > >>> > > >
> > >>> > > > Yes, please try to use this approach.
> > >>> > > >
> > >>> > > > Thanks!
> > >>> > > >
> > >>> > > > Binbin
> > >>> > > >
> > >>> > > > Binbin
> > >>> > > >
> > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via RT
<
> > >>> > > > met_help at ucar.edu> wrote:
> > >>> > > >
> > >>> > > >> Binbin,
> > >>> > > >>
> > >>> > > >> I used the "-PDS10" option for wgrib to dump those
records:
> > >>> > > >>
> > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > >>> > > >>
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > >>> > > >> | egrep "^11:|^26:|^43:"
> > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0
0 1 0
> 0
> > 1
> > >>> 0 0
> > >>> > 0
> > >>> > > 21
> > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0
0 1 0
> 0
> > 1
> > >>> 0 0
> > >>> > 0
> > >>> > > 21
> > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6 0
0 1 0
> 0
> > 1
> > >>> 0 0
> > >>> > 0
> > >>> > > 21
> > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > >>> > > >>
> > >>> > > >> Looking VERY closely, there is a difference in the 42-
nd
> column
> > >>> which
> > >>> > is
> > >>> > > >> the 4-th from the last... records 11, 26, and 43 have
values
> of
> > >>> 3, 1,
> > >>> > > and
> > >>> > > >> 2
> > >>> > > >> respectively.
> > >>> > > >>
> > >>> > > >> On this page (
> > >>> > >
https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > >>> > > >> ),
> > >>> > > >> byte 42 = 1 means "unperturbed control forecast".
> > >>> > > >>
> > >>> > > >> So we *should* be able to use the "GRIB_ens" config
option to
> > >>> > > >> differentiate
> > >>> > > >> between those records. BUT when I try, it doesn't work.
And
> > that's
> > >>> > > because
> > >>> > > >> of this line:
> > >>> > > >>
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > >>> > > >>
> > >>> > > >> We only apply those filtering options to GRIB files
created by
> > >>> NCEP.
> > >>> > If
> > >>> > > >> you'd like, I could try removing that condition to see
if we
> can
> > >>> pick
> > >>> > > out
> > >>> > > >> the single record of interest.
> > >>> > > >>
> > >>> > > >> John
> > >>> > > >>
> > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov via
RT <
> > >>> > > >> met_help at ucar.edu> wrote:
> > >>> > > >>
> > >>> > > >> >
> > >>> > > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > >>> > > >> >
> > >>> > > >> > John,
> > >>> > > >> >
> > >>> > > >> > Look at my
> > >>> > > >> > directory
> > >>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is only
for 500
> > >>> HGT.
> > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
anl:type=analysis
> > >>> > > >> > I get 3 records:
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >>> > > >> > mb:anl:type=analysis:NAve=0
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >>> > > >> > mb:anl:type=analysis:NAve=0
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >>> > > >> > mb:anl:type=analysis:NAve=0
> > >>> > > >> >
> > >>> > > >> > From its grib2 file, we know only record#26 is
analysis
> > low-res
> > >>> > > >> control.
> > >>> > > >> > If you have some way to determine which record is
analysis
> > >>> low-res
> > >>> > > >> control
> > >>> > > >> > from the grib1 file,
> > >>> > > >> > then we can use grib1 directly (don't consider the
size
> > issue).
> > >>> > > >> >
> > >>> > > >> > Binbin
> > >>> > > >> >
> > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
Affiliate
> <
> > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > >>> > > >> >
> > >>> > > >> > > John,
> > >>> > > >> > >
> > >>> > > >> > > Grib1 data is too large, another issue is the
low-res
> > >>> control
> > >>> > > >> analysis
> > >>> > > >> > > data can not be retrieved
> > >>> > > >> > > from the grib1 ecmwf data files. If using wgrib, we
can
> see
> > 3
> > >>> > > analysis
> > >>> > > >> > > fields, but don't know
> > >>> > > >> > > which is low-res control. After convert to grib2,
wgrib2
> can
> > >>> show
> > >>> > > >> which
> > >>> > > >> > > record is low-res control.
> > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set on
WCOSS.
> > >>> > > >> > >
> > >>> > > >> > > Binbin
> > >>> > > >> > >
> > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway
via RT <
> > >>> > > >> > > met_help at ucar.edu> wrote:
> > >>> > > >> > >
> > >>> > > >> > >> Binbin,
> > >>> > > >> > >>
> > >>> > > >> > >> Sounds like there's a lot of moving parts there.
How
> would
> > >>> you
> > >>> > like
> > >>> > > >> to
> > >>> > > >> > >> proceed?
> > >>> > > >> > >>
> > >>> > > >> > >> Should we check to see if MET can read those ECMWF
GRIB1
> > >>> files
> > >>> > > >> directly
> > >>> > > >> > >> without needing to reformat?
> > >>> > > >> > >> Or do you just want to create a custom GRIB table
for the
> > >>> result
> > >>> > of
> > >>> > > >> your
> > >>> > > >> > >> pre-processing and conversion steps?
> > >>> > > >> > >>
> > >>> > > >> > >> John
> > >>> > > >> > >>
> > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
Binbin.Zhou at noaa.gov via
> > RT <
> > >>> > > >> > >> met_help at ucar.edu> wrote:
> > >>> > > >> > >>
> > >>> > > >> > >> >
> > >>> > > >> > >> > <URL:
> > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >>> > >
> > >>> > > >> > >> >
> > >>> > > >> > >> > John,
> > >>> > > >> > >> >
> > >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
converted
> from
> > >>> its
> > >>> > > grib1
> > >>> > > >> > >> format
> > >>> > > >> > >> > data.
> > >>> > > >> > >> > Before conversion, the parameter names are
there, but
> > after
> > >>> > > >> > conversion,
> > >>> > > >> > >> the
> > >>> > > >> > >> > names
> > >>> > > >> > >> > are gone. It is because of unknown tables. The
grib1
> data
> > >>> is
> > >>> > > still
> > >>> > > >> > >> > original. The original
> > >>> > > >> > >> > grib1 data files are very strange data format we
can
> not
> > >>> read.
> > >>> > So
> > >>> > > >> we
> > >>> > > >> > >> have
> > >>> > > >> > >> > to use
> > >>> > > >> > >> > a Frotran code to re-arrange the fields and save
them
> as
> > >>> > > NCEP-like
> > >>> > > >> > grib1
> > >>> > > >> > >> > data files.
> > >>> > > >> > >> > So after several processes, some important info
like
> > tables
> > >>> > could
> > >>> > > >> be
> > >>> > > >> > >> lost.
> > >>> > > >> > >> >
> > >>> > > >> > >> > I'll test your first suggestion since the
field
> record
> > >>> order
> > >>> > > is
> > >>> > > >> > >> fixed.
> > >>> > > >> > >> > Currently we still
> > >>> > > >> > >> > obtained grib1 data from ECWMF, eventually, we
will get
> > >>> grib2
> > >>> > > data
> > >>> > > >> > >> > directly. So before
> > >>> > > >> > >> > we can get original grib2 files from ECMWF, we
just use
> > >>> this
> > >>> > > >> temporal
> > >>> > > >> > >> > solution.
> > >>> > > >> > >> > '
> > >>> > > >> > >> > Thanks!
> > >>> > > >> > >> >
> > >>> > > >> > >> >
> > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley
Gotway via
> RT
> > <
> > >>> > > >> > >> > met_help at ucar.edu>
> > >>> > > >> > >> > wrote:
> > >>> > > >> > >> >
> > >>> > > >> > >> > > Binbin,
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > OK, working on Venus.
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > You have a typo in the command you sent.
> > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 a.ps
> > >>> > > >> > >> > >
> > >>> > > >> > >>
> > >>> > > >>
> > >>> >
> > >>>
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >>> > > >> > >> > -v
> > >>> > > >> > >> > > 3
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
> > "GRIB2_parm_cat".
> > >>> > After
> > >>> > > >> > >> > correcting
> > >>> > > >> > >> > > that I did still get an error but it's a very
useful
> > one!
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > module use
> > >>> > > >> > >>
> > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > >>> > > >> > >> > > module load met/9.1
> > >>> > > >> > >> > > module load grib_util/1.1.1
> > >>> > > >> > >> > > cd
> > >>> > > >> > >> > >
> > >>> > > >> > >> > >
> > >>> > > >> > >> >
> > >>> > > >> > >>
> > >>> > > >> >
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > >>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/
> > plot.ps
> > >>> > > >> > >> > 'GRIB2_disc=255;
> > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3; GRIB2_parm=5;
> > >>> level="P500";'
> > >>> > > >> > >> > >
> > >>> > > >> > >> > >
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict() ->
no
> > >>> parameter
> > >>> > > found
> > >>> > > >> > with
> > >>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
> GRIB2_parm
> > >>> (5).
> > >>> > > Use
> > >>> > > >> the
> > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to define
custom
> > >>> GRIB
> > >>> > > >> > >> tables.ERROR
> > >>> > > >> > >> > :
> > >>> > > >> > >> > > *
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > So this table is not yet defined in MET. I
checked
> > their
> > >>> > > website:
> > >>> > > >> > >> > >
> > >>> > > >> > >> > >
> > >>> > > >> > >> >
> > >>> > > >> > >>
> > >>> > > >> >
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > But I don't see an entry defined for
GRIB2_parm=5;
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > 3 things...
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > (1) As a crude temporary solution, you could
refer to
> > it
> > >>> by
> > >>> > > it's
> > >>> > > >> > >> record
> > >>> > > >> > >> > > number. Let's assume you know that record 1
has
> > >>> temperature
> > >>> > > data:
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > plot_data_plane ecme.ens01.t00z.grd3.f12.grib2
~/
> > plot.ps
> > >>> > > >> > 'name="TMP";
> > >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > Here I'm reading record 1, naming it TMP and
the
> using
> > >>> > > >> > set_attr_level
> > >>> > > >> > >> > write
> > >>> > > >> > >> > > the output level as P500 instead of R1.
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > (2) You could create a custom GRIB table
yourself
> > >>> locally,
> > >>> > and
> > >>> > > >> then
> > >>> > > >> > >> set
> > >>> > > >> > >> > the
> > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to point
MET to
> > it.
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > (3) The best solution is contacting ECMWF and
> > requesting
> > >>> they
> > >>> > > >> send
> > >>> > > >> > >> over
> > >>> > > >> > >> > the
> > >>> > > >> > >> > > GRIB2 table we should be using to decode this
data.
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > Thanks,
> > >>> > > >> > >> > > John
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley
Gotway <
> > >>> > > >> johnhg at ucar.edu>
> > >>> > > >> > >> > wrote:
> > >>> > > >> > >> > >
> > >>> > > >> > >> > > > We might already have them. I'm not sure.
> > >>> > > >> > >> > > >
> > >>> > > >> > >> > > > But there's an additional required piece of
the
> > puzzle.
> > >>> > > Binbin
> > >>> > > >> > runs
> > >>> > > >> > >> > > > "wgrib2" to figure out what "name" he should
use in
> > >>> the MET
> > >>> > > >> config
> > >>> > > >> > >> > files.
> > >>> > > >> > >> > > > If wgrib2 doesn't tell him the right "name",
the
> > >>> problem
> > >>> > will
> > >>> > > >> > >> remain.
> > >>> > > >> > >> > > > wgrib2 can be configured to define
additional
> tables
> > as
> > >>> > well,
> > >>> > > >> but
> > >>> > > >> > >> for
> > >>> > > >> > >> > > this
> > >>> > > >> > >> > > > to be easy for Binbin, that'd mean updating
the
> > script
> > >>> > that's
> > >>> > > >> run
> > >>> > > >> > >> for
> > >>> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
guessing
> > >>> that's
> > >>> > > not
> > >>> > > >> > very
> > >>> > > >> > >> > easy.
> > >>> > > >> > >> > > >
> > >>> > > >> > >> > > > Yes, I do agree that the best solution is
getting
> the
> > >>> ECMWF
> > >>> > > >> GRIB
> > >>> > > >> > >> tables
> > >>> > > >> > >> > > > and adding them for both wgrib2 and MET.
> > >>> > > >> > >> > > >
> > >>> > > >> > >> > > > John
> > >>> > > >> > >> > > >
> > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen <
> > >>> > jensen at ucar.edu>
> > >>> > > >> > wrote:
> > >>> > > >> > >> > > >
> > >>> > > >> > >> > > >> John,
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> It seems to me that it would be worthwhile
to just
> > ask
> > >>> > ECMWF
> > >>> > > >> for
> > >>> > > >> > >> their
> > >>> > > >> > >> > > >> Grib2 Table for their Ensemble files. And
other
> > >>> > operational
> > >>> > > >> > >> centers
> > >>> > > >> > >> > for
> > >>> > > >> > >> > > >> that matter. Do you agree this would be a
good
> > >>> solution?
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> Thanks, Tara
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> ---------- Forwarded message ---------
> > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
> > met_help at ucar.edu>
> > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769]
urgent
> > >>> question for
> > >>> > > >> > reading
> > >>> > > >> > >> > ECMWF
> > >>> > > >> > >> > > >> ensmeble data
> > >>> > > >> > >> > > >> To:
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> <URL:
> > >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >>> > > >> >
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> John,
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> I tested using following command:
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > a.ps
> > >>> > > >> > >> > > >>
> > >>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >>> > > >> -v3
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> ERROR :
> > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name
> or
> > >>> > > >> GRIB2_disc,
> > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified in
> > >>> field
> > >>> > > >> > >> information
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > a.ps
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> >
> > >>> > > >> >
> > >>> > >
> > >>>
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >>> > > >> > >> > > >> -v
> > >>> > > >> > >> > > >> 3
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> Still got error
> > >>> > > >> > >> > > >> ERROR :
> > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() -> either
name
> or
> > >>> > > >> GRIB2_disc,
> > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified in
> > >>> field
> > >>> > > >> > >> information
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> Looks like discipline=255 is not recognized
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> Binbin
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John Halley
Gotway
> > >>> via RT
> > >>> > <
> > >>> > > >> > >> > > >> met_help at ucar.edu>
> > >>> > > >> > >> > > >> wrote:
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> > Binbin,
> > >>> > > >> > >> > > >> > You can do something similar with MET.
Please go
> > to
> > >>> this
> > >>> > > >> link:
> > >>> > > >> > >> > > >> >
> > >>> > > >>
> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and
you'll
> find
> > >>> the
> > >>> > > >> content
> > >>> > > >> > >> > > >> > copied/pasted below. You can specify
these
> INSTEAD
> > >>> OF
> > >>> > > >> > specifying
> > >>> > > >> > >> a
> > >>> > > >> > >> > > >> > requested name.
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an integer
> > >>> specifying the
> > >>> > > >> master
> > >>> > > >> > >> > table
> > >>> > > >> > >> > > >> > number.
> > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an integer
> > >>> specifying the
> > >>> > > >> local
> > >>> > > >> > >> table
> > >>> > > >> > >> > > >> > number.
> > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an integer
> > >>> specifying the
> > >>> > > >> GRIB2
> > >>> > > >> > >> > > >> discipline
> > >>> > > >> > >> > > >> > code.
> > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an
integer
> > >>> specifying
> > >>> > > the
> > >>> > > >> > >> > parameter
> > >>> > > >> > >> > > >> > category
> > >>> > > >> > >> > > >> > // code.
> > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an integer
> > >>> specifying the
> > >>> > > >> > >> parameter
> > >>> > > >> > >> > > code.
> > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an integer
> > specifying
> > >>> the
> > >>> > > >> product
> > >>> > > >> > >> > > >> definition
> > >>> > > >> > >> > > >> > // template (Table 4.0).
> > >>> > > >> > >> > > >> > // - The "GRIB2_process" is an
integer
> > >>> specifying
> > >>> > > the
> > >>> > > >> > >> > generating
> > >>> > > >> > >> > > >> > process
> > >>> > > >> > >> > > >> > // (Table 4.3).
> > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an integer
> > >>> specifying the
> > >>> > > >> > >> originating
> > >>> > > >> > >> > > >> > center.
> > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an
integer
> > >>> specifying
> > >>> > > the
> > >>> > > >> > >> > ensemble
> > >>> > > >> > >> > > >> type
> > >>> > > >> > >> > > >> > // (Table 4.6).
> > >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an
integer
> > >>> specifying
> > >>> > > the
> > >>> > > >> > >> derived
> > >>> > > >> > >> > > >> > product
> > >>> > > >> > >> > > >> > // type (Table 4.7).
> > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an
integer
> > >>> > specifying
> > >>> > > >> the
> > >>> > > >> > >> > > >> statistical
> > >>> > > >> > >> > > >> > // processing type (Table 4.10).
> > >>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index" and
> > >>> > > "GRIB2_ipdtmpl_val"
> > >>> > > >> > >> entries
> > >>> > > >> > >> > > are
> > >>> > > >> > >> > > >> > arrays
> > >>> > > >> > >> > > >> > // of integers which specify the
product
> > >>> > > description
> > >>> > > >> > >> > template
> > >>> > > >> > >> > > >> > values to
> > >>> > > >> > >> > > >> > // be used. The indices are 0-
based. For
> > >>> > example,
> > >>> > > >> use
> > >>> > > >> > the
> > >>> > > >> > >> > > >> > following to
> > >>> > > >> > >> > > >> > // request a GRIB2 record whose
9-th and
> > >>> 27-th
> > >>> > > >> product
> > >>> > > >> > >> > > >> description
> > >>> > > >> > >> > > >> > // template values are 1 and 2,
> > >>> respectively:
> > >>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8,
26];
> > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > >>> > > >> > >> 2];
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > While I don't have any ECMWF data on hand
for
> > >>> testing, I
> > >>> > > do
> > >>> > > >> > have
> > >>> > > >> > >> a
> > >>> > > >> > >> > > file
> > >>> > > >> > >> > > >> > from the Air Force from which wgrib2
prints a
> > >>> similar
> > >>> > > >> result:
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
discipline=0
> > >>> center=57
> > >>> > > >> > >> local_table=1
> > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour fcst:
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > So I used that for testing. I ran the
following
> > >>> > > >> plot_data_plane
> > >>> > > >> > >> > > command
> > >>> > > >> > >> > > >> > WITHOUT specifying a requested name:
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > plot_data_plane
GALWEM_20170610_CY00_FH18.GR2
> > >>> plot.ps
> > >>> > > >> > >> > 'GRIB2_disc=0;
> > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
GRIB2_parm_cat=1;
> > >>> > > >> GRIB2_parm=192;
> > >>> > > >> > >> > > >> level="Z0";'
> > >>> > > >> > >> > > >> > -v 3
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > And then I looked closely at the log
message to
> > >>> confirm
> > >>> > > >> that it
> > >>> > > >> > >> used
> > >>> > > >> > >> > > >> > record 549, as expected:
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane()
-> Found
> > >>> exact
> > >>> > > match
> > >>> > > >> > for
> > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1 of
GRIB2
> > file
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > >
> > >>> > > >> > >> >
> > >>> > > >> > >>
> > >>> > > >> >
> > >>> > > >>
> > >>> > >
> > >>> >
> > >>>
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I defined
to do
> a
> > >>> table
> > >>> > > >> lookup
> > >>> > > >> > >> and
> > >>> > > >> > >> > > find
> > >>> > > >> > >> > > >> a
> > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >>
> > >>> > >
> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > And it determined that that field should
be
> called
> > >>> > > "CTSTM".
> > >>> > > >> > >> > Hopefully,
> > >>> > > >> > >> > > >> the
> > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined
there in
> > >>> MET. If
> > >>> > > >> not, we
> > >>> > > >> > >> can
> > >>> > > >> > >> > > >> always
> > >>> > > >> > >> > > >> > add them.
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > Please try running plot_data_plane at
verbosity
> > >>> level 3
> > >>> > on
> > >>> > > >> one
> > >>> > > >> > of
> > >>> > > >> > >> > > these
> > >>> > > >> > >> > > >> > files first. And make sure it runs
without error
> > >>> and the
> > >>> > > log
> > >>> > > >> > >> message
> > >>> > > >> > >> > > >> prints
> > >>> > > >> > >> > > >> > out the "name" it found for that data. If
so,
> you
> > >>> can
> > >>> > just
> > >>> > > >> skip
> > >>> > > >> > >> > using
> > >>> > > >> > >> > > >> all
> > >>> > > >> > >> > > >> > those integers and revert back to using
the name
> > MET
> > >>> > > knows.
> > >>> > > >> If
> > >>> > > >> > >> not,
> > >>> > > >> > >> > > >> then we
> > >>> > > >> > >> > > >> > may need to define a custom GRIB table
for this
> > >>> ECMWF
> > >>> > > GRIB2
> > >>> > > >> > data.
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >> > Thanks,
> > >>> > > >> > >> > > >> > John
> > >>> > > >> > >> > > >> >
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> --
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> Binbin Zhou
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> 5830 University Research Ct.
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> College Park, MD 20740
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> 301-683-3683
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >> --
> > >>> > > >> > >> > > >>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >>> > > >> > >> > > >> Tara Jensen
> > >>> > > >> > >> > > >> Project Manager II
> > >>> > > >> > >> > > >> NCAR RAL and DTC
> > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > >>> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > >>> > > >> > >> > > >>
> > >>> > > >> > >> > > >
> > >>> > > >> > >> > >
> > >>> > > >> > >> > >
> > >>> > > >> > >> >
> > >>> > > >> > >> > --
> > >>> > > >> > >> >
> > >>> > > >> > >> > Binbin Zhou
> > >>> > > >> > >> >
> > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >>> > > >> > >> >
> > >>> > > >> > >> > 5830 University Research Ct.
> > >>> > > >> > >> >
> > >>> > > >> > >> > College Park, MD 20740
> > >>> > > >> > >> >
> > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > >>> > > >> > >> >
> > >>> > > >> > >> > 301-683-3683
> > >>> > > >> > >> >
> > >>> > > >> > >> >
> > >>> > > >> > >>
> > >>> > > >> > >>
> > >>> > > >> > >
> > >>> > > >> > > --
> > >>> > > >> > >
> > >>> > > >> > > Binbin Zhou
> > >>> > > >> > >
> > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > >>> > > >> > >
> > >>> > > >> > > 5830 University Research Ct.
> > >>> > > >> > >
> > >>> > > >> > > College Park, MD 20740
> > >>> > > >> > >
> > >>> > > >> > > Binbin.Zhou at noaa.gov
> > >>> > > >> > >
> > >>> > > >> > > 301-683-3683
> > >>> > > >> > >
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >> > --
> > >>> > > >> >
> > >>> > > >> > Binbin Zhou
> > >>> > > >> >
> > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >>> > > >> >
> > >>> > > >> > 5830 University Research Ct.
> > >>> > > >> >
> > >>> > > >> > College Park, MD 20740
> > >>> > > >> >
> > >>> > > >> > Binbin.Zhou at noaa.gov
> > >>> > > >> >
> > >>> > > >> > 301-683-3683
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >>
> > >>> > > >>
> > >>> > > >
> > >>> > > > --
> > >>> > > >
> > >>> > > > Binbin Zhou
> > >>> > > >
> > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > >>> > > >
> > >>> > > > 5830 University Research Ct.
> > >>> > > >
> > >>> > > > College Park, MD 20740
> > >>> > > >
> > >>> > > > Binbin.Zhou at noaa.gov
> > >>> > > >
> > >>> > > > 301-683-3683
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>> > > --
> > >>> > >
> > >>> > > Binbin Zhou
> > >>> > >
> > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > >>> > >
> > >>> > > 5830 University Research Ct.
> > >>> > >
> > >>> > > College Park, MD 20740
> > >>> > >
> > >>> > > Binbin.Zhou at noaa.gov
> > >>> > >
> > >>> > > 301-683-3683
> > >>> > >
> > >>> > >
> > >>> >
> > >>> >
> > >>>
> > >>> --
> > >>>
> > >>> Binbin Zhou
> > >>>
> > >>> IMSG at NOAA/NWS/NCEP/EMC
> > >>>
> > >>> 5830 University Research Ct.
> > >>>
> > >>> College Park, MD 20740
> > >>>
> > >>> Binbin.Zhou at noaa.gov
> > >>>
> > >>> 301-683-3683
> > >>>
> > >>>
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Thu Dec 10 12:05:57 2020
Thanks Julie.
And Binbin, here's an EnsembleStatConfig file where climos are used:
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
Since the list of forecast fields is exactly the same as the climo
fields,
this config file just copies them over using:
climo_mean = fcst;
In your case, when they differ, you'd just specify the "field" array
inside
the "climo_mean" dictionary. For example:
climo_mean = {
field = [
{ name = "TMP"; level = "Z2"; },
{ name = "TMP"; level = "P850"; }
];
...
}
On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> Thanks, John. That worked great.
>
> Binbin, this version of MET is now accessible to you by running the
> following on venus:
>
> module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> module load met/ens_config_test
>
> Please let us know if you encounter any problems.
>
> Julie
>
> On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > Julie,
> >
> > Building the tarfile is very easy. I just ran this script from the
MET
> repo
> > and specified the branch to be built:
> > MET/scripts/met_checkout_and_build.sh branch
feature_1604_ens_config
> >
> > And that created a tarfile which I posted to RAL's anonymous ftp
server:
> >
> >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> >
> > I assume that'll work for you.
> >
> > Thanks,
> > John
> >
> > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >
> > > John,
> > >
> > > I am able to help with this install on WCOSS.
> > >
> > > Would it be easier to use git to checkout the repo on WCOSS? Or
would
> it
> > be
> > > > easier if I built a tarfile for this version of the code?
> > > >
> > > The last time I tried to build from git on WCOSS (a while ago),
I
> > > couldn't. I don't remember all of the details, but I think I
couldn't
> > run
> > > bootstrap and maybe it had to do with the version of autoconf
being too
> > > old? I'm happy to try it again if you'd like though.
Otherwise, I
> know
> > a
> > > tarfile would work, but I know that is extra work for you.
> > >
> > > Julie
> > >
> > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> > >
> > > > Julie,
> > > >
> > > > I have created a bugfix branch for main_v9.1 named
> > > > "feature_1604_ens_config" and it's up on GitHub.
> > > >
> > > > Only a single file has changed in it. But I need to get this
version
> > > > compiled on WCOSS with which Binbin can test.
> > > >
> > > > Are you to help me with this? Would it be easier to use git to
> checkout
> > > > the repo on WCOSS? Or would it be easier if I built a tarfile
for
> this
> > > > version of the code?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > > >
> > > >> Binbin,
> > > >>
> > > >> In order for you to test this change out, we'd need to
recompile a
> > > >> version of the code on WCOSS. I wrote this up as a GitHub
issue:
> > > >> https://github.com/dtcenter/MET/issues/1604
> > > >>
> > > >> And I'll create a branch from main_v9.1 named
> "bugfix_1604_ens_config"
> > > >> and commit the change there. Then we can checkout that code
on
> WCOSS,
> > > >> compile it, and you can test it.
> > > >>
> > > >> Yes, you're right, if the fcst fields differ from the climo
fields,
> > > you'd
> > > >> need to specify them separately. That's pretty easy in the
> > Ensemble-Stat
> > > >> config file... but are you asking how to control that via
METplus?
> > > >>
> > > >> John
> > > >>
> > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >>>
> > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > >>>
> > > >>> John,
> > > >>>
> > > >>> Great! Let me test it at first before make change in
MET9.1
> > > >>> Another question:
> > > >>> Since climatology data is still in NCEP's grib2 format,
In the
> > > >>> ensemble_stat
> > > >>> config file, climo entry settings should be different
from those
> > of
> > > >>> fcst, obs entries?
> > > >>> How to set climo (name, level, etc.)?
> > > >>>
> > > >>> Binbin
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> > > >>> met_help at ucar.edu>
> > > >>> wrote:
> > > >>>
> > > >>> > Binbin,
> > > >>> >
> > > >>> > Success! When I recompile with that one line commented
out,
> > > >>> plot_data_plane
> > > >>> > is able to filter down all those records to the single one
that
> we
> > > >>> want.
> > > >>> > Note in the example below that I used the "lead_time" and
> > "GRIB_ens"
> > > >>> > options to filter the input records down.
> > > >>> >
> > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
> > > >>> level="P500";
> > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found
exact
> match
> > > for
> > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > > >>> "ensposte.t00z.z500hr".*
> > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1
GRIB
> > records
> > > >>> > matching VarInfo "HGT/P500" in GRIB file
"ensposte.t00z.z500hr".
> > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > >>> >
> > > >>> > Also note that this selects out record number 26.
> > > >>> >
> > > >>> > Shall I make this change as a "bugfix" for met-9.1...which
could
> be
> > > >>> > included in a met-9.1.2 bugfix release? Or shall I just
include
> it
> > in
> > > >>> > development for the next release (planned for March 2021)?
> > > >>> >
> > > >>> > Thanks,
> > > >>> > John
> > > >>> >
> > > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT
<
> > > >>> > met_help at ucar.edu> wrote:
> > > >>> >
> > > >>> > >
> > > >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > > >>> > >
> > > >>> > > John,
> > > >>> > >
> > > >>> > > If this removing impacts other applications, or takes
some
> time
> > > >>> > > to be effective, then won't try. I will use another
approach,
> > such
> > > >>> as
> > > >>> > > borrow GEFS's grib2 data structure as tank to save EC
ensemble
> > > >>> > > data.
> > > >>> > >
> > > >>> > > Binbin
> > > >>> > >
> > > >>> > >
> > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
Affiliate <
> > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > >>> > >
> > > >>> > > > John,
> > > >>> > > >
> > > >>> > > > Yes, please try to use this approach.
> > > >>> > > >
> > > >>> > > > Thanks!
> > > >>> > > >
> > > >>> > > > Binbin
> > > >>> > > >
> > > >>> > > > Binbin
> > > >>> > > >
> > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via
RT <
> > > >>> > > > met_help at ucar.edu> wrote:
> > > >>> > > >
> > > >>> > > >> Binbin,
> > > >>> > > >>
> > > >>> > > >> I used the "-PDS10" option for wgrib to dump those
records:
> > > >>> > > >>
> > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6
0 0 1
> 0
> > 0
> > > 1
> > > >>> 0 0
> > > >>> > 0
> > > >>> > > 21
> > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6
0 0 1
> 0
> > 0
> > > 1
> > > >>> 0 0
> > > >>> > 0
> > > >>> > > 21
> > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6
0 0 1
> 0
> > 0
> > > 1
> > > >>> 0 0
> > > >>> > 0
> > > >>> > > 21
> > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > >>> > > >>
> > > >>> > > >> Looking VERY closely, there is a difference in the
42-nd
> > column
> > > >>> which
> > > >>> > is
> > > >>> > > >> the 4-th from the last... records 11, 26, and 43 have
values
> > of
> > > >>> 3, 1,
> > > >>> > > and
> > > >>> > > >> 2
> > > >>> > > >> respectively.
> > > >>> > > >>
> > > >>> > > >> On this page (
> > > >>> > >
https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > >>> > > >> ),
> > > >>> > > >> byte 42 = 1 means "unperturbed control forecast".
> > > >>> > > >>
> > > >>> > > >> So we *should* be able to use the "GRIB_ens" config
option
> to
> > > >>> > > >> differentiate
> > > >>> > > >> between those records. BUT when I try, it doesn't
work. And
> > > that's
> > > >>> > > because
> > > >>> > > >> of this line:
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > >>> > > >>
> > > >>> > > >> We only apply those filtering options to GRIB files
created
> by
> > > >>> NCEP.
> > > >>> > If
> > > >>> > > >> you'd like, I could try removing that condition to
see if we
> > can
> > > >>> pick
> > > >>> > > out
> > > >>> > > >> the single record of interest.
> > > >>> > > >>
> > > >>> > > >> John
> > > >>> > > >>
> > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov
via RT
> <
> > > >>> > > >> met_help at ucar.edu> wrote:
> > > >>> > > >>
> > > >>> > > >> >
> > > >>> > > >> > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >
> > > >>> > > >> >
> > > >>> > > >> > John,
> > > >>> > > >> >
> > > >>> > > >> > Look at my
> > > >>> > > >> > directory
> > > >>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is
only for
> 500
> > > >>> HGT.
> > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
anl:type=analysis
> > > >>> > > >> > I get 3 records:
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >>> > > >> >
> > > >>> > > >> > From its grib2 file, we know only record#26 is
analysis
> > > low-res
> > > >>> > > >> control.
> > > >>> > > >> > If you have some way to determine which record is
analysis
> > > >>> low-res
> > > >>> > > >> control
> > > >>> > > >> > from the grib1 file,
> > > >>> > > >> > then we can use grib1 directly (don't consider the
size
> > > issue).
> > > >>> > > >> >
> > > >>> > > >> > Binbin
> > > >>> > > >> >
> > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
> Affiliate
> > <
> > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > >>> > > >> >
> > > >>> > > >> > > John,
> > > >>> > > >> > >
> > > >>> > > >> > > Grib1 data is too large, another issue is the
low-res
> > > >>> control
> > > >>> > > >> analysis
> > > >>> > > >> > > data can not be retrieved
> > > >>> > > >> > > from the grib1 ecmwf data files. If using wgrib,
we can
> > see
> > > 3
> > > >>> > > analysis
> > > >>> > > >> > > fields, but don't know
> > > >>> > > >> > > which is low-res control. After convert to grib2,
wgrib2
> > can
> > > >>> show
> > > >>> > > >> which
> > > >>> > > >> > > record is low-res control.
> > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set
on
> WCOSS.
> > > >>> > > >> > >
> > > >>> > > >> > > Binbin
> > > >>> > > >> > >
> > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway
via
> RT <
> > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > >>> > > >> > >
> > > >>> > > >> > >> Binbin,
> > > >>> > > >> > >>
> > > >>> > > >> > >> Sounds like there's a lot of moving parts there.
How
> > would
> > > >>> you
> > > >>> > like
> > > >>> > > >> to
> > > >>> > > >> > >> proceed?
> > > >>> > > >> > >>
> > > >>> > > >> > >> Should we check to see if MET can read those
ECMWF
> GRIB1
> > > >>> files
> > > >>> > > >> directly
> > > >>> > > >> > >> without needing to reformat?
> > > >>> > > >> > >> Or do you just want to create a custom GRIB
table for
> the
> > > >>> result
> > > >>> > of
> > > >>> > > >> your
> > > >>> > > >> > >> pre-processing and conversion steps?
> > > >>> > > >> > >>
> > > >>> > > >> > >> John
> > > >>> > > >> > >>
> > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
Binbin.Zhou at noaa.gov
> via
> > > RT <
> > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > >>> > > >> > >>
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > <URL:
> > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >>> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > John,
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
converted
> > from
> > > >>> its
> > > >>> > > grib1
> > > >>> > > >> > >> format
> > > >>> > > >> > >> > data.
> > > >>> > > >> > >> > Before conversion, the parameter names are
there, but
> > > after
> > > >>> > > >> > conversion,
> > > >>> > > >> > >> the
> > > >>> > > >> > >> > names
> > > >>> > > >> > >> > are gone. It is because of unknown tables. The
grib1
> > data
> > > >>> is
> > > >>> > > still
> > > >>> > > >> > >> > original. The original
> > > >>> > > >> > >> > grib1 data files are very strange data format
we can
> > not
> > > >>> read.
> > > >>> > So
> > > >>> > > >> we
> > > >>> > > >> > >> have
> > > >>> > > >> > >> > to use
> > > >>> > > >> > >> > a Frotran code to re-arrange the fields and
save them
> > as
> > > >>> > > NCEP-like
> > > >>> > > >> > grib1
> > > >>> > > >> > >> > data files.
> > > >>> > > >> > >> > So after several processes, some important
info like
> > > tables
> > > >>> > could
> > > >>> > > >> be
> > > >>> > > >> > >> lost.
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > I'll test your first suggestion since the
field
> > record
> > > >>> order
> > > >>> > > is
> > > >>> > > >> > >> fixed.
> > > >>> > > >> > >> > Currently we still
> > > >>> > > >> > >> > obtained grib1 data from ECWMF, eventually, we
will
> get
> > > >>> grib2
> > > >>> > > data
> > > >>> > > >> > >> > directly. So before
> > > >>> > > >> > >> > we can get original grib2 files from ECMWF, we
just
> use
> > > >>> this
> > > >>> > > >> temporal
> > > >>> > > >> > >> > solution.
> > > >>> > > >> > >> > '
> > > >>> > > >> > >> > Thanks!
> > > >>> > > >> > >> >
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley
Gotway via
> > RT
> > > <
> > > >>> > > >> > >> > met_help at ucar.edu>
> > > >>> > > >> > >> > wrote:
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > > Binbin,
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > OK, working on Venus.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > You have a typo in the command you sent.
> > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> a.ps
> > > >>> > > >> > >> > >
> > > >>> > > >> > >>
> > > >>> > > >>
> > > >>> >
> > > >>>
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >>> > > >> > >> > -v
> > > >>> > > >> > >> > > 3
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
> > > "GRIB2_parm_cat".
> > > >>> > After
> > > >>> > > >> > >> > correcting
> > > >>> > > >> > >> > > that I did still get an error but it's a
very
> useful
> > > one!
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > module use
> > > >>> > > >> > >>
> > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > >>> > > >> > >> > > module load met/9.1
> > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > >>> > > >> > >> > > cd
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 ~/
> > > plot.ps
> > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
GRIB2_parm=5;
> > > >>> level="P500";'
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict()
-> no
> > > >>> parameter
> > > >>> > > found
> > > >>> > > >> > with
> > > >>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
> > GRIB2_parm
> > > >>> (5).
> > > >>> > > Use
> > > >>> > > >> the
> > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
define
> custom
> > > >>> GRIB
> > > >>> > > >> > >> tables.ERROR
> > > >>> > > >> > >> > :
> > > >>> > > >> > >> > > *
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > So this table is not yet defined in MET. I
checked
> > > their
> > > >>> > > website:
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > But I don't see an entry defined for
GRIB2_parm=5;
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > 3 things...
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > (1) As a crude temporary solution, you could
refer
> to
> > > it
> > > >>> by
> > > >>> > > it's
> > > >>> > > >> > >> record
> > > >>> > > >> > >> > > number. Let's assume you know that record 1
has
> > > >>> temperature
> > > >>> > > data:
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 ~/
> > > plot.ps
> > > >>> > > >> > 'name="TMP";
> > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > Here I'm reading record 1, naming it TMP and
the
> > using
> > > >>> > > >> > set_attr_level
> > > >>> > > >> > >> > write
> > > >>> > > >> > >> > > the output level as P500 instead of R1.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > (2) You could create a custom GRIB table
yourself
> > > >>> locally,
> > > >>> > and
> > > >>> > > >> then
> > > >>> > > >> > >> set
> > > >>> > > >> > >> > the
> > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
point MET
> to
> > > it.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > (3) The best solution is contacting ECMWF
and
> > > requesting
> > > >>> they
> > > >>> > > >> send
> > > >>> > > >> > >> over
> > > >>> > > >> > >> > the
> > > >>> > > >> > >> > > GRIB2 table we should be using to decode
this data.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > Thanks,
> > > >>> > > >> > >> > > John
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley
Gotway <
> > > >>> > > >> johnhg at ucar.edu>
> > > >>> > > >> > >> > wrote:
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > > We might already have them. I'm not sure.
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > But there's an additional required piece
of the
> > > puzzle.
> > > >>> > > Binbin
> > > >>> > > >> > runs
> > > >>> > > >> > >> > > > "wgrib2" to figure out what "name" he
should use
> in
> > > >>> the MET
> > > >>> > > >> config
> > > >>> > > >> > >> > files.
> > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the right
"name", the
> > > >>> problem
> > > >>> > will
> > > >>> > > >> > >> remain.
> > > >>> > > >> > >> > > > wgrib2 can be configured to define
additional
> > tables
> > > as
> > > >>> > well,
> > > >>> > > >> but
> > > >>> > > >> > >> for
> > > >>> > > >> > >> > > this
> > > >>> > > >> > >> > > > to be easy for Binbin, that'd mean
updating the
> > > script
> > > >>> > that's
> > > >>> > > >> run
> > > >>> > > >> > >> for
> > > >>> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
> guessing
> > > >>> that's
> > > >>> > > not
> > > >>> > > >> > very
> > > >>> > > >> > >> > easy.
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > Yes, I do agree that the best solution is
getting
> > the
> > > >>> ECMWF
> > > >>> > > >> GRIB
> > > >>> > > >> > >> tables
> > > >>> > > >> > >> > > > and adding them for both wgrib2 and MET.
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > John
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen
<
> > > >>> > jensen at ucar.edu>
> > > >>> > > >> > wrote:
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > >> John,
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> It seems to me that it would be
worthwhile to
> just
> > > ask
> > > >>> > ECMWF
> > > >>> > > >> for
> > > >>> > > >> > >> their
> > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble files.
And other
> > > >>> > operational
> > > >>> > > >> > >> centers
> > > >>> > > >> > >> > for
> > > >>> > > >> > >> > > >> that matter. Do you agree this would be
a good
> > > >>> solution?
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Thanks, Tara
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> ---------- Forwarded message ---------
> > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
> > > met_help at ucar.edu>
> > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769]
urgent
> > > >>> question for
> > > >>> > > >> > reading
> > > >>> > > >> > >> > ECMWF
> > > >>> > > >> > >> > > >> ensmeble data
> > > >>> > > >> > >> > > >> To:
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> <URL:
> > > >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >>> > > >> >
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> John,
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> I tested using following command:
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > > a.ps
> > > >>> > > >> > >> > > >>
> > > >>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >>> > > >> -v3
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> ERROR :
> > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either name
> > or
> > > >>> > > >> GRIB2_disc,
> > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified
> in
> > > >>> field
> > > >>> > > >> > >> information
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > > a.ps
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> >
> > > >>> > > >> >
> > > >>> > >
> > > >>>
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >>> > > >> > >> > > >> -v
> > > >>> > > >> > >> > > >> 3
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Still got error
> > > >>> > > >> > >> > > >> ERROR :
> > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either name
> > or
> > > >>> > > >> GRIB2_disc,
> > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified
> in
> > > >>> field
> > > >>> > > >> > >> information
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Looks like discipline=255 is not
recognized
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Binbin
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John
Halley
> Gotway
> > > >>> via RT
> > > >>> > <
> > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > >>> > > >> > >> > > >> wrote:
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> > Binbin,
> > > >>> > > >> > >> > > >> > You can do something similar with MET.
Please
> go
> > > to
> > > >>> this
> > > >>> > > >> link:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >>
> > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and
you'll
> > find
> > > >>> the
> > > >>> > > >> content
> > > >>> > > >> > >> > > >> > copied/pasted below. You can specify
these
> > INSTEAD
> > > >>> OF
> > > >>> > > >> > specifying
> > > >>> > > >> > >> a
> > > >>> > > >> > >> > > >> > requested name.
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an
integer
> > > >>> specifying the
> > > >>> > > >> master
> > > >>> > > >> > >> > table
> > > >>> > > >> > >> > > >> > number.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an
integer
> > > >>> specifying the
> > > >>> > > >> local
> > > >>> > > >> > >> table
> > > >>> > > >> > >> > > >> > number.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an
integer
> > > >>> specifying the
> > > >>> > > >> GRIB2
> > > >>> > > >> > >> > > >> discipline
> > > >>> > > >> > >> > > >> > code.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> > parameter
> > > >>> > > >> > >> > > >> > category
> > > >>> > > >> > >> > > >> > // code.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an
integer
> > > >>> specifying the
> > > >>> > > >> > >> parameter
> > > >>> > > >> > >> > > code.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an
integer
> > > specifying
> > > >>> the
> > > >>> > > >> product
> > > >>> > > >> > >> > > >> definition
> > > >>> > > >> > >> > > >> > // template (Table 4.0).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_process" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> > generating
> > > >>> > > >> > >> > > >> > process
> > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an
integer
> > > >>> specifying the
> > > >>> > > >> > >> originating
> > > >>> > > >> > >> > > >> > center.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> > ensemble
> > > >>> > > >> > >> > > >> type
> > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> derived
> > > >>> > > >> > >> > > >> > product
> > > >>> > > >> > >> > > >> > // type (Table 4.7).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an
integer
> > > >>> > specifying
> > > >>> > > >> the
> > > >>> > > >> > >> > > >> statistical
> > > >>> > > >> > >> > > >> > // processing type (Table
4.10).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index"
and
> > > >>> > > "GRIB2_ipdtmpl_val"
> > > >>> > > >> > >> entries
> > > >>> > > >> > >> > > are
> > > >>> > > >> > >> > > >> > arrays
> > > >>> > > >> > >> > > >> > // of integers which specify
the
> product
> > > >>> > > description
> > > >>> > > >> > >> > template
> > > >>> > > >> > >> > > >> > values to
> > > >>> > > >> > >> > > >> > // be used. The indices are 0-
based.
> For
> > > >>> > example,
> > > >>> > > >> use
> > > >>> > > >> > the
> > > >>> > > >> > >> > > >> > following to
> > > >>> > > >> > >> > > >> > // request a GRIB2 record whose
9-th
> and
> > > >>> 27-th
> > > >>> > > >> product
> > > >>> > > >> > >> > > >> description
> > > >>> > > >> > >> > > >> > // template values are 1 and 2,
> > > >>> respectively:
> > > >>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8,
26];
> > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > >>> > > >> > >> 2];
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > While I don't have any ECMWF data on
hand for
> > > >>> testing, I
> > > >>> > > do
> > > >>> > > >> > have
> > > >>> > > >> > >> a
> > > >>> > > >> > >> > > file
> > > >>> > > >> > >> > > >> > from the Air Force from which wgrib2
prints a
> > > >>> similar
> > > >>> > > >> result:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
discipline=0
> > > >>> center=57
> > > >>> > > >> > >> local_table=1
> > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour
fcst:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > So I used that for testing. I ran the
> following
> > > >>> > > >> plot_data_plane
> > > >>> > > >> > >> > > command
> > > >>> > > >> > >> > > >> > WITHOUT specifying a requested name:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > plot_data_plane
GALWEM_20170610_CY00_FH18.GR2
> > > >>> plot.ps
> > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
GRIB2_parm_cat=1;
> > > >>> > > >> GRIB2_parm=192;
> > > >>> > > >> > >> > > >> level="Z0";'
> > > >>> > > >> > >> > > >> > -v 3
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > And then I looked closely at the log
message
> to
> > > >>> confirm
> > > >>> > > >> that it
> > > >>> > > >> > >> used
> > > >>> > > >> > >> > > >> > record 549, as expected:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane()
->
> Found
> > > >>> exact
> > > >>> > > match
> > > >>> > > >> > for
> > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1
of
> GRIB2
> > > file
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I
defined to
> do
> > a
> > > >>> table
> > > >>> > > >> lookup
> > > >>> > > >> > >> and
> > > >>> > > >> > >> > > find
> > > >>> > > >> > >> > > >> a
> > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >>
> > > >>> > >
> > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > And it determined that that field
should be
> > called
> > > >>> > > "CTSTM".
> > > >>> > > >> > >> > Hopefully,
> > > >>> > > >> > >> > > >> the
> > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined
there
> in
> > > >>> MET. If
> > > >>> > > >> not, we
> > > >>> > > >> > >> can
> > > >>> > > >> > >> > > >> always
> > > >>> > > >> > >> > > >> > add them.
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > Please try running plot_data_plane at
> verbosity
> > > >>> level 3
> > > >>> > on
> > > >>> > > >> one
> > > >>> > > >> > of
> > > >>> > > >> > >> > > these
> > > >>> > > >> > >> > > >> > files first. And make sure it runs
without
> error
> > > >>> and the
> > > >>> > > log
> > > >>> > > >> > >> message
> > > >>> > > >> > >> > > >> prints
> > > >>> > > >> > >> > > >> > out the "name" it found for that data.
If so,
> > you
> > > >>> can
> > > >>> > just
> > > >>> > > >> skip
> > > >>> > > >> > >> > using
> > > >>> > > >> > >> > > >> all
> > > >>> > > >> > >> > > >> > those integers and revert back to using
the
> name
> > > MET
> > > >>> > > knows.
> > > >>> > > >> If
> > > >>> > > >> > >> not,
> > > >>> > > >> > >> > > >> then we
> > > >>> > > >> > >> > > >> > may need to define a custom GRIB table
for
> this
> > > >>> ECMWF
> > > >>> > > GRIB2
> > > >>> > > >> > data.
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > Thanks,
> > > >>> > > >> > >> > > >> > John
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> --
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Binbin Zhou
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> College Park, MD 20740
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> 301-683-3683
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> --
> > > >>> > > >> > >> > > >>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >>> > > >> > >> > > >> Tara Jensen
> > > >>> > > >> > >> > > >> Project Manager II
> > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > > >>> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > --
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > Binbin Zhou
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > 5830 University Research Ct.
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > College Park, MD 20740
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > 301-683-3683
> > > >>> > > >> > >> >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> > >>
> > > >>> > > >> > >
> > > >>> > > >> > > --
> > > >>> > > >> > >
> > > >>> > > >> > > Binbin Zhou
> > > >>> > > >> > >
> > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> > >
> > > >>> > > >> > > 5830 University Research Ct.
> > > >>> > > >> > >
> > > >>> > > >> > > College Park, MD 20740
> > > >>> > > >> > >
> > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > >>> > > >> > >
> > > >>> > > >> > > 301-683-3683
> > > >>> > > >> > >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> > --
> > > >>> > > >> >
> > > >>> > > >> > Binbin Zhou
> > > >>> > > >> >
> > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> >
> > > >>> > > >> > 5830 University Research Ct.
> > > >>> > > >> >
> > > >>> > > >> > College Park, MD 20740
> > > >>> > > >> >
> > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > >>> > > >> >
> > > >>> > > >> > 301-683-3683
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > >
> > > >>> > > > Binbin Zhou
> > > >>> > > >
> > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >
> > > >>> > > > 5830 University Research Ct.
> > > >>> > > >
> > > >>> > > > College Park, MD 20740
> > > >>> > > >
> > > >>> > > > Binbin.Zhou at noaa.gov
> > > >>> > > >
> > > >>> > > > 301-683-3683
> > > >>> > > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > >
> > > >>> > > Binbin Zhou
> > > >>> > >
> > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > >
> > > >>> > > 5830 University Research Ct.
> > > >>> > >
> > > >>> > > College Park, MD 20740
> > > >>> > >
> > > >>> > > Binbin.Zhou at noaa.gov
> > > >>> > >
> > > >>> > > 301-683-3683
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Binbin Zhou
> > > >>>
> > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > >>>
> > > >>> 5830 University Research Ct.
> > > >>>
> > > >>> College Park, MD 20740
> > > >>>
> > > >>> Binbin.Zhou at noaa.gov
> > > >>>
> > > >>> 301-683-3683
> > > >>>
> > > >>>
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Thu Dec 10 13:17:08 2020
Julie,
I'll test it, Thanks!
Binbin
On Thu, Dec 10, 2020 at 1:59 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Thanks, John. That worked great.
>
> Binbin, this version of MET is now accessible to you by running the
> following on venus:
>
> module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> module load met/ens_config_test
>
> Please let us know if you encounter any problems.
>
> Julie
>
> On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > Julie,
> >
> > Building the tarfile is very easy. I just ran this script from the
MET
> repo
> > and specified the branch to be built:
> > MET/scripts/met_checkout_and_build.sh branch
feature_1604_ens_config
> >
> > And that created a tarfile which I posted to RAL's anonymous ftp
server:
> >
> >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> >
> > I assume that'll work for you.
> >
> > Thanks,
> > John
> >
> > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >
> > > John,
> > >
> > > I am able to help with this install on WCOSS.
> > >
> > > Would it be easier to use git to checkout the repo on WCOSS? Or
would
> it
> > be
> > > > easier if I built a tarfile for this version of the code?
> > > >
> > > The last time I tried to build from git on WCOSS (a while ago),
I
> > > couldn't. I don't remember all of the details, but I think I
couldn't
> > run
> > > bootstrap and maybe it had to do with the version of autoconf
being too
> > > old? I'm happy to try it again if you'd like though.
Otherwise, I
> know
> > a
> > > tarfile would work, but I know that is extra work for you.
> > >
> > > Julie
> > >
> > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> > >
> > > > Julie,
> > > >
> > > > I have created a bugfix branch for main_v9.1 named
> > > > "feature_1604_ens_config" and it's up on GitHub.
> > > >
> > > > Only a single file has changed in it. But I need to get this
version
> > > > compiled on WCOSS with which Binbin can test.
> > > >
> > > > Are you to help me with this? Would it be easier to use git to
> checkout
> > > > the repo on WCOSS? Or would it be easier if I built a tarfile
for
> this
> > > > version of the code?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > > >
> > > >> Binbin,
> > > >>
> > > >> In order for you to test this change out, we'd need to
recompile a
> > > >> version of the code on WCOSS. I wrote this up as a GitHub
issue:
> > > >> https://github.com/dtcenter/MET/issues/1604
> > > >>
> > > >> And I'll create a branch from main_v9.1 named
> "bugfix_1604_ens_config"
> > > >> and commit the change there. Then we can checkout that code
on
> WCOSS,
> > > >> compile it, and you can test it.
> > > >>
> > > >> Yes, you're right, if the fcst fields differ from the climo
fields,
> > > you'd
> > > >> need to specify them separately. That's pretty easy in the
> > Ensemble-Stat
> > > >> config file... but are you asking how to control that via
METplus?
> > > >>
> > > >> John
> > > >>
> > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >>>
> > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > >>>
> > > >>> John,
> > > >>>
> > > >>> Great! Let me test it at first before make change in
MET9.1
> > > >>> Another question:
> > > >>> Since climatology data is still in NCEP's grib2 format,
In the
> > > >>> ensemble_stat
> > > >>> config file, climo entry settings should be different
from those
> > of
> > > >>> fcst, obs entries?
> > > >>> How to set climo (name, level, etc.)?
> > > >>>
> > > >>> Binbin
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> > > >>> met_help at ucar.edu>
> > > >>> wrote:
> > > >>>
> > > >>> > Binbin,
> > > >>> >
> > > >>> > Success! When I recompile with that one line commented
out,
> > > >>> plot_data_plane
> > > >>> > is able to filter down all those records to the single one
that
> we
> > > >>> want.
> > > >>> > Note in the example below that I used the "lead_time" and
> > "GRIB_ens"
> > > >>> > options to filter the input records down.
> > > >>> >
> > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps '*name="HGT";
> > > >>> level="P500";
> > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found
exact
> match
> > > for
> > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > > >>> "ensposte.t00z.z500hr".*
> > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1
GRIB
> > records
> > > >>> > matching VarInfo "HGT/P500" in GRIB file
"ensposte.t00z.z500hr".
> > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > >>> >
> > > >>> > Also note that this selects out record number 26.
> > > >>> >
> > > >>> > Shall I make this change as a "bugfix" for met-9.1...which
could
> be
> > > >>> > included in a met-9.1.2 bugfix release? Or shall I just
include
> it
> > in
> > > >>> > development for the next release (planned for March 2021)?
> > > >>> >
> > > >>> > Thanks,
> > > >>> > John
> > > >>> >
> > > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via RT
<
> > > >>> > met_help at ucar.edu> wrote:
> > > >>> >
> > > >>> > >
> > > >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > > >>> > >
> > > >>> > > John,
> > > >>> > >
> > > >>> > > If this removing impacts other applications, or takes
some
> time
> > > >>> > > to be effective, then won't try. I will use another
approach,
> > such
> > > >>> as
> > > >>> > > borrow GEFS's grib2 data structure as tank to save EC
ensemble
> > > >>> > > data.
> > > >>> > >
> > > >>> > > Binbin
> > > >>> > >
> > > >>> > >
> > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
Affiliate <
> > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > >>> > >
> > > >>> > > > John,
> > > >>> > > >
> > > >>> > > > Yes, please try to use this approach.
> > > >>> > > >
> > > >>> > > > Thanks!
> > > >>> > > >
> > > >>> > > > Binbin
> > > >>> > > >
> > > >>> > > > Binbin
> > > >>> > > >
> > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway via
RT <
> > > >>> > > > met_help at ucar.edu> wrote:
> > > >>> > > >
> > > >>> > > >> Binbin,
> > > >>> > > >>
> > > >>> > > >> I used the "-PDS10" option for wgrib to dump those
records:
> > > >>> > > >>
> > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6
0 0 1
> 0
> > 0
> > > 1
> > > >>> 0 0
> > > >>> > 0
> > > >>> > > 21
> > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6
0 0 1
> 0
> > 0
> > > 1
> > > >>> 0 0
> > > >>> > 0
> > > >>> > > 21
> > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12 6
0 0 1
> 0
> > 0
> > > 1
> > > >>> 0 0
> > > >>> > 0
> > > >>> > > 21
> > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > >>> > > >>
> > > >>> > > >> Looking VERY closely, there is a difference in the
42-nd
> > column
> > > >>> which
> > > >>> > is
> > > >>> > > >> the 4-th from the last... records 11, 26, and 43 have
values
> > of
> > > >>> 3, 1,
> > > >>> > > and
> > > >>> > > >> 2
> > > >>> > > >> respectively.
> > > >>> > > >>
> > > >>> > > >> On this page (
> > > >>> > >
https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > >>> > > >> ),
> > > >>> > > >> byte 42 = 1 means "unperturbed control forecast".
> > > >>> > > >>
> > > >>> > > >> So we *should* be able to use the "GRIB_ens" config
option
> to
> > > >>> > > >> differentiate
> > > >>> > > >> between those records. BUT when I try, it doesn't
work. And
> > > that's
> > > >>> > > because
> > > >>> > > >> of this line:
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > >>> > > >>
> > > >>> > > >> We only apply those filtering options to GRIB files
created
> by
> > > >>> NCEP.
> > > >>> > If
> > > >>> > > >> you'd like, I could try removing that condition to
see if we
> > can
> > > >>> pick
> > > >>> > > out
> > > >>> > > >> the single record of interest.
> > > >>> > > >>
> > > >>> > > >> John
> > > >>> > > >>
> > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov
via RT
> <
> > > >>> > > >> met_help at ucar.edu> wrote:
> > > >>> > > >>
> > > >>> > > >> >
> > > >>> > > >> > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >
> > > >>> > > >> >
> > > >>> > > >> > John,
> > > >>> > > >> >
> > > >>> > > >> > Look at my
> > > >>> > > >> > directory
> > > >>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is
only for
> 500
> > > >>> HGT.
> > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
anl:type=analysis
> > > >>> > > >> > I get 3 records:
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >>> > > >> >
> > > >>> > > >> > From its grib2 file, we know only record#26 is
analysis
> > > low-res
> > > >>> > > >> control.
> > > >>> > > >> > If you have some way to determine which record is
analysis
> > > >>> low-res
> > > >>> > > >> control
> > > >>> > > >> > from the grib1 file,
> > > >>> > > >> > then we can use grib1 directly (don't consider the
size
> > > issue).
> > > >>> > > >> >
> > > >>> > > >> > Binbin
> > > >>> > > >> >
> > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
> Affiliate
> > <
> > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > >>> > > >> >
> > > >>> > > >> > > John,
> > > >>> > > >> > >
> > > >>> > > >> > > Grib1 data is too large, another issue is the
low-res
> > > >>> control
> > > >>> > > >> analysis
> > > >>> > > >> > > data can not be retrieved
> > > >>> > > >> > > from the grib1 ecmwf data files. If using wgrib,
we can
> > see
> > > 3
> > > >>> > > analysis
> > > >>> > > >> > > fields, but don't know
> > > >>> > > >> > > which is low-res control. After convert to grib2,
wgrib2
> > can
> > > >>> show
> > > >>> > > >> which
> > > >>> > > >> > > record is low-res control.
> > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set
on
> WCOSS.
> > > >>> > > >> > >
> > > >>> > > >> > > Binbin
> > > >>> > > >> > >
> > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley Gotway
via
> RT <
> > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > >>> > > >> > >
> > > >>> > > >> > >> Binbin,
> > > >>> > > >> > >>
> > > >>> > > >> > >> Sounds like there's a lot of moving parts there.
How
> > would
> > > >>> you
> > > >>> > like
> > > >>> > > >> to
> > > >>> > > >> > >> proceed?
> > > >>> > > >> > >>
> > > >>> > > >> > >> Should we check to see if MET can read those
ECMWF
> GRIB1
> > > >>> files
> > > >>> > > >> directly
> > > >>> > > >> > >> without needing to reformat?
> > > >>> > > >> > >> Or do you just want to create a custom GRIB
table for
> the
> > > >>> result
> > > >>> > of
> > > >>> > > >> your
> > > >>> > > >> > >> pre-processing and conversion steps?
> > > >>> > > >> > >>
> > > >>> > > >> > >> John
> > > >>> > > >> > >>
> > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
Binbin.Zhou at noaa.gov
> via
> > > RT <
> > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > >>> > > >> > >>
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > <URL:
> > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >>> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > John,
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
converted
> > from
> > > >>> its
> > > >>> > > grib1
> > > >>> > > >> > >> format
> > > >>> > > >> > >> > data.
> > > >>> > > >> > >> > Before conversion, the parameter names are
there, but
> > > after
> > > >>> > > >> > conversion,
> > > >>> > > >> > >> the
> > > >>> > > >> > >> > names
> > > >>> > > >> > >> > are gone. It is because of unknown tables. The
grib1
> > data
> > > >>> is
> > > >>> > > still
> > > >>> > > >> > >> > original. The original
> > > >>> > > >> > >> > grib1 data files are very strange data format
we can
> > not
> > > >>> read.
> > > >>> > So
> > > >>> > > >> we
> > > >>> > > >> > >> have
> > > >>> > > >> > >> > to use
> > > >>> > > >> > >> > a Frotran code to re-arrange the fields and
save them
> > as
> > > >>> > > NCEP-like
> > > >>> > > >> > grib1
> > > >>> > > >> > >> > data files.
> > > >>> > > >> > >> > So after several processes, some important
info like
> > > tables
> > > >>> > could
> > > >>> > > >> be
> > > >>> > > >> > >> lost.
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > I'll test your first suggestion since the
field
> > record
> > > >>> order
> > > >>> > > is
> > > >>> > > >> > >> fixed.
> > > >>> > > >> > >> > Currently we still
> > > >>> > > >> > >> > obtained grib1 data from ECWMF, eventually, we
will
> get
> > > >>> grib2
> > > >>> > > data
> > > >>> > > >> > >> > directly. So before
> > > >>> > > >> > >> > we can get original grib2 files from ECMWF, we
just
> use
> > > >>> this
> > > >>> > > >> temporal
> > > >>> > > >> > >> > solution.
> > > >>> > > >> > >> > '
> > > >>> > > >> > >> > Thanks!
> > > >>> > > >> > >> >
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley
Gotway via
> > RT
> > > <
> > > >>> > > >> > >> > met_help at ucar.edu>
> > > >>> > > >> > >> > wrote:
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > > Binbin,
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > OK, working on Venus.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > You have a typo in the command you sent.
> > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> a.ps
> > > >>> > > >> > >> > >
> > > >>> > > >> > >>
> > > >>> > > >>
> > > >>> >
> > > >>>
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >>> > > >> > >> > -v
> > > >>> > > >> > >> > > 3
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
> > > "GRIB2_parm_cat".
> > > >>> > After
> > > >>> > > >> > >> > correcting
> > > >>> > > >> > >> > > that I did still get an error but it's a
very
> useful
> > > one!
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > module use
> > > >>> > > >> > >>
> > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > >>> > > >> > >> > > module load met/9.1
> > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > >>> > > >> > >> > > cd
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 ~/
> > > plot.ps
> > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
GRIB2_parm=5;
> > > >>> level="P500";'
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > *ERROR : ERROR : VarInfoGrib2::set_dict()
-> no
> > > >>> parameter
> > > >>> > > found
> > > >>> > > >> > with
> > > >>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat (3)
> > GRIB2_parm
> > > >>> (5).
> > > >>> > > Use
> > > >>> > > >> the
> > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
define
> custom
> > > >>> GRIB
> > > >>> > > >> > >> tables.ERROR
> > > >>> > > >> > >> > :
> > > >>> > > >> > >> > > *
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > So this table is not yet defined in MET. I
checked
> > > their
> > > >>> > > website:
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > But I don't see an entry defined for
GRIB2_parm=5;
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > 3 things...
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > (1) As a crude temporary solution, you could
refer
> to
> > > it
> > > >>> by
> > > >>> > > it's
> > > >>> > > >> > >> record
> > > >>> > > >> > >> > > number. Let's assume you know that record 1
has
> > > >>> temperature
> > > >>> > > data:
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 ~/
> > > plot.ps
> > > >>> > > >> > 'name="TMP";
> > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > Here I'm reading record 1, naming it TMP and
the
> > using
> > > >>> > > >> > set_attr_level
> > > >>> > > >> > >> > write
> > > >>> > > >> > >> > > the output level as P500 instead of R1.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > (2) You could create a custom GRIB table
yourself
> > > >>> locally,
> > > >>> > and
> > > >>> > > >> then
> > > >>> > > >> > >> set
> > > >>> > > >> > >> > the
> > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
point MET
> to
> > > it.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > (3) The best solution is contacting ECMWF
and
> > > requesting
> > > >>> they
> > > >>> > > >> send
> > > >>> > > >> > >> over
> > > >>> > > >> > >> > the
> > > >>> > > >> > >> > > GRIB2 table we should be using to decode
this data.
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > Thanks,
> > > >>> > > >> > >> > > John
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley
Gotway <
> > > >>> > > >> johnhg at ucar.edu>
> > > >>> > > >> > >> > wrote:
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > > > We might already have them. I'm not sure.
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > But there's an additional required piece
of the
> > > puzzle.
> > > >>> > > Binbin
> > > >>> > > >> > runs
> > > >>> > > >> > >> > > > "wgrib2" to figure out what "name" he
should use
> in
> > > >>> the MET
> > > >>> > > >> config
> > > >>> > > >> > >> > files.
> > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the right
"name", the
> > > >>> problem
> > > >>> > will
> > > >>> > > >> > >> remain.
> > > >>> > > >> > >> > > > wgrib2 can be configured to define
additional
> > tables
> > > as
> > > >>> > well,
> > > >>> > > >> but
> > > >>> > > >> > >> for
> > > >>> > > >> > >> > > this
> > > >>> > > >> > >> > > > to be easy for Binbin, that'd mean
updating the
> > > script
> > > >>> > that's
> > > >>> > > >> run
> > > >>> > > >> > >> for
> > > >>> > > >> > >> > > > "module load grib_utils" on WCOSS. And I'm
> guessing
> > > >>> that's
> > > >>> > > not
> > > >>> > > >> > very
> > > >>> > > >> > >> > easy.
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > Yes, I do agree that the best solution is
getting
> > the
> > > >>> ECMWF
> > > >>> > > >> GRIB
> > > >>> > > >> > >> tables
> > > >>> > > >> > >> > > > and adding them for both wgrib2 and MET.
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > John
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara Jensen
<
> > > >>> > jensen at ucar.edu>
> > > >>> > > >> > wrote:
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > > >> John,
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> It seems to me that it would be
worthwhile to
> just
> > > ask
> > > >>> > ECMWF
> > > >>> > > >> for
> > > >>> > > >> > >> their
> > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble files.
And other
> > > >>> > operational
> > > >>> > > >> > >> centers
> > > >>> > > >> > >> > for
> > > >>> > > >> > >> > > >> that matter. Do you agree this would be
a good
> > > >>> solution?
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Thanks, Tara
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> ---------- Forwarded message ---------
> > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
> > > met_help at ucar.edu>
> > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769]
urgent
> > > >>> question for
> > > >>> > > >> > reading
> > > >>> > > >> > >> > ECMWF
> > > >>> > > >> > >> > > >> ensmeble data
> > > >>> > > >> > >> > > >> To:
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> <URL:
> > > >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >>> > > >> >
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> John,
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> I tested using following command:
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > > a.ps
> > > >>> > > >> > >> > > >>
> > > >>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >>> > > >> -v3
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> ERROR :
> > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either name
> > or
> > > >>> > > >> GRIB2_disc,
> > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified
> in
> > > >>> field
> > > >>> > > >> > >> information
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > > a.ps
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> >
> > > >>> > > >> >
> > > >>> > >
> > > >>>
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >>> > > >> > >> > > >> -v
> > > >>> > > >> > >> > > >> 3
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Still got error
> > > >>> > > >> > >> > > >> ERROR :
> > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either name
> > or
> > > >>> > > >> GRIB2_disc,
> > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
specified
> in
> > > >>> field
> > > >>> > > >> > >> information
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Looks like discipline=255 is not
recognized
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Binbin
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John
Halley
> Gotway
> > > >>> via RT
> > > >>> > <
> > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > >>> > > >> > >> > > >> wrote:
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> > Binbin,
> > > >>> > > >> > >> > > >> > You can do something similar with MET.
Please
> go
> > > to
> > > >>> this
> > > >>> > > >> link:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >>
> > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and
you'll
> > find
> > > >>> the
> > > >>> > > >> content
> > > >>> > > >> > >> > > >> > copied/pasted below. You can specify
these
> > INSTEAD
> > > >>> OF
> > > >>> > > >> > specifying
> > > >>> > > >> > >> a
> > > >>> > > >> > >> > > >> > requested name.
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an
integer
> > > >>> specifying the
> > > >>> > > >> master
> > > >>> > > >> > >> > table
> > > >>> > > >> > >> > > >> > number.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an
integer
> > > >>> specifying the
> > > >>> > > >> local
> > > >>> > > >> > >> table
> > > >>> > > >> > >> > > >> > number.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an
integer
> > > >>> specifying the
> > > >>> > > >> GRIB2
> > > >>> > > >> > >> > > >> discipline
> > > >>> > > >> > >> > > >> > code.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> > parameter
> > > >>> > > >> > >> > > >> > category
> > > >>> > > >> > >> > > >> > // code.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an
integer
> > > >>> specifying the
> > > >>> > > >> > >> parameter
> > > >>> > > >> > >> > > code.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an
integer
> > > specifying
> > > >>> the
> > > >>> > > >> product
> > > >>> > > >> > >> > > >> definition
> > > >>> > > >> > >> > > >> > // template (Table 4.0).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_process" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> > generating
> > > >>> > > >> > >> > > >> > process
> > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an
integer
> > > >>> specifying the
> > > >>> > > >> > >> originating
> > > >>> > > >> > >> > > >> > center.
> > > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> > ensemble
> > > >>> > > >> > >> > > >> type
> > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an
integer
> > > >>> specifying
> > > >>> > > the
> > > >>> > > >> > >> derived
> > > >>> > > >> > >> > > >> > product
> > > >>> > > >> > >> > > >> > // type (Table 4.7).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is an
integer
> > > >>> > specifying
> > > >>> > > >> the
> > > >>> > > >> > >> > > >> statistical
> > > >>> > > >> > >> > > >> > // processing type (Table
4.10).
> > > >>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index"
and
> > > >>> > > "GRIB2_ipdtmpl_val"
> > > >>> > > >> > >> entries
> > > >>> > > >> > >> > > are
> > > >>> > > >> > >> > > >> > arrays
> > > >>> > > >> > >> > > >> > // of integers which specify
the
> product
> > > >>> > > description
> > > >>> > > >> > >> > template
> > > >>> > > >> > >> > > >> > values to
> > > >>> > > >> > >> > > >> > // be used. The indices are 0-
based.
> For
> > > >>> > example,
> > > >>> > > >> use
> > > >>> > > >> > the
> > > >>> > > >> > >> > > >> > following to
> > > >>> > > >> > >> > > >> > // request a GRIB2 record whose
9-th
> and
> > > >>> 27-th
> > > >>> > > >> product
> > > >>> > > >> > >> > > >> description
> > > >>> > > >> > >> > > >> > // template values are 1 and 2,
> > > >>> respectively:
> > > >>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8,
26];
> > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > >>> > > >> > >> 2];
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > While I don't have any ECMWF data on
hand for
> > > >>> testing, I
> > > >>> > > do
> > > >>> > > >> > have
> > > >>> > > >> > >> a
> > > >>> > > >> > >> > > file
> > > >>> > > >> > >> > > >> > from the Air Force from which wgrib2
prints a
> > > >>> similar
> > > >>> > > >> result:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
discipline=0
> > > >>> center=57
> > > >>> > > >> > >> local_table=1
> > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour
fcst:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > So I used that for testing. I ran the
> following
> > > >>> > > >> plot_data_plane
> > > >>> > > >> > >> > > command
> > > >>> > > >> > >> > > >> > WITHOUT specifying a requested name:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > plot_data_plane
GALWEM_20170610_CY00_FH18.GR2
> > > >>> plot.ps
> > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
GRIB2_parm_cat=1;
> > > >>> > > >> GRIB2_parm=192;
> > > >>> > > >> > >> > > >> level="Z0";'
> > > >>> > > >> > >> > > >> > -v 3
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > And then I looked closely at the log
message
> to
> > > >>> confirm
> > > >>> > > >> that it
> > > >>> > > >> > >> used
> > > >>> > > >> > >> > > >> > record 549, as expected:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > DEBUG 3: MetGrib2DataFile::data_plane()
->
> Found
> > > >>> exact
> > > >>> > > match
> > > >>> > > >> > for
> > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field 1
of
> GRIB2
> > > file
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I
defined to
> do
> > a
> > > >>> table
> > > >>> > > >> lookup
> > > >>> > > >> > >> and
> > > >>> > > >> > >> > > find
> > > >>> > > >> > >> > > >> a
> > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >>
> > > >>> > >
> > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > And it determined that that field
should be
> > called
> > > >>> > > "CTSTM".
> > > >>> > > >> > >> > Hopefully,
> > > >>> > > >> > >> > > >> the
> > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already defined
there
> in
> > > >>> MET. If
> > > >>> > > >> not, we
> > > >>> > > >> > >> can
> > > >>> > > >> > >> > > >> always
> > > >>> > > >> > >> > > >> > add them.
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > Please try running plot_data_plane at
> verbosity
> > > >>> level 3
> > > >>> > on
> > > >>> > > >> one
> > > >>> > > >> > of
> > > >>> > > >> > >> > > these
> > > >>> > > >> > >> > > >> > files first. And make sure it runs
without
> error
> > > >>> and the
> > > >>> > > log
> > > >>> > > >> > >> message
> > > >>> > > >> > >> > > >> prints
> > > >>> > > >> > >> > > >> > out the "name" it found for that data.
If so,
> > you
> > > >>> can
> > > >>> > just
> > > >>> > > >> skip
> > > >>> > > >> > >> > using
> > > >>> > > >> > >> > > >> all
> > > >>> > > >> > >> > > >> > those integers and revert back to using
the
> name
> > > MET
> > > >>> > > knows.
> > > >>> > > >> If
> > > >>> > > >> > >> not,
> > > >>> > > >> > >> > > >> then we
> > > >>> > > >> > >> > > >> > may need to define a custom GRIB table
for
> this
> > > >>> ECMWF
> > > >>> > > GRIB2
> > > >>> > > >> > data.
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >> > Thanks,
> > > >>> > > >> > >> > > >> > John
> > > >>> > > >> > >> > > >> >
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> --
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Binbin Zhou
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> College Park, MD 20740
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> 301-683-3683
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >> --
> > > >>> > > >> > >> > > >>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >>> > > >> > >> > > >> Tara Jensen
> > > >>> > > >> > >> > > >> Project Manager II
> > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307 USA
> > > >>> > > >> > >> > > >> +1 303-497-8479 jensen at ucar.edu
> > > >>> > > >> > >> > > >>
> > > >>> > > >> > >> > > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> > >
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > --
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > Binbin Zhou
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > 5830 University Research Ct.
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > College Park, MD 20740
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > >>> > > >> > >> >
> > > >>> > > >> > >> > 301-683-3683
> > > >>> > > >> > >> >
> > > >>> > > >> > >> >
> > > >>> > > >> > >>
> > > >>> > > >> > >>
> > > >>> > > >> > >
> > > >>> > > >> > > --
> > > >>> > > >> > >
> > > >>> > > >> > > Binbin Zhou
> > > >>> > > >> > >
> > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> > >
> > > >>> > > >> > > 5830 University Research Ct.
> > > >>> > > >> > >
> > > >>> > > >> > > College Park, MD 20740
> > > >>> > > >> > >
> > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > >>> > > >> > >
> > > >>> > > >> > > 301-683-3683
> > > >>> > > >> > >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >> > --
> > > >>> > > >> >
> > > >>> > > >> > Binbin Zhou
> > > >>> > > >> >
> > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >> >
> > > >>> > > >> > 5830 University Research Ct.
> > > >>> > > >> >
> > > >>> > > >> > College Park, MD 20740
> > > >>> > > >> >
> > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > >>> > > >> >
> > > >>> > > >> > 301-683-3683
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > >
> > > >>> > > > Binbin Zhou
> > > >>> > > >
> > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > > >
> > > >>> > > > 5830 University Research Ct.
> > > >>> > > >
> > > >>> > > > College Park, MD 20740
> > > >>> > > >
> > > >>> > > > Binbin.Zhou at noaa.gov
> > > >>> > > >
> > > >>> > > > 301-683-3683
> > > >>> > > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > >
> > > >>> > > Binbin Zhou
> > > >>> > >
> > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >>> > >
> > > >>> > > 5830 University Research Ct.
> > > >>> > >
> > > >>> > > College Park, MD 20740
> > > >>> > >
> > > >>> > > Binbin.Zhou at noaa.gov
> > > >>> > >
> > > >>> > > 301-683-3683
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>>
> > > >>> --
> > > >>>
> > > >>> Binbin Zhou
> > > >>>
> > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > >>>
> > > >>> 5830 University Research Ct.
> > > >>>
> > > >>> College Park, MD 20740
> > > >>>
> > > >>> Binbin.Zhou at noaa.gov
> > > >>>
> > > >>> 301-683-3683
> > > >>>
> > > >>>
> > >
> > > --
> > > Julie Prestopnik (she/her)
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Thu Dec 10 13:18:00 2020
John,
Thanks!
Binbin
On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Thanks Julie.
>
> And Binbin, here's an EnsembleStatConfig file where climos are used:
>
>
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
>
> Since the list of forecast fields is exactly the same as the climo
fields,
> this config file just copies them over using:
> climo_mean = fcst;
>
> In your case, when they differ, you'd just specify the "field" array
inside
> the "climo_mean" dictionary. For example:
>
> climo_mean = {
>
> field = [
> { name = "TMP"; level = "Z2"; },
> { name = "TMP"; level = "P850"; }
> ];
> ...
> }
>
> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > Thanks, John. That worked great.
> >
> > Binbin, this version of MET is now accessible to you by running
the
> > following on venus:
> >
> > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > module load met/ens_config_test
> >
> > Please let us know if you encounter any problems.
> >
> > Julie
> >
> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >
> > > Julie,
> > >
> > > Building the tarfile is very easy. I just ran this script from
the MET
> > repo
> > > and specified the branch to be built:
> > > MET/scripts/met_checkout_and_build.sh branch
feature_1604_ens_config
> > >
> > > And that created a tarfile which I posted to RAL's anonymous ftp
> server:
> > >
> > >
> > >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> > >
> > > I assume that'll work for you.
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > > >
> > > > John,
> > > >
> > > > I am able to help with this install on WCOSS.
> > > >
> > > > Would it be easier to use git to checkout the repo on WCOSS?
Or would
> > it
> > > be
> > > > > easier if I built a tarfile for this version of the code?
> > > > >
> > > > The last time I tried to build from git on WCOSS (a while
ago), I
> > > > couldn't. I don't remember all of the details, but I think I
> couldn't
> > > run
> > > > bootstrap and maybe it had to do with the version of autoconf
being
> too
> > > > old? I'm happy to try it again if you'd like though.
Otherwise, I
> > know
> > > a
> > > > tarfile would work, but I know that is extra work for you.
> > > >
> > > > Julie
> > > >
> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > > >
> > > > > Julie,
> > > > >
> > > > > I have created a bugfix branch for main_v9.1 named
> > > > > "feature_1604_ens_config" and it's up on GitHub.
> > > > >
> > > > > Only a single file has changed in it. But I need to get this
> version
> > > > > compiled on WCOSS with which Binbin can test.
> > > > >
> > > > > Are you to help me with this? Would it be easier to use git
to
> > checkout
> > > > > the repo on WCOSS? Or would it be easier if I built a
tarfile for
> > this
> > > > > version of the code?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway
<johnhg at ucar.edu
> >
> > > > wrote:
> > > > >
> > > > >> Binbin,
> > > > >>
> > > > >> In order for you to test this change out, we'd need to
recompile a
> > > > >> version of the code on WCOSS. I wrote this up as a GitHub
issue:
> > > > >> https://github.com/dtcenter/MET/issues/1604
> > > > >>
> > > > >> And I'll create a branch from main_v9.1 named
> > "bugfix_1604_ens_config"
> > > > >> and commit the change there. Then we can checkout that code
on
> > WCOSS,
> > > > >> compile it, and you can test it.
> > > > >>
> > > > >> Yes, you're right, if the fcst fields differ from the climo
> fields,
> > > > you'd
> > > > >> need to specify them separately. That's pretty easy in the
> > > Ensemble-Stat
> > > > >> config file... but are you asking how to control that via
METplus?
> > > > >>
> > > > >> John
> > > > >>
> > > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT
<
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >>>
> > > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > > >>>
> > > > >>> John,
> > > > >>>
> > > > >>> Great! Let me test it at first before make change in
MET9.1
> > > > >>> Another question:
> > > > >>> Since climatology data is still in NCEP's grib2 format,
In the
> > > > >>> ensemble_stat
> > > > >>> config file, climo entry settings should be different
from
> those
> > > of
> > > > >>> fcst, obs entries?
> > > > >>> How to set climo (name, level, etc.)?
> > > > >>>
> > > > >>> Binbin
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT <
> > > > >>> met_help at ucar.edu>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Binbin,
> > > > >>> >
> > > > >>> > Success! When I recompile with that one line commented
out,
> > > > >>> plot_data_plane
> > > > >>> > is able to filter down all those records to the single
one that
> > we
> > > > >>> want.
> > > > >>> > Note in the example below that I used the "lead_time"
and
> > > "GRIB_ens"
> > > > >>> > options to filter the input records down.
> > > > >>> >
> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
'*name="HGT";
> > > > >>> level="P500";
> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found
exact
> > match
> > > > for
> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > > > >>> "ensposte.t00z.z500hr".*
> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1
GRIB
> > > records
> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> "ensposte.t00z.z500hr".
> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > > >>> >
> > > > >>> > Also note that this selects out record number 26.
> > > > >>> >
> > > > >>> > Shall I make this change as a "bugfix" for met-
9.1...which
> could
> > be
> > > > >>> > included in a met-9.1.2 bugfix release? Or shall I just
include
> > it
> > > in
> > > > >>> > development for the next release (planned for March
2021)?
> > > > >>> >
> > > > >>> > Thanks,
> > > > >>> > John
> > > > >>> >
> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via
RT <
> > > > >>> > met_help at ucar.edu> wrote:
> > > > >>> >
> > > > >>> > >
> > > > >>> > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > > > >>> > >
> > > > >>> > > John,
> > > > >>> > >
> > > > >>> > > If this removing impacts other applications, or
takes some
> > time
> > > > >>> > > to be effective, then won't try. I will use another
> approach,
> > > such
> > > > >>> as
> > > > >>> > > borrow GEFS's grib2 data structure as tank to save EC
> ensemble
> > > > >>> > > data.
> > > > >>> > >
> > > > >>> > > Binbin
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
Affiliate <
> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > > >>> > >
> > > > >>> > > > John,
> > > > >>> > > >
> > > > >>> > > > Yes, please try to use this approach.
> > > > >>> > > >
> > > > >>> > > > Thanks!
> > > > >>> > > >
> > > > >>> > > > Binbin
> > > > >>> > > >
> > > > >>> > > > Binbin
> > > > >>> > > >
> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway
via RT <
> > > > >>> > > > met_help at ucar.edu> wrote:
> > > > >>> > > >
> > > > >>> > > >> Binbin,
> > > > >>> > > >>
> > > > >>> > > >> I used the "-PDS10" option for wgrib to dump those
> records:
> > > > >>> > > >>
> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > > >>> > > >>
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12
6 0 0
> 1
> > 0
> > > 0
> > > > 1
> > > > >>> 0 0
> > > > >>> > 0
> > > > >>> > > 21
> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12
6 0 0
> 1
> > 0
> > > 0
> > > > 1
> > > > >>> 0 0
> > > > >>> > 0
> > > > >>> > > 21
> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20 12
6 0 0
> 1
> > 0
> > > 0
> > > > 1
> > > > >>> 0 0
> > > > >>> > 0
> > > > >>> > > 21
> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > > >>> > > >>
> > > > >>> > > >> Looking VERY closely, there is a difference in the
42-nd
> > > column
> > > > >>> which
> > > > >>> > is
> > > > >>> > > >> the 4-th from the last... records 11, 26, and 43
have
> values
> > > of
> > > > >>> 3, 1,
> > > > >>> > > and
> > > > >>> > > >> 2
> > > > >>> > > >> respectively.
> > > > >>> > > >>
> > > > >>> > > >> On this page (
> > > > >>> > >
https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > > >>> > > >> ),
> > > > >>> > > >> byte 42 = 1 means "unperturbed control forecast".
> > > > >>> > > >>
> > > > >>> > > >> So we *should* be able to use the "GRIB_ens" config
option
> > to
> > > > >>> > > >> differentiate
> > > > >>> > > >> between those records. BUT when I try, it doesn't
work.
> And
> > > > that's
> > > > >>> > > because
> > > > >>> > > >> of this line:
> > > > >>> > > >>
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > > >>> > > >>
> > > > >>> > > >> We only apply those filtering options to GRIB files
> created
> > by
> > > > >>> NCEP.
> > > > >>> > If
> > > > >>> > > >> you'd like, I could try removing that condition to
see if
> we
> > > can
> > > > >>> pick
> > > > >>> > > out
> > > > >>> > > >> the single record of interest.
> > > > >>> > > >>
> > > > >>> > > >> John
> > > > >>> > > >>
> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM Binbin.Zhou at noaa.gov
via
> RT
> > <
> > > > >>> > > >> met_help at ucar.edu> wrote:
> > > > >>> > > >>
> > > > >>> > > >> >
> > > > >>> > > >> > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >
> > > > >>> > > >> >
> > > > >>> > > >> > John,
> > > > >>> > > >> >
> > > > >>> > > >> > Look at my
> > > > >>> > > >> > directory
> > > > >>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is
only for
> > 500
> > > > >>> HGT.
> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
> anl:type=analysis
> > > > >>> > > >> > I get 3 records:
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > >>> > > >> >
> > > > >>> > > >> > From its grib2 file, we know only record#26 is
analysis
> > > > low-res
> > > > >>> > > >> control.
> > > > >>> > > >> > If you have some way to determine which record is
> analysis
> > > > >>> low-res
> > > > >>> > > >> control
> > > > >>> > > >> > from the grib1 file,
> > > > >>> > > >> > then we can use grib1 directly (don't consider
the size
> > > > issue).
> > > > >>> > > >> >
> > > > >>> > > >> > Binbin
> > > > >>> > > >> >
> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou - NOAA
> > Affiliate
> > > <
> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > > >>> > > >> >
> > > > >>> > > >> > > John,
> > > > >>> > > >> > >
> > > > >>> > > >> > > Grib1 data is too large, another issue is the
> low-res
> > > > >>> control
> > > > >>> > > >> analysis
> > > > >>> > > >> > > data can not be retrieved
> > > > >>> > > >> > > from the grib1 ecmwf data files. If using
wgrib, we
> can
> > > see
> > > > 3
> > > > >>> > > analysis
> > > > >>> > > >> > > fields, but don't know
> > > > >>> > > >> > > which is low-res control. After convert to
grib2,
> wgrib2
> > > can
> > > > >>> show
> > > > >>> > > >> which
> > > > >>> > > >> > > record is low-res control.
> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data set
on
> > WCOSS.
> > > > >>> > > >> > >
> > > > >>> > > >> > > Binbin
> > > > >>> > > >> > >
> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley
Gotway via
> > RT <
> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > > >>> > > >> > >
> > > > >>> > > >> > >> Binbin,
> > > > >>> > > >> > >>
> > > > >>> > > >> > >> Sounds like there's a lot of moving parts
there. How
> > > would
> > > > >>> you
> > > > >>> > like
> > > > >>> > > >> to
> > > > >>> > > >> > >> proceed?
> > > > >>> > > >> > >>
> > > > >>> > > >> > >> Should we check to see if MET can read those
ECMWF
> > GRIB1
> > > > >>> files
> > > > >>> > > >> directly
> > > > >>> > > >> > >> without needing to reformat?
> > > > >>> > > >> > >> Or do you just want to create a custom GRIB
table for
> > the
> > > > >>> result
> > > > >>> > of
> > > > >>> > > >> your
> > > > >>> > > >> > >> pre-processing and conversion steps?
> > > > >>> > > >> > >>
> > > > >>> > > >> > >> John
> > > > >>> > > >> > >>
> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
Binbin.Zhou at noaa.gov
> > via
> > > > RT <
> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > > >>> > > >> > >>
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > <URL:
> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >>> > >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > John,
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
converted
> > > from
> > > > >>> its
> > > > >>> > > grib1
> > > > >>> > > >> > >> format
> > > > >>> > > >> > >> > data.
> > > > >>> > > >> > >> > Before conversion, the parameter names are
there,
> but
> > > > after
> > > > >>> > > >> > conversion,
> > > > >>> > > >> > >> the
> > > > >>> > > >> > >> > names
> > > > >>> > > >> > >> > are gone. It is because of unknown tables.
The
> grib1
> > > data
> > > > >>> is
> > > > >>> > > still
> > > > >>> > > >> > >> > original. The original
> > > > >>> > > >> > >> > grib1 data files are very strange data
format we
> can
> > > not
> > > > >>> read.
> > > > >>> > So
> > > > >>> > > >> we
> > > > >>> > > >> > >> have
> > > > >>> > > >> > >> > to use
> > > > >>> > > >> > >> > a Frotran code to re-arrange the fields and
save
> them
> > > as
> > > > >>> > > NCEP-like
> > > > >>> > > >> > grib1
> > > > >>> > > >> > >> > data files.
> > > > >>> > > >> > >> > So after several processes, some important
info
> like
> > > > tables
> > > > >>> > could
> > > > >>> > > >> be
> > > > >>> > > >> > >> lost.
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > I'll test your first suggestion since the
field
> > > record
> > > > >>> order
> > > > >>> > > is
> > > > >>> > > >> > >> fixed.
> > > > >>> > > >> > >> > Currently we still
> > > > >>> > > >> > >> > obtained grib1 data from ECWMF, eventually,
we will
> > get
> > > > >>> grib2
> > > > >>> > > data
> > > > >>> > > >> > >> > directly. So before
> > > > >>> > > >> > >> > we can get original grib2 files from ECMWF,
we just
> > use
> > > > >>> this
> > > > >>> > > >> temporal
> > > > >>> > > >> > >> > solution.
> > > > >>> > > >> > >> > '
> > > > >>> > > >> > >> > Thanks!
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley
Gotway
> via
> > > RT
> > > > <
> > > > >>> > > >> > >> > met_help at ucar.edu>
> > > > >>> > > >> > >> > wrote:
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > > Binbin,
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > OK, working on Venus.
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > You have a typo in the command you sent.
> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> > a.ps
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >>
> > > > >>> > > >>
> > > > >>> >
> > > > >>>
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >>> > > >> > >> > -v
> > > > >>> > > >> > >> > > 3
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
> > > > "GRIB2_parm_cat".
> > > > >>> > After
> > > > >>> > > >> > >> > correcting
> > > > >>> > > >> > >> > > that I did still get an error but it's a
very
> > useful
> > > > one!
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > module use
> > > > >>> > > >> > >>
> > > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > >>> > > >> > >> > > module load met/9.1
> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > > >>> > > >> > >> > > cd
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >>
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 ~/
> > > > plot.ps
> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
GRIB2_parm=5;
> > > > >>> level="P500";'
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > *ERROR : ERROR :
VarInfoGrib2::set_dict() -> no
> > > > >>> parameter
> > > > >>> > > found
> > > > >>> > > >> > with
> > > > >>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat
(3)
> > > GRIB2_parm
> > > > >>> (5).
> > > > >>> > > Use
> > > > >>> > > >> the
> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
define
> > custom
> > > > >>> GRIB
> > > > >>> > > >> > >> tables.ERROR
> > > > >>> > > >> > >> > :
> > > > >>> > > >> > >> > > *
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > So this table is not yet defined in MET. I
> checked
> > > > their
> > > > >>> > > website:
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >>
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > But I don't see an entry defined for
> GRIB2_parm=5;
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > 3 things...
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > (1) As a crude temporary solution, you
could
> refer
> > to
> > > > it
> > > > >>> by
> > > > >>> > > it's
> > > > >>> > > >> > >> record
> > > > >>> > > >> > >> > > number. Let's assume you know that record
1 has
> > > > >>> temperature
> > > > >>> > > data:
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2 ~/
> > > > plot.ps
> > > > >>> > > >> > 'name="TMP";
> > > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > Here I'm reading record 1, naming it TMP
and the
> > > using
> > > > >>> > > >> > set_attr_level
> > > > >>> > > >> > >> > write
> > > > >>> > > >> > >> > > the output level as P500 instead of R1.
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > (2) You could create a custom GRIB table
yourself
> > > > >>> locally,
> > > > >>> > and
> > > > >>> > > >> then
> > > > >>> > > >> > >> set
> > > > >>> > > >> > >> > the
> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
point MET
> > to
> > > > it.
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > (3) The best solution is contacting ECMWF
and
> > > > requesting
> > > > >>> they
> > > > >>> > > >> send
> > > > >>> > > >> > >> over
> > > > >>> > > >> > >> > the
> > > > >>> > > >> > >> > > GRIB2 table we should be using to decode
this
> data.
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > Thanks,
> > > > >>> > > >> > >> > > John
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John Halley
> Gotway <
> > > > >>> > > >> johnhg at ucar.edu>
> > > > >>> > > >> > >> > wrote:
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > > > We might already have them. I'm not
sure.
> > > > >>> > > >> > >> > > >
> > > > >>> > > >> > >> > > > But there's an additional required piece
of the
> > > > puzzle.
> > > > >>> > > Binbin
> > > > >>> > > >> > runs
> > > > >>> > > >> > >> > > > "wgrib2" to figure out what "name" he
should
> use
> > in
> > > > >>> the MET
> > > > >>> > > >> config
> > > > >>> > > >> > >> > files.
> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the right
"name",
> the
> > > > >>> problem
> > > > >>> > will
> > > > >>> > > >> > >> remain.
> > > > >>> > > >> > >> > > > wgrib2 can be configured to define
additional
> > > tables
> > > > as
> > > > >>> > well,
> > > > >>> > > >> but
> > > > >>> > > >> > >> for
> > > > >>> > > >> > >> > > this
> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd mean
updating the
> > > > script
> > > > >>> > that's
> > > > >>> > > >> run
> > > > >>> > > >> > >> for
> > > > >>> > > >> > >> > > > "module load grib_utils" on WCOSS. And
I'm
> > guessing
> > > > >>> that's
> > > > >>> > > not
> > > > >>> > > >> > very
> > > > >>> > > >> > >> > easy.
> > > > >>> > > >> > >> > > >
> > > > >>> > > >> > >> > > > Yes, I do agree that the best solution
is
> getting
> > > the
> > > > >>> ECMWF
> > > > >>> > > >> GRIB
> > > > >>> > > >> > >> tables
> > > > >>> > > >> > >> > > > and adding them for both wgrib2 and MET.
> > > > >>> > > >> > >> > > >
> > > > >>> > > >> > >> > > > John
> > > > >>> > > >> > >> > > >
> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara
Jensen <
> > > > >>> > jensen at ucar.edu>
> > > > >>> > > >> > wrote:
> > > > >>> > > >> > >> > > >
> > > > >>> > > >> > >> > > >> John,
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> It seems to me that it would be
worthwhile to
> > just
> > > > ask
> > > > >>> > ECMWF
> > > > >>> > > >> for
> > > > >>> > > >> > >> their
> > > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble files.
And
> other
> > > > >>> > operational
> > > > >>> > > >> > >> centers
> > > > >>> > > >> > >> > for
> > > > >>> > > >> > >> > > >> that matter. Do you agree this would
be a
> good
> > > > >>> solution?
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> Thanks, Tara
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> ---------- Forwarded message ---------
> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
> > > > met_help at ucar.edu>
> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769]
urgent
> > > > >>> question for
> > > > >>> > > >> > reading
> > > > >>> > > >> > >> > ECMWF
> > > > >>> > > >> > >> > > >> ensmeble data
> > > > >>> > > >> > >> > > >> To:
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> <URL:
> > > > >>> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >>> > > >> >
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> John,
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> I tested using following command:
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> plot_data_plane
> ecme.ens01.t00z.grd3.f12.grib2
> > > > a.ps
> > > > >>> > > >> > >> > > >>
> > > > >>> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >>> > > >> -v3
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> ERROR :
> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either
> name
> > > or
> > > > >>> > > >> GRIB2_disc,
> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
> specified
> > in
> > > > >>> field
> > > > >>> > > >> > >> information
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> plot_data_plane
> ecme.ens01.t00z.grd3.f12.grib2
> > > > a.ps
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> >
> > > > >>> > > >> >
> > > > >>> > >
> > > > >>>
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >>> > > >> > >> > > >> -v
> > > > >>> > > >> > >> > > >> 3
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> Still got error
> > > > >>> > > >> > >> > > >> ERROR :
> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either
> name
> > > or
> > > > >>> > > >> GRIB2_disc,
> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
> specified
> > in
> > > > >>> field
> > > > >>> > > >> > >> information
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> Looks like discipline=255 is not
recognized
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> Binbin
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John
Halley
> > Gotway
> > > > >>> via RT
> > > > >>> > <
> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > > >>> > > >> > >> > > >> wrote:
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> > Binbin,
> > > > >>> > > >> > >> > > >> > You can do something similar with
MET.
> Please
> > go
> > > > to
> > > > >>> this
> > > > >>> > > >> link:
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >>
> > > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > And then search for "GRIB2_mtab", and
you'll
> > > find
> > > > >>> the
> > > > >>> > > >> content
> > > > >>> > > >> > >> > > >> > copied/pasted below. You can specify
these
> > > INSTEAD
> > > > >>> OF
> > > > >>> > > >> > specifying
> > > > >>> > > >> > >> a
> > > > >>> > > >> > >> > > >> > requested name.
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an
integer
> > > > >>> specifying the
> > > > >>> > > >> master
> > > > >>> > > >> > >> > table
> > > > >>> > > >> > >> > > >> > number.
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an
integer
> > > > >>> specifying the
> > > > >>> > > >> local
> > > > >>> > > >> > >> table
> > > > >>> > > >> > >> > > >> > number.
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an
integer
> > > > >>> specifying the
> > > > >>> > > >> GRIB2
> > > > >>> > > >> > >> > > >> discipline
> > > > >>> > > >> > >> > > >> > code.
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is an
> integer
> > > > >>> specifying
> > > > >>> > > the
> > > > >>> > > >> > >> > parameter
> > > > >>> > > >> > >> > > >> > category
> > > > >>> > > >> > >> > > >> > // code.
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an
integer
> > > > >>> specifying the
> > > > >>> > > >> > >> parameter
> > > > >>> > > >> > >> > > code.
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an
integer
> > > > specifying
> > > > >>> the
> > > > >>> > > >> product
> > > > >>> > > >> > >> > > >> definition
> > > > >>> > > >> > >> > > >> > // template (Table 4.0).
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_process" is an
integer
> > > > >>> specifying
> > > > >>> > > the
> > > > >>> > > >> > >> > generating
> > > > >>> > > >> > >> > > >> > process
> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an
integer
> > > > >>> specifying the
> > > > >>> > > >> > >> originating
> > > > >>> > > >> > >> > > >> > center.
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is an
> integer
> > > > >>> specifying
> > > > >>> > > the
> > > > >>> > > >> > >> > ensemble
> > > > >>> > > >> > >> > > >> type
> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is an
> integer
> > > > >>> specifying
> > > > >>> > > the
> > > > >>> > > >> > >> derived
> > > > >>> > > >> > >> > > >> > product
> > > > >>> > > >> > >> > > >> > // type (Table 4.7).
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is
an
> integer
> > > > >>> > specifying
> > > > >>> > > >> the
> > > > >>> > > >> > >> > > >> statistical
> > > > >>> > > >> > >> > > >> > // processing type (Table
4.10).
> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index"
and
> > > > >>> > > "GRIB2_ipdtmpl_val"
> > > > >>> > > >> > >> entries
> > > > >>> > > >> > >> > > are
> > > > >>> > > >> > >> > > >> > arrays
> > > > >>> > > >> > >> > > >> > // of integers which specify
the
> > product
> > > > >>> > > description
> > > > >>> > > >> > >> > template
> > > > >>> > > >> > >> > > >> > values to
> > > > >>> > > >> > >> > > >> > // be used. The indices are
0-based.
> > For
> > > > >>> > example,
> > > > >>> > > >> use
> > > > >>> > > >> > the
> > > > >>> > > >> > >> > > >> > following to
> > > > >>> > > >> > >> > > >> > // request a GRIB2 record
whose 9-th
> > and
> > > > >>> 27-th
> > > > >>> > > >> product
> > > > >>> > > >> > >> > > >> description
> > > > >>> > > >> > >> > > >> > // template values are 1 and
2,
> > > > >>> respectively:
> > > > >>> > > >> > >> > > >> > // GRIB2_ipdtmpl_index=[8,
26];
> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > > >>> > > >> > >> 2];
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > While I don't have any ECMWF data on
hand
> for
> > > > >>> testing, I
> > > > >>> > > do
> > > > >>> > > >> > have
> > > > >>> > > >> > >> a
> > > > >>> > > >> > >> > > file
> > > > >>> > > >> > >> > > >> > from the Air Force from which wgrib2
prints
> a
> > > > >>> similar
> > > > >>> > > >> result:
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
discipline=0
> > > > >>> center=57
> > > > >>> > > >> > >> local_table=1
> > > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour
fcst:
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > So I used that for testing. I ran the
> > following
> > > > >>> > > >> plot_data_plane
> > > > >>> > > >> > >> > > command
> > > > >>> > > >> > >> > > >> > WITHOUT specifying a requested name:
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > plot_data_plane
> GALWEM_20170610_CY00_FH18.GR2
> > > > >>> plot.ps
> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
> GRIB2_parm_cat=1;
> > > > >>> > > >> GRIB2_parm=192;
> > > > >>> > > >> > >> > > >> level="Z0";'
> > > > >>> > > >> > >> > > >> > -v 3
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > And then I looked closely at the log
message
> > to
> > > > >>> confirm
> > > > >>> > > >> that it
> > > > >>> > > >> > >> used
> > > > >>> > > >> > >> > > >> > record 549, as expected:
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > DEBUG 3:
MetGrib2DataFile::data_plane() ->
> > Found
> > > > >>> exact
> > > > >>> > > match
> > > > >>> > > >> > for
> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field
1 of
> > GRIB2
> > > > file
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >>
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I
defined to
> > do
> > > a
> > > > >>> table
> > > > >>> > > >> lookup
> > > > >>> > > >> > >> and
> > > > >>> > > >> > >> > > find
> > > > >>> > > >> > >> > > >> a
> > > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >>
> > > > >>> > >
> > > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > And it determined that that field
should be
> > > called
> > > > >>> > > "CTSTM".
> > > > >>> > > >> > >> > Hopefully,
> > > > >>> > > >> > >> > > >> the
> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already
defined there
> > in
> > > > >>> MET. If
> > > > >>> > > >> not, we
> > > > >>> > > >> > >> can
> > > > >>> > > >> > >> > > >> always
> > > > >>> > > >> > >> > > >> > add them.
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > Please try running plot_data_plane at
> > verbosity
> > > > >>> level 3
> > > > >>> > on
> > > > >>> > > >> one
> > > > >>> > > >> > of
> > > > >>> > > >> > >> > > these
> > > > >>> > > >> > >> > > >> > files first. And make sure it runs
without
> > error
> > > > >>> and the
> > > > >>> > > log
> > > > >>> > > >> > >> message
> > > > >>> > > >> > >> > > >> prints
> > > > >>> > > >> > >> > > >> > out the "name" it found for that
data. If
> so,
> > > you
> > > > >>> can
> > > > >>> > just
> > > > >>> > > >> skip
> > > > >>> > > >> > >> > using
> > > > >>> > > >> > >> > > >> all
> > > > >>> > > >> > >> > > >> > those integers and revert back to
using the
> > name
> > > > MET
> > > > >>> > > knows.
> > > > >>> > > >> If
> > > > >>> > > >> > >> not,
> > > > >>> > > >> > >> > > >> then we
> > > > >>> > > >> > >> > > >> > may need to define a custom GRIB
table for
> > this
> > > > >>> ECMWF
> > > > >>> > > GRIB2
> > > > >>> > > >> > data.
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >> > Thanks,
> > > > >>> > > >> > >> > > >> > John
> > > > >>> > > >> > >> > > >> >
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> --
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> Binbin Zhou
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> College Park, MD 20740
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> 301-683-3683
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >> --
> > > > >>> > > >> > >> > > >>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > >>> > > >> > >> > > >> Tara Jensen
> > > > >>> > > >> > >> > > >> Project Manager II
> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307
USA
> > > > >>> > > >> > >> > > >> +1 303-497-8479
jensen at ucar.edu
> > > > >>> > > >> > >> > > >>
> > > > >>> > > >> > >> > > >
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> > >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > --
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > Binbin Zhou
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > 5830 University Research Ct.
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > College Park, MD 20740
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> > 301-683-3683
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >> >
> > > > >>> > > >> > >>
> > > > >>> > > >> > >>
> > > > >>> > > >> > >
> > > > >>> > > >> > > --
> > > > >>> > > >> > >
> > > > >>> > > >> > > Binbin Zhou
> > > > >>> > > >> > >
> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >>> > > >> > >
> > > > >>> > > >> > > 5830 University Research Ct.
> > > > >>> > > >> > >
> > > > >>> > > >> > > College Park, MD 20740
> > > > >>> > > >> > >
> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > > >>> > > >> > >
> > > > >>> > > >> > > 301-683-3683
> > > > >>> > > >> > >
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >> > --
> > > > >>> > > >> >
> > > > >>> > > >> > Binbin Zhou
> > > > >>> > > >> >
> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >>> > > >> >
> > > > >>> > > >> > 5830 University Research Ct.
> > > > >>> > > >> >
> > > > >>> > > >> > College Park, MD 20740
> > > > >>> > > >> >
> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > > >>> > > >> >
> > > > >>> > > >> > 301-683-3683
> > > > >>> > > >> >
> > > > >>> > > >> >
> > > > >>> > > >>
> > > > >>> > > >>
> > > > >>> > > >
> > > > >>> > > > --
> > > > >>> > > >
> > > > >>> > > > Binbin Zhou
> > > > >>> > > >
> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >>> > > >
> > > > >>> > > > 5830 University Research Ct.
> > > > >>> > > >
> > > > >>> > > > College Park, MD 20740
> > > > >>> > > >
> > > > >>> > > > Binbin.Zhou at noaa.gov
> > > > >>> > > >
> > > > >>> > > > 301-683-3683
> > > > >>> > > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > --
> > > > >>> > >
> > > > >>> > > Binbin Zhou
> > > > >>> > >
> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >>> > >
> > > > >>> > > 5830 University Research Ct.
> > > > >>> > >
> > > > >>> > > College Park, MD 20740
> > > > >>> > >
> > > > >>> > > Binbin.Zhou at noaa.gov
> > > > >>> > >
> > > > >>> > > 301-683-3683
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > > >>> --
> > > > >>>
> > > > >>> Binbin Zhou
> > > > >>>
> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > > >>>
> > > > >>> 5830 University Research Ct.
> > > > >>>
> > > > >>> College Park, MD 20740
> > > > >>>
> > > > >>> Binbin.Zhou at noaa.gov
> > > > >>>
> > > > >>> 301-683-3683
> > > > >>>
> > > > >>>
> > > >
> > > > --
> > > > Julie Prestopnik (she/her)
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her)
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Fri Dec 11 10:20:46 2020
John,
As I tested following plotting
plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps '*name="HGT";
level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
the error message is
ERROR :
ERROR : yyerror() -> syntax error in file "/tmp/met_config_9350_0"
ERROR :
ERROR : line = 1
ERROR :
ERROR : column = 1
ERROR :
ERROR : text = "*"
ERROR :
ERROR :
ERROR : *name="HGT"; level="P500"; lead_time="00";
GRIB_ens="low_res_ctl";*
ERROR :
^__________________________________________________________________
ERROR :
After I removed "*"
plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps 'name="HGT";
level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
It shows 3 records matched
DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "HGT/P500" in GRIB record 1 of GRIB file
"ecmanl.t12z.grid3.f00.grib1".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "HGT/P500" in GRIB record 2 of GRIB file
"ecmanl.t12z.grid3.f00.grib1".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "HGT/P500" in GRIB record 3 of GRIB file
"ecmanl.t12z.grid3.f00.grib1".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3 GRIB records
matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
DEBUG 1: Creating postscript file: plot.ps
My testing directory on Venus
is /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
test script is ploting.sh
Can you look at it?
Thanks!
Binbin
On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA Affiliate <
binbin.zhou at noaa.gov> wrote:
> John,
>
> Thanks!
>
> Binbin
>
> On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Thanks Julie.
>>
>> And Binbin, here's an EnsembleStatConfig file where climos are
used:
>>
>>
>>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
>>
>> Since the list of forecast fields is exactly the same as the climo
fields,
>> this config file just copies them over using:
>> climo_mean = fcst;
>>
>> In your case, when they differ, you'd just specify the "field"
array
>> inside
>> the "climo_mean" dictionary. For example:
>>
>> climo_mean = {
>>
>> field = [
>> { name = "TMP"; level = "Z2"; },
>> { name = "TMP"; level = "P850"; }
>> ];
>> ...
>> }
>>
>> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> >
>> > Thanks, John. That worked great.
>> >
>> > Binbin, this version of MET is now accessible to you by running
the
>> > following on venus:
>> >
>> > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
>> > module load met/ens_config_test
>> >
>> > Please let us know if you encounter any problems.
>> >
>> > Julie
>> >
>> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> > >
>> > > Julie,
>> > >
>> > > Building the tarfile is very easy. I just ran this script from
the MET
>> > repo
>> > > and specified the branch to be built:
>> > > MET/scripts/met_checkout_and_build.sh branch
>> feature_1604_ens_config
>> > >
>> > > And that created a tarfile which I posted to RAL's anonymous
ftp
>> server:
>> > >
>> > >
>> > >
>> >
>> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
>> > >
>> > > I assume that'll work for you.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
>> > met_help at ucar.edu
>> > > >
>> > > wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
>> > > >
>> > > > John,
>> > > >
>> > > > I am able to help with this install on WCOSS.
>> > > >
>> > > > Would it be easier to use git to checkout the repo on WCOSS?
Or
>> would
>> > it
>> > > be
>> > > > > easier if I built a tarfile for this version of the code?
>> > > > >
>> > > > The last time I tried to build from git on WCOSS (a while
ago), I
>> > > > couldn't. I don't remember all of the details, but I think I
>> couldn't
>> > > run
>> > > > bootstrap and maybe it had to do with the version of autoconf
being
>> too
>> > > > old? I'm happy to try it again if you'd like though.
Otherwise, I
>> > know
>> > > a
>> > > > tarfile would work, but I know that is extra work for you.
>> > > >
>> > > > Julie
>> > > >
>> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway
<johnhg at ucar.edu>
>> > > wrote:
>> > > >
>> > > > > Julie,
>> > > > >
>> > > > > I have created a bugfix branch for main_v9.1 named
>> > > > > "feature_1604_ens_config" and it's up on GitHub.
>> > > > >
>> > > > > Only a single file has changed in it. But I need to get
this
>> version
>> > > > > compiled on WCOSS with which Binbin can test.
>> > > > >
>> > > > > Are you to help me with this? Would it be easier to use git
to
>> > checkout
>> > > > > the repo on WCOSS? Or would it be easier if I built a
tarfile for
>> > this
>> > > > > version of the code?
>> > > > >
>> > > > > Thanks,
>> > > > > John
>> > > > >
>> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <
>> johnhg at ucar.edu>
>> > > > wrote:
>> > > > >
>> > > > >> Binbin,
>> > > > >>
>> > > > >> In order for you to test this change out, we'd need to
recompile
>> a
>> > > > >> version of the code on WCOSS. I wrote this up as a GitHub
issue:
>> > > > >> https://github.com/dtcenter/MET/issues/1604
>> > > > >>
>> > > > >> And I'll create a branch from main_v9.1 named
>> > "bugfix_1604_ens_config"
>> > > > >> and commit the change there. Then we can checkout that
code on
>> > WCOSS,
>> > > > >> compile it, and you can test it.
>> > > > >>
>> > > > >> Yes, you're right, if the fcst fields differ from the
climo
>> fields,
>> > > > you'd
>> > > > >> need to specify them separately. That's pretty easy in the
>> > > Ensemble-Stat
>> > > > >> config file... but are you asking how to control that via
>> METplus?
>> > > > >>
>> > > > >> John
>> > > > >>
>> > > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via RT
<
>> > > > >> met_help at ucar.edu> wrote:
>> > > > >>
>> > > > >>>
>> > > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>> > > > >>>
>> > > > >>> John,
>> > > > >>>
>> > > > >>> Great! Let me test it at first before make change in
MET9.1
>> > > > >>> Another question:
>> > > > >>> Since climatology data is still in NCEP's grib2
format, In
>> the
>> > > > >>> ensemble_stat
>> > > > >>> config file, climo entry settings should be different
from
>> those
>> > > of
>> > > > >>> fcst, obs entries?
>> > > > >>> How to set climo (name, level, etc.)?
>> > > > >>>
>> > > > >>> Binbin
>> > > > >>>
>> > > > >>>
>> > > > >>>
>> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via RT
<
>> > > > >>> met_help at ucar.edu>
>> > > > >>> wrote:
>> > > > >>>
>> > > > >>> > Binbin,
>> > > > >>> >
>> > > > >>> > Success! When I recompile with that one line commented
out,
>> > > > >>> plot_data_plane
>> > > > >>> > is able to filter down all those records to the single
one
>> that
>> > we
>> > > > >>> want.
>> > > > >>> > Note in the example below that I used the "lead_time"
and
>> > > "GRIB_ens"
>> > > > >>> > options to filter the input records down.
>> > > > >>> >
>> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
'*name="HGT";
>> > > > >>> level="P500";
>> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
>> > > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
>> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> *Found
exact
>> > match
>> > > > for
>> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
>> > > > >>> "ensposte.t00z.z500hr".*
>> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found
1 GRIB
>> > > records
>> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
>> "ensposte.t00z.z500hr".
>> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
>> > > > >>> >
>> > > > >>> > Also note that this selects out record number 26.
>> > > > >>> >
>> > > > >>> > Shall I make this change as a "bugfix" for met-
9.1...which
>> could
>> > be
>> > > > >>> > included in a met-9.1.2 bugfix release? Or shall I just
>> include
>> > it
>> > > in
>> > > > >>> > development for the next release (planned for March
2021)?
>> > > > >>> >
>> > > > >>> > Thanks,
>> > > > >>> > John
>> > > > >>> >
>> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov via
RT <
>> > > > >>> > met_help at ucar.edu> wrote:
>> > > > >>> >
>> > > > >>> > >
>> > > > >>> > > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> > >
>> > > > >>> > >
>> > > > >>> > > John,
>> > > > >>> > >
>> > > > >>> > > If this removing impacts other applications, or
takes some
>> > time
>> > > > >>> > > to be effective, then won't try. I will use another
>> approach,
>> > > such
>> > > > >>> as
>> > > > >>> > > borrow GEFS's grib2 data structure as tank to save EC
>> ensemble
>> > > > >>> > > data.
>> > > > >>> > >
>> > > > >>> > > Binbin
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
Affiliate
>> <
>> > > > >>> > > binbin.zhou at noaa.gov> wrote:
>> > > > >>> > >
>> > > > >>> > > > John,
>> > > > >>> > > >
>> > > > >>> > > > Yes, please try to use this approach.
>> > > > >>> > > >
>> > > > >>> > > > Thanks!
>> > > > >>> > > >
>> > > > >>> > > > Binbin
>> > > > >>> > > >
>> > > > >>> > > > Binbin
>> > > > >>> > > >
>> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway
via RT <
>> > > > >>> > > > met_help at ucar.edu> wrote:
>> > > > >>> > > >
>> > > > >>> > > >> Binbin,
>> > > > >>> > > >>
>> > > > >>> > > >> I used the "-PDS10" option for wgrib to dump those
>> records:
>> > > > >>> > > >>
>> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
>> > > > >>> > > >>
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
>> > > > >>> > > >> | egrep "^11:|^26:|^43:"
>> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20
12 6 0
>> 0 1
>> > 0
>> > > 0
>> > > > 1
>> > > > >>> 0 0
>> > > > >>> > 0
>> > > > >>> > > 21
>> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
>> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20
12 6 0
>> 0 1
>> > 0
>> > > 0
>> > > > 1
>> > > > >>> 0 0
>> > > > >>> > 0
>> > > > >>> > > 21
>> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
>> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20
12 6 0
>> 0 1
>> > 0
>> > > 0
>> > > > 1
>> > > > >>> 0 0
>> > > > >>> > 0
>> > > > >>> > > 21
>> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
>> > > > >>> > > >>
>> > > > >>> > > >> Looking VERY closely, there is a difference in the
42-nd
>> > > column
>> > > > >>> which
>> > > > >>> > is
>> > > > >>> > > >> the 4-th from the last... records 11, 26, and 43
have
>> values
>> > > of
>> > > > >>> 3, 1,
>> > > > >>> > > and
>> > > > >>> > > >> 2
>> > > > >>> > > >> respectively.
>> > > > >>> > > >>
>> > > > >>> > > >> On this page (
>> > > > >>> > >
https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
>> > > > >>> > > >> ),
>> > > > >>> > > >> byte 42 = 1 means "unperturbed control forecast".
>> > > > >>> > > >>
>> > > > >>> > > >> So we *should* be able to use the "GRIB_ens"
config
>> option
>> > to
>> > > > >>> > > >> differentiate
>> > > > >>> > > >> between those records. BUT when I try, it doesn't
work.
>> And
>> > > > that's
>> > > > >>> > > because
>> > > > >>> > > >> of this line:
>> > > > >>> > > >>
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
>> > > > >>> > > >>
>> > > > >>> > > >> We only apply those filtering options to GRIB
files
>> created
>> > by
>> > > > >>> NCEP.
>> > > > >>> > If
>> > > > >>> > > >> you'd like, I could try removing that condition to
see
>> if we
>> > > can
>> > > > >>> pick
>> > > > >>> > > out
>> > > > >>> > > >> the single record of interest.
>> > > > >>> > > >>
>> > > > >>> > > >> John
>> > > > >>> > > >>
>> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
Binbin.Zhou at noaa.gov via
>> RT
>> > <
>> > > > >>> > > >> met_help at ucar.edu> wrote:
>> > > > >>> > > >>
>> > > > >>> > > >> >
>> > > > >>> > > >> > <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> > > > >
>> > > > >>> > > >> >
>> > > > >>> > > >> > John,
>> > > > >>> > > >> >
>> > > > >>> > > >> > Look at my
>> > > > >>> > > >> > directory
>> > > > >>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
>> > > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this is
only
>> for
>> > 500
>> > > > >>> HGT.
>> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
>> anl:type=analysis
>> > > > >>> > > >> > I get 3 records:
>> > > > >>> > > >> >
>> > > > >>> > > >> >
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
>> > > > >>> > > >> >
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
>> > > > >>> > > >> >
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
>> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
>> > > > >>> > > >> >
>> > > > >>> > > >> > From its grib2 file, we know only record#26 is
>> analysis
>> > > > low-res
>> > > > >>> > > >> control.
>> > > > >>> > > >> > If you have some way to determine which record
is
>> analysis
>> > > > >>> low-res
>> > > > >>> > > >> control
>> > > > >>> > > >> > from the grib1 file,
>> > > > >>> > > >> > then we can use grib1 directly (don't consider
the size
>> > > > issue).
>> > > > >>> > > >> >
>> > > > >>> > > >> > Binbin
>> > > > >>> > > >> >
>> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou -
NOAA
>> > Affiliate
>> > > <
>> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
>> > > > >>> > > >> >
>> > > > >>> > > >> > > John,
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > Grib1 data is too large, another issue is
the
>> low-res
>> > > > >>> control
>> > > > >>> > > >> analysis
>> > > > >>> > > >> > > data can not be retrieved
>> > > > >>> > > >> > > from the grib1 ecmwf data files. If using
wgrib, we
>> can
>> > > see
>> > > > 3
>> > > > >>> > > analysis
>> > > > >>> > > >> > > fields, but don't know
>> > > > >>> > > >> > > which is low-res control. After convert to
grib2,
>> wgrib2
>> > > can
>> > > > >>> show
>> > > > >>> > > >> which
>> > > > >>> > > >> > > record is low-res control.
>> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data
set on
>> > WCOSS.
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > Binbin
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley
Gotway via
>> > RT <
>> > > > >>> > > >> > > met_help at ucar.edu> wrote:
>> > > > >>> > > >> > >
>> > > > >>> > > >> > >> Binbin,
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >> Sounds like there's a lot of moving parts
there. How
>> > > would
>> > > > >>> you
>> > > > >>> > like
>> > > > >>> > > >> to
>> > > > >>> > > >> > >> proceed?
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >> Should we check to see if MET can read those
ECMWF
>> > GRIB1
>> > > > >>> files
>> > > > >>> > > >> directly
>> > > > >>> > > >> > >> without needing to reformat?
>> > > > >>> > > >> > >> Or do you just want to create a custom GRIB
table
>> for
>> > the
>> > > > >>> result
>> > > > >>> > of
>> > > > >>> > > >> your
>> > > > >>> > > >> > >> pre-processing and conversion steps?
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >> John
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
Binbin.Zhou at noaa.gov
>> > via
>> > > > RT <
>> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > <URL:
>> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> > > > >>> > >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > John,
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
>> converted
>> > > from
>> > > > >>> its
>> > > > >>> > > grib1
>> > > > >>> > > >> > >> format
>> > > > >>> > > >> > >> > data.
>> > > > >>> > > >> > >> > Before conversion, the parameter names are
there,
>> but
>> > > > after
>> > > > >>> > > >> > conversion,
>> > > > >>> > > >> > >> the
>> > > > >>> > > >> > >> > names
>> > > > >>> > > >> > >> > are gone. It is because of unknown tables.
The
>> grib1
>> > > data
>> > > > >>> is
>> > > > >>> > > still
>> > > > >>> > > >> > >> > original. The original
>> > > > >>> > > >> > >> > grib1 data files are very strange data
format we
>> can
>> > > not
>> > > > >>> read.
>> > > > >>> > So
>> > > > >>> > > >> we
>> > > > >>> > > >> > >> have
>> > > > >>> > > >> > >> > to use
>> > > > >>> > > >> > >> > a Frotran code to re-arrange the fields and
save
>> them
>> > > as
>> > > > >>> > > NCEP-like
>> > > > >>> > > >> > grib1
>> > > > >>> > > >> > >> > data files.
>> > > > >>> > > >> > >> > So after several processes, some important
info
>> like
>> > > > tables
>> > > > >>> > could
>> > > > >>> > > >> be
>> > > > >>> > > >> > >> lost.
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > I'll test your first suggestion since
the field
>> > > record
>> > > > >>> order
>> > > > >>> > > is
>> > > > >>> > > >> > >> fixed.
>> > > > >>> > > >> > >> > Currently we still
>> > > > >>> > > >> > >> > obtained grib1 data from ECWMF, eventually,
we
>> will
>> > get
>> > > > >>> grib2
>> > > > >>> > > data
>> > > > >>> > > >> > >> > directly. So before
>> > > > >>> > > >> > >> > we can get original grib2 files from ECMWF,
we
>> just
>> > use
>> > > > >>> this
>> > > > >>> > > >> temporal
>> > > > >>> > > >> > >> > solution.
>> > > > >>> > > >> > >> > '
>> > > > >>> > > >> > >> > Thanks!
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John Halley
Gotway
>> via
>> > > RT
>> > > > <
>> > > > >>> > > >> > >> > met_help at ucar.edu>
>> > > > >>> > > >> > >> > wrote:
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > > Binbin,
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > OK, working on Venus.
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > You have a typo in the command you sent.
>> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
>> > a.ps
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >>
>> > > > >>> > > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > > >>> > > >> > >> > -v
>> > > > >>> > > >> > >> > > 3
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
>> > > > "GRIB2_parm_cat".
>> > > > >>> > After
>> > > > >>> > > >> > >> > correcting
>> > > > >>> > > >> > >> > > that I did still get an error but it's a
very
>> > useful
>> > > > one!
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > module use
>> > > > >>> > > >> > >>
>> > > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
>> > > > >>> > > >> > >> > > module load met/9.1
>> > > > >>> > > >> > >> > > module load grib_util/1.1.1
>> > > > >>> > > >> > >> > > cd
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >>
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
>> > > > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
>> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
>> ~/
>> > > > plot.ps
>> > > > >>> > > >> > >> > 'GRIB2_disc=255;
>> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
GRIB2_parm=5;
>> > > > >>> level="P500";'
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > *ERROR : ERROR :
VarInfoGrib2::set_dict() ->
>> no
>> > > > >>> parameter
>> > > > >>> > > found
>> > > > >>> > > >> > with
>> > > > >>> > > >> > >> > > matching GRIB2_disc (255) GRIB2_parm_cat
(3)
>> > > GRIB2_parm
>> > > > >>> (5).
>> > > > >>> > > Use
>> > > > >>> > > >> the
>> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
define
>> > custom
>> > > > >>> GRIB
>> > > > >>> > > >> > >> tables.ERROR
>> > > > >>> > > >> > >> > :
>> > > > >>> > > >> > >> > > *
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > So this table is not yet defined in MET.
I
>> checked
>> > > > their
>> > > > >>> > > website:
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >>
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > But I don't see an entry defined for
>> GRIB2_parm=5;
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > 3 things...
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > (1) As a crude temporary solution, you
could
>> refer
>> > to
>> > > > it
>> > > > >>> by
>> > > > >>> > > it's
>> > > > >>> > > >> > >> record
>> > > > >>> > > >> > >> > > number. Let's assume you know that record
1 has
>> > > > >>> temperature
>> > > > >>> > > data:
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
>> ~/
>> > > > plot.ps
>> > > > >>> > > >> > 'name="TMP";
>> > > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v 5
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > Here I'm reading record 1, naming it TMP
and the
>> > > using
>> > > > >>> > > >> > set_attr_level
>> > > > >>> > > >> > >> > write
>> > > > >>> > > >> > >> > > the output level as P500 instead of R1.
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > (2) You could create a custom GRIB table
>> yourself
>> > > > >>> locally,
>> > > > >>> > and
>> > > > >>> > > >> then
>> > > > >>> > > >> > >> set
>> > > > >>> > > >> > >> > the
>> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
point
>> MET
>> > to
>> > > > it.
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > (3) The best solution is contacting ECMWF
and
>> > > > requesting
>> > > > >>> they
>> > > > >>> > > >> send
>> > > > >>> > > >> > >> over
>> > > > >>> > > >> > >> > the
>> > > > >>> > > >> > >> > > GRIB2 table we should be using to decode
this
>> data.
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > Thanks,
>> > > > >>> > > >> > >> > > John
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John
Halley
>> Gotway <
>> > > > >>> > > >> johnhg at ucar.edu>
>> > > > >>> > > >> > >> > wrote:
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > > > We might already have them. I'm not
sure.
>> > > > >>> > > >> > >> > > >
>> > > > >>> > > >> > >> > > > But there's an additional required
piece of
>> the
>> > > > puzzle.
>> > > > >>> > > Binbin
>> > > > >>> > > >> > runs
>> > > > >>> > > >> > >> > > > "wgrib2" to figure out what "name" he
should
>> use
>> > in
>> > > > >>> the MET
>> > > > >>> > > >> config
>> > > > >>> > > >> > >> > files.
>> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the right
"name",
>> the
>> > > > >>> problem
>> > > > >>> > will
>> > > > >>> > > >> > >> remain.
>> > > > >>> > > >> > >> > > > wgrib2 can be configured to define
additional
>> > > tables
>> > > > as
>> > > > >>> > well,
>> > > > >>> > > >> but
>> > > > >>> > > >> > >> for
>> > > > >>> > > >> > >> > > this
>> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd mean
updating
>> the
>> > > > script
>> > > > >>> > that's
>> > > > >>> > > >> run
>> > > > >>> > > >> > >> for
>> > > > >>> > > >> > >> > > > "module load grib_utils" on WCOSS. And
I'm
>> > guessing
>> > > > >>> that's
>> > > > >>> > > not
>> > > > >>> > > >> > very
>> > > > >>> > > >> > >> > easy.
>> > > > >>> > > >> > >> > > >
>> > > > >>> > > >> > >> > > > Yes, I do agree that the best solution
is
>> getting
>> > > the
>> > > > >>> ECMWF
>> > > > >>> > > >> GRIB
>> > > > >>> > > >> > >> tables
>> > > > >>> > > >> > >> > > > and adding them for both wgrib2 and
MET.
>> > > > >>> > > >> > >> > > >
>> > > > >>> > > >> > >> > > > John
>> > > > >>> > > >> > >> > > >
>> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara
Jensen <
>> > > > >>> > jensen at ucar.edu>
>> > > > >>> > > >> > wrote:
>> > > > >>> > > >> > >> > > >
>> > > > >>> > > >> > >> > > >> John,
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> It seems to me that it would be
worthwhile to
>> > just
>> > > > ask
>> > > > >>> > ECMWF
>> > > > >>> > > >> for
>> > > > >>> > > >> > >> their
>> > > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble files.
And
>> other
>> > > > >>> > operational
>> > > > >>> > > >> > >> centers
>> > > > >>> > > >> > >> > for
>> > > > >>> > > >> > >> > > >> that matter. Do you agree this would
be a
>> good
>> > > > >>> solution?
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> Thanks, Tara
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> ---------- Forwarded message ---------
>> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
>> > > > met_help at ucar.edu>
>> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
>> > > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu #97769]
urgent
>> > > > >>> question for
>> > > > >>> > > >> > reading
>> > > > >>> > > >> > >> > ECMWF
>> > > > >>> > > >> > >> > > >> ensmeble data
>> > > > >>> > > >> > >> > > >> To:
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> <URL:
>> > > > >>> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>> > > > >>> > > >> >
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> John,
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> I tested using following command:
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> plot_data_plane
>> ecme.ens01.t00z.grd3.f12.grib2
>> > > > a.ps
>> > > > >>> > > >> > >> > > >>
>> > > > >>>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > > >>> > > >> -v3
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> ERROR :
>> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either
>> name
>> > > or
>> > > > >>> > > >> GRIB2_disc,
>> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
>> specified
>> > in
>> > > > >>> field
>> > > > >>> > > >> > >> information
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> plot_data_plane
>> ecme.ens01.t00z.grd3.f12.grib2
>> > > > a.ps
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> >
>> > > > >>> > >
>> > > > >>>
>> > > >
>> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
>> > > > >>> > > >> > >> > > >> -v
>> > > > >>> > > >> > >> > > >> 3
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> Still got error
>> > > > >>> > > >> > >> > > >> ERROR :
>> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either
>> name
>> > > or
>> > > > >>> > > >> GRIB2_disc,
>> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must be
>> specified
>> > in
>> > > > >>> field
>> > > > >>> > > >> > >> information
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> Looks like discipline=255 is not
recognized
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> Binbin
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John
Halley
>> > Gotway
>> > > > >>> via RT
>> > > > >>> > <
>> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
>> > > > >>> > > >> > >> > > >> wrote:
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> > Binbin,
>> > > > >>> > > >> > >> > > >> > You can do something similar with
MET.
>> Please
>> > go
>> > > > to
>> > > > >>> this
>> > > > >>> > > >> link:
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >>
>> > > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > And then search for "GRIB2_mtab",
and
>> you'll
>> > > find
>> > > > >>> the
>> > > > >>> > > >> content
>> > > > >>> > > >> > >> > > >> > copied/pasted below. You can specify
these
>> > > INSTEAD
>> > > > >>> OF
>> > > > >>> > > >> > specifying
>> > > > >>> > > >> > >> a
>> > > > >>> > > >> > >> > > >> > requested name.
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an
integer
>> > > > >>> specifying the
>> > > > >>> > > >> master
>> > > > >>> > > >> > >> > table
>> > > > >>> > > >> > >> > > >> > number.
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an
integer
>> > > > >>> specifying the
>> > > > >>> > > >> local
>> > > > >>> > > >> > >> table
>> > > > >>> > > >> > >> > > >> > number.
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an
integer
>> > > > >>> specifying the
>> > > > >>> > > >> GRIB2
>> > > > >>> > > >> > >> > > >> discipline
>> > > > >>> > > >> > >> > > >> > code.
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is
an
>> integer
>> > > > >>> specifying
>> > > > >>> > > the
>> > > > >>> > > >> > >> > parameter
>> > > > >>> > > >> > >> > > >> > category
>> > > > >>> > > >> > >> > > >> > // code.
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an
integer
>> > > > >>> specifying the
>> > > > >>> > > >> > >> parameter
>> > > > >>> > > >> > >> > > code.
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an
integer
>> > > > specifying
>> > > > >>> the
>> > > > >>> > > >> product
>> > > > >>> > > >> > >> > > >> definition
>> > > > >>> > > >> > >> > > >> > // template (Table 4.0).
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_process" is an
>> integer
>> > > > >>> specifying
>> > > > >>> > > the
>> > > > >>> > > >> > >> > generating
>> > > > >>> > > >> > >> > > >> > process
>> > > > >>> > > >> > >> > > >> > // (Table 4.3).
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an
integer
>> > > > >>> specifying the
>> > > > >>> > > >> > >> originating
>> > > > >>> > > >> > >> > > >> > center.
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is
an
>> integer
>> > > > >>> specifying
>> > > > >>> > > the
>> > > > >>> > > >> > >> > ensemble
>> > > > >>> > > >> > >> > > >> type
>> > > > >>> > > >> > >> > > >> > // (Table 4.6).
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is
an
>> integer
>> > > > >>> specifying
>> > > > >>> > > the
>> > > > >>> > > >> > >> derived
>> > > > >>> > > >> > >> > > >> > product
>> > > > >>> > > >> > >> > > >> > // type (Table 4.7).
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type" is
an
>> integer
>> > > > >>> > specifying
>> > > > >>> > > >> the
>> > > > >>> > > >> > >> > > >> statistical
>> > > > >>> > > >> > >> > > >> > // processing type (Table
4.10).
>> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ipdtmpl_index"
and
>> > > > >>> > > "GRIB2_ipdtmpl_val"
>> > > > >>> > > >> > >> entries
>> > > > >>> > > >> > >> > > are
>> > > > >>> > > >> > >> > > >> > arrays
>> > > > >>> > > >> > >> > > >> > // of integers which specify
the
>> > product
>> > > > >>> > > description
>> > > > >>> > > >> > >> > template
>> > > > >>> > > >> > >> > > >> > values to
>> > > > >>> > > >> > >> > > >> > // be used. The indices are
>> 0-based.
>> > For
>> > > > >>> > example,
>> > > > >>> > > >> use
>> > > > >>> > > >> > the
>> > > > >>> > > >> > >> > > >> > following to
>> > > > >>> > > >> > >> > > >> > // request a GRIB2 record
whose
>> 9-th
>> > and
>> > > > >>> 27-th
>> > > > >>> > > >> product
>> > > > >>> > > >> > >> > > >> description
>> > > > >>> > > >> > >> > > >> > // template values are 1 and
2,
>> > > > >>> respectively:
>> > > > >>> > > >> > >> > > >> > //
GRIB2_ipdtmpl_index=[8, 26];
>> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
>> > > > >>> > > >> > >> 2];
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > While I don't have any ECMWF data on
hand
>> for
>> > > > >>> testing, I
>> > > > >>> > > do
>> > > > >>> > > >> > have
>> > > > >>> > > >> > >> a
>> > > > >>> > > >> > >> > > file
>> > > > >>> > > >> > >> > > >> > from the Air Force from which wgrib2
>> prints a
>> > > > >>> similar
>> > > > >>> > > >> result:
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
discipline=0
>> > > > >>> center=57
>> > > > >>> > > >> > >> local_table=1
>> > > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour
fcst:
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > So I used that for testing. I ran
the
>> > following
>> > > > >>> > > >> plot_data_plane
>> > > > >>> > > >> > >> > > command
>> > > > >>> > > >> > >> > > >> > WITHOUT specifying a requested name:
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > plot_data_plane
>> GALWEM_20170610_CY00_FH18.GR2
>> > > > >>> plot.ps
>> > > > >>> > > >> > >> > 'GRIB2_disc=0;
>> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
>> GRIB2_parm_cat=1;
>> > > > >>> > > >> GRIB2_parm=192;
>> > > > >>> > > >> > >> > > >> level="Z0";'
>> > > > >>> > > >> > >> > > >> > -v 3
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > And then I looked closely at the log
>> message
>> > to
>> > > > >>> confirm
>> > > > >>> > > >> that it
>> > > > >>> > > >> > >> used
>> > > > >>> > > >> > >> > > >> > record 549, as expected:
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > DEBUG 3:
MetGrib2DataFile::data_plane() ->
>> > Found
>> > > > >>> exact
>> > > > >>> > > match
>> > > > >>> > > >> > for
>> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549 field
1 of
>> > GRIB2
>> > > > file
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >>
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I
defined
>> to
>> > do
>> > > a
>> > > > >>> table
>> > > > >>> > > >> lookup
>> > > > >>> > > >> > >> and
>> > > > >>> > > >> > >> > > find
>> > > > >>> > > >> > >> > > >> a
>> > > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >>
>> > > > >>> > >
>> > > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > And it determined that that field
should be
>> > > called
>> > > > >>> > > "CTSTM".
>> > > > >>> > > >> > >> > Hopefully,
>> > > > >>> > > >> > >> > > >> the
>> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already
defined
>> there
>> > in
>> > > > >>> MET. If
>> > > > >>> > > >> not, we
>> > > > >>> > > >> > >> can
>> > > > >>> > > >> > >> > > >> always
>> > > > >>> > > >> > >> > > >> > add them.
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > Please try running plot_data_plane
at
>> > verbosity
>> > > > >>> level 3
>> > > > >>> > on
>> > > > >>> > > >> one
>> > > > >>> > > >> > of
>> > > > >>> > > >> > >> > > these
>> > > > >>> > > >> > >> > > >> > files first. And make sure it runs
without
>> > error
>> > > > >>> and the
>> > > > >>> > > log
>> > > > >>> > > >> > >> message
>> > > > >>> > > >> > >> > > >> prints
>> > > > >>> > > >> > >> > > >> > out the "name" it found for that
data. If
>> so,
>> > > you
>> > > > >>> can
>> > > > >>> > just
>> > > > >>> > > >> skip
>> > > > >>> > > >> > >> > using
>> > > > >>> > > >> > >> > > >> all
>> > > > >>> > > >> > >> > > >> > those integers and revert back to
using the
>> > name
>> > > > MET
>> > > > >>> > > knows.
>> > > > >>> > > >> If
>> > > > >>> > > >> > >> not,
>> > > > >>> > > >> > >> > > >> then we
>> > > > >>> > > >> > >> > > >> > may need to define a custom GRIB
table for
>> > this
>> > > > >>> ECMWF
>> > > > >>> > > GRIB2
>> > > > >>> > > >> > data.
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >> > Thanks,
>> > > > >>> > > >> > >> > > >> > John
>> > > > >>> > > >> > >> > > >> >
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> --
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> Binbin Zhou
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> College Park, MD 20740
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> 301-683-3683
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >> --
>> > > > >>> > > >> > >> > > >>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > > > >>> > > >> > >> > > >> Tara Jensen
>> > > > >>> > > >> > >> > > >> Project Manager II
>> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
>> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307
USA
>> > > > >>> > > >> > >> > > >> +1 303-497-8479
jensen at ucar.edu
>> > > > >>> > > >> > >> > > >>
>> > > > >>> > > >> > >> > > >
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> > >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > --
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > Binbin Zhou
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > 5830 University Research Ct.
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > College Park, MD 20740
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> > 301-683-3683
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >> >
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >>
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > --
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > Binbin Zhou
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > 5830 University Research Ct.
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > College Park, MD 20740
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
>> > > > >>> > > >> > >
>> > > > >>> > > >> > > 301-683-3683
>> > > > >>> > > >> > >
>> > > > >>> > > >> >
>> > > > >>> > > >> >
>> > > > >>> > > >> > --
>> > > > >>> > > >> >
>> > > > >>> > > >> > Binbin Zhou
>> > > > >>> > > >> >
>> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>> > > >> >
>> > > > >>> > > >> > 5830 University Research Ct.
>> > > > >>> > > >> >
>> > > > >>> > > >> > College Park, MD 20740
>> > > > >>> > > >> >
>> > > > >>> > > >> > Binbin.Zhou at noaa.gov
>> > > > >>> > > >> >
>> > > > >>> > > >> > 301-683-3683
>> > > > >>> > > >> >
>> > > > >>> > > >> >
>> > > > >>> > > >>
>> > > > >>> > > >>
>> > > > >>> > > >
>> > > > >>> > > > --
>> > > > >>> > > >
>> > > > >>> > > > Binbin Zhou
>> > > > >>> > > >
>> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>> > > >
>> > > > >>> > > > 5830 University Research Ct.
>> > > > >>> > > >
>> > > > >>> > > > College Park, MD 20740
>> > > > >>> > > >
>> > > > >>> > > > Binbin.Zhou at noaa.gov
>> > > > >>> > > >
>> > > > >>> > > > 301-683-3683
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > > --
>> > > > >>> > >
>> > > > >>> > > Binbin Zhou
>> > > > >>> > >
>> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>> > >
>> > > > >>> > > 5830 University Research Ct.
>> > > > >>> > >
>> > > > >>> > > College Park, MD 20740
>> > > > >>> > >
>> > > > >>> > > Binbin.Zhou at noaa.gov
>> > > > >>> > >
>> > > > >>> > > 301-683-3683
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > > >>> --
>> > > > >>>
>> > > > >>> Binbin Zhou
>> > > > >>>
>> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
>> > > > >>>
>> > > > >>> 5830 University Research Ct.
>> > > > >>>
>> > > > >>> College Park, MD 20740
>> > > > >>>
>> > > > >>> Binbin.Zhou at noaa.gov
>> > > > >>>
>> > > > >>> 301-683-3683
>> > > > >>>
>> > > > >>>
>> > > >
>> > > > --
>> > > > Julie Prestopnik (she/her)
>> > > > Software Engineer
>> > > > National Center for Atmospheric Research
>> > > > Research Applications Laboratory
>> > > > Email: jpresto at ucar.edu
>> > > >
>> > > > My working day may not be your working day. Please do not
feel
>> obliged
>> > > to
>> > > > reply to this email outside of your normal working hours.
>> > > >
>> > > >
>> > >
>> > >
>> >
>> > --
>> > Julie Prestopnik (she/her)
>> > Software Engineer
>> > National Center for Atmospheric Research
>> > Research Applications Laboratory
>> > Email: jpresto at ucar.edu
>> >
>> > My working day may not be your working day. Please do not feel
obliged
>> to
>> > reply to this email outside of your normal working hours.
>> >
>> >
>>
>>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Fri Dec 11 10:47:46 2020
Binbin,
Those * characters don't belong in the plot_data_plane command. That's
the
way the mail system formats bold text. To be totally clear, here are
the
exact commands I ran after logging on to Venus to test:
cd /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
module load met/ens_config_test
plot_data_plane ecmanl.t12z.grid3.f00.grib1 ~/plot.ps 'name="HGT";
level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "HGT/P500" in GRIB record 3 of GRIB file
"ecmanl.t12z.grid3.f00.grib1".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
DEBUG 1: Creating postscript file: /u/John.H.Gotway/plot.ps
That matches a single record, namely the 3rd record in that GRIB file.
Do
you see different behavior? Perhaps you failed to load the
"ens_config_test" version of the code before testing?
John
On Fri, Dec 11, 2020 at 10:20 AM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> As I tested following plotting
>
> plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps '*name="HGT";
> level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
>
> the error message is
>
> ERROR :
> ERROR : yyerror() -> syntax error in file "/tmp/met_config_9350_0"
> ERROR :
> ERROR : line = 1
> ERROR :
> ERROR : column = 1
> ERROR :
> ERROR : text = "*"
> ERROR :
> ERROR :
> ERROR : *name="HGT"; level="P500"; lead_time="00";
> GRIB_ens="low_res_ctl";*
> ERROR :
> ^__________________________________________________________________
> ERROR :
>
> After I removed "*"
> plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps 'name="HGT";
> level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> It shows 3 records matched
>
> DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "HGT/P500" in GRIB record 1 of GRIB file
> "ecmanl.t12z.grid3.f00.grib1".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "HGT/P500" in GRIB record 2 of GRIB file
> "ecmanl.t12z.grid3.f00.grib1".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> "ecmanl.t12z.grid3.f00.grib1".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3 GRIB
records
> matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
> DEBUG 1: Creating postscript file: plot.ps
>
> My testing directory on Venus
> is /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> test script is ploting.sh
>
> Can you look at it?
>
> Thanks!
> Binbin
>
>
> On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA Affiliate <
> binbin.zhou at noaa.gov> wrote:
>
> > John,
> >
> > Thanks!
> >
> > Binbin
> >
> > On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Thanks Julie.
> >>
> >> And Binbin, here's an EnsembleStatConfig file where climos are
used:
> >>
> >>
> >>
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
> >>
> >> Since the list of forecast fields is exactly the same as the
climo
> fields,
> >> this config file just copies them over using:
> >> climo_mean = fcst;
> >>
> >> In your case, when they differ, you'd just specify the "field"
array
> >> inside
> >> the "climo_mean" dictionary. For example:
> >>
> >> climo_mean = {
> >>
> >> field = [
> >> { name = "TMP"; level = "Z2"; },
> >> { name = "TMP"; level = "P850"; }
> >> ];
> >> ...
> >> }
> >>
> >> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >> >
> >> > Thanks, John. That worked great.
> >> >
> >> > Binbin, this version of MET is now accessible to you by running
the
> >> > following on venus:
> >> >
> >> > module use
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> >> > module load met/ens_config_test
> >> >
> >> > Please let us know if you encounter any problems.
> >> >
> >> > Julie
> >> >
> >> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > >
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> >> > >
> >> > > Julie,
> >> > >
> >> > > Building the tarfile is very easy. I just ran this script
from the
> MET
> >> > repo
> >> > > and specified the branch to be built:
> >> > > MET/scripts/met_checkout_and_build.sh branch
> >> feature_1604_ens_config
> >> > >
> >> > > And that created a tarfile which I posted to RAL's anonymous
ftp
> >> server:
> >> > >
> >> > >
> >> > >
> >> >
> >>
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> >> > >
> >> > > I assume that'll work for you.
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
> >> > met_help at ucar.edu
> >> > > >
> >> > > wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >> > > >
> >> > > > John,
> >> > > >
> >> > > > I am able to help with this install on WCOSS.
> >> > > >
> >> > > > Would it be easier to use git to checkout the repo on
WCOSS? Or
> >> would
> >> > it
> >> > > be
> >> > > > > easier if I built a tarfile for this version of the code?
> >> > > > >
> >> > > > The last time I tried to build from git on WCOSS (a while
ago), I
> >> > > > couldn't. I don't remember all of the details, but I think
I
> >> couldn't
> >> > > run
> >> > > > bootstrap and maybe it had to do with the version of
autoconf
> being
> >> too
> >> > > > old? I'm happy to try it again if you'd like though.
Otherwise,
> I
> >> > know
> >> > > a
> >> > > > tarfile would work, but I know that is extra work for you.
> >> > > >
> >> > > > Julie
> >> > > >
> >> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <
> johnhg at ucar.edu>
> >> > > wrote:
> >> > > >
> >> > > > > Julie,
> >> > > > >
> >> > > > > I have created a bugfix branch for main_v9.1 named
> >> > > > > "feature_1604_ens_config" and it's up on GitHub.
> >> > > > >
> >> > > > > Only a single file has changed in it. But I need to get
this
> >> version
> >> > > > > compiled on WCOSS with which Binbin can test.
> >> > > > >
> >> > > > > Are you to help me with this? Would it be easier to use
git to
> >> > checkout
> >> > > > > the repo on WCOSS? Or would it be easier if I built a
tarfile
> for
> >> > this
> >> > > > > version of the code?
> >> > > > >
> >> > > > > Thanks,
> >> > > > > John
> >> > > > >
> >> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <
> >> johnhg at ucar.edu>
> >> > > > wrote:
> >> > > > >
> >> > > > >> Binbin,
> >> > > > >>
> >> > > > >> In order for you to test this change out, we'd need to
> recompile
> >> a
> >> > > > >> version of the code on WCOSS. I wrote this up as a
GitHub
> issue:
> >> > > > >> https://github.com/dtcenter/MET/issues/1604
> >> > > > >>
> >> > > > >> And I'll create a branch from main_v9.1 named
> >> > "bugfix_1604_ens_config"
> >> > > > >> and commit the change there. Then we can checkout that
code on
> >> > WCOSS,
> >> > > > >> compile it, and you can test it.
> >> > > > >>
> >> > > > >> Yes, you're right, if the fcst fields differ from the
climo
> >> fields,
> >> > > > you'd
> >> > > > >> need to specify them separately. That's pretty easy in
the
> >> > > Ensemble-Stat
> >> > > > >> config file... but are you asking how to control that
via
> >> METplus?
> >> > > > >>
> >> > > > >> John
> >> > > > >>
> >> > > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov via
RT <
> >> > > > >> met_help at ucar.edu> wrote:
> >> > > > >>
> >> > > > >>>
> >> > > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> >> > > > >>>
> >> > > > >>> John,
> >> > > > >>>
> >> > > > >>> Great! Let me test it at first before make change in
MET9.1
> >> > > > >>> Another question:
> >> > > > >>> Since climatology data is still in NCEP's grib2
format, In
> >> the
> >> > > > >>> ensemble_stat
> >> > > > >>> config file, climo entry settings should be
different from
> >> those
> >> > > of
> >> > > > >>> fcst, obs entries?
> >> > > > >>> How to set climo (name, level, etc.)?
> >> > > > >>>
> >> > > > >>> Binbin
> >> > > > >>>
> >> > > > >>>
> >> > > > >>>
> >> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via
RT <
> >> > > > >>> met_help at ucar.edu>
> >> > > > >>> wrote:
> >> > > > >>>
> >> > > > >>> > Binbin,
> >> > > > >>> >
> >> > > > >>> > Success! When I recompile with that one line
commented out,
> >> > > > >>> plot_data_plane
> >> > > > >>> > is able to filter down all those records to the
single one
> >> that
> >> > we
> >> > > > >>> want.
> >> > > > >>> > Note in the example below that I used the "lead_time"
and
> >> > > "GRIB_ens"
> >> > > > >>> > options to filter the input records down.
> >> > > > >>> >
> >> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
'*name="HGT";
> >> > > > >>> level="P500";
> >> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> >> > > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() ->
*Found
> exact
> >> > match
> >> > > > for
> >> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> >> > > > >>> "ensposte.t00z.z500hr".*
> >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() ->
Found 1
> GRIB
> >> > > records
> >> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> >> "ensposte.t00z.z500hr".
> >> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> >> > > > >>> >
> >> > > > >>> > Also note that this selects out record number 26.
> >> > > > >>> >
> >> > > > >>> > Shall I make this change as a "bugfix" for met-
9.1...which
> >> could
> >> > be
> >> > > > >>> > included in a met-9.1.2 bugfix release? Or shall I
just
> >> include
> >> > it
> >> > > in
> >> > > > >>> > development for the next release (planned for March
2021)?
> >> > > > >>> >
> >> > > > >>> > Thanks,
> >> > > > >>> > John
> >> > > > >>> >
> >> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov
via RT
> <
> >> > > > >>> > met_help at ucar.edu> wrote:
> >> > > > >>> >
> >> > > > >>> > >
> >> > > > >>> > > <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >> > >
> >> > > > >>> > >
> >> > > > >>> > > John,
> >> > > > >>> > >
> >> > > > >>> > > If this removing impacts other applications, or
takes
> some
> >> > time
> >> > > > >>> > > to be effective, then won't try. I will use
another
> >> approach,
> >> > > such
> >> > > > >>> as
> >> > > > >>> > > borrow GEFS's grib2 data structure as tank to save
EC
> >> ensemble
> >> > > > >>> > > data.
> >> > > > >>> > >
> >> > > > >>> > > Binbin
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
> Affiliate
> >> <
> >> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> >> > > > >>> > >
> >> > > > >>> > > > John,
> >> > > > >>> > > >
> >> > > > >>> > > > Yes, please try to use this approach.
> >> > > > >>> > > >
> >> > > > >>> > > > Thanks!
> >> > > > >>> > > >
> >> > > > >>> > > > Binbin
> >> > > > >>> > > >
> >> > > > >>> > > > Binbin
> >> > > > >>> > > >
> >> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley Gotway
via
> RT <
> >> > > > >>> > > > met_help at ucar.edu> wrote:
> >> > > > >>> > > >
> >> > > > >>> > > >> Binbin,
> >> > > > >>> > > >>
> >> > > > >>> > > >> I used the "-PDS10" option for wgrib to dump
those
> >> records:
> >> > > > >>> > > >>
> >> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> >> > > > >>> > > >>
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> >> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> >> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20
12 6 0
> >> 0 1
> >> > 0
> >> > > 0
> >> > > > 1
> >> > > > >>> 0 0
> >> > > > >>> > 0
> >> > > > >>> > > 21
> >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> >> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20
12 6 0
> >> 0 1
> >> > 0
> >> > > 0
> >> > > > 1
> >> > > > >>> 0 0
> >> > > > >>> > 0
> >> > > > >>> > > 21
> >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> >> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244 20
12 6 0
> >> 0 1
> >> > 0
> >> > > 0
> >> > > > 1
> >> > > > >>> 0 0
> >> > > > >>> > 0
> >> > > > >>> > > 21
> >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> >> > > > >>> > > >>
> >> > > > >>> > > >> Looking VERY closely, there is a difference in
the
> 42-nd
> >> > > column
> >> > > > >>> which
> >> > > > >>> > is
> >> > > > >>> > > >> the 4-th from the last... records 11, 26, and 43
have
> >> values
> >> > > of
> >> > > > >>> 3, 1,
> >> > > > >>> > > and
> >> > > > >>> > > >> 2
> >> > > > >>> > > >> respectively.
> >> > > > >>> > > >>
> >> > > > >>> > > >> On this page (
> >> > > > >>> > >
> https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> >> > > > >>> > > >> ),
> >> > > > >>> > > >> byte 42 = 1 means "unperturbed control
forecast".
> >> > > > >>> > > >>
> >> > > > >>> > > >> So we *should* be able to use the "GRIB_ens"
config
> >> option
> >> > to
> >> > > > >>> > > >> differentiate
> >> > > > >>> > > >> between those records. BUT when I try, it
doesn't work.
> >> And
> >> > > > that's
> >> > > > >>> > > because
> >> > > > >>> > > >> of this line:
> >> > > > >>> > > >>
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> >> > > > >>> > > >>
> >> > > > >>> > > >> We only apply those filtering options to GRIB
files
> >> created
> >> > by
> >> > > > >>> NCEP.
> >> > > > >>> > If
> >> > > > >>> > > >> you'd like, I could try removing that condition
to see
> >> if we
> >> > > can
> >> > > > >>> pick
> >> > > > >>> > > out
> >> > > > >>> > > >> the single record of interest.
> >> > > > >>> > > >>
> >> > > > >>> > > >> John
> >> > > > >>> > > >>
> >> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
Binbin.Zhou at noaa.gov
> via
> >> RT
> >> > <
> >> > > > >>> > > >> met_help at ucar.edu> wrote:
> >> > > > >>> > > >>
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > <URL:
> >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >> > > > >
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > John,
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > Look at my
> >> > > > >>> > > >> > directory
> >> > > > >>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> >> > > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this
is only
> >> for
> >> > 500
> >> > > > >>> HGT.
> >> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
> >> anl:type=analysis
> >> > > > >>> > > >> > I get 3 records:
> >> > > > >>> > > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> >> > > > >>> > > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> >> > > > >>> > > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > From its grib2 file, we know only record#26
is
> >> analysis
> >> > > > low-res
> >> > > > >>> > > >> control.
> >> > > > >>> > > >> > If you have some way to determine which record
is
> >> analysis
> >> > > > >>> low-res
> >> > > > >>> > > >> control
> >> > > > >>> > > >> > from the grib1 file,
> >> > > > >>> > > >> > then we can use grib1 directly (don't consider
the
> size
> >> > > > issue).
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > Binbin
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou -
NOAA
> >> > Affiliate
> >> > > <
> >> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > > John,
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > Grib1 data is too large, another issue is
the
> >> low-res
> >> > > > >>> control
> >> > > > >>> > > >> analysis
> >> > > > >>> > > >> > > data can not be retrieved
> >> > > > >>> > > >> > > from the grib1 ecmwf data files. If using
wgrib, we
> >> can
> >> > > see
> >> > > > 3
> >> > > > >>> > > analysis
> >> > > > >>> > > >> > > fields, but don't know
> >> > > > >>> > > >> > > which is low-res control. After convert to
grib2,
> >> wgrib2
> >> > > can
> >> > > > >>> show
> >> > > > >>> > > >> which
> >> > > > >>> > > >> > > record is low-res control.
> >> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1 data
set on
> >> > WCOSS.
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > Binbin
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley
Gotway
> via
> >> > RT <
> >> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > >> Binbin,
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >> Sounds like there's a lot of moving parts
there.
> How
> >> > > would
> >> > > > >>> you
> >> > > > >>> > like
> >> > > > >>> > > >> to
> >> > > > >>> > > >> > >> proceed?
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >> Should we check to see if MET can read
those ECMWF
> >> > GRIB1
> >> > > > >>> files
> >> > > > >>> > > >> directly
> >> > > > >>> > > >> > >> without needing to reformat?
> >> > > > >>> > > >> > >> Or do you just want to create a custom GRIB
table
> >> for
> >> > the
> >> > > > >>> result
> >> > > > >>> > of
> >> > > > >>> > > >> your
> >> > > > >>> > > >> > >> pre-processing and conversion steps?
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >> John
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
> Binbin.Zhou at noaa.gov
> >> > via
> >> > > > RT <
> >> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > <URL:
> >> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >> > > > >>> > >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > John,
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files are
> >> converted
> >> > > from
> >> > > > >>> its
> >> > > > >>> > > grib1
> >> > > > >>> > > >> > >> format
> >> > > > >>> > > >> > >> > data.
> >> > > > >>> > > >> > >> > Before conversion, the parameter names
are
> there,
> >> but
> >> > > > after
> >> > > > >>> > > >> > conversion,
> >> > > > >>> > > >> > >> the
> >> > > > >>> > > >> > >> > names
> >> > > > >>> > > >> > >> > are gone. It is because of unknown
tables. The
> >> grib1
> >> > > data
> >> > > > >>> is
> >> > > > >>> > > still
> >> > > > >>> > > >> > >> > original. The original
> >> > > > >>> > > >> > >> > grib1 data files are very strange data
format we
> >> can
> >> > > not
> >> > > > >>> read.
> >> > > > >>> > So
> >> > > > >>> > > >> we
> >> > > > >>> > > >> > >> have
> >> > > > >>> > > >> > >> > to use
> >> > > > >>> > > >> > >> > a Frotran code to re-arrange the fields
and save
> >> them
> >> > > as
> >> > > > >>> > > NCEP-like
> >> > > > >>> > > >> > grib1
> >> > > > >>> > > >> > >> > data files.
> >> > > > >>> > > >> > >> > So after several processes, some
important info
> >> like
> >> > > > tables
> >> > > > >>> > could
> >> > > > >>> > > >> be
> >> > > > >>> > > >> > >> lost.
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > I'll test your first suggestion since
the
> field
> >> > > record
> >> > > > >>> order
> >> > > > >>> > > is
> >> > > > >>> > > >> > >> fixed.
> >> > > > >>> > > >> > >> > Currently we still
> >> > > > >>> > > >> > >> > obtained grib1 data from ECWMF,
eventually, we
> >> will
> >> > get
> >> > > > >>> grib2
> >> > > > >>> > > data
> >> > > > >>> > > >> > >> > directly. So before
> >> > > > >>> > > >> > >> > we can get original grib2 files from
ECMWF, we
> >> just
> >> > use
> >> > > > >>> this
> >> > > > >>> > > >> temporal
> >> > > > >>> > > >> > >> > solution.
> >> > > > >>> > > >> > >> > '
> >> > > > >>> > > >> > >> > Thanks!
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John
Halley
> Gotway
> >> via
> >> > > RT
> >> > > > <
> >> > > > >>> > > >> > >> > met_help at ucar.edu>
> >> > > > >>> > > >> > >> > wrote:
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > > Binbin,
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > OK, working on Venus.
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > You have a typo in the command you
sent.
> >> > > > >>> > > >> > >> > > plot_data_plane
> ecme.ens01.t00z.grd3.f12.grib2
> >> > a.ps
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >>
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > > > >>> > > >> > >> > -v
> >> > > > >>> > > >> > >> > > 3
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that should be
> >> > > > "GRIB2_parm_cat".
> >> > > > >>> > After
> >> > > > >>> > > >> > >> > correcting
> >> > > > >>> > > >> > >> > > that I did still get an error but it's
a very
> >> > useful
> >> > > > one!
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > module use
> >> > > > >>> > > >> > >>
> >> > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> >> > > > >>> > > >> > >> > > module load met/9.1
> >> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> >> > > > >>> > > >> > >> > > cd
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> >> > > > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> >> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> >> ~/
> >> > > > plot.ps
> >> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> >> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
GRIB2_parm=5;
> >> > > > >>> level="P500";'
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > *ERROR : ERROR :
VarInfoGrib2::set_dict() ->
> >> no
> >> > > > >>> parameter
> >> > > > >>> > > found
> >> > > > >>> > > >> > with
> >> > > > >>> > > >> > >> > > matching GRIB2_disc (255)
GRIB2_parm_cat (3)
> >> > > GRIB2_parm
> >> > > > >>> (5).
> >> > > > >>> > > Use
> >> > > > >>> > > >> the
> >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
define
> >> > custom
> >> > > > >>> GRIB
> >> > > > >>> > > >> > >> tables.ERROR
> >> > > > >>> > > >> > >> > :
> >> > > > >>> > > >> > >> > > *
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > So this table is not yet defined in
MET. I
> >> checked
> >> > > > their
> >> > > > >>> > > website:
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > But I don't see an entry defined for
> >> GRIB2_parm=5;
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > 3 things...
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > (1) As a crude temporary solution, you
could
> >> refer
> >> > to
> >> > > > it
> >> > > > >>> by
> >> > > > >>> > > it's
> >> > > > >>> > > >> > >> record
> >> > > > >>> > > >> > >> > > number. Let's assume you know that
record 1
> has
> >> > > > >>> temperature
> >> > > > >>> > > data:
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > plot_data_plane
ecme.ens01.t00z.grd3.f12.grib2
> >> ~/
> >> > > > plot.ps
> >> > > > >>> > > >> > 'name="TMP";
> >> > > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";' -v
5
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > Here I'm reading record 1, naming it
TMP and
> the
> >> > > using
> >> > > > >>> > > >> > set_attr_level
> >> > > > >>> > > >> > >> > write
> >> > > > >>> > > >> > >> > > the output level as P500 instead of R1.
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > (2) You could create a custom GRIB
table
> >> yourself
> >> > > > >>> locally,
> >> > > > >>> > and
> >> > > > >>> > > >> then
> >> > > > >>> > > >> > >> set
> >> > > > >>> > > >> > >> > the
> >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable to
point
> >> MET
> >> > to
> >> > > > it.
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > (3) The best solution is contacting
ECMWF and
> >> > > > requesting
> >> > > > >>> they
> >> > > > >>> > > >> send
> >> > > > >>> > > >> > >> over
> >> > > > >>> > > >> > >> > the
> >> > > > >>> > > >> > >> > > GRIB2 table we should be using to
decode this
> >> data.
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > Thanks,
> >> > > > >>> > > >> > >> > > John
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John
Halley
> >> Gotway <
> >> > > > >>> > > >> johnhg at ucar.edu>
> >> > > > >>> > > >> > >> > wrote:
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > > > We might already have them. I'm not
sure.
> >> > > > >>> > > >> > >> > > >
> >> > > > >>> > > >> > >> > > > But there's an additional required
piece of
> >> the
> >> > > > puzzle.
> >> > > > >>> > > Binbin
> >> > > > >>> > > >> > runs
> >> > > > >>> > > >> > >> > > > "wgrib2" to figure out what "name" he
should
> >> use
> >> > in
> >> > > > >>> the MET
> >> > > > >>> > > >> config
> >> > > > >>> > > >> > >> > files.
> >> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the right
"name",
> >> the
> >> > > > >>> problem
> >> > > > >>> > will
> >> > > > >>> > > >> > >> remain.
> >> > > > >>> > > >> > >> > > > wgrib2 can be configured to define
> additional
> >> > > tables
> >> > > > as
> >> > > > >>> > well,
> >> > > > >>> > > >> but
> >> > > > >>> > > >> > >> for
> >> > > > >>> > > >> > >> > > this
> >> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd mean
updating
> >> the
> >> > > > script
> >> > > > >>> > that's
> >> > > > >>> > > >> run
> >> > > > >>> > > >> > >> for
> >> > > > >>> > > >> > >> > > > "module load grib_utils" on WCOSS.
And I'm
> >> > guessing
> >> > > > >>> that's
> >> > > > >>> > > not
> >> > > > >>> > > >> > very
> >> > > > >>> > > >> > >> > easy.
> >> > > > >>> > > >> > >> > > >
> >> > > > >>> > > >> > >> > > > Yes, I do agree that the best
solution is
> >> getting
> >> > > the
> >> > > > >>> ECMWF
> >> > > > >>> > > >> GRIB
> >> > > > >>> > > >> > >> tables
> >> > > > >>> > > >> > >> > > > and adding them for both wgrib2 and
MET.
> >> > > > >>> > > >> > >> > > >
> >> > > > >>> > > >> > >> > > > John
> >> > > > >>> > > >> > >> > > >
> >> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara
Jensen <
> >> > > > >>> > jensen at ucar.edu>
> >> > > > >>> > > >> > wrote:
> >> > > > >>> > > >> > >> > > >
> >> > > > >>> > > >> > >> > > >> John,
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> It seems to me that it would be
worthwhile
> to
> >> > just
> >> > > > ask
> >> > > > >>> > ECMWF
> >> > > > >>> > > >> for
> >> > > > >>> > > >> > >> their
> >> > > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble
files. And
> >> other
> >> > > > >>> > operational
> >> > > > >>> > > >> > >> centers
> >> > > > >>> > > >> > >> > for
> >> > > > >>> > > >> > >> > > >> that matter. Do you agree this
would be a
> >> good
> >> > > > >>> solution?
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> Thanks, Tara
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> ---------- Forwarded message
---------
> >> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT <
> >> > > > met_help at ucar.edu>
> >> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> >> > > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu
#97769]
> urgent
> >> > > > >>> question for
> >> > > > >>> > > >> > reading
> >> > > > >>> > > >> > >> > ECMWF
> >> > > > >>> > > >> > >> > > >> ensmeble data
> >> > > > >>> > > >> > >> > > >> To:
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> <URL:
> >> > > > >>> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> John,
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> I tested using following command:
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> plot_data_plane
> >> ecme.ens01.t00z.grd3.f12.grib2
> >> > > > a.ps
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > > > >>> > > >> -v3
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> ERROR :
> >> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either
> >> name
> >> > > or
> >> > > > >>> > > >> GRIB2_disc,
> >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must
be
> >> specified
> >> > in
> >> > > > >>> field
> >> > > > >>> > > >> > >> information
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> plot_data_plane
> >> ecme.ens01.t00z.grd3.f12.grib2
> >> > > > a.ps
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > >
> >> > > > >>>
> >> > > >
> >> >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> >> > > > >>> > > >> > >> > > >> -v
> >> > > > >>> > > >> > >> > > >> 3
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> Still got error
> >> > > > >>> > > >> > >> > > >> ERROR :
> >> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict() ->
either
> >> name
> >> > > or
> >> > > > >>> > > >> GRIB2_disc,
> >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must
be
> >> specified
> >> > in
> >> > > > >>> field
> >> > > > >>> > > >> > >> information
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> Looks like discipline=255 is not
recognized
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> Binbin
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM John
Halley
> >> > Gotway
> >> > > > >>> via RT
> >> > > > >>> > <
> >> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> >> > > > >>> > > >> > >> > > >> wrote:
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> > Binbin,
> >> > > > >>> > > >> > >> > > >> > You can do something similar with
MET.
> >> Please
> >> > go
> >> > > > to
> >> > > > >>> this
> >> > > > >>> > > >> link:
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >>
> >> > >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > And then search for "GRIB2_mtab",
and
> >> you'll
> >> > > find
> >> > > > >>> the
> >> > > > >>> > > >> content
> >> > > > >>> > > >> > >> > > >> > copied/pasted below. You can
specify
> these
> >> > > INSTEAD
> >> > > > >>> OF
> >> > > > >>> > > >> > specifying
> >> > > > >>> > > >> > >> a
> >> > > > >>> > > >> > >> > > >> > requested name.
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is an
integer
> >> > > > >>> specifying the
> >> > > > >>> > > >> master
> >> > > > >>> > > >> > >> > table
> >> > > > >>> > > >> > >> > > >> > number.
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is an
integer
> >> > > > >>> specifying the
> >> > > > >>> > > >> local
> >> > > > >>> > > >> > >> table
> >> > > > >>> > > >> > >> > > >> > number.
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is an
integer
> >> > > > >>> specifying the
> >> > > > >>> > > >> GRIB2
> >> > > > >>> > > >> > >> > > >> discipline
> >> > > > >>> > > >> > >> > > >> > code.
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat" is
an
> >> integer
> >> > > > >>> specifying
> >> > > > >>> > > the
> >> > > > >>> > > >> > >> > parameter
> >> > > > >>> > > >> > >> > > >> > category
> >> > > > >>> > > >> > >> > > >> > // code.
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is an
integer
> >> > > > >>> specifying the
> >> > > > >>> > > >> > >> parameter
> >> > > > >>> > > >> > >> > > code.
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an
integer
> >> > > > specifying
> >> > > > >>> the
> >> > > > >>> > > >> product
> >> > > > >>> > > >> > >> > > >> definition
> >> > > > >>> > > >> > >> > > >> > // template (Table 4.0).
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_process" is
an
> >> integer
> >> > > > >>> specifying
> >> > > > >>> > > the
> >> > > > >>> > > >> > >> > generating
> >> > > > >>> > > >> > >> > > >> > process
> >> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is an
integer
> >> > > > >>> specifying the
> >> > > > >>> > > >> > >> originating
> >> > > > >>> > > >> > >> > > >> > center.
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type" is
an
> >> integer
> >> > > > >>> specifying
> >> > > > >>> > > the
> >> > > > >>> > > >> > >> > ensemble
> >> > > > >>> > > >> > >> > > >> type
> >> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_der_type" is
an
> >> integer
> >> > > > >>> specifying
> >> > > > >>> > > the
> >> > > > >>> > > >> > >> derived
> >> > > > >>> > > >> > >> > > >> > product
> >> > > > >>> > > >> > >> > > >> > // type (Table 4.7).
> >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type"
is an
> >> integer
> >> > > > >>> > specifying
> >> > > > >>> > > >> the
> >> > > > >>> > > >> > >> > > >> statistical
> >> > > > >>> > > >> > >> > > >> > // processing type (Table
4.10).
> >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ipdtmpl_index" and
> >> > > > >>> > > "GRIB2_ipdtmpl_val"
> >> > > > >>> > > >> > >> entries
> >> > > > >>> > > >> > >> > > are
> >> > > > >>> > > >> > >> > > >> > arrays
> >> > > > >>> > > >> > >> > > >> > // of integers which
specify the
> >> > product
> >> > > > >>> > > description
> >> > > > >>> > > >> > >> > template
> >> > > > >>> > > >> > >> > > >> > values to
> >> > > > >>> > > >> > >> > > >> > // be used. The indices
are
> >> 0-based.
> >> > For
> >> > > > >>> > example,
> >> > > > >>> > > >> use
> >> > > > >>> > > >> > the
> >> > > > >>> > > >> > >> > > >> > following to
> >> > > > >>> > > >> > >> > > >> > // request a GRIB2 record
whose
> >> 9-th
> >> > and
> >> > > > >>> 27-th
> >> > > > >>> > > >> product
> >> > > > >>> > > >> > >> > > >> description
> >> > > > >>> > > >> > >> > > >> > // template values are 1
and 2,
> >> > > > >>> respectively:
> >> > > > >>> > > >> > >> > > >> > //
GRIB2_ipdtmpl_index=[8,
> 26];
> >> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> >> > > > >>> > > >> > >> 2];
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > While I don't have any ECMWF data
on hand
> >> for
> >> > > > >>> testing, I
> >> > > > >>> > > do
> >> > > > >>> > > >> > have
> >> > > > >>> > > >> > >> a
> >> > > > >>> > > >> > >> > > file
> >> > > > >>> > > >> > >> > > >> > from the Air Force from which
wgrib2
> >> prints a
> >> > > > >>> similar
> >> > > > >>> > > >> result:
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
> discipline=0
> >> > > > >>> center=57
> >> > > > >>> > > >> > >> local_table=1
> >> > > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18 hour
fcst:
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > So I used that for testing. I ran
the
> >> > following
> >> > > > >>> > > >> plot_data_plane
> >> > > > >>> > > >> > >> > > command
> >> > > > >>> > > >> > >> > > >> > WITHOUT specifying a requested
name:
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > plot_data_plane
> >> GALWEM_20170610_CY00_FH18.GR2
> >> > > > >>> plot.ps
> >> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> >> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
> >> GRIB2_parm_cat=1;
> >> > > > >>> > > >> GRIB2_parm=192;
> >> > > > >>> > > >> > >> > > >> level="Z0";'
> >> > > > >>> > > >> > >> > > >> > -v 3
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > And then I looked closely at the
log
> >> message
> >> > to
> >> > > > >>> confirm
> >> > > > >>> > > >> that it
> >> > > > >>> > > >> > >> used
> >> > > > >>> > > >> > >> > > >> > record 549, as expected:
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > DEBUG 3:
MetGrib2DataFile::data_plane()
> ->
> >> > Found
> >> > > > >>> exact
> >> > > > >>> > > match
> >> > > > >>> > > >> > for
> >> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549
field 1 of
> >> > GRIB2
> >> > > > file
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers I
defined
> >> to
> >> > do
> >> > > a
> >> > > > >>> table
> >> > > > >>> > > >> lookup
> >> > > > >>> > > >> > >> and
> >> > > > >>> > > >> > >> > > find
> >> > > > >>> > > >> > >> > > >> a
> >> > > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >>
> >> > > > >>> > >
> >> > > >
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > And it determined that that field
should
> be
> >> > > called
> >> > > > >>> > > "CTSTM".
> >> > > > >>> > > >> > >> > Hopefully,
> >> > > > >>> > > >> > >> > > >> the
> >> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already
defined
> >> there
> >> > in
> >> > > > >>> MET. If
> >> > > > >>> > > >> not, we
> >> > > > >>> > > >> > >> can
> >> > > > >>> > > >> > >> > > >> always
> >> > > > >>> > > >> > >> > > >> > add them.
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > Please try running plot_data_plane
at
> >> > verbosity
> >> > > > >>> level 3
> >> > > > >>> > on
> >> > > > >>> > > >> one
> >> > > > >>> > > >> > of
> >> > > > >>> > > >> > >> > > these
> >> > > > >>> > > >> > >> > > >> > files first. And make sure it runs
> without
> >> > error
> >> > > > >>> and the
> >> > > > >>> > > log
> >> > > > >>> > > >> > >> message
> >> > > > >>> > > >> > >> > > >> prints
> >> > > > >>> > > >> > >> > > >> > out the "name" it found for that
data. If
> >> so,
> >> > > you
> >> > > > >>> can
> >> > > > >>> > just
> >> > > > >>> > > >> skip
> >> > > > >>> > > >> > >> > using
> >> > > > >>> > > >> > >> > > >> all
> >> > > > >>> > > >> > >> > > >> > those integers and revert back to
using
> the
> >> > name
> >> > > > MET
> >> > > > >>> > > knows.
> >> > > > >>> > > >> If
> >> > > > >>> > > >> > >> not,
> >> > > > >>> > > >> > >> > > >> then we
> >> > > > >>> > > >> > >> > > >> > may need to define a custom GRIB
table
> for
> >> > this
> >> > > > >>> ECMWF
> >> > > > >>> > > GRIB2
> >> > > > >>> > > >> > data.
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >> > Thanks,
> >> > > > >>> > > >> > >> > > >> > John
> >> > > > >>> > > >> > >> > > >> >
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> --
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> Binbin Zhou
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> College Park, MD 20740
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> 301-683-3683
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >> --
> >> > > > >>> > > >> > >> > > >>
> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> > > > >>> > > >> > >> > > >> Tara Jensen
> >> > > > >>> > > >> > >> > > >> Project Manager II
> >> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> >> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado 80307
USA
> >> > > > >>> > > >> > >> > > >> +1 303-497-8479
jensen at ucar.edu
> >> > > > >>> > > >> > >> > > >>
> >> > > > >>> > > >> > >> > > >
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> > >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > --
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > Binbin Zhou
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > 5830 University Research Ct.
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > College Park, MD 20740
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> > 301-683-3683
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >> >
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >>
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > --
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > Binbin Zhou
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > 5830 University Research Ct.
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > College Park, MD 20740
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> > > 301-683-3683
> >> > > > >>> > > >> > >
> >> > > > >>> > > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > --
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > Binbin Zhou
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > 5830 University Research Ct.
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > College Park, MD 20740
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> >> > > > >>> > > >> >
> >> > > > >>> > > >> > 301-683-3683
> >> > > > >>> > > >> >
> >> > > > >>> > > >> >
> >> > > > >>> > > >>
> >> > > > >>> > > >>
> >> > > > >>> > > >
> >> > > > >>> > > > --
> >> > > > >>> > > >
> >> > > > >>> > > > Binbin Zhou
> >> > > > >>> > > >
> >> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>> > > >
> >> > > > >>> > > > 5830 University Research Ct.
> >> > > > >>> > > >
> >> > > > >>> > > > College Park, MD 20740
> >> > > > >>> > > >
> >> > > > >>> > > > Binbin.Zhou at noaa.gov
> >> > > > >>> > > >
> >> > > > >>> > > > 301-683-3683
> >> > > > >>> > > >
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> > > --
> >> > > > >>> > >
> >> > > > >>> > > Binbin Zhou
> >> > > > >>> > >
> >> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>> > >
> >> > > > >>> > > 5830 University Research Ct.
> >> > > > >>> > >
> >> > > > >>> > > College Park, MD 20740
> >> > > > >>> > >
> >> > > > >>> > > Binbin.Zhou at noaa.gov
> >> > > > >>> > >
> >> > > > >>> > > 301-683-3683
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>>
> >> > > > >>> --
> >> > > > >>>
> >> > > > >>> Binbin Zhou
> >> > > > >>>
> >> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >>>
> >> > > > >>> 5830 University Research Ct.
> >> > > > >>>
> >> > > > >>> College Park, MD 20740
> >> > > > >>>
> >> > > > >>> Binbin.Zhou at noaa.gov
> >> > > > >>>
> >> > > > >>> 301-683-3683
> >> > > > >>>
> >> > > > >>>
> >> > > >
> >> > > > --
> >> > > > Julie Prestopnik (she/her)
> >> > > > Software Engineer
> >> > > > National Center for Atmospheric Research
> >> > > > Research Applications Laboratory
> >> > > > Email: jpresto at ucar.edu
> >> > > >
> >> > > > My working day may not be your working day. Please do not
feel
> >> obliged
> >> > > to
> >> > > > reply to this email outside of your normal working hours.
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > Julie Prestopnik (she/her)
> >> > Software Engineer
> >> > National Center for Atmospheric Research
> >> > Research Applications Laboratory
> >> > Email: jpresto at ucar.edu
> >> >
> >> > My working day may not be your working day. Please do not feel
> obliged
> >> to
> >> > reply to this email outside of your normal working hours.
> >> >
> >> >
> >>
> >>
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Fri Dec 11 12:05:06 2020
John,
This it works!
Thanks!
Binbin
On Fri, Dec 11, 2020 at 12:47 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Binbin,
>
> Those * characters don't belong in the plot_data_plane command.
That's the
> way the mail system formats bold text. To be totally clear, here are
the
> exact commands I ran after logging on to Venus to test:
>
> cd /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> module load met/ens_config_test
> plot_data_plane ecmanl.t12z.grid3.f00.grib1 ~/plot.ps 'name="HGT";
> level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
>
> DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> "ecmanl.t12z.grid3.f00.grib1".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
> DEBUG 1: Creating postscript file: /u/John.H.Gotway/plot.ps
>
> That matches a single record, namely the 3rd record in that GRIB
file. Do
> you see different behavior? Perhaps you failed to load the
> "ens_config_test" version of the code before testing?
>
> John
>
> On Fri, Dec 11, 2020 at 10:20 AM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > As I tested following plotting
> >
> > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps '*name="HGT";
> > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> >
> > the error message is
> >
> > ERROR :
> > ERROR : yyerror() -> syntax error in file
"/tmp/met_config_9350_0"
> > ERROR :
> > ERROR : line = 1
> > ERROR :
> > ERROR : column = 1
> > ERROR :
> > ERROR : text = "*"
> > ERROR :
> > ERROR :
> > ERROR : *name="HGT"; level="P500"; lead_time="00";
> > GRIB_ens="low_res_ctl";*
> > ERROR :
> >
^__________________________________________________________________
> > ERROR :
> >
> > After I removed "*"
> > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps 'name="HGT";
> > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > It shows 3 records matched
> >
> > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "HGT/P500" in GRIB record 1 of GRIB file
> > "ecmanl.t12z.grid3.f00.grib1".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "HGT/P500" in GRIB record 2 of GRIB file
> > "ecmanl.t12z.grid3.f00.grib1".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > "ecmanl.t12z.grid3.f00.grib1".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3 GRIB
records
> > matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
> > DEBUG 1: Creating postscript file: plot.ps
> >
> > My testing directory on Venus
> > is /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > test script is ploting.sh
> >
> > Can you look at it?
> >
> > Thanks!
> > Binbin
> >
> >
> > On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA Affiliate <
> > binbin.zhou at noaa.gov> wrote:
> >
> > > John,
> > >
> > > Thanks!
> > >
> > > Binbin
> > >
> > > On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Thanks Julie.
> > >>
> > >> And Binbin, here's an EnsembleStatConfig file where climos are
used:
> > >>
> > >>
> > >>
> >
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
> > >>
> > >> Since the list of forecast fields is exactly the same as the
climo
> > fields,
> > >> this config file just copies them over using:
> > >> climo_mean = fcst;
> > >>
> > >> In your case, when they differ, you'd just specify the "field"
array
> > >> inside
> > >> the "climo_mean" dictionary. For example:
> > >>
> > >> climo_mean = {
> > >>
> > >> field = [
> > >> { name = "TMP"; level = "Z2"; },
> > >> { name = "TMP"; level = "P850"; }
> > >> ];
> > >> ...
> > >> }
> > >>
> > >> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT <
> > >> met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > >> >
> > >> > Thanks, John. That worked great.
> > >> >
> > >> > Binbin, this version of MET is now accessible to you by
running the
> > >> > following on venus:
> > >> >
> > >> > module use
> > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > >> > module load met/ens_config_test
> > >> >
> > >> > Please let us know if you encounter any problems.
> > >> >
> > >> > Julie
> > >> >
> > >> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT <
> > >> > met_help at ucar.edu> wrote:
> > >> >
> > >> > >
> > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >> > >
> > >> > > Julie,
> > >> > >
> > >> > > Building the tarfile is very easy. I just ran this script
from the
> > MET
> > >> > repo
> > >> > > and specified the branch to be built:
> > >> > > MET/scripts/met_checkout_and_build.sh branch
> > >> feature_1604_ens_config
> > >> > >
> > >> > > And that created a tarfile which I posted to RAL's
anonymous ftp
> > >> server:
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> > >> > >
> > >> > > I assume that'll work for you.
> > >> > >
> > >> > > Thanks,
> > >> > > John
> > >> > >
> > >> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
> > >> > met_help at ucar.edu
> > >> > > >
> > >> > > wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >> > > >
> > >> > > > John,
> > >> > > >
> > >> > > > I am able to help with this install on WCOSS.
> > >> > > >
> > >> > > > Would it be easier to use git to checkout the repo on
WCOSS? Or
> > >> would
> > >> > it
> > >> > > be
> > >> > > > > easier if I built a tarfile for this version of the
code?
> > >> > > > >
> > >> > > > The last time I tried to build from git on WCOSS (a while
ago),
> I
> > >> > > > couldn't. I don't remember all of the details, but I
think I
> > >> couldn't
> > >> > > run
> > >> > > > bootstrap and maybe it had to do with the version of
autoconf
> > being
> > >> too
> > >> > > > old? I'm happy to try it again if you'd like though.
> Otherwise,
> > I
> > >> > know
> > >> > > a
> > >> > > > tarfile would work, but I know that is extra work for
you.
> > >> > > >
> > >> > > > Julie
> > >> > > >
> > >> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <
> > johnhg at ucar.edu>
> > >> > > wrote:
> > >> > > >
> > >> > > > > Julie,
> > >> > > > >
> > >> > > > > I have created a bugfix branch for main_v9.1 named
> > >> > > > > "feature_1604_ens_config" and it's up on GitHub.
> > >> > > > >
> > >> > > > > Only a single file has changed in it. But I need to get
this
> > >> version
> > >> > > > > compiled on WCOSS with which Binbin can test.
> > >> > > > >
> > >> > > > > Are you to help me with this? Would it be easier to use
git to
> > >> > checkout
> > >> > > > > the repo on WCOSS? Or would it be easier if I built a
tarfile
> > for
> > >> > this
> > >> > > > > version of the code?
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > John
> > >> > > > >
> > >> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <
> > >> johnhg at ucar.edu>
> > >> > > > wrote:
> > >> > > > >
> > >> > > > >> Binbin,
> > >> > > > >>
> > >> > > > >> In order for you to test this change out, we'd need to
> > recompile
> > >> a
> > >> > > > >> version of the code on WCOSS. I wrote this up as a
GitHub
> > issue:
> > >> > > > >> https://github.com/dtcenter/MET/issues/1604
> > >> > > > >>
> > >> > > > >> And I'll create a branch from main_v9.1 named
> > >> > "bugfix_1604_ens_config"
> > >> > > > >> and commit the change there. Then we can checkout that
code
> on
> > >> > WCOSS,
> > >> > > > >> compile it, and you can test it.
> > >> > > > >>
> > >> > > > >> Yes, you're right, if the fcst fields differ from the
climo
> > >> fields,
> > >> > > > you'd
> > >> > > > >> need to specify them separately. That's pretty easy in
the
> > >> > > Ensemble-Stat
> > >> > > > >> config file... but are you asking how to control that
via
> > >> METplus?
> > >> > > > >>
> > >> > > > >> John
> > >> > > > >>
> > >> > > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov
via RT <
> > >> > > > >> met_help at ucar.edu> wrote:
> > >> > > > >>
> > >> > > > >>>
> > >> > > > >>> <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > >> > > > >>>
> > >> > > > >>> John,
> > >> > > > >>>
> > >> > > > >>> Great! Let me test it at first before make change
in
> MET9.1
> > >> > > > >>> Another question:
> > >> > > > >>> Since climatology data is still in NCEP's grib2
format,
> In
> > >> the
> > >> > > > >>> ensemble_stat
> > >> > > > >>> config file, climo entry settings should be
different
> from
> > >> those
> > >> > > of
> > >> > > > >>> fcst, obs entries?
> > >> > > > >>> How to set climo (name, level, etc.)?
> > >> > > > >>>
> > >> > > > >>> Binbin
> > >> > > > >>>
> > >> > > > >>>
> > >> > > > >>>
> > >> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway via
RT <
> > >> > > > >>> met_help at ucar.edu>
> > >> > > > >>> wrote:
> > >> > > > >>>
> > >> > > > >>> > Binbin,
> > >> > > > >>> >
> > >> > > > >>> > Success! When I recompile with that one line
commented
> out,
> > >> > > > >>> plot_data_plane
> > >> > > > >>> > is able to filter down all those records to the
single one
> > >> that
> > >> > we
> > >> > > > >>> want.
> > >> > > > >>> > Note in the example below that I used the
"lead_time" and
> > >> > > "GRIB_ens"
> > >> > > > >>> > options to filter the input records down.
> > >> > > > >>> >
> > >> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
> '*name="HGT";
> > >> > > > >>> level="P500";
> > >> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > >> > > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() ->
*Found
> > exact
> > >> > match
> > >> > > > for
> > >> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > >> > > > >>> "ensposte.t00z.z500hr".*
> > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() ->
Found 1
> > GRIB
> > >> > > records
> > >> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> > >> "ensposte.t00z.z500hr".
> > >> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > >> > > > >>> >
> > >> > > > >>> > Also note that this selects out record number 26.
> > >> > > > >>> >
> > >> > > > >>> > Shall I make this change as a "bugfix" for met-
9.1...which
> > >> could
> > >> > be
> > >> > > > >>> > included in a met-9.1.2 bugfix release? Or shall I
just
> > >> include
> > >> > it
> > >> > > in
> > >> > > > >>> > development for the next release (planned for March
2021)?
> > >> > > > >>> >
> > >> > > > >>> > Thanks,
> > >> > > > >>> > John
> > >> > > > >>> >
> > >> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM Binbin.Zhou at noaa.gov
via
> RT
> > <
> > >> > > > >>> > met_help at ucar.edu> wrote:
> > >> > > > >>> >
> > >> > > > >>> > >
> > >> > > > >>> > > <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >> > >
> > >> > > > >>> > >
> > >> > > > >>> > > John,
> > >> > > > >>> > >
> > >> > > > >>> > > If this removing impacts other applications, or
takes
> > some
> > >> > time
> > >> > > > >>> > > to be effective, then won't try. I will use
another
> > >> approach,
> > >> > > such
> > >> > > > >>> as
> > >> > > > >>> > > borrow GEFS's grib2 data structure as tank to
save EC
> > >> ensemble
> > >> > > > >>> > > data.
> > >> > > > >>> > >
> > >> > > > >>> > > Binbin
> > >> > > > >>> > >
> > >> > > > >>> > >
> > >> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou - NOAA
> > Affiliate
> > >> <
> > >> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> > >> > > > >>> > >
> > >> > > > >>> > > > John,
> > >> > > > >>> > > >
> > >> > > > >>> > > > Yes, please try to use this approach.
> > >> > > > >>> > > >
> > >> > > > >>> > > > Thanks!
> > >> > > > >>> > > >
> > >> > > > >>> > > > Binbin
> > >> > > > >>> > > >
> > >> > > > >>> > > > Binbin
> > >> > > > >>> > > >
> > >> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley
Gotway via
> > RT <
> > >> > > > >>> > > > met_help at ucar.edu> wrote:
> > >> > > > >>> > > >
> > >> > > > >>> > > >> Binbin,
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> I used the "-PDS10" option for wgrib to dump
those
> > >> records:
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > >> > > > >>> > > >>
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > >> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> > >> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244
20 12
> 6 0
> > >> 0 1
> > >> > 0
> > >> > > 0
> > >> > > > 1
> > >> > > > >>> 0 0
> > >> > > > >>> > 0
> > >> > > > >>> > > 21
> > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > >> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244
20 12
> 6 0
> > >> 0 1
> > >> > 0
> > >> > > 0
> > >> > > > 1
> > >> > > > >>> 0 0
> > >> > > > >>> > 0
> > >> > > > >>> > > 21
> > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > >> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1 244
20 12
> 6 0
> > >> 0 1
> > >> > 0
> > >> > > 0
> > >> > > > 1
> > >> > > > >>> 0 0
> > >> > > > >>> > 0
> > >> > > > >>> > > 21
> > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> Looking VERY closely, there is a difference in
the
> > 42-nd
> > >> > > column
> > >> > > > >>> which
> > >> > > > >>> > is
> > >> > > > >>> > > >> the 4-th from the last... records 11, 26, and
43 have
> > >> values
> > >> > > of
> > >> > > > >>> 3, 1,
> > >> > > > >>> > > and
> > >> > > > >>> > > >> 2
> > >> > > > >>> > > >> respectively.
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> On this page (
> > >> > > > >>> > >
> > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > >> > > > >>> > > >> ),
> > >> > > > >>> > > >> byte 42 = 1 means "unperturbed control
forecast".
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> So we *should* be able to use the "GRIB_ens"
config
> > >> option
> > >> > to
> > >> > > > >>> > > >> differentiate
> > >> > > > >>> > > >> between those records. BUT when I try, it
doesn't
> work.
> > >> And
> > >> > > > that's
> > >> > > > >>> > > because
> > >> > > > >>> > > >> of this line:
> > >> > > > >>> > > >>
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> We only apply those filtering options to GRIB
files
> > >> created
> > >> > by
> > >> > > > >>> NCEP.
> > >> > > > >>> > If
> > >> > > > >>> > > >> you'd like, I could try removing that
condition to
> see
> > >> if we
> > >> > > can
> > >> > > > >>> pick
> > >> > > > >>> > > out
> > >> > > > >>> > > >> the single record of interest.
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> John
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
Binbin.Zhou at noaa.gov
> > via
> > >> RT
> > >> > <
> > >> > > > >>> > > >> met_help at ucar.edu> wrote:
> > >> > > > >>> > > >>
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > <URL:
> > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >> > > > >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > John,
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > Look at my
> > >> > > > >>> > > >> > directory
> > >> > > > >>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > >> > > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr, this
is
> only
> > >> for
> > >> > 500
> > >> > > > >>> HGT.
> > >> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
> > >> anl:type=analysis
> > >> > > > >>> > > >> > I get 3 records:
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > From its grib2 file, we know only record#26
is
> > >> analysis
> > >> > > > low-res
> > >> > > > >>> > > >> control.
> > >> > > > >>> > > >> > If you have some way to determine which
record is
> > >> analysis
> > >> > > > >>> low-res
> > >> > > > >>> > > >> control
> > >> > > > >>> > > >> > from the grib1 file,
> > >> > > > >>> > > >> > then we can use grib1 directly (don't
consider the
> > size
> > >> > > > issue).
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > Binbin
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou -
NOAA
> > >> > Affiliate
> > >> > > <
> > >> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > > John,
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > Grib1 data is too large, another issue
is the
> > >> low-res
> > >> > > > >>> control
> > >> > > > >>> > > >> analysis
> > >> > > > >>> > > >> > > data can not be retrieved
> > >> > > > >>> > > >> > > from the grib1 ecmwf data files. If using
wgrib,
> we
> > >> can
> > >> > > see
> > >> > > > 3
> > >> > > > >>> > > analysis
> > >> > > > >>> > > >> > > fields, but don't know
> > >> > > > >>> > > >> > > which is low-res control. After convert to
grib2,
> > >> wgrib2
> > >> > > can
> > >> > > > >>> show
> > >> > > > >>> > > >> which
> > >> > > > >>> > > >> > > record is low-res control.
> > >> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1
data set
> on
> > >> > WCOSS.
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > Binbin
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John Halley
Gotway
> > via
> > >> > RT <
> > >> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > >> Binbin,
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >> Sounds like there's a lot of moving parts
there.
> > How
> > >> > > would
> > >> > > > >>> you
> > >> > > > >>> > like
> > >> > > > >>> > > >> to
> > >> > > > >>> > > >> > >> proceed?
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >> Should we check to see if MET can read
those
> ECMWF
> > >> > GRIB1
> > >> > > > >>> files
> > >> > > > >>> > > >> directly
> > >> > > > >>> > > >> > >> without needing to reformat?
> > >> > > > >>> > > >> > >> Or do you just want to create a custom
GRIB
> table
> > >> for
> > >> > the
> > >> > > > >>> result
> > >> > > > >>> > of
> > >> > > > >>> > > >> your
> > >> > > > >>> > > >> > >> pre-processing and conversion steps?
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >> John
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
> > Binbin.Zhou at noaa.gov
> > >> > via
> > >> > > > RT <
> > >> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > <URL:
> > >> > > > >>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >> > > > >>> > >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > John,
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files
are
> > >> converted
> > >> > > from
> > >> > > > >>> its
> > >> > > > >>> > > grib1
> > >> > > > >>> > > >> > >> format
> > >> > > > >>> > > >> > >> > data.
> > >> > > > >>> > > >> > >> > Before conversion, the parameter names
are
> > there,
> > >> but
> > >> > > > after
> > >> > > > >>> > > >> > conversion,
> > >> > > > >>> > > >> > >> the
> > >> > > > >>> > > >> > >> > names
> > >> > > > >>> > > >> > >> > are gone. It is because of unknown
tables. The
> > >> grib1
> > >> > > data
> > >> > > > >>> is
> > >> > > > >>> > > still
> > >> > > > >>> > > >> > >> > original. The original
> > >> > > > >>> > > >> > >> > grib1 data files are very strange data
format
> we
> > >> can
> > >> > > not
> > >> > > > >>> read.
> > >> > > > >>> > So
> > >> > > > >>> > > >> we
> > >> > > > >>> > > >> > >> have
> > >> > > > >>> > > >> > >> > to use
> > >> > > > >>> > > >> > >> > a Frotran code to re-arrange the fields
and
> save
> > >> them
> > >> > > as
> > >> > > > >>> > > NCEP-like
> > >> > > > >>> > > >> > grib1
> > >> > > > >>> > > >> > >> > data files.
> > >> > > > >>> > > >> > >> > So after several processes, some
important
> info
> > >> like
> > >> > > > tables
> > >> > > > >>> > could
> > >> > > > >>> > > >> be
> > >> > > > >>> > > >> > >> lost.
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > I'll test your first suggestion
since the
> > field
> > >> > > record
> > >> > > > >>> order
> > >> > > > >>> > > is
> > >> > > > >>> > > >> > >> fixed.
> > >> > > > >>> > > >> > >> > Currently we still
> > >> > > > >>> > > >> > >> > obtained grib1 data from ECWMF,
eventually, we
> > >> will
> > >> > get
> > >> > > > >>> grib2
> > >> > > > >>> > > data
> > >> > > > >>> > > >> > >> > directly. So before
> > >> > > > >>> > > >> > >> > we can get original grib2 files from
ECMWF, we
> > >> just
> > >> > use
> > >> > > > >>> this
> > >> > > > >>> > > >> temporal
> > >> > > > >>> > > >> > >> > solution.
> > >> > > > >>> > > >> > >> > '
> > >> > > > >>> > > >> > >> > Thanks!
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John
Halley
> > Gotway
> > >> via
> > >> > > RT
> > >> > > > <
> > >> > > > >>> > > >> > >> > met_help at ucar.edu>
> > >> > > > >>> > > >> > >> > wrote:
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > > Binbin,
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > OK, working on Venus.
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > You have a typo in the command you
sent.
> > >> > > > >>> > > >> > >> > > plot_data_plane
> > ecme.ens01.t00z.grd3.f12.grib2
> > >> > a.ps
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >>
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > > > >>> > > >> > >> > -v
> > >> > > > >>> > > >> > >> > > 3
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that should
be
> > >> > > > "GRIB2_parm_cat".
> > >> > > > >>> > After
> > >> > > > >>> > > >> > >> > correcting
> > >> > > > >>> > > >> > >> > > that I did still get an error but
it's a
> very
> > >> > useful
> > >> > > > one!
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > module use
> > >> > > > >>> > > >> > >>
> > >> > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > >> > > > >>> > > >> > >> > > module load met/9.1
> > >> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> > >> > > > >>> > > >> > >> > > cd
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > >> > > > >>> > > >> > >> > > $WGRIB2 ensposte.t00z.z500.grd3.grib2
> > >> > > > >>> > > >> > >> > > plot_data_plane
> ecme.ens01.t00z.grd3.f12.grib2
> > >> ~/
> > >> > > > plot.ps
> > >> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> > >> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
> GRIB2_parm=5;
> > >> > > > >>> level="P500";'
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > *ERROR : ERROR :
VarInfoGrib2::set_dict()
> ->
> > >> no
> > >> > > > >>> parameter
> > >> > > > >>> > > found
> > >> > > > >>> > > >> > with
> > >> > > > >>> > > >> > >> > > matching GRIB2_disc (255)
GRIB2_parm_cat (3)
> > >> > > GRIB2_parm
> > >> > > > >>> (5).
> > >> > > > >>> > > Use
> > >> > > > >>> > > >> the
> > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable
to
> define
> > >> > custom
> > >> > > > >>> GRIB
> > >> > > > >>> > > >> > >> tables.ERROR
> > >> > > > >>> > > >> > >> > :
> > >> > > > >>> > > >> > >> > > *
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > So this table is not yet defined in
MET. I
> > >> checked
> > >> > > > their
> > >> > > > >>> > > website:
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > But I don't see an entry defined for
> > >> GRIB2_parm=5;
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > 3 things...
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > (1) As a crude temporary solution,
you could
> > >> refer
> > >> > to
> > >> > > > it
> > >> > > > >>> by
> > >> > > > >>> > > it's
> > >> > > > >>> > > >> > >> record
> > >> > > > >>> > > >> > >> > > number. Let's assume you know that
record 1
> > has
> > >> > > > >>> temperature
> > >> > > > >>> > > data:
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > plot_data_plane
> ecme.ens01.t00z.grd3.f12.grib2
> > >> ~/
> > >> > > > plot.ps
> > >> > > > >>> > > >> > 'name="TMP";
> > >> > > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";'
-v 5
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > Here I'm reading record 1, naming it
TMP and
> > the
> > >> > > using
> > >> > > > >>> > > >> > set_attr_level
> > >> > > > >>> > > >> > >> > write
> > >> > > > >>> > > >> > >> > > the output level as P500 instead of
R1.
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > (2) You could create a custom GRIB
table
> > >> yourself
> > >> > > > >>> locally,
> > >> > > > >>> > and
> > >> > > > >>> > > >> then
> > >> > > > >>> > > >> > >> set
> > >> > > > >>> > > >> > >> > the
> > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment variable
to
> point
> > >> MET
> > >> > to
> > >> > > > it.
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > (3) The best solution is contacting
ECMWF
> and
> > >> > > > requesting
> > >> > > > >>> they
> > >> > > > >>> > > >> send
> > >> > > > >>> > > >> > >> over
> > >> > > > >>> > > >> > >> > the
> > >> > > > >>> > > >> > >> > > GRIB2 table we should be using to
decode
> this
> > >> data.
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > Thanks,
> > >> > > > >>> > > >> > >> > > John
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John
Halley
> > >> Gotway <
> > >> > > > >>> > > >> johnhg at ucar.edu>
> > >> > > > >>> > > >> > >> > wrote:
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > > > We might already have them. I'm not
sure.
> > >> > > > >>> > > >> > >> > > >
> > >> > > > >>> > > >> > >> > > > But there's an additional required
piece
> of
> > >> the
> > >> > > > puzzle.
> > >> > > > >>> > > Binbin
> > >> > > > >>> > > >> > runs
> > >> > > > >>> > > >> > >> > > > "wgrib2" to figure out what "name"
he
> should
> > >> use
> > >> > in
> > >> > > > >>> the MET
> > >> > > > >>> > > >> config
> > >> > > > >>> > > >> > >> > files.
> > >> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the
right
> "name",
> > >> the
> > >> > > > >>> problem
> > >> > > > >>> > will
> > >> > > > >>> > > >> > >> remain.
> > >> > > > >>> > > >> > >> > > > wgrib2 can be configured to define
> > additional
> > >> > > tables
> > >> > > > as
> > >> > > > >>> > well,
> > >> > > > >>> > > >> but
> > >> > > > >>> > > >> > >> for
> > >> > > > >>> > > >> > >> > > this
> > >> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd mean
> updating
> > >> the
> > >> > > > script
> > >> > > > >>> > that's
> > >> > > > >>> > > >> run
> > >> > > > >>> > > >> > >> for
> > >> > > > >>> > > >> > >> > > > "module load grib_utils" on WCOSS.
And I'm
> > >> > guessing
> > >> > > > >>> that's
> > >> > > > >>> > > not
> > >> > > > >>> > > >> > very
> > >> > > > >>> > > >> > >> > easy.
> > >> > > > >>> > > >> > >> > > >
> > >> > > > >>> > > >> > >> > > > Yes, I do agree that the best
solution is
> > >> getting
> > >> > > the
> > >> > > > >>> ECMWF
> > >> > > > >>> > > >> GRIB
> > >> > > > >>> > > >> > >> tables
> > >> > > > >>> > > >> > >> > > > and adding them for both wgrib2 and
MET.
> > >> > > > >>> > > >> > >> > > >
> > >> > > > >>> > > >> > >> > > > John
> > >> > > > >>> > > >> > >> > > >
> > >> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM Tara
> Jensen <
> > >> > > > >>> > jensen at ucar.edu>
> > >> > > > >>> > > >> > wrote:
> > >> > > > >>> > > >> > >> > > >
> > >> > > > >>> > > >> > >> > > >> John,
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> It seems to me that it would be
> worthwhile
> > to
> > >> > just
> > >> > > > ask
> > >> > > > >>> > ECMWF
> > >> > > > >>> > > >> for
> > >> > > > >>> > > >> > >> their
> > >> > > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble
files.
> And
> > >> other
> > >> > > > >>> > operational
> > >> > > > >>> > > >> > >> centers
> > >> > > > >>> > > >> > >> > for
> > >> > > > >>> > > >> > >> > > >> that matter. Do you agree this
would be
> a
> > >> good
> > >> > > > >>> solution?
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> Thanks, Tara
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> ---------- Forwarded message
---------
> > >> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via RT
<
> > >> > > > met_help at ucar.edu>
> > >> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06 AM
> > >> > > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu
#97769]
> > urgent
> > >> > > > >>> question for
> > >> > > > >>> > > >> > reading
> > >> > > > >>> > > >> > >> > ECMWF
> > >> > > > >>> > > >> > >> > > >> ensmeble data
> > >> > > > >>> > > >> > >> > > >> To:
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> <URL:
> > >> > > > >>> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> John,
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> I tested using following
command:
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > >> ecme.ens01.t00z.grd3.f12.grib2
> > >> > > > a.ps
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > > > >>> > > >> -v3
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> ERROR :
> > >> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict()
->
> either
> > >> name
> > >> > > or
> > >> > > > >>> > > >> GRIB2_disc,
> > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must
be
> > >> specified
> > >> > in
> > >> > > > >>> field
> > >> > > > >>> > > >> > >> information
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > >> ecme.ens01.t00z.grd3.f12.grib2
> > >> > > > a.ps
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > >
> > >> > > > >>>
> > >> > > >
> > >> >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > >> > > > >>> > > >> > >> > > >> -v
> > >> > > > >>> > > >> > >> > > >> 3
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> Still got error
> > >> > > > >>> > > >> > >> > > >> ERROR :
> > >> > > > >>> > > >> > >> > > >> ERROR : VarInfoGrib2::set_dict()
->
> either
> > >> name
> > >> > > or
> > >> > > > >>> > > >> GRIB2_disc,
> > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm must
be
> > >> specified
> > >> > in
> > >> > > > >>> field
> > >> > > > >>> > > >> > >> information
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> Looks like discipline=255 is not
> recognized
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> Binbin
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM
John
> Halley
> > >> > Gotway
> > >> > > > >>> via RT
> > >> > > > >>> > <
> > >> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > >> > > > >>> > > >> > >> > > >> wrote:
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> > Binbin,
> > >> > > > >>> > > >> > >> > > >> > You can do something similar
with MET.
> > >> Please
> > >> > go
> > >> > > > to
> > >> > > > >>> this
> > >> > > > >>> > > >> link:
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >>
> > >> > >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > And then search for
"GRIB2_mtab", and
> > >> you'll
> > >> > > find
> > >> > > > >>> the
> > >> > > > >>> > > >> content
> > >> > > > >>> > > >> > >> > > >> > copied/pasted below. You can
specify
> > these
> > >> > > INSTEAD
> > >> > > > >>> OF
> > >> > > > >>> > > >> > specifying
> > >> > > > >>> > > >> > >> a
> > >> > > > >>> > > >> > >> > > >> > requested name.
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is
an
> integer
> > >> > > > >>> specifying the
> > >> > > > >>> > > >> master
> > >> > > > >>> > > >> > >> > table
> > >> > > > >>> > > >> > >> > > >> > number.
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is
an
> integer
> > >> > > > >>> specifying the
> > >> > > > >>> > > >> local
> > >> > > > >>> > > >> > >> table
> > >> > > > >>> > > >> > >> > > >> > number.
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is
an
> integer
> > >> > > > >>> specifying the
> > >> > > > >>> > > >> GRIB2
> > >> > > > >>> > > >> > >> > > >> discipline
> > >> > > > >>> > > >> > >> > > >> > code.
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm_cat"
is an
> > >> integer
> > >> > > > >>> specifying
> > >> > > > >>> > > the
> > >> > > > >>> > > >> > >> > parameter
> > >> > > > >>> > > >> > >> > > >> > category
> > >> > > > >>> > > >> > >> > > >> > // code.
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is
an
> integer
> > >> > > > >>> specifying the
> > >> > > > >>> > > >> > >> parameter
> > >> > > > >>> > > >> > >> > > code.
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is an
> integer
> > >> > > > specifying
> > >> > > > >>> the
> > >> > > > >>> > > >> product
> > >> > > > >>> > > >> > >> > > >> definition
> > >> > > > >>> > > >> > >> > > >> > // template (Table 4.0).
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_process"
is an
> > >> integer
> > >> > > > >>> specifying
> > >> > > > >>> > > the
> > >> > > > >>> > > >> > >> > generating
> > >> > > > >>> > > >> > >> > > >> > process
> > >> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is
an
> integer
> > >> > > > >>> specifying the
> > >> > > > >>> > > >> > >> originating
> > >> > > > >>> > > >> > >> > > >> > center.
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ens_type"
is an
> > >> integer
> > >> > > > >>> specifying
> > >> > > > >>> > > the
> > >> > > > >>> > > >> > >> > ensemble
> > >> > > > >>> > > >> > >> > > >> type
> > >> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_der_type"
is an
> > >> integer
> > >> > > > >>> specifying
> > >> > > > >>> > > the
> > >> > > > >>> > > >> > >> derived
> > >> > > > >>> > > >> > >> > > >> > product
> > >> > > > >>> > > >> > >> > > >> > // type (Table 4.7).
> > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_stat_type"
is an
> > >> integer
> > >> > > > >>> > specifying
> > >> > > > >>> > > >> the
> > >> > > > >>> > > >> > >> > > >> statistical
> > >> > > > >>> > > >> > >> > > >> > // processing type
(Table
> 4.10).
> > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ipdtmpl_index"
> and
> > >> > > > >>> > > "GRIB2_ipdtmpl_val"
> > >> > > > >>> > > >> > >> entries
> > >> > > > >>> > > >> > >> > > are
> > >> > > > >>> > > >> > >> > > >> > arrays
> > >> > > > >>> > > >> > >> > > >> > // of integers which
specify
> the
> > >> > product
> > >> > > > >>> > > description
> > >> > > > >>> > > >> > >> > template
> > >> > > > >>> > > >> > >> > > >> > values to
> > >> > > > >>> > > >> > >> > > >> > // be used. The indices
are
> > >> 0-based.
> > >> > For
> > >> > > > >>> > example,
> > >> > > > >>> > > >> use
> > >> > > > >>> > > >> > the
> > >> > > > >>> > > >> > >> > > >> > following to
> > >> > > > >>> > > >> > >> > > >> > // request a GRIB2
record whose
> > >> 9-th
> > >> > and
> > >> > > > >>> 27-th
> > >> > > > >>> > > >> product
> > >> > > > >>> > > >> > >> > > >> description
> > >> > > > >>> > > >> > >> > > >> > // template values are 1
and 2,
> > >> > > > >>> respectively:
> > >> > > > >>> > > >> > >> > > >> > //
GRIB2_ipdtmpl_index=[8,
> > 26];
> > >> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > >> > > > >>> > > >> > >> 2];
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > While I don't have any ECMWF
data on
> hand
> > >> for
> > >> > > > >>> testing, I
> > >> > > > >>> > > do
> > >> > > > >>> > > >> > have
> > >> > > > >>> > > >> > >> a
> > >> > > > >>> > > >> > >> > > file
> > >> > > > >>> > > >> > >> > > >> > from the Air Force from which
wgrib2
> > >> prints a
> > >> > > > >>> similar
> > >> > > > >>> > > >> result:
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
> > discipline=0
> > >> > > > >>> center=57
> > >> > > > >>> > > >> > >> local_table=1
> > >> > > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18
hour
> fcst:
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > So I used that for testing. I
ran the
> > >> > following
> > >> > > > >>> > > >> plot_data_plane
> > >> > > > >>> > > >> > >> > > command
> > >> > > > >>> > > >> > >> > > >> > WITHOUT specifying a requested
name:
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > plot_data_plane
> > >> GALWEM_20170610_CY00_FH18.GR2
> > >> > > > >>> plot.ps
> > >> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> > >> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
> > >> GRIB2_parm_cat=1;
> > >> > > > >>> > > >> GRIB2_parm=192;
> > >> > > > >>> > > >> > >> > > >> level="Z0";'
> > >> > > > >>> > > >> > >> > > >> > -v 3
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > And then I looked closely at the
log
> > >> message
> > >> > to
> > >> > > > >>> confirm
> > >> > > > >>> > > >> that it
> > >> > > > >>> > > >> > >> used
> > >> > > > >>> > > >> > >> > > >> > record 549, as expected:
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > DEBUG 3:
MetGrib2DataFile::data_plane()
> > ->
> > >> > Found
> > >> > > > >>> exact
> > >> > > > >>> > > match
> > >> > > > >>> > > >> > for
> > >> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549
field 1
> of
> > >> > GRIB2
> > >> > > > file
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > So MET used these GRIB2 numbers
I
> defined
> > >> to
> > >> > do
> > >> > > a
> > >> > > > >>> table
> > >> > > > >>> > > >> lookup
> > >> > > > >>> > > >> > >> and
> > >> > > > >>> > > >> > >> > > find
> > >> > > > >>> > > >> > >> > > >> a
> > >> > > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > >
> > >> > > >
> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > And it determined that that
field
> should
> > be
> > >> > > called
> > >> > > > >>> > > "CTSTM".
> > >> > > > >>> > > >> > >> > Hopefully,
> > >> > > > >>> > > >> > >> > > >> the
> > >> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already
defined
> > >> there
> > >> > in
> > >> > > > >>> MET. If
> > >> > > > >>> > > >> not, we
> > >> > > > >>> > > >> > >> can
> > >> > > > >>> > > >> > >> > > >> always
> > >> > > > >>> > > >> > >> > > >> > add them.
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > Please try running
plot_data_plane at
> > >> > verbosity
> > >> > > > >>> level 3
> > >> > > > >>> > on
> > >> > > > >>> > > >> one
> > >> > > > >>> > > >> > of
> > >> > > > >>> > > >> > >> > > these
> > >> > > > >>> > > >> > >> > > >> > files first. And make sure it
runs
> > without
> > >> > error
> > >> > > > >>> and the
> > >> > > > >>> > > log
> > >> > > > >>> > > >> > >> message
> > >> > > > >>> > > >> > >> > > >> prints
> > >> > > > >>> > > >> > >> > > >> > out the "name" it found for that
data.
> If
> > >> so,
> > >> > > you
> > >> > > > >>> can
> > >> > > > >>> > just
> > >> > > > >>> > > >> skip
> > >> > > > >>> > > >> > >> > using
> > >> > > > >>> > > >> > >> > > >> all
> > >> > > > >>> > > >> > >> > > >> > those integers and revert back
to using
> > the
> > >> > name
> > >> > > > MET
> > >> > > > >>> > > knows.
> > >> > > > >>> > > >> If
> > >> > > > >>> > > >> > >> not,
> > >> > > > >>> > > >> > >> > > >> then we
> > >> > > > >>> > > >> > >> > > >> > may need to define a custom GRIB
table
> > for
> > >> > this
> > >> > > > >>> ECMWF
> > >> > > > >>> > > GRIB2
> > >> > > > >>> > > >> > data.
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >> > Thanks,
> > >> > > > >>> > > >> > >> > > >> > John
> > >> > > > >>> > > >> > >> > > >> >
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> --
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> Binbin Zhou
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> College Park, MD 20740
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> 301-683-3683
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >> --
> > >> > > > >>> > > >> > >> > > >>
> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> > > > >>> > > >> > >> > > >> Tara Jensen
> > >> > > > >>> > > >> > >> > > >> Project Manager II
> > >> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > >> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado
80307 USA
> > >> > > > >>> > > >> > >> > > >> +1 303-497-8479
jensen at ucar.edu
> > >> > > > >>> > > >> > >> > > >>
> > >> > > > >>> > > >> > >> > > >
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> > >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > --
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > Binbin Zhou
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > 5830 University Research Ct.
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > College Park, MD 20740
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> > 301-683-3683
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >> >
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >>
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > --
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > Binbin Zhou
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > 5830 University Research Ct.
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > College Park, MD 20740
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> > > 301-683-3683
> > >> > > > >>> > > >> > >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > --
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > Binbin Zhou
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > 5830 University Research Ct.
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > College Park, MD 20740
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> > 301-683-3683
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >> >
> > >> > > > >>> > > >>
> > >> > > > >>> > > >>
> > >> > > > >>> > > >
> > >> > > > >>> > > > --
> > >> > > > >>> > > >
> > >> > > > >>> > > > Binbin Zhou
> > >> > > > >>> > > >
> > >> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>> > > >
> > >> > > > >>> > > > 5830 University Research Ct.
> > >> > > > >>> > > >
> > >> > > > >>> > > > College Park, MD 20740
> > >> > > > >>> > > >
> > >> > > > >>> > > > Binbin.Zhou at noaa.gov
> > >> > > > >>> > > >
> > >> > > > >>> > > > 301-683-3683
> > >> > > > >>> > > >
> > >> > > > >>> > >
> > >> > > > >>> > >
> > >> > > > >>> > > --
> > >> > > > >>> > >
> > >> > > > >>> > > Binbin Zhou
> > >> > > > >>> > >
> > >> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>> > >
> > >> > > > >>> > > 5830 University Research Ct.
> > >> > > > >>> > >
> > >> > > > >>> > > College Park, MD 20740
> > >> > > > >>> > >
> > >> > > > >>> > > Binbin.Zhou at noaa.gov
> > >> > > > >>> > >
> > >> > > > >>> > > 301-683-3683
> > >> > > > >>> > >
> > >> > > > >>> > >
> > >> > > > >>> >
> > >> > > > >>> >
> > >> > > > >>>
> > >> > > > >>> --
> > >> > > > >>>
> > >> > > > >>> Binbin Zhou
> > >> > > > >>>
> > >> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >>>
> > >> > > > >>> 5830 University Research Ct.
> > >> > > > >>>
> > >> > > > >>> College Park, MD 20740
> > >> > > > >>>
> > >> > > > >>> Binbin.Zhou at noaa.gov
> > >> > > > >>>
> > >> > > > >>> 301-683-3683
> > >> > > > >>>
> > >> > > > >>>
> > >> > > >
> > >> > > > --
> > >> > > > Julie Prestopnik (she/her)
> > >> > > > Software Engineer
> > >> > > > National Center for Atmospheric Research
> > >> > > > Research Applications Laboratory
> > >> > > > Email: jpresto at ucar.edu
> > >> > > >
> > >> > > > My working day may not be your working day. Please do
not feel
> > >> obliged
> > >> > > to
> > >> > > > reply to this email outside of your normal working hours.
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> > --
> > >> > Julie Prestopnik (she/her)
> > >> > Software Engineer
> > >> > National Center for Atmospheric Research
> > >> > Research Applications Laboratory
> > >> > Email: jpresto at ucar.edu
> > >> >
> > >> > My working day may not be your working day. Please do not
feel
> > obliged
> > >> to
> > >> > reply to this email outside of your normal working hours.
> > >> >
> > >> >
> > >>
> > >>
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Fri Dec 11 12:54:42 2020
Binbin,
Great, thanks for confirming. OK, next question is about versions and
timelines.
Do we need to create a met-9.1.2 bugfix version that includes this
change?
Or is it sufficient to just include in the met-10.0.0 release?
If we need to create a met-9.1.2 bugfix release, do you have a
deadline for
that?
Thanks,
John
On Fri, Dec 11, 2020 at 12:05 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> This it works!
>
> Thanks!
>
> Binbin
>
> On Fri, Dec 11, 2020 at 12:47 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Binbin,
> >
> > Those * characters don't belong in the plot_data_plane command.
That's
> the
> > way the mail system formats bold text. To be totally clear, here
are the
> > exact commands I ran after logging on to Venus to test:
> >
> > cd /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > module load met/ens_config_test
> > plot_data_plane ecmanl.t12z.grid3.f00.grib1 ~/plot.ps 'name="HGT";
> > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> >
> > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > "ecmanl.t12z.grid3.f00.grib1".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
> > DEBUG 1: Creating postscript file: /u/John.H.Gotway/plot.ps
> >
> > That matches a single record, namely the 3rd record in that GRIB
file. Do
> > you see different behavior? Perhaps you failed to load the
> > "ens_config_test" version of the code before testing?
> >
> > John
> >
> > On Fri, Dec 11, 2020 at 10:20 AM Binbin.Zhou at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >
> > > John,
> > >
> > > As I tested following plotting
> > >
> > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
'*name="HGT";
> > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > >
> > > the error message is
> > >
> > > ERROR :
> > > ERROR : yyerror() -> syntax error in file
"/tmp/met_config_9350_0"
> > > ERROR :
> > > ERROR : line = 1
> > > ERROR :
> > > ERROR : column = 1
> > > ERROR :
> > > ERROR : text = "*"
> > > ERROR :
> > > ERROR :
> > > ERROR : *name="HGT"; level="P500"; lead_time="00";
> > > GRIB_ens="low_res_ctl";*
> > > ERROR :
> > >
^__________________________________________________________________
> > > ERROR :
> > >
> > > After I removed "*"
> > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
'name="HGT";
> > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > > It shows 3 records matched
> > >
> > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match for
> > > VarInfo "HGT/P500" in GRIB record 1 of GRIB file
> > > "ecmanl.t12z.grid3.f00.grib1".
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match for
> > > VarInfo "HGT/P500" in GRIB record 2 of GRIB file
> > > "ecmanl.t12z.grid3.f00.grib1".
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match for
> > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > "ecmanl.t12z.grid3.f00.grib1".
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3 GRIB
records
> > > matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
> > > DEBUG 1: Creating postscript file: plot.ps
> > >
> > > My testing directory on Venus
> > > is
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > test script is ploting.sh
> > >
> > > Can you look at it?
> > >
> > > Thanks!
> > > Binbin
> > >
> > >
> > > On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA Affiliate <
> > > binbin.zhou at noaa.gov> wrote:
> > >
> > > > John,
> > > >
> > > > Thanks!
> > > >
> > > > Binbin
> > > >
> > > > On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Thanks Julie.
> > > >>
> > > >> And Binbin, here's an EnsembleStatConfig file where climos
are used:
> > > >>
> > > >>
> > > >>
> > >
> >
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
> > > >>
> > > >> Since the list of forecast fields is exactly the same as the
climo
> > > fields,
> > > >> this config file just copies them over using:
> > > >> climo_mean = fcst;
> > > >>
> > > >> In your case, when they differ, you'd just specify the
"field" array
> > > >> inside
> > > >> the "climo_mean" dictionary. For example:
> > > >>
> > > >> climo_mean = {
> > > >>
> > > >> field = [
> > > >> { name = "TMP"; level = "Z2"; },
> > > >> { name = "TMP"; level = "P850"; }
> > > >> ];
> > > >> ...
> > > >> }
> > > >>
> > > >> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT <
> > > >> met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > >> >
> > > >> > Thanks, John. That worked great.
> > > >> >
> > > >> > Binbin, this version of MET is now accessible to you by
running
> the
> > > >> > following on venus:
> > > >> >
> > > >> > module use
> > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > >> > module load met/ens_config_test
> > > >> >
> > > >> > Please let us know if you encounter any problems.
> > > >> >
> > > >> > Julie
> > > >> >
> > > >> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via RT
<
> > > >> > met_help at ucar.edu> wrote:
> > > >> >
> > > >> > >
> > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > >> > >
> > > >> > > Julie,
> > > >> > >
> > > >> > > Building the tarfile is very easy. I just ran this script
from
> the
> > > MET
> > > >> > repo
> > > >> > > and specified the branch to be built:
> > > >> > > MET/scripts/met_checkout_and_build.sh branch
> > > >> feature_1604_ens_config
> > > >> > >
> > > >> > > And that created a tarfile which I posted to RAL's
anonymous ftp
> > > >> server:
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> > > >> > >
> > > >> > > I assume that'll work for you.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > John
> > > >> > >
> > > >> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT <
> > > >> > met_help at ucar.edu
> > > >> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > > >> > > >
> > > >> > > > John,
> > > >> > > >
> > > >> > > > I am able to help with this install on WCOSS.
> > > >> > > >
> > > >> > > > Would it be easier to use git to checkout the repo on
WCOSS?
> Or
> > > >> would
> > > >> > it
> > > >> > > be
> > > >> > > > > easier if I built a tarfile for this version of the
code?
> > > >> > > > >
> > > >> > > > The last time I tried to build from git on WCOSS (a
while
> ago),
> > I
> > > >> > > > couldn't. I don't remember all of the details, but I
think I
> > > >> couldn't
> > > >> > > run
> > > >> > > > bootstrap and maybe it had to do with the version of
autoconf
> > > being
> > > >> too
> > > >> > > > old? I'm happy to try it again if you'd like though.
> > Otherwise,
> > > I
> > > >> > know
> > > >> > > a
> > > >> > > > tarfile would work, but I know that is extra work for
you.
> > > >> > > >
> > > >> > > > Julie
> > > >> > > >
> > > >> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <
> > > johnhg at ucar.edu>
> > > >> > > wrote:
> > > >> > > >
> > > >> > > > > Julie,
> > > >> > > > >
> > > >> > > > > I have created a bugfix branch for main_v9.1 named
> > > >> > > > > "feature_1604_ens_config" and it's up on GitHub.
> > > >> > > > >
> > > >> > > > > Only a single file has changed in it. But I need to
get this
> > > >> version
> > > >> > > > > compiled on WCOSS with which Binbin can test.
> > > >> > > > >
> > > >> > > > > Are you to help me with this? Would it be easier to
use git
> to
> > > >> > checkout
> > > >> > > > > the repo on WCOSS? Or would it be easier if I built a
> tarfile
> > > for
> > > >> > this
> > > >> > > > > version of the code?
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > John
> > > >> > > > >
> > > >> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <
> > > >> johnhg at ucar.edu>
> > > >> > > > wrote:
> > > >> > > > >
> > > >> > > > >> Binbin,
> > > >> > > > >>
> > > >> > > > >> In order for you to test this change out, we'd need
to
> > > recompile
> > > >> a
> > > >> > > > >> version of the code on WCOSS. I wrote this up as a
GitHub
> > > issue:
> > > >> > > > >> https://github.com/dtcenter/MET/issues/1604
> > > >> > > > >>
> > > >> > > > >> And I'll create a branch from main_v9.1 named
> > > >> > "bugfix_1604_ens_config"
> > > >> > > > >> and commit the change there. Then we can checkout
that code
> > on
> > > >> > WCOSS,
> > > >> > > > >> compile it, and you can test it.
> > > >> > > > >>
> > > >> > > > >> Yes, you're right, if the fcst fields differ from
the climo
> > > >> fields,
> > > >> > > > you'd
> > > >> > > > >> need to specify them separately. That's pretty easy
in the
> > > >> > > Ensemble-Stat
> > > >> > > > >> config file... but are you asking how to control
that via
> > > >> METplus?
> > > >> > > > >>
> > > >> > > > >> John
> > > >> > > > >>
> > > >> > > > >> On Wed, Dec 9, 2020 at 2:29 PM Binbin.Zhou at noaa.gov
via
> RT <
> > > >> > > > >> met_help at ucar.edu> wrote:
> > > >> > > > >>
> > > >> > > > >>>
> > > >> > > > >>> <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >
> > > >> > > > >>>
> > > >> > > > >>> John,
> > > >> > > > >>>
> > > >> > > > >>> Great! Let me test it at first before make change
in
> > MET9.1
> > > >> > > > >>> Another question:
> > > >> > > > >>> Since climatology data is still in NCEP's grib2
format,
> > In
> > > >> the
> > > >> > > > >>> ensemble_stat
> > > >> > > > >>> config file, climo entry settings should be
different
> > from
> > > >> those
> > > >> > > of
> > > >> > > > >>> fcst, obs entries?
> > > >> > > > >>> How to set climo (name, level, etc.)?
> > > >> > > > >>>
> > > >> > > > >>> Binbin
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway
via RT <
> > > >> > > > >>> met_help at ucar.edu>
> > > >> > > > >>> wrote:
> > > >> > > > >>>
> > > >> > > > >>> > Binbin,
> > > >> > > > >>> >
> > > >> > > > >>> > Success! When I recompile with that one line
commented
> > out,
> > > >> > > > >>> plot_data_plane
> > > >> > > > >>> > is able to filter down all those records to the
single
> one
> > > >> that
> > > >> > we
> > > >> > > > >>> want.
> > > >> > > > >>> > Note in the example below that I used the
"lead_time"
> and
> > > >> > > "GRIB_ens"
> > > >> > > > >>> > options to filter the input records down.
> > > >> > > > >>> >
> > > >> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
> > '*name="HGT";
> > > >> > > > >>> level="P500";
> > > >> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > >> > > > >>> > DEBUG 1: Opening data file: ensposte.t00z.z500hr
> > > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() ->
*Found
> > > exact
> > > >> > match
> > > >> > > > for
> > > >> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB file
> > > >> > > > >>> "ensposte.t00z.z500hr".*
> > > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array() ->
Found 1
> > > GRIB
> > > >> > > records
> > > >> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> > > >> "ensposte.t00z.z500hr".
> > > >> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > >> > > > >>> >
> > > >> > > > >>> > Also note that this selects out record number 26.
> > > >> > > > >>> >
> > > >> > > > >>> > Shall I make this change as a "bugfix" for
> met-9.1...which
> > > >> could
> > > >> > be
> > > >> > > > >>> > included in a met-9.1.2 bugfix release? Or shall
I just
> > > >> include
> > > >> > it
> > > >> > > in
> > > >> > > > >>> > development for the next release (planned for
March
> 2021)?
> > > >> > > > >>> >
> > > >> > > > >>> > Thanks,
> > > >> > > > >>> > John
> > > >> > > > >>> >
> > > >> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM
Binbin.Zhou at noaa.gov via
> > RT
> > > <
> > > >> > > > >>> > met_help at ucar.edu> wrote:
> > > >> > > > >>> >
> > > >> > > > >>> > >
> > > >> > > > >>> > > <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >> > >
> > > >> > > > >>> > >
> > > >> > > > >>> > > John,
> > > >> > > > >>> > >
> > > >> > > > >>> > > If this removing impacts other applications,
or
> takes
> > > some
> > > >> > time
> > > >> > > > >>> > > to be effective, then won't try. I will use
another
> > > >> approach,
> > > >> > > such
> > > >> > > > >>> as
> > > >> > > > >>> > > borrow GEFS's grib2 data structure as tank to
save EC
> > > >> ensemble
> > > >> > > > >>> > > data.
> > > >> > > > >>> > >
> > > >> > > > >>> > > Binbin
> > > >> > > > >>> > >
> > > >> > > > >>> > >
> > > >> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou -
NOAA
> > > Affiliate
> > > >> <
> > > >> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > >> > > > >>> > >
> > > >> > > > >>> > > > John,
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > Yes, please try to use this approach.
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > Thanks!
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > Binbin
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > Binbin
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley
Gotway
> via
> > > RT <
> > > >> > > > >>> > > > met_help at ucar.edu> wrote:
> > > >> > > > >>> > > >
> > > >> > > > >>> > > >> Binbin,
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> I used the "-PDS10" option for wgrib to dump
those
> > > >> records:
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > >> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > >> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1
244 20 12
> > 6 0
> > > >> 0 1
> > > >> > 0
> > > >> > > 0
> > > >> > > > 1
> > > >> > > > >>> 0 0
> > > >> > > > >>> > 0
> > > >> > > > >>> > > 21
> > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > >> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1
244 20 12
> > 6 0
> > > >> 0 1
> > > >> > 0
> > > >> > > 0
> > > >> > > > 1
> > > >> > > > >>> 0 0
> > > >> > > > >>> > 0
> > > >> > > > >>> > > 21
> > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > >> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1
244 20 12
> > 6 0
> > > >> 0 1
> > > >> > 0
> > > >> > > 0
> > > >> > > > 1
> > > >> > > > >>> 0 0
> > > >> > > > >>> > 0
> > > >> > > > >>> > > 21
> > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> Looking VERY closely, there is a difference
in the
> > > 42-nd
> > > >> > > column
> > > >> > > > >>> which
> > > >> > > > >>> > is
> > > >> > > > >>> > > >> the 4-th from the last... records 11, 26,
and 43
> have
> > > >> values
> > > >> > > of
> > > >> > > > >>> 3, 1,
> > > >> > > > >>> > > and
> > > >> > > > >>> > > >> 2
> > > >> > > > >>> > > >> respectively.
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> On this page (
> > > >> > > > >>> > >
> > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > >> > > > >>> > > >> ),
> > > >> > > > >>> > > >> byte 42 = 1 means "unperturbed control
forecast".
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> So we *should* be able to use the "GRIB_ens"
config
> > > >> option
> > > >> > to
> > > >> > > > >>> > > >> differentiate
> > > >> > > > >>> > > >> between those records. BUT when I try, it
doesn't
> > work.
> > > >> And
> > > >> > > > that's
> > > >> > > > >>> > > because
> > > >> > > > >>> > > >> of this line:
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> We only apply those filtering options to
GRIB files
> > > >> created
> > > >> > by
> > > >> > > > >>> NCEP.
> > > >> > > > >>> > If
> > > >> > > > >>> > > >> you'd like, I could try removing that
condition to
> > see
> > > >> if we
> > > >> > > can
> > > >> > > > >>> pick
> > > >> > > > >>> > > out
> > > >> > > > >>> > > >> the single record of interest.
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> John
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
> Binbin.Zhou at noaa.gov
> > > via
> > > >> RT
> > > >> > <
> > > >> > > > >>> > > >> met_help at ucar.edu> wrote:
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > <URL:
> > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >> > > > >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > John,
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > Look at my
> > > >> > > > >>> > > >> > directory
> > > >> > > > >>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > >> > > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr,
this is
> > only
> > > >> for
> > > >> > 500
> > > >> > > > >>> HGT.
> > > >> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
> > > >> anl:type=analysis
> > > >> > > > >>> > > >> > I get 3 records:
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > From its grib2 file, we know only
record#26 is
> > > >> analysis
> > > >> > > > low-res
> > > >> > > > >>> > > >> control.
> > > >> > > > >>> > > >> > If you have some way to determine which
record is
> > > >> analysis
> > > >> > > > >>> low-res
> > > >> > > > >>> > > >> control
> > > >> > > > >>> > > >> > from the grib1 file,
> > > >> > > > >>> > > >> > then we can use grib1 directly (don't
consider
> the
> > > size
> > > >> > > > issue).
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > Binbin
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin Zhou
- NOAA
> > > >> > Affiliate
> > > >> > > <
> > > >> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > > John,
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > Grib1 data is too large, another issue
is the
> > > >> low-res
> > > >> > > > >>> control
> > > >> > > > >>> > > >> analysis
> > > >> > > > >>> > > >> > > data can not be retrieved
> > > >> > > > >>> > > >> > > from the grib1 ecmwf data files. If
using
> wgrib,
> > we
> > > >> can
> > > >> > > see
> > > >> > > > 3
> > > >> > > > >>> > > analysis
> > > >> > > > >>> > > >> > > fields, but don't know
> > > >> > > > >>> > > >> > > which is low-res control. After convert
to
> grib2,
> > > >> wgrib2
> > > >> > > can
> > > >> > > > >>> show
> > > >> > > > >>> > > >> which
> > > >> > > > >>> > > >> > > record is low-res control.
> > > >> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1
data set
> > on
> > > >> > WCOSS.
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > Binbin
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John
Halley
> Gotway
> > > via
> > > >> > RT <
> > > >> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > >> Binbin,
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >> Sounds like there's a lot of moving
parts
> there.
> > > How
> > > >> > > would
> > > >> > > > >>> you
> > > >> > > > >>> > like
> > > >> > > > >>> > > >> to
> > > >> > > > >>> > > >> > >> proceed?
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >> Should we check to see if MET can read
those
> > ECMWF
> > > >> > GRIB1
> > > >> > > > >>> files
> > > >> > > > >>> > > >> directly
> > > >> > > > >>> > > >> > >> without needing to reformat?
> > > >> > > > >>> > > >> > >> Or do you just want to create a custom
GRIB
> > table
> > > >> for
> > > >> > the
> > > >> > > > >>> result
> > > >> > > > >>> > of
> > > >> > > > >>> > > >> your
> > > >> > > > >>> > > >> > >> pre-processing and conversion steps?
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >> John
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
> > > Binbin.Zhou at noaa.gov
> > > >> > via
> > > >> > > > RT <
> > > >> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > <URL:
> > > >> > > > >>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >> > > > >>> > >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > John,
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data files
are
> > > >> converted
> > > >> > > from
> > > >> > > > >>> its
> > > >> > > > >>> > > grib1
> > > >> > > > >>> > > >> > >> format
> > > >> > > > >>> > > >> > >> > data.
> > > >> > > > >>> > > >> > >> > Before conversion, the parameter
names are
> > > there,
> > > >> but
> > > >> > > > after
> > > >> > > > >>> > > >> > conversion,
> > > >> > > > >>> > > >> > >> the
> > > >> > > > >>> > > >> > >> > names
> > > >> > > > >>> > > >> > >> > are gone. It is because of unknown
tables.
> The
> > > >> grib1
> > > >> > > data
> > > >> > > > >>> is
> > > >> > > > >>> > > still
> > > >> > > > >>> > > >> > >> > original. The original
> > > >> > > > >>> > > >> > >> > grib1 data files are very strange
data
> format
> > we
> > > >> can
> > > >> > > not
> > > >> > > > >>> read.
> > > >> > > > >>> > So
> > > >> > > > >>> > > >> we
> > > >> > > > >>> > > >> > >> have
> > > >> > > > >>> > > >> > >> > to use
> > > >> > > > >>> > > >> > >> > a Frotran code to re-arrange the
fields and
> > save
> > > >> them
> > > >> > > as
> > > >> > > > >>> > > NCEP-like
> > > >> > > > >>> > > >> > grib1
> > > >> > > > >>> > > >> > >> > data files.
> > > >> > > > >>> > > >> > >> > So after several processes, some
important
> > info
> > > >> like
> > > >> > > > tables
> > > >> > > > >>> > could
> > > >> > > > >>> > > >> be
> > > >> > > > >>> > > >> > >> lost.
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > I'll test your first suggestion
since the
> > > field
> > > >> > > record
> > > >> > > > >>> order
> > > >> > > > >>> > > is
> > > >> > > > >>> > > >> > >> fixed.
> > > >> > > > >>> > > >> > >> > Currently we still
> > > >> > > > >>> > > >> > >> > obtained grib1 data from ECWMF,
eventually,
> we
> > > >> will
> > > >> > get
> > > >> > > > >>> grib2
> > > >> > > > >>> > > data
> > > >> > > > >>> > > >> > >> > directly. So before
> > > >> > > > >>> > > >> > >> > we can get original grib2 files from
ECMWF,
> we
> > > >> just
> > > >> > use
> > > >> > > > >>> this
> > > >> > > > >>> > > >> temporal
> > > >> > > > >>> > > >> > >> > solution.
> > > >> > > > >>> > > >> > >> > '
> > > >> > > > >>> > > >> > >> > Thanks!
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John
Halley
> > > Gotway
> > > >> via
> > > >> > > RT
> > > >> > > > <
> > > >> > > > >>> > > >> > >> > met_help at ucar.edu>
> > > >> > > > >>> > > >> > >> > wrote:
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > > Binbin,
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > OK, working on Venus.
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > You have a typo in the command you
sent.
> > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > ecme.ens01.t00z.grd3.f12.grib2
> > > >> > a.ps
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >>
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> >
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> > > > >>> > > >> > >> > -v
> > > >> > > > >>> > > >> > >> > > 3
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that
should be
> > > >> > > > "GRIB2_parm_cat".
> > > >> > > > >>> > After
> > > >> > > > >>> > > >> > >> > correcting
> > > >> > > > >>> > > >> > >> > > that I did still get an error but
it's a
> > very
> > > >> > useful
> > > >> > > > one!
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > module use
> > > >> > > > >>> > > >> > >>
> > > >> > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > >> > > > >>> > > >> > >> > > module load met/9.1
> > > >> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > >> > > > >>> > > >> > >> > > cd
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > >> > > > >>> > > >> > >> > > $WGRIB2
ensposte.t00z.z500.grd3.grib2
> > > >> > > > >>> > > >> > >> > > plot_data_plane
> > ecme.ens01.t00z.grd3.f12.grib2
> > > >> ~/
> > > >> > > > plot.ps
> > > >> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > >> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
> > GRIB2_parm=5;
> > > >> > > > >>> level="P500";'
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > *ERROR : ERROR :
> VarInfoGrib2::set_dict()
> > ->
> > > >> no
> > > >> > > > >>> parameter
> > > >> > > > >>> > > found
> > > >> > > > >>> > > >> > with
> > > >> > > > >>> > > >> > >> > > matching GRIB2_disc (255)
GRIB2_parm_cat
> (3)
> > > >> > > GRIB2_parm
> > > >> > > > >>> (5).
> > > >> > > > >>> > > Use
> > > >> > > > >>> > > >> the
> > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable to
> > define
> > > >> > custom
> > > >> > > > >>> GRIB
> > > >> > > > >>> > > >> > >> tables.ERROR
> > > >> > > > >>> > > >> > >> > :
> > > >> > > > >>> > > >> > >> > > *
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > So this table is not yet defined in
MET. I
> > > >> checked
> > > >> > > > their
> > > >> > > > >>> > > website:
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > But I don't see an entry defined
for
> > > >> GRIB2_parm=5;
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > 3 things...
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > (1) As a crude temporary solution,
you
> could
> > > >> refer
> > > >> > to
> > > >> > > > it
> > > >> > > > >>> by
> > > >> > > > >>> > > it's
> > > >> > > > >>> > > >> > >> record
> > > >> > > > >>> > > >> > >> > > number. Let's assume you know that
record
> 1
> > > has
> > > >> > > > >>> temperature
> > > >> > > > >>> > > data:
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > plot_data_plane
> > ecme.ens01.t00z.grd3.f12.grib2
> > > >> ~/
> > > >> > > > plot.ps
> > > >> > > > >>> > > >> > 'name="TMP";
> > > >> > > > >>> > > >> > >> > > level="R1"; set_attr_level="P500";'
-v 5
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > Here I'm reading record 1, naming
it TMP
> and
> > > the
> > > >> > > using
> > > >> > > > >>> > > >> > set_attr_level
> > > >> > > > >>> > > >> > >> > write
> > > >> > > > >>> > > >> > >> > > the output level as P500 instead of
R1.
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > (2) You could create a custom GRIB
table
> > > >> yourself
> > > >> > > > >>> locally,
> > > >> > > > >>> > and
> > > >> > > > >>> > > >> then
> > > >> > > > >>> > > >> > >> set
> > > >> > > > >>> > > >> > >> > the
> > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable to
> > point
> > > >> MET
> > > >> > to
> > > >> > > > it.
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > (3) The best solution is contacting
ECMWF
> > and
> > > >> > > > requesting
> > > >> > > > >>> they
> > > >> > > > >>> > > >> send
> > > >> > > > >>> > > >> > >> over
> > > >> > > > >>> > > >> > >> > the
> > > >> > > > >>> > > >> > >> > > GRIB2 table we should be using to
decode
> > this
> > > >> data.
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > Thanks,
> > > >> > > > >>> > > >> > >> > > John
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM John
Halley
> > > >> Gotway <
> > > >> > > > >>> > > >> johnhg at ucar.edu>
> > > >> > > > >>> > > >> > >> > wrote:
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > > > We might already have them. I'm
not
> sure.
> > > >> > > > >>> > > >> > >> > > >
> > > >> > > > >>> > > >> > >> > > > But there's an additional
required piece
> > of
> > > >> the
> > > >> > > > puzzle.
> > > >> > > > >>> > > Binbin
> > > >> > > > >>> > > >> > runs
> > > >> > > > >>> > > >> > >> > > > "wgrib2" to figure out what
"name" he
> > should
> > > >> use
> > > >> > in
> > > >> > > > >>> the MET
> > > >> > > > >>> > > >> config
> > > >> > > > >>> > > >> > >> > files.
> > > >> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the
right
> > "name",
> > > >> the
> > > >> > > > >>> problem
> > > >> > > > >>> > will
> > > >> > > > >>> > > >> > >> remain.
> > > >> > > > >>> > > >> > >> > > > wgrib2 can be configured to
define
> > > additional
> > > >> > > tables
> > > >> > > > as
> > > >> > > > >>> > well,
> > > >> > > > >>> > > >> but
> > > >> > > > >>> > > >> > >> for
> > > >> > > > >>> > > >> > >> > > this
> > > >> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd
mean
> > updating
> > > >> the
> > > >> > > > script
> > > >> > > > >>> > that's
> > > >> > > > >>> > > >> run
> > > >> > > > >>> > > >> > >> for
> > > >> > > > >>> > > >> > >> > > > "module load grib_utils" on
WCOSS. And
> I'm
> > > >> > guessing
> > > >> > > > >>> that's
> > > >> > > > >>> > > not
> > > >> > > > >>> > > >> > very
> > > >> > > > >>> > > >> > >> > easy.
> > > >> > > > >>> > > >> > >> > > >
> > > >> > > > >>> > > >> > >> > > > Yes, I do agree that the best
solution
> is
> > > >> getting
> > > >> > > the
> > > >> > > > >>> ECMWF
> > > >> > > > >>> > > >> GRIB
> > > >> > > > >>> > > >> > >> tables
> > > >> > > > >>> > > >> > >> > > > and adding them for both wgrib2
and MET.
> > > >> > > > >>> > > >> > >> > > >
> > > >> > > > >>> > > >> > >> > > > John
> > > >> > > > >>> > > >> > >> > > >
> > > >> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM
Tara
> > Jensen <
> > > >> > > > >>> > jensen at ucar.edu>
> > > >> > > > >>> > > >> > wrote:
> > > >> > > > >>> > > >> > >> > > >
> > > >> > > > >>> > > >> > >> > > >> John,
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> It seems to me that it would be
> > worthwhile
> > > to
> > > >> > just
> > > >> > > > ask
> > > >> > > > >>> > ECMWF
> > > >> > > > >>> > > >> for
> > > >> > > > >>> > > >> > >> their
> > > >> > > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble
files.
> > And
> > > >> other
> > > >> > > > >>> > operational
> > > >> > > > >>> > > >> > >> centers
> > > >> > > > >>> > > >> > >> > for
> > > >> > > > >>> > > >> > >> > > >> that matter. Do you agree this
would
> be
> > a
> > > >> good
> > > >> > > > >>> solution?
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> Thanks, Tara
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> ---------- Forwarded message
---------
> > > >> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via
RT <
> > > >> > > > met_help at ucar.edu>
> > > >> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at 11:06
AM
> > > >> > > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu
#97769]
> > > urgent
> > > >> > > > >>> question for
> > > >> > > > >>> > > >> > reading
> > > >> > > > >>> > > >> > >> > ECMWF
> > > >> > > > >>> > > >> > >> > > >> ensmeble data
> > > >> > > > >>> > > >> > >> > > >> To:
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> <URL:
> > > >> > > > >>> > >
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> John,
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> I tested using following
command:
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > >> > > > a.ps
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>>
'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> > > > >>> > > >> -v3
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict() ->
> > either
> > > >> name
> > > >> > > or
> > > >> > > > >>> > > >> GRIB2_disc,
> > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm
must be
> > > >> specified
> > > >> > in
> > > >> > > > >>> field
> > > >> > > > >>> > > >> > >> information
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > >> > > > a.ps
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > >
> > > >> > > > >>>
> > > >> > > >
> > > >> >
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > >> > > > >>> > > >> > >> > > >> -v
> > > >> > > > >>> > > >> > >> > > >> 3
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> Still got error
> > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict() ->
> > either
> > > >> name
> > > >> > > or
> > > >> > > > >>> > > >> GRIB2_disc,
> > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm
must be
> > > >> specified
> > > >> > in
> > > >> > > > >>> field
> > > >> > > > >>> > > >> > >> information
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> Looks like discipline=255 is not
> > recognized
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> Binbin
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57 AM
John
> > Halley
> > > >> > Gotway
> > > >> > > > >>> via RT
> > > >> > > > >>> > <
> > > >> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > >> > > > >>> > > >> > >> > > >> wrote:
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> > Binbin,
> > > >> > > > >>> > > >> > >> > > >> > You can do something similar
with
> MET.
> > > >> Please
> > > >> > go
> > > >> > > > to
> > > >> > > > >>> this
> > > >> > > > >>> > > >> link:
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >>
> > > >> > >
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > And then search for
"GRIB2_mtab", and
> > > >> you'll
> > > >> > > find
> > > >> > > > >>> the
> > > >> > > > >>> > > >> content
> > > >> > > > >>> > > >> > >> > > >> > copied/pasted below. You can
specify
> > > these
> > > >> > > INSTEAD
> > > >> > > > >>> OF
> > > >> > > > >>> > > >> > specifying
> > > >> > > > >>> > > >> > >> a
> > > >> > > > >>> > > >> > >> > > >> > requested name.
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab" is
an
> > integer
> > > >> > > > >>> specifying the
> > > >> > > > >>> > > >> master
> > > >> > > > >>> > > >> > >> > table
> > > >> > > > >>> > > >> > >> > > >> > number.
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab" is
an
> > integer
> > > >> > > > >>> specifying the
> > > >> > > > >>> > > >> local
> > > >> > > > >>> > > >> > >> table
> > > >> > > > >>> > > >> > >> > > >> > number.
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_disc" is
an
> > integer
> > > >> > > > >>> specifying the
> > > >> > > > >>> > > >> GRIB2
> > > >> > > > >>> > > >> > >> > > >> discipline
> > > >> > > > >>> > > >> > >> > > >> > code.
> > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_parm_cat" is an
> > > >> integer
> > > >> > > > >>> specifying
> > > >> > > > >>> > > the
> > > >> > > > >>> > > >> > >> > parameter
> > > >> > > > >>> > > >> > >> > > >> > category
> > > >> > > > >>> > > >> > >> > > >> > // code.
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm" is
an
> > integer
> > > >> > > > >>> specifying the
> > > >> > > > >>> > > >> > >> parameter
> > > >> > > > >>> > > >> > >> > > code.
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt" is
an
> > integer
> > > >> > > > specifying
> > > >> > > > >>> the
> > > >> > > > >>> > > >> product
> > > >> > > > >>> > > >> > >> > > >> definition
> > > >> > > > >>> > > >> > >> > > >> > // template (Table
4.0).
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_process"
is an
> > > >> integer
> > > >> > > > >>> specifying
> > > >> > > > >>> > > the
> > > >> > > > >>> > > >> > >> > generating
> > > >> > > > >>> > > >> > >> > > >> > process
> > > >> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr" is
an
> > integer
> > > >> > > > >>> specifying the
> > > >> > > > >>> > > >> > >> originating
> > > >> > > > >>> > > >> > >> > > >> > center.
> > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ens_type" is an
> > > >> integer
> > > >> > > > >>> specifying
> > > >> > > > >>> > > the
> > > >> > > > >>> > > >> > >> > ensemble
> > > >> > > > >>> > > >> > >> > > >> type
> > > >> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_der_type" is an
> > > >> integer
> > > >> > > > >>> specifying
> > > >> > > > >>> > > the
> > > >> > > > >>> > > >> > >> derived
> > > >> > > > >>> > > >> > >> > > >> > product
> > > >> > > > >>> > > >> > >> > > >> > // type (Table 4.7).
> > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_stat_type" is
> an
> > > >> integer
> > > >> > > > >>> > specifying
> > > >> > > > >>> > > >> the
> > > >> > > > >>> > > >> > >> > > >> statistical
> > > >> > > > >>> > > >> > >> > > >> > // processing type
(Table
> > 4.10).
> > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ipdtmpl_index"
> > and
> > > >> > > > >>> > > "GRIB2_ipdtmpl_val"
> > > >> > > > >>> > > >> > >> entries
> > > >> > > > >>> > > >> > >> > > are
> > > >> > > > >>> > > >> > >> > > >> > arrays
> > > >> > > > >>> > > >> > >> > > >> > // of integers which
specify
> > the
> > > >> > product
> > > >> > > > >>> > > description
> > > >> > > > >>> > > >> > >> > template
> > > >> > > > >>> > > >> > >> > > >> > values to
> > > >> > > > >>> > > >> > >> > > >> > // be used. The
indices are
> > > >> 0-based.
> > > >> > For
> > > >> > > > >>> > example,
> > > >> > > > >>> > > >> use
> > > >> > > > >>> > > >> > the
> > > >> > > > >>> > > >> > >> > > >> > following to
> > > >> > > > >>> > > >> > >> > > >> > // request a GRIB2
record
> whose
> > > >> 9-th
> > > >> > and
> > > >> > > > >>> 27-th
> > > >> > > > >>> > > >> product
> > > >> > > > >>> > > >> > >> > > >> description
> > > >> > > > >>> > > >> > >> > > >> > // template values are
1 and
> 2,
> > > >> > > > >>> respectively:
> > > >> > > > >>> > > >> > >> > > >> > //
GRIB2_ipdtmpl_index=[8,
> > > 26];
> > > >> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > >> > > > >>> > > >> > >> 2];
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > While I don't have any ECMWF
data on
> > hand
> > > >> for
> > > >> > > > >>> testing, I
> > > >> > > > >>> > > do
> > > >> > > > >>> > > >> > have
> > > >> > > > >>> > > >> > >> a
> > > >> > > > >>> > > >> > >> > > file
> > > >> > > > >>> > > >> > >> > > >> > from the Air Force from which
wgrib2
> > > >> prints a
> > > >> > > > >>> similar
> > > >> > > > >>> > > >> result:
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > 549:687031550:d=2017061000:var
> > > discipline=0
> > > >> > > > >>> center=57
> > > >> > > > >>> > > >> > >> local_table=1
> > > >> > > > >>> > > >> > >> > > >> > parmcat=1 parm=192:surface:18
hour
> > fcst:
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > So I used that for testing. I
ran the
> > > >> > following
> > > >> > > > >>> > > >> plot_data_plane
> > > >> > > > >>> > > >> > >> > > command
> > > >> > > > >>> > > >> > >> > > >> > WITHOUT specifying a requested
name:
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > plot_data_plane
> > > >> GALWEM_20170610_CY00_FH18.GR2
> > > >> > > > >>> plot.ps
> > > >> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > >> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
> > > >> GRIB2_parm_cat=1;
> > > >> > > > >>> > > >> GRIB2_parm=192;
> > > >> > > > >>> > > >> > >> > > >> level="Z0";'
> > > >> > > > >>> > > >> > >> > > >> > -v 3
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > And then I looked closely at
the log
> > > >> message
> > > >> > to
> > > >> > > > >>> confirm
> > > >> > > > >>> > > >> that it
> > > >> > > > >>> > > >> > >> used
> > > >> > > > >>> > > >> > >> > > >> > record 549, as expected:
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > DEBUG 3:
> MetGrib2DataFile::data_plane()
> > > ->
> > > >> > Found
> > > >> > > > >>> exact
> > > >> > > > >>> > > match
> > > >> > > > >>> > > >> > for
> > > >> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record 549
field
> 1
> > of
> > > >> > GRIB2
> > > >> > > > file
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > So MET used these GRIB2
numbers I
> > defined
> > > >> to
> > > >> > do
> > > >> > > a
> > > >> > > > >>> table
> > > >> > > > >>> > > >> lookup
> > > >> > > > >>> > > >> > >> and
> > > >> > > > >>> > > >> > >> > > find
> > > >> > > > >>> > > >> > >> > > >> a
> > > >> > > > >>> > > >> > >> > > >> > match in the MET GRIB tables:
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > >
> > > >> > > >
> > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > And it determined that that
field
> > should
> > > be
> > > >> > > called
> > > >> > > > >>> > > "CTSTM".
> > > >> > > > >>> > > >> > >> > Hopefully,
> > > >> > > > >>> > > >> > >> > > >> the
> > > >> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are already
> defined
> > > >> there
> > > >> > in
> > > >> > > > >>> MET. If
> > > >> > > > >>> > > >> not, we
> > > >> > > > >>> > > >> > >> can
> > > >> > > > >>> > > >> > >> > > >> always
> > > >> > > > >>> > > >> > >> > > >> > add them.
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > Please try running
plot_data_plane at
> > > >> > verbosity
> > > >> > > > >>> level 3
> > > >> > > > >>> > on
> > > >> > > > >>> > > >> one
> > > >> > > > >>> > > >> > of
> > > >> > > > >>> > > >> > >> > > these
> > > >> > > > >>> > > >> > >> > > >> > files first. And make sure it
runs
> > > without
> > > >> > error
> > > >> > > > >>> and the
> > > >> > > > >>> > > log
> > > >> > > > >>> > > >> > >> message
> > > >> > > > >>> > > >> > >> > > >> prints
> > > >> > > > >>> > > >> > >> > > >> > out the "name" it found for
that
> data.
> > If
> > > >> so,
> > > >> > > you
> > > >> > > > >>> can
> > > >> > > > >>> > just
> > > >> > > > >>> > > >> skip
> > > >> > > > >>> > > >> > >> > using
> > > >> > > > >>> > > >> > >> > > >> all
> > > >> > > > >>> > > >> > >> > > >> > those integers and revert back
to
> using
> > > the
> > > >> > name
> > > >> > > > MET
> > > >> > > > >>> > > knows.
> > > >> > > > >>> > > >> If
> > > >> > > > >>> > > >> > >> not,
> > > >> > > > >>> > > >> > >> > > >> then we
> > > >> > > > >>> > > >> > >> > > >> > may need to define a custom
GRIB
> table
> > > for
> > > >> > this
> > > >> > > > >>> ECMWF
> > > >> > > > >>> > > GRIB2
> > > >> > > > >>> > > >> > data.
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >> > Thanks,
> > > >> > > > >>> > > >> > >> > > >> > John
> > > >> > > > >>> > > >> > >> > > >> >
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> --
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> Binbin Zhou
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> College Park, MD 20740
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> 301-683-3683
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >> --
> > > >> > > > >>> > > >> > >> > > >>
> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >> > > > >>> > > >> > >> > > >> Tara Jensen
> > > >> > > > >>> > > >> > >> > > >> Project Manager II
> > > >> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > >> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado
80307
> USA
> > > >> > > > >>> > > >> > >> > > >> +1 303-497-8479
> jensen at ucar.edu
> > > >> > > > >>> > > >> > >> > > >>
> > > >> > > > >>> > > >> > >> > > >
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> > >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > --
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > Binbin Zhou
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > 5830 University Research Ct.
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > College Park, MD 20740
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> > 301-683-3683
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >> >
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >>
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > --
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > Binbin Zhou
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > 5830 University Research Ct.
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > College Park, MD 20740
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> > > 301-683-3683
> > > >> > > > >>> > > >> > >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > --
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > Binbin Zhou
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > 5830 University Research Ct.
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > College Park, MD 20740
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> > 301-683-3683
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >> >
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >>
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > --
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > Binbin Zhou
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > 5830 University Research Ct.
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > College Park, MD 20740
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > Binbin.Zhou at noaa.gov
> > > >> > > > >>> > > >
> > > >> > > > >>> > > > 301-683-3683
> > > >> > > > >>> > > >
> > > >> > > > >>> > >
> > > >> > > > >>> > >
> > > >> > > > >>> > > --
> > > >> > > > >>> > >
> > > >> > > > >>> > > Binbin Zhou
> > > >> > > > >>> > >
> > > >> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>> > >
> > > >> > > > >>> > > 5830 University Research Ct.
> > > >> > > > >>> > >
> > > >> > > > >>> > > College Park, MD 20740
> > > >> > > > >>> > >
> > > >> > > > >>> > > Binbin.Zhou at noaa.gov
> > > >> > > > >>> > >
> > > >> > > > >>> > > 301-683-3683
> > > >> > > > >>> > >
> > > >> > > > >>> > >
> > > >> > > > >>> >
> > > >> > > > >>> >
> > > >> > > > >>>
> > > >> > > > >>> --
> > > >> > > > >>>
> > > >> > > > >>> Binbin Zhou
> > > >> > > > >>>
> > > >> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >>>
> > > >> > > > >>> 5830 University Research Ct.
> > > >> > > > >>>
> > > >> > > > >>> College Park, MD 20740
> > > >> > > > >>>
> > > >> > > > >>> Binbin.Zhou at noaa.gov
> > > >> > > > >>>
> > > >> > > > >>> 301-683-3683
> > > >> > > > >>>
> > > >> > > > >>>
> > > >> > > >
> > > >> > > > --
> > > >> > > > Julie Prestopnik (she/her)
> > > >> > > > Software Engineer
> > > >> > > > National Center for Atmospheric Research
> > > >> > > > Research Applications Laboratory
> > > >> > > > Email: jpresto at ucar.edu
> > > >> > > >
> > > >> > > > My working day may not be your working day. Please do
not
> feel
> > > >> obliged
> > > >> > > to
> > > >> > > > reply to this email outside of your normal working
hours.
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >> > --
> > > >> > Julie Prestopnik (she/her)
> > > >> > Software Engineer
> > > >> > National Center for Atmospheric Research
> > > >> > Research Applications Laboratory
> > > >> > Email: jpresto at ucar.edu
> > > >> >
> > > >> > My working day may not be your working day. Please do not
feel
> > > obliged
> > > >> to
> > > >> > reply to this email outside of your normal working hours.
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > >
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> > >
> >
> >
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Fri Dec 11 13:20:28 2020
John,
Including in MET-10.0.0 should be fine.
Binbin
On Fri, Dec 11, 2020 at 2:54 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> Great, thanks for confirming. OK, next question is about versions
and
> timelines.
>
> Do we need to create a met-9.1.2 bugfix version that includes this
change?
> Or is it sufficient to just include in the met-10.0.0 release?
>
> If we need to create a met-9.1.2 bugfix release, do you have a
deadline for
> that?
>
> Thanks,
> John
>
> On Fri, Dec 11, 2020 at 12:05 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > This it works!
> >
> > Thanks!
> >
> > Binbin
> >
> > On Fri, Dec 11, 2020 at 12:47 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Binbin,
> > >
> > > Those * characters don't belong in the plot_data_plane command.
That's
> > the
> > > way the mail system formats bold text. To be totally clear, here
are
> the
> > > exact commands I ran after logging on to Venus to test:
> > >
> > > cd
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > module use
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > module load met/ens_config_test
> > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 ~/plot.ps
'name="HGT";
> > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > >
> > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match for
> > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > "ecmanl.t12z.grid3.f00.grib1".
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > > matching VarInfo "HGT/P500" in GRIB file
"ecmanl.t12z.grid3.f00.grib1".
> > > DEBUG 1: Creating postscript file: /u/John.H.Gotway/plot.ps
> > >
> > > That matches a single record, namely the 3rd record in that GRIB
file.
> Do
> > > you see different behavior? Perhaps you failed to load the
> > > "ens_config_test" version of the code before testing?
> > >
> > > John
> > >
> > > On Fri, Dec 11, 2020 at 10:20 AM Binbin.Zhou at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > > >
> > > > John,
> > > >
> > > > As I tested following plotting
> > > >
> > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
'*name="HGT";
> > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > >
> > > > the error message is
> > > >
> > > > ERROR :
> > > > ERROR : yyerror() -> syntax error in file
"/tmp/met_config_9350_0"
> > > > ERROR :
> > > > ERROR : line = 1
> > > > ERROR :
> > > > ERROR : column = 1
> > > > ERROR :
> > > > ERROR : text = "*"
> > > > ERROR :
> > > > ERROR :
> > > > ERROR : *name="HGT"; level="P500"; lead_time="00";
> > > > GRIB_ens="low_res_ctl";*
> > > > ERROR :
> > > >
^__________________________________________________________________
> > > > ERROR :
> > > >
> > > > After I removed "*"
> > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
'name="HGT";
> > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > > > It shows 3 records matched
> > > >
> > > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> for
> > > > VarInfo "HGT/P500" in GRIB record 1 of GRIB file
> > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> for
> > > > VarInfo "HGT/P500" in GRIB record 2 of GRIB file
> > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> for
> > > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3 GRIB
records
> > > > matching VarInfo "HGT/P500" in GRIB file
> "ecmanl.t12z.grid3.f00.grib1".
> > > > DEBUG 1: Creating postscript file: plot.ps
> > > >
> > > > My testing directory on Venus
> > > > is
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > > test script is ploting.sh
> > > >
> > > > Can you look at it?
> > > >
> > > > Thanks!
> > > > Binbin
> > > >
> > > >
> > > > On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA Affiliate <
> > > > binbin.zhou at noaa.gov> wrote:
> > > >
> > > > > John,
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Binbin
> > > > >
> > > > > On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Thanks Julie.
> > > > >>
> > > > >> And Binbin, here's an EnsembleStatConfig file where climos
are
> used:
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
> > > > >>
> > > > >> Since the list of forecast fields is exactly the same as
the climo
> > > > fields,
> > > > >> this config file just copies them over using:
> > > > >> climo_mean = fcst;
> > > > >>
> > > > >> In your case, when they differ, you'd just specify the
"field"
> array
> > > > >> inside
> > > > >> the "climo_mean" dictionary. For example:
> > > > >>
> > > > >> climo_mean = {
> > > > >>
> > > > >> field = [
> > > > >> { name = "TMP"; level = "Z2"; },
> > > > >> { name = "TMP"; level = "P850"; }
> > > > >> ];
> > > > >> ...
> > > > >> }
> > > > >>
> > > > >> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT <
> > > > >> met_help at ucar.edu>
> > > > >> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > > >> >
> > > > >> > Thanks, John. That worked great.
> > > > >> >
> > > > >> > Binbin, this version of MET is now accessible to you by
running
> > the
> > > > >> > following on venus:
> > > > >> >
> > > > >> > module use
> > > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > >> > module load met/ens_config_test
> > > > >> >
> > > > >> > Please let us know if you encounter any problems.
> > > > >> >
> > > > >> > Julie
> > > > >> >
> > > > >> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via
RT <
> > > > >> > met_help at ucar.edu> wrote:
> > > > >> >
> > > > >> > >
> > > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > > > >> > >
> > > > >> > > Julie,
> > > > >> > >
> > > > >> > > Building the tarfile is very easy. I just ran this
script from
> > the
> > > > MET
> > > > >> > repo
> > > > >> > > and specified the branch to be built:
> > > > >> > > MET/scripts/met_checkout_and_build.sh branch
> > > > >> feature_1604_ens_config
> > > > >> > >
> > > > >> > > And that created a tarfile which I posted to RAL's
anonymous
> ftp
> > > > >> server:
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> > > > >> > >
> > > > >> > > I assume that'll work for you.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > John
> > > > >> > >
> > > > >> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via RT
<
> > > > >> > met_help at ucar.edu
> > > > >> > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > > > >> > > >
> > > > >> > > > John,
> > > > >> > > >
> > > > >> > > > I am able to help with this install on WCOSS.
> > > > >> > > >
> > > > >> > > > Would it be easier to use git to checkout the repo on
WCOSS?
> > Or
> > > > >> would
> > > > >> > it
> > > > >> > > be
> > > > >> > > > > easier if I built a tarfile for this version of the
code?
> > > > >> > > > >
> > > > >> > > > The last time I tried to build from git on WCOSS (a
while
> > ago),
> > > I
> > > > >> > > > couldn't. I don't remember all of the details, but I
think
> I
> > > > >> couldn't
> > > > >> > > run
> > > > >> > > > bootstrap and maybe it had to do with the version of
> autoconf
> > > > being
> > > > >> too
> > > > >> > > > old? I'm happy to try it again if you'd like though.
> > > Otherwise,
> > > > I
> > > > >> > know
> > > > >> > > a
> > > > >> > > > tarfile would work, but I know that is extra work for
you.
> > > > >> > > >
> > > > >> > > > Julie
> > > > >> > > >
> > > > >> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <
> > > > johnhg at ucar.edu>
> > > > >> > > wrote:
> > > > >> > > >
> > > > >> > > > > Julie,
> > > > >> > > > >
> > > > >> > > > > I have created a bugfix branch for main_v9.1 named
> > > > >> > > > > "feature_1604_ens_config" and it's up on GitHub.
> > > > >> > > > >
> > > > >> > > > > Only a single file has changed in it. But I need to
get
> this
> > > > >> version
> > > > >> > > > > compiled on WCOSS with which Binbin can test.
> > > > >> > > > >
> > > > >> > > > > Are you to help me with this? Would it be easier to
use
> git
> > to
> > > > >> > checkout
> > > > >> > > > > the repo on WCOSS? Or would it be easier if I built
a
> > tarfile
> > > > for
> > > > >> > this
> > > > >> > > > > version of the code?
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > John
> > > > >> > > > >
> > > > >> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway <
> > > > >> johnhg at ucar.edu>
> > > > >> > > > wrote:
> > > > >> > > > >
> > > > >> > > > >> Binbin,
> > > > >> > > > >>
> > > > >> > > > >> In order for you to test this change out, we'd
need to
> > > > recompile
> > > > >> a
> > > > >> > > > >> version of the code on WCOSS. I wrote this up as a
GitHub
> > > > issue:
> > > > >> > > > >> https://github.com/dtcenter/MET/issues/1604
> > > > >> > > > >>
> > > > >> > > > >> And I'll create a branch from main_v9.1 named
> > > > >> > "bugfix_1604_ens_config"
> > > > >> > > > >> and commit the change there. Then we can checkout
that
> code
> > > on
> > > > >> > WCOSS,
> > > > >> > > > >> compile it, and you can test it.
> > > > >> > > > >>
> > > > >> > > > >> Yes, you're right, if the fcst fields differ from
the
> climo
> > > > >> fields,
> > > > >> > > > you'd
> > > > >> > > > >> need to specify them separately. That's pretty
easy in
> the
> > > > >> > > Ensemble-Stat
> > > > >> > > > >> config file... but are you asking how to control
that via
> > > > >> METplus?
> > > > >> > > > >>
> > > > >> > > > >> John
> > > > >> > > > >>
> > > > >> > > > >> On Wed, Dec 9, 2020 at 2:29 PM
Binbin.Zhou at noaa.gov via
> > RT <
> > > > >> > > > >> met_help at ucar.edu> wrote:
> > > > >> > > > >>
> > > > >> > > > >>>
> > > > >> > > > >>> <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >
> > > > >> > > > >>>
> > > > >> > > > >>> John,
> > > > >> > > > >>>
> > > > >> > > > >>> Great! Let me test it at first before make
change in
> > > MET9.1
> > > > >> > > > >>> Another question:
> > > > >> > > > >>> Since climatology data is still in NCEP's
grib2
> format,
> > > In
> > > > >> the
> > > > >> > > > >>> ensemble_stat
> > > > >> > > > >>> config file, climo entry settings should be
different
> > > from
> > > > >> those
> > > > >> > > of
> > > > >> > > > >>> fcst, obs entries?
> > > > >> > > > >>> How to set climo (name, level, etc.)?
> > > > >> > > > >>>
> > > > >> > > > >>> Binbin
> > > > >> > > > >>>
> > > > >> > > > >>>
> > > > >> > > > >>>
> > > > >> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley Gotway
via
> RT <
> > > > >> > > > >>> met_help at ucar.edu>
> > > > >> > > > >>> wrote:
> > > > >> > > > >>>
> > > > >> > > > >>> > Binbin,
> > > > >> > > > >>> >
> > > > >> > > > >>> > Success! When I recompile with that one line
commented
> > > out,
> > > > >> > > > >>> plot_data_plane
> > > > >> > > > >>> > is able to filter down all those records to the
single
> > one
> > > > >> that
> > > > >> > we
> > > > >> > > > >>> want.
> > > > >> > > > >>> > Note in the example below that I used the
"lead_time"
> > and
> > > > >> > > "GRIB_ens"
> > > > >> > > > >>> > options to filter the input records down.
> > > > >> > > > >>> >
> > > > >> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
> > > '*name="HGT";
> > > > >> > > > >>> level="P500";
> > > > >> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > > >> > > > >>> > DEBUG 1: Opening data file:
ensposte.t00z.z500hr
> > > > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array()
->
> *Found
> > > > exact
> > > > >> > match
> > > > >> > > > for
> > > > >> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB
file
> > > > >> > > > >>> "ensposte.t00z.z500hr".*
> > > > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array()
->
> Found 1
> > > > GRIB
> > > > >> > > records
> > > > >> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> > > > >> "ensposte.t00z.z500hr".
> > > > >> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > > >> > > > >>> >
> > > > >> > > > >>> > Also note that this selects out record number
26.
> > > > >> > > > >>> >
> > > > >> > > > >>> > Shall I make this change as a "bugfix" for
> > met-9.1...which
> > > > >> could
> > > > >> > be
> > > > >> > > > >>> > included in a met-9.1.2 bugfix release? Or
shall I
> just
> > > > >> include
> > > > >> > it
> > > > >> > > in
> > > > >> > > > >>> > development for the next release (planned for
March
> > 2021)?
> > > > >> > > > >>> >
> > > > >> > > > >>> > Thanks,
> > > > >> > > > >>> > John
> > > > >> > > > >>> >
> > > > >> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM
Binbin.Zhou at noaa.gov
> via
> > > RT
> > > > <
> > > > >> > > > >>> > met_help at ucar.edu> wrote:
> > > > >> > > > >>> >
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >> > >
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > John,
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > If this removing impacts other
applications, or
> > takes
> > > > some
> > > > >> > time
> > > > >> > > > >>> > > to be effective, then won't try. I will use
another
> > > > >> approach,
> > > > >> > > such
> > > > >> > > > >>> as
> > > > >> > > > >>> > > borrow GEFS's grib2 data structure as tank to
save
> EC
> > > > >> ensemble
> > > > >> > > > >>> > > data.
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > Binbin
> > > > >> > > > >>> > >
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou -
NOAA
> > > > Affiliate
> > > > >> <
> > > > >> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > > John,
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > Yes, please try to use this approach.
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > Thanks!
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > Binbin
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > Binbin
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John Halley
Gotway
> > via
> > > > RT <
> > > > >> > > > >>> > > > met_help at ucar.edu> wrote:
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > >> Binbin,
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> I used the "-PDS10" option for wgrib to
dump
> those
> > > > >> records:
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > > >> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > > >> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100 1
244 20
> 12
> > > 6 0
> > > > >> 0 1
> > > > >> > 0
> > > > >> > > 0
> > > > >> > > > 1
> > > > >> > > > >>> 0 0
> > > > >> > > > >>> > 0
> > > > >> > > > >>> > > 21
> > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1 255
> > > > >> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100 1
244 20
> 12
> > > 6 0
> > > > >> 0 1
> > > > >> > 0
> > > > >> > > 0
> > > > >> > > > 1
> > > > >> > > > >>> 0 0
> > > > >> > > > >>> > 0
> > > > >> > > > >>> > > 21
> > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1 255
> > > > >> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100 1
244 20
> 12
> > > 6 0
> > > > >> 0 1
> > > > >> > 0
> > > > >> > > 0
> > > > >> > > > 1
> > > > >> > > > >>> 0 0
> > > > >> > > > >>> > 0
> > > > >> > > > >>> > > 21
> > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1 255
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> Looking VERY closely, there is a
difference in
> the
> > > > 42-nd
> > > > >> > > column
> > > > >> > > > >>> which
> > > > >> > > > >>> > is
> > > > >> > > > >>> > > >> the 4-th from the last... records 11, 26,
and 43
> > have
> > > > >> values
> > > > >> > > of
> > > > >> > > > >>> 3, 1,
> > > > >> > > > >>> > > and
> > > > >> > > > >>> > > >> 2
> > > > >> > > > >>> > > >> respectively.
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> On this page (
> > > > >> > > > >>> > >
> > > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > > >> > > > >>> > > >> ),
> > > > >> > > > >>> > > >> byte 42 = 1 means "unperturbed control
forecast".
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> So we *should* be able to use the
"GRIB_ens"
> config
> > > > >> option
> > > > >> > to
> > > > >> > > > >>> > > >> differentiate
> > > > >> > > > >>> > > >> between those records. BUT when I try, it
doesn't
> > > work.
> > > > >> And
> > > > >> > > > that's
> > > > >> > > > >>> > > because
> > > > >> > > > >>> > > >> of this line:
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> We only apply those filtering options to
GRIB
> files
> > > > >> created
> > > > >> > by
> > > > >> > > > >>> NCEP.
> > > > >> > > > >>> > If
> > > > >> > > > >>> > > >> you'd like, I could try removing that
condition
> to
> > > see
> > > > >> if we
> > > > >> > > can
> > > > >> > > > >>> pick
> > > > >> > > > >>> > > out
> > > > >> > > > >>> > > >> the single record of interest.
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> John
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
> > Binbin.Zhou at noaa.gov
> > > > via
> > > > >> RT
> > > > >> > <
> > > > >> > > > >>> > > >> met_help at ucar.edu> wrote:
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > <URL:
> > > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >> > > > >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > John,
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > Look at my
> > > > >> > > > >>> > > >> > directory
> > > > >> > > > >>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > > >> > > > >>> > > >> > The grib1 file is ensposte.t00z.z500hr,
this is
> > > only
> > > > >> for
> > > > >> > 500
> > > > >> > > > >>> HGT.
> > > > >> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep anl|grep
> > > > >> anl:type=analysis
> > > > >> > > > >>> > > >> > I get 3 records:
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > From its grib2 file, we know only
record#26 is
> > > > >> analysis
> > > > >> > > > low-res
> > > > >> > > > >>> > > >> control.
> > > > >> > > > >>> > > >> > If you have some way to determine which
record
> is
> > > > >> analysis
> > > > >> > > > >>> low-res
> > > > >> > > > >>> > > >> control
> > > > >> > > > >>> > > >> > from the grib1 file,
> > > > >> > > > >>> > > >> > then we can use grib1 directly (don't
consider
> > the
> > > > size
> > > > >> > > > issue).
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > Binbin
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin
Zhou -
> NOAA
> > > > >> > Affiliate
> > > > >> > > <
> > > > >> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > > John,
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > Grib1 data is too large, another
issue is
> the
> > > > >> low-res
> > > > >> > > > >>> control
> > > > >> > > > >>> > > >> analysis
> > > > >> > > > >>> > > >> > > data can not be retrieved
> > > > >> > > > >>> > > >> > > from the grib1 ecmwf data files. If
using
> > wgrib,
> > > we
> > > > >> can
> > > > >> > > see
> > > > >> > > > 3
> > > > >> > > > >>> > > analysis
> > > > >> > > > >>> > > >> > > fields, but don't know
> > > > >> > > > >>> > > >> > > which is low-res control. After
convert to
> > grib2,
> > > > >> wgrib2
> > > > >> > > can
> > > > >> > > > >>> show
> > > > >> > > > >>> > > >> which
> > > > >> > > > >>> > > >> > > record is low-res control.
> > > > >> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf grib1
data
> set
> > > on
> > > > >> > WCOSS.
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > Binbin
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John
Halley
> > Gotway
> > > > via
> > > > >> > RT <
> > > > >> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > >> Binbin,
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >> Sounds like there's a lot of moving
parts
> > there.
> > > > How
> > > > >> > > would
> > > > >> > > > >>> you
> > > > >> > > > >>> > like
> > > > >> > > > >>> > > >> to
> > > > >> > > > >>> > > >> > >> proceed?
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >> Should we check to see if MET can
read those
> > > ECMWF
> > > > >> > GRIB1
> > > > >> > > > >>> files
> > > > >> > > > >>> > > >> directly
> > > > >> > > > >>> > > >> > >> without needing to reformat?
> > > > >> > > > >>> > > >> > >> Or do you just want to create a
custom GRIB
> > > table
> > > > >> for
> > > > >> > the
> > > > >> > > > >>> result
> > > > >> > > > >>> > of
> > > > >> > > > >>> > > >> your
> > > > >> > > > >>> > > >> > >> pre-processing and conversion steps?
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >> John
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
> > > > Binbin.Zhou at noaa.gov
> > > > >> > via
> > > > >> > > > RT <
> > > > >> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > <URL:
> > > > >> > > > >>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > John,
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data
files are
> > > > >> converted
> > > > >> > > from
> > > > >> > > > >>> its
> > > > >> > > > >>> > > grib1
> > > > >> > > > >>> > > >> > >> format
> > > > >> > > > >>> > > >> > >> > data.
> > > > >> > > > >>> > > >> > >> > Before conversion, the parameter
names are
> > > > there,
> > > > >> but
> > > > >> > > > after
> > > > >> > > > >>> > > >> > conversion,
> > > > >> > > > >>> > > >> > >> the
> > > > >> > > > >>> > > >> > >> > names
> > > > >> > > > >>> > > >> > >> > are gone. It is because of unknown
tables.
> > The
> > > > >> grib1
> > > > >> > > data
> > > > >> > > > >>> is
> > > > >> > > > >>> > > still
> > > > >> > > > >>> > > >> > >> > original. The original
> > > > >> > > > >>> > > >> > >> > grib1 data files are very strange
data
> > format
> > > we
> > > > >> can
> > > > >> > > not
> > > > >> > > > >>> read.
> > > > >> > > > >>> > So
> > > > >> > > > >>> > > >> we
> > > > >> > > > >>> > > >> > >> have
> > > > >> > > > >>> > > >> > >> > to use
> > > > >> > > > >>> > > >> > >> > a Frotran code to re-arrange the
fields
> and
> > > save
> > > > >> them
> > > > >> > > as
> > > > >> > > > >>> > > NCEP-like
> > > > >> > > > >>> > > >> > grib1
> > > > >> > > > >>> > > >> > >> > data files.
> > > > >> > > > >>> > > >> > >> > So after several processes, some
important
> > > info
> > > > >> like
> > > > >> > > > tables
> > > > >> > > > >>> > could
> > > > >> > > > >>> > > >> be
> > > > >> > > > >>> > > >> > >> lost.
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > I'll test your first suggestion
since
> the
> > > > field
> > > > >> > > record
> > > > >> > > > >>> order
> > > > >> > > > >>> > > is
> > > > >> > > > >>> > > >> > >> fixed.
> > > > >> > > > >>> > > >> > >> > Currently we still
> > > > >> > > > >>> > > >> > >> > obtained grib1 data from ECWMF,
> eventually,
> > we
> > > > >> will
> > > > >> > get
> > > > >> > > > >>> grib2
> > > > >> > > > >>> > > data
> > > > >> > > > >>> > > >> > >> > directly. So before
> > > > >> > > > >>> > > >> > >> > we can get original grib2 files
from
> ECMWF,
> > we
> > > > >> just
> > > > >> > use
> > > > >> > > > >>> this
> > > > >> > > > >>> > > >> temporal
> > > > >> > > > >>> > > >> > >> > solution.
> > > > >> > > > >>> > > >> > >> > '
> > > > >> > > > >>> > > >> > >> > Thanks!
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM John
Halley
> > > > Gotway
> > > > >> via
> > > > >> > > RT
> > > > >> > > > <
> > > > >> > > > >>> > > >> > >> > met_help at ucar.edu>
> > > > >> > > > >>> > > >> > >> > wrote:
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > > Binbin,
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > OK, working on Venus.
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > You have a typo in the command
you sent.
> > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > >> > a.ps
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> >
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >> > > > >>> > > >> > >> > -v
> > > > >> > > > >>> > > >> > >> > > 3
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that
should be
> > > > >> > > > "GRIB2_parm_cat".
> > > > >> > > > >>> > After
> > > > >> > > > >>> > > >> > >> > correcting
> > > > >> > > > >>> > > >> > >> > > that I did still get an error but
it's a
> > > very
> > > > >> > useful
> > > > >> > > > one!
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > module use
> > > > >> > > > >>> > > >> > >>
> > > > >> > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > >> > > > >>> > > >> > >> > > module load met/9.1
> > > > >> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > > >> > > > >>> > > >> > >> > > cd
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > > >> > > > >>> > > >> > >> > > $WGRIB2
ensposte.t00z.z500.grd3.grib2
> > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > ecme.ens01.t00z.grd3.f12.grib2
> > > > >> ~/
> > > > >> > > > plot.ps
> > > > >> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > > >> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
> > > GRIB2_parm=5;
> > > > >> > > > >>> level="P500";'
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > *ERROR : ERROR :
> > VarInfoGrib2::set_dict()
> > > ->
> > > > >> no
> > > > >> > > > >>> parameter
> > > > >> > > > >>> > > found
> > > > >> > > > >>> > > >> > with
> > > > >> > > > >>> > > >> > >> > > matching GRIB2_disc (255)
GRIB2_parm_cat
> > (3)
> > > > >> > > GRIB2_parm
> > > > >> > > > >>> (5).
> > > > >> > > > >>> > > Use
> > > > >> > > > >>> > > >> the
> > > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable to
> > > define
> > > > >> > custom
> > > > >> > > > >>> GRIB
> > > > >> > > > >>> > > >> > >> tables.ERROR
> > > > >> > > > >>> > > >> > >> > :
> > > > >> > > > >>> > > >> > >> > > *
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > So this table is not yet defined
in
> MET. I
> > > > >> checked
> > > > >> > > > their
> > > > >> > > > >>> > > website:
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > But I don't see an entry defined
for
> > > > >> GRIB2_parm=5;
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > 3 things...
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > (1) As a crude temporary
solution, you
> > could
> > > > >> refer
> > > > >> > to
> > > > >> > > > it
> > > > >> > > > >>> by
> > > > >> > > > >>> > > it's
> > > > >> > > > >>> > > >> > >> record
> > > > >> > > > >>> > > >> > >> > > number. Let's assume you know
that
> record
> > 1
> > > > has
> > > > >> > > > >>> temperature
> > > > >> > > > >>> > > data:
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > ecme.ens01.t00z.grd3.f12.grib2
> > > > >> ~/
> > > > >> > > > plot.ps
> > > > >> > > > >>> > > >> > 'name="TMP";
> > > > >> > > > >>> > > >> > >> > > level="R1";
set_attr_level="P500";' -v 5
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > Here I'm reading record 1, naming
it TMP
> > and
> > > > the
> > > > >> > > using
> > > > >> > > > >>> > > >> > set_attr_level
> > > > >> > > > >>> > > >> > >> > write
> > > > >> > > > >>> > > >> > >> > > the output level as P500 instead
of R1.
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > (2) You could create a custom
GRIB table
> > > > >> yourself
> > > > >> > > > >>> locally,
> > > > >> > > > >>> > and
> > > > >> > > > >>> > > >> then
> > > > >> > > > >>> > > >> > >> set
> > > > >> > > > >>> > > >> > >> > the
> > > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable to
> > > point
> > > > >> MET
> > > > >> > to
> > > > >> > > > it.
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > (3) The best solution is
contacting
> ECMWF
> > > and
> > > > >> > > > requesting
> > > > >> > > > >>> they
> > > > >> > > > >>> > > >> send
> > > > >> > > > >>> > > >> > >> over
> > > > >> > > > >>> > > >> > >> > the
> > > > >> > > > >>> > > >> > >> > > GRIB2 table we should be using to
decode
> > > this
> > > > >> data.
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > Thanks,
> > > > >> > > > >>> > > >> > >> > > John
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM
John
> Halley
> > > > >> Gotway <
> > > > >> > > > >>> > > >> johnhg at ucar.edu>
> > > > >> > > > >>> > > >> > >> > wrote:
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > > > We might already have them. I'm
not
> > sure.
> > > > >> > > > >>> > > >> > >> > > >
> > > > >> > > > >>> > > >> > >> > > > But there's an additional
required
> piece
> > > of
> > > > >> the
> > > > >> > > > puzzle.
> > > > >> > > > >>> > > Binbin
> > > > >> > > > >>> > > >> > runs
> > > > >> > > > >>> > > >> > >> > > > "wgrib2" to figure out what
"name" he
> > > should
> > > > >> use
> > > > >> > in
> > > > >> > > > >>> the MET
> > > > >> > > > >>> > > >> config
> > > > >> > > > >>> > > >> > >> > files.
> > > > >> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him the
right
> > > "name",
> > > > >> the
> > > > >> > > > >>> problem
> > > > >> > > > >>> > will
> > > > >> > > > >>> > > >> > >> remain.
> > > > >> > > > >>> > > >> > >> > > > wgrib2 can be configured to
define
> > > > additional
> > > > >> > > tables
> > > > >> > > > as
> > > > >> > > > >>> > well,
> > > > >> > > > >>> > > >> but
> > > > >> > > > >>> > > >> > >> for
> > > > >> > > > >>> > > >> > >> > > this
> > > > >> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd
mean
> > > updating
> > > > >> the
> > > > >> > > > script
> > > > >> > > > >>> > that's
> > > > >> > > > >>> > > >> run
> > > > >> > > > >>> > > >> > >> for
> > > > >> > > > >>> > > >> > >> > > > "module load grib_utils" on
WCOSS. And
> > I'm
> > > > >> > guessing
> > > > >> > > > >>> that's
> > > > >> > > > >>> > > not
> > > > >> > > > >>> > > >> > very
> > > > >> > > > >>> > > >> > >> > easy.
> > > > >> > > > >>> > > >> > >> > > >
> > > > >> > > > >>> > > >> > >> > > > Yes, I do agree that the best
solution
> > is
> > > > >> getting
> > > > >> > > the
> > > > >> > > > >>> ECMWF
> > > > >> > > > >>> > > >> GRIB
> > > > >> > > > >>> > > >> > >> tables
> > > > >> > > > >>> > > >> > >> > > > and adding them for both wgrib2
and
> MET.
> > > > >> > > > >>> > > >> > >> > > >
> > > > >> > > > >>> > > >> > >> > > > John
> > > > >> > > > >>> > > >> > >> > > >
> > > > >> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32 PM
Tara
> > > Jensen <
> > > > >> > > > >>> > jensen at ucar.edu>
> > > > >> > > > >>> > > >> > wrote:
> > > > >> > > > >>> > > >> > >> > > >
> > > > >> > > > >>> > > >> > >> > > >> John,
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> It seems to me that it would
be
> > > worthwhile
> > > > to
> > > > >> > just
> > > > >> > > > ask
> > > > >> > > > >>> > ECMWF
> > > > >> > > > >>> > > >> for
> > > > >> > > > >>> > > >> > >> their
> > > > >> > > > >>> > > >> > >> > > >> Grib2 Table for their Ensemble
files.
> > > And
> > > > >> other
> > > > >> > > > >>> > operational
> > > > >> > > > >>> > > >> > >> centers
> > > > >> > > > >>> > > >> > >> > for
> > > > >> > > > >>> > > >> > >> > > >> that matter. Do you agree
this would
> > be
> > > a
> > > > >> good
> > > > >> > > > >>> solution?
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> Thanks, Tara
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> ---------- Forwarded message
> ---------
> > > > >> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov via
RT <
> > > > >> > > > met_help at ucar.edu>
> > > > >> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at
11:06 AM
> > > > >> > > > >>> > > >> > >> > > >> Subject: Re: [rt.rap.ucar.edu
> #97769]
> > > > urgent
> > > > >> > > > >>> question for
> > > > >> > > > >>> > > >> > reading
> > > > >> > > > >>> > > >> > >> > ECMWF
> > > > >> > > > >>> > > >> > >> > > >> ensmeble data
> > > > >> > > > >>> > > >> > >> > > >> To:
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> <URL:
> > > > >> > > > >>> > >
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> John,
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> I tested using following
command:
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > > >> > > > a.ps
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>>
> 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >> > > > >>> > > >> -v3
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict() ->
> > > either
> > > > >> name
> > > > >> > > or
> > > > >> > > > >>> > > >> GRIB2_disc,
> > > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm
must be
> > > > >> specified
> > > > >> > in
> > > > >> > > > >>> field
> > > > >> > > > >>> > > >> > >> information
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > > >> > > > a.ps
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> >
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > >> > > > >>> > > >> > >> > > >> -v
> > > > >> > > > >>> > > >> > >> > > >> 3
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> Still got error
> > > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict() ->
> > > either
> > > > >> name
> > > > >> > > or
> > > > >> > > > >>> > > >> GRIB2_disc,
> > > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and GRIB2_parm
must be
> > > > >> specified
> > > > >> > in
> > > > >> > > > >>> field
> > > > >> > > > >>> > > >> > >> information
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> Looks like discipline=255 is
not
> > > recognized
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> Binbin
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57
AM John
> > > Halley
> > > > >> > Gotway
> > > > >> > > > >>> via RT
> > > > >> > > > >>> > <
> > > > >> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > > >> > > > >>> > > >> > >> > > >> wrote:
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> > Binbin,
> > > > >> > > > >>> > > >> > >> > > >> > You can do something similar
with
> > MET.
> > > > >> Please
> > > > >> > go
> > > > >> > > > to
> > > > >> > > > >>> this
> > > > >> > > > >>> > > >> link:
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > >
> https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > And then search for
"GRIB2_mtab",
> and
> > > > >> you'll
> > > > >> > > find
> > > > >> > > > >>> the
> > > > >> > > > >>> > > >> content
> > > > >> > > > >>> > > >> > >> > > >> > copied/pasted below. You can
> specify
> > > > these
> > > > >> > > INSTEAD
> > > > >> > > > >>> OF
> > > > >> > > > >>> > > >> > specifying
> > > > >> > > > >>> > > >> > >> a
> > > > >> > > > >>> > > >> > >> > > >> > requested name.
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_mtab"
is an
> > > integer
> > > > >> > > > >>> specifying the
> > > > >> > > > >>> > > >> master
> > > > >> > > > >>> > > >> > >> > table
> > > > >> > > > >>> > > >> > >> > > >> > number.
> > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_ltab"
is an
> > > integer
> > > > >> > > > >>> specifying the
> > > > >> > > > >>> > > >> local
> > > > >> > > > >>> > > >> > >> table
> > > > >> > > > >>> > > >> > >> > > >> > number.
> > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_disc"
is an
> > > integer
> > > > >> > > > >>> specifying the
> > > > >> > > > >>> > > >> GRIB2
> > > > >> > > > >>> > > >> > >> > > >> discipline
> > > > >> > > > >>> > > >> > >> > > >> > code.
> > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_parm_cat" is
> an
> > > > >> integer
> > > > >> > > > >>> specifying
> > > > >> > > > >>> > > the
> > > > >> > > > >>> > > >> > >> > parameter
> > > > >> > > > >>> > > >> > >> > > >> > category
> > > > >> > > > >>> > > >> > >> > > >> > // code.
> > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_parm"
is an
> > > integer
> > > > >> > > > >>> specifying the
> > > > >> > > > >>> > > >> > >> parameter
> > > > >> > > > >>> > > >> > >> > > code.
> > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt"
is an
> > > integer
> > > > >> > > > specifying
> > > > >> > > > >>> the
> > > > >> > > > >>> > > >> product
> > > > >> > > > >>> > > >> > >> > > >> definition
> > > > >> > > > >>> > > >> > >> > > >> > // template (Table
4.0).
> > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_process" is
> an
> > > > >> integer
> > > > >> > > > >>> specifying
> > > > >> > > > >>> > > the
> > > > >> > > > >>> > > >> > >> > generating
> > > > >> > > > >>> > > >> > >> > > >> > process
> > > > >> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_cntr"
is an
> > > integer
> > > > >> > > > >>> specifying the
> > > > >> > > > >>> > > >> > >> originating
> > > > >> > > > >>> > > >> > >> > > >> > center.
> > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ens_type" is
> an
> > > > >> integer
> > > > >> > > > >>> specifying
> > > > >> > > > >>> > > the
> > > > >> > > > >>> > > >> > >> > ensemble
> > > > >> > > > >>> > > >> > >> > > >> type
> > > > >> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_der_type" is
> an
> > > > >> integer
> > > > >> > > > >>> specifying
> > > > >> > > > >>> > > the
> > > > >> > > > >>> > > >> > >> derived
> > > > >> > > > >>> > > >> > >> > > >> > product
> > > > >> > > > >>> > > >> > >> > > >> > // type (Table 4.7).
> > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_stat_type" is
> > an
> > > > >> integer
> > > > >> > > > >>> > specifying
> > > > >> > > > >>> > > >> the
> > > > >> > > > >>> > > >> > >> > > >> statistical
> > > > >> > > > >>> > > >> > >> > > >> > // processing type
(Table
> > > 4.10).
> > > > >> > > > >>> > > >> > >> > > >> > // - The
> "GRIB2_ipdtmpl_index"
> > > and
> > > > >> > > > >>> > > "GRIB2_ipdtmpl_val"
> > > > >> > > > >>> > > >> > >> entries
> > > > >> > > > >>> > > >> > >> > > are
> > > > >> > > > >>> > > >> > >> > > >> > arrays
> > > > >> > > > >>> > > >> > >> > > >> > // of integers which
> specify
> > > the
> > > > >> > product
> > > > >> > > > >>> > > description
> > > > >> > > > >>> > > >> > >> > template
> > > > >> > > > >>> > > >> > >> > > >> > values to
> > > > >> > > > >>> > > >> > >> > > >> > // be used. The
indices are
> > > > >> 0-based.
> > > > >> > For
> > > > >> > > > >>> > example,
> > > > >> > > > >>> > > >> use
> > > > >> > > > >>> > > >> > the
> > > > >> > > > >>> > > >> > >> > > >> > following to
> > > > >> > > > >>> > > >> > >> > > >> > // request a GRIB2
record
> > whose
> > > > >> 9-th
> > > > >> > and
> > > > >> > > > >>> 27-th
> > > > >> > > > >>> > > >> product
> > > > >> > > > >>> > > >> > >> > > >> description
> > > > >> > > > >>> > > >> > >> > > >> > // template values
are 1
> and
> > 2,
> > > > >> > > > >>> respectively:
> > > > >> > > > >>> > > >> > >> > > >> > //
> GRIB2_ipdtmpl_index=[8,
> > > > 26];
> > > > >> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > > >> > > > >>> > > >> > >> 2];
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > While I don't have any ECMWF
data
> on
> > > hand
> > > > >> for
> > > > >> > > > >>> testing, I
> > > > >> > > > >>> > > do
> > > > >> > > > >>> > > >> > have
> > > > >> > > > >>> > > >> > >> a
> > > > >> > > > >>> > > >> > >> > > file
> > > > >> > > > >>> > > >> > >> > > >> > from the Air Force from
which
> wgrib2
> > > > >> prints a
> > > > >> > > > >>> similar
> > > > >> > > > >>> > > >> result:
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> >
549:687031550:d=2017061000:var
> > > > discipline=0
> > > > >> > > > >>> center=57
> > > > >> > > > >>> > > >> > >> local_table=1
> > > > >> > > > >>> > > >> > >> > > >> > parmcat=1
parm=192:surface:18 hour
> > > fcst:
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > So I used that for testing.
I ran
> the
> > > > >> > following
> > > > >> > > > >>> > > >> plot_data_plane
> > > > >> > > > >>> > > >> > >> > > command
> > > > >> > > > >>> > > >> > >> > > >> > WITHOUT specifying a
requested
> name:
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > plot_data_plane
> > > > >> GALWEM_20170610_CY00_FH18.GR2
> > > > >> > > > >>> plot.ps
> > > > >> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > > >> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57; GRIB2_ltab=1;
> > > > >> GRIB2_parm_cat=1;
> > > > >> > > > >>> > > >> GRIB2_parm=192;
> > > > >> > > > >>> > > >> > >> > > >> level="Z0";'
> > > > >> > > > >>> > > >> > >> > > >> > -v 3
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > And then I looked closely at
the
> log
> > > > >> message
> > > > >> > to
> > > > >> > > > >>> confirm
> > > > >> > > > >>> > > >> that it
> > > > >> > > > >>> > > >> > >> used
> > > > >> > > > >>> > > >> > >> > > >> > record 549, as expected:
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > DEBUG 3:
> > MetGrib2DataFile::data_plane()
> > > > ->
> > > > >> > Found
> > > > >> > > > >>> exact
> > > > >> > > > >>> > > match
> > > > >> > > > >>> > > >> > for
> > > > >> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record
549
> field
> > 1
> > > of
> > > > >> > GRIB2
> > > > >> > > > file
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > So MET used these GRIB2
numbers I
> > > defined
> > > > >> to
> > > > >> > do
> > > > >> > > a
> > > > >> > > > >>> table
> > > > >> > > > >>> > > >> lookup
> > > > >> > > > >>> > > >> > >> and
> > > > >> > > > >>> > > >> > >> > > find
> > > > >> > > > >>> > > >> > >> > > >> a
> > > > >> > > > >>> > > >> > >> > > >> > match in the MET GRIB
tables:
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > >
> > > > >> > > >
> > > >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > And it determined that that
field
> > > should
> > > > be
> > > > >> > > called
> > > > >> > > > >>> > > "CTSTM".
> > > > >> > > > >>> > > >> > >> > Hopefully,
> > > > >> > > > >>> > > >> > >> > > >> the
> > > > >> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are
already
> > defined
> > > > >> there
> > > > >> > in
> > > > >> > > > >>> MET. If
> > > > >> > > > >>> > > >> not, we
> > > > >> > > > >>> > > >> > >> can
> > > > >> > > > >>> > > >> > >> > > >> always
> > > > >> > > > >>> > > >> > >> > > >> > add them.
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > Please try running
plot_data_plane
> at
> > > > >> > verbosity
> > > > >> > > > >>> level 3
> > > > >> > > > >>> > on
> > > > >> > > > >>> > > >> one
> > > > >> > > > >>> > > >> > of
> > > > >> > > > >>> > > >> > >> > > these
> > > > >> > > > >>> > > >> > >> > > >> > files first. And make sure
it runs
> > > > without
> > > > >> > error
> > > > >> > > > >>> and the
> > > > >> > > > >>> > > log
> > > > >> > > > >>> > > >> > >> message
> > > > >> > > > >>> > > >> > >> > > >> prints
> > > > >> > > > >>> > > >> > >> > > >> > out the "name" it found for
that
> > data.
> > > If
> > > > >> so,
> > > > >> > > you
> > > > >> > > > >>> can
> > > > >> > > > >>> > just
> > > > >> > > > >>> > > >> skip
> > > > >> > > > >>> > > >> > >> > using
> > > > >> > > > >>> > > >> > >> > > >> all
> > > > >> > > > >>> > > >> > >> > > >> > those integers and revert
back to
> > using
> > > > the
> > > > >> > name
> > > > >> > > > MET
> > > > >> > > > >>> > > knows.
> > > > >> > > > >>> > > >> If
> > > > >> > > > >>> > > >> > >> not,
> > > > >> > > > >>> > > >> > >> > > >> then we
> > > > >> > > > >>> > > >> > >> > > >> > may need to define a custom
GRIB
> > table
> > > > for
> > > > >> > this
> > > > >> > > > >>> ECMWF
> > > > >> > > > >>> > > GRIB2
> > > > >> > > > >>> > > >> > data.
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >> > Thanks,
> > > > >> > > > >>> > > >> > >> > > >> > John
> > > > >> > > > >>> > > >> > >> > > >> >
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> --
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> Binbin Zhou
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> College Park, MD 20740
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> 301-683-3683
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >> --
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > >> > > > >>> > > >> > >> > > >> Tara Jensen
> > > > >> > > > >>> > > >> > >> > > >> Project Manager II
> > > > >> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > > >> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder, Colorado
80307
> > USA
> > > > >> > > > >>> > > >> > >> > > >> +1 303-497-8479
> > jensen at ucar.edu
> > > > >> > > > >>> > > >> > >> > > >>
> > > > >> > > > >>> > > >> > >> > > >
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> > >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > --
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > Binbin Zhou
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > 5830 University Research Ct.
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > College Park, MD 20740
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> > 301-683-3683
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >> >
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >>
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > --
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > Binbin Zhou
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > 5830 University Research Ct.
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > College Park, MD 20740
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> > > 301-683-3683
> > > > >> > > > >>> > > >> > >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > --
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > Binbin Zhou
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > 5830 University Research Ct.
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > College Park, MD 20740
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> > 301-683-3683
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >> >
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >>
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > --
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > Binbin Zhou
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > 5830 University Research Ct.
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > College Park, MD 20740
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > Binbin.Zhou at noaa.gov
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > > > 301-683-3683
> > > > >> > > > >>> > > >
> > > > >> > > > >>> > >
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > --
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > Binbin Zhou
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > 5830 University Research Ct.
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > College Park, MD 20740
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > Binbin.Zhou at noaa.gov
> > > > >> > > > >>> > >
> > > > >> > > > >>> > > 301-683-3683
> > > > >> > > > >>> > >
> > > > >> > > > >>> > >
> > > > >> > > > >>> >
> > > > >> > > > >>> >
> > > > >> > > > >>>
> > > > >> > > > >>> --
> > > > >> > > > >>>
> > > > >> > > > >>> Binbin Zhou
> > > > >> > > > >>>
> > > > >> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >>>
> > > > >> > > > >>> 5830 University Research Ct.
> > > > >> > > > >>>
> > > > >> > > > >>> College Park, MD 20740
> > > > >> > > > >>>
> > > > >> > > > >>> Binbin.Zhou at noaa.gov
> > > > >> > > > >>>
> > > > >> > > > >>> 301-683-3683
> > > > >> > > > >>>
> > > > >> > > > >>>
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > Julie Prestopnik (she/her)
> > > > >> > > > Software Engineer
> > > > >> > > > National Center for Atmospheric Research
> > > > >> > > > Research Applications Laboratory
> > > > >> > > > Email: jpresto at ucar.edu
> > > > >> > > >
> > > > >> > > > My working day may not be your working day. Please
do not
> > feel
> > > > >> obliged
> > > > >> > > to
> > > > >> > > > reply to this email outside of your normal working
hours.
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> > --
> > > > >> > Julie Prestopnik (she/her)
> > > > >> > Software Engineer
> > > > >> > National Center for Atmospheric Research
> > > > >> > Research Applications Laboratory
> > > > >> > Email: jpresto at ucar.edu
> > > > >> >
> > > > >> > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > >> to
> > > > >> > reply to this email outside of your normal working hours.
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > >
> > > > > Binbin Zhou
> > > > >
> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >
> > > > > 5830 University Research Ct.
> > > > >
> > > > > College Park, MD 20740
> > > > >
> > > > > Binbin.Zhou at noaa.gov
> > > > >
> > > > > 301-683-3683
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: John Halley Gotway
Time: Mon Dec 14 12:20:10 2020
Binbin,
FYI, I just submitted a pull request to merge these changes in for the
met-10.0.0 release.
https://github.com/dtcenter/MET/pull/1608
And once Julie reviews those changes, we'll merge them for inclusion
in the
met-10.0.0 release.
I'll go ahead and resolve this ticket.
Thanks,
John
On Fri, Dec 11, 2020 at 1:21 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
>
> John,
>
> Including in MET-10.0.0 should be fine.
>
> Binbin
>
> On Fri, Dec 11, 2020 at 2:54 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binbin,
> >
> > Great, thanks for confirming. OK, next question is about versions
and
> > timelines.
> >
> > Do we need to create a met-9.1.2 bugfix version that includes this
> change?
> > Or is it sufficient to just include in the met-10.0.0 release?
> >
> > If we need to create a met-9.1.2 bugfix release, do you have a
deadline
> for
> > that?
> >
> > Thanks,
> > John
> >
> > On Fri, Dec 11, 2020 at 12:05 PM Binbin.Zhou at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > >
> > > John,
> > >
> > > This it works!
> > >
> > > Thanks!
> > >
> > > Binbin
> > >
> > > On Fri, Dec 11, 2020 at 12:47 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Binbin,
> > > >
> > > > Those * characters don't belong in the plot_data_plane
command.
> That's
> > > the
> > > > way the mail system formats bold text. To be totally clear,
here are
> > the
> > > > exact commands I ran after logging on to Venus to test:
> > > >
> > > > cd
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > > module use
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > module load met/ens_config_test
> > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 ~/plot.ps
'name="HGT";
> > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > > >
> > > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> for
> > > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > > > matching VarInfo "HGT/P500" in GRIB file
> "ecmanl.t12z.grid3.f00.grib1".
> > > > DEBUG 1: Creating postscript file: /u/John.H.Gotway/plot.ps
> > > >
> > > > That matches a single record, namely the 3rd record in that
GRIB
> file.
> > Do
> > > > you see different behavior? Perhaps you failed to load the
> > > > "ens_config_test" version of the code before testing?
> > > >
> > > > John
> > > >
> > > > On Fri, Dec 11, 2020 at 10:20 AM Binbin.Zhou at noaa.gov via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > > >
> > > > > John,
> > > > >
> > > > > As I tested following plotting
> > > > >
> > > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
'*name="HGT";
> > > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v 3
> > > > >
> > > > > the error message is
> > > > >
> > > > > ERROR :
> > > > > ERROR : yyerror() -> syntax error in file
"/tmp/met_config_9350_0"
> > > > > ERROR :
> > > > > ERROR : line = 1
> > > > > ERROR :
> > > > > ERROR : column = 1
> > > > > ERROR :
> > > > > ERROR : text = "*"
> > > > > ERROR :
> > > > > ERROR :
> > > > > ERROR : *name="HGT"; level="P500"; lead_time="00";
> > > > > GRIB_ens="low_res_ctl";*
> > > > > ERROR :
> > > > >
^__________________________________________________________________
> > > > > ERROR :
> > > > >
> > > > > After I removed "*"
> > > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
'name="HGT";
> > > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > > > > It shows 3 records matched
> > > > >
> > > > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> > for
> > > > > VarInfo "HGT/P500" in GRIB record 1 of GRIB file
> > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> > for
> > > > > VarInfo "HGT/P500" in GRIB record 2 of GRIB file
> > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> > for
> > > > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3
GRIB
> records
> > > > > matching VarInfo "HGT/P500" in GRIB file
> > "ecmanl.t12z.grid3.f00.grib1".
> > > > > DEBUG 1: Creating postscript file: plot.ps
> > > > >
> > > > > My testing directory on Venus
> > > > > is
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > > > test script is ploting.sh
> > > > >
> > > > > Can you look at it?
> > > > >
> > > > > Thanks!
> > > > > Binbin
> > > > >
> > > > >
> > > > > On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA Affiliate
<
> > > > > binbin.zhou at noaa.gov> wrote:
> > > > >
> > > > > > John,
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Binbin
> > > > > >
> > > > > > On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > >> Thanks Julie.
> > > > > >>
> > > > > >> And Binbin, here's an EnsembleStatConfig file where
climos are
> > used:
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
> > > > > >>
> > > > > >> Since the list of forecast fields is exactly the same as
the
> climo
> > > > > fields,
> > > > > >> this config file just copies them over using:
> > > > > >> climo_mean = fcst;
> > > > > >>
> > > > > >> In your case, when they differ, you'd just specify the
"field"
> > array
> > > > > >> inside
> > > > > >> the "climo_mean" dictionary. For example:
> > > > > >>
> > > > > >> climo_mean = {
> > > > > >>
> > > > > >> field = [
> > > > > >> { name = "TMP"; level = "Z2"; },
> > > > > >> { name = "TMP"; level = "P850"; }
> > > > > >> ];
> > > > > >> ...
> > > > > >> }
> > > > > >>
> > > > > >> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via RT
<
> > > > > >> met_help at ucar.edu>
> > > > > >> wrote:
> > > > > >>
> > > > > >> >
> > > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> >
> > > > > >> >
> > > > > >> > Thanks, John. That worked great.
> > > > > >> >
> > > > > >> > Binbin, this version of MET is now accessible to you by
> running
> > > the
> > > > > >> > following on venus:
> > > > > >> >
> > > > > >> > module use
> > > > > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > >> > module load met/ens_config_test
> > > > > >> >
> > > > > >> > Please let us know if you encounter any problems.
> > > > > >> >
> > > > > >> > Julie
> > > > > >> >
> > > > > >> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway via
RT <
> > > > > >> > met_help at ucar.edu> wrote:
> > > > > >> >
> > > > > >> > >
> > > > > >> > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > > > > >> > >
> > > > > >> > > Julie,
> > > > > >> > >
> > > > > >> > > Building the tarfile is very easy. I just ran this
script
> from
> > > the
> > > > > MET
> > > > > >> > repo
> > > > > >> > > and specified the branch to be built:
> > > > > >> > > MET/scripts/met_checkout_and_build.sh branch
> > > > > >> feature_1604_ens_config
> > > > > >> > >
> > > > > >> > > And that created a tarfile which I posted to RAL's
anonymous
> > ftp
> > > > > >> server:
> > > > > >> > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> > > > > >> > >
> > > > > >> > > I assume that'll work for you.
> > > > > >> > >
> > > > > >> > > Thanks,
> > > > > >> > > John
> > > > > >> > >
> > > > > >> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik via
RT <
> > > > > >> > met_help at ucar.edu
> > > > > >> > > >
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > >
> > > > > >> > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >
> > > > > >> > > >
> > > > > >> > > > John,
> > > > > >> > > >
> > > > > >> > > > I am able to help with this install on WCOSS.
> > > > > >> > > >
> > > > > >> > > > Would it be easier to use git to checkout the repo
on
> WCOSS?
> > > Or
> > > > > >> would
> > > > > >> > it
> > > > > >> > > be
> > > > > >> > > > > easier if I built a tarfile for this version of
the
> code?
> > > > > >> > > > >
> > > > > >> > > > The last time I tried to build from git on WCOSS (a
while
> > > ago),
> > > > I
> > > > > >> > > > couldn't. I don't remember all of the details, but
I
> think
> > I
> > > > > >> couldn't
> > > > > >> > > run
> > > > > >> > > > bootstrap and maybe it had to do with the version
of
> > autoconf
> > > > > being
> > > > > >> too
> > > > > >> > > > old? I'm happy to try it again if you'd like
though.
> > > > Otherwise,
> > > > > I
> > > > > >> > know
> > > > > >> > > a
> > > > > >> > > > tarfile would work, but I know that is extra work
for you.
> > > > > >> > > >
> > > > > >> > > > Julie
> > > > > >> > > >
> > > > > >> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway <
> > > > > johnhg at ucar.edu>
> > > > > >> > > wrote:
> > > > > >> > > >
> > > > > >> > > > > Julie,
> > > > > >> > > > >
> > > > > >> > > > > I have created a bugfix branch for main_v9.1
named
> > > > > >> > > > > "feature_1604_ens_config" and it's up on GitHub.
> > > > > >> > > > >
> > > > > >> > > > > Only a single file has changed in it. But I need
to get
> > this
> > > > > >> version
> > > > > >> > > > > compiled on WCOSS with which Binbin can test.
> > > > > >> > > > >
> > > > > >> > > > > Are you to help me with this? Would it be easier
to use
> > git
> > > to
> > > > > >> > checkout
> > > > > >> > > > > the repo on WCOSS? Or would it be easier if I
built a
> > > tarfile
> > > > > for
> > > > > >> > this
> > > > > >> > > > > version of the code?
> > > > > >> > > > >
> > > > > >> > > > > Thanks,
> > > > > >> > > > > John
> > > > > >> > > > >
> > > > > >> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley Gotway
<
> > > > > >> johnhg at ucar.edu>
> > > > > >> > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > >> Binbin,
> > > > > >> > > > >>
> > > > > >> > > > >> In order for you to test this change out, we'd
need to
> > > > > recompile
> > > > > >> a
> > > > > >> > > > >> version of the code on WCOSS. I wrote this up as
a
> GitHub
> > > > > issue:
> > > > > >> > > > >> https://github.com/dtcenter/MET/issues/1604
> > > > > >> > > > >>
> > > > > >> > > > >> And I'll create a branch from main_v9.1 named
> > > > > >> > "bugfix_1604_ens_config"
> > > > > >> > > > >> and commit the change there. Then we can
checkout that
> > code
> > > > on
> > > > > >> > WCOSS,
> > > > > >> > > > >> compile it, and you can test it.
> > > > > >> > > > >>
> > > > > >> > > > >> Yes, you're right, if the fcst fields differ
from the
> > climo
> > > > > >> fields,
> > > > > >> > > > you'd
> > > > > >> > > > >> need to specify them separately. That's pretty
easy in
> > the
> > > > > >> > > Ensemble-Stat
> > > > > >> > > > >> config file... but are you asking how to control
that
> via
> > > > > >> METplus?
> > > > > >> > > > >>
> > > > > >> > > > >> John
> > > > > >> > > > >>
> > > > > >> > > > >> On Wed, Dec 9, 2020 at 2:29 PM
Binbin.Zhou at noaa.gov
> via
> > > RT <
> > > > > >> > > > >> met_help at ucar.edu> wrote:
> > > > > >> > > > >>
> > > > > >> > > > >>>
> > > > > >> > > > >>> <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > >
> > > > > >> > > > >>>
> > > > > >> > > > >>> John,
> > > > > >> > > > >>>
> > > > > >> > > > >>> Great! Let me test it at first before make
change in
> > > > MET9.1
> > > > > >> > > > >>> Another question:
> > > > > >> > > > >>> Since climatology data is still in NCEP's
grib2
> > format,
> > > > In
> > > > > >> the
> > > > > >> > > > >>> ensemble_stat
> > > > > >> > > > >>> config file, climo entry settings should be
> different
> > > > from
> > > > > >> those
> > > > > >> > > of
> > > > > >> > > > >>> fcst, obs entries?
> > > > > >> > > > >>> How to set climo (name, level, etc.)?
> > > > > >> > > > >>>
> > > > > >> > > > >>> Binbin
> > > > > >> > > > >>>
> > > > > >> > > > >>>
> > > > > >> > > > >>>
> > > > > >> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley
Gotway via
> > RT <
> > > > > >> > > > >>> met_help at ucar.edu>
> > > > > >> > > > >>> wrote:
> > > > > >> > > > >>>
> > > > > >> > > > >>> > Binbin,
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > Success! When I recompile with that one line
> commented
> > > > out,
> > > > > >> > > > >>> plot_data_plane
> > > > > >> > > > >>> > is able to filter down all those records to
the
> single
> > > one
> > > > > >> that
> > > > > >> > we
> > > > > >> > > > >>> want.
> > > > > >> > > > >>> > Note in the example below that I used the
> "lead_time"
> > > and
> > > > > >> > > "GRIB_ens"
> > > > > >> > > > >>> > options to filter the input records down.
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > plot_data_plane ensposte.t00z.z500hr plot.ps
> > > > '*name="HGT";
> > > > > >> > > > >>> level="P500";
> > > > > >> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*' -v
3
> > > > > >> > > > >>> > DEBUG 1: Opening data file:
ensposte.t00z.z500hr
> > > > > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array()
->
> > *Found
> > > > > exact
> > > > > >> > match
> > > > > >> > > > for
> > > > > >> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of GRIB
file
> > > > > >> > > > >>> "ensposte.t00z.z500hr".*
> > > > > >> > > > >>> > DEBUG 3: MetGrib1DataFile::data_plane_array()
->
> > Found 1
> > > > > GRIB
> > > > > >> > > records
> > > > > >> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> > > > > >> "ensposte.t00z.z500hr".
> > > > > >> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > Also note that this selects out record number
26.
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > Shall I make this change as a "bugfix" for
> > > met-9.1...which
> > > > > >> could
> > > > > >> > be
> > > > > >> > > > >>> > included in a met-9.1.2 bugfix release? Or
shall I
> > just
> > > > > >> include
> > > > > >> > it
> > > > > >> > > in
> > > > > >> > > > >>> > development for the next release (planned for
March
> > > 2021)?
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > Thanks,
> > > > > >> > > > >>> > John
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM
Binbin.Zhou at noaa.gov
> > via
> > > > RT
> > > > > <
> > > > > >> > > > >>> > met_help at ucar.edu> wrote:
> > > > > >> > > > >>> >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > <URL:
> > > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > >> > >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > John,
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > If this removing impacts other
applications, or
> > > takes
> > > > > some
> > > > > >> > time
> > > > > >> > > > >>> > > to be effective, then won't try. I will
use
> another
> > > > > >> approach,
> > > > > >> > > such
> > > > > >> > > > >>> as
> > > > > >> > > > >>> > > borrow GEFS's grib2 data structure as tank
to save
> > EC
> > > > > >> ensemble
> > > > > >> > > > >>> > > data.
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > Binbin
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin Zhou
- NOAA
> > > > > Affiliate
> > > > > >> <
> > > > > >> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > > John,
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > Yes, please try to use this approach.
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > Thanks!
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > Binbin
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > Binbin
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John
Halley
> Gotway
> > > via
> > > > > RT <
> > > > > >> > > > >>> > > > met_help at ucar.edu> wrote:
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > >> Binbin,
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> I used the "-PDS10" option for wgrib to
dump
> > those
> > > > > >> records:
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > > > >> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > > > >> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7 100
1 244
> 20
> > 12
> > > > 6 0
> > > > > >> 0 1
> > > > > >> > 0
> > > > > >> > > 0
> > > > > >> > > > 1
> > > > > >> > > > >>> 0 0
> > > > > >> > > > >>> > 0
> > > > > >> > > > >>> > > 21
> > > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1
255
> > > > > >> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7 100
1 244
> 20
> > 12
> > > > 6 0
> > > > > >> 0 1
> > > > > >> > 0
> > > > > >> > > 0
> > > > > >> > > > 1
> > > > > >> > > > >>> 0 0
> > > > > >> > > > >>> > 0
> > > > > >> > > > >>> > > 21
> > > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1
255
> > > > > >> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7 100
1 244
> 20
> > 12
> > > > 6 0
> > > > > >> 0 1
> > > > > >> > 0
> > > > > >> > > 0
> > > > > >> > > > 1
> > > > > >> > > > >>> 0 0
> > > > > >> > > > >>> > 0
> > > > > >> > > > >>> > > 21
> > > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1
255
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> Looking VERY closely, there is a
difference in
> > the
> > > > > 42-nd
> > > > > >> > > column
> > > > > >> > > > >>> which
> > > > > >> > > > >>> > is
> > > > > >> > > > >>> > > >> the 4-th from the last... records 11,
26, and
> 43
> > > have
> > > > > >> values
> > > > > >> > > of
> > > > > >> > > > >>> 3, 1,
> > > > > >> > > > >>> > > and
> > > > > >> > > > >>> > > >> 2
> > > > > >> > > > >>> > > >> respectively.
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> On this page (
> > > > > >> > > > >>> > >
> > > > > https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > > > >> > > > >>> > > >> ),
> > > > > >> > > > >>> > > >> byte 42 = 1 means "unperturbed control
> forecast".
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> So we *should* be able to use the
"GRIB_ens"
> > config
> > > > > >> option
> > > > > >> > to
> > > > > >> > > > >>> > > >> differentiate
> > > > > >> > > > >>> > > >> between those records. BUT when I try,
it
> doesn't
> > > > work.
> > > > > >> And
> > > > > >> > > > that's
> > > > > >> > > > >>> > > because
> > > > > >> > > > >>> > > >> of this line:
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> We only apply those filtering options to
GRIB
> > files
> > > > > >> created
> > > > > >> > by
> > > > > >> > > > >>> NCEP.
> > > > > >> > > > >>> > If
> > > > > >> > > > >>> > > >> you'd like, I could try removing that
condition
> > to
> > > > see
> > > > > >> if we
> > > > > >> > > can
> > > > > >> > > > >>> pick
> > > > > >> > > > >>> > > out
> > > > > >> > > > >>> > > >> the single record of interest.
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> John
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
> > > Binbin.Zhou at noaa.gov
> > > > > via
> > > > > >> RT
> > > > > >> > <
> > > > > >> > > > >>> > > >> met_help at ucar.edu> wrote:
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > <URL:
> > > > > >> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > >> > > > >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > John,
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > Look at my
> > > > > >> > > > >>> > > >> > directory
> > > > > >> > > > >>>
> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > > > >> > > > >>> > > >> > The grib1 file is
ensposte.t00z.z500hr, this
> is
> > > > only
> > > > > >> for
> > > > > >> > 500
> > > > > >> > > > >>> HGT.
> > > > > >> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep
anl|grep
> > > > > >> anl:type=analysis
> > > > > >> > > > >>> > > >> > I get 3 records:
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > From its grib2 file, we know only
record#26
> is
> > > > > >> analysis
> > > > > >> > > > low-res
> > > > > >> > > > >>> > > >> control.
> > > > > >> > > > >>> > > >> > If you have some way to determine
which
> record
> > is
> > > > > >> analysis
> > > > > >> > > > >>> low-res
> > > > > >> > > > >>> > > >> control
> > > > > >> > > > >>> > > >> > from the grib1 file,
> > > > > >> > > > >>> > > >> > then we can use grib1 directly (don't
> consider
> > > the
> > > > > size
> > > > > >> > > > issue).
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > Binbin
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM Binbin
Zhou -
> > NOAA
> > > > > >> > Affiliate
> > > > > >> > > <
> > > > > >> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > > John,
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > Grib1 data is too large, another
issue is
> > the
> > > > > >> low-res
> > > > > >> > > > >>> control
> > > > > >> > > > >>> > > >> analysis
> > > > > >> > > > >>> > > >> > > data can not be retrieved
> > > > > >> > > > >>> > > >> > > from the grib1 ecmwf data files. If
using
> > > wgrib,
> > > > we
> > > > > >> can
> > > > > >> > > see
> > > > > >> > > > 3
> > > > > >> > > > >>> > > analysis
> > > > > >> > > > >>> > > >> > > fields, but don't know
> > > > > >> > > > >>> > > >> > > which is low-res control. After
convert to
> > > grib2,
> > > > > >> wgrib2
> > > > > >> > > can
> > > > > >> > > > >>> show
> > > > > >> > > > >>> > > >> which
> > > > > >> > > > >>> > > >> > > record is low-res control.
> > > > > >> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf
grib1 data
> > set
> > > > on
> > > > > >> > WCOSS.
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > Binbin
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM John
Halley
> > > Gotway
> > > > > via
> > > > > >> > RT <
> > > > > >> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > >> Binbin,
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >> Sounds like there's a lot of moving
parts
> > > there.
> > > > > How
> > > > > >> > > would
> > > > > >> > > > >>> you
> > > > > >> > > > >>> > like
> > > > > >> > > > >>> > > >> to
> > > > > >> > > > >>> > > >> > >> proceed?
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >> Should we check to see if MET can
read
> those
> > > > ECMWF
> > > > > >> > GRIB1
> > > > > >> > > > >>> files
> > > > > >> > > > >>> > > >> directly
> > > > > >> > > > >>> > > >> > >> without needing to reformat?
> > > > > >> > > > >>> > > >> > >> Or do you just want to create a
custom
> GRIB
> > > > table
> > > > > >> for
> > > > > >> > the
> > > > > >> > > > >>> result
> > > > > >> > > > >>> > of
> > > > > >> > > > >>> > > >> your
> > > > > >> > > > >>> > > >> > >> pre-processing and conversion
steps?
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >> John
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
> > > > > Binbin.Zhou at noaa.gov
> > > > > >> > via
> > > > > >> > > > RT <
> > > > > >> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > <URL:
> > > > > >> > > > >>>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > John,
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data
files
> are
> > > > > >> converted
> > > > > >> > > from
> > > > > >> > > > >>> its
> > > > > >> > > > >>> > > grib1
> > > > > >> > > > >>> > > >> > >> format
> > > > > >> > > > >>> > > >> > >> > data.
> > > > > >> > > > >>> > > >> > >> > Before conversion, the parameter
names
> are
> > > > > there,
> > > > > >> but
> > > > > >> > > > after
> > > > > >> > > > >>> > > >> > conversion,
> > > > > >> > > > >>> > > >> > >> the
> > > > > >> > > > >>> > > >> > >> > names
> > > > > >> > > > >>> > > >> > >> > are gone. It is because of
unknown
> tables.
> > > The
> > > > > >> grib1
> > > > > >> > > data
> > > > > >> > > > >>> is
> > > > > >> > > > >>> > > still
> > > > > >> > > > >>> > > >> > >> > original. The original
> > > > > >> > > > >>> > > >> > >> > grib1 data files are very strange
data
> > > format
> > > > we
> > > > > >> can
> > > > > >> > > not
> > > > > >> > > > >>> read.
> > > > > >> > > > >>> > So
> > > > > >> > > > >>> > > >> we
> > > > > >> > > > >>> > > >> > >> have
> > > > > >> > > > >>> > > >> > >> > to use
> > > > > >> > > > >>> > > >> > >> > a Frotran code to re-arrange the
fields
> > and
> > > > save
> > > > > >> them
> > > > > >> > > as
> > > > > >> > > > >>> > > NCEP-like
> > > > > >> > > > >>> > > >> > grib1
> > > > > >> > > > >>> > > >> > >> > data files.
> > > > > >> > > > >>> > > >> > >> > So after several processes, some
> important
> > > > info
> > > > > >> like
> > > > > >> > > > tables
> > > > > >> > > > >>> > could
> > > > > >> > > > >>> > > >> be
> > > > > >> > > > >>> > > >> > >> lost.
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > I'll test your first
suggestion since
> > the
> > > > > field
> > > > > >> > > record
> > > > > >> > > > >>> order
> > > > > >> > > > >>> > > is
> > > > > >> > > > >>> > > >> > >> fixed.
> > > > > >> > > > >>> > > >> > >> > Currently we still
> > > > > >> > > > >>> > > >> > >> > obtained grib1 data from ECWMF,
> > eventually,
> > > we
> > > > > >> will
> > > > > >> > get
> > > > > >> > > > >>> grib2
> > > > > >> > > > >>> > > data
> > > > > >> > > > >>> > > >> > >> > directly. So before
> > > > > >> > > > >>> > > >> > >> > we can get original grib2 files
from
> > ECMWF,
> > > we
> > > > > >> just
> > > > > >> > use
> > > > > >> > > > >>> this
> > > > > >> > > > >>> > > >> temporal
> > > > > >> > > > >>> > > >> > >> > solution.
> > > > > >> > > > >>> > > >> > >> > '
> > > > > >> > > > >>> > > >> > >> > Thanks!
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM
John
> Halley
> > > > > Gotway
> > > > > >> via
> > > > > >> > > RT
> > > > > >> > > > <
> > > > > >> > > > >>> > > >> > >> > met_help at ucar.edu>
> > > > > >> > > > >>> > > >> > >> > wrote:
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > > Binbin,
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > OK, working on Venus.
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > You have a typo in the command
you
> sent.
> > > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > > >> > a.ps
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> >
> > > > >
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > > >> > > > >>> > > >> > >> > -v
> > > > > >> > > > >>> > > >> > >> > > 3
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that
should
> be
> > > > > >> > > > "GRIB2_parm_cat".
> > > > > >> > > > >>> > After
> > > > > >> > > > >>> > > >> > >> > correcting
> > > > > >> > > > >>> > > >> > >> > > that I did still get an error
but
> it's a
> > > > very
> > > > > >> > useful
> > > > > >> > > > one!
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > module use
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > >
> /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > >> > > > >>> > > >> > >> > > module load met/9.1
> > > > > >> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > > > >> > > > >>> > > >> > >> > > cd
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > > > >> > > > >>> > > >> > >> > > $WGRIB2
ensposte.t00z.z500.grd3.grib2
> > > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > > >> ~/
> > > > > >> > > > plot.ps
> > > > > >> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > > > >> > > > >>> > > >> > >> > > GRIB2_mtab=2; GRIB2_parm_cat=3;
> > > > GRIB2_parm=5;
> > > > > >> > > > >>> level="P500";'
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > *ERROR : ERROR :
> > > VarInfoGrib2::set_dict()
> > > > ->
> > > > > >> no
> > > > > >> > > > >>> parameter
> > > > > >> > > > >>> > > found
> > > > > >> > > > >>> > > >> > with
> > > > > >> > > > >>> > > >> > >> > > matching GRIB2_disc (255)
> GRIB2_parm_cat
> > > (3)
> > > > > >> > > GRIB2_parm
> > > > > >> > > > >>> (5).
> > > > > >> > > > >>> > > Use
> > > > > >> > > > >>> > > >> the
> > > > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable
> to
> > > > define
> > > > > >> > custom
> > > > > >> > > > >>> GRIB
> > > > > >> > > > >>> > > >> > >> tables.ERROR
> > > > > >> > > > >>> > > >> > >> > :
> > > > > >> > > > >>> > > >> > >> > > *
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > So this table is not yet
defined in
> > MET. I
> > > > > >> checked
> > > > > >> > > > their
> > > > > >> > > > >>> > > website:
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > But I don't see an entry
defined for
> > > > > >> GRIB2_parm=5;
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > 3 things...
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > (1) As a crude temporary
solution, you
> > > could
> > > > > >> refer
> > > > > >> > to
> > > > > >> > > > it
> > > > > >> > > > >>> by
> > > > > >> > > > >>> > > it's
> > > > > >> > > > >>> > > >> > >> record
> > > > > >> > > > >>> > > >> > >> > > number. Let's assume you know
that
> > record
> > > 1
> > > > > has
> > > > > >> > > > >>> temperature
> > > > > >> > > > >>> > > data:
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > > >> ~/
> > > > > >> > > > plot.ps
> > > > > >> > > > >>> > > >> > 'name="TMP";
> > > > > >> > > > >>> > > >> > >> > > level="R1";
set_attr_level="P500";'
> -v 5
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > Here I'm reading record 1,
naming it
> TMP
> > > and
> > > > > the
> > > > > >> > > using
> > > > > >> > > > >>> > > >> > set_attr_level
> > > > > >> > > > >>> > > >> > >> > write
> > > > > >> > > > >>> > > >> > >> > > the output level as P500
instead of
> R1.
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > (2) You could create a custom
GRIB
> table
> > > > > >> yourself
> > > > > >> > > > >>> locally,
> > > > > >> > > > >>> > and
> > > > > >> > > > >>> > > >> then
> > > > > >> > > > >>> > > >> > >> set
> > > > > >> > > > >>> > > >> > >> > the
> > > > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable
> to
> > > > point
> > > > > >> MET
> > > > > >> > to
> > > > > >> > > > it.
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > (3) The best solution is
contacting
> > ECMWF
> > > > and
> > > > > >> > > > requesting
> > > > > >> > > > >>> they
> > > > > >> > > > >>> > > >> send
> > > > > >> > > > >>> > > >> > >> over
> > > > > >> > > > >>> > > >> > >> > the
> > > > > >> > > > >>> > > >> > >> > > GRIB2 table we should be using
to
> decode
> > > > this
> > > > > >> data.
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > Thanks,
> > > > > >> > > > >>> > > >> > >> > > John
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30 PM
John
> > Halley
> > > > > >> Gotway <
> > > > > >> > > > >>> > > >> johnhg at ucar.edu>
> > > > > >> > > > >>> > > >> > >> > wrote:
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > > > We might already have them.
I'm not
> > > sure.
> > > > > >> > > > >>> > > >> > >> > > >
> > > > > >> > > > >>> > > >> > >> > > > But there's an additional
required
> > piece
> > > > of
> > > > > >> the
> > > > > >> > > > puzzle.
> > > > > >> > > > >>> > > Binbin
> > > > > >> > > > >>> > > >> > runs
> > > > > >> > > > >>> > > >> > >> > > > "wgrib2" to figure out what
"name"
> he
> > > > should
> > > > > >> use
> > > > > >> > in
> > > > > >> > > > >>> the MET
> > > > > >> > > > >>> > > >> config
> > > > > >> > > > >>> > > >> > >> > files.
> > > > > >> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him
the right
> > > > "name",
> > > > > >> the
> > > > > >> > > > >>> problem
> > > > > >> > > > >>> > will
> > > > > >> > > > >>> > > >> > >> remain.
> > > > > >> > > > >>> > > >> > >> > > > wgrib2 can be configured to
define
> > > > > additional
> > > > > >> > > tables
> > > > > >> > > > as
> > > > > >> > > > >>> > well,
> > > > > >> > > > >>> > > >> but
> > > > > >> > > > >>> > > >> > >> for
> > > > > >> > > > >>> > > >> > >> > > this
> > > > > >> > > > >>> > > >> > >> > > > to be easy for Binbin, that'd
mean
> > > > updating
> > > > > >> the
> > > > > >> > > > script
> > > > > >> > > > >>> > that's
> > > > > >> > > > >>> > > >> run
> > > > > >> > > > >>> > > >> > >> for
> > > > > >> > > > >>> > > >> > >> > > > "module load grib_utils" on
WCOSS.
> And
> > > I'm
> > > > > >> > guessing
> > > > > >> > > > >>> that's
> > > > > >> > > > >>> > > not
> > > > > >> > > > >>> > > >> > very
> > > > > >> > > > >>> > > >> > >> > easy.
> > > > > >> > > > >>> > > >> > >> > > >
> > > > > >> > > > >>> > > >> > >> > > > Yes, I do agree that the best
> solution
> > > is
> > > > > >> getting
> > > > > >> > > the
> > > > > >> > > > >>> ECMWF
> > > > > >> > > > >>> > > >> GRIB
> > > > > >> > > > >>> > > >> > >> tables
> > > > > >> > > > >>> > > >> > >> > > > and adding them for both
wgrib2 and
> > MET.
> > > > > >> > > > >>> > > >> > >> > > >
> > > > > >> > > > >>> > > >> > >> > > > John
> > > > > >> > > > >>> > > >> > >> > > >
> > > > > >> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32
PM Tara
> > > > Jensen <
> > > > > >> > > > >>> > jensen at ucar.edu>
> > > > > >> > > > >>> > > >> > wrote:
> > > > > >> > > > >>> > > >> > >> > > >
> > > > > >> > > > >>> > > >> > >> > > >> John,
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> It seems to me that it would
be
> > > > worthwhile
> > > > > to
> > > > > >> > just
> > > > > >> > > > ask
> > > > > >> > > > >>> > ECMWF
> > > > > >> > > > >>> > > >> for
> > > > > >> > > > >>> > > >> > >> their
> > > > > >> > > > >>> > > >> > >> > > >> Grib2 Table for their
Ensemble
> files.
> > > > And
> > > > > >> other
> > > > > >> > > > >>> > operational
> > > > > >> > > > >>> > > >> > >> centers
> > > > > >> > > > >>> > > >> > >> > for
> > > > > >> > > > >>> > > >> > >> > > >> that matter. Do you agree
this
> would
> > > be
> > > > a
> > > > > >> good
> > > > > >> > > > >>> solution?
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> Thanks, Tara
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> ---------- Forwarded message
> > ---------
> > > > > >> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov
via RT
> <
> > > > > >> > > > met_help at ucar.edu>
> > > > > >> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at
11:06 AM
> > > > > >> > > > >>> > > >> > >> > > >> Subject: Re:
[rt.rap.ucar.edu
> > #97769]
> > > > > urgent
> > > > > >> > > > >>> question for
> > > > > >> > > > >>> > > >> > reading
> > > > > >> > > > >>> > > >> > >> > ECMWF
> > > > > >> > > > >>> > > >> > >> > > >> ensmeble data
> > > > > >> > > > >>> > > >> > >> > > >> To:
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> <URL:
> > > > > >> > > > >>> > >
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> John,
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> I tested using following
command:
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > > > >> > > > a.ps
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>>
> > 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > > >> > > > >>> > > >> -v3
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict()
> ->
> > > > either
> > > > > >> name
> > > > > >> > > or
> > > > > >> > > > >>> > > >> GRIB2_disc,
> > > > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and
GRIB2_parm must
> be
> > > > > >> specified
> > > > > >> > in
> > > > > >> > > > >>> field
> > > > > >> > > > >>> > > >> > >> information
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > > > >> > > > a.ps
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> >
> > > > >
> > >
>
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > > >> > > > >>> > > >> > >> > > >> -v
> > > > > >> > > > >>> > > >> > >> > > >> 3
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> Still got error
> > > > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict()
> ->
> > > > either
> > > > > >> name
> > > > > >> > > or
> > > > > >> > > > >>> > > >> GRIB2_disc,
> > > > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and
GRIB2_parm must
> be
> > > > > >> specified
> > > > > >> > in
> > > > > >> > > > >>> field
> > > > > >> > > > >>> > > >> > >> information
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> Looks like discipline=255 is
not
> > > > recognized
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> Binbin
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at 11:57
AM
> John
> > > > Halley
> > > > > >> > Gotway
> > > > > >> > > > >>> via RT
> > > > > >> > > > >>> > <
> > > > > >> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > > > >> > > > >>> > > >> > >> > > >> wrote:
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> > Binbin,
> > > > > >> > > > >>> > > >> > >> > > >> > You can do something
similar with
> > > MET.
> > > > > >> Please
> > > > > >> > go
> > > > > >> > > > to
> > > > > >> > > > >>> this
> > > > > >> > > > >>> > > >> link:
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > >
> > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > And then search for
"GRIB2_mtab",
> > and
> > > > > >> you'll
> > > > > >> > > find
> > > > > >> > > > >>> the
> > > > > >> > > > >>> > > >> content
> > > > > >> > > > >>> > > >> > >> > > >> > copied/pasted below. You
can
> > specify
> > > > > these
> > > > > >> > > INSTEAD
> > > > > >> > > > >>> OF
> > > > > >> > > > >>> > > >> > specifying
> > > > > >> > > > >>> > > >> > >> a
> > > > > >> > > > >>> > > >> > >> > > >> > requested name.
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_mtab" is an
> > > > integer
> > > > > >> > > > >>> specifying the
> > > > > >> > > > >>> > > >> master
> > > > > >> > > > >>> > > >> > >> > table
> > > > > >> > > > >>> > > >> > >> > > >> > number.
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ltab" is an
> > > > integer
> > > > > >> > > > >>> specifying the
> > > > > >> > > > >>> > > >> local
> > > > > >> > > > >>> > > >> > >> table
> > > > > >> > > > >>> > > >> > >> > > >> > number.
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_disc" is an
> > > > integer
> > > > > >> > > > >>> specifying the
> > > > > >> > > > >>> > > >> GRIB2
> > > > > >> > > > >>> > > >> > >> > > >> discipline
> > > > > >> > > > >>> > > >> > >> > > >> > code.
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_parm_cat"
> is
> > an
> > > > > >> integer
> > > > > >> > > > >>> specifying
> > > > > >> > > > >>> > > the
> > > > > >> > > > >>> > > >> > >> > parameter
> > > > > >> > > > >>> > > >> > >> > > >> > category
> > > > > >> > > > >>> > > >> > >> > > >> > // code.
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_parm" is an
> > > > integer
> > > > > >> > > > >>> specifying the
> > > > > >> > > > >>> > > >> > >> parameter
> > > > > >> > > > >>> > > >> > >> > > code.
> > > > > >> > > > >>> > > >> > >> > > >> > // - The "GRIB2_pdt"
is an
> > > > integer
> > > > > >> > > > specifying
> > > > > >> > > > >>> the
> > > > > >> > > > >>> > > >> product
> > > > > >> > > > >>> > > >> > >> > > >> definition
> > > > > >> > > > >>> > > >> > >> > > >> > // template (Table
4.0).
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_process" is
> > an
> > > > > >> integer
> > > > > >> > > > >>> specifying
> > > > > >> > > > >>> > > the
> > > > > >> > > > >>> > > >> > >> > generating
> > > > > >> > > > >>> > > >> > >> > > >> > process
> > > > > >> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_cntr" is an
> > > > integer
> > > > > >> > > > >>> specifying the
> > > > > >> > > > >>> > > >> > >> originating
> > > > > >> > > > >>> > > >> > >> > > >> > center.
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ens_type"
> is
> > an
> > > > > >> integer
> > > > > >> > > > >>> specifying
> > > > > >> > > > >>> > > the
> > > > > >> > > > >>> > > >> > >> > ensemble
> > > > > >> > > > >>> > > >> > >> > > >> type
> > > > > >> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_der_type"
> is
> > an
> > > > > >> integer
> > > > > >> > > > >>> specifying
> > > > > >> > > > >>> > > the
> > > > > >> > > > >>> > > >> > >> derived
> > > > > >> > > > >>> > > >> > >> > > >> > product
> > > > > >> > > > >>> > > >> > >> > > >> > // type (Table
4.7).
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_stat_type"
> is
> > > an
> > > > > >> integer
> > > > > >> > > > >>> > specifying
> > > > > >> > > > >>> > > >> the
> > > > > >> > > > >>> > > >> > >> > > >> statistical
> > > > > >> > > > >>> > > >> > >> > > >> > // processing type
(Table
> > > > 4.10).
> > > > > >> > > > >>> > > >> > >> > > >> > // - The
> > "GRIB2_ipdtmpl_index"
> > > > and
> > > > > >> > > > >>> > > "GRIB2_ipdtmpl_val"
> > > > > >> > > > >>> > > >> > >> entries
> > > > > >> > > > >>> > > >> > >> > > are
> > > > > >> > > > >>> > > >> > >> > > >> > arrays
> > > > > >> > > > >>> > > >> > >> > > >> > // of integers
which
> > specify
> > > > the
> > > > > >> > product
> > > > > >> > > > >>> > > description
> > > > > >> > > > >>> > > >> > >> > template
> > > > > >> > > > >>> > > >> > >> > > >> > values to
> > > > > >> > > > >>> > > >> > >> > > >> > // be used. The
indices
> are
> > > > > >> 0-based.
> > > > > >> > For
> > > > > >> > > > >>> > example,
> > > > > >> > > > >>> > > >> use
> > > > > >> > > > >>> > > >> > the
> > > > > >> > > > >>> > > >> > >> > > >> > following to
> > > > > >> > > > >>> > > >> > >> > > >> > // request a GRIB2
record
> > > whose
> > > > > >> 9-th
> > > > > >> > and
> > > > > >> > > > >>> 27-th
> > > > > >> > > > >>> > > >> product
> > > > > >> > > > >>> > > >> > >> > > >> description
> > > > > >> > > > >>> > > >> > >> > > >> > // template values
are 1
> > and
> > > 2,
> > > > > >> > > > >>> respectively:
> > > > > >> > > > >>> > > >> > >> > > >> > //
> > GRIB2_ipdtmpl_index=[8,
> > > > > 26];
> > > > > >> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > > > >> > > > >>> > > >> > >> 2];
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > While I don't have any
ECMWF data
> > on
> > > > hand
> > > > > >> for
> > > > > >> > > > >>> testing, I
> > > > > >> > > > >>> > > do
> > > > > >> > > > >>> > > >> > have
> > > > > >> > > > >>> > > >> > >> a
> > > > > >> > > > >>> > > >> > >> > > file
> > > > > >> > > > >>> > > >> > >> > > >> > from the Air Force from
which
> > wgrib2
> > > > > >> prints a
> > > > > >> > > > >>> similar
> > > > > >> > > > >>> > > >> result:
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> >
549:687031550:d=2017061000:var
> > > > > discipline=0
> > > > > >> > > > >>> center=57
> > > > > >> > > > >>> > > >> > >> local_table=1
> > > > > >> > > > >>> > > >> > >> > > >> > parmcat=1
parm=192:surface:18
> hour
> > > > fcst:
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > So I used that for
testing. I ran
> > the
> > > > > >> > following
> > > > > >> > > > >>> > > >> plot_data_plane
> > > > > >> > > > >>> > > >> > >> > > command
> > > > > >> > > > >>> > > >> > >> > > >> > WITHOUT specifying a
requested
> > name:
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > plot_data_plane
> > > > > >> GALWEM_20170610_CY00_FH18.GR2
> > > > > >> > > > >>> plot.ps
> > > > > >> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > > > >> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57;
GRIB2_ltab=1;
> > > > > >> GRIB2_parm_cat=1;
> > > > > >> > > > >>> > > >> GRIB2_parm=192;
> > > > > >> > > > >>> > > >> > >> > > >> level="Z0";'
> > > > > >> > > > >>> > > >> > >> > > >> > -v 3
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > And then I looked closely
at the
> > log
> > > > > >> message
> > > > > >> > to
> > > > > >> > > > >>> confirm
> > > > > >> > > > >>> > > >> that it
> > > > > >> > > > >>> > > >> > >> used
> > > > > >> > > > >>> > > >> > >> > > >> > record 549, as expected:
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > DEBUG 3:
> > > MetGrib2DataFile::data_plane()
> > > > > ->
> > > > > >> > Found
> > > > > >> > > > >>> exact
> > > > > >> > > > >>> > > match
> > > > > >> > > > >>> > > >> > for
> > > > > >> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2 record
549
> > field
> > > 1
> > > > of
> > > > > >> > GRIB2
> > > > > >> > > > file
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > So MET used these GRIB2
numbers I
> > > > defined
> > > > > >> to
> > > > > >> > do
> > > > > >> > > a
> > > > > >> > > > >>> table
> > > > > >> > > > >>> > > >> lookup
> > > > > >> > > > >>> > > >> > >> and
> > > > > >> > > > >>> > > >> > >> > > find
> > > > > >> > > > >>> > > >> > >> > > >> a
> > > > > >> > > > >>> > > >> > >> > > >> > match in the MET GRIB
tables:
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > >
> > > > > >> > > >
> > > > >
> https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > And it determined that
that field
> > > > should
> > > > > be
> > > > > >> > > called
> > > > > >> > > > >>> > > "CTSTM".
> > > > > >> > > > >>> > > >> > >> > Hopefully,
> > > > > >> > > > >>> > > >> > >> > > >> the
> > > > > >> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are
already
> > > defined
> > > > > >> there
> > > > > >> > in
> > > > > >> > > > >>> MET. If
> > > > > >> > > > >>> > > >> not, we
> > > > > >> > > > >>> > > >> > >> can
> > > > > >> > > > >>> > > >> > >> > > >> always
> > > > > >> > > > >>> > > >> > >> > > >> > add them.
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > Please try running
> plot_data_plane
> > at
> > > > > >> > verbosity
> > > > > >> > > > >>> level 3
> > > > > >> > > > >>> > on
> > > > > >> > > > >>> > > >> one
> > > > > >> > > > >>> > > >> > of
> > > > > >> > > > >>> > > >> > >> > > these
> > > > > >> > > > >>> > > >> > >> > > >> > files first. And make sure
it
> runs
> > > > > without
> > > > > >> > error
> > > > > >> > > > >>> and the
> > > > > >> > > > >>> > > log
> > > > > >> > > > >>> > > >> > >> message
> > > > > >> > > > >>> > > >> > >> > > >> prints
> > > > > >> > > > >>> > > >> > >> > > >> > out the "name" it found
for that
> > > data.
> > > > If
> > > > > >> so,
> > > > > >> > > you
> > > > > >> > > > >>> can
> > > > > >> > > > >>> > just
> > > > > >> > > > >>> > > >> skip
> > > > > >> > > > >>> > > >> > >> > using
> > > > > >> > > > >>> > > >> > >> > > >> all
> > > > > >> > > > >>> > > >> > >> > > >> > those integers and revert
back to
> > > using
> > > > > the
> > > > > >> > name
> > > > > >> > > > MET
> > > > > >> > > > >>> > > knows.
> > > > > >> > > > >>> > > >> If
> > > > > >> > > > >>> > > >> > >> not,
> > > > > >> > > > >>> > > >> > >> > > >> then we
> > > > > >> > > > >>> > > >> > >> > > >> > may need to define a
custom GRIB
> > > table
> > > > > for
> > > > > >> > this
> > > > > >> > > > >>> ECMWF
> > > > > >> > > > >>> > > GRIB2
> > > > > >> > > > >>> > > >> > data.
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >> > Thanks,
> > > > > >> > > > >>> > > >> > >> > > >> > John
> > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> --
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> Binbin Zhou
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> 5830 University Research Ct.
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> College Park, MD 20740
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> 301-683-3683
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >> --
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > >> > > > >>> > > >> > >> > > >> Tara Jensen
> > > > > >> > > > >>> > > >> > >> > > >> Project Manager II
> > > > > >> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > > > >> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder,
Colorado
> 80307
> > > USA
> > > > > >> > > > >>> > > >> > >> > > >> +1 303-497-8479
> > > jensen at ucar.edu
> > > > > >> > > > >>> > > >> > >> > > >>
> > > > > >> > > > >>> > > >> > >> > > >
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> > >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > --
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > Binbin Zhou
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > 5830 University Research Ct.
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > College Park, MD 20740
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> > 301-683-3683
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >> >
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >>
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > --
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > Binbin Zhou
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > 5830 University Research Ct.
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > College Park, MD 20740
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> > > 301-683-3683
> > > > > >> > > > >>> > > >> > >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > --
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > Binbin Zhou
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > 5830 University Research Ct.
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > College Park, MD 20740
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> > 301-683-3683
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >> >
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >>
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > --
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > Binbin Zhou
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > 5830 University Research Ct.
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > College Park, MD 20740
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > Binbin.Zhou at noaa.gov
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > > > 301-683-3683
> > > > > >> > > > >>> > > >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > --
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > Binbin Zhou
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > 5830 University Research Ct.
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > College Park, MD 20740
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > Binbin.Zhou at noaa.gov
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > > 301-683-3683
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> > >
> > > > > >> > > > >>> >
> > > > > >> > > > >>> >
> > > > > >> > > > >>>
> > > > > >> > > > >>> --
> > > > > >> > > > >>>
> > > > > >> > > > >>> Binbin Zhou
> > > > > >> > > > >>>
> > > > > >> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > > > >> > > > >>>
> > > > > >> > > > >>> 5830 University Research Ct.
> > > > > >> > > > >>>
> > > > > >> > > > >>> College Park, MD 20740
> > > > > >> > > > >>>
> > > > > >> > > > >>> Binbin.Zhou at noaa.gov
> > > > > >> > > > >>>
> > > > > >> > > > >>> 301-683-3683
> > > > > >> > > > >>>
> > > > > >> > > > >>>
> > > > > >> > > >
> > > > > >> > > > --
> > > > > >> > > > Julie Prestopnik (she/her)
> > > > > >> > > > Software Engineer
> > > > > >> > > > National Center for Atmospheric Research
> > > > > >> > > > Research Applications Laboratory
> > > > > >> > > > Email: jpresto at ucar.edu
> > > > > >> > > >
> > > > > >> > > > My working day may not be your working day. Please
do not
> > > feel
> > > > > >> obliged
> > > > > >> > > to
> > > > > >> > > > reply to this email outside of your normal working
hours.
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >> > --
> > > > > >> > Julie Prestopnik (she/her)
> > > > > >> > Software Engineer
> > > > > >> > National Center for Atmospheric Research
> > > > > >> > Research Applications Laboratory
> > > > > >> > Email: jpresto at ucar.edu
> > > > > >> >
> > > > > >> > My working day may not be your working day. Please do
not
> feel
> > > > > obliged
> > > > > >> to
> > > > > >> > reply to this email outside of your normal working
hours.
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Binbin Zhou
> > > > > >
> > > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >
> > > > > > 5830 University Research Ct.
> > > > > >
> > > > > > College Park, MD 20740
> > > > > >
> > > > > > Binbin.Zhou at noaa.gov
> > > > > >
> > > > > > 301-683-3683
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Binbin Zhou
> > > > >
> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >
> > > > > 5830 University Research Ct.
> > > > >
> > > > > College Park, MD 20740
> > > > >
> > > > > Binbin.Zhou at noaa.gov
> > > > >
> > > > > 301-683-3683
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> > >
> >
> >
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: urgent question for reading ECMWF ensmeble data
From: Binbin.Zhou at noaa.gov
Time: Mon Dec 14 12:37:31 2020
John,
Thanks!
Binbin
On Mon, Dec 14, 2020 at 2:20 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> FYI, I just submitted a pull request to merge these changes in for
the
> met-10.0.0 release.
> https://github.com/dtcenter/MET/pull/1608
>
> And once Julie reviews those changes, we'll merge them for inclusion
in the
> met-10.0.0 release.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Fri, Dec 11, 2020 at 1:21 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> >
> > John,
> >
> > Including in MET-10.0.0 should be fine.
> >
> > Binbin
> >
> > On Fri, Dec 11, 2020 at 2:54 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binbin,
> > >
> > > Great, thanks for confirming. OK, next question is about
versions and
> > > timelines.
> > >
> > > Do we need to create a met-9.1.2 bugfix version that includes
this
> > change?
> > > Or is it sufficient to just include in the met-10.0.0 release?
> > >
> > > If we need to create a met-9.1.2 bugfix release, do you have a
deadline
> > for
> > > that?
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Dec 11, 2020 at 12:05 PM Binbin.Zhou at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
>
> > > >
> > > > John,
> > > >
> > > > This it works!
> > > >
> > > > Thanks!
> > > >
> > > > Binbin
> > > >
> > > > On Fri, Dec 11, 2020 at 12:47 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Binbin,
> > > > >
> > > > > Those * characters don't belong in the plot_data_plane
command.
> > That's
> > > > the
> > > > > way the mail system formats bold text. To be totally clear,
here
> are
> > > the
> > > > > exact commands I ran after logging on to Venus to test:
> > > > >
> > > > > cd
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > > > module use
> > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > module load met/ens_config_test
> > > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 ~/plot.ps
'name="HGT";
> > > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v 3
> > > > >
> > > > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match
> > for
> > > > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1
GRIB
> records
> > > > > matching VarInfo "HGT/P500" in GRIB file
> > "ecmanl.t12z.grid3.f00.grib1".
> > > > > DEBUG 1: Creating postscript file: /u/John.H.Gotway/plot.ps
> > > > >
> > > > > That matches a single record, namely the 3rd record in that
GRIB
> > file.
> > > Do
> > > > > you see different behavior? Perhaps you failed to load the
> > > > > "ens_config_test" version of the code before testing?
> > > > >
> > > > > John
> > > > >
> > > > > On Fri, Dec 11, 2020 at 10:20 AM Binbin.Zhou at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769 >
> > > > > >
> > > > > > John,
> > > > > >
> > > > > > As I tested following plotting
> > > > > >
> > > > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
> '*name="HGT";
> > > > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";*' -v
3
> > > > > >
> > > > > > the error message is
> > > > > >
> > > > > > ERROR :
> > > > > > ERROR : yyerror() -> syntax error in file
> "/tmp/met_config_9350_0"
> > > > > > ERROR :
> > > > > > ERROR : line = 1
> > > > > > ERROR :
> > > > > > ERROR : column = 1
> > > > > > ERROR :
> > > > > > ERROR : text = "*"
> > > > > > ERROR :
> > > > > > ERROR :
> > > > > > ERROR : *name="HGT"; level="P500"; lead_time="00";
> > > > > > GRIB_ens="low_res_ctl";*
> > > > > > ERROR :
> > > > > >
> ^__________________________________________________________________
> > > > > > ERROR :
> > > > > >
> > > > > > After I removed "*"
> > > > > > plot_data_plane ecmanl.t12z.grid3.f00.grib1 plot.ps
> 'name="HGT";
> > > > > > level="P500"; lead_time="00"; GRIB_ens="low_res_ctl";' -v
3
> > > > > > It shows 3 records matched
> > > > > >
> > > > > > DEBUG 1: Opening data file: ecmanl.t12z.grid3.f00.grib1
> > > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found
exact
> match
> > > for
> > > > > > VarInfo "HGT/P500" in GRIB record 1 of GRIB file
> > > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found
exact
> match
> > > for
> > > > > > VarInfo "HGT/P500" in GRIB record 2 of GRIB file
> > > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found
exact
> match
> > > for
> > > > > > VarInfo "HGT/P500" in GRIB record 3 of GRIB file
> > > > > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 3
GRIB
> > records
> > > > > > matching VarInfo "HGT/P500" in GRIB file
> > > "ecmanl.t12z.grid3.f00.grib1".
> > > > > > DEBUG 1: Creating postscript file: plot.ps
> > > > > >
> > > > > > My testing directory on Venus
> > > > > > is
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0_grib1
> > > > > > test script is ploting.sh
> > > > > >
> > > > > > Can you look at it?
> > > > > >
> > > > > > Thanks!
> > > > > > Binbin
> > > > > >
> > > > > >
> > > > > > On Thu, Dec 10, 2020 at 3:17 PM Binbin Zhou - NOAA
Affiliate <
> > > > > > binbin.zhou at noaa.gov> wrote:
> > > > > >
> > > > > > > John,
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Binbin
> > > > > > >
> > > > > > > On Thu, Dec 10, 2020 at 2:06 PM John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > >> Thanks Julie.
> > > > > > >>
> > > > > > >> And Binbin, here's an EnsembleStatConfig file where
climos are
> > > used:
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/main_v9.1/test/config/EnsembleStatConfig_climo
> > > > > > >>
> > > > > > >> Since the list of forecast fields is exactly the same
as the
> > climo
> > > > > > fields,
> > > > > > >> this config file just copies them over using:
> > > > > > >> climo_mean = fcst;
> > > > > > >>
> > > > > > >> In your case, when they differ, you'd just specify the
"field"
> > > array
> > > > > > >> inside
> > > > > > >> the "climo_mean" dictionary. For example:
> > > > > > >>
> > > > > > >> climo_mean = {
> > > > > > >>
> > > > > > >> field = [
> > > > > > >> { name = "TMP"; level = "Z2"; },
> > > > > > >> { name = "TMP"; level = "P850"; }
> > > > > > >> ];
> > > > > > >> ...
> > > > > > >> }
> > > > > > >>
> > > > > > >> On Thu, Dec 10, 2020 at 12:00 PM Julie Prestopnik via
RT <
> > > > > > >> met_help at ucar.edu>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> >
> > > > > > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > >
> > > > > > >> >
> > > > > > >> > Thanks, John. That worked great.
> > > > > > >> >
> > > > > > >> > Binbin, this version of MET is now accessible to you
by
> > running
> > > > the
> > > > > > >> > following on venus:
> > > > > > >> >
> > > > > > >> > module use
> > > > > >
/gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > > >> > module load met/ens_config_test
> > > > > > >> >
> > > > > > >> > Please let us know if you encounter any problems.
> > > > > > >> >
> > > > > > >> > Julie
> > > > > > >> >
> > > > > > >> > On Thu, Dec 10, 2020 at 10:05 AM John Halley Gotway
via RT <
> > > > > > >> > met_help at ucar.edu> wrote:
> > > > > > >> >
> > > > > > >> > >
> > > > > > >> > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > >
> > > > > > >> > >
> > > > > > >> > > Julie,
> > > > > > >> > >
> > > > > > >> > > Building the tarfile is very easy. I just ran this
script
> > from
> > > > the
> > > > > > MET
> > > > > > >> > repo
> > > > > > >> > > and specified the branch to be built:
> > > > > > >> > > MET/scripts/met_checkout_and_build.sh branch
> > > > > > >> feature_1604_ens_config
> > > > > > >> > >
> > > > > > >> > > And that created a tarfile which I posted to RAL's
> anonymous
> > > ftp
> > > > > > >> server:
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> ftp://ftp.rap.ucar.edu/incoming/irap/met_help/met-
feature_1604_ens_config.20201210.tar.gz
> > > > > > >> > >
> > > > > > >> > > I assume that'll work for you.
> > > > > > >> > >
> > > > > > >> > > Thanks,
> > > > > > >> > > John
> > > > > > >> > >
> > > > > > >> > > On Thu, Dec 10, 2020 at 9:25 AM Julie Prestopnik
via RT <
> > > > > > >> > met_help at ucar.edu
> > > > > > >> > > >
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > >
> > > > > > >> > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > >
> > > > > > >> > > >
> > > > > > >> > > > John,
> > > > > > >> > > >
> > > > > > >> > > > I am able to help with this install on WCOSS.
> > > > > > >> > > >
> > > > > > >> > > > Would it be easier to use git to checkout the
repo on
> > WCOSS?
> > > > Or
> > > > > > >> would
> > > > > > >> > it
> > > > > > >> > > be
> > > > > > >> > > > > easier if I built a tarfile for this version of
the
> > code?
> > > > > > >> > > > >
> > > > > > >> > > > The last time I tried to build from git on WCOSS
(a
> while
> > > > ago),
> > > > > I
> > > > > > >> > > > couldn't. I don't remember all of the details,
but I
> > think
> > > I
> > > > > > >> couldn't
> > > > > > >> > > run
> > > > > > >> > > > bootstrap and maybe it had to do with the version
of
> > > autoconf
> > > > > > being
> > > > > > >> too
> > > > > > >> > > > old? I'm happy to try it again if you'd like
though.
> > > > > Otherwise,
> > > > > > I
> > > > > > >> > know
> > > > > > >> > > a
> > > > > > >> > > > tarfile would work, but I know that is extra work
for
> you.
> > > > > > >> > > >
> > > > > > >> > > > Julie
> > > > > > >> > > >
> > > > > > >> > > > On Wed, Dec 9, 2020 at 5:18 PM John Halley Gotway
<
> > > > > > johnhg at ucar.edu>
> > > > > > >> > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Julie,
> > > > > > >> > > > >
> > > > > > >> > > > > I have created a bugfix branch for main_v9.1
named
> > > > > > >> > > > > "feature_1604_ens_config" and it's up on
GitHub.
> > > > > > >> > > > >
> > > > > > >> > > > > Only a single file has changed in it. But I
need to
> get
> > > this
> > > > > > >> version
> > > > > > >> > > > > compiled on WCOSS with which Binbin can test.
> > > > > > >> > > > >
> > > > > > >> > > > > Are you to help me with this? Would it be
easier to
> use
> > > git
> > > > to
> > > > > > >> > checkout
> > > > > > >> > > > > the repo on WCOSS? Or would it be easier if I
built a
> > > > tarfile
> > > > > > for
> > > > > > >> > this
> > > > > > >> > > > > version of the code?
> > > > > > >> > > > >
> > > > > > >> > > > > Thanks,
> > > > > > >> > > > > John
> > > > > > >> > > > >
> > > > > > >> > > > > On Wed, Dec 9, 2020 at 4:52 PM John Halley
Gotway <
> > > > > > >> johnhg at ucar.edu>
> > > > > > >> > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > >> Binbin,
> > > > > > >> > > > >>
> > > > > > >> > > > >> In order for you to test this change out, we'd
need
> to
> > > > > > recompile
> > > > > > >> a
> > > > > > >> > > > >> version of the code on WCOSS. I wrote this up
as a
> > GitHub
> > > > > > issue:
> > > > > > >> > > > >> https://github.com/dtcenter/MET/issues/1604
> > > > > > >> > > > >>
> > > > > > >> > > > >> And I'll create a branch from main_v9.1 named
> > > > > > >> > "bugfix_1604_ens_config"
> > > > > > >> > > > >> and commit the change there. Then we can
checkout
> that
> > > code
> > > > > on
> > > > > > >> > WCOSS,
> > > > > > >> > > > >> compile it, and you can test it.
> > > > > > >> > > > >>
> > > > > > >> > > > >> Yes, you're right, if the fcst fields differ
from the
> > > climo
> > > > > > >> fields,
> > > > > > >> > > > you'd
> > > > > > >> > > > >> need to specify them separately. That's pretty
easy
> in
> > > the
> > > > > > >> > > Ensemble-Stat
> > > > > > >> > > > >> config file... but are you asking how to
control that
> > via
> > > > > > >> METplus?
> > > > > > >> > > > >>
> > > > > > >> > > > >> John
> > > > > > >> > > > >>
> > > > > > >> > > > >> On Wed, Dec 9, 2020 at 2:29 PM
Binbin.Zhou at noaa.gov
> > via
> > > > RT <
> > > > > > >> > > > >> met_help at ucar.edu> wrote:
> > > > > > >> > > > >>
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > > >
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> John,
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> Great! Let me test it at first before make
change
> in
> > > > > MET9.1
> > > > > > >> > > > >>> Another question:
> > > > > > >> > > > >>> Since climatology data is still in NCEP's
grib2
> > > format,
> > > > > In
> > > > > > >> the
> > > > > > >> > > > >>> ensemble_stat
> > > > > > >> > > > >>> config file, climo entry settings should
be
> > different
> > > > > from
> > > > > > >> those
> > > > > > >> > > of
> > > > > > >> > > > >>> fcst, obs entries?
> > > > > > >> > > > >>> How to set climo (name, level, etc.)?
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> Binbin
> > > > > > >> > > > >>>
> > > > > > >> > > > >>>
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> On Wed, Dec 9, 2020 at 4:14 PM John Halley
Gotway
> via
> > > RT <
> > > > > > >> > > > >>> met_help at ucar.edu>
> > > > > > >> > > > >>> wrote:
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> > Binbin,
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > Success! When I recompile with that one
line
> > commented
> > > > > out,
> > > > > > >> > > > >>> plot_data_plane
> > > > > > >> > > > >>> > is able to filter down all those records to
the
> > single
> > > > one
> > > > > > >> that
> > > > > > >> > we
> > > > > > >> > > > >>> want.
> > > > > > >> > > > >>> > Note in the example below that I used the
> > "lead_time"
> > > > and
> > > > > > >> > > "GRIB_ens"
> > > > > > >> > > > >>> > options to filter the input records down.
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > plot_data_plane ensposte.t00z.z500hr
plot.ps
> > > > > '*name="HGT";
> > > > > > >> > > > >>> level="P500";
> > > > > > >> > > > >>> > lead_time="00"; GRIB_ens="low_res_ctl";*'
-v 3
> > > > > > >> > > > >>> > DEBUG 1: Opening data file:
ensposte.t00z.z500hr
> > > > > > >> > > > >>> > DEBUG 3:
MetGrib1DataFile::data_plane_array() ->
> > > *Found
> > > > > > exact
> > > > > > >> > match
> > > > > > >> > > > for
> > > > > > >> > > > >>> > VarInfo "HGT/P500" in GRIB record 26 of
GRIB file
> > > > > > >> > > > >>> "ensposte.t00z.z500hr".*
> > > > > > >> > > > >>> > DEBUG 3:
MetGrib1DataFile::data_plane_array() ->
> > > Found 1
> > > > > > GRIB
> > > > > > >> > > records
> > > > > > >> > > > >>> > matching VarInfo "HGT/P500" in GRIB file
> > > > > > >> "ensposte.t00z.z500hr".
> > > > > > >> > > > >>> > DEBUG 1: Creating postscript file: plot.ps
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > Also note that this selects out record
number 26.
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > Shall I make this change as a "bugfix" for
> > > > met-9.1...which
> > > > > > >> could
> > > > > > >> > be
> > > > > > >> > > > >>> > included in a met-9.1.2 bugfix release? Or
shall I
> > > just
> > > > > > >> include
> > > > > > >> > it
> > > > > > >> > > in
> > > > > > >> > > > >>> > development for the next release (planned
for
> March
> > > > 2021)?
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > Thanks,
> > > > > > >> > > > >>> > John
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > On Tue, Dec 8, 2020 at 5:27 PM
> Binbin.Zhou at noaa.gov
> > > via
> > > > > RT
> > > > > > <
> > > > > > >> > > > >>> > met_help at ucar.edu> wrote:
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > <URL:
> > > > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > > >> > >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > John,
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > If this removing impacts other
applications,
> or
> > > > takes
> > > > > > some
> > > > > > >> > time
> > > > > > >> > > > >>> > > to be effective, then won't try. I will
use
> > another
> > > > > > >> approach,
> > > > > > >> > > such
> > > > > > >> > > > >>> as
> > > > > > >> > > > >>> > > borrow GEFS's grib2 data structure as
tank to
> save
> > > EC
> > > > > > >> ensemble
> > > > > > >> > > > >>> > > data.
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > Binbin
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > On Tue, Dec 8, 2020 at 6:46 PM Binbin
Zhou -
> NOAA
> > > > > > Affiliate
> > > > > > >> <
> > > > > > >> > > > >>> > > binbin.zhou at noaa.gov> wrote:
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > > John,
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > Yes, please try to use this approach.
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > Thanks!
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > Binbin
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > Binbin
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > On Tue, Dec 8, 2020 at 5:43 PM John
Halley
> > Gotway
> > > > via
> > > > > > RT <
> > > > > > >> > > > >>> > > > met_help at ucar.edu> wrote:
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > >> Binbin,
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> I used the "-PDS10" option for wgrib
to dump
> > > those
> > > > > > >> records:
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> [John.H.Gotway at v71a1 ~]$ $WGRIB -PDS10
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0/ensposte.t00z.z500hr
> > > > > > >> > > > >>> > > >> | egrep "^11:|^26:|^43:"
> > > > > > >> > > > >>> > > >> 11:PDS10= 0 0 45 2 98 151 255 128 7
100 1 244
> > 20
> > > 12
> > > > > 6 0
> > > > > > >> 0 1
> > > > > > >> > 0
> > > > > > >> > > 0
> > > > > > >> > > > 1
> > > > > > >> > > > >>> 0 0
> > > > > > >> > > > >>> > 0
> > > > > > >> > > > >>> > > 21
> > > > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3 2 1
255
> > > > > > >> > > > >>> > > >> 26:PDS10= 0 0 45 2 98 151 255 128 7
100 1 244
> > 20
> > > 12
> > > > > 6 0
> > > > > > >> 0 1
> > > > > > >> > 0
> > > > > > >> > > 0
> > > > > > >> > > > 1
> > > > > > >> > > > >>> 0 0
> > > > > > >> > > > >>> > 0
> > > > > > >> > > > >>> > > 21
> > > > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 2 1
255
> > > > > > >> > > > >>> > > >> 43:PDS10= 0 0 45 2 98 151 255 128 7
100 1 244
> > 20
> > > 12
> > > > > 6 0
> > > > > > >> 0 1
> > > > > > >> > 0
> > > > > > >> > > 0
> > > > > > >> > > > 1
> > > > > > >> > > > >>> 0 0
> > > > > > >> > > > >>> > 0
> > > > > > >> > > > >>> > > 21
> > > > > > >> > > > >>> > > >> 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 1
255
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> Looking VERY closely, there is a
difference
> in
> > > the
> > > > > > 42-nd
> > > > > > >> > > column
> > > > > > >> > > > >>> which
> > > > > > >> > > > >>> > is
> > > > > > >> > > > >>> > > >> the 4-th from the last... records 11,
26, and
> > 43
> > > > have
> > > > > > >> values
> > > > > > >> > > of
> > > > > > >> > > > >>> 3, 1,
> > > > > > >> > > > >>> > > and
> > > > > > >> > > > >>> > > >> 2
> > > > > > >> > > > >>> > > >> respectively.
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> On this page (
> > > > > > >> > > > >>> > >
> > > > > >
https://www.nco.ncep.noaa.gov/pmb/docs/on388/appendixc.html
> > > > > > >> > > > >>> > > >> ),
> > > > > > >> > > > >>> > > >> byte 42 = 1 means "unperturbed control
> > forecast".
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> So we *should* be able to use the
"GRIB_ens"
> > > config
> > > > > > >> option
> > > > > > >> > to
> > > > > > >> > > > >>> > > >> differentiate
> > > > > > >> > > > >>> > > >> between those records. BUT when I try,
it
> > doesn't
> > > > > work.
> > > > > > >> And
> > > > > > >> > > > that's
> > > > > > >> > > > >>> > > because
> > > > > > >> > > > >>> > > >> of this line:
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/17e1c9740224bf84a8bd0e84b5d254ccc5981bb8/met/src/libcode/vx_data2d_grib/data2d_grib_utils.cc#L89
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> We only apply those filtering options
to GRIB
> > > files
> > > > > > >> created
> > > > > > >> > by
> > > > > > >> > > > >>> NCEP.
> > > > > > >> > > > >>> > If
> > > > > > >> > > > >>> > > >> you'd like, I could try removing that
> condition
> > > to
> > > > > see
> > > > > > >> if we
> > > > > > >> > > can
> > > > > > >> > > > >>> pick
> > > > > > >> > > > >>> > > out
> > > > > > >> > > > >>> > > >> the single record of interest.
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> John
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> On Tue, Dec 8, 2020 at 2:51 PM
> > > > Binbin.Zhou at noaa.gov
> > > > > > via
> > > > > > >> RT
> > > > > > >> > <
> > > > > > >> > > > >>> > > >> met_help at ucar.edu> wrote:
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > <URL:
> > > > > > >> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > > >> > > > >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > John,
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > Look at my
> > > > > > >> > > > >>> > > >> > directory
> > > > > > >> > > > >>>
> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/ecme1p0
> > > > > > >> > > > >>> > > >> > The grib1 file is
ensposte.t00z.z500hr,
> this
> > is
> > > > > only
> > > > > > >> for
> > > > > > >> > 500
> > > > > > >> > > > >>> HGT.
> > > > > > >> > > > >>> > > >> > wgrib ensposte.t00z.z500hr|grep
anl|grep
> > > > > > >> anl:type=analysis
> > > > > > >> > > > >>> > > >> > I get 3 records:
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
11:896950:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
26:2242375:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
43:3767190:d=20120600:HGT:kpds5=7:kpds6=100:kpds7=500:TR=1:P1=0:P2=0:TimeU=1:500
> > > > > > >> > > > >>> > > >> > mb:anl:type=analysis:NAve=0
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > From its grib2 file, we know only
> record#26
> > is
> > > > > > >> analysis
> > > > > > >> > > > low-res
> > > > > > >> > > > >>> > > >> control.
> > > > > > >> > > > >>> > > >> > If you have some way to determine
which
> > record
> > > is
> > > > > > >> analysis
> > > > > > >> > > > >>> low-res
> > > > > > >> > > > >>> > > >> control
> > > > > > >> > > > >>> > > >> > from the grib1 file,
> > > > > > >> > > > >>> > > >> > then we can use grib1 directly
(don't
> > consider
> > > > the
> > > > > > size
> > > > > > >> > > > issue).
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > Binbin
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > On Tue, Dec 8, 2020 at 4:36 PM
Binbin Zhou
> -
> > > NOAA
> > > > > > >> > Affiliate
> > > > > > >> > > <
> > > > > > >> > > > >>> > > >> > binbin.zhou at noaa.gov> wrote:
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > > John,
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > Grib1 data is too large, another
issue
> is
> > > the
> > > > > > >> low-res
> > > > > > >> > > > >>> control
> > > > > > >> > > > >>> > > >> analysis
> > > > > > >> > > > >>> > > >> > > data can not be retrieved
> > > > > > >> > > > >>> > > >> > > from the grib1 ecmwf data files.
If using
> > > > wgrib,
> > > > > we
> > > > > > >> can
> > > > > > >> > > see
> > > > > > >> > > > 3
> > > > > > >> > > > >>> > > analysis
> > > > > > >> > > > >>> > > >> > > fields, but don't know
> > > > > > >> > > > >>> > > >> > > which is low-res control. After
convert
> to
> > > > grib2,
> > > > > > >> wgrib2
> > > > > > >> > > can
> > > > > > >> > > > >>> show
> > > > > > >> > > > >>> > > >> which
> > > > > > >> > > > >>> > > >> > > record is low-res control.
> > > > > > >> > > > >>> > > >> > > Tomorrow I'll show you the ecmwf
grib1
> data
> > > set
> > > > > on
> > > > > > >> > WCOSS.
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > Binbin
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > On Tue, Dec 8, 2020 at 3:57 PM
John
> Halley
> > > > Gotway
> > > > > > via
> > > > > > >> > RT <
> > > > > > >> > > > >>> > > >> > > met_help at ucar.edu> wrote:
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > >> Binbin,
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >> Sounds like there's a lot of
moving
> parts
> > > > there.
> > > > > > How
> > > > > > >> > > would
> > > > > > >> > > > >>> you
> > > > > > >> > > > >>> > like
> > > > > > >> > > > >>> > > >> to
> > > > > > >> > > > >>> > > >> > >> proceed?
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >> Should we check to see if MET can
read
> > those
> > > > > ECMWF
> > > > > > >> > GRIB1
> > > > > > >> > > > >>> files
> > > > > > >> > > > >>> > > >> directly
> > > > > > >> > > > >>> > > >> > >> without needing to reformat?
> > > > > > >> > > > >>> > > >> > >> Or do you just want to create a
custom
> > GRIB
> > > > > table
> > > > > > >> for
> > > > > > >> > the
> > > > > > >> > > > >>> result
> > > > > > >> > > > >>> > of
> > > > > > >> > > > >>> > > >> your
> > > > > > >> > > > >>> > > >> > >> pre-processing and conversion
steps?
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >> John
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >> On Mon, Dec 7, 2020 at 2:49 PM
> > > > > > Binbin.Zhou at noaa.gov
> > > > > > >> > via
> > > > > > >> > > > RT <
> > > > > > >> > > > >>> > > >> > >> met_help at ucar.edu> wrote:
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > <URL:
> > > > > > >> > > > >>>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > John,
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > Our ECMWF grib2 ensemble data
files
> > are
> > > > > > >> converted
> > > > > > >> > > from
> > > > > > >> > > > >>> its
> > > > > > >> > > > >>> > > grib1
> > > > > > >> > > > >>> > > >> > >> format
> > > > > > >> > > > >>> > > >> > >> > data.
> > > > > > >> > > > >>> > > >> > >> > Before conversion, the
parameter names
> > are
> > > > > > there,
> > > > > > >> but
> > > > > > >> > > > after
> > > > > > >> > > > >>> > > >> > conversion,
> > > > > > >> > > > >>> > > >> > >> the
> > > > > > >> > > > >>> > > >> > >> > names
> > > > > > >> > > > >>> > > >> > >> > are gone. It is because of
unknown
> > tables.
> > > > The
> > > > > > >> grib1
> > > > > > >> > > data
> > > > > > >> > > > >>> is
> > > > > > >> > > > >>> > > still
> > > > > > >> > > > >>> > > >> > >> > original. The original
> > > > > > >> > > > >>> > > >> > >> > grib1 data files are very
strange data
> > > > format
> > > > > we
> > > > > > >> can
> > > > > > >> > > not
> > > > > > >> > > > >>> read.
> > > > > > >> > > > >>> > So
> > > > > > >> > > > >>> > > >> we
> > > > > > >> > > > >>> > > >> > >> have
> > > > > > >> > > > >>> > > >> > >> > to use
> > > > > > >> > > > >>> > > >> > >> > a Frotran code to re-arrange
the
> fields
> > > and
> > > > > save
> > > > > > >> them
> > > > > > >> > > as
> > > > > > >> > > > >>> > > NCEP-like
> > > > > > >> > > > >>> > > >> > grib1
> > > > > > >> > > > >>> > > >> > >> > data files.
> > > > > > >> > > > >>> > > >> > >> > So after several processes,
some
> > important
> > > > > info
> > > > > > >> like
> > > > > > >> > > > tables
> > > > > > >> > > > >>> > could
> > > > > > >> > > > >>> > > >> be
> > > > > > >> > > > >>> > > >> > >> lost.
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > I'll test your first
suggestion
> since
> > > the
> > > > > > field
> > > > > > >> > > record
> > > > > > >> > > > >>> order
> > > > > > >> > > > >>> > > is
> > > > > > >> > > > >>> > > >> > >> fixed.
> > > > > > >> > > > >>> > > >> > >> > Currently we still
> > > > > > >> > > > >>> > > >> > >> > obtained grib1 data from ECWMF,
> > > eventually,
> > > > we
> > > > > > >> will
> > > > > > >> > get
> > > > > > >> > > > >>> grib2
> > > > > > >> > > > >>> > > data
> > > > > > >> > > > >>> > > >> > >> > directly. So before
> > > > > > >> > > > >>> > > >> > >> > we can get original grib2 files
from
> > > ECMWF,
> > > > we
> > > > > > >> just
> > > > > > >> > use
> > > > > > >> > > > >>> this
> > > > > > >> > > > >>> > > >> temporal
> > > > > > >> > > > >>> > > >> > >> > solution.
> > > > > > >> > > > >>> > > >> > >> > '
> > > > > > >> > > > >>> > > >> > >> > Thanks!
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > On Fri, Dec 4, 2020 at 5:48 PM
John
> > Halley
> > > > > > Gotway
> > > > > > >> via
> > > > > > >> > > RT
> > > > > > >> > > > <
> > > > > > >> > > > >>> > > >> > >> > met_help at ucar.edu>
> > > > > > >> > > > >>> > > >> > >> > wrote:
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > > Binbin,
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > OK, working on Venus.
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > You have a typo in the
command you
> > sent.
> > > > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > > > >> > a.ps
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> >
> > > > > >
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > > > >> > > > >>> > > >> > >> > -v
> > > > > > >> > > > >>> > > >> > >> > > 3
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > You have "RIB2_parm_cat" that
should
> > be
> > > > > > >> > > > "GRIB2_parm_cat".
> > > > > > >> > > > >>> > After
> > > > > > >> > > > >>> > > >> > >> > correcting
> > > > > > >> > > > >>> > > >> > >> > > that I did still get an error
but
> > it's a
> > > > > very
> > > > > > >> > useful
> > > > > > >> > > > one!
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > module use
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > >
> > /gpfs/dell2/emc/verification/noscrub/emc.metplus/modulefiles
> > > > > > >> > > > >>> > > >> > >> > > module load met/9.1
> > > > > > >> > > > >>> > > >> > >> > > module load grib_util/1.1.1
> > > > > > >> > > > >>> > > >> > >> > > cd
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/g2g/com/verf/dev/test_ecmanl/grib2
> > > > > > >> > > > >>> > > >> > >> > > $WGRIB2
> ensposte.t00z.z500.grd3.grib2
> > > > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > > > >> ~/
> > > > > > >> > > > plot.ps
> > > > > > >> > > > >>> > > >> > >> > 'GRIB2_disc=255;
> > > > > > >> > > > >>> > > >> > >> > > GRIB2_mtab=2;
GRIB2_parm_cat=3;
> > > > > GRIB2_parm=5;
> > > > > > >> > > > >>> level="P500";'
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > *ERROR : ERROR :
> > > > VarInfoGrib2::set_dict()
> > > > > ->
> > > > > > >> no
> > > > > > >> > > > >>> parameter
> > > > > > >> > > > >>> > > found
> > > > > > >> > > > >>> > > >> > with
> > > > > > >> > > > >>> > > >> > >> > > matching GRIB2_disc (255)
> > GRIB2_parm_cat
> > > > (3)
> > > > > > >> > > GRIB2_parm
> > > > > > >> > > > >>> (5).
> > > > > > >> > > > >>> > > Use
> > > > > > >> > > > >>> > > >> the
> > > > > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable
> > to
> > > > > define
> > > > > > >> > custom
> > > > > > >> > > > >>> GRIB
> > > > > > >> > > > >>> > > >> > >> tables.ERROR
> > > > > > >> > > > >>> > > >> > >> > :
> > > > > > >> > > > >>> > > >> > >> > > *
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > So this table is not yet
defined in
> > > MET. I
> > > > > > >> checked
> > > > > > >> > > > their
> > > > > > >> > > > >>> > > website:
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://apps.ecmwf.int/codes/grib/param-
db?&discipline=255&category=3&filter=grib2
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > But I don't see an entry
defined for
> > > > > > >> GRIB2_parm=5;
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > 3 things...
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > (1) As a crude temporary
solution,
> you
> > > > could
> > > > > > >> refer
> > > > > > >> > to
> > > > > > >> > > > it
> > > > > > >> > > > >>> by
> > > > > > >> > > > >>> > > it's
> > > > > > >> > > > >>> > > >> > >> record
> > > > > > >> > > > >>> > > >> > >> > > number. Let's assume you know
that
> > > record
> > > > 1
> > > > > > has
> > > > > > >> > > > >>> temperature
> > > > > > >> > > > >>> > > data:
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > plot_data_plane
> > > > > ecme.ens01.t00z.grd3.f12.grib2
> > > > > > >> ~/
> > > > > > >> > > > plot.ps
> > > > > > >> > > > >>> > > >> > 'name="TMP";
> > > > > > >> > > > >>> > > >> > >> > > level="R1";
set_attr_level="P500";'
> > -v 5
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > Here I'm reading record 1,
naming it
> > TMP
> > > > and
> > > > > > the
> > > > > > >> > > using
> > > > > > >> > > > >>> > > >> > set_attr_level
> > > > > > >> > > > >>> > > >> > >> > write
> > > > > > >> > > > >>> > > >> > >> > > the output level as P500
instead of
> > R1.
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > (2) You could create a custom
GRIB
> > table
> > > > > > >> yourself
> > > > > > >> > > > >>> locally,
> > > > > > >> > > > >>> > and
> > > > > > >> > > > >>> > > >> then
> > > > > > >> > > > >>> > > >> > >> set
> > > > > > >> > > > >>> > > >> > >> > the
> > > > > > >> > > > >>> > > >> > >> > > MET_GRIB_TABLES environment
variable
> > to
> > > > > point
> > > > > > >> MET
> > > > > > >> > to
> > > > > > >> > > > it.
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > (3) The best solution is
contacting
> > > ECMWF
> > > > > and
> > > > > > >> > > > requesting
> > > > > > >> > > > >>> they
> > > > > > >> > > > >>> > > >> send
> > > > > > >> > > > >>> > > >> > >> over
> > > > > > >> > > > >>> > > >> > >> > the
> > > > > > >> > > > >>> > > >> > >> > > GRIB2 table we should be
using to
> > decode
> > > > > this
> > > > > > >> data.
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > Thanks,
> > > > > > >> > > > >>> > > >> > >> > > John
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > On Fri, Dec 4, 2020 at 3:30
PM John
> > > Halley
> > > > > > >> Gotway <
> > > > > > >> > > > >>> > > >> johnhg at ucar.edu>
> > > > > > >> > > > >>> > > >> > >> > wrote:
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > > > We might already have them.
I'm
> not
> > > > sure.
> > > > > > >> > > > >>> > > >> > >> > > >
> > > > > > >> > > > >>> > > >> > >> > > > But there's an additional
required
> > > piece
> > > > > of
> > > > > > >> the
> > > > > > >> > > > puzzle.
> > > > > > >> > > > >>> > > Binbin
> > > > > > >> > > > >>> > > >> > runs
> > > > > > >> > > > >>> > > >> > >> > > > "wgrib2" to figure out what
"name"
> > he
> > > > > should
> > > > > > >> use
> > > > > > >> > in
> > > > > > >> > > > >>> the MET
> > > > > > >> > > > >>> > > >> config
> > > > > > >> > > > >>> > > >> > >> > files.
> > > > > > >> > > > >>> > > >> > >> > > > If wgrib2 doesn't tell him
the
> right
> > > > > "name",
> > > > > > >> the
> > > > > > >> > > > >>> problem
> > > > > > >> > > > >>> > will
> > > > > > >> > > > >>> > > >> > >> remain.
> > > > > > >> > > > >>> > > >> > >> > > > wgrib2 can be configured to
define
> > > > > > additional
> > > > > > >> > > tables
> > > > > > >> > > > as
> > > > > > >> > > > >>> > well,
> > > > > > >> > > > >>> > > >> but
> > > > > > >> > > > >>> > > >> > >> for
> > > > > > >> > > > >>> > > >> > >> > > this
> > > > > > >> > > > >>> > > >> > >> > > > to be easy for Binbin,
that'd mean
> > > > > updating
> > > > > > >> the
> > > > > > >> > > > script
> > > > > > >> > > > >>> > that's
> > > > > > >> > > > >>> > > >> run
> > > > > > >> > > > >>> > > >> > >> for
> > > > > > >> > > > >>> > > >> > >> > > > "module load grib_utils" on
WCOSS.
> > And
> > > > I'm
> > > > > > >> > guessing
> > > > > > >> > > > >>> that's
> > > > > > >> > > > >>> > > not
> > > > > > >> > > > >>> > > >> > very
> > > > > > >> > > > >>> > > >> > >> > easy.
> > > > > > >> > > > >>> > > >> > >> > > >
> > > > > > >> > > > >>> > > >> > >> > > > Yes, I do agree that the
best
> > solution
> > > > is
> > > > > > >> getting
> > > > > > >> > > the
> > > > > > >> > > > >>> ECMWF
> > > > > > >> > > > >>> > > >> GRIB
> > > > > > >> > > > >>> > > >> > >> tables
> > > > > > >> > > > >>> > > >> > >> > > > and adding them for both
wgrib2
> and
> > > MET.
> > > > > > >> > > > >>> > > >> > >> > > >
> > > > > > >> > > > >>> > > >> > >> > > > John
> > > > > > >> > > > >>> > > >> > >> > > >
> > > > > > >> > > > >>> > > >> > >> > > > On Fri, Dec 4, 2020 at 2:32
PM
> Tara
> > > > > Jensen <
> > > > > > >> > > > >>> > jensen at ucar.edu>
> > > > > > >> > > > >>> > > >> > wrote:
> > > > > > >> > > > >>> > > >> > >> > > >
> > > > > > >> > > > >>> > > >> > >> > > >> John,
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> It seems to me that it
would be
> > > > > worthwhile
> > > > > > to
> > > > > > >> > just
> > > > > > >> > > > ask
> > > > > > >> > > > >>> > ECMWF
> > > > > > >> > > > >>> > > >> for
> > > > > > >> > > > >>> > > >> > >> their
> > > > > > >> > > > >>> > > >> > >> > > >> Grib2 Table for their
Ensemble
> > files.
> > > > > And
> > > > > > >> other
> > > > > > >> > > > >>> > operational
> > > > > > >> > > > >>> > > >> > >> centers
> > > > > > >> > > > >>> > > >> > >> > for
> > > > > > >> > > > >>> > > >> > >> > > >> that matter. Do you agree
this
> > would
> > > > be
> > > > > a
> > > > > > >> good
> > > > > > >> > > > >>> solution?
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> Thanks, Tara
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> ---------- Forwarded
message
> > > ---------
> > > > > > >> > > > >>> > > >> > >> > > >> From: Binbin.Zhou at noaa.gov
via
> RT
> > <
> > > > > > >> > > > met_help at ucar.edu>
> > > > > > >> > > > >>> > > >> > >> > > >> Date: Fri, Dec 4, 2020 at
11:06
> AM
> > > > > > >> > > > >>> > > >> > >> > > >> Subject: Re:
[rt.rap.ucar.edu
> > > #97769]
> > > > > > urgent
> > > > > > >> > > > >>> question for
> > > > > > >> > > > >>> > > >> > reading
> > > > > > >> > > > >>> > > >> > >> > ECMWF
> > > > > > >> > > > >>> > > >> > >> > > >> ensmeble data
> > > > > > >> > > > >>> > > >> > >> > > >> To:
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> <URL:
> > > > > > >> > > > >>> > >
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97769
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> John,
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> I tested using following
> command:
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > > > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > > > > >> > > > a.ps
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>>
> > > 'GRIB2_mtab=2;GRIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > > > >> > > > >>> > > >> -v3
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > > > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict()
> > ->
> > > > > either
> > > > > > >> name
> > > > > > >> > > or
> > > > > > >> > > > >>> > > >> GRIB2_disc,
> > > > > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and
GRIB2_parm
> must
> > be
> > > > > > >> specified
> > > > > > >> > in
> > > > > > >> > > > >>> field
> > > > > > >> > > > >>> > > >> > >> information
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> plot_data_plane
> > > > > > >> ecme.ens01.t00z.grd3.f12.grib2
> > > > > > >> > > > a.ps
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> >
> > > > > >
> > > >
> >
'GRIB2_mtab=2;GRIB2_disc=255;RIB2_parm_cat=3;GRIB2_parm=5;level="P500";'
> > > > > > >> > > > >>> > > >> > >> > > >> -v
> > > > > > >> > > > >>> > > >> > >> > > >> 3
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> Still got error
> > > > > > >> > > > >>> > > >> > >> > > >> ERROR :
> > > > > > >> > > > >>> > > >> > >> > > >> ERROR :
VarInfoGrib2::set_dict()
> > ->
> > > > > either
> > > > > > >> name
> > > > > > >> > > or
> > > > > > >> > > > >>> > > >> GRIB2_disc,
> > > > > > >> > > > >>> > > >> > >> > > >> GRIB2_parm_cat and
GRIB2_parm
> must
> > be
> > > > > > >> specified
> > > > > > >> > in
> > > > > > >> > > > >>> field
> > > > > > >> > > > >>> > > >> > >> information
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> Looks like discipline=255
is not
> > > > > recognized
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> Binbin
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> On Fri, Dec 4, 2020 at
11:57 AM
> > John
> > > > > Halley
> > > > > > >> > Gotway
> > > > > > >> > > > >>> via RT
> > > > > > >> > > > >>> > <
> > > > > > >> > > > >>> > > >> > >> > > >> met_help at ucar.edu>
> > > > > > >> > > > >>> > > >> > >> > > >> wrote:
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> > Binbin,
> > > > > > >> > > > >>> > > >> > >> > > >> > You can do something
similar
> with
> > > > MET.
> > > > > > >> Please
> > > > > > >> > go
> > > > > > >> > > > to
> > > > > > >> > > > >>> this
> > > > > > >> > > > >>> > > >> link:
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > >
> > > https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > And then search for
> "GRIB2_mtab",
> > > and
> > > > > > >> you'll
> > > > > > >> > > find
> > > > > > >> > > > >>> the
> > > > > > >> > > > >>> > > >> content
> > > > > > >> > > > >>> > > >> > >> > > >> > copied/pasted below. You
can
> > > specify
> > > > > > these
> > > > > > >> > > INSTEAD
> > > > > > >> > > > >>> OF
> > > > > > >> > > > >>> > > >> > specifying
> > > > > > >> > > > >>> > > >> > >> a
> > > > > > >> > > > >>> > > >> > >> > > >> > requested name.
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_mtab" is
> an
> > > > > integer
> > > > > > >> > > > >>> specifying the
> > > > > > >> > > > >>> > > >> master
> > > > > > >> > > > >>> > > >> > >> > table
> > > > > > >> > > > >>> > > >> > >> > > >> > number.
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ltab" is
> an
> > > > > integer
> > > > > > >> > > > >>> specifying the
> > > > > > >> > > > >>> > > >> local
> > > > > > >> > > > >>> > > >> > >> table
> > > > > > >> > > > >>> > > >> > >> > > >> > number.
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_disc" is
> an
> > > > > integer
> > > > > > >> > > > >>> specifying the
> > > > > > >> > > > >>> > > >> GRIB2
> > > > > > >> > > > >>> > > >> > >> > > >> discipline
> > > > > > >> > > > >>> > > >> > >> > > >> > code.
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_parm_cat"
> > is
> > > an
> > > > > > >> integer
> > > > > > >> > > > >>> specifying
> > > > > > >> > > > >>> > > the
> > > > > > >> > > > >>> > > >> > >> > parameter
> > > > > > >> > > > >>> > > >> > >> > > >> > category
> > > > > > >> > > > >>> > > >> > >> > > >> > // code.
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_parm" is
> an
> > > > > integer
> > > > > > >> > > > >>> specifying the
> > > > > > >> > > > >>> > > >> > >> parameter
> > > > > > >> > > > >>> > > >> > >> > > code.
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_pdt" is
> an
> > > > > integer
> > > > > > >> > > > specifying
> > > > > > >> > > > >>> the
> > > > > > >> > > > >>> > > >> product
> > > > > > >> > > > >>> > > >> > >> > > >> definition
> > > > > > >> > > > >>> > > >> > >> > > >> > // template
(Table
> 4.0).
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_process"
> is
> > > an
> > > > > > >> integer
> > > > > > >> > > > >>> specifying
> > > > > > >> > > > >>> > > the
> > > > > > >> > > > >>> > > >> > >> > generating
> > > > > > >> > > > >>> > > >> > >> > > >> > process
> > > > > > >> > > > >>> > > >> > >> > > >> > // (Table 4.3).
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_cntr" is
> an
> > > > > integer
> > > > > > >> > > > >>> specifying the
> > > > > > >> > > > >>> > > >> > >> originating
> > > > > > >> > > > >>> > > >> > >> > > >> > center.
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_ens_type"
> > is
> > > an
> > > > > > >> integer
> > > > > > >> > > > >>> specifying
> > > > > > >> > > > >>> > > the
> > > > > > >> > > > >>> > > >> > >> > ensemble
> > > > > > >> > > > >>> > > >> > >> > > >> type
> > > > > > >> > > > >>> > > >> > >> > > >> > // (Table 4.6).
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
"GRIB2_der_type"
> > is
> > > an
> > > > > > >> integer
> > > > > > >> > > > >>> specifying
> > > > > > >> > > > >>> > > the
> > > > > > >> > > > >>> > > >> > >> derived
> > > > > > >> > > > >>> > > >> > >> > > >> > product
> > > > > > >> > > > >>> > > >> > >> > > >> > // type (Table
4.7).
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
> "GRIB2_stat_type"
> > is
> > > > an
> > > > > > >> integer
> > > > > > >> > > > >>> > specifying
> > > > > > >> > > > >>> > > >> the
> > > > > > >> > > > >>> > > >> > >> > > >> statistical
> > > > > > >> > > > >>> > > >> > >> > > >> > // processing
type
> (Table
> > > > > 4.10).
> > > > > > >> > > > >>> > > >> > >> > > >> > // - The
> > > "GRIB2_ipdtmpl_index"
> > > > > and
> > > > > > >> > > > >>> > > "GRIB2_ipdtmpl_val"
> > > > > > >> > > > >>> > > >> > >> entries
> > > > > > >> > > > >>> > > >> > >> > > are
> > > > > > >> > > > >>> > > >> > >> > > >> > arrays
> > > > > > >> > > > >>> > > >> > >> > > >> > // of integers
which
> > > specify
> > > > > the
> > > > > > >> > product
> > > > > > >> > > > >>> > > description
> > > > > > >> > > > >>> > > >> > >> > template
> > > > > > >> > > > >>> > > >> > >> > > >> > values to
> > > > > > >> > > > >>> > > >> > >> > > >> > // be used. The
indices
> > are
> > > > > > >> 0-based.
> > > > > > >> > For
> > > > > > >> > > > >>> > example,
> > > > > > >> > > > >>> > > >> use
> > > > > > >> > > > >>> > > >> > the
> > > > > > >> > > > >>> > > >> > >> > > >> > following to
> > > > > > >> > > > >>> > > >> > >> > > >> > // request a
GRIB2
> record
> > > > whose
> > > > > > >> 9-th
> > > > > > >> > and
> > > > > > >> > > > >>> 27-th
> > > > > > >> > > > >>> > > >> product
> > > > > > >> > > > >>> > > >> > >> > > >> description
> > > > > > >> > > > >>> > > >> > >> > > >> > // template
values are
> 1
> > > and
> > > > 2,
> > > > > > >> > > > >>> respectively:
> > > > > > >> > > > >>> > > >> > >> > > >> > //
> > > GRIB2_ipdtmpl_index=[8,
> > > > > > 26];
> > > > > > >> > > > >>> > > >> > GRIB2_ipdtmpl_val=[1,
> > > > > > >> > > > >>> > > >> > >> 2];
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > While I don't have any
ECMWF
> data
> > > on
> > > > > hand
> > > > > > >> for
> > > > > > >> > > > >>> testing, I
> > > > > > >> > > > >>> > > do
> > > > > > >> > > > >>> > > >> > have
> > > > > > >> > > > >>> > > >> > >> a
> > > > > > >> > > > >>> > > >> > >> > > file
> > > > > > >> > > > >>> > > >> > >> > > >> > from the Air Force from
which
> > > wgrib2
> > > > > > >> prints a
> > > > > > >> > > > >>> similar
> > > > > > >> > > > >>> > > >> result:
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> >
549:687031550:d=2017061000:var
> > > > > > discipline=0
> > > > > > >> > > > >>> center=57
> > > > > > >> > > > >>> > > >> > >> local_table=1
> > > > > > >> > > > >>> > > >> > >> > > >> > parmcat=1
parm=192:surface:18
> > hour
> > > > > fcst:
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > So I used that for
testing. I
> ran
> > > the
> > > > > > >> > following
> > > > > > >> > > > >>> > > >> plot_data_plane
> > > > > > >> > > > >>> > > >> > >> > > command
> > > > > > >> > > > >>> > > >> > >> > > >> > WITHOUT specifying a
requested
> > > name:
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > plot_data_plane
> > > > > > >> GALWEM_20170610_CY00_FH18.GR2
> > > > > > >> > > > >>> plot.ps
> > > > > > >> > > > >>> > > >> > >> > 'GRIB2_disc=0;
> > > > > > >> > > > >>> > > >> > >> > > >> > GRIB2_cntr=57;
GRIB2_ltab=1;
> > > > > > >> GRIB2_parm_cat=1;
> > > > > > >> > > > >>> > > >> GRIB2_parm=192;
> > > > > > >> > > > >>> > > >> > >> > > >> level="Z0";'
> > > > > > >> > > > >>> > > >> > >> > > >> > -v 3
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > And then I looked
closely at
> the
> > > log
> > > > > > >> message
> > > > > > >> > to
> > > > > > >> > > > >>> confirm
> > > > > > >> > > > >>> > > >> that it
> > > > > > >> > > > >>> > > >> > >> used
> > > > > > >> > > > >>> > > >> > >> > > >> > record 549, as expected:
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > DEBUG 3:
> > > > MetGrib2DataFile::data_plane()
> > > > > > ->
> > > > > > >> > Found
> > > > > > >> > > > >>> exact
> > > > > > >> > > > >>> > > match
> > > > > > >> > > > >>> > > >> > for
> > > > > > >> > > > >>> > > >> > >> > > >> > 'CTSTM/Z0' in GRIB2
record 549
> > > field
> > > > 1
> > > > > of
> > > > > > >> > GRIB2
> > > > > > >> > > > file
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
'/Volumes/d1/projects/MET/MET_unit_test/MET_test_input/model_data/grib2/galwem/GALWEM_20170610_CY00_FH18.GR2'
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > So MET used these GRIB2
> numbers I
> > > > > defined
> > > > > > >> to
> > > > > > >> > do
> > > > > > >> > > a
> > > > > > >> > > > >>> table
> > > > > > >> > > > >>> > > >> lookup
> > > > > > >> > > > >>> > > >> > >> and
> > > > > > >> > > > >>> > > >> > >> > > find
> > > > > > >> > > > >>> > > >> > >> > > >> a
> > > > > > >> > > > >>> > > >> > >> > > >> > match in the MET GRIB
tables:
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > >
> > > > > > >> > > >
> > > > > >
> >
https://github.com/dtcenter/MET/tree/main_v9.1/met/data/table_files
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > And it determined that
that
> field
> > > > > should
> > > > > > be
> > > > > > >> > > called
> > > > > > >> > > > >>> > > "CTSTM".
> > > > > > >> > > > >>> > > >> > >> > Hopefully,
> > > > > > >> > > > >>> > > >> > >> > > >> the
> > > > > > >> > > > >>> > > >> > >> > > >> > ECMWF GRIB2 tables are
already
> > > > defined
> > > > > > >> there
> > > > > > >> > in
> > > > > > >> > > > >>> MET. If
> > > > > > >> > > > >>> > > >> not, we
> > > > > > >> > > > >>> > > >> > >> can
> > > > > > >> > > > >>> > > >> > >> > > >> always
> > > > > > >> > > > >>> > > >> > >> > > >> > add them.
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > Please try running
> > plot_data_plane
> > > at
> > > > > > >> > verbosity
> > > > > > >> > > > >>> level 3
> > > > > > >> > > > >>> > on
> > > > > > >> > > > >>> > > >> one
> > > > > > >> > > > >>> > > >> > of
> > > > > > >> > > > >>> > > >> > >> > > these
> > > > > > >> > > > >>> > > >> > >> > > >> > files first. And make
sure it
> > runs
> > > > > > without
> > > > > > >> > error
> > > > > > >> > > > >>> and the
> > > > > > >> > > > >>> > > log
> > > > > > >> > > > >>> > > >> > >> message
> > > > > > >> > > > >>> > > >> > >> > > >> prints
> > > > > > >> > > > >>> > > >> > >> > > >> > out the "name" it found
for
> that
> > > > data.
> > > > > If
> > > > > > >> so,
> > > > > > >> > > you
> > > > > > >> > > > >>> can
> > > > > > >> > > > >>> > just
> > > > > > >> > > > >>> > > >> skip
> > > > > > >> > > > >>> > > >> > >> > using
> > > > > > >> > > > >>> > > >> > >> > > >> all
> > > > > > >> > > > >>> > > >> > >> > > >> > those integers and
revert back
> to
> > > > using
> > > > > > the
> > > > > > >> > name
> > > > > > >> > > > MET
> > > > > > >> > > > >>> > > knows.
> > > > > > >> > > > >>> > > >> If
> > > > > > >> > > > >>> > > >> > >> not,
> > > > > > >> > > > >>> > > >> > >> > > >> then we
> > > > > > >> > > > >>> > > >> > >> > > >> > may need to define a
custom
> GRIB
> > > > table
> > > > > > for
> > > > > > >> > this
> > > > > > >> > > > >>> ECMWF
> > > > > > >> > > > >>> > > GRIB2
> > > > > > >> > > > >>> > > >> > data.
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >> > Thanks,
> > > > > > >> > > > >>> > > >> > >> > > >> > John
> > > > > > >> > > > >>> > > >> > >> > > >> >
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> --
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> Binbin Zhou
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> 5830 University Research
Ct.
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> College Park, MD 20740
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> 301-683-3683
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >> --
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > >> > > > >>> > > >> > >> > > >> Tara Jensen
> > > > > > >> > > > >>> > > >> > >> > > >> Project Manager II
> > > > > > >> > > > >>> > > >> > >> > > >> NCAR RAL and DTC
> > > > > > >> > > > >>> > > >> > >> > > >> PO Box 3000, Boulder,
Colorado
> > 80307
> > > > USA
> > > > > > >> > > > >>> > > >> > >> > > >> +1 303-497-8479
> > > > jensen at ucar.edu
> > > > > > >> > > > >>> > > >> > >> > > >>
> > > > > > >> > > > >>> > > >> > >> > > >
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> > >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > --
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > Binbin Zhou
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > 5830 University Research Ct.
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > College Park, MD 20740
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> > 301-683-3683
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >> >
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >>
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > --
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > Binbin Zhou
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > 5830 University Research Ct.
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > College Park, MD 20740
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> > > 301-683-3683
> > > > > > >> > > > >>> > > >> > >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > --
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > Binbin Zhou
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > 5830 University Research Ct.
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > College Park, MD 20740
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> > 301-683-3683
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >> >
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >>
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > --
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > Binbin Zhou
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > 5830 University Research Ct.
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > College Park, MD 20740
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > > > 301-683-3683
> > > > > > >> > > > >>> > > >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > --
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > Binbin Zhou
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > 5830 University Research Ct.
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > College Park, MD 20740
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > > 301-683-3683
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> > >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>> >
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> --
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> Binbin Zhou
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> 5830 University Research Ct.
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> College Park, MD 20740
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> Binbin.Zhou at noaa.gov
> > > > > > >> > > > >>>
> > > > > > >> > > > >>> 301-683-3683
> > > > > > >> > > > >>>
> > > > > > >> > > > >>>
> > > > > > >> > > >
> > > > > > >> > > > --
> > > > > > >> > > > Julie Prestopnik (she/her)
> > > > > > >> > > > Software Engineer
> > > > > > >> > > > National Center for Atmospheric Research
> > > > > > >> > > > Research Applications Laboratory
> > > > > > >> > > > Email: jpresto at ucar.edu
> > > > > > >> > > >
> > > > > > >> > > > My working day may not be your working day.
Please do
> not
> > > > feel
> > > > > > >> obliged
> > > > > > >> > > to
> > > > > > >> > > > reply to this email outside of your normal
working
> hours.
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Julie Prestopnik (she/her)
> > > > > > >> > Software Engineer
> > > > > > >> > National Center for Atmospheric Research
> > > > > > >> > Research Applications Laboratory
> > > > > > >> > Email: jpresto at ucar.edu
> > > > > > >> >
> > > > > > >> > My working day may not be your working day. Please
do not
> > feel
> > > > > > obliged
> > > > > > >> to
> > > > > > >> > reply to this email outside of your normal working
hours.
> > > > > > >> >
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Binbin Zhou
> > > > > > >
> > > > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > > >
> > > > > > > 5830 University Research Ct.
> > > > > > >
> > > > > > > College Park, MD 20740
> > > > > > >
> > > > > > > Binbin.Zhou at noaa.gov
> > > > > > >
> > > > > > > 301-683-3683
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Binbin Zhou
> > > > > >
> > > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > > >
> > > > > > 5830 University Research Ct.
> > > > > >
> > > > > > College Park, MD 20740
> > > > > >
> > > > > > Binbin.Zhou at noaa.gov
> > > > > >
> > > > > > 301-683-3683
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > > >
> > >
> > >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
More information about the Met_help
mailing list