[Met_help] [rt.rap.ucar.edu #93371] History for How to match the time between obs and model record for grid_stat
John Halley Gotway via RT
met_help at ucar.edu
Fri Jan 17 14:35:47 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
To Whom It May Concern:
I have files of MODIS observed column dust concentration that is one hour
per file, ${OBSV_FILE}. I have HYSPLIT column dust concentration that is
48 fcst hour per file, ${FCST_FILE} .
When I do
grid_stat ${FCST_FILE} ${OBSV_FILE} ${CONFIG_FILE} -outdir . -v 2
How do I write configure file to match the records between
${FCST_FILE} ${OBSV_FILE} for statistic?
Note
(1) Both are on the same grid
(2) When NESDIS encode the ${OBS_FILE}, there is no grib2 table that
defined column dust concentration, so it randomly pick a grib2 table for
output. In this case, it is using variable DEN, and it has value of
concentration in kg/cm^3.
(3) FCST_FILE also has the same issue, so it is using variable LIPMF and it
is the value of the power of 10. (in ug/cm^3)
The unfinished configure file I am working for HYSPLIT column dust
verification can be found in
/gpfs/dell2/emc/modeling/noscrub/Ho-Chun.Huang/MET_verif/EMC_verif-aq-g2g/parm/verf_g2g_config_hysplitdust
The observation file (20191204, ##Z) of and model output fcst file (by date
in dustcs.yyyymmdd) can be found in
/gpfs/dell2/emc/modeling/noscrub/Ho-Chun.Huang/methelp
For example, MYDdust.aod_conc.v6.3.4.20191204.hr21.grib should match
dustcs.20191204/dustcs.t12z.pbl.1hr.grib2 fcst_hour 9
dustcs.20191204/dustcs.t06z.pbl.1hr.grib2 fcst_hour 15
dustcs.20191203/dustcs.t12z.pbl.1hr.grib2 fcst_hour 33
dustcs.20191203/dustcs.t06z.pbl.1hr.grib2 fcst_hour 39
where we perform verification upto 48 fcst hours.
What should be included in my configuration,
(1) using observation time as reference
(2) The line be added into fcst field, for proper time match?
fcst = {
wind_thresh = [ NA ];
field = [
{
name = "LIPMF";
convert(LIPMF) = 10**LIPMF
level = [ "L0" ];
cat_thresh = [ >=0.5, >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5,
>=4.0, >=4.5, >=5.0 ];
}
];
};
(3) What else needs to be added in my configuration file.
Thank you very much for your help.
Ho-Chun Huang
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct., Rm. 2792
College Park, MD 20740
Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
301-683-3958
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: How to match the time between obs and model record for grid_stat
From: Minna Win
Time: Thu Dec 05 14:50:31 2019
Hi Ho-Chun,
It looks like you are having issues setting up the grid stat config
files
for your data. I'm assigning this ticket to John Halley Gotway,
please
allow a few days for a response.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Dec 5, 2019 at 2:12 PM Ho-Chun Huang - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Thu Dec 05 14:11:44 2019: Request 93371 was acted upon.
> Transaction: Ticket created by ho-chun.huang at noaa.gov
> Queue: met_help
> Subject: How to match the time between obs and model record for
> grid_stat
> Owner: Nobody
> Requestors: ho-chun.huang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93371 >
>
>
> To Whom It May Concern:
>
> I have files of MODIS observed column dust concentration that is one
hour
> per file, ${OBSV_FILE}. I have HYSPLIT column dust concentration
that is
> 48 fcst hour per file, ${FCST_FILE} .
>
> When I do
> grid_stat ${FCST_FILE} ${OBSV_FILE} ${CONFIG_FILE} -outdir . -v 2
>
> How do I write configure file to match the records between
> ${FCST_FILE} ${OBSV_FILE} for statistic?
>
> Note
> (1) Both are on the same grid
> (2) When NESDIS encode the ${OBS_FILE}, there is no grib2 table
that
> defined column dust concentration, so it randomly pick a grib2
table for
> output. In this case, it is using variable DEN, and it has value of
> concentration in kg/cm^3.
> (3) FCST_FILE also has the same issue, so it is using variable LIPMF
and it
> is the value of the power of 10. (in ug/cm^3)
>
> The unfinished configure file I am working for HYSPLIT column dust
> verification can be found in
>
> /gpfs/dell2/emc/modeling/noscrub/Ho-Chun.Huang/MET_verif/EMC_verif-
aq-g2g/parm/verf_g2g_config_hysplitdust
>
> The observation file (20191204, ##Z) of and model output fcst file
(by date
> in dustcs.yyyymmdd) can be found in
> /gpfs/dell2/emc/modeling/noscrub/Ho-Chun.Huang/methelp
>
> For example, MYDdust.aod_conc.v6.3.4.20191204.hr21.grib should
match
> dustcs.20191204/dustcs.t12z.pbl.1hr.grib2 fcst_hour 9
> dustcs.20191204/dustcs.t06z.pbl.1hr.grib2 fcst_hour 15
> dustcs.20191203/dustcs.t12z.pbl.1hr.grib2 fcst_hour 33
> dustcs.20191203/dustcs.t06z.pbl.1hr.grib2 fcst_hour 39
>
> where we perform verification upto 48 fcst hours.
>
> What should be included in my configuration,
> (1) using observation time as reference
> (2) The line be added into fcst field, for proper time match?
>
> fcst = {
> wind_thresh = [ NA ];
>
> field = [
> {
> name = "LIPMF";
> convert(LIPMF) = 10**LIPMF
> level = [ "L0" ];
> cat_thresh = [ >=0.5, >=1.0, >=1.5, >=2.0, >=2.5, >=3.0,
>=3.5,
> >=4.0, >=4.5, >=5.0 ];
> }
> ];
>
> };
>
> (3) What else needs to be added in my configuration file.
>
> Thank you very much for your help.
>
> Ho-Chun Huang
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct., Rm. 2792
>
> College Park, MD 20740
>
> Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
>
> 301-683-3958
>
>
------------------------------------------------
Subject: How to match the time between obs and model record for grid_stat
From: John Halley Gotway
Time: Wed Dec 11 17:30:58 2019
Ho Chun,
I apologize for the long delay in getting back to you on this issue.
I've
been focused on development tasks for the next software release.
I logged on to luna and took a look at your data. I see that your
observation file contains data for a single time while the forecast
files
contain output for all 48 lead times.
Yes, we can setup the Grid-Stat configuration file to handle this
situation. When you run Grid-Stat, you define a single input gridded
forecast file and observation file. Often the output for each lead
time is
written to a different file, but in your case, they're all grouped
together.
So we can run Grid-Stat many times for each forecast file, but using
the
"lead_time" config option to specify which record we actually want.
Here's
an example of extracting and plotting forecast hour 39 from one of
these
files:
*module load grib_util/1.0.5*
*module use
/gpfs/hps3/emc/global/noscrub/Julie.Prestopnik/modulefilesmodule load
met/8.1*
*plot_data_plane \*
*/gpfs/dell2/emc/modeling/noscrub/Ho-
Chun.Huang/methelp/dustcs.20191203/dustcs.t06z.pbl.1hr.grib2
\*
*LIPMF_f39.ps 'name="LIPMF"; level="Z0-5000"; lead_time="39";' -v 3*
DEBUG 1: Opening data file:
/gpfs/dell2/emc/modeling/noscrub/Ho-
Chun.Huang/methelp/dustcs.20191203/dustcs.t06z.pbl.1hr.grib2
DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
'LIPMF/Z5000-0' in GRIB2 record 39 field 1 of GRIB2 file
'/gpfs/dell2/emc/modeling/noscrub/Ho-
Chun.Huang/methelp/dustcs.20191203/dustcs.t06z.pbl.1hr.grib2'
DEBUG 1: Creating postscript file: LIPMF_f39.ps
Please try running this command to confirm that it works for you.
So in your Grid-Stat config file, we should setup the fcst dictionary
like
this...
Note the changes...
(1) In convert function, use '^' for exponentiation instead of '**'
(2) You forgot to terminate the convert function with a semicolon.
(3) Specify the level as "Z0-5000"
(4) Specify lead_time = "${CUR_LEAD_HOUR}" where that environment
variable
is set to the current lead time.
(5) Make similar changes in the "obs" dictionary, but you don't need
to
specify "lead_time" since it only has one record.
If you find it logically easier, instead of specifying "lead_time =
"HH";"
you could instead specify "valid_time = "YYYYMMDD_HHMMSS";". That
would
logically have the same effect of pulling out the correct record from
your
forecast file.
fcst = {
wind_thresh = [ NA ];
field = [
{
name = "LIPMF";
convert(LIPMF) = 10^LIPMF;
level = [ "Z0-5000" ];
lead_time = "${CUR_LEAD_HOUR}";
cat_thresh = [ >=0.5, >=1.0, >=1.5, >=2.0, >=2.5, >=3.0,
>=3.5,
>=4.0, >=4.5, >=5.0 ];
}
];
};
Now are you doing this yourself manually by writing your own scripts?
Or
are you doing it using METplus?
If the latter, then I'll get George McCabe involved to advise on how
to
setup the METplus configuration file to handle this logic.
Thanks,
John Halley Gotway
On Thu, Dec 5, 2019 at 2:51 PM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93371 >
>
> Hi Ho-Chun,
>
> It looks like you are having issues setting up the grid stat config
files
> for your data. I'm assigning this ticket to John Halley Gotway,
please
> allow a few days for a response.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Thu, Dec 5, 2019 at 2:12 PM Ho-Chun Huang - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > Thu Dec 05 14:11:44 2019: Request 93371 was acted upon.
> > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > Queue: met_help
> > Subject: How to match the time between obs and model record
for
> > grid_stat
> > Owner: Nobody
> > Requestors: ho-chun.huang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93371 >
> >
> >
> > To Whom It May Concern:
> >
> > I have files of MODIS observed column dust concentration that is
one hour
> > per file, ${OBSV_FILE}. I have HYSPLIT column dust concentration
that is
> > 48 fcst hour per file, ${FCST_FILE} .
> >
> > When I do
> > grid_stat ${FCST_FILE} ${OBSV_FILE} ${CONFIG_FILE} -outdir . -v 2
> >
> > How do I write configure file to match the records between
> > ${FCST_FILE} ${OBSV_FILE} for statistic?
> >
> > Note
> > (1) Both are on the same grid
> > (2) When NESDIS encode the ${OBS_FILE}, there is no grib2 table
that
> > defined column dust concentration, so it randomly pick a grib2
table for
> > output. In this case, it is using variable DEN, and it has value
of
> > concentration in kg/cm^3.
> > (3) FCST_FILE also has the same issue, so it is using variable
LIPMF and
> it
> > is the value of the power of 10. (in ug/cm^3)
> >
> > The unfinished configure file I am working for HYSPLIT column dust
> > verification can be found in
> >
> >
> /gpfs/dell2/emc/modeling/noscrub/Ho-Chun.Huang/MET_verif/EMC_verif-
aq-g2g/parm/verf_g2g_config_hysplitdust
> >
> > The observation file (20191204, ##Z) of and model output fcst file
(by
> date
> > in dustcs.yyyymmdd) can be found in
> > /gpfs/dell2/emc/modeling/noscrub/Ho-Chun.Huang/methelp
> >
> > For example, MYDdust.aod_conc.v6.3.4.20191204.hr21.grib should
match
> > dustcs.20191204/dustcs.t12z.pbl.1hr.grib2 fcst_hour 9
> > dustcs.20191204/dustcs.t06z.pbl.1hr.grib2 fcst_hour 15
> > dustcs.20191203/dustcs.t12z.pbl.1hr.grib2 fcst_hour 33
> > dustcs.20191203/dustcs.t06z.pbl.1hr.grib2 fcst_hour 39
> >
> > where we perform verification upto 48 fcst hours.
> >
> > What should be included in my configuration,
> > (1) using observation time as reference
> > (2) The line be added into fcst field, for proper time match?
> >
> > fcst = {
> > wind_thresh = [ NA ];
> >
> > field = [
> > {
> > name = "LIPMF";
> > convert(LIPMF) = 10**LIPMF
> > level = [ "L0" ];
> > cat_thresh = [ >=0.5, >=1.0, >=1.5, >=2.0, >=2.5, >=3.0,
>=3.5,
> > >=4.0, >=4.5, >=5.0 ];
> > }
> > ];
> >
> > };
> >
> > (3) What else needs to be added in my configuration file.
> >
> > Thank you very much for your help.
> >
> > Ho-Chun Huang
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct., Rm. 2792
> >
> > College Park, MD 20740
> >
> > Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
> >
> > 301-683-3958
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list