[Met_help] [rt.rap.ucar.edu #93333] History for met-series stat
John Halley Gotway via RT
met_help at ucar.edu
Fri Jan 17 14:27:25 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am running met series-stat using version 8 to verify two nearly identical
model forecasts but getting very different stats when looking into Bias etc
fields. I have used identical configuration file for both of them. Attached
figure is total aerosol optical depth from two test runs verifying against
same analysis dataset. I need help whether I need to change any settings in
the configuration file, as I am not expecting that huge difference between
these two sets of runs.
.sh and Seriesstat config file are in wcoss tide at :
/global/noscrub/Partha.Bhattacharjee/MET
Let me know if you can access them.
Thanks
Partha
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: met-series stat
From: Julie Prestopnik
Time: Wed Dec 04 12:20:29 2019
Hi Partha.
I have assigned this ticket to John. Please allow a few days for a
response. Thanks!
Julie
On Wed, Dec 4, 2019 at 11:59 AM Partha Bhattacharjee - NOAA Affiliate
via
RT <met_help at ucar.edu> wrote:
>
> Wed Dec 04 11:58:37 2019: Request 93333 was acted upon.
> Transaction: Ticket created by partha.bhattacharjee at noaa.gov
> Queue: met_help
> Subject: met-series stat
> Owner: Nobody
> Requestors: partha.bhattacharjee at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
>
>
> Hello,
> I am running met series-stat using version 8 to verify two nearly
identical
> model forecasts but getting very different stats when looking into
Bias etc
> fields. I have used identical configuration file for both of them.
Attached
> figure is total aerosol optical depth from two test runs verifying
against
> same analysis dataset. I need help whether I need to change any
settings in
> the configuration file, as I am not expecting that huge difference
between
> these two sets of runs.
>
> .sh and Seriesstat config file are in wcoss tide at :
> /global/noscrub/Partha.Bhattacharjee/MET
>
> Let me know if you can access them.
>
> Thanks
> Partha
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: met-series stat
From: Partha Bhattacharjee - NOAA Affiliate
Time: Wed Dec 04 12:22:36 2019
Sure, no problem.
Thanks
Partha
On Wed, Dec 4, 2019 at 2:20 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi Partha.
>
> I have assigned this ticket to John. Please allow a few days for a
> response. Thanks!
>
> Julie
>
> On Wed, Dec 4, 2019 at 11:59 AM Partha Bhattacharjee - NOAA
Affiliate via
> RT <met_help at ucar.edu> wrote:
>
> >
> > Wed Dec 04 11:58:37 2019: Request 93333 was acted upon.
> > Transaction: Ticket created by partha.bhattacharjee at noaa.gov
> > Queue: met_help
> > Subject: met-series stat
> > Owner: Nobody
> > Requestors: partha.bhattacharjee at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
> >
> >
> > Hello,
> > I am running met series-stat using version 8 to verify two nearly
> identical
> > model forecasts but getting very different stats when looking into
Bias
> etc
> > fields. I have used identical configuration file for both of them.
> Attached
> > figure is total aerosol optical depth from two test runs
verifying
> against
> > same analysis dataset. I need help whether I need to change any
settings
> in
> > the configuration file, as I am not expecting that huge difference
> between
> > these two sets of runs.
> >
> > .sh and Seriesstat config file are in wcoss tide at :
> > /global/noscrub/Partha.Bhattacharjee/MET
> >
> > Let me know if you can access them.
> >
> > Thanks
> > Partha
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: met-series stat
From: John Halley Gotway
Time: Wed Dec 04 12:55:44 2019
Hi Partha,
I see you have questions about the Series-Analysis tool in MET.
I'm looking at your config file on tide
in
/global/noscrub/Partha.Bhattacharjee/MET/SeriesAnlConfig_July19_Test12
Here are some things I notice...
(1) In the "fcst" section, you do NOT need to list level = [ "A0",
"A0",
... "A0" ]; multiple times. Since you're reading the same data from
each
input file, you only need to list it once:
fcst = { field = [ { name = "AOTK"; level = [ "A0" ]; } ]; }
(2) The same is true in the "obs" section. You only need to list
"(0,*,*)"
once.
The fcst.field and obs.field settings are technically an array. If
you're
reading the same data from each of the input files (as you are), that
array
only needs to be length one. If you're reading different data from
each
input file, then those arrays must match the number of input files.
If this is running really slow, you could consider removing KT_CORR
and
SP_CORR output from the cnt line type and setting rank_corr_flag =
FALSE.
That'd make it run faster.
I do notice in your run script (series_stat_test2_201907.sh) that
there's a
1.5 hour offset between the timestamps of the forecast and observation
data. So series-analysis is probably printing a bunch of warning
messages
about that.
I don't see any obvious problems in your config file, other than the
suggestions above. You say you were surprised to see such large
differences. You could create and look at a time-series of plots for
the
raw fields. Perhaps there's some discrepancy in the data you aren't
expecting?
For example, you could pick a reasonable min/max value for the raw
data.
Loop over the files in the input directory and run MET's
plot_data_plane
tool with the -plot_range command line option to keep the min/max
plotting
value fixed. Once that's done you could convert the PostScript files
to an
animated gif:
convert -density 288 -geometry 50% -rotate 90 -delay 100 -background
white
-dispose Background *.ps anim.gif
Then do the same thing for the obs data. Look at those animated gifs
side-by-side to see if there's anything surprising in the data that
you
weren't expecting.
John
On Wed, Dec 4, 2019 at 12:23 PM Partha Bhattacharjee - NOAA Affiliate
via
RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
>
> Sure, no problem.
> Thanks
> Partha
>
> On Wed, Dec 4, 2019 at 2:20 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Partha.
> >
> > I have assigned this ticket to John. Please allow a few days for
a
> > response. Thanks!
> >
> > Julie
> >
> > On Wed, Dec 4, 2019 at 11:59 AM Partha Bhattacharjee - NOAA
Affiliate via
> > RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Wed Dec 04 11:58:37 2019: Request 93333 was acted upon.
> > > Transaction: Ticket created by partha.bhattacharjee at noaa.gov
> > > Queue: met_help
> > > Subject: met-series stat
> > > Owner: Nobody
> > > Requestors: partha.bhattacharjee at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333
> >
> > >
> > >
> > > Hello,
> > > I am running met series-stat using version 8 to verify two
nearly
> > identical
> > > model forecasts but getting very different stats when looking
into Bias
> > etc
> > > fields. I have used identical configuration file for both of
them.
> > Attached
> > > figure is total aerosol optical depth from two test runs
verifying
> > against
> > > same analysis dataset. I need help whether I need to change any
> settings
> > in
> > > the configuration file, as I am not expecting that huge
difference
> > between
> > > these two sets of runs.
> > >
> > > .sh and Seriesstat config file are in wcoss tide at :
> > > /global/noscrub/Partha.Bhattacharjee/MET
> > >
> > > Let me know if you can access them.
> > >
> > > Thanks
> > > Partha
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
------------------------------------------------
Subject: met-series stat
From: Partha Bhattacharjee - NOAA Affiliate
Time: Wed Dec 04 16:58:30 2019
Thanks John for looking into it and your suggestions. I will look
closely
into test2 data set the way you suggested. For GRIB2_process=62000 I
am
having this problem with test2 dataset, for other grib2 records it is
working fine and close to what I am getting from test1 and
irrespective of
any obs data I use I am still seeing this issue. Anyway, I will let
you
know my findings.
Partha
On Wed, Dec 4, 2019 at 2:55 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Partha,
>
> I see you have questions about the Series-Analysis tool in MET.
>
> I'm looking at your config file on tide
> in
/global/noscrub/Partha.Bhattacharjee/MET/SeriesAnlConfig_July19_Test12
>
> Here are some things I notice...
> (1) In the "fcst" section, you do NOT need to list level = [ "A0",
"A0",
> ... "A0" ]; multiple times. Since you're reading the same data from
each
> input file, you only need to list it once:
> fcst = { field = [ { name = "AOTK"; level = [ "A0" ]; } ]; }
> (2) The same is true in the "obs" section. You only need to list
"(0,*,*)"
> once.
>
> The fcst.field and obs.field settings are technically an array. If
you're
> reading the same data from each of the input files (as you are),
that array
> only needs to be length one. If you're reading different data from
each
> input file, then those arrays must match the number of input files.
>
> If this is running really slow, you could consider removing KT_CORR
and
> SP_CORR output from the cnt line type and setting rank_corr_flag =
FALSE.
> That'd make it run faster.
>
> I do notice in your run script (series_stat_test2_201907.sh) that
there's a
> 1.5 hour offset between the timestamps of the forecast and
observation
> data. So series-analysis is probably printing a bunch of warning
messages
> about that.
>
> I don't see any obvious problems in your config file, other than the
> suggestions above. You say you were surprised to see such large
> differences. You could create and look at a time-series of plots
for the
> raw fields. Perhaps there's some discrepancy in the data you aren't
> expecting?
>
> For example, you could pick a reasonable min/max value for the raw
data.
> Loop over the files in the input directory and run MET's
plot_data_plane
> tool with the -plot_range command line option to keep the min/max
plotting
> value fixed. Once that's done you could convert the PostScript
files to an
> animated gif:
>
> convert -density 288 -geometry 50% -rotate 90 -delay 100 -background
white
> -dispose Background *.ps anim.gif
>
> Then do the same thing for the obs data. Look at those animated
gifs
> side-by-side to see if there's anything surprising in the data that
you
> weren't expecting.
>
> John
>
>
> On Wed, Dec 4, 2019 at 12:23 PM Partha Bhattacharjee - NOAA
Affiliate via
> RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
> >
> > Sure, no problem.
> > Thanks
> > Partha
> >
> > On Wed, Dec 4, 2019 at 2:20 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Hi Partha.
> > >
> > > I have assigned this ticket to John. Please allow a few days
for a
> > > response. Thanks!
> > >
> > > Julie
> > >
> > > On Wed, Dec 4, 2019 at 11:59 AM Partha Bhattacharjee - NOAA
Affiliate
> via
> > > RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Wed Dec 04 11:58:37 2019: Request 93333 was acted upon.
> > > > Transaction: Ticket created by partha.bhattacharjee at noaa.gov
> > > > Queue: met_help
> > > > Subject: met-series stat
> > > > Owner: Nobody
> > > > Requestors: partha.bhattacharjee at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333
> > >
> > > >
> > > >
> > > > Hello,
> > > > I am running met series-stat using version 8 to verify two
nearly
> > > identical
> > > > model forecasts but getting very different stats when looking
into
> Bias
> > > etc
> > > > fields. I have used identical configuration file for both of
them.
> > > Attached
> > > > figure is total aerosol optical depth from two test runs
verifying
> > > against
> > > > same analysis dataset. I need help whether I need to change
any
> > settings
> > > in
> > > > the configuration file, as I am not expecting that huge
difference
> > > between
> > > > these two sets of runs.
> > > >
> > > > .sh and Seriesstat config file are in wcoss tide at :
> > > > /global/noscrub/Partha.Bhattacharjee/MET
> > > >
> > > > Let me know if you can access them.
> > > >
> > > > Thanks
> > > > Partha
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: met-series stat
From: Partha Bhattacharjee - NOAA Affiliate
Time: Fri Dec 06 17:04:57 2019
John,
I am looking into Grib record and trying to match whether MET Config
file
picking up exactly the same.
In the config file I sent you earlier you can see I am trying to read
grib record name = "AOTK"; GRIB2_process=62000. level A0 (total AOD at
550nm to match against obs), from all the input grib2 files. So, for
each
day it looks into 0, 6, 12 and 18 hour forecast for the entire month.
When
I look into the log file, I found MET is picking up :
MetGrib2DataFile::data_plane() - Found exact match for 'AOTK/A0' in
GRIB2
record 205 field 1 of GRIB2 file './20190705/gfs.t00z.pgrb2.0p25.f000'
DEBUG 3: MetGrib2DataFile::data_plane() - Found exact match for
'AOTK/A0'
in GRIB2 record 177 field 1 of GRIB2 file
'./20190705/gfs.t00z.pgrb2.0p25.f006'
DEBUG 3: MetGrib2DataFile::data_plane() - Found exact match for
'AOTK/A0'
in GRIB2 record 177 field 1 of GRIB2 file
'./20190705/gfs.t00z.pgrb2.0p25.f012'
but wgrib on the file shows
205:127608157:d=2019070500:AOTK:entire atmosphere:anl:chemical=Total
Aerosol:aerosol_size <2e-05:aerosol_wavelength 3.38e-07-3.42e-07
it is total AOD but at different wavelength (needs to read 209
instead).
209:131963245:d=2019070500:AOTK:entire atmosphere:anl:chemical=Total
Aerosol:aerosol_size <2e-05:aerosol_wavelength 5.45e-07-5.65e-07
Similarly, for other forecast hours, it should read record 182, not
177.
It seems to be GRIB2_process=62000 is not enough as some other
wavelength may have the same variable. Should I just put
'name="var209"
instead ?=in the config ?
Thanks
Partha
On Wed, Dec 4, 2019 at 2:55 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Partha,
>
> I see you have questions about the Series-Analysis tool in MET.
>
> I'm looking at your config file on tide
> in
/global/noscrub/Partha.Bhattacharjee/MET/SeriesAnlConfig_July19_Test12
>
> Here are some things I notice...
> (1) In the "fcst" section, you do NOT need to list level = [ "A0",
"A0",
> ... "A0" ]; multiple times. Since you're reading the same data from
each
> input file, you only need to list it once:
> fcst = { field = [ { name = "AOTK"; level = [ "A0" ]; } ]; }
> (2) The same is true in the "obs" section. You only need to list
"(0,*,*)"
> once.
>
> The fcst.field and obs.field settings are technically an array. If
you're
> reading the same data from each of the input files (as you are),
that array
> only needs to be length one. If you're reading different data from
each
> input file, then those arrays must match the number of input files.
>
> If this is running really slow, you could consider removing KT_CORR
and
> SP_CORR output from the cnt line type and setting rank_corr_flag =
FALSE.
> That'd make it run faster.
>
> I do notice in your run script (series_stat_test2_201907.sh) that
there's a
> 1.5 hour offset between the timestamps of the forecast and
observation
> data. So series-analysis is probably printing a bunch of warning
messages
> about that.
>
> I don't see any obvious problems in your config file, other than the
> suggestions above. You say you were surprised to see such large
> differences. You could create and look at a time-series of plots
for the
> raw fields. Perhaps there's some discrepancy in the data you aren't
> expecting?
>
> For example, you could pick a reasonable min/max value for the raw
data.
> Loop over the files in the input directory and run MET's
plot_data_plane
> tool with the -plot_range command line option to keep the min/max
plotting
> value fixed. Once that's done you could convert the PostScript
files to an
> animated gif:
>
> convert -density 288 -geometry 50% -rotate 90 -delay 100 -background
white
> -dispose Background *.ps anim.gif
>
> Then do the same thing for the obs data. Look at those animated
gifs
> side-by-side to see if there's anything surprising in the data that
you
> weren't expecting.
>
> John
>
>
> On Wed, Dec 4, 2019 at 12:23 PM Partha Bhattacharjee - NOAA
Affiliate via
> RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
> >
> > Sure, no problem.
> > Thanks
> > Partha
> >
> > On Wed, Dec 4, 2019 at 2:20 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Hi Partha.
> > >
> > > I have assigned this ticket to John. Please allow a few days
for a
> > > response. Thanks!
> > >
> > > Julie
> > >
> > > On Wed, Dec 4, 2019 at 11:59 AM Partha Bhattacharjee - NOAA
Affiliate
> via
> > > RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Wed Dec 04 11:58:37 2019: Request 93333 was acted upon.
> > > > Transaction: Ticket created by partha.bhattacharjee at noaa.gov
> > > > Queue: met_help
> > > > Subject: met-series stat
> > > > Owner: Nobody
> > > > Requestors: partha.bhattacharjee at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333
> > >
> > > >
> > > >
> > > > Hello,
> > > > I am running met series-stat using version 8 to verify two
nearly
> > > identical
> > > > model forecasts but getting very different stats when looking
into
> Bias
> > > etc
> > > > fields. I have used identical configuration file for both of
them.
> > > Attached
> > > > figure is total aerosol optical depth from two test runs
verifying
> > > against
> > > > same analysis dataset. I need help whether I need to change
any
> > settings
> > > in
> > > > the configuration file, as I am not expecting that huge
difference
> > > between
> > > > these two sets of runs.
> > > >
> > > > .sh and Seriesstat config file are in wcoss tide at :
> > > > /global/noscrub/Partha.Bhattacharjee/MET
> > > >
> > > > Let me know if you can access them.
> > > >
> > > > Thanks
> > > > Partha
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
>
--
Partha S. Bhattacharjee, Ph.D.
I.M. Systems Group
Environmental Modeling Centre
NCWCP
5830 University Research Court.
College Park, MD 20740
------------------------------------------------
Subject: met-series stat
From: John Halley Gotway
Time: Tue Dec 10 16:28:40 2019
Partha,
Sorry for the delay in getting you a response. I've been focused on
some
development tasks for the next release.
I see that MET is not matching the GRIB records you intend and you'd
like
to figure out how to match the one you want. Specifically, in the
file
listed below, MET is matching record 205, but you want 209:
/global/noscrub/Partha.Bhattacharjee/NRT_NGACv2/GSD_Chem/LiPan_v88V2/20190705/gfs.t00z.pgrb2.0p25.f000
# Here's the commands I used to look at this issue
*module load grib_util*
*$WGRIB2
/global/noscrub/Partha.Bhattacharjee/NRT_NGACv2/GSD_Chem/LiPan_v88V2/20190705/gfs.t00z.pgrb2.0p25.f000
| egrep AOTK*
*module use /global/noscrub/Julie.Prestopnik/modulefilesmodule load
met/8.1*
# When I run it through plot_data_plane, I do see a warning about
"multiple
matches"
*plot_data_plane
/global/noscrub/Partha.Bhattacharjee/NRT_NGACv2/GSD_Chem/LiPan_v88V2/20190705/gfs.t00z.pgrb2.0p25.f000
~/aotk.ps <http://aotk.ps> 'name="AOTK"; level="A0";'*
WARNING: MetGrib2DataFile::data_plane() -> Using the first of 12
matching
records for "AOTK/A0":
WARNING: record 205 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 338, 9, 342, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 208 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 430, 9, 450, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 209 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 545, 9, 565, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 211 field 1: ipdtmpl[26] = 20, 102, 62001, 0, 6, 20,
0, 0,
7, 9, 545, 9, 565, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 213 field 1: ipdtmpl[26] = 20, 102, 62008, 0, 6, 20,
0, 0,
7, 9, 545, 9, 565, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 215 field 1: ipdtmpl[26] = 20, 102, 62006, 0, 6, 20,
0, 0,
7, 9, 545, 9, 565, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 217 field 1: ipdtmpl[26] = 20, 102, 62010, 0, 6, 20,
0, 0,
7, 9, 545, 9, 565, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 219 field 1: ipdtmpl[26] = 20, 102, 62009, 0, 6, 20,
0, 0,
7, 9, 545, 9, 565, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 221 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 620, 9, 670, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 222 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 841, 9, 876, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 223 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 1628, 9, 1652, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: record 224 field 1: ipdtmpl[26] = 20, 102, 62000, 0, 6, 20,
0, 0,
7, 9, 11000, 9, 11200, 2, 0, 81, 0, 0, 1, 0, 10, 0, 0, 255, 0, 0
WARNING: Try setting the 0-based "GRIB2_ipdtmpl_index" and
"GRIB2_ipdtmpl_val" config file options to select one.
# Since you want records 209, you could use "GRIB2_ipdtmpl_index = [
2, 10
]; GRIB2_ipdtmpl_val = [ 62000, 545 ];"
*plot_data_plane
/global/noscrub/Partha.Bhattacharjee/NRT_NGACv2/GSD_Chem/LiPan_v88V2/20190705/gfs.t00z.pgrb2.0p25.f000
~/aotk.ps <http://aotk.ps> 'name="AOTK"; level="A0";
GRIB2_ipdtmpl_index =
[ 2, 10 ]; GRIB2_ipdtmpl_val = [ 62000, 545 ];'*
And that produces the desired match:
DEBUG 1: Opening data file:
/global/noscrub/Partha.Bhattacharjee/NRT_NGACv2/GSD_Chem/LiPan_v88V2/20190705/gfs.t00z.pgrb2.0p25.f000
DEBUG 3: MetGrib2DataFile::data_plane() -> *Found exact match for
'AOTK/A0'
in GRIB2 record 209 field 1* of GRIB2 file
'/global/noscrub/Partha.Bhattacharjee/NRT_NGACv2/GSD_Chem/LiPan_v88V2/20190705/gfs.t00z.pgrb2.0p25.f000'
DEBUG 1: Creating postscript file: /u/John.H.Gotway/aotk.ps
So you'd just put those additional filtering settings into the config
file. Hope that helps.
Thanks,
John
On Fri, Dec 6, 2019 at 5:05 PM Partha Bhattacharjee - NOAA Affiliate
via RT
<met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
>
> John,
> I am looking into Grib record and trying to match whether MET Config
file
> picking up exactly the same.
> In the config file I sent you earlier you can see I am trying to
read
> grib record name = "AOTK"; GRIB2_process=62000. level A0 (total AOD
at
> 550nm to match against obs), from all the input grib2 files. So, for
each
> day it looks into 0, 6, 12 and 18 hour forecast for the entire
month. When
> I look into the log file, I found MET is picking up :
>
> MetGrib2DataFile::data_plane() - Found exact match for 'AOTK/A0' in
GRIB2
> record 205 field 1 of GRIB2 file
'./20190705/gfs.t00z.pgrb2.0p25.f000'
> DEBUG 3: MetGrib2DataFile::data_plane() - Found exact match for
'AOTK/A0'
> in GRIB2 record 177 field 1 of GRIB2 file
> './20190705/gfs.t00z.pgrb2.0p25.f006'
> DEBUG 3: MetGrib2DataFile::data_plane() - Found exact match for
'AOTK/A0'
> in GRIB2 record 177 field 1 of GRIB2 file
> './20190705/gfs.t00z.pgrb2.0p25.f012'
>
> but wgrib on the file shows
> 205:127608157:d=2019070500:AOTK:entire atmosphere:anl:chemical=Total
> Aerosol:aerosol_size <2e-05:aerosol_wavelength 3.38e-07-3.42e-07
> it is total AOD but at different wavelength (needs to read 209
instead).
> 209:131963245:d=2019070500:AOTK:entire atmosphere:anl:chemical=Total
> Aerosol:aerosol_size <2e-05:aerosol_wavelength 5.45e-07-5.65e-07
>
> Similarly, for other forecast hours, it should read record 182, not
177.
>
> It seems to be GRIB2_process=62000 is not enough as some other
> wavelength may have the same variable. Should I just put
'name="var209"
> instead ?=in the config ?
>
> Thanks
> Partha
>
>
>
>
>
> On Wed, Dec 4, 2019 at 2:55 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Partha,
> >
> > I see you have questions about the Series-Analysis tool in MET.
> >
> > I'm looking at your config file on tide
> > in
/global/noscrub/Partha.Bhattacharjee/MET/SeriesAnlConfig_July19_Test12
> >
> > Here are some things I notice...
> > (1) In the "fcst" section, you do NOT need to list level = [ "A0",
"A0",
> > ... "A0" ]; multiple times. Since you're reading the same data
from each
> > input file, you only need to list it once:
> > fcst = { field = [ { name = "AOTK"; level = [ "A0" ]; } ]; }
> > (2) The same is true in the "obs" section. You only need to list
> "(0,*,*)"
> > once.
> >
> > The fcst.field and obs.field settings are technically an array.
If
> you're
> > reading the same data from each of the input files (as you are),
that
> array
> > only needs to be length one. If you're reading different data
from each
> > input file, then those arrays must match the number of input
files.
> >
> > If this is running really slow, you could consider removing
KT_CORR and
> > SP_CORR output from the cnt line type and setting rank_corr_flag =
FALSE.
> > That'd make it run faster.
> >
> > I do notice in your run script (series_stat_test2_201907.sh) that
> there's a
> > 1.5 hour offset between the timestamps of the forecast and
observation
> > data. So series-analysis is probably printing a bunch of warning
> messages
> > about that.
> >
> > I don't see any obvious problems in your config file, other than
the
> > suggestions above. You say you were surprised to see such large
> > differences. You could create and look at a time-series of plots
for the
> > raw fields. Perhaps there's some discrepancy in the data you
aren't
> > expecting?
> >
> > For example, you could pick a reasonable min/max value for the raw
data.
> > Loop over the files in the input directory and run MET's
plot_data_plane
> > tool with the -plot_range command line option to keep the min/max
> plotting
> > value fixed. Once that's done you could convert the PostScript
files to
> an
> > animated gif:
> >
> > convert -density 288 -geometry 50% -rotate 90 -delay 100
-background
> white
> > -dispose Background *.ps anim.gif
> >
> > Then do the same thing for the obs data. Look at those animated
gifs
> > side-by-side to see if there's anything surprising in the data
that you
> > weren't expecting.
> >
> > John
> >
> >
> > On Wed, Dec 4, 2019 at 12:23 PM Partha Bhattacharjee - NOAA
Affiliate via
> > RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333 >
> > >
> > > Sure, no problem.
> > > Thanks
> > > Partha
> > >
> > > On Wed, Dec 4, 2019 at 2:20 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Partha.
> > > >
> > > > I have assigned this ticket to John. Please allow a few days
for a
> > > > response. Thanks!
> > > >
> > > > Julie
> > > >
> > > > On Wed, Dec 4, 2019 at 11:59 AM Partha Bhattacharjee - NOAA
Affiliate
> > via
> > > > RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Wed Dec 04 11:58:37 2019: Request 93333 was acted upon.
> > > > > Transaction: Ticket created by partha.bhattacharjee at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: met-series stat
> > > > > Owner: Nobody
> > > > > Requestors: partha.bhattacharjee at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93333
> > > >
> > > > >
> > > > >
> > > > > Hello,
> > > > > I am running met series-stat using version 8 to verify two
nearly
> > > > identical
> > > > > model forecasts but getting very different stats when
looking into
> > Bias
> > > > etc
> > > > > fields. I have used identical configuration file for both of
them.
> > > > Attached
> > > > > figure is total aerosol optical depth from two test runs
verifying
> > > > against
> > > > > same analysis dataset. I need help whether I need to change
any
> > > settings
> > > > in
> > > > > the configuration file, as I am not expecting that huge
difference
> > > > between
> > > > > these two sets of runs.
> > > > >
> > > > > .sh and Seriesstat config file are in wcoss tide at :
> > > > > /global/noscrub/Partha.Bhattacharjee/MET
> > > > >
> > > > > Let me know if you can access them.
> > > > >
> > > > > Thanks
> > > > > Partha
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> >
>
> --
> Partha S. Bhattacharjee, Ph.D.
> I.M. Systems Group
> Environmental Modeling Centre
> NCWCP
> 5830 University Research Court.
> College Park, MD 20740
>
>
------------------------------------------------
More information about the Met_help
mailing list