[Met_help] [rt.rap.ucar.edu #99857] History for MET V9.0 plot_point_obs (UNCLASSIFIED)
John Halley Gotway via RT
met_help at ucar.edu
Thu May 13 09:33:02 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
CLASSIFICATION: UNCLASSIFIED
I ran point-stat with an input obs file containing point obs for a large
domain and an input GRIB forecast file for a much smaller domain located in
the middle of the larger domain. The point-stat log file showed that there
were 2100 ADPSFC TMP/Z2 obs designated "off the grid". I assume this means
that they fell outside the smaller forecast domain. I then ran
plot_point_obs using the same input obs file and used the argument
-data_file to point to the forecast file to produce the plot over the
smaller domain. The plot_point_obs log file shows that 1610 ADPSFC TMP/Z2
obs were skipped off the grid. Why is there such a large difference in the
determination of obs located outside the smaller domain?
I noticed that the online MET User's Guide contains what appears to be
duplicate sections describing the plot_point_obs configuration file. These
sections are 27.1.2 and 27.1.3.
https://met.readthedocs.io/en/latest/Users_Guide/plotting.html
Thanks.
R/
John
Mr. John W. Raby
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
Office:(575) 678-2004 DSN 258-2004
Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
CLASSIFICATION: UNCLASSIFIED
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: Julie Prestopnik
Time: Tue May 11 17:05:20 2021
Hi John.
I have requested that John HG take a look. Please allow a few
business
days for a response.
Thanks for pointing out the duplicate information in the
documentation. It
looks like they're not quite identical as 27.1.2 has a “regrid” entry,
but
27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there and
27.1.3
is not. We'll get that figured out and will update the documentation
appropriately.
Julie
On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I ran point-stat with an input obs file containing point obs for a
large
> domain and an input GRIB forecast file for a much smaller domain
located in
> the middle of the larger domain. The point-stat log file showed that
there
> were 2100 ADPSFC TMP/Z2 obs designated "off the grid". I assume this
means
> that they fell outside the smaller forecast domain. I then ran
> plot_point_obs using the same input obs file and used the argument
> -data_file to point to the forecast file to produce the plot over
the
> smaller domain. The plot_point_obs log file shows that 1610 ADPSFC
TMP/Z2
> obs were skipped off the grid. Why is there such a large difference
in the
> determination of obs located outside the smaller domain?
>
> I noticed that the online MET User's Guide contains what appears to
be
> duplicate sections describing the plot_point_obs configuration file.
These
> sections are 27.1.2 and 27.1.3.
> https://met.readthedocs.io/en/latest/Users_Guide/plotting.html
>
> Thanks.
>
> R/
> John
>
> Mr. John W. Raby
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> Office:(575) 678-2004 DSN 258-2004
> Teleworking: Contact Admin Specialist at (410) 278-6974
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed May 12 07:03:26 2021
CLASSIFICATION: UNCLASSIFIED
Hi Julie -
Thanks for passing on my request and for considering an update to the
documentation.
R/
John
-----Original Message-----
From: Julie Prestopnik via RT <met_help at ucar.edu>
Sent: Tuesday, May 11, 2021 5:05 PM
To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
plot_point_obs (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
Hi John.
I have requested that John HG take a look. Please allow a few
business days for a response.
Thanks for pointing out the duplicate information in the
documentation. It looks like they're not quite identical as 27.1.2
has a “regrid” entry, but
27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there and
27.1.3 is not. We'll get that figured out and will update the
documentation appropriately.
Julie
On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I ran point-stat with an input obs file containing point obs for a
> large domain and an input GRIB forecast file for a much smaller
domain
> located in the middle of the larger domain. The point-stat log file
> showed that there were 2100 ADPSFC TMP/Z2 obs designated "off the
> grid". I assume this means that they fell outside the smaller
forecast
> domain. I then ran plot_point_obs using the same input obs file and
> used the argument -data_file to point to the forecast file to
produce
> the plot over the smaller domain. The plot_point_obs log file shows
> that 1610 ADPSFC TMP/Z2 obs were skipped off the grid. Why is there
> such a large difference in the determination of obs located outside
the smaller domain?
>
> I noticed that the online MET User's Guide contains what appears to
be
> duplicate sections describing the plot_point_obs configuration file.
> These sections are 27.1.2 and 27.1.3.
> Caution-
https://met.readthedocs.io/en/latest/Users_Guide/plotting.html
>
> Thanks.
>
> R/
> John
>
> Mr. John W. Raby
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> Office:(575) 678-2004 DSN 258-2004
> Teleworking: Contact Admin Specialist at (410) 278-6974 FAX (575)
> 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research Research Applications
Laboratory
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to reply to this email outside of your normal working hours.
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed May 12 10:48:44 2021
John,
Your question is, why do the number of observations reported as being
off
the grid differ between point-stat and plot-point-obs? You're finding
that
the number reported by point-stat is higher than the number reported
by
plot-point-obs... 2100 from point-stat and 1600 from plot-point-obs.
I did some testing and have an explanation.
The point-stat and plot-point-obs tools just count things and apply
filtering logic in different ways. I ran with some sample data
included in
the MET release. Here's some log output from point-stat when verifying
2m
temperature:
DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation type
ADPSFC,
over region FULL, for interpolation method NEAREST(1), using 557
matched
pairs.
DEBUG 3: Number of matched pairs = 557
DEBUG 3: Observations processed = 87753
DEBUG 3: Rejected: station id = 0
DEBUG 3: Rejected: obs var name = 77648
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 7571
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 1612
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 59
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: bad climo mean = 0
DEBUG 3: Rejected: bad climo stdev = 0
DEBUG 3: Rejected: mpr filter = 0
DEBUG 3: Rejected: duplicates = 306
The order of the log messages matches the order in which the filtering
logic is applied. So point-stat processed 87753 individual
observations and
discarded 77648 because they weren't TMP. The next non-zero thing is
"off
the grid". Of the 10,105 observations of temperature, 7571 were not on
my
verification grid. Then additional filtering logic is applied to the
remaining 2534 observations. Some of those are discarded for not being
at 2
meters and others for being the wrong message type.
I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP and
-plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat counts
individual
observations, plot-point-obs counts observation LOCATIONS. In this
case,
plot-point-obs discards 1636 locations for being off the grid. Note
that
plot-point-obs only filters by message type (ADPSFC) and variable name
(TMP). It has not filtered by vertical level. If one station reports
100
times in your point obs file, that location is only counted and
plotted
once. So a dot is plotted in the output if there was at least one
observation of ADPSFC TMP at that location.
Another exercise is going the other way... can you get point-stat to
report
the exact same number of matched pairs that are plotted by plot-point-
obs?
I was eventually able to do so, but had to tweak my point-stat config
file.
Since each station can report multiple times, I setup point-stat to
only
use the one closest to valid time of my forecast... that way multiple
reports at 1 station result in only 1 matched pair being created:
obs_summary = NEAREST;
And with that, point-stat reported 557 matched pairs and plot-point-
obs
plotted 557 locations:
>From point-stat...
DEBUG 3: Number of matched pairs = 557
>From plot-point-obs...
DEBUG 2: Finished plotting 557 locations.
Does that all make sense?
Thanks,
John HG
On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> Hi Julie -
>
> Thanks for passing on my request and for considering an update to
the
> documentation.
>
> R/
> John
>
> -----Original Message-----
> From: Julie Prestopnik via RT <met_help at ucar.edu>
> Sent: Tuesday, May 11, 2021 5:05 PM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> Hi John.
>
> I have requested that John HG take a look. Please allow a few
business
> days for a response.
>
> Thanks for pointing out the duplicate information in the
documentation.
> It looks like they're not quite identical as 27.1.2 has a “regrid”
entry,
> but
> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there and
27.1.3
> is not. We'll get that figured out and will update the
documentation
> appropriately.
>
> Julie
>
> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > I ran point-stat with an input obs file containing point obs for a
> > large domain and an input GRIB forecast file for a much smaller
domain
> > located in the middle of the larger domain. The point-stat log
file
> > showed that there were 2100 ADPSFC TMP/Z2 obs designated "off the
> > grid". I assume this means that they fell outside the smaller
forecast
> > domain. I then ran plot_point_obs using the same input obs file
and
> > used the argument -data_file to point to the forecast file to
produce
> > the plot over the smaller domain. The plot_point_obs log file
shows
> > that 1610 ADPSFC TMP/Z2 obs were skipped off the grid. Why is
there
> > such a large difference in the determination of obs located
outside the
> smaller domain?
> >
> > I noticed that the online MET User's Guide contains what appears
to be
> > duplicate sections describing the plot_point_obs configuration
file.
> > These sections are 27.1.2 and 27.1.3.
> > Caution-
https://met.readthedocs.io/en/latest/Users_Guide/plotting.html
> >
> > Thanks.
> >
> > R/
> > John
> >
> > Mr. John W. Raby
> > U.S. Army Research Laboratory
> > White Sands Missile Range, NM 88002
> > Office:(575) 678-2004 DSN 258-2004
> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX (575)
> > 678-1230 DSN 258-1230
> > Email: john.w.raby2.civ at mail.mil
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> --
> Julie Prestopnik (she/her)
> Software Engineer
> National Center for Atmospheric Research Research Applications
Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed May 12 11:22:28 2021
And John, thanks for the heads up about the docs. I agree about the
duplication and am fixing it via:
https://github.com/dtcenter/MET/issues/1789
Once the changes are accepted and the docs are rebuilt, they will be
corrected here:
https://met.readthedocs.io/en/latest/
Thanks,
John
On Wed, May 12, 2021 at 10:48 AM John Halley Gotway <johnhg at ucar.edu>
wrote:
> John,
>
> Your question is, why do the number of observations reported as
being off
> the grid differ between point-stat and plot-point-obs? You're
finding that
> the number reported by point-stat is higher than the number reported
by
> plot-point-obs... 2100 from point-stat and 1600 from plot-point-obs.
>
> I did some testing and have an explanation.
>
> The point-stat and plot-point-obs tools just count things and apply
> filtering logic in different ways. I ran with some sample data
included in
> the MET release. Here's some log output from point-stat when
verifying 2m
> temperature:
>
> DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
557
> matched pairs.
> DEBUG 3: Number of matched pairs = 557
> DEBUG 3: Observations processed = 87753
> DEBUG 3: Rejected: station id = 0
> DEBUG 3: Rejected: obs var name = 77648
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 7571
> DEBUG 3: Rejected: topography = 0
> DEBUG 3: Rejected: level mismatch = 1612
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 59
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: bad climo mean = 0
> DEBUG 3: Rejected: bad climo stdev = 0
> DEBUG 3: Rejected: mpr filter = 0
> DEBUG 3: Rejected: duplicates = 306
>
> The order of the log messages matches the order in which the
filtering
> logic is applied. So point-stat processed 87753 individual
observations and
> discarded 77648 because they weren't TMP. The next non-zero thing is
"off
> the grid". Of the 10,105 observations of temperature, 7571 were not
on my
> verification grid. Then additional filtering logic is applied to the
> remaining 2534 observations. Some of those are discarded for not
being at 2
> meters and others for being the wrong message type.
>
> I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP and
> -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat counts
individual
> observations, plot-point-obs counts observation LOCATIONS. In this
case,
> plot-point-obs discards 1636 locations for being off the grid. Note
that
> plot-point-obs only filters by message type (ADPSFC) and variable
name
> (TMP). It has not filtered by vertical level. If one station reports
100
> times in your point obs file, that location is only counted and
plotted
> once. So a dot is plotted in the output if there was at least one
> observation of ADPSFC TMP at that location.
>
> Another exercise is going the other way... can you get point-stat to
> report the exact same number of matched pairs that are plotted by
> plot-point-obs? I was eventually able to do so, but had to tweak my
> point-stat config file. Since each station can report multiple
times, I
> setup point-stat to only use the one closest to valid time of my
> forecast... that way multiple reports at 1 station result in only 1
matched
> pair being created:
> obs_summary = NEAREST;
>
> And with that, point-stat reported 557 matched pairs and plot-point-
obs
> plotted 557 locations:
>
> From point-stat...
> DEBUG 3: Number of matched pairs = 557
>
> From plot-point-obs...
> DEBUG 2: Finished plotting 557 locations.
>
> Does that all make sense?
>
> Thanks,
> John HG
>
>
>
> On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>>
>> CLASSIFICATION: UNCLASSIFIED
>>
>> Hi Julie -
>>
>> Thanks for passing on my request and for considering an update to
the
>> documentation.
>>
>> R/
>> John
>>
>> -----Original Message-----
>> From: Julie Prestopnik via RT <met_help at ucar.edu>
>> Sent: Tuesday, May 11, 2021 5:05 PM
>> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
>> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
>> plot_point_obs (UNCLASSIFIED)
>>
>> All active links contained in this email were disabled. Please
verify
>> the identity of the sender, and confirm the authenticity of all
links
>> contained within the message prior to copying and pasting the
address to a
>> Web browser.
>>
>>
>>
>>
>> ----
>>
>> Hi John.
>>
>> I have requested that John HG take a look. Please allow a few
business
>> days for a response.
>>
>> Thanks for pointing out the duplicate information in the
documentation.
>> It looks like they're not quite identical as 27.1.2 has a “regrid”
entry,
>> but
>> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there and
>> 27.1.3 is not. We'll get that figured out and will update the
>> documentation appropriately.
>>
>> Julie
>>
>> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
>> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> > Queue: met_help
>> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
>> > Owner: Nobody
>> > Requestors: john.w.raby2.civ at mail.mil
>> > Status: new
>> > Ticket <Caution-url:
>> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>> >
>> >
>> > CLASSIFICATION: UNCLASSIFIED
>> >
>> > I ran point-stat with an input obs file containing point obs for
a
>> > large domain and an input GRIB forecast file for a much smaller
domain
>> > located in the middle of the larger domain. The point-stat log
file
>> > showed that there were 2100 ADPSFC TMP/Z2 obs designated "off the
>> > grid". I assume this means that they fell outside the smaller
forecast
>> > domain. I then ran plot_point_obs using the same input obs file
and
>> > used the argument -data_file to point to the forecast file to
produce
>> > the plot over the smaller domain. The plot_point_obs log file
shows
>> > that 1610 ADPSFC TMP/Z2 obs were skipped off the grid. Why is
there
>> > such a large difference in the determination of obs located
outside the
>> smaller domain?
>> >
>> > I noticed that the online MET User's Guide contains what appears
to be
>> > duplicate sections describing the plot_point_obs configuration
file.
>> > These sections are 27.1.2 and 27.1.3.
>> > Caution-
https://met.readthedocs.io/en/latest/Users_Guide/plotting.html
>> >
>> > Thanks.
>> >
>> > R/
>> > John
>> >
>> > Mr. John W. Raby
>> > U.S. Army Research Laboratory
>> > White Sands Missile Range, NM 88002
>> > Office:(575) 678-2004 DSN 258-2004
>> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX (575)
>> > 678-1230 DSN 258-1230
>> > Email: john.w.raby2.civ at mail.mil
>> >
>> >
>> > CLASSIFICATION: UNCLASSIFIED
>> >
>> >
>>
>> --
>> Julie Prestopnik (she/her)
>> Software Engineer
>> National Center for Atmospheric Research Research Applications
Laboratory
>> Email: jpresto at ucar.edu
>>
>> My working day may not be your working day. Please do not feel
obliged
>> to reply to this email outside of your normal working hours.
>>
>> CLASSIFICATION: UNCLASSIFIED
>>
>>
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed May 12 15:04:18 2021
CLASSIFICATION: UNCLASSIFIED
Hi John -
Thanks for initiating the fix process for the duplication in the
User's Guide.
Thanks for creating the data to replicate my situation. I understand
that it is expected that there are differences between the way point-
stat filters the observations and decides how many are off the grid
and the way that plot-point-obs does it. I was thinking that it had
something to do with this, but thought maybe it would be worth it to
investigate this further. The point-stat config file I used has the
following settings:
duplicate_flag = UNIQUE;
obs_summary = NEAREST;
My understanding is that the duplicate_flag setting of UNIQUE will
select one ob in the event there are matching obs.
Having the obs_summary = NEAREST, assures that only the ob closest
in time to the valid time of the forecast. So having these settings in
tandem would have effectively worked the same as your exercise going
the other way. So I went back and looked at the number of matched
pairs from point-stat and the number of obs plotted by plot-point-obs
and they were different. Was there any other tweak you did to get the
numbers of matched pairs and the number of obs plotted to match?
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, May 12, 2021 11:22 AM
To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
plot_point_obs (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
And John, thanks for the heads up about the docs. I agree about the
duplication and am fixing it via:
Caution-https://github.com/dtcenter/MET/issues/1789
Once the changes are accepted and the docs are rebuilt, they will be
corrected here:
Caution-https://met.readthedocs.io/en/latest/
Thanks,
John
On Wed, May 12, 2021 at 10:48 AM John Halley Gotway <johnhg at ucar.edu>
wrote:
> John,
>
> Your question is, why do the number of observations reported as
being
> off the grid differ between point-stat and plot-point-obs? You're
> finding that the number reported by point-stat is higher than the
> number reported by plot-point-obs... 2100 from point-stat and 1600
from plot-point-obs.
>
> I did some testing and have an explanation.
>
> The point-stat and plot-point-obs tools just count things and apply
> filtering logic in different ways. I ran with some sample data
> included in the MET release. Here's some log output from point-stat
> when verifying 2m
> temperature:
>
> DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
> 557 matched pairs.
> DEBUG 3: Number of matched pairs = 557
> DEBUG 3: Observations processed = 87753
> DEBUG 3: Rejected: station id = 0
> DEBUG 3: Rejected: obs var name = 77648
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 7571
> DEBUG 3: Rejected: topography = 0
> DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected: quality
> marker = 0
> DEBUG 3: Rejected: message type = 59
> DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad fcst
> value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG 3: Rejected:
> bad climo stdev = 0
> DEBUG 3: Rejected: mpr filter = 0
> DEBUG 3: Rejected: duplicates = 306
>
> The order of the log messages matches the order in which the
filtering
> logic is applied. So point-stat processed 87753 individual
> observations and discarded 77648 because they weren't TMP. The next
> non-zero thing is "off the grid". Of the 10,105 observations of
> temperature, 7571 were not on my verification grid. Then additional
> filtering logic is applied to the remaining 2534 observations. Some
of
> those are discarded for not being at 2 meters and others for being
the wrong message type.
>
> I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP and
> -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat counts
> individual observations, plot-point-obs counts observation
LOCATIONS.
> In this case, plot-point-obs discards 1636 locations for being off
the
> grid. Note that plot-point-obs only filters by message type (ADPSFC)
> and variable name (TMP). It has not filtered by vertical level. If
one
> station reports 100 times in your point obs file, that location is
> only counted and plotted once. So a dot is plotted in the output if
> there was at least one observation of ADPSFC TMP at that location.
>
> Another exercise is going the other way... can you get point-stat to
> report the exact same number of matched pairs that are plotted by
> plot-point-obs? I was eventually able to do so, but had to tweak my
> point-stat config file. Since each station can report multiple
times,
> I setup point-stat to only use the one closest to valid time of my
> forecast... that way multiple reports at 1 station result in only 1
> matched pair being created:
> obs_summary = NEAREST;
>
> And with that, point-stat reported 557 matched pairs and
> plot-point-obs plotted 557 locations:
>
> From point-stat...
> DEBUG 3: Number of matched pairs = 557
>
> From plot-point-obs...
> DEBUG 2: Finished plotting 557 locations.
>
> Does that all make sense?
>
> Thanks,
> John HG
>
>
>
> On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <Caution-url:
>> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>>
>> CLASSIFICATION: UNCLASSIFIED
>>
>> Hi Julie -
>>
>> Thanks for passing on my request and for considering an update to
the
>> documentation.
>>
>> R/
>> John
>>
>> -----Original Message-----
>> From: Julie Prestopnik via RT <met_help at ucar.edu>
>> Sent: Tuesday, May 11, 2021 5:05 PM
>> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
>> <john.w.raby2.civ at mail.mil>
>> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
>> plot_point_obs (UNCLASSIFIED)
>>
>> All active links contained in this email were disabled. Please
>> verify the identity of the sender, and confirm the authenticity of
>> all links contained within the message prior to copying and pasting
>> the address to a Web browser.
>>
>>
>>
>>
>> ----
>>
>> Hi John.
>>
>> I have requested that John HG take a look. Please allow a few
>> business days for a response.
>>
>> Thanks for pointing out the duplicate information in the
documentation.
>> It looks like they're not quite identical as 27.1.2 has a “regrid”
>> entry, but
>> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there and
>> 27.1.3 is not. We'll get that figured out and will update the
>> documentation appropriately.
>>
>> Julie
>>
>> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
>> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> > Queue: met_help
>> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
>> > Owner: Nobody
>> > Requestors: john.w.raby2.civ at mail.mil
>> > Status: new
>> > Ticket <Caution-Caution-url:
>> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=9
>> > 9857 >
>> >
>> >
>> > CLASSIFICATION: UNCLASSIFIED
>> >
>> > I ran point-stat with an input obs file containing point obs for
a
>> > large domain and an input GRIB forecast file for a much smaller
>> > domain located in the middle of the larger domain. The point-stat
>> > log file showed that there were 2100 ADPSFC TMP/Z2 obs designated
>> > "off the grid". I assume this means that they fell outside the
>> > smaller forecast domain. I then ran plot_point_obs using the same
>> > input obs file and used the argument -data_file to point to the
>> > forecast file to produce the plot over the smaller domain. The
>> > plot_point_obs log file shows that 1610 ADPSFC TMP/Z2 obs were
>> > skipped off the grid. Why is there such a large difference in the
>> > determination of obs located outside the
>> smaller domain?
>> >
>> > I noticed that the online MET User's Guide contains what appears
to
>> > be duplicate sections describing the plot_point_obs configuration
file.
>> > These sections are 27.1.2 and 27.1.3.
>> > Caution-Caution-
https://met.readthedocs.io/en/latest/Users_Guide/pl
>> > otting.html
>> >
>> > Thanks.
>> >
>> > R/
>> > John
>> >
>> > Mr. John W. Raby
>> > U.S. Army Research Laboratory
>> > White Sands Missile Range, NM 88002
>> > Office:(575) 678-2004 DSN 258-2004
>> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX (575)
>> > 678-1230 DSN 258-1230
>> > Email: john.w.raby2.civ at mail.mil
>> >
>> >
>> > CLASSIFICATION: UNCLASSIFIED
>> >
>> >
>>
>> --
>> Julie Prestopnik (she/her)
>> Software Engineer
>> National Center for Atmospheric Research Research Applications
>> Laboratory
>> Email: jpresto at ucar.edu
>>
>> My working day may not be your working day. Please do not feel
>> obliged to reply to this email outside of your normal working
hours.
>>
>> CLASSIFICATION: UNCLASSIFIED
>>
>>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed May 12 15:11:23 2021
John,
I did also open up the obs_window setting in the Point-Stat config
file by
adding a few 0's to the default setting:
obs_window = { beg = -5400000; end = 5400000; }
I just wanted to make sure that Point-Stat used obs from ALL times in
the
input file rather than subsetting by time. Plot-Point-Obs is NOT
filtering
by time, so I didn't want Point-Stat to either.
Just all depends on the specifics of your data. In general, Plot-
Point-Obs
is doing less filtering than Point-Stat.
John
On Wed, May 12, 2021 at 3:04 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> Hi John -
>
> Thanks for initiating the fix process for the duplication in the
User's
> Guide.
>
> Thanks for creating the data to replicate my situation. I understand
that
> it is expected that there are differences between the way point-stat
> filters the observations and decides how many are off the grid and
the way
> that plot-point-obs does it. I was thinking that it had something to
do
> with this, but thought maybe it would be worth it to investigate
this
> further. The point-stat config file I used has the following
settings:
>
> duplicate_flag = UNIQUE;
> obs_summary = NEAREST;
>
> My understanding is that the duplicate_flag setting of UNIQUE will
select
> one ob in the event there are matching obs.
> Having the obs_summary = NEAREST, assures that only the ob
closest in
> time to the valid time of the forecast. So having these settings in
tandem
> would have effectively worked the same as your exercise going the
other
> way. So I went back and looked at the number of matched pairs from
> point-stat and the number of obs plotted by plot-point-obs and they
were
> different. Was there any other tweak you did to get the numbers of
matched
> pairs and the number of obs plotted to match?
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, May 12, 2021 11:22 AM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> And John, thanks for the heads up about the docs. I agree about the
> duplication and am fixing it via:
> Caution-https://github.com/dtcenter/MET/issues/1789
>
> Once the changes are accepted and the docs are rebuilt, they will be
> corrected here:
> Caution-https://met.readthedocs.io/en/latest/
>
> Thanks,
> John
>
> On Wed, May 12, 2021 at 10:48 AM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > John,
> >
> > Your question is, why do the number of observations reported as
being
> > off the grid differ between point-stat and plot-point-obs? You're
> > finding that the number reported by point-stat is higher than the
> > number reported by plot-point-obs... 2100 from point-stat and 1600
from
> plot-point-obs.
> >
> > I did some testing and have an explanation.
> >
> > The point-stat and plot-point-obs tools just count things and
apply
> > filtering logic in different ways. I ran with some sample data
> > included in the MET release. Here's some log output from point-
stat
> > when verifying 2m
> > temperature:
> >
> > DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation
type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 557 matched pairs.
> > DEBUG 3: Number of matched pairs = 557
> > DEBUG 3: Observations processed = 87753
> > DEBUG 3: Rejected: station id = 0
> > DEBUG 3: Rejected: obs var name = 77648
> > DEBUG 3: Rejected: valid time = 0
> > DEBUG 3: Rejected: bad obs value = 0
> > DEBUG 3: Rejected: off the grid = 7571
> > DEBUG 3: Rejected: topography = 0
> > DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected:
quality
> > marker = 0
> > DEBUG 3: Rejected: message type = 59
> > DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad fcst
> > value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG 3:
Rejected:
> > bad climo stdev = 0
> > DEBUG 3: Rejected: mpr filter = 0
> > DEBUG 3: Rejected: duplicates = 306
> >
> > The order of the log messages matches the order in which the
filtering
> > logic is applied. So point-stat processed 87753 individual
> > observations and discarded 77648 because they weren't TMP. The
next
> > non-zero thing is "off the grid". Of the 10,105 observations of
> > temperature, 7571 were not on my verification grid. Then
additional
> > filtering logic is applied to the remaining 2534 observations.
Some of
> > those are discarded for not being at 2 meters and others for being
the
> wrong message type.
> >
> > I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP
and
> > -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat counts
> > individual observations, plot-point-obs counts observation
LOCATIONS.
> > In this case, plot-point-obs discards 1636 locations for being off
the
> > grid. Note that plot-point-obs only filters by message type
(ADPSFC)
> > and variable name (TMP). It has not filtered by vertical level. If
one
> > station reports 100 times in your point obs file, that location is
> > only counted and plotted once. So a dot is plotted in the output
if
> > there was at least one observation of ADPSFC TMP at that location.
> >
> > Another exercise is going the other way... can you get point-stat
to
> > report the exact same number of matched pairs that are plotted by
> > plot-point-obs? I was eventually able to do so, but had to tweak
my
> > point-stat config file. Since each station can report multiple
times,
> > I setup point-stat to only use the one closest to valid time of my
> > forecast... that way multiple reports at 1 station result in only
1
> > matched pair being created:
> > obs_summary = NEAREST;
> >
> > And with that, point-stat reported 557 matched pairs and
> > plot-point-obs plotted 557 locations:
> >
> > From point-stat...
> > DEBUG 3: Number of matched pairs = 557
> >
> > From plot-point-obs...
> > DEBUG 2: Finished plotting 557 locations.
> >
> > Does that all make sense?
> >
> > Thanks,
> > John HG
> >
> >
> >
> > On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <Caution-url:
> >> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
> >>
> >> CLASSIFICATION: UNCLASSIFIED
> >>
> >> Hi Julie -
> >>
> >> Thanks for passing on my request and for considering an update to
the
> >> documentation.
> >>
> >> R/
> >> John
> >>
> >> -----Original Message-----
> >> From: Julie Prestopnik via RT <met_help at ucar.edu>
> >> Sent: Tuesday, May 11, 2021 5:05 PM
> >> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> >> <john.w.raby2.civ at mail.mil>
> >> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> >> plot_point_obs (UNCLASSIFIED)
> >>
> >> All active links contained in this email were disabled. Please
> >> verify the identity of the sender, and confirm the authenticity
of
> >> all links contained within the message prior to copying and
pasting
> >> the address to a Web browser.
> >>
> >>
> >>
> >>
> >> ----
> >>
> >> Hi John.
> >>
> >> I have requested that John HG take a look. Please allow a few
> >> business days for a response.
> >>
> >> Thanks for pointing out the duplicate information in the
documentation.
> >> It looks like they're not quite identical as 27.1.2 has a
“regrid”
> >> entry, but
> >> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there
and
> >> 27.1.3 is not. We'll get that figured out and will update the
> >> documentation appropriately.
> >>
> >> Julie
> >>
> >> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> >> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> >> > Queue: met_help
> >> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> >> > Owner: Nobody
> >> > Requestors: john.w.raby2.civ at mail.mil
> >> > Status: new
> >> > Ticket <Caution-Caution-url:
> >> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=9
> >> > 9857 >
> >> >
> >> >
> >> > CLASSIFICATION: UNCLASSIFIED
> >> >
> >> > I ran point-stat with an input obs file containing point obs
for a
> >> > large domain and an input GRIB forecast file for a much smaller
> >> > domain located in the middle of the larger domain. The point-
stat
> >> > log file showed that there were 2100 ADPSFC TMP/Z2 obs
designated
> >> > "off the grid". I assume this means that they fell outside the
> >> > smaller forecast domain. I then ran plot_point_obs using the
same
> >> > input obs file and used the argument -data_file to point to the
> >> > forecast file to produce the plot over the smaller domain. The
> >> > plot_point_obs log file shows that 1610 ADPSFC TMP/Z2 obs were
> >> > skipped off the grid. Why is there such a large difference in
the
> >> > determination of obs located outside the
> >> smaller domain?
> >> >
> >> > I noticed that the online MET User's Guide contains what
appears to
> >> > be duplicate sections describing the plot_point_obs
configuration
> file.
> >> > These sections are 27.1.2 and 27.1.3.
> >> > Caution-Caution-
https://met.readthedocs.io/en/latest/Users_Guide/pl
> >> > otting.html
> >> >
> >> > Thanks.
> >> >
> >> > R/
> >> > John
> >> >
> >> > Mr. John W. Raby
> >> > U.S. Army Research Laboratory
> >> > White Sands Missile Range, NM 88002
> >> > Office:(575) 678-2004 DSN 258-2004
> >> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> >> > 678-1230 DSN 258-1230
> >> > Email: john.w.raby2.civ at mail.mil
> >> >
> >> >
> >> > CLASSIFICATION: UNCLASSIFIED
> >> >
> >> >
> >>
> >> --
> >> Julie Prestopnik (she/her)
> >> Software Engineer
> >> National Center for Atmospheric Research Research Applications
> >> Laboratory
> >> Email: jpresto at ucar.edu
> >>
> >> My working day may not be your working day. Please do not feel
> >> obliged to reply to this email outside of your normal working
hours.
> >>
> >> CLASSIFICATION: UNCLASSIFIED
> >>
> >>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed May 12 15:28:05 2021
CLASSIFICATION: UNCLASSIFIED
John -
My obs_window setting was much more restrictive:
obs_window = {
beg = -1200;
end = 1200;
I guess this setting alone would have reduced the number of matched
pairs from point-stat compared to those plotted by plot-point-obs. In
my case, the numbers of matched pairs was 1439 and the number of obs
plotted was 1725.
Thanks for sharing those details about the differences between point-
stat and plot-point-obs. Good info to retain for the future.
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, May 12, 2021 3:11 PM
To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
plot_point_obs (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
I did also open up the obs_window setting in the Point-Stat config
file by adding a few 0's to the default setting:
obs_window = { beg = -5400000; end = 5400000; }
I just wanted to make sure that Point-Stat used obs from ALL times in
the input file rather than subsetting by time. Plot-Point-Obs is NOT
filtering by time, so I didn't want Point-Stat to either.
Just all depends on the specifics of your data. In general, Plot-
Point-Obs is doing less filtering than Point-Stat.
John
On Wed, May 12, 2021 at 3:04 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> Hi John -
>
> Thanks for initiating the fix process for the duplication in the
> User's Guide.
>
> Thanks for creating the data to replicate my situation. I understand
> that it is expected that there are differences between the way
> point-stat filters the observations and decides how many are off the
> grid and the way that plot-point-obs does it. I was thinking that it
> had something to do with this, but thought maybe it would be worth
it
> to investigate this further. The point-stat config file I used has
the following settings:
>
> duplicate_flag = UNIQUE;
> obs_summary = NEAREST;
>
> My understanding is that the duplicate_flag setting of UNIQUE will
> select one ob in the event there are matching obs.
> Having the obs_summary = NEAREST, assures that only the ob
closest in
> time to the valid time of the forecast. So having these settings in
> tandem would have effectively worked the same as your exercise going
> the other way. So I went back and looked at the number of matched
> pairs from point-stat and the number of obs plotted by plot-point-
obs
> and they were different. Was there any other tweak you did to get
the
> numbers of matched pairs and the number of obs plotted to match?
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, May 12, 2021 11:22 AM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> <john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> And John, thanks for the heads up about the docs. I agree about the
> duplication and am fixing it via:
> Caution-Caution-https://github.com/dtcenter/MET/issues/1789
>
> Once the changes are accepted and the docs are rebuilt, they will be
> corrected here:
> Caution-Caution-https://met.readthedocs.io/en/latest/
>
> Thanks,
> John
>
> On Wed, May 12, 2021 at 10:48 AM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > John,
> >
> > Your question is, why do the number of observations reported as
> > being off the grid differ between point-stat and plot-point-obs?
> > You're finding that the number reported by point-stat is higher
than
> > the number reported by plot-point-obs... 2100 from point-stat and
> > 1600 from
> plot-point-obs.
> >
> > I did some testing and have an explanation.
> >
> > The point-stat and plot-point-obs tools just count things and
apply
> > filtering logic in different ways. I ran with some sample data
> > included in the MET release. Here's some log output from point-
stat
> > when verifying 2m
> > temperature:
> >
> > DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation
type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 557 matched pairs.
> > DEBUG 3: Number of matched pairs = 557
> > DEBUG 3: Observations processed = 87753
> > DEBUG 3: Rejected: station id = 0
> > DEBUG 3: Rejected: obs var name = 77648
> > DEBUG 3: Rejected: valid time = 0
> > DEBUG 3: Rejected: bad obs value = 0
> > DEBUG 3: Rejected: off the grid = 7571
> > DEBUG 3: Rejected: topography = 0
> > DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected:
quality
> > marker = 0
> > DEBUG 3: Rejected: message type = 59
> > DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad fcst
> > value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG 3:
Rejected:
> > bad climo stdev = 0
> > DEBUG 3: Rejected: mpr filter = 0
> > DEBUG 3: Rejected: duplicates = 306
> >
> > The order of the log messages matches the order in which the
> > filtering logic is applied. So point-stat processed 87753
individual
> > observations and discarded 77648 because they weren't TMP. The
next
> > non-zero thing is "off the grid". Of the 10,105 observations of
> > temperature, 7571 were not on my verification grid. Then
additional
> > filtering logic is applied to the remaining 2534 observations.
Some
> > of those are discarded for not being at 2 meters and others for
> > being the
> wrong message type.
> >
> > I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP
and
> > -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat counts
> > individual observations, plot-point-obs counts observation
LOCATIONS.
> > In this case, plot-point-obs discards 1636 locations for being off
> > the grid. Note that plot-point-obs only filters by message type
> > (ADPSFC) and variable name (TMP). It has not filtered by vertical
> > level. If one station reports 100 times in your point obs file,
that
> > location is only counted and plotted once. So a dot is plotted in
> > the output if there was at least one observation of ADPSFC TMP at
that location.
> >
> > Another exercise is going the other way... can you get point-stat
to
> > report the exact same number of matched pairs that are plotted by
> > plot-point-obs? I was eventually able to do so, but had to tweak
my
> > point-stat config file. Since each station can report multiple
> > times, I setup point-stat to only use the one closest to valid
time
> > of my forecast... that way multiple reports at 1 station result in
> > only 1 matched pair being created:
> > obs_summary = NEAREST;
> >
> > And with that, point-stat reported 557 matched pairs and
> > plot-point-obs plotted 557 locations:
> >
> > From point-stat...
> > DEBUG 3: Number of matched pairs = 557
> >
> > From plot-point-obs...
> > DEBUG 2: Finished plotting 557 locations.
> >
> > Does that all make sense?
> >
> > Thanks,
> > John HG
> >
> >
> >
> > On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <Caution-Caution-url:
> >> Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=9
> >> 9857 >
> >>
> >> CLASSIFICATION: UNCLASSIFIED
> >>
> >> Hi Julie -
> >>
> >> Thanks for passing on my request and for considering an update to
> >> the documentation.
> >>
> >> R/
> >> John
> >>
> >> -----Original Message-----
> >> From: Julie Prestopnik via RT <met_help at ucar.edu>
> >> Sent: Tuesday, May 11, 2021 5:05 PM
> >> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> >> <john.w.raby2.civ at mail.mil>
> >> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> >> plot_point_obs (UNCLASSIFIED)
> >>
> >> All active links contained in this email were disabled. Please
> >> verify the identity of the sender, and confirm the authenticity
of
> >> all links contained within the message prior to copying and
pasting
> >> the address to a Web browser.
> >>
> >>
> >>
> >>
> >> ----
> >>
> >> Hi John.
> >>
> >> I have requested that John HG take a look. Please allow a few
> >> business days for a response.
> >>
> >> Thanks for pointing out the duplicate information in the
documentation.
> >> It looks like they're not quite identical as 27.1.2 has a
“regrid”
> >> entry, but
> >> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there
and
> >> 27.1.3 is not. We'll get that figured out and will update the
> >> documentation appropriately.
> >>
> >> Julie
> >>
> >> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> >> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> >> > Queue: met_help
> >> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> >> > Owner: Nobody
> >> > Requestors: john.w.raby2.civ at mail.mil
> >> > Status: new
> >> > Ticket <Caution-Caution-Caution-url:
> >> > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display
> >> > .html?id=9
> >> > 9857 >
> >> >
> >> >
> >> > CLASSIFICATION: UNCLASSIFIED
> >> >
> >> > I ran point-stat with an input obs file containing point obs
for
> >> > a large domain and an input GRIB forecast file for a much
smaller
> >> > domain located in the middle of the larger domain. The point-
stat
> >> > log file showed that there were 2100 ADPSFC TMP/Z2 obs
designated
> >> > "off the grid". I assume this means that they fell outside the
> >> > smaller forecast domain. I then ran plot_point_obs using the
same
> >> > input obs file and used the argument -data_file to point to the
> >> > forecast file to produce the plot over the smaller domain. The
> >> > plot_point_obs log file shows that 1610 ADPSFC TMP/Z2 obs were
> >> > skipped off the grid. Why is there such a large difference in
the
> >> > determination of obs located outside the
> >> smaller domain?
> >> >
> >> > I noticed that the online MET User's Guide contains what
appears
> >> > to be duplicate sections describing the plot_point_obs
> >> > configuration
> file.
> >> > These sections are 27.1.2 and 27.1.3.
> >> > Caution-Caution-Caution-
https://met.readthedocs.io/en/latest/User
> >> > s_Guide/pl
> >> > otting.html
> >> >
> >> > Thanks.
> >> >
> >> > R/
> >> > John
> >> >
> >> > Mr. John W. Raby
> >> > U.S. Army Research Laboratory
> >> > White Sands Missile Range, NM 88002
> >> > Office:(575) 678-2004 DSN 258-2004
> >> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> >> > 678-1230 DSN 258-1230
> >> > Email: john.w.raby2.civ at mail.mil
> >> >
> >> >
> >> > CLASSIFICATION: UNCLASSIFIED
> >> >
> >> >
> >>
> >> --
> >> Julie Prestopnik (she/her)
> >> Software Engineer
> >> National Center for Atmospheric Research Research Applications
> >> Laboratory
> >> Email: jpresto at ucar.edu
> >>
> >> My working day may not be your working day. Please do not feel
> >> obliged to reply to this email outside of your normal working
hours.
> >>
> >> CLASSIFICATION: UNCLASSIFIED
> >>
> >>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed May 12 15:34:11 2021
John,
There was another met-help question today about plot_point_obs. Since
you're also asking about it, I'll forward my response about a major
overhaul we've made to it for MET version 10.0.0, released on Monday.
...
We did indeed add [filtering by quality mark] as part of a major
overhaul
to the plot_point_obs tool for MET version 10.0.0, released on Monday.
Here's a link to the line in the plot_point_obs config file that
supports
that filtering:
https://github.com/dtcenter/MET/blob/3becf7ef698862a1ce5d52640b2787e49e290e84/met/data/config/PlotPointObsConfig_default#L42
In MET 10.0.0, you can really customize the plot_point_obs output a
ton.
For example, you can plot a field of data and then overlay dots of
various
sizes and colors. Here's an example of an image that's generated as
part of
our unit tests:
[image: Screen Shot 2021-05-12 at 1.37.56 PM.png]
The big blue dots are from surface stations and the little blue dots
are
the location of satellite obs. Here's the plot_point_obs config file
which
produces this output:
https://github.com/dtcenter/MET/blob/main_v10.0/test/config/PlotPointObsConfig
Note the 3 entries in the "point_data" array which define different
plotting options for each filtered subset of point observations.
John
On Wed, May 12, 2021 at 3:28 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> My obs_window setting was much more restrictive:
>
> obs_window = {
> beg = -1200;
> end = 1200;
>
> I guess this setting alone would have reduced the number of matched
pairs
> from point-stat compared to those plotted by plot-point-obs. In my
case,
> the numbers of matched pairs was 1439 and the number of obs plotted
was
> 1725.
>
> Thanks for sharing those details about the differences between
point-stat
> and plot-point-obs. Good info to retain for the future.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, May 12, 2021 3:11 PM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I did also open up the obs_window setting in the Point-Stat config
file by
> adding a few 0's to the default setting:
> obs_window = { beg = -5400000; end = 5400000; }
>
> I just wanted to make sure that Point-Stat used obs from ALL times
in the
> input file rather than subsetting by time. Plot-Point-Obs is NOT
filtering
> by time, so I didn't want Point-Stat to either.
>
> Just all depends on the specifics of your data. In general, Plot-
Point-Obs
> is doing less filtering than Point-Stat.
>
> John
>
> On Wed, May 12, 2021 at 3:04 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > Hi John -
> >
> > Thanks for initiating the fix process for the duplication in the
> > User's Guide.
> >
> > Thanks for creating the data to replicate my situation. I
understand
> > that it is expected that there are differences between the way
> > point-stat filters the observations and decides how many are off
the
> > grid and the way that plot-point-obs does it. I was thinking that
it
> > had something to do with this, but thought maybe it would be worth
it
> > to investigate this further. The point-stat config file I used has
the
> following settings:
> >
> > duplicate_flag = UNIQUE;
> > obs_summary = NEAREST;
> >
> > My understanding is that the duplicate_flag setting of UNIQUE will
> > select one ob in the event there are matching obs.
> > Having the obs_summary = NEAREST, assures that only the ob
closest in
> > time to the valid time of the forecast. So having these settings
in
> > tandem would have effectively worked the same as your exercise
going
> > the other way. So I went back and looked at the number of matched
> > pairs from point-stat and the number of obs plotted by plot-point-
obs
> > and they were different. Was there any other tweak you did to get
the
> > numbers of matched pairs and the number of obs plotted to match?
> >
> > R/
> > John
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, May 12, 2021 11:22 AM
> > To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > <john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > plot_point_obs (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > And John, thanks for the heads up about the docs. I agree about
the
> > duplication and am fixing it via:
> > Caution-Caution-https://github.com/dtcenter/MET/issues/1789
> >
> > Once the changes are accepted and the docs are rebuilt, they will
be
> > corrected here:
> > Caution-Caution-https://met.readthedocs.io/en/latest/
> >
> > Thanks,
> > John
> >
> > On Wed, May 12, 2021 at 10:48 AM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> >
> > > John,
> > >
> > > Your question is, why do the number of observations reported as
> > > being off the grid differ between point-stat and plot-point-obs?
> > > You're finding that the number reported by point-stat is higher
than
> > > the number reported by plot-point-obs... 2100 from point-stat
and
> > > 1600 from
> > plot-point-obs.
> > >
> > > I did some testing and have an explanation.
> > >
> > > The point-stat and plot-point-obs tools just count things and
apply
> > > filtering logic in different ways. I ran with some sample data
> > > included in the MET release. Here's some log output from point-
stat
> > > when verifying 2m
> > > temperature:
> > >
> > > DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation
type
> > > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > > 557 matched pairs.
> > > DEBUG 3: Number of matched pairs = 557
> > > DEBUG 3: Observations processed = 87753
> > > DEBUG 3: Rejected: station id = 0
> > > DEBUG 3: Rejected: obs var name = 77648
> > > DEBUG 3: Rejected: valid time = 0
> > > DEBUG 3: Rejected: bad obs value = 0
> > > DEBUG 3: Rejected: off the grid = 7571
> > > DEBUG 3: Rejected: topography = 0
> > > DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected:
quality
> > > marker = 0
> > > DEBUG 3: Rejected: message type = 59
> > > DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad
fcst
> > > value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG 3:
Rejected:
> > > bad climo stdev = 0
> > > DEBUG 3: Rejected: mpr filter = 0
> > > DEBUG 3: Rejected: duplicates = 306
> > >
> > > The order of the log messages matches the order in which the
> > > filtering logic is applied. So point-stat processed 87753
individual
> > > observations and discarded 77648 because they weren't TMP. The
next
> > > non-zero thing is "off the grid". Of the 10,105 observations of
> > > temperature, 7571 were not on my verification grid. Then
additional
> > > filtering logic is applied to the remaining 2534 observations.
Some
> > > of those are discarded for not being at 2 meters and others for
> > > being the
> > wrong message type.
> > >
> > > I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP
and
> > > -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat counts
> > > individual observations, plot-point-obs counts observation
LOCATIONS.
> > > In this case, plot-point-obs discards 1636 locations for being
off
> > > the grid. Note that plot-point-obs only filters by message type
> > > (ADPSFC) and variable name (TMP). It has not filtered by
vertical
> > > level. If one station reports 100 times in your point obs file,
that
> > > location is only counted and plotted once. So a dot is plotted
in
> > > the output if there was at least one observation of ADPSFC TMP
at that
> location.
> > >
> > > Another exercise is going the other way... can you get point-
stat to
> > > report the exact same number of matched pairs that are plotted
by
> > > plot-point-obs? I was eventually able to do so, but had to tweak
my
> > > point-stat config file. Since each station can report multiple
> > > times, I setup point-stat to only use the one closest to valid
time
> > > of my forecast... that way multiple reports at 1 station result
in
> > > only 1 matched pair being created:
> > > obs_summary = NEAREST;
> > >
> > > And with that, point-stat reported 557 matched pairs and
> > > plot-point-obs plotted 557 locations:
> > >
> > > From point-stat...
> > > DEBUG 3: Number of matched pairs = 557
> > >
> > > From plot-point-obs...
> > > DEBUG 2: Finished plotting 557 locations.
> > >
> > > Does that all make sense?
> > >
> > > Thanks,
> > > John HG
> > >
> > >
> > >
> > > On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <Caution-Caution-url:
> > >> Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=9
> > >> 9857 >
> > >>
> > >> CLASSIFICATION: UNCLASSIFIED
> > >>
> > >> Hi Julie -
> > >>
> > >> Thanks for passing on my request and for considering an update
to
> > >> the documentation.
> > >>
> > >> R/
> > >> John
> > >>
> > >> -----Original Message-----
> > >> From: Julie Prestopnik via RT <met_help at ucar.edu>
> > >> Sent: Tuesday, May 11, 2021 5:05 PM
> > >> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > >> <john.w.raby2.civ at mail.mil>
> > >> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> > >> plot_point_obs (UNCLASSIFIED)
> > >>
> > >> All active links contained in this email were disabled. Please
> > >> verify the identity of the sender, and confirm the authenticity
of
> > >> all links contained within the message prior to copying and
pasting
> > >> the address to a Web browser.
> > >>
> > >>
> > >>
> > >>
> > >> ----
> > >>
> > >> Hi John.
> > >>
> > >> I have requested that John HG take a look. Please allow a few
> > >> business days for a response.
> > >>
> > >> Thanks for pointing out the duplicate information in the
> documentation.
> > >> It looks like they're not quite identical as 27.1.2 has a
“regrid”
> > >> entry, but
> > >> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there
and
> > >> 27.1.3 is not. We'll get that figured out and will update the
> > >> documentation appropriately.
> > >>
> > >> Julie
> > >>
> > >> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> > >> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > >> > Queue: met_help
> > >> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> > >> > Owner: Nobody
> > >> > Requestors: john.w.raby2.civ at mail.mil
> > >> > Status: new
> > >> > Ticket <Caution-Caution-Caution-url:
> > >> > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display
> > >> > .html?id=9
> > >> > 9857 >
> > >> >
> > >> >
> > >> > CLASSIFICATION: UNCLASSIFIED
> > >> >
> > >> > I ran point-stat with an input obs file containing point obs
for
> > >> > a large domain and an input GRIB forecast file for a much
smaller
> > >> > domain located in the middle of the larger domain. The point-
stat
> > >> > log file showed that there were 2100 ADPSFC TMP/Z2 obs
designated
> > >> > "off the grid". I assume this means that they fell outside
the
> > >> > smaller forecast domain. I then ran plot_point_obs using the
same
> > >> > input obs file and used the argument -data_file to point to
the
> > >> > forecast file to produce the plot over the smaller domain.
The
> > >> > plot_point_obs log file shows that 1610 ADPSFC TMP/Z2 obs
were
> > >> > skipped off the grid. Why is there such a large difference in
the
> > >> > determination of obs located outside the
> > >> smaller domain?
> > >> >
> > >> > I noticed that the online MET User's Guide contains what
appears
> > >> > to be duplicate sections describing the plot_point_obs
> > >> > configuration
> > file.
> > >> > These sections are 27.1.2 and 27.1.3.
> > >> > Caution-Caution-Caution-
https://met.readthedocs.io/en/latest/User
> > >> > s_Guide/pl
> > >> > otting.html
> > >> >
> > >> > Thanks.
> > >> >
> > >> > R/
> > >> > John
> > >> >
> > >> > Mr. John W. Raby
> > >> > U.S. Army Research Laboratory
> > >> > White Sands Missile Range, NM 88002
> > >> > Office:(575) 678-2004 DSN 258-2004
> > >> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > >> > 678-1230 DSN 258-1230
> > >> > Email: john.w.raby2.civ at mail.mil
> > >> >
> > >> >
> > >> > CLASSIFICATION: UNCLASSIFIED
> > >> >
> > >> >
> > >>
> > >> --
> > >> Julie Prestopnik (she/her)
> > >> Software Engineer
> > >> National Center for Atmospheric Research Research Applications
> > >> Laboratory
> > >> Email: jpresto at ucar.edu
> > >>
> > >> My working day may not be your working day. Please do not feel
> > >> obliged to reply to this email outside of your normal working
hours.
> > >>
> > >> CLASSIFICATION: UNCLASSIFIED
> > >>
> > >>
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed May 12 16:00:58 2021
CLASSIFICATION: UNCLASSIFIED
John -
You aren’t kidding about the overhaul! Impressive. You know, as a
matter of fact, I did notice the change in plot-point-obs by way of
the testing I did yesterday for my project. I was using the container
image for MET V9.1 for running plot-point-obs and had used the online
User's Guide as the basis for setting the arguments in the run
command. I discovered that there were certain arguments such as
-plot_grid name and obs_var name which were rejected outright or
ignored, and I thought it curious that plot-point-obs for V9.1 would
have been that much different from V10.0. I had to drop back to my PDF
copy of the V9.0 User's Guide to make sure I was using the right
arguments. Another thing I noticed was the possibility of using a
config file. I searched for a config file in the MET distribution for
V9.0 and didn't find one, so I realized that the V10.0 version was
indeed significantly different from the V9.0 version.
Are online versions of older User's Guides available? I find that
searches in the online guides are much quicker than in the PDFs.
Do you have the container image available for MET V10.0? I would like
to copy it to my HPC and convert it to a Singularity container for my
use as I've done before with previous versions of MET.
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, May 12, 2021 3:34 PM
To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
plot_point_obs (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
There was another met-help question today about plot_point_obs. Since
you're also asking about it, I'll forward my response about a major
overhaul we've made to it for MET version 10.0.0, released on Monday.
...
We did indeed add [filtering by quality mark] as part of a major
overhaul to the plot_point_obs tool for MET version 10.0.0, released
on Monday.
Here's a link to the line in the plot_point_obs config file that
supports that filtering:
Caution-
https://github.com/dtcenter/MET/blob/3becf7ef698862a1ce5d52640b2787e49e290e84/met/data/config/PlotPointObsConfig_default#L42
In MET 10.0.0, you can really customize the plot_point_obs output a
ton.
For example, you can plot a field of data and then overlay dots of
various sizes and colors. Here's an example of an image that's
generated as part of our unit tests:
[image: Screen Shot 2021-05-12 at 1.37.56 PM.png]
The big blue dots are from surface stations and the little blue dots
are the location of satellite obs. Here's the plot_point_obs config
file which produces this output:
Caution-
https://github.com/dtcenter/MET/blob/main_v10.0/test/config/PlotPointObsConfig
Note the 3 entries in the "point_data" array which define different
plotting options for each filtered subset of point observations.
John
On Wed, May 12, 2021 at 3:28 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> My obs_window setting was much more restrictive:
>
> obs_window = {
> beg = -1200;
> end = 1200;
>
> I guess this setting alone would have reduced the number of matched
> pairs from point-stat compared to those plotted by plot-point-obs.
In
> my case, the numbers of matched pairs was 1439 and the number of obs
> plotted was 1725.
>
> Thanks for sharing those details about the differences between
> point-stat and plot-point-obs. Good info to retain for the future.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, May 12, 2021 3:11 PM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> <john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> I did also open up the obs_window setting in the Point-Stat config
> file by adding a few 0's to the default setting:
> obs_window = { beg = -5400000; end = 5400000; }
>
> I just wanted to make sure that Point-Stat used obs from ALL times
in
> the input file rather than subsetting by time. Plot-Point-Obs is NOT
> filtering by time, so I didn't want Point-Stat to either.
>
> Just all depends on the specifics of your data. In general,
> Plot-Point-Obs is doing less filtering than Point-Stat.
>
> John
>
> On Wed, May 12, 2021 at 3:04 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99
> > 857 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > Hi John -
> >
> > Thanks for initiating the fix process for the duplication in the
> > User's Guide.
> >
> > Thanks for creating the data to replicate my situation. I
understand
> > that it is expected that there are differences between the way
> > point-stat filters the observations and decides how many are off
the
> > grid and the way that plot-point-obs does it. I was thinking that
it
> > had something to do with this, but thought maybe it would be worth
> > it to investigate this further. The point-stat config file I used
> > has the
> following settings:
> >
> > duplicate_flag = UNIQUE;
> > obs_summary = NEAREST;
> >
> > My understanding is that the duplicate_flag setting of UNIQUE will
> > select one ob in the event there are matching obs.
> > Having the obs_summary = NEAREST, assures that only the ob
closest in
> > time to the valid time of the forecast. So having these settings
in
> > tandem would have effectively worked the same as your exercise
going
> > the other way. So I went back and looked at the number of matched
> > pairs from point-stat and the number of obs plotted by
> > plot-point-obs and they were different. Was there any other tweak
> > you did to get the numbers of matched pairs and the number of obs
plotted to match?
> >
> > R/
> > John
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, May 12, 2021 11:22 AM
> > To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > <john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > plot_point_obs (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
> > verify the identity of the sender, and confirm the authenticity of
> > all links contained within the message prior to copying and
pasting
> > the address to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > And John, thanks for the heads up about the docs. I agree about
the
> > duplication and am fixing it via:
> > Caution-Caution-Caution-
https://github.com/dtcenter/MET/issues/1789
> >
> > Once the changes are accepted and the docs are rebuilt, they will
be
> > corrected here:
> > Caution-Caution-Caution-https://met.readthedocs.io/en/latest/
> >
> > Thanks,
> > John
> >
> > On Wed, May 12, 2021 at 10:48 AM John Halley Gotway
> > <johnhg at ucar.edu>
> > wrote:
> >
> > > John,
> > >
> > > Your question is, why do the number of observations reported as
> > > being off the grid differ between point-stat and plot-point-obs?
> > > You're finding that the number reported by point-stat is higher
> > > than the number reported by plot-point-obs... 2100 from point-
stat
> > > and
> > > 1600 from
> > plot-point-obs.
> > >
> > > I did some testing and have an explanation.
> > >
> > > The point-stat and plot-point-obs tools just count things and
> > > apply filtering logic in different ways. I ran with some sample
> > > data included in the MET release. Here's some log output from
> > > point-stat when verifying 2m
> > > temperature:
> > >
> > > DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation
> > > type ADPSFC, over region FULL, for interpolation method
> > > NEAREST(1), using
> > > 557 matched pairs.
> > > DEBUG 3: Number of matched pairs = 557
> > > DEBUG 3: Observations processed = 87753
> > > DEBUG 3: Rejected: station id = 0
> > > DEBUG 3: Rejected: obs var name = 77648
> > > DEBUG 3: Rejected: valid time = 0
> > > DEBUG 3: Rejected: bad obs value = 0
> > > DEBUG 3: Rejected: off the grid = 7571
> > > DEBUG 3: Rejected: topography = 0
> > > DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected:
> > > quality marker = 0
> > > DEBUG 3: Rejected: message type = 59
> > > DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad
fcst
> > > value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG 3:
Rejected:
> > > bad climo stdev = 0
> > > DEBUG 3: Rejected: mpr filter = 0
> > > DEBUG 3: Rejected: duplicates = 306
> > >
> > > The order of the log messages matches the order in which the
> > > filtering logic is applied. So point-stat processed 87753
> > > individual observations and discarded 77648 because they weren't
> > > TMP. The next non-zero thing is "off the grid". Of the 10,105
> > > observations of temperature, 7571 were not on my verification
> > > grid. Then additional filtering logic is applied to the
remaining
> > > 2534 observations. Some of those are discarded for not being at
2
> > > meters and others for being the
> > wrong message type.
> > >
> > > I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var TMP
> > > and -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat
counts
> > > individual observations, plot-point-obs counts observation
LOCATIONS.
> > > In this case, plot-point-obs discards 1636 locations for being
off
> > > the grid. Note that plot-point-obs only filters by message type
> > > (ADPSFC) and variable name (TMP). It has not filtered by
vertical
> > > level. If one station reports 100 times in your point obs file,
> > > that location is only counted and plotted once. So a dot is
> > > plotted in the output if there was at least one observation of
> > > ADPSFC TMP at that
> location.
> > >
> > > Another exercise is going the other way... can you get point-
stat
> > > to report the exact same number of matched pairs that are
plotted
> > > by plot-point-obs? I was eventually able to do so, but had to
> > > tweak my point-stat config file. Since each station can report
> > > multiple times, I setup point-stat to only use the one closest
to
> > > valid time of my forecast... that way multiple reports at 1
> > > station result in only 1 matched pair being created:
> > > obs_summary = NEAREST;
> > >
> > > And with that, point-stat reported 557 matched pairs and
> > > plot-point-obs plotted 557 locations:
> > >
> > > From point-stat...
> > > DEBUG 3: Number of matched pairs = 557
> > >
> > > From plot-point-obs...
> > > DEBUG 2: Finished plotting 557 locations.
> > >
> > > Does that all make sense?
> > >
> > > Thanks,
> > > John HG
> > >
> > >
> > >
> > > On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <Caution-Caution-Caution-url:
> > >> Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display
> > >> .html?id=9
> > >> 9857 >
> > >>
> > >> CLASSIFICATION: UNCLASSIFIED
> > >>
> > >> Hi Julie -
> > >>
> > >> Thanks for passing on my request and for considering an update
to
> > >> the documentation.
> > >>
> > >> R/
> > >> John
> > >>
> > >> -----Original Message-----
> > >> From: Julie Prestopnik via RT <met_help at ucar.edu>
> > >> Sent: Tuesday, May 11, 2021 5:05 PM
> > >> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > >> <john.w.raby2.civ at mail.mil>
> > >> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> > >> plot_point_obs (UNCLASSIFIED)
> > >>
> > >> All active links contained in this email were disabled. Please
> > >> verify the identity of the sender, and confirm the authenticity
> > >> of all links contained within the message prior to copying and
> > >> pasting the address to a Web browser.
> > >>
> > >>
> > >>
> > >>
> > >> ----
> > >>
> > >> Hi John.
> > >>
> > >> I have requested that John HG take a look. Please allow a few
> > >> business days for a response.
> > >>
> > >> Thanks for pointing out the duplicate information in the
> documentation.
> > >> It looks like they're not quite identical as 27.1.2 has a
“regrid”
> > >> entry, but
> > >> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be there
> > >> and
> > >> 27.1.3 is not. We'll get that figured out and will update the
> > >> documentation appropriately.
> > >>
> > >> Julie
> > >>
> > >> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> > >> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > >> > Queue: met_help
> > >> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> > >> > Owner: Nobody
> > >> > Requestors: john.w.raby2.civ at mail.mil
> > >> > Status: new
> > >> > Ticket <Caution-Caution-Caution-Caution-url:
> > >> > Caution-Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Tick
> > >> > et/Display
> > >> > .html?id=9
> > >> > 9857 >
> > >> >
> > >> >
> > >> > CLASSIFICATION: UNCLASSIFIED
> > >> >
> > >> > I ran point-stat with an input obs file containing point obs
> > >> > for a large domain and an input GRIB forecast file for a much
> > >> > smaller domain located in the middle of the larger domain.
The
> > >> > point-stat log file showed that there were 2100 ADPSFC TMP/Z2
> > >> > obs designated "off the grid". I assume this means that they
> > >> > fell outside the smaller forecast domain. I then ran
> > >> > plot_point_obs using the same input obs file and used the
> > >> > argument -data_file to point to the forecast file to produce
> > >> > the plot over the smaller domain. The plot_point_obs log file
> > >> > shows that 1610 ADPSFC TMP/Z2 obs were skipped off the grid.
> > >> > Why is there such a large difference in the determination of
> > >> > obs located outside the
> > >> smaller domain?
> > >> >
> > >> > I noticed that the online MET User's Guide contains what
> > >> > appears to be duplicate sections describing the
plot_point_obs
> > >> > configuration
> > file.
> > >> > These sections are 27.1.2 and 27.1.3.
> > >> > Caution-Caution-Caution-Caution-
https://met.readthedocs.io/en/l
> > >> > atest/User
> > >> > s_Guide/pl
> > >> > otting.html
> > >> >
> > >> > Thanks.
> > >> >
> > >> > R/
> > >> > John
> > >> >
> > >> > Mr. John W. Raby
> > >> > U.S. Army Research Laboratory
> > >> > White Sands Missile Range, NM 88002
> > >> > Office:(575) 678-2004 DSN 258-2004
> > >> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
> > >> > (575)
> > >> > 678-1230 DSN 258-1230
> > >> > Email: john.w.raby2.civ at mail.mil
> > >> >
> > >> >
> > >> > CLASSIFICATION: UNCLASSIFIED
> > >> >
> > >> >
> > >>
> > >> --
> > >> Julie Prestopnik (she/her)
> > >> Software Engineer
> > >> National Center for Atmospheric Research Research Applications
> > >> Laboratory
> > >> Email: jpresto at ucar.edu
> > >>
> > >> My working day may not be your working day. Please do not feel
> > >> obliged to reply to this email outside of your normal working
hours.
> > >>
> > >> CLASSIFICATION: UNCLASSIFIED
> > >>
> > >>
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed May 12 17:13:02 2021
John,
Older versions of the user's guide can be found here:
https://dtcenter.org/community-code/model-evaluation-tools-
met/documentation
Just click on "Previous Versions".
Yes, whenever we tag new versions of the code, a new image is
automatically
built on DockerHub. Here's the list of them:
https://hub.docker.com/repository/registry-
1.docker.io/dtcenter/met/tags?page=1&ordering=last_updated
This command will retrieve version 10.0.0:
docker pull dtcenter/met:10.0.0
John
On Wed, May 12, 2021 at 4:01 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> You aren’t kidding about the overhaul! Impressive. You know, as a
matter
> of fact, I did notice the change in plot-point-obs by way of the
testing I
> did yesterday for my project. I was using the container image for
MET V9.1
> for running plot-point-obs and had used the online User's Guide as
the
> basis for setting the arguments in the run command. I discovered
that there
> were certain arguments such as -plot_grid name and obs_var name
which were
> rejected outright or ignored, and I thought it curious that plot-
point-obs
> for V9.1 would have been that much different from V10.0. I had to
drop back
> to my PDF copy of the V9.0 User's Guide to make sure I was using the
right
> arguments. Another thing I noticed was the possibility of using a
config
> file. I searched for a config file in the MET distribution for V9.0
and
> didn't find one, so I realized that the V10.0 version was indeed
> significantly different from the V9.0 version.
>
> Are online versions of older User's Guides available? I find that
searches
> in the online guides are much quicker than in the PDFs.
>
> Do you have the container image available for MET V10.0? I would
like to
> copy it to my HPC and convert it to a Singularity container for my
use as
> I've done before with previous versions of MET.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, May 12, 2021 3:34 PM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> There was another met-help question today about plot_point_obs.
Since
> you're also asking about it, I'll forward my response about a major
> overhaul we've made to it for MET version 10.0.0, released on
Monday.
>
> ...
>
> We did indeed add [filtering by quality mark] as part of a major
overhaul
> to the plot_point_obs tool for MET version 10.0.0, released on
Monday.
>
> Here's a link to the line in the plot_point_obs config file that
supports
> that filtering:
> Caution-
>
https://github.com/dtcenter/MET/blob/3becf7ef698862a1ce5d52640b2787e49e290e84/met/data/config/PlotPointObsConfig_default#L42
>
> In MET 10.0.0, you can really customize the plot_point_obs output a
ton.
> For example, you can plot a field of data and then overlay dots of
various
> sizes and colors. Here's an example of an image that's generated as
part of
> our unit tests:
>
> [image: Screen Shot 2021-05-12 at 1.37.56 PM.png]
>
> The big blue dots are from surface stations and the little blue dots
are
> the location of satellite obs. Here's the plot_point_obs config file
which
> produces this output:
> Caution-
>
https://github.com/dtcenter/MET/blob/main_v10.0/test/config/PlotPointObsConfig
>
> Note the 3 entries in the "point_data" array which define different
> plotting options for each filtered subset of point observations.
>
> John
>
>
>
> On Wed, May 12, 2021 at 3:28 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > My obs_window setting was much more restrictive:
> >
> > obs_window = {
> > beg = -1200;
> > end = 1200;
> >
> > I guess this setting alone would have reduced the number of
matched
> > pairs from point-stat compared to those plotted by plot-point-obs.
In
> > my case, the numbers of matched pairs was 1439 and the number of
obs
> > plotted was 1725.
> >
> > Thanks for sharing those details about the differences between
> > point-stat and plot-point-obs. Good info to retain for the future.
> >
> > R/
> > John
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, May 12, 2021 3:11 PM
> > To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > <john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > plot_point_obs (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > I did also open up the obs_window setting in the Point-Stat config
> > file by adding a few 0's to the default setting:
> > obs_window = { beg = -5400000; end = 5400000; }
> >
> > I just wanted to make sure that Point-Stat used obs from ALL times
in
> > the input file rather than subsetting by time. Plot-Point-Obs is
NOT
> > filtering by time, so I didn't want Point-Stat to either.
> >
> > Just all depends on the specifics of your data. In general,
> > Plot-Point-Obs is doing less filtering than Point-Stat.
> >
> > John
> >
> > On Wed, May 12, 2021 at 3:04 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url:
> > > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99
> > > 857 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > Hi John -
> > >
> > > Thanks for initiating the fix process for the duplication in the
> > > User's Guide.
> > >
> > > Thanks for creating the data to replicate my situation. I
understand
> > > that it is expected that there are differences between the way
> > > point-stat filters the observations and decides how many are off
the
> > > grid and the way that plot-point-obs does it. I was thinking
that it
> > > had something to do with this, but thought maybe it would be
worth
> > > it to investigate this further. The point-stat config file I
used
> > > has the
> > following settings:
> > >
> > > duplicate_flag = UNIQUE;
> > > obs_summary = NEAREST;
> > >
> > > My understanding is that the duplicate_flag setting of UNIQUE
will
> > > select one ob in the event there are matching obs.
> > > Having the obs_summary = NEAREST, assures that only the ob
closest
> in
> > > time to the valid time of the forecast. So having these settings
in
> > > tandem would have effectively worked the same as your exercise
going
> > > the other way. So I went back and looked at the number of
matched
> > > pairs from point-stat and the number of obs plotted by
> > > plot-point-obs and they were different. Was there any other
tweak
> > > you did to get the numbers of matched pairs and the number of
obs
> plotted to match?
> > >
> > > R/
> > > John
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Wednesday, May 12, 2021 11:22 AM
> > > To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > > <john.w.raby2.civ at mail.mil>
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > > plot_point_obs (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify the identity of the sender, and confirm the authenticity
of
> > > all links contained within the message prior to copying and
pasting
> > > the address to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > And John, thanks for the heads up about the docs. I agree about
the
> > > duplication and am fixing it via:
> > > Caution-Caution-Caution-
https://github.com/dtcenter/MET/issues/1789
> > >
> > > Once the changes are accepted and the docs are rebuilt, they
will be
> > > corrected here:
> > > Caution-Caution-Caution-https://met.readthedocs.io/en/latest/
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, May 12, 2021 at 10:48 AM John Halley Gotway
> > > <johnhg at ucar.edu>
> > > wrote:
> > >
> > > > John,
> > > >
> > > > Your question is, why do the number of observations reported
as
> > > > being off the grid differ between point-stat and plot-point-
obs?
> > > > You're finding that the number reported by point-stat is
higher
> > > > than the number reported by plot-point-obs... 2100 from point-
stat
> > > > and
> > > > 1600 from
> > > plot-point-obs.
> > > >
> > > > I did some testing and have an explanation.
> > > >
> > > > The point-stat and plot-point-obs tools just count things and
> > > > apply filtering logic in different ways. I ran with some
sample
> > > > data included in the MET release. Here's some log output from
> > > > point-stat when verifying 2m
> > > > temperature:
> > > >
> > > > DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation
> > > > type ADPSFC, over region FULL, for interpolation method
> > > > NEAREST(1), using
> > > > 557 matched pairs.
> > > > DEBUG 3: Number of matched pairs = 557
> > > > DEBUG 3: Observations processed = 87753
> > > > DEBUG 3: Rejected: station id = 0
> > > > DEBUG 3: Rejected: obs var name = 77648
> > > > DEBUG 3: Rejected: valid time = 0
> > > > DEBUG 3: Rejected: bad obs value = 0
> > > > DEBUG 3: Rejected: off the grid = 7571
> > > > DEBUG 3: Rejected: topography = 0
> > > > DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected:
> > > > quality marker = 0
> > > > DEBUG 3: Rejected: message type = 59
> > > > DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad
fcst
> > > > value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG 3:
Rejected:
> > > > bad climo stdev = 0
> > > > DEBUG 3: Rejected: mpr filter = 0
> > > > DEBUG 3: Rejected: duplicates = 306
> > > >
> > > > The order of the log messages matches the order in which the
> > > > filtering logic is applied. So point-stat processed 87753
> > > > individual observations and discarded 77648 because they
weren't
> > > > TMP. The next non-zero thing is "off the grid". Of the 10,105
> > > > observations of temperature, 7571 were not on my verification
> > > > grid. Then additional filtering logic is applied to the
remaining
> > > > 2534 observations. Some of those are discarded for not being
at 2
> > > > meters and others for being the
> > > wrong message type.
> > > >
> > > > I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var
TMP
> > > > and -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat
counts
> > > > individual observations, plot-point-obs counts observation
LOCATIONS.
> > > > In this case, plot-point-obs discards 1636 locations for being
off
> > > > the grid. Note that plot-point-obs only filters by message
type
> > > > (ADPSFC) and variable name (TMP). It has not filtered by
vertical
> > > > level. If one station reports 100 times in your point obs
file,
> > > > that location is only counted and plotted once. So a dot is
> > > > plotted in the output if there was at least one observation of
> > > > ADPSFC TMP at that
> > location.
> > > >
> > > > Another exercise is going the other way... can you get point-
stat
> > > > to report the exact same number of matched pairs that are
plotted
> > > > by plot-point-obs? I was eventually able to do so, but had to
> > > > tweak my point-stat config file. Since each station can report
> > > > multiple times, I setup point-stat to only use the one closest
to
> > > > valid time of my forecast... that way multiple reports at 1
> > > > station result in only 1 matched pair being created:
> > > > obs_summary = NEAREST;
> > > >
> > > > And with that, point-stat reported 557 matched pairs and
> > > > plot-point-obs plotted 557 locations:
> > > >
> > > > From point-stat...
> > > > DEBUG 3: Number of matched pairs = 557
> > > >
> > > > From plot-point-obs...
> > > > DEBUG 2: Finished plotting 557 locations.
> > > >
> > > > Does that all make sense?
> > > >
> > > > Thanks,
> > > > John HG
> > > >
> > > >
> > > >
> > > > On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <Caution-Caution-Caution-url:
> > > >> Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display
> > > >> .html?id=9
> > > >> 9857 >
> > > >>
> > > >> CLASSIFICATION: UNCLASSIFIED
> > > >>
> > > >> Hi Julie -
> > > >>
> > > >> Thanks for passing on my request and for considering an
update to
> > > >> the documentation.
> > > >>
> > > >> R/
> > > >> John
> > > >>
> > > >> -----Original Message-----
> > > >> From: Julie Prestopnik via RT <met_help at ucar.edu>
> > > >> Sent: Tuesday, May 11, 2021 5:05 PM
> > > >> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > > >> <john.w.raby2.civ at mail.mil>
> > > >> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > > >> plot_point_obs (UNCLASSIFIED)
> > > >>
> > > >> All active links contained in this email were disabled.
Please
> > > >> verify the identity of the sender, and confirm the
authenticity
> > > >> of all links contained within the message prior to copying
and
> > > >> pasting the address to a Web browser.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> ----
> > > >>
> > > >> Hi John.
> > > >>
> > > >> I have requested that John HG take a look. Please allow a
few
> > > >> business days for a response.
> > > >>
> > > >> Thanks for pointing out the duplicate information in the
> > documentation.
> > > >> It looks like they're not quite identical as 27.1.2 has a
“regrid”
> > > >> entry, but
> > > >> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be
there
> > > >> and
> > > >> 27.1.3 is not. We'll get that figured out and will update
the
> > > >> documentation appropriately.
> > > >>
> > > >> Julie
> > > >>
> > > >> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> > > >> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > >> > Queue: met_help
> > > >> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> > > >> > Owner: Nobody
> > > >> > Requestors: john.w.raby2.civ at mail.mil
> > > >> > Status: new
> > > >> > Ticket <Caution-Caution-Caution-Caution-url:
> > > >> > Caution-Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Tick
> > > >> > et/Display
> > > >> > .html?id=9
> > > >> > 9857 >
> > > >> >
> > > >> >
> > > >> > CLASSIFICATION: UNCLASSIFIED
> > > >> >
> > > >> > I ran point-stat with an input obs file containing point
obs
> > > >> > for a large domain and an input GRIB forecast file for a
much
> > > >> > smaller domain located in the middle of the larger domain.
The
> > > >> > point-stat log file showed that there were 2100 ADPSFC
TMP/Z2
> > > >> > obs designated "off the grid". I assume this means that
they
> > > >> > fell outside the smaller forecast domain. I then ran
> > > >> > plot_point_obs using the same input obs file and used the
> > > >> > argument -data_file to point to the forecast file to
produce
> > > >> > the plot over the smaller domain. The plot_point_obs log
file
> > > >> > shows that 1610 ADPSFC TMP/Z2 obs were skipped off the
grid.
> > > >> > Why is there such a large difference in the determination
of
> > > >> > obs located outside the
> > > >> smaller domain?
> > > >> >
> > > >> > I noticed that the online MET User's Guide contains what
> > > >> > appears to be duplicate sections describing the
plot_point_obs
> > > >> > configuration
> > > file.
> > > >> > These sections are 27.1.2 and 27.1.3.
> > > >> > Caution-Caution-Caution-Caution-
https://met.readthedocs.io/en/l
> > > >> > atest/User
> > > >> > s_Guide/pl
> > > >> > otting.html
> > > >> >
> > > >> > Thanks.
> > > >> >
> > > >> > R/
> > > >> > John
> > > >> >
> > > >> > Mr. John W. Raby
> > > >> > U.S. Army Research Laboratory
> > > >> > White Sands Missile Range, NM 88002
> > > >> > Office:(575) 678-2004 DSN 258-2004
> > > >> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
> > > >> > (575)
> > > >> > 678-1230 DSN 258-1230
> > > >> > Email: john.w.raby2.civ at mail.mil
> > > >> >
> > > >> >
> > > >> > CLASSIFICATION: UNCLASSIFIED
> > > >> >
> > > >> >
> > > >>
> > > >> --
> > > >> Julie Prestopnik (she/her)
> > > >> Software Engineer
> > > >> National Center for Atmospheric Research Research
Applications
> > > >> Laboratory
> > > >> Email: jpresto at ucar.edu
> > > >>
> > > >> My working day may not be your working day. Please do not
feel
> > > >> obliged to reply to this email outside of your normal working
hours.
> > > >>
> > > >> CLASSIFICATION: UNCLASSIFIED
> > > >>
> > > >>
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu May 13 07:00:19 2021
CLASSIFICATION: UNCLASSIFIED
John -
Thanks for pointing the "Previous Versions" link out to access the
online versions. This is very helpful. Somehow, I missed this on
previous visits to this site.
Appreciate the info on the process for maintaining and distributing
the container images and for the pull command. I'll get and install
V10.0 today.
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, May 12, 2021 5:13 PM
To: Raby, John W CIV USARMY DEVCOM ARL (USA)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
plot_point_obs (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Older versions of the user's guide can be found here:
Caution-https://dtcenter.org/community-code/model-evaluation-tools-
met/documentation
Just click on "Previous Versions".
Yes, whenever we tag new versions of the code, a new image is
automatically built on DockerHub. Here's the list of them:
Caution-https://hub.docker.com/repository/registry-
1.docker.io/dtcenter/met/tags?page=1&ordering=last_updated
This command will retrieve version 10.0.0:
docker pull dtcenter/met:10.0.0
John
On Wed, May 12, 2021 at 4:01 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99857 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> You aren’t kidding about the overhaul! Impressive. You know, as a
> matter of fact, I did notice the change in plot-point-obs by way of
> the testing I did yesterday for my project. I was using the
container
> image for MET V9.1 for running plot-point-obs and had used the
online
> User's Guide as the basis for setting the arguments in the run
> command. I discovered that there were certain arguments such as
> -plot_grid name and obs_var name which were rejected outright or
> ignored, and I thought it curious that plot-point-obs for V9.1 would
> have been that much different from V10.0. I had to drop back to my
PDF
> copy of the V9.0 User's Guide to make sure I was using the right
> arguments. Another thing I noticed was the possibility of using a
> config file. I searched for a config file in the MET distribution
for
> V9.0 and didn't find one, so I realized that the V10.0 version was
indeed significantly different from the V9.0 version.
>
> Are online versions of older User's Guides available? I find that
> searches in the online guides are much quicker than in the PDFs.
>
> Do you have the container image available for MET V10.0? I would
like
> to copy it to my HPC and convert it to a Singularity container for
my
> use as I've done before with previous versions of MET.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, May 12, 2021 3:34 PM
> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> <john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET V9.0
> plot_point_obs (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> There was another met-help question today about plot_point_obs.
Since
> you're also asking about it, I'll forward my response about a major
> overhaul we've made to it for MET version 10.0.0, released on
Monday.
>
> ...
>
> We did indeed add [filtering by quality mark] as part of a major
> overhaul to the plot_point_obs tool for MET version 10.0.0, released
on Monday.
>
> Here's a link to the line in the plot_point_obs config file that
> supports that filtering:
> Caution-
> Caution-
https://github.com/dtcenter/MET/blob/3becf7ef698862a1ce5d52640
> b2787e49e290e84/met/data/config/PlotPointObsConfig_default#L42
>
> In MET 10.0.0, you can really customize the plot_point_obs output a
ton.
> For example, you can plot a field of data and then overlay dots of
> various sizes and colors. Here's an example of an image that's
> generated as part of our unit tests:
>
> [image: Screen Shot 2021-05-12 at 1.37.56 PM.png]
>
> The big blue dots are from surface stations and the little blue dots
> are the location of satellite obs. Here's the plot_point_obs config
> file which produces this output:
> Caution-
> Caution-
https://github.com/dtcenter/MET/blob/main_v10.0/test/config/Pl
> otPointObsConfig
>
> Note the 3 entries in the "point_data" array which define different
> plotting options for each filtered subset of point observations.
>
> John
>
>
>
> On Wed, May 12, 2021 at 3:28 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99
> > 857 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > My obs_window setting was much more restrictive:
> >
> > obs_window = {
> > beg = -1200;
> > end = 1200;
> >
> > I guess this setting alone would have reduced the number of
matched
> > pairs from point-stat compared to those plotted by plot-point-obs.
> > In my case, the numbers of matched pairs was 1439 and the number
of
> > obs plotted was 1725.
> >
> > Thanks for sharing those details about the differences between
> > point-stat and plot-point-obs. Good info to retain for the future.
> >
> > R/
> > John
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, May 12, 2021 3:11 PM
> > To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > <john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > plot_point_obs (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
> > verify the identity of the sender, and confirm the authenticity of
> > all links contained within the message prior to copying and
pasting
> > the address to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > I did also open up the obs_window setting in the Point-Stat config
> > file by adding a few 0's to the default setting:
> > obs_window = { beg = -5400000; end = 5400000; }
> >
> > I just wanted to make sure that Point-Stat used obs from ALL times
> > in the input file rather than subsetting by time. Plot-Point-Obs
is
> > NOT filtering by time, so I didn't want Point-Stat to either.
> >
> > Just all depends on the specifics of your data. In general,
> > Plot-Point-Obs is doing less filtering than Point-Stat.
> >
> > John
> >
> > On Wed, May 12, 2021 at 3:04 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url:
> > > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.
> > > html?id=99
> > > 857 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > Hi John -
> > >
> > > Thanks for initiating the fix process for the duplication in the
> > > User's Guide.
> > >
> > > Thanks for creating the data to replicate my situation. I
> > > understand that it is expected that there are differences
between
> > > the way point-stat filters the observations and decides how many
> > > are off the grid and the way that plot-point-obs does it. I was
> > > thinking that it had something to do with this, but thought
maybe
> > > it would be worth it to investigate this further. The point-stat
> > > config file I used has the
> > following settings:
> > >
> > > duplicate_flag = UNIQUE;
> > > obs_summary = NEAREST;
> > >
> > > My understanding is that the duplicate_flag setting of UNIQUE
will
> > > select one ob in the event there are matching obs.
> > > Having the obs_summary = NEAREST, assures that only the ob
closest
> in
> > > time to the valid time of the forecast. So having these settings
> > > in tandem would have effectively worked the same as your
exercise
> > > going the other way. So I went back and looked at the number of
> > > matched pairs from point-stat and the number of obs plotted by
> > > plot-point-obs and they were different. Was there any other
tweak
> > > you did to get the numbers of matched pairs and the number of
obs
> plotted to match?
> > >
> > > R/
> > > John
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Wednesday, May 12, 2021 11:22 AM
> > > To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > > <john.w.raby2.civ at mail.mil>
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
> > > V9.0 plot_point_obs (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify the identity of the sender, and confirm the authenticity
of
> > > all links contained within the message prior to copying and
> > > pasting the address to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > And John, thanks for the heads up about the docs. I agree about
> > > the duplication and am fixing it via:
> > > Caution-Caution-Caution-Caution-
https://github.com/dtcenter/MET/is
> > > sues/1789
> > >
> > > Once the changes are accepted and the docs are rebuilt, they
will
> > > be corrected here:
> > > Caution-Caution-Caution-Caution-
https://met.readthedocs.io/en/late
> > > st/
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, May 12, 2021 at 10:48 AM John Halley Gotway
> > > <johnhg at ucar.edu>
> > > wrote:
> > >
> > > > John,
> > > >
> > > > Your question is, why do the number of observations reported
as
> > > > being off the grid differ between point-stat and plot-point-
obs?
> > > > You're finding that the number reported by point-stat is
higher
> > > > than the number reported by plot-point-obs... 2100 from
> > > > point-stat and
> > > > 1600 from
> > > plot-point-obs.
> > > >
> > > > I did some testing and have an explanation.
> > > >
> > > > The point-stat and plot-point-obs tools just count things and
> > > > apply filtering logic in different ways. I ran with some
sample
> > > > data included in the MET release. Here's some log output from
> > > > point-stat when verifying 2m
> > > > temperature:
> > > >
> > > > DEBUG 2: Processing TMP_Z2(*,*) versus TMP/Z2, for observation
> > > > type ADPSFC, over region FULL, for interpolation method
> > > > NEAREST(1), using
> > > > 557 matched pairs.
> > > > DEBUG 3: Number of matched pairs = 557
> > > > DEBUG 3: Observations processed = 87753
> > > > DEBUG 3: Rejected: station id = 0
> > > > DEBUG 3: Rejected: obs var name = 77648
> > > > DEBUG 3: Rejected: valid time = 0
> > > > DEBUG 3: Rejected: bad obs value = 0
> > > > DEBUG 3: Rejected: off the grid = 7571
> > > > DEBUG 3: Rejected: topography = 0
> > > > DEBUG 3: Rejected: level mismatch = 1612 DEBUG 3: Rejected:
> > > > quality marker = 0
> > > > DEBUG 3: Rejected: message type = 59
> > > > DEBUG 3: Rejected: masking region = 0 DEBUG 3: Rejected: bad
> > > > fcst value = 0 DEBUG 3: Rejected: bad climo mean = 0 DEBUG
3: Rejected:
> > > > bad climo stdev = 0
> > > > DEBUG 3: Rejected: mpr filter = 0
> > > > DEBUG 3: Rejected: duplicates = 306
> > > >
> > > > The order of the log messages matches the order in which the
> > > > filtering logic is applied. So point-stat processed 87753
> > > > individual observations and discarded 77648 because they
weren't
> > > > TMP. The next non-zero thing is "off the grid". Of the 10,105
> > > > observations of temperature, 7571 were not on my verification
> > > > grid. Then additional filtering logic is applied to the
> > > > remaining
> > > > 2534 observations. Some of those are discarded for not being
at
> > > > 2 meters and others for being the
> > > wrong message type.
> > > >
> > > > I ran plot-point-obs using the -msg_typ ADPSFC and -obs_var
TMP
> > > > and -plot_grid TMP_Z2_DTC165.nc options. Whereas point-stat
> > > > counts individual observations, plot-point-obs counts
observation LOCATIONS.
> > > > In this case, plot-point-obs discards 1636 locations for being
> > > > off the grid. Note that plot-point-obs only filters by message
> > > > type
> > > > (ADPSFC) and variable name (TMP). It has not filtered by
> > > > vertical level. If one station reports 100 times in your point
> > > > obs file, that location is only counted and plotted once. So a
> > > > dot is plotted in the output if there was at least one
> > > > observation of ADPSFC TMP at that
> > location.
> > > >
> > > > Another exercise is going the other way... can you get
> > > > point-stat to report the exact same number of matched pairs
that
> > > > are plotted by plot-point-obs? I was eventually able to do so,
> > > > but had to tweak my point-stat config file. Since each station
> > > > can report multiple times, I setup point-stat to only use the
> > > > one closest to valid time of my forecast... that way multiple
> > > > reports at 1 station result in only 1 matched pair being
created:
> > > > obs_summary = NEAREST;
> > > >
> > > > And with that, point-stat reported 557 matched pairs and
> > > > plot-point-obs plotted 557 locations:
> > > >
> > > > From point-stat...
> > > > DEBUG 3: Number of matched pairs = 557
> > > >
> > > > From plot-point-obs...
> > > > DEBUG 2: Finished plotting 557 locations.
> > > >
> > > > Does that all make sense?
> > > >
> > > > Thanks,
> > > > John HG
> > > >
> > > >
> > > >
> > > > On Wed, May 12, 2021 at 7:11 AM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <Caution-Caution-Caution-Caution-url:
> > > >> Caution-Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Tick
> > > >> et/Display
> > > >> .html?id=9
> > > >> 9857 >
> > > >>
> > > >> CLASSIFICATION: UNCLASSIFIED
> > > >>
> > > >> Hi Julie -
> > > >>
> > > >> Thanks for passing on my request and for considering an
update
> > > >> to the documentation.
> > > >>
> > > >> R/
> > > >> John
> > > >>
> > > >> -----Original Message-----
> > > >> From: Julie Prestopnik via RT <met_help at ucar.edu>
> > > >> Sent: Tuesday, May 11, 2021 5:05 PM
> > > >> To: Raby, John W CIV USARMY DEVCOM ARL (USA)
> > > >> <john.w.raby2.civ at mail.mil>
> > > >> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99857] MET
V9.0
> > > >> plot_point_obs (UNCLASSIFIED)
> > > >>
> > > >> All active links contained in this email were disabled.
Please
> > > >> verify the identity of the sender, and confirm the
authenticity
> > > >> of all links contained within the message prior to copying
and
> > > >> pasting the address to a Web browser.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> ----
> > > >>
> > > >> Hi John.
> > > >>
> > > >> I have requested that John HG take a look. Please allow a
few
> > > >> business days for a response.
> > > >>
> > > >> Thanks for pointing out the duplicate information in the
> > documentation.
> > > >> It looks like they're not quite identical as 27.1.2 has a
“regrid”
> > > >> entry, but
> > > >> 27.1.3 does not, so I’m thinking 27.1.2 is supposed to be
there
> > > >> and
> > > >> 27.1.3 is not. We'll get that figured out and will update
the
> > > >> documentation appropriately.
> > > >>
> > > >> Julie
> > > >>
> > > >> On Tue, May 11, 2021 at 4:23 PM Raby, John W USA CIV via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > Tue May 11 16:23:21 2021: Request 99857 was acted upon.
> > > >> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > >> > Queue: met_help
> > > >> > Subject: MET V9.0 plot_point_obs (UNCLASSIFIED)
> > > >> > Owner: Nobody
> > > >> > Requestors: john.w.raby2.civ at mail.mil
> > > >> > Status: new
> > > >> > Ticket <Caution-Caution-Caution-Caution-Caution-url:
> > > >> > Caution-Caution-Caution-Caution-Caution-
https://rt.rap.ucar.e
> > > >> > du/rt/Tick
> > > >> > et/Display
> > > >> > .html?id=9
> > > >> > 9857 >
> > > >> >
> > > >> >
> > > >> > CLASSIFICATION: UNCLASSIFIED
> > > >> >
> > > >> > I ran point-stat with an input obs file containing point
obs
> > > >> > for a large domain and an input GRIB forecast file for a
much
> > > >> > smaller domain located in the middle of the larger domain.
> > > >> > The point-stat log file showed that there were 2100 ADPSFC
> > > >> > TMP/Z2 obs designated "off the grid". I assume this means
> > > >> > that they fell outside the smaller forecast domain. I then
> > > >> > ran plot_point_obs using the same input obs file and used
the
> > > >> > argument -data_file to point to the forecast file to
produce
> > > >> > the plot over the smaller domain. The plot_point_obs log
file
> > > >> > shows that 1610 ADPSFC TMP/Z2 obs were skipped off the
grid.
> > > >> > Why is there such a large difference in the determination
of
> > > >> > obs located outside the
> > > >> smaller domain?
> > > >> >
> > > >> > I noticed that the online MET User's Guide contains what
> > > >> > appears to be duplicate sections describing the
> > > >> > plot_point_obs configuration
> > > file.
> > > >> > These sections are 27.1.2 and 27.1.3.
> > > >> > Caution-Caution-Caution-Caution-Caution-
https://met.readthedo
> > > >> > cs.io/en/l
> > > >> > atest/User
> > > >> > s_Guide/pl
> > > >> > otting.html
> > > >> >
> > > >> > Thanks.
> > > >> >
> > > >> > R/
> > > >> > John
> > > >> >
> > > >> > Mr. John W. Raby
> > > >> > U.S. Army Research Laboratory White Sands Missile Range, NM
> > > >> > 88002
> > > >> > Office:(575) 678-2004 DSN 258-2004
> > > >> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
> > > >> > (575)
> > > >> > 678-1230 DSN 258-1230
> > > >> > Email: john.w.raby2.civ at mail.mil
> > > >> >
> > > >> >
> > > >> > CLASSIFICATION: UNCLASSIFIED
> > > >> >
> > > >> >
> > > >>
> > > >> --
> > > >> Julie Prestopnik (she/her)
> > > >> Software Engineer
> > > >> National Center for Atmospheric Research Research
Applications
> > > >> Laboratory
> > > >> Email: jpresto at ucar.edu
> > > >>
> > > >> My working day may not be your working day. Please do not
feel
> > > >> obliged to reply to this email outside of your normal working
hours.
> > > >>
> > > >> CLASSIFICATION: UNCLASSIFIED
> > > >>
> > > >>
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
More information about the Met_help
mailing list