[Met_help] [rt.rap.ucar.edu #90874] History for Use of MET/METplus to verify Particulate Matter (PM)

Thu Jul 11 09:32:12 MDT 2019

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi, there,

I don't know if y'all recall, but you had helped me sometime last year to
verify Particulate Matter (PM) using MET, which at one time had worked and
gave me output in the .stat file using point_stat.

Now, that seems to no longer work despite not making any changes to any of
the configuration files.  Tara had asked me to re-run MET using an older
version that had worked.

MET 7.0 is still available on our machine (I don't recall when that was
implemented), and I get the same result with MET 7.0 that I do when I run
with MET 8.1.  Therefore, I'm not sure I can get back to what had worked.

If anyone remembers helping with PM previously and has access to WCOSS, I
can point you to data/scripts/commands and see if you can help.

Thanks!

Perry

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Tue Jul 02 13:04:09 2019

Hi Perry,

I can log on and take a look today.  Can you point me in the right
direction on WCOSS?

Thanks,
John

On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> Transaction: Ticket created by perry.shafran at noaa.gov
>        Queue: met_help
>      Subject: Use of MET/METplus to verify Particulate Matter (PM)
>        Owner: Nobody
>   Requestors: perry.shafran at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
>
> Hi, there,
>
> I don't know if y'all recall, but you had helped me sometime last
year to
> verify Particulate Matter (PM) using MET, which at one time had
worked and
> gave me output in the .stat file using point_stat.
>
> Now, that seems to no longer work despite not making any changes to
any of
> the configuration files.  Tara had asked me to re-run MET using an
older
> version that had worked.
>
> MET 7.0 is still available on our machine (I don't recall when that
was
> implemented), and I get the same result with MET 7.0 that I do when
I run
> with MET 8.1.  Therefore, I'm not sure I can get back to what had
worked.
>
> If anyone remembers helping with PM previously and has access to
WCOSS, I
> can point you to data/scripts/commands and see if you can help.
>
> Thanks!
>
> Perry
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Tue Jul 02 13:08:39 2019

Hi, John,

OK, you can find things in the working directory:
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12

MET commands I've been using:

 /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc prepda.2019062912
prepda.nc.2019062912
/meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3

 /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat AWIP3D06.tm00
prepda.nc.2019062912
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM -v 3

This is using 7.0 but I see the same problem with the current version
8.1
as well.

Here is the output from point_stat:

DEBUG 1: Forecast File: AWIP3D06.tm00
DEBUG 1: Observation File: prepda.nc.2019062912
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for PMTF/A1.
DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact match for
"PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file "AWIP3D06.tm00"
DEBUG 3: Applying censor thresholds "NA" and replacing with values
"(nul)".
DEBUG 3: Censored values for 0 of 117130 grid points.
DEBUG 3: Use the matching forecast and observation grids.
DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 442 Ny:
265
Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha: 1037.975
Cone:
0.630 Bx: 208.6215 By: 7
84.2266
DEBUG 2: Processing masking regions.
DEBUG 3: Processing grid mask: FULL
DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology mean
levels,
and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Searching 21786 observations from 21786 messages.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation type
ANOWPM,
over region FULL, for interpolation method NEAREST(1), using 0 pairs.
DEBUG 3: Number of matched pairs  = 0
DEBUG 3: Observations processed   = 21786
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: obs type       = 0
DEBUG 3: Rejected: valid time     = 0
DEBUG 3: Rejected: bad obs value  = 0
DEBUG 3: Rejected: off the grid   = 1095
DEBUG 3: Rejected: level mismatch = 20691
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type   = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates     = 0
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file:
./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat

Perry

On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Hi Perry,
>
> I can log on and take a look today.  Can you point me in the right
> direction on WCOSS?
>
> Thanks,
> John
>
> On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > Transaction: Ticket created by perry.shafran at noaa.gov
> >        Queue: met_help
> >      Subject: Use of MET/METplus to verify Particulate Matter (PM)
> >        Owner: Nobody
> >   Requestors: perry.shafran at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> >
> > Hi, there,
> >
> > I don't know if y'all recall, but you had helped me sometime last
year to
> > verify Particulate Matter (PM) using MET, which at one time had
worked
> and
> > gave me output in the .stat file using point_stat.
> >
> > Now, that seems to no longer work despite not making any changes
to any
> of
> > the configuration files.  Tara had asked me to re-run MET using an
older
> > version that had worked.
> >
> > MET 7.0 is still available on our machine (I don't recall when
that was
> > implemented), and I get the same result with MET 7.0 that I do
when I run
> > with MET 8.1.  Therefore, I'm not sure I can get back to what had
worked.
> >
> > If anyone remembers helping with PM previously and has access to
WCOSS, I
> > can point you to data/scripts/commands and see if you can help.
> >
> > Thanks!
> >
> > Perry
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Tue Jul 09 15:25:57 2019

Hi, John,

This issue has not been resolved.  As a matter of fact, I was about to
ask
about your status on this request.

Perry

On Tue, Jul 9, 2019 at 2:17 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> According to our records, your request has been resolved. If you
have any
> further questions or concerns, please respond to this message.
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Wed Jul 10 12:00:21 2019

Perry,

I found data to test this behavior out in this directory:

   /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12

Here's the problem, pb2nc is reporting the accumulation interval for
the
COPOPM observation as being 0.  Changing the Point-Stat config file
like
this does indeed produce matched pairs:

*obs = {   convert(x) = x * 10^9;   field = [      { name = "COPOPM";
level
= "A0";' }   ];}*

If I remember correctly though, these files should contain both 1-hour
and
8-hour accumulations of COPOPM.  I do know that this logic used to
work...
and the fact that it's now NOT working with both met-7.0 and met-8.1
means
that the most likely explanation is that the data has changed.
Perhaps
something changed in the data files after the recent GFS upgrade?

We could test this theory like this...
1. Get a PREPBUFR file from a year ago (prepda.2018070712).
2. Get a recent PREPBUFR file (prepda.201907012).
3. Run them both through PB2NC using the same version of MET and the
same
config file.
4. Run both of them them through an Rscript to dump the NetCDF obs to
ascii

Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R prepda.nc.2018070712
>prepda.nc.2018070712.txt
Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R prepda.nc.2019070712
>prepda.nc.2019070712.txt

Compare the 8-th column of the output.  The theory is that the 2018
data
will have 1's and 8's there while the 2019 data will have all 0's.

If that's the case, we'll need to fix pb2nc to parse the accumulation
interval correctly from the updated file format.

Thanks,
John

On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
> Hi, John,
>
> OK, you can find things in the working directory:
> /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
>
> MET commands I've been using:
>
>  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
prepda.2019062912
> prepda.nc.2019062912
> /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3
>
>  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
AWIP3D06.tm00
> prepda.nc.2019062912
> /meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM -v
3
>
> This is using 7.0 but I see the same problem with the current
version 8.1
> as well.
>
> Here is the output from point_stat:
>
> DEBUG 1: Forecast File: AWIP3D06.tm00
> DEBUG 1: Observation File: prepda.nc.2019062912
> DEBUG 2:
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for PMTF/A1.
> DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact match
for
> "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file "AWIP3D06.tm00"
> DEBUG 3: Applying censor thresholds "NA" and replacing with values
"(nul)".
> DEBUG 3: Censored values for 0 of 117130 grid points.
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 442 Ny:
265
> Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha: 1037.975
Cone:
> 0.630 Bx: 208.6215 By: 7
> 84.2266
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 21786 observations from 21786 messages.
> DEBUG 2:
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation type
ANOWPM,
> over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> DEBUG 3: Number of matched pairs  = 0
> DEBUG 3: Observations processed   = 21786
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 0
> DEBUG 3: Rejected: valid time     = 0
> DEBUG 3: Rejected: bad obs value  = 0
> DEBUG 3: Rejected: off the grid   = 1095
> DEBUG 3: Rejected: level mismatch = 20691
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type   = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates     = 0
> DEBUG 2:
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file:
> ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
>
> Perry
>
> On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Perry,
> >
> > I can log on and take a look today.  Can you point me in the right
> > direction on WCOSS?
> >
> > Thanks,
> > John
> >
> > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > Transaction: Ticket created by perry.shafran at noaa.gov
> > >        Queue: met_help
> > >      Subject: Use of MET/METplus to verify Particulate Matter
(PM)
> > >        Owner: Nobody
> > >   Requestors: perry.shafran at noaa.gov
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >
> > >
> > >
> > > Hi, there,
> > >
> > > I don't know if y'all recall, but you had helped me sometime
last year
> to
> > > verify Particulate Matter (PM) using MET, which at one time had
worked
> > and
> > > gave me output in the .stat file using point_stat.
> > >
> > > Now, that seems to no longer work despite not making any changes
to any
> > of
> > > the configuration files.  Tara had asked me to re-run MET using
an
> older
> > > version that had worked.
> > >
> > > MET 7.0 is still available on our machine (I don't recall when
that was
> > > implemented), and I get the same result with MET 7.0 that I do
when I
> run
> > > with MET 8.1.  Therefore, I'm not sure I can get back to what
had
> worked.
> > >
> > > If anyone remembers helping with PM previously and has access to
> WCOSS, I
> > > can point you to data/scripts/commands and see if you can help.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Wed Jul 10 12:18:42 2019

Hi, John - Just to note, PM does *not* have 8-hr records.  They only
have
1-hr records.  You are thinking of ozone, which does have 1- and 8-hr
records (and they are averages, not accumulations).

I don't know if this changes your answer, but whatever logic that was
used
for PM did indeed work previously.  It could be that something you
were
using in the PM bufr file has changed.

For ozone, it's still working as expected, and that gives me
verification
output in the .stat files for both 1-hr and 8-hr averages.

These are files that would not have been affected by the GFS upgrade,
since
they originate from the EPA.

Do you know the full path to Rscript in the MET code?

Thanks!

Perry

On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> I found data to test this behavior out in this directory:
>
>    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
>
>
> Here's the problem, pb2nc is reporting the accumulation interval for
the
> COPOPM observation as being 0.  Changing the Point-Stat config file
like
> this does indeed produce matched pairs:
>
>
>
>
>
>
>
> *obs = {   convert(x) = x * 10^9;   field = [      { name =
"COPOPM"; level
> = "A0";' }   ];}*
>
>
>
> If I remember correctly though, these files should contain both 1-
hour and
> 8-hour accumulations of COPOPM.  I do know that this logic used to
work...
> and the fact that it's now NOT working with both met-7.0 and met-8.1
means
> that the most likely explanation is that the data has changed.
Perhaps
> something changed in the data files after the recent GFS upgrade?
>
> We could test this theory like this...
> 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> 2. Get a recent PREPBUFR file (prepda.201907012).
> 3. Run them both through PB2NC using the same version of MET and the
same
> config file.
> 4. Run both of them them through an Rscript to dump the NetCDF obs
to ascii
>
> Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2018070712
> >prepda.nc.2018070712.txt
> Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2019070712
> >prepda.nc.2019070712.txt
>
> Compare the 8-th column of the output.  The theory is that the 2018
data
> will have 1's and 8's there while the 2019 data will have all 0's.
>
> If that's the case, we'll need to fix pb2nc to parse the
accumulation
> interval correctly from the updated file format.
>
> Thanks,
> John
>
>
> On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > OK, you can find things in the working directory:
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> >
> > MET commands I've been using:
> >
> >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
prepda.2019062912
> > prepda.nc.2019062912
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3
> >
> >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
AWIP3D06.tm00
> > prepda.nc.2019062912
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
-v 3
> >
> > This is using 7.0 but I see the same problem with the current
version 8.1
> > as well.
> >
> > Here is the output from point_stat:
> >
> > DEBUG 1: Forecast File: AWIP3D06.tm00
> > DEBUG 1: Observation File: prepda.nc.2019062912
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for PMTF/A1.
> > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact match
for
> > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file "AWIP3D06.tm00"
> > DEBUG 3: Applying censor thresholds "NA" and replacing with values
> "(nul)".
> > DEBUG 3: Censored values for 0 of 117130 grid points.
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 442
Ny: 265
> > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha: 1037.975
Cone:
> > 0.630 Bx: 208.6215 By: 7
> > 84.2266
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Searching 21786 observations from 21786 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation type
> ANOWPM,
> > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > DEBUG 3: Number of matched pairs  = 0
> > DEBUG 3: Observations processed   = 21786
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 0
> > DEBUG 3: Rejected: valid time     = 0
> > DEBUG 3: Rejected: bad obs value  = 0
> > DEBUG 3: Rejected: off the grid   = 1095
> > DEBUG 3: Rejected: level mismatch = 20691
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type   = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates     = 0
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 1: Output file:
> > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> >
> > Perry
> >
> > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry,
> > >
> > > I can log on and take a look today.  Can you point me in the
right
> > > direction on WCOSS?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > >        Queue: met_help
> > > >      Subject: Use of MET/METplus to verify Particulate Matter
(PM)
> > > >        Owner: Nobody
> > > >   Requestors: perry.shafran at noaa.gov
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > >
> > > >
> > > >
> > > > Hi, there,
> > > >
> > > > I don't know if y'all recall, but you had helped me sometime
last
> year
> > to
> > > > verify Particulate Matter (PM) using MET, which at one time
had
> worked
> > > and
> > > > gave me output in the .stat file using point_stat.
> > > >
> > > > Now, that seems to no longer work despite not making any
changes to
> any
> > > of
> > > > the configuration files.  Tara had asked me to re-run MET
using an
> > older
> > > > version that had worked.
> > > >
> > > > MET 7.0 is still available on our machine (I don't recall when
that
> was
> > > > implemented), and I get the same result with MET 7.0 that I do
when I
> > run
> > > > with MET 8.1.  Therefore, I'm not sure I can get back to what
had
> > worked.
> > > >
> > > > If anyone remembers helping with PM previously and has access
to
> > WCOSS, I
> > > > can point you to data/scripts/commands and see if you can
help.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Wed Jul 10 12:28:30 2019

Hi, John,

I do have an year-old bufr file online.  Look on Tide in:

/meso/noscrub/Perry.Shafran/com/hourly/prod/hourly.20180707/aqm.t12z.anowpm.pb.tm024

It is the file of the same type you should be looking at now.

Perry

On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> I found data to test this behavior out in this directory:
>
>    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
>
>
> Here's the problem, pb2nc is reporting the accumulation interval for
the
> COPOPM observation as being 0.  Changing the Point-Stat config file
like
> this does indeed produce matched pairs:
>
>
>
>
>
>
>
> *obs = {   convert(x) = x * 10^9;   field = [      { name =
"COPOPM"; level
> = "A0";' }   ];}*
>
>
>
> If I remember correctly though, these files should contain both 1-
hour and
> 8-hour accumulations of COPOPM.  I do know that this logic used to
work...
> and the fact that it's now NOT working with both met-7.0 and met-8.1
means
> that the most likely explanation is that the data has changed.
Perhaps
> something changed in the data files after the recent GFS upgrade?
>
> We could test this theory like this...
> 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> 2. Get a recent PREPBUFR file (prepda.201907012).
> 3. Run them both through PB2NC using the same version of MET and the
same
> config file.
> 4. Run both of them them through an Rscript to dump the NetCDF obs
to ascii
>
> Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2018070712
> >prepda.nc.2018070712.txt
> Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2019070712
> >prepda.nc.2019070712.txt
>
> Compare the 8-th column of the output.  The theory is that the 2018
data
> will have 1's and 8's there while the 2019 data will have all 0's.
>
> If that's the case, we'll need to fix pb2nc to parse the
accumulation
> interval correctly from the updated file format.
>
> Thanks,
> John
>
>
> On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > OK, you can find things in the working directory:
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> >
> > MET commands I've been using:
> >
> >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
prepda.2019062912
> > prepda.nc.2019062912
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3
> >
> >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
AWIP3D06.tm00
> > prepda.nc.2019062912
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
-v 3
> >
> > This is using 7.0 but I see the same problem with the current
version 8.1
> > as well.
> >
> > Here is the output from point_stat:
> >
> > DEBUG 1: Forecast File: AWIP3D06.tm00
> > DEBUG 1: Observation File: prepda.nc.2019062912
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for PMTF/A1.
> > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact match
for
> > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file "AWIP3D06.tm00"
> > DEBUG 3: Applying censor thresholds "NA" and replacing with values
> "(nul)".
> > DEBUG 3: Censored values for 0 of 117130 grid points.
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 442
Ny: 265
> > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha: 1037.975
Cone:
> > 0.630 Bx: 208.6215 By: 7
> > 84.2266
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Searching 21786 observations from 21786 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation type
> ANOWPM,
> > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > DEBUG 3: Number of matched pairs  = 0
> > DEBUG 3: Observations processed   = 21786
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 0
> > DEBUG 3: Rejected: valid time     = 0
> > DEBUG 3: Rejected: bad obs value  = 0
> > DEBUG 3: Rejected: off the grid   = 1095
> > DEBUG 3: Rejected: level mismatch = 20691
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type   = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates     = 0
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 1: Output file:
> > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> >
> > Perry
> >
> > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry,
> > >
> > > I can log on and take a look today.  Can you point me in the
right
> > > direction on WCOSS?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > >        Queue: met_help
> > > >      Subject: Use of MET/METplus to verify Particulate Matter
(PM)
> > > >        Owner: Nobody
> > > >   Requestors: perry.shafran at noaa.gov
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > >
> > > >
> > > >
> > > > Hi, there,
> > > >
> > > > I don't know if y'all recall, but you had helped me sometime
last
> year
> > to
> > > > verify Particulate Matter (PM) using MET, which at one time
had
> worked
> > > and
> > > > gave me output in the .stat file using point_stat.
> > > >
> > > > Now, that seems to no longer work despite not making any
changes to
> any
> > > of
> > > > the configuration files.  Tara had asked me to re-run MET
using an
> > older
> > > > version that had worked.
> > > >
> > > > MET 7.0 is still available on our machine (I don't recall when
that
> was
> > > > implemented), and I get the same result with MET 7.0 that I do
when I
> > run
> > > > with MET 8.1.  Therefore, I'm not sure I can get back to what
had
> > worked.
> > > >
> > > > If anyone remembers helping with PM previously and has access
to
> > WCOSS, I
> > > > can point you to data/scripts/commands and see if you can
help.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Wed Jul 10 12:42:56 2019

Hi, John,

One more thing.  I don't know how you get the interval you've been
talking
about, but I've been using a local tool called debufr (I don't know if
you
have this tool), but it lists out what is in the prepbufr file
directly.
Seems the interval is gotten from a bufr variable called TPHR, which
is
listed in the debufr output as "Time Period or Displacement".  For the
ozone file these have values of -1.0 and -8.0 (note the minus signs).
For
the PM file, they all have a value of -1.

This is the same as in the 2018 file as it is for the 2019 file.

I'm also going to run PM using the change from A1 to A0 as you have
done.

Perry

On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> I found data to test this behavior out in this directory:
>
>    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
>
>
> Here's the problem, pb2nc is reporting the accumulation interval for
the
> COPOPM observation as being 0.  Changing the Point-Stat config file
like
> this does indeed produce matched pairs:
>
>
>
>
>
>
>
> *obs = {   convert(x) = x * 10^9;   field = [      { name =
"COPOPM"; level
> = "A0";' }   ];}*
>
>
>
> If I remember correctly though, these files should contain both 1-
hour and
> 8-hour accumulations of COPOPM.  I do know that this logic used to
work...
> and the fact that it's now NOT working with both met-7.0 and met-8.1
means
> that the most likely explanation is that the data has changed.
Perhaps
> something changed in the data files after the recent GFS upgrade?
>
> We could test this theory like this...
> 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> 2. Get a recent PREPBUFR file (prepda.201907012).
> 3. Run them both through PB2NC using the same version of MET and the
same
> config file.
> 4. Run both of them them through an Rscript to dump the NetCDF obs
to ascii
>
> Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2018070712
> >prepda.nc.2018070712.txt
> Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2019070712
> >prepda.nc.2019070712.txt
>
> Compare the 8-th column of the output.  The theory is that the 2018
data
> will have 1's and 8's there while the 2019 data will have all 0's.
>
> If that's the case, we'll need to fix pb2nc to parse the
accumulation
> interval correctly from the updated file format.
>
> Thanks,
> John
>
>
> On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > OK, you can find things in the working directory:
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> >
> > MET commands I've been using:
> >
> >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
prepda.2019062912
> > prepda.nc.2019062912
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3
> >
> >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
AWIP3D06.tm00
> > prepda.nc.2019062912
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
-v 3
> >
> > This is using 7.0 but I see the same problem with the current
version 8.1
> > as well.
> >
> > Here is the output from point_stat:
> >
> > DEBUG 1: Forecast File: AWIP3D06.tm00
> > DEBUG 1: Observation File: prepda.nc.2019062912
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for PMTF/A1.
> > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact match
for
> > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file "AWIP3D06.tm00"
> > DEBUG 3: Applying censor thresholds "NA" and replacing with values
> "(nul)".
> > DEBUG 3: Censored values for 0 of 117130 grid points.
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 442
Ny: 265
> > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha: 1037.975
Cone:
> > 0.630 Bx: 208.6215 By: 7
> > 84.2266
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Searching 21786 observations from 21786 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation type
> ANOWPM,
> > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > DEBUG 3: Number of matched pairs  = 0
> > DEBUG 3: Observations processed   = 21786
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 0
> > DEBUG 3: Rejected: valid time     = 0
> > DEBUG 3: Rejected: bad obs value  = 0
> > DEBUG 3: Rejected: off the grid   = 1095
> > DEBUG 3: Rejected: level mismatch = 20691
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type   = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates     = 0
> > DEBUG 2:
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 1: Output file:
> > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> >
> > Perry
> >
> > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry,
> > >
> > > I can log on and take a look today.  Can you point me in the
right
> > > direction on WCOSS?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > >        Queue: met_help
> > > >      Subject: Use of MET/METplus to verify Particulate Matter
(PM)
> > > >        Owner: Nobody
> > > >   Requestors: perry.shafran at noaa.gov
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > >
> > > >
> > > >
> > > > Hi, there,
> > > >
> > > > I don't know if y'all recall, but you had helped me sometime
last
> year
> > to
> > > > verify Particulate Matter (PM) using MET, which at one time
had
> worked
> > > and
> > > > gave me output in the .stat file using point_stat.
> > > >
> > > > Now, that seems to no longer work despite not making any
changes to
> any
> > > of
> > > > the configuration files.  Tara had asked me to re-run MET
using an
> > older
> > > > version that had worked.
> > > >
> > > > MET 7.0 is still available on our machine (I don't recall when
that
> was
> > > > implemented), and I get the same result with MET 7.0 that I do
when I
> > run
> > > > with MET 8.1.  Therefore, I'm not sure I can get back to what
had
> > worked.
> > > >
> > > > If anyone remembers helping with PM previously and has access
to
> > WCOSS, I
> > > > can point you to data/scripts/commands and see if you can
help.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Wed Jul 10 13:19:09 2019

Perry,

I ran through the debugger and found that TPHR is only being checked
for
AIRNOW observations:

*      // Special handling for "AIRNOW"      bool is_airnow = (0 ==
strcmp("AIRNOW", hdr_typ));*

And the message type for these obs is "ANOWPM".  I tested this change:

*      // Special handling for "AIRNOW" and "ANOWPM"      bool
is_airnow =
(0 == strcmp("AIRNOW", hdr_typ) ||                                  0
==
strcmp("ANOWPM", hdr_typ));*

And that makes PB2NC correctly handle the 1-hour averaging period.

Are you sure this was ever working?  I remember that we got things
working
for ozone, but don't remember the PM.

Howard, do you have any recollection of this?

The debufr tool sounds useful.  Is that on wcoss?  I don't have it in
my
path and can't find it in a module.

That Rscript can be found in
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R

Thanks,
John

On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
> Hi, John,
>
> One more thing.  I don't know how you get the interval you've been
talking
> about, but I've been using a local tool called debufr (I don't know
if you
> have this tool), but it lists out what is in the prepbufr file
directly.
> Seems the interval is gotten from a bufr variable called TPHR, which
is
> listed in the debufr output as "Time Period or Displacement".  For
the
> ozone file these have values of -1.0 and -8.0 (note the minus
signs).  For
> the PM file, they all have a value of -1.
>
> This is the same as in the 2018 file as it is for the 2019 file.
>
> I'm also going to run PM using the change from A1 to A0 as you have
done.
>
> Perry
>
> On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Perry,
> >
> > I found data to test this behavior out in this directory:
> >
> >    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> >
> >
> > Here's the problem, pb2nc is reporting the accumulation interval
for the
> > COPOPM observation as being 0.  Changing the Point-Stat config
file like
> > this does indeed produce matched pairs:
> >
> >
> >
> >
> >
> >
> >
> > *obs = {   convert(x) = x * 10^9;   field = [      { name =
"COPOPM";
> level
> > = "A0";' }   ];}*
> >
> >
> >
> > If I remember correctly though, these files should contain both 1-
hour
> and
> > 8-hour accumulations of COPOPM.  I do know that this logic used to
> work...
> > and the fact that it's now NOT working with both met-7.0 and met-
8.1
> means
> > that the most likely explanation is that the data has changed.
Perhaps
> > something changed in the data files after the recent GFS upgrade?
> >
> > We could test this theory like this...
> > 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> > 2. Get a recent PREPBUFR file (prepda.201907012).
> > 3. Run them both through PB2NC using the same version of MET and
the same
> > config file.
> > 4. Run both of them them through an Rscript to dump the NetCDF obs
to
> ascii
> >
> > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2018070712
> > >prepda.nc.2018070712.txt
> > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2019070712
> > >prepda.nc.2019070712.txt
> >
> > Compare the 8-th column of the output.  The theory is that the
2018 data
> > will have 1's and 8's there while the 2019 data will have all 0's.
> >
> > If that's the case, we'll need to fix pb2nc to parse the
accumulation
> > interval correctly from the updated file format.
> >
> > Thanks,
> > John
> >
> >
> > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > >
> > > Hi, John,
> > >
> > > OK, you can find things in the working directory:
> > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > >
> > > MET commands I've been using:
> > >
> > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
prepda.2019062912
> > > prepda.nc.2019062912
> > > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3
> > >
> > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
AWIP3D06.tm00
> > > prepda.nc.2019062912
> > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM -v 3
> > >
> > > This is using 7.0 but I see the same problem with the current
version
> 8.1
> > > as well.
> > >
> > > Here is the output from point_stat:
> > >
> > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for PMTF/A1.
> > > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact
match for
> > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
"AWIP3D06.tm00"
> > > DEBUG 3: Applying censor thresholds "NA" and replacing with
values
> > "(nul)".
> > > DEBUG 3: Censored values for 0 of 117130 grid points.
> > > DEBUG 3: Use the matching forecast and observation grids.
> > > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 442
Ny: 265
> > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha:
1037.975 Cone:
> > > 0.630 Bx: 208.6215 By: 7
> > > 84.2266
> > > DEBUG 2: Processing masking regions.
> > > DEBUG 3: Processing grid mask: FULL
> > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology mean
> levels,
> > > and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Searching 21786 observations from 21786 messages.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation
type
> > ANOWPM,
> > > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > > DEBUG 3: Number of matched pairs  = 0
> > > DEBUG 3: Observations processed   = 21786
> > > DEBUG 3: Rejected: SID exclusion  = 0
> > > DEBUG 3: Rejected: obs type       = 0
> > > DEBUG 3: Rejected: valid time     = 0
> > > DEBUG 3: Rejected: bad obs value  = 0
> > > DEBUG 3: Rejected: off the grid   = 1095
> > > DEBUG 3: Rejected: level mismatch = 20691
> > > DEBUG 3: Rejected: quality marker = 0
> > > DEBUG 3: Rejected: message type   = 0
> > > DEBUG 3: Rejected: masking region = 0
> > > DEBUG 3: Rejected: bad fcst value = 0
> > > DEBUG 3: Rejected: duplicates     = 0
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 1: Output file:
> > > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > >
> > > Perry
> > >
> > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Perry,
> > > >
> > > > I can log on and take a look today.  Can you point me in the
right
> > > > direction on WCOSS?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > >        Queue: met_help
> > > > >      Subject: Use of MET/METplus to verify Particulate
Matter (PM)
> > > > >        Owner: Nobody
> > > > >   Requestors: perry.shafran at noaa.gov
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > >
> > > > >
> > > > >
> > > > > Hi, there,
> > > > >
> > > > > I don't know if y'all recall, but you had helped me sometime
last
> > year
> > > to
> > > > > verify Particulate Matter (PM) using MET, which at one time
had
> > worked
> > > > and
> > > > > gave me output in the .stat file using point_stat.
> > > > >
> > > > > Now, that seems to no longer work despite not making any
changes to
> > any
> > > > of
> > > > > the configuration files.  Tara had asked me to re-run MET
using an
> > > older
> > > > > version that had worked.
> > > > >
> > > > > MET 7.0 is still available on our machine (I don't recall
when that
> > was
> > > > > implemented), and I get the same result with MET 7.0 that I
do
> when I
> > > run
> > > > > with MET 8.1.  Therefore, I'm not sure I can get back to
what had
> > > worked.
> > > > >
> > > > > If anyone remembers helping with PM previously and has
access to
> > > WCOSS, I
> > > > > can point you to data/scripts/commands and see if you can
help.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Wed Jul 10 13:25:25 2019

Hi, John,

I clearly have older .stat files with PM verification lines listed in
there, so it must have been working at some point.  (Which is why I
was
satisfied at the time to move on to something else.)

Sounds like you are saying that I make that replacement that the thing
will
now work?  Where is that replacement being done?  I don't see it in
the
PB2NC config file.

BTW - the full path to debufr is /nwprod/util/exec/debufr.  Try it on
any
prepbufr file, you get lots of information on each observations.  It
produces a large output file, though, since it goes through each
observations individually.

Thanks!

Perry

On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> I ran through the debugger and found that TPHR is only being checked
for
> AIRNOW observations:
>
>
> *      // Special handling for "AIRNOW"      bool is_airnow = (0 ==
> strcmp("AIRNOW", hdr_typ));*
>
> And the message type for these obs is "ANOWPM".  I tested this
change:
>
>
>
> *      // Special handling for "AIRNOW" and "ANOWPM"      bool
is_airnow =
> (0 == strcmp("AIRNOW", hdr_typ) ||
0 ==
> strcmp("ANOWPM", hdr_typ));*
>
> And that makes PB2NC correctly handle the 1-hour averaging period.
>
> Are you sure this was ever working?  I remember that we got things
working
> for ozone, but don't remember the PM.
>
> Howard, do you have any recollection of this?
>
> The debufr tool sounds useful.  Is that on wcoss?  I don't have it
in my
> path and can't find it in a module.
>
> That Rscript can be found in
>
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
>
> Thanks,
> John
>
> On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > One more thing.  I don't know how you get the interval you've been
> talking
> > about, but I've been using a local tool called debufr (I don't
know if
> you
> > have this tool), but it lists out what is in the prepbufr file
directly.
> > Seems the interval is gotten from a bufr variable called TPHR,
which is
> > listed in the debufr output as "Time Period or Displacement".  For
the
> > ozone file these have values of -1.0 and -8.0 (note the minus
signs).
> For
> > the PM file, they all have a value of -1.
> >
> > This is the same as in the 2018 file as it is for the 2019 file.
> >
> > I'm also going to run PM using the change from A1 to A0 as you
have done.
> >
> > Perry
> >
> > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Perry,
> > >
> > > I found data to test this behavior out in this directory:
> > >
> > >    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > >
> > >
> > > Here's the problem, pb2nc is reporting the accumulation interval
for
> the
> > > COPOPM observation as being 0.  Changing the Point-Stat config
file
> like
> > > this does indeed produce matched pairs:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *obs = {   convert(x) = x * 10^9;   field = [      { name =
"COPOPM";
> > level
> > > = "A0";' }   ];}*
> > >
> > >
> > >
> > > If I remember correctly though, these files should contain both
1-hour
> > and
> > > 8-hour accumulations of COPOPM.  I do know that this logic used
to
> > work...
> > > and the fact that it's now NOT working with both met-7.0 and
met-8.1
> > means
> > > that the most likely explanation is that the data has changed.
Perhaps
> > > something changed in the data files after the recent GFS
upgrade?
> > >
> > > We could test this theory like this...
> > > 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > 3. Run them both through PB2NC using the same version of MET and
the
> same
> > > config file.
> > > 4. Run both of them them through an Rscript to dump the NetCDF
obs to
> > ascii
> > >
> > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2018070712
> > > >prepda.nc.2018070712.txt
> > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2019070712
> > > >prepda.nc.2019070712.txt
> > >
> > > Compare the 8-th column of the output.  The theory is that the
2018
> data
> > > will have 1's and 8's there while the 2019 data will have all
0's.
> > >
> > > If that's the case, we'll need to fix pb2nc to parse the
accumulation
> > > interval correctly from the updated file format.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>
> > > >
> > > > Hi, John,
> > > >
> > > > OK, you can find things in the working directory:
> > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > >
> > > > MET commands I've been using:
> > > >
> > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
prepda.2019062912
> > > > prepda.nc.2019062912
> > > > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v
3
> > > >
> > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> AWIP3D06.tm00
> > > > prepda.nc.2019062912
> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM -v
> 3
> > > >
> > > > This is using 7.0 but I see the same problem with the current
version
> > 8.1
> > > > as well.
> > > >
> > > > Here is the output from point_stat:
> > > >
> > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > DEBUG 2:
> > > > DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > DEBUG 2:
> > > > DEBUG 2: Reading data for PMTF/A1.
> > > > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact
match for
> > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
"AWIP3D06.tm00"
> > > > DEBUG 3: Applying censor thresholds "NA" and replacing with
values
> > > "(nul)".
> > > > DEBUG 3: Censored values for 0 of 117130 grid points.
> > > > DEBUG 3: Use the matching forecast and observation grids.
> > > > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx:
442 Ny:
> 265
> > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha:
1037.975
> Cone:
> > > > 0.630 Bx: 208.6215 By: 7
> > > > 84.2266
> > > > DEBUG 2: Processing masking regions.
> > > > DEBUG 3: Processing grid mask: FULL
> > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology
mean
> > levels,
> > > > and 0 climatology standard deviation levels.
> > > > DEBUG 2:
> > > > DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > DEBUG 2:
> > > > DEBUG 2: Searching 21786 observations from 21786 messages.
> > > > DEBUG 2:
> > > > DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > DEBUG 2:
> > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for observation
type
> > > ANOWPM,
> > > > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > > > DEBUG 3: Number of matched pairs  = 0
> > > > DEBUG 3: Observations processed   = 21786
> > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > DEBUG 3: Rejected: obs type       = 0
> > > > DEBUG 3: Rejected: valid time     = 0
> > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > DEBUG 3: Rejected: quality marker = 0
> > > > DEBUG 3: Rejected: message type   = 0
> > > > DEBUG 3: Rejected: masking region = 0
> > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > DEBUG 3: Rejected: duplicates     = 0
> > > > DEBUG 2:
> > > > DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > DEBUG 2:
> > > > DEBUG 1: Output file:
> > > > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > >
> > > > Perry
> > > >
> > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry,
> > > > >
> > > > > I can log on and take a look today.  Can you point me in the
right
> > > > > direction on WCOSS?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > >        Queue: met_help
> > > > > >      Subject: Use of MET/METplus to verify Particulate
Matter
> (PM)
> > > > > >        Owner: Nobody
> > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > >       Status: new
> > > > > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi, there,
> > > > > >
> > > > > > I don't know if y'all recall, but you had helped me
sometime last
> > > year
> > > > to
> > > > > > verify Particulate Matter (PM) using MET, which at one
time had
> > > worked
> > > > > and
> > > > > > gave me output in the .stat file using point_stat.
> > > > > >
> > > > > > Now, that seems to no longer work despite not making any
changes
> to
> > > any
> > > > > of
> > > > > > the configuration files.  Tara had asked me to re-run MET
using
> an
> > > > older
> > > > > > version that had worked.
> > > > > >
> > > > > > MET 7.0 is still available on our machine (I don't recall
when
> that
> > > was
> > > > > > implemented), and I get the same result with MET 7.0 that
I do
> > when I
> > > > run
> > > > > > with MET 8.1.  Therefore, I'm not sure I can get back to
what had
> > > > worked.
> > > > > >
> > > > > > If anyone remembers helping with PM previously and has
access to
> > > > WCOSS, I
> > > > > > can point you to data/scripts/commands and see if you can
help.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Wed Jul 10 13:35:43 2019

Perry,

Can you please take a look at those older stat files in the "OBS_LEV"
output column?  Does it say "A0" or "A1"?  Because using "A0" will
indeed
work now.

But searching through the source code for older versions of pb2nc, I
can
find now reference to it ever including ANOWPM.

Thanks,
John

On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
> Hi, John,
>
> I clearly have older .stat files with PM verification lines listed
in
> there, so it must have been working at some point.  (Which is why I
was
> satisfied at the time to move on to something else.)
>
> Sounds like you are saying that I make that replacement that the
thing will
> now work?  Where is that replacement being done?  I don't see it in
the
> PB2NC config file.
>
> BTW - the full path to debufr is /nwprod/util/exec/debufr.  Try it
on any
> prepbufr file, you get lots of information on each observations.  It
> produces a large output file, though, since it goes through each
> observations individually.
>
> Thanks!
>
> Perry
>
> On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Perry,
> >
> > I ran through the debugger and found that TPHR is only being
checked for
> > AIRNOW observations:
> >
> >
> > *      // Special handling for "AIRNOW"      bool is_airnow = (0
==
> > strcmp("AIRNOW", hdr_typ));*
> >
> > And the message type for these obs is "ANOWPM".  I tested this
change:
> >
> >
> >
> > *      // Special handling for "AIRNOW" and "ANOWPM"      bool
is_airnow
> =
> > (0 == strcmp("AIRNOW", hdr_typ) ||
0 ==
> > strcmp("ANOWPM", hdr_typ));*
> >
> > And that makes PB2NC correctly handle the 1-hour averaging period.
> >
> > Are you sure this was ever working?  I remember that we got things
> working
> > for ozone, but don't remember the PM.
> >
> > Howard, do you have any recollection of this?
> >
> > The debufr tool sounds useful.  Is that on wcoss?  I don't have it
in my
> > path and can't find it in a module.
> >
> > That Rscript can be found in
> >
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> >
> > Thanks,
> > John
> >
> > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > >
> > > Hi, John,
> > >
> > > One more thing.  I don't know how you get the interval you've
been
> > talking
> > > about, but I've been using a local tool called debufr (I don't
know if
> > you
> > > have this tool), but it lists out what is in the prepbufr file
> directly.
> > > Seems the interval is gotten from a bufr variable called TPHR,
which is
> > > listed in the debufr output as "Time Period or Displacement".
For the
> > > ozone file these have values of -1.0 and -8.0 (note the minus
signs).
> > For
> > > the PM file, they all have a value of -1.
> > >
> > > This is the same as in the 2018 file as it is for the 2019 file.
> > >
> > > I'm also going to run PM using the change from A1 to A0 as you
have
> done.
> > >
> > > Perry
> > >
> > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Perry,
> > > >
> > > > I found data to test this behavior out in this directory:
> > > >
> > > >    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > > >
> > > >
> > > > Here's the problem, pb2nc is reporting the accumulation
interval for
> > the
> > > > COPOPM observation as being 0.  Changing the Point-Stat config
file
> > like
> > > > this does indeed produce matched pairs:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *obs = {   convert(x) = x * 10^9;   field = [      { name =
"COPOPM";
> > > level
> > > > = "A0";' }   ];}*
> > > >
> > > >
> > > >
> > > > If I remember correctly though, these files should contain
both
> 1-hour
> > > and
> > > > 8-hour accumulations of COPOPM.  I do know that this logic
used to
> > > work...
> > > > and the fact that it's now NOT working with both met-7.0 and
met-8.1
> > > means
> > > > that the most likely explanation is that the data has changed.
> Perhaps
> > > > something changed in the data files after the recent GFS
upgrade?
> > > >
> > > > We could test this theory like this...
> > > > 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > > 3. Run them both through PB2NC using the same version of MET
and the
> > same
> > > > config file.
> > > > 4. Run both of them them through an Rscript to dump the NetCDF
obs to
> > > ascii
> > > >
> > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2018070712
> > > > >prepda.nc.2018070712.txt
> > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
prepda.nc.2019070712
> > > > >prepda.nc.2019070712.txt
> > > >
> > > > Compare the 8-th column of the output.  The theory is that the
2018
> > data
> > > > will have 1's and 8's there while the 2019 data will have all
0's.
> > > >
> > > > If that's the case, we'll need to fix pb2nc to parse the
accumulation
> > > > interval correctly from the updated file format.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > > > >
> > > > > Hi, John,
> > > > >
> > > > > OK, you can find things in the working directory:
> > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > > >
> > > > > MET commands I've been using:
> > > > >
> > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> prepda.2019062912
> > > > > prepda.nc.2019062912
> > > > > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
-v 3
> > > > >
> > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> > AWIP3D06.tm00
> > > > > prepda.nc.2019062912
> > > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> -v
> > 3
> > > > >
> > > > > This is using 7.0 but I see the same problem with the
current
> version
> > > 8.1
> > > > > as well.
> > > > >
> > > > > Here is the output from point_stat:
> > > > >
> > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > > DEBUG 2:
> > > > > DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > DEBUG 2:
> > > > > DEBUG 2: Reading data for PMTF/A1.
> > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact
match
> for
> > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
"AWIP3D06.tm00"
> > > > > DEBUG 3: Applying censor thresholds "NA" and replacing with
values
> > > > "(nul)".
> > > > > DEBUG 3: Censored values for 0 of 117130 grid points.
> > > > > DEBUG 3: Use the matching forecast and observation grids.
> > > > > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx:
442 Ny:
> > 265
> > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha:
1037.975
> > Cone:
> > > > > 0.630 Bx: 208.6215 By: 7
> > > > > 84.2266
> > > > > DEBUG 2: Processing masking regions.
> > > > > DEBUG 3: Processing grid mask: FULL
> > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0 climatology
mean
> > > levels,
> > > > > and 0 climatology standard deviation levels.
> > > > > DEBUG 2:
> > > > > DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > DEBUG 2:
> > > > > DEBUG 2: Searching 21786 observations from 21786 messages.
> > > > > DEBUG 2:
> > > > > DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > DEBUG 2:
> > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
observation type
> > > > ANOWPM,
> > > > > over region FULL, for interpolation method NEAREST(1), using
0
> pairs.
> > > > > DEBUG 3: Number of matched pairs  = 0
> > > > > DEBUG 3: Observations processed   = 21786
> > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > DEBUG 3: Rejected: obs type       = 0
> > > > > DEBUG 3: Rejected: valid time     = 0
> > > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > > DEBUG 3: Rejected: quality marker = 0
> > > > > DEBUG 3: Rejected: message type   = 0
> > > > > DEBUG 3: Rejected: masking region = 0
> > > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > > DEBUG 3: Rejected: duplicates     = 0
> > > > > DEBUG 2:
> > > > > DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > DEBUG 2:
> > > > > DEBUG 1: Output file:
> > > > > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > > >
> > > > > Perry
> > > > >
> > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Perry,
> > > > > >
> > > > > > I can log on and take a look today.  Can you point me in
the
> right
> > > > > > direction on WCOSS?
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov via
RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted upon.
> > > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > > >        Queue: met_help
> > > > > > >      Subject: Use of MET/METplus to verify Particulate
Matter
> > (PM)
> > > > > > >        Owner: Nobody
> > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > >       Status: new
> > > > > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi, there,
> > > > > > >
> > > > > > > I don't know if y'all recall, but you had helped me
sometime
> last
> > > > year
> > > > > to
> > > > > > > verify Particulate Matter (PM) using MET, which at one
time had
> > > > worked
> > > > > > and
> > > > > > > gave me output in the .stat file using point_stat.
> > > > > > >
> > > > > > > Now, that seems to no longer work despite not making any
> changes
> > to
> > > > any
> > > > > > of
> > > > > > > the configuration files.  Tara had asked me to re-run
MET using
> > an
> > > > > older
> > > > > > > version that had worked.
> > > > > > >
> > > > > > > MET 7.0 is still available on our machine (I don't
recall when
> > that
> > > > was
> > > > > > > implemented), and I get the same result with MET 7.0
that I do
> > > when I
> > > > > run
> > > > > > > with MET 8.1.  Therefore, I'm not sure I can get back to
what
> had
> > > > > worked.
> > > > > > >
> > > > > > > If anyone remembers helping with PM previously and has
access
> to
> > > > > WCOSS, I
> > > > > > > can point you to data/scripts/commands and see if you
can help.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Wed Jul 10 13:37:48 2019

Hi, John,

Those old stat files have "A1" in them.

Perry

On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> Can you please take a look at those older stat files in the
"OBS_LEV"
> output column?  Does it say "A0" or "A1"?  Because using "A0" will
indeed
> work now.
>
> But searching through the source code for older versions of pb2nc, I
can
> find now reference to it ever including ANOWPM.
>
> Thanks,
> John
>
> On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > I clearly have older .stat files with PM verification lines listed
in
> > there, so it must have been working at some point.  (Which is why
I was
> > satisfied at the time to move on to something else.)
> >
> > Sounds like you are saying that I make that replacement that the
thing
> will
> > now work?  Where is that replacement being done?  I don't see it
in the
> > PB2NC config file.
> >
> > BTW - the full path to debufr is /nwprod/util/exec/debufr.  Try it
on any
> > prepbufr file, you get lots of information on each observations.
It
> > produces a large output file, though, since it goes through each
> > observations individually.
> >
> > Thanks!
> >
> > Perry
> >
> > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Perry,
> > >
> > > I ran through the debugger and found that TPHR is only being
checked
> for
> > > AIRNOW observations:
> > >
> > >
> > > *      // Special handling for "AIRNOW"      bool is_airnow = (0
==
> > > strcmp("AIRNOW", hdr_typ));*
> > >
> > > And the message type for these obs is "ANOWPM".  I tested this
change:
> > >
> > >
> > >
> > > *      // Special handling for "AIRNOW" and "ANOWPM"      bool
> is_airnow
> > =
> > > (0 == strcmp("AIRNOW", hdr_typ) ||
0
> ==
> > > strcmp("ANOWPM", hdr_typ));*
> > >
> > > And that makes PB2NC correctly handle the 1-hour averaging
period.
> > >
> > > Are you sure this was ever working?  I remember that we got
things
> > working
> > > for ozone, but don't remember the PM.
> > >
> > > Howard, do you have any recollection of this?
> > >
> > > The debufr tool sounds useful.  Is that on wcoss?  I don't have
it in
> my
> > > path and can't find it in a module.
> > >
> > > That Rscript can be found in
> > >
>
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>
> > > >
> > > > Hi, John,
> > > >
> > > > One more thing.  I don't know how you get the interval you've
been
> > > talking
> > > > about, but I've been using a local tool called debufr (I don't
know
> if
> > > you
> > > > have this tool), but it lists out what is in the prepbufr file
> > directly.
> > > > Seems the interval is gotten from a bufr variable called TPHR,
which
> is
> > > > listed in the debufr output as "Time Period or Displacement".
For
> the
> > > > ozone file these have values of -1.0 and -8.0 (note the minus
signs).
> > > For
> > > > the PM file, they all have a value of -1.
> > > >
> > > > This is the same as in the 2018 file as it is for the 2019
file.
> > > >
> > > > I'm also going to run PM using the change from A1 to A0 as you
have
> > done.
> > > >
> > > > Perry
> > > >
> > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Perry,
> > > > >
> > > > > I found data to test this behavior out in this directory:
> > > > >
> > > > >    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > > > >
> > > > >
> > > > > Here's the problem, pb2nc is reporting the accumulation
interval
> for
> > > the
> > > > > COPOPM observation as being 0.  Changing the Point-Stat
config file
> > > like
> > > > > this does indeed produce matched pairs:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *obs = {   convert(x) = x * 10^9;   field = [      { name =
> "COPOPM";
> > > > level
> > > > > = "A0";' }   ];}*
> > > > >
> > > > >
> > > > >
> > > > > If I remember correctly though, these files should contain
both
> > 1-hour
> > > > and
> > > > > 8-hour accumulations of COPOPM.  I do know that this logic
used to
> > > > work...
> > > > > and the fact that it's now NOT working with both met-7.0 and
> met-8.1
> > > > means
> > > > > that the most likely explanation is that the data has
changed.
> > Perhaps
> > > > > something changed in the data files after the recent GFS
upgrade?
> > > > >
> > > > > We could test this theory like this...
> > > > > 1. Get a PREPBUFR file from a year ago (prepda.2018070712).
> > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > > > 3. Run them both through PB2NC using the same version of MET
and
> the
> > > same
> > > > > config file.
> > > > > 4. Run both of them them through an Rscript to dump the
NetCDF obs
> to
> > > > ascii
> > > > >
> > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> prepda.nc.2018070712
> > > > > >prepda.nc.2018070712.txt
> > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> prepda.nc.2019070712
> > > > > >prepda.nc.2019070712.txt
> > > > >
> > > > > Compare the 8-th column of the output.  The theory is that
the 2018
> > > data
> > > > > will have 1's and 8's there while the 2019 data will have
all 0's.
> > > > >
> > > > > If that's the case, we'll need to fix pb2nc to parse the
> accumulation
> > > > > interval correctly from the updated file format.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > > > > >
> > > > > > Hi, John,
> > > > > >
> > > > > > OK, you can find things in the working directory:
> > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > > > >
> > > > > > MET commands I've been using:
> > > > > >
> > > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> > prepda.2019062912
> > > > > > prepda.nc.2019062912
> > > > > > /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
-v 3
> > > > > >
> > > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> > > AWIP3D06.tm00
> > > > > > prepda.nc.2019062912
> > > > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> > -v
> > > 3
> > > > > >
> > > > > > This is using 7.0 but I see the same problem with the
current
> > version
> > > > 8.1
> > > > > > as well.
> > > > > >
> > > > > > Here is the output from point_stat:
> > > > > >
> > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > > > DEBUG 2:
> > > > > > DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > DEBUG 2:
> > > > > > DEBUG 2: Reading data for PMTF/A1.
> > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found
exact match
> > for
> > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
"AWIP3D06.tm00"
> > > > > > DEBUG 3: Applying censor thresholds "NA" and replacing
with
> values
> > > > > "(nul)".
> > > > > > DEBUG 3: Censored values for 0 of 117130 grid points.
> > > > > > DEBUG 3: Use the matching forecast and observation grids.
> > > > > > DEBUG 3: Grid Definition: Projection: Lambert Conformal
Nx: 442
> Ny:
> > > 265
> > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha:
1037.975
> > > Cone:
> > > > > > 0.630 Bx: 208.6215 By: 7
> > > > > > 84.2266
> > > > > > DEBUG 2: Processing masking regions.
> > > > > > DEBUG 3: Processing grid mask: FULL
> > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
climatology mean
> > > > levels,
> > > > > > and 0 climatology standard deviation levels.
> > > > > > DEBUG 2:
> > > > > > DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > DEBUG 2:
> > > > > > DEBUG 2: Searching 21786 observations from 21786 messages.
> > > > > > DEBUG 2:
> > > > > > DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > DEBUG 2:
> > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
observation
> type
> > > > > ANOWPM,
> > > > > > over region FULL, for interpolation method NEAREST(1),
using 0
> > pairs.
> > > > > > DEBUG 3: Number of matched pairs  = 0
> > > > > > DEBUG 3: Observations processed   = 21786
> > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > DEBUG 3: Rejected: obs type       = 0
> > > > > > DEBUG 3: Rejected: valid time     = 0
> > > > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > > > DEBUG 3: Rejected: quality marker = 0
> > > > > > DEBUG 3: Rejected: message type   = 0
> > > > > > DEBUG 3: Rejected: masking region = 0
> > > > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > > > DEBUG 3: Rejected: duplicates     = 0
> > > > > > DEBUG 2:
> > > > > > DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > DEBUG 2:
> > > > > > DEBUG 1: Output file:
> > > > > > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT <
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry,
> > > > > > >
> > > > > > > I can log on and take a look today.  Can you point me in
the
> > right
> > > > > > > direction on WCOSS?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov
via RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted
upon.
> > > > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > > > >        Queue: met_help
> > > > > > > >      Subject: Use of MET/METplus to verify Particulate
Matter
> > > (PM)
> > > > > > > >        Owner: Nobody
> > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > >       Status: new
> > > > > > > >  Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi, there,
> > > > > > > >
> > > > > > > > I don't know if y'all recall, but you had helped me
sometime
> > last
> > > > > year
> > > > > > to
> > > > > > > > verify Particulate Matter (PM) using MET, which at one
time
> had
> > > > > worked
> > > > > > > and
> > > > > > > > gave me output in the .stat file using point_stat.
> > > > > > > >
> > > > > > > > Now, that seems to no longer work despite not making
any
> > changes
> > > to
> > > > > any
> > > > > > > of
> > > > > > > > the configuration files.  Tara had asked me to re-run
MET
> using
> > > an
> > > > > > older
> > > > > > > > version that had worked.
> > > > > > > >
> > > > > > > > MET 7.0 is still available on our machine (I don't
recall
> when
> > > that
> > > > > was
> > > > > > > > implemented), and I get the same result with MET 7.0
that I
> do
> > > > when I
> > > > > > run
> > > > > > > > with MET 8.1.  Therefore, I'm not sure I can get back
to what
> > had
> > > > > > worked.
> > > > > > > >
> > > > > > > > If anyone remembers helping with PM previously and has
access
> > to
> > > > > > WCOSS, I
> > > > > > > > can point you to data/scripts/commands and see if you
can
> help.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Wed Jul 10 13:57:30 2019

Perry,

I grepped through the version control logs and found this smoking gun:

commit f88b9f36c3d73a9ed771e2d4995bc059b9b05df7
Author: John Halley Gotway <johnhg at ucar.edu>
Date:   Wed Mar 28 20:51:07 2018 +0000

    Add the ANOWPM message type to the is_airnow boolean logic.

Now the question is, how did that changed get dropped from the code?

Thanks,
John

On Wed, Jul 10, 2019 at 1:38 PM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
> Hi, John,
>
> Those old stat files have "A1" in them.
>
> Perry
>
> On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Perry,
> >
> > Can you please take a look at those older stat files in the
"OBS_LEV"
> > output column?  Does it say "A0" or "A1"?  Because using "A0" will
indeed
> > work now.
> >
> > But searching through the source code for older versions of pb2nc,
I can
> > find now reference to it ever including ANOWPM.
> >
> > Thanks,
> > John
> >
> > On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > >
> > > Hi, John,
> > >
> > > I clearly have older .stat files with PM verification lines
listed in
> > > there, so it must have been working at some point.  (Which is
why I was
> > > satisfied at the time to move on to something else.)
> > >
> > > Sounds like you are saying that I make that replacement that the
thing
> > will
> > > now work?  Where is that replacement being done?  I don't see it
in the
> > > PB2NC config file.
> > >
> > > BTW - the full path to debufr is /nwprod/util/exec/debufr.  Try
it on
> any
> > > prepbufr file, you get lots of information on each observations.
It
> > > produces a large output file, though, since it goes through each
> > > observations individually.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Perry,
> > > >
> > > > I ran through the debugger and found that TPHR is only being
checked
> > for
> > > > AIRNOW observations:
> > > >
> > > >
> > > > *      // Special handling for "AIRNOW"      bool is_airnow =
(0 ==
> > > > strcmp("AIRNOW", hdr_typ));*
> > > >
> > > > And the message type for these obs is "ANOWPM".  I tested this
> change:
> > > >
> > > >
> > > >
> > > > *      // Special handling for "AIRNOW" and "ANOWPM"      bool
> > is_airnow
> > > =
> > > > (0 == strcmp("AIRNOW", hdr_typ) ||
0
> > ==
> > > > strcmp("ANOWPM", hdr_typ));*
> > > >
> > > > And that makes PB2NC correctly handle the 1-hour averaging
period.
> > > >
> > > > Are you sure this was ever working?  I remember that we got
things
> > > working
> > > > for ozone, but don't remember the PM.
> > > >
> > > > Howard, do you have any recollection of this?
> > > >
> > > > The debufr tool sounds useful.  Is that on wcoss?  I don't
have it in
> > my
> > > > path and can't find it in a module.
> > > >
> > > > That Rscript can be found in
> > > >
> >
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov via RT
<
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > > > >
> > > > > Hi, John,
> > > > >
> > > > > One more thing.  I don't know how you get the interval
you've been
> > > > talking
> > > > > about, but I've been using a local tool called debufr (I
don't know
> > if
> > > > you
> > > > > have this tool), but it lists out what is in the prepbufr
file
> > > directly.
> > > > > Seems the interval is gotten from a bufr variable called
TPHR,
> which
> > is
> > > > > listed in the debufr output as "Time Period or
Displacement".  For
> > the
> > > > > ozone file these have values of -1.0 and -8.0 (note the
minus
> signs).
> > > > For
> > > > > the PM file, they all have a value of -1.
> > > > >
> > > > > This is the same as in the 2018 file as it is for the 2019
file.
> > > > >
> > > > > I'm also going to run PM using the change from A1 to A0 as
you have
> > > done.
> > > > >
> > > > > Perry
> > > > >
> > > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Perry,
> > > > > >
> > > > > > I found data to test this behavior out in this directory:
> > > > > >
> > > > > >    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > > > > >
> > > > > >
> > > > > > Here's the problem, pb2nc is reporting the accumulation
interval
> > for
> > > > the
> > > > > > COPOPM observation as being 0.  Changing the Point-Stat
config
> file
> > > > like
> > > > > > this does indeed produce matched pairs:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *obs = {   convert(x) = x * 10^9;   field = [      { name
=
> > "COPOPM";
> > > > > level
> > > > > > = "A0";' }   ];}*
> > > > > >
> > > > > >
> > > > > >
> > > > > > If I remember correctly though, these files should contain
both
> > > 1-hour
> > > > > and
> > > > > > 8-hour accumulations of COPOPM.  I do know that this logic
used
> to
> > > > > work...
> > > > > > and the fact that it's now NOT working with both met-7.0
and
> > met-8.1
> > > > > means
> > > > > > that the most likely explanation is that the data has
changed.
> > > Perhaps
> > > > > > something changed in the data files after the recent GFS
upgrade?
> > > > > >
> > > > > > We could test this theory like this...
> > > > > > 1. Get a PREPBUFR file from a year ago
(prepda.2018070712).
> > > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > > > > 3. Run them both through PB2NC using the same version of
MET and
> > the
> > > > same
> > > > > > config file.
> > > > > > 4. Run both of them them through an Rscript to dump the
NetCDF
> obs
> > to
> > > > > ascii
> > > > > >
> > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > prepda.nc.2018070712
> > > > > > >prepda.nc.2018070712.txt
> > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > prepda.nc.2019070712
> > > > > > >prepda.nc.2019070712.txt
> > > > > >
> > > > > > Compare the 8-th column of the output.  The theory is that
the
> 2018
> > > > data
> > > > > > will have 1's and 8's there while the 2019 data will have
all
> 0's.
> > > > > >
> > > > > > If that's the case, we'll need to fix pb2nc to parse the
> > accumulation
> > > > > > interval correctly from the updated file format.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov via
RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >
> > > > > > >
> > > > > > > Hi, John,
> > > > > > >
> > > > > > > OK, you can find things in the working directory:
> > > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > > > > >
> > > > > > > MET commands I've been using:
> > > > > > >
> > > > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> > > prepda.2019062912
> > > > > > > prepda.nc.2019062912
> > > > > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm -v 3
> > > > > > >
> > > > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> > > > AWIP3D06.tm00
> > > > > > > prepda.nc.2019062912
> > > > > > >
> /meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> > > -v
> > > > 3
> > > > > > >
> > > > > > > This is using 7.0 but I see the same problem with the
current
> > > version
> > > > > 8.1
> > > > > > > as well.
> > > > > > >
> > > > > > > Here is the output from point_stat:
> > > > > > >
> > > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2: Reading data for PMTF/A1.
> > > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found
exact
> match
> > > for
> > > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
> "AWIP3D06.tm00"
> > > > > > > DEBUG 3: Applying censor thresholds "NA" and replacing
with
> > values
> > > > > > "(nul)".
> > > > > > > DEBUG 3: Censored values for 0 of 117130 grid points.
> > > > > > > DEBUG 3: Use the matching forecast and observation
grids.
> > > > > > > DEBUG 3: Grid Definition: Projection: Lambert Conformal
Nx: 442
> > Ny:
> > > > 265
> > > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000 Alpha:
> 1037.975
> > > > Cone:
> > > > > > > 0.630 Bx: 208.6215 By: 7
> > > > > > > 84.2266
> > > > > > > DEBUG 2: Processing masking regions.
> > > > > > > DEBUG 3: Processing grid mask: FULL
> > > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
climatology
> mean
> > > > > levels,
> > > > > > > and 0 climatology standard deviation levels.
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2: Searching 21786 observations from 21786
messages.
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
observation
> > type
> > > > > > ANOWPM,
> > > > > > > over region FULL, for interpolation method NEAREST(1),
using 0
> > > pairs.
> > > > > > > DEBUG 3: Number of matched pairs  = 0
> > > > > > > DEBUG 3: Observations processed   = 21786
> > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > DEBUG 3: Rejected: obs type       = 0
> > > > > > > DEBUG 3: Rejected: valid time     = 0
> > > > > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > > > > DEBUG 3: Rejected: quality marker = 0
> > > > > > > DEBUG 3: Rejected: message type   = 0
> > > > > > > DEBUG 3: Rejected: masking region = 0
> > > > > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > > > > DEBUG 3: Rejected: duplicates     = 0
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 1: Output file:
> > > > > > > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Perry,
> > > > > > > >
> > > > > > > > I can log on and take a look today.  Can you point me
in the
> > > right
> > > > > > > > direction on WCOSS?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > > On Tue, Jul 2, 2019 at 8:13 AM perry.shafran at noaa.gov
via
> RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted
upon.
> > > > > > > > > Transaction: Ticket created by
perry.shafran at noaa.gov
> > > > > > > > >        Queue: met_help
> > > > > > > > >      Subject: Use of MET/METplus to verify
Particulate
> Matter
> > > > (PM)
> > > > > > > > >        Owner: Nobody
> > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > >       Status: new
> > > > > > > > >  Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi, there,
> > > > > > > > >
> > > > > > > > > I don't know if y'all recall, but you had helped me
> sometime
> > > last
> > > > > > year
> > > > > > > to
> > > > > > > > > verify Particulate Matter (PM) using MET, which at
one time
> > had
> > > > > > worked
> > > > > > > > and
> > > > > > > > > gave me output in the .stat file using point_stat.
> > > > > > > > >
> > > > > > > > > Now, that seems to no longer work despite not making
any
> > > changes
> > > > to
> > > > > > any
> > > > > > > > of
> > > > > > > > > the configuration files.  Tara had asked me to re-
run MET
> > using
> > > > an
> > > > > > > older
> > > > > > > > > version that had worked.
> > > > > > > > >
> > > > > > > > > MET 7.0 is still available on our machine (I don't
recall
> > when
> > > > that
> > > > > > was
> > > > > > > > > implemented), and I get the same result with MET 7.0
that I
> > do
> > > > > when I
> > > > > > > run
> > > > > > > > > with MET 8.1.  Therefore, I'm not sure I can get
back to
> what
> > > had
> > > > > > > worked.
> > > > > > > > >
> > > > > > > > > If anyone remembers helping with PM previously and
has
> access
> > > to
> > > > > > > WCOSS, I
> > > > > > > > > can point you to data/scripts/commands and see if
you can
> > help.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > Perry
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Wed Jul 10 13:59:55 2019

Hi, John,

Interesting.  I have no clue how that got dropped.

Thanks for that info!

Perry

On Wed, Jul 10, 2019 at 3:57 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> I grepped through the version control logs and found this smoking
gun:
>
> commit f88b9f36c3d73a9ed771e2d4995bc059b9b05df7
> Author: John Halley Gotway <johnhg at ucar.edu>
> Date:   Wed Mar 28 20:51:07 2018 +0000
>
>     Add the ANOWPM message type to the is_airnow boolean logic.
>
> Now the question is, how did that changed get dropped from the code?
>
> Thanks,
> John
>
> On Wed, Jul 10, 2019 at 1:38 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > Those old stat files have "A1" in them.
> >
> > Perry
> >
> > On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Perry,
> > >
> > > Can you please take a look at those older stat files in the
"OBS_LEV"
> > > output column?  Does it say "A0" or "A1"?  Because using "A0"
will
> indeed
> > > work now.
> > >
> > > But searching through the source code for older versions of
pb2nc, I
> can
> > > find now reference to it ever including ANOWPM.
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>
> > > >
> > > > Hi, John,
> > > >
> > > > I clearly have older .stat files with PM verification lines
listed in
> > > > there, so it must have been working at some point.  (Which is
why I
> was
> > > > satisfied at the time to move on to something else.)
> > > >
> > > > Sounds like you are saying that I make that replacement that
the
> thing
> > > will
> > > > now work?  Where is that replacement being done?  I don't see
it in
> the
> > > > PB2NC config file.
> > > >
> > > > BTW - the full path to debufr is /nwprod/util/exec/debufr.
Try it on
> > any
> > > > prepbufr file, you get lots of information on each
observations.  It
> > > > produces a large output file, though, since it goes through
each
> > > > observations individually.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Perry,
> > > > >
> > > > > I ran through the debugger and found that TPHR is only being
> checked
> > > for
> > > > > AIRNOW observations:
> > > > >
> > > > >
> > > > > *      // Special handling for "AIRNOW"      bool is_airnow
= (0 ==
> > > > > strcmp("AIRNOW", hdr_typ));*
> > > > >
> > > > > And the message type for these obs is "ANOWPM".  I tested
this
> > change:
> > > > >
> > > > >
> > > > >
> > > > > *      // Special handling for "AIRNOW" and "ANOWPM"
bool
> > > is_airnow
> > > > =
> > > > > (0 == strcmp("AIRNOW", hdr_typ) ||
>   0
> > > ==
> > > > > strcmp("ANOWPM", hdr_typ));*
> > > > >
> > > > > And that makes PB2NC correctly handle the 1-hour averaging
period.
> > > > >
> > > > > Are you sure this was ever working?  I remember that we got
things
> > > > working
> > > > > for ozone, but don't remember the PM.
> > > > >
> > > > > Howard, do you have any recollection of this?
> > > > >
> > > > > The debufr tool sounds useful.  Is that on wcoss?  I don't
have it
> in
> > > my
> > > > > path and can't find it in a module.
> > > > >
> > > > > That Rscript can be found in
> > > > >
> > >
>
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > > > > >
> > > > > > Hi, John,
> > > > > >
> > > > > > One more thing.  I don't know how you get the interval
you've
> been
> > > > > talking
> > > > > > about, but I've been using a local tool called debufr (I
don't
> know
> > > if
> > > > > you
> > > > > > have this tool), but it lists out what is in the prepbufr
file
> > > > directly.
> > > > > > Seems the interval is gotten from a bufr variable called
TPHR,
> > which
> > > is
> > > > > > listed in the debufr output as "Time Period or
Displacement".
> For
> > > the
> > > > > > ozone file these have values of -1.0 and -8.0 (note the
minus
> > signs).
> > > > > For
> > > > > > the PM file, they all have a value of -1.
> > > > > >
> > > > > > This is the same as in the 2018 file as it is for the 2019
file.
> > > > > >
> > > > > > I'm also going to run PM using the change from A1 to A0 as
you
> have
> > > > done.
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Perry,
> > > > > > >
> > > > > > > I found data to test this behavior out in this
directory:
> > > > > > >
> > > > > > >    /stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > > > > > >
> > > > > > >
> > > > > > > Here's the problem, pb2nc is reporting the accumulation
> interval
> > > for
> > > > > the
> > > > > > > COPOPM observation as being 0.  Changing the Point-Stat
config
> > file
> > > > > like
> > > > > > > this does indeed produce matched pairs:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *obs = {   convert(x) = x * 10^9;   field = [      {
name =
> > > "COPOPM";
> > > > > > level
> > > > > > > = "A0";' }   ];}*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > If I remember correctly though, these files should
contain both
> > > > 1-hour
> > > > > > and
> > > > > > > 8-hour accumulations of COPOPM.  I do know that this
logic used
> > to
> > > > > > work...
> > > > > > > and the fact that it's now NOT working with both met-7.0
and
> > > met-8.1
> > > > > > means
> > > > > > > that the most likely explanation is that the data has
changed.
> > > > Perhaps
> > > > > > > something changed in the data files after the recent GFS
> upgrade?
> > > > > > >
> > > > > > > We could test this theory like this...
> > > > > > > 1. Get a PREPBUFR file from a year ago
(prepda.2018070712).
> > > > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > > > > > 3. Run them both through PB2NC using the same version of
MET
> and
> > > the
> > > > > same
> > > > > > > config file.
> > > > > > > 4. Run both of them them through an Rscript to dump the
NetCDF
> > obs
> > > to
> > > > > > ascii
> > > > > > >
> > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > prepda.nc.2018070712
> > > > > > > >prepda.nc.2018070712.txt
> > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > prepda.nc.2019070712
> > > > > > > >prepda.nc.2019070712.txt
> > > > > > >
> > > > > > > Compare the 8-th column of the output.  The theory is
that the
> > 2018
> > > > > data
> > > > > > > will have 1's and 8's there while the 2019 data will
have all
> > 0's.
> > > > > > >
> > > > > > > If that's the case, we'll need to fix pb2nc to parse the
> > > accumulation
> > > > > > > interval correctly from the updated file format.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov
via RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > >
> > > > > > > >
> > > > > > > > Hi, John,
> > > > > > > >
> > > > > > > > OK, you can find things in the working directory:
> > > > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > > > > > >
> > > > > > > > MET commands I've been using:
> > > > > > > >
> > > > > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> > > > prepda.2019062912
> > > > > > > > prepda.nc.2019062912
> > > > > > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
> -v 3
> > > > > > > >
> > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> > > > > AWIP3D06.tm00
> > > > > > > > prepda.nc.2019062912
> > > > > > > >
> > /meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> > > > -v
> > > > > 3
> > > > > > > >
> > > > > > > > This is using 7.0 but I see the same problem with the
current
> > > > version
> > > > > > 8.1
> > > > > > > > as well.
> > > > > > > >
> > > > > > > > Here is the output from point_stat:
> > > > > > > >
> > > > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2: Reading data for PMTF/A1.
> > > > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() - Found
exact
> > match
> > > > for
> > > > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
> > "AWIP3D06.tm00"
> > > > > > > > DEBUG 3: Applying censor thresholds "NA" and replacing
with
> > > values
> > > > > > > "(nul)".
> > > > > > > > DEBUG 3: Censored values for 0 of 117130 grid points.
> > > > > > > > DEBUG 3: Use the matching forecast and observation
grids.
> > > > > > > > DEBUG 3: Grid Definition: Projection: Lambert
Conformal Nx:
> 442
> > > Ny:
> > > > > 265
> > > > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000
Alpha:
> > 1037.975
> > > > > Cone:
> > > > > > > > 0.630 Bx: 208.6215 By: 7
> > > > > > > > 84.2266
> > > > > > > > DEBUG 2: Processing masking regions.
> > > > > > > > DEBUG 3: Processing grid mask: FULL
> > > > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
climatology
> > mean
> > > > > > levels,
> > > > > > > > and 0 climatology standard deviation levels.
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2: Searching 21786 observations from 21786
messages.
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
observation
> > > type
> > > > > > > ANOWPM,
> > > > > > > > over region FULL, for interpolation method NEAREST(1),
using
> 0
> > > > pairs.
> > > > > > > > DEBUG 3: Number of matched pairs  = 0
> > > > > > > > DEBUG 3: Observations processed   = 21786
> > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > DEBUG 3: Rejected: obs type       = 0
> > > > > > > > DEBUG 3: Rejected: valid time     = 0
> > > > > > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > > > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > > > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > > > > > DEBUG 3: Rejected: quality marker = 0
> > > > > > > > DEBUG 3: Rejected: message type   = 0
> > > > > > > > DEBUG 3: Rejected: masking region = 0
> > > > > > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > > > > > DEBUG 3: Rejected: duplicates     = 0
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 1: Output file:
> > > > > > > > ./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Perry,
> > > > > > > > >
> > > > > > > > > I can log on and take a look today.  Can you point
me in
> the
> > > > right
> > > > > > > > > direction on WCOSS?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > > On Tue, Jul 2, 2019 at 8:13 AM
perry.shafran at noaa.gov via
> > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was acted
upon.
> > > > > > > > > > Transaction: Ticket created by
perry.shafran at noaa.gov
> > > > > > > > > >        Queue: met_help
> > > > > > > > > >      Subject: Use of MET/METplus to verify
Particulate
> > Matter
> > > > > (PM)
> > > > > > > > > >        Owner: Nobody
> > > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > > >       Status: new
> > > > > > > > > >  Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi, there,
> > > > > > > > > >
> > > > > > > > > > I don't know if y'all recall, but you had helped
me
> > sometime
> > > > last
> > > > > > > year
> > > > > > > > to
> > > > > > > > > > verify Particulate Matter (PM) using MET, which at
one
> time
> > > had
> > > > > > > worked
> > > > > > > > > and
> > > > > > > > > > gave me output in the .stat file using point_stat.
> > > > > > > > > >
> > > > > > > > > > Now, that seems to no longer work despite not
making any
> > > > changes
> > > > > to
> > > > > > > any
> > > > > > > > > of
> > > > > > > > > > the configuration files.  Tara had asked me to re-
run MET
> > > using
> > > > > an
> > > > > > > > older
> > > > > > > > > > version that had worked.
> > > > > > > > > >
> > > > > > > > > > MET 7.0 is still available on our machine (I don't
recall
> > > when
> > > > > that
> > > > > > > was
> > > > > > > > > > implemented), and I get the same result with MET
7.0
> that I
> > > do
> > > > > > when I
> > > > > > > > run
> > > > > > > > > > with MET 8.1.  Therefore, I'm not sure I can get
back to
> > what
> > > > had
> > > > > > > > worked.
> > > > > > > > > >
> > > > > > > > > > If anyone remembers helping with PM previously and
has
> > access
> > > > to
> > > > > > > > WCOSS, I
> > > > > > > > > > can point you to data/scripts/commands and see if
you can
> > > help.
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > >
> > > > > > > > > > Perry
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Wed Jul 10 16:17:44 2019

Perry,

I did some forensic work on the repository and found exactly where the
ANOWPM logic got dropped.  I added the details to this GitHub issue:
   https://github.com/NCAR/MET/issues/1162

Adding it back in to the master_v8.1 bugfix branch and the develop
branch
(for future versions) is easy.  Figuring out when/how to get the
change to
you is the hard part.  We just did the first met-8.1.1 bugfix release
on
Monday.  This change will be included in met-8.1.2... but how soon
should
we do that?

Thanks,
John

On Wed, Jul 10, 2019 at 1:59 PM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
> Hi, John,
>
> Interesting.  I have no clue how that got dropped.
>
> Thanks for that info!
>
> Perry
>
> On Wed, Jul 10, 2019 at 3:57 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Perry,
> >
> > I grepped through the version control logs and found this smoking
gun:
> >
> > commit f88b9f36c3d73a9ed771e2d4995bc059b9b05df7
> > Author: John Halley Gotway <johnhg at ucar.edu>
> > Date:   Wed Mar 28 20:51:07 2018 +0000
> >
> >     Add the ANOWPM message type to the is_airnow boolean logic.
> >
> > Now the question is, how did that changed get dropped from the
code?
> >
> > Thanks,
> > John
> >
> > On Wed, Jul 10, 2019 at 1:38 PM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > >
> > > Hi, John,
> > >
> > > Those old stat files have "A1" in them.
> > >
> > > Perry
> > >
> > > On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Perry,
> > > >
> > > > Can you please take a look at those older stat files in the
"OBS_LEV"
> > > > output column?  Does it say "A0" or "A1"?  Because using "A0"
will
> > indeed
> > > > work now.
> > > >
> > > > But searching through the source code for older versions of
pb2nc, I
> > can
> > > > find now reference to it ever including ANOWPM.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via RT
<
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > > > >
> > > > > Hi, John,
> > > > >
> > > > > I clearly have older .stat files with PM verification lines
listed
> in
> > > > > there, so it must have been working at some point.  (Which
is why I
> > was
> > > > > satisfied at the time to move on to something else.)
> > > > >
> > > > > Sounds like you are saying that I make that replacement that
the
> > thing
> > > > will
> > > > > now work?  Where is that replacement being done?  I don't
see it in
> > the
> > > > > PB2NC config file.
> > > > >
> > > > > BTW - the full path to debufr is /nwprod/util/exec/debufr.
Try it
> on
> > > any
> > > > > prepbufr file, you get lots of information on each
observations.
> It
> > > > > produces a large output file, though, since it goes through
each
> > > > > observations individually.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Perry,
> > > > > >
> > > > > > I ran through the debugger and found that TPHR is only
being
> > checked
> > > > for
> > > > > > AIRNOW observations:
> > > > > >
> > > > > >
> > > > > > *      // Special handling for "AIRNOW"      bool
is_airnow = (0
> ==
> > > > > > strcmp("AIRNOW", hdr_typ));*
> > > > > >
> > > > > > And the message type for these obs is "ANOWPM".  I tested
this
> > > change:
> > > > > >
> > > > > >
> > > > > >
> > > > > > *      // Special handling for "AIRNOW" and "ANOWPM"
bool
> > > > is_airnow
> > > > > =
> > > > > > (0 == strcmp("AIRNOW", hdr_typ) ||
> >   0
> > > > ==
> > > > > > strcmp("ANOWPM", hdr_typ));*
> > > > > >
> > > > > > And that makes PB2NC correctly handle the 1-hour averaging
> period.
> > > > > >
> > > > > > Are you sure this was ever working?  I remember that we
got
> things
> > > > > working
> > > > > > for ozone, but don't remember the PM.
> > > > > >
> > > > > > Howard, do you have any recollection of this?
> > > > > >
> > > > > > The debufr tool sounds useful.  Is that on wcoss?  I don't
have
> it
> > in
> > > > my
> > > > > > path and can't find it in a module.
> > > > > >
> > > > > > That Rscript can be found in
> > > > > >
> > > >
> >
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov
via RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >
> > > > > > >
> > > > > > > Hi, John,
> > > > > > >
> > > > > > > One more thing.  I don't know how you get the interval
you've
> > been
> > > > > > talking
> > > > > > > about, but I've been using a local tool called debufr (I
don't
> > know
> > > > if
> > > > > > you
> > > > > > > have this tool), but it lists out what is in the
prepbufr file
> > > > > directly.
> > > > > > > Seems the interval is gotten from a bufr variable called
TPHR,
> > > which
> > > > is
> > > > > > > listed in the debufr output as "Time Period or
Displacement".
> > For
> > > > the
> > > > > > > ozone file these have values of -1.0 and -8.0 (note the
minus
> > > signs).
> > > > > > For
> > > > > > > the PM file, they all have a value of -1.
> > > > > > >
> > > > > > > This is the same as in the 2018 file as it is for the
2019
> file.
> > > > > > >
> > > > > > > I'm also going to run PM using the change from A1 to A0
as you
> > have
> > > > > done.
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Perry,
> > > > > > > >
> > > > > > > > I found data to test this behavior out in this
directory:
> > > > > > > >
> > > > > > > >
/stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > > > > > > >
> > > > > > > >
> > > > > > > > Here's the problem, pb2nc is reporting the
accumulation
> > interval
> > > > for
> > > > > > the
> > > > > > > > COPOPM observation as being 0.  Changing the Point-
Stat
> config
> > > file
> > > > > > like
> > > > > > > > this does indeed produce matched pairs:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *obs = {   convert(x) = x * 10^9;   field = [      {
name =
> > > > "COPOPM";
> > > > > > > level
> > > > > > > > = "A0";' }   ];}*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > If I remember correctly though, these files should
contain
> both
> > > > > 1-hour
> > > > > > > and
> > > > > > > > 8-hour accumulations of COPOPM.  I do know that this
logic
> used
> > > to
> > > > > > > work...
> > > > > > > > and the fact that it's now NOT working with both met-
7.0 and
> > > > met-8.1
> > > > > > > means
> > > > > > > > that the most likely explanation is that the data has
> changed.
> > > > > Perhaps
> > > > > > > > something changed in the data files after the recent
GFS
> > upgrade?
> > > > > > > >
> > > > > > > > We could test this theory like this...
> > > > > > > > 1. Get a PREPBUFR file from a year ago
(prepda.2018070712).
> > > > > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > > > > > > 3. Run them both through PB2NC using the same version
of MET
> > and
> > > > the
> > > > > > same
> > > > > > > > config file.
> > > > > > > > 4. Run both of them them through an Rscript to dump
the
> NetCDF
> > > obs
> > > > to
> > > > > > > ascii
> > > > > > > >
> > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > prepda.nc.2018070712
> > > > > > > > >prepda.nc.2018070712.txt
> > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > prepda.nc.2019070712
> > > > > > > > >prepda.nc.2019070712.txt
> > > > > > > >
> > > > > > > > Compare the 8-th column of the output.  The theory is
that
> the
> > > 2018
> > > > > > data
> > > > > > > > will have 1's and 8's there while the 2019 data will
have all
> > > 0's.
> > > > > > > >
> > > > > > > > If that's the case, we'll need to fix pb2nc to parse
the
> > > > accumulation
> > > > > > > > interval correctly from the updated file format.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Jul 2, 2019 at 1:44 PM perry.shafran at noaa.gov
via
> RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > >
> > > > > > > > >
> > > > > > > > > Hi, John,
> > > > > > > > >
> > > > > > > > > OK, you can find things in the working directory:
> > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > > > > > > >
> > > > > > > > > MET commands I've been using:
> > > > > > > > >
> > > > > > > > >  /global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> > > > > prepda.2019062912
> > > > > > > > > prepda.nc.2019062912
> > > > > > > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
> > -v 3
> > > > > > > > >
> > > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> > > > > > AWIP3D06.tm00
> > > > > > > > > prepda.nc.2019062912
> > > > > > > > >
> > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> > > > > -v
> > > > > > 3
> > > > > > > > >
> > > > > > > > > This is using 7.0 but I see the same problem with
the
> current
> > > > > version
> > > > > > > 8.1
> > > > > > > > > as well.
> > > > > > > > >
> > > > > > > > > Here is the output from point_stat:
> > > > > > > > >
> > > > > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2: Reading data for PMTF/A1.
> > > > > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() -
Found exact
> > > match
> > > > > for
> > > > > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
> > > "AWIP3D06.tm00"
> > > > > > > > > DEBUG 3: Applying censor thresholds "NA" and
replacing with
> > > > values
> > > > > > > > "(nul)".
> > > > > > > > > DEBUG 3: Censored values for 0 of 117130 grid
points.
> > > > > > > > > DEBUG 3: Use the matching forecast and observation
grids.
> > > > > > > > > DEBUG 3: Grid Definition: Projection: Lambert
Conformal Nx:
> > 442
> > > > Ny:
> > > > > > 265
> > > > > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000
Alpha:
> > > 1037.975
> > > > > > Cone:
> > > > > > > > > 0.630 Bx: 208.6215 By: 7
> > > > > > > > > 84.2266
> > > > > > > > > DEBUG 2: Processing masking regions.
> > > > > > > > > DEBUG 3: Processing grid mask: FULL
> > > > > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
climatology
> > > mean
> > > > > > > levels,
> > > > > > > > > and 0 climatology standard deviation levels.
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2: Searching 21786 observations from 21786
messages.
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
> observation
> > > > type
> > > > > > > > ANOWPM,
> > > > > > > > > over region FULL, for interpolation method
NEAREST(1),
> using
> > 0
> > > > > pairs.
> > > > > > > > > DEBUG 3: Number of matched pairs  = 0
> > > > > > > > > DEBUG 3: Observations processed   = 21786
> > > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > > DEBUG 3: Rejected: obs type       = 0
> > > > > > > > > DEBUG 3: Rejected: valid time     = 0
> > > > > > > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > > > > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > > > > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > > > > > > DEBUG 3: Rejected: quality marker = 0
> > > > > > > > > DEBUG 3: Rejected: message type   = 0
> > > > > > > > > DEBUG 3: Rejected: masking region = 0
> > > > > > > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > > > > > > DEBUG 3: Rejected: duplicates     = 0
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 1: Output file:
> > > > > > > > >
./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > > > > > > >
> > > > > > > > > Perry
> > > > > > > > >
> > > > > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway
via RT <
> > > > > > > > > met_help at ucar.edu>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Perry,
> > > > > > > > > >
> > > > > > > > > > I can log on and take a look today.  Can you point
me in
> > the
> > > > > right
> > > > > > > > > > direction on WCOSS?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > John
> > > > > > > > > >
> > > > > > > > > > On Tue, Jul 2, 2019 at 8:13 AM
perry.shafran at noaa.gov
> via
> > > RT <
> > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was
acted upon.
> > > > > > > > > > > Transaction: Ticket created by
perry.shafran at noaa.gov
> > > > > > > > > > >        Queue: met_help
> > > > > > > > > > >      Subject: Use of MET/METplus to verify
Particulate
> > > Matter
> > > > > > (PM)
> > > > > > > > > > >        Owner: Nobody
> > > > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > > > >       Status: new
> > > > > > > > > > >  Ticket <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Hi, there,
> > > > > > > > > > >
> > > > > > > > > > > I don't know if y'all recall, but you had helped
me
> > > sometime
> > > > > last
> > > > > > > > year
> > > > > > > > > to
> > > > > > > > > > > verify Particulate Matter (PM) using MET, which
at one
> > time
> > > > had
> > > > > > > > worked
> > > > > > > > > > and
> > > > > > > > > > > gave me output in the .stat file using
point_stat.
> > > > > > > > > > >
> > > > > > > > > > > Now, that seems to no longer work despite not
making
> any
> > > > > changes
> > > > > > to
> > > > > > > > any
> > > > > > > > > > of
> > > > > > > > > > > the configuration files.  Tara had asked me to
re-run
> MET
> > > > using
> > > > > > an
> > > > > > > > > older
> > > > > > > > > > > version that had worked.
> > > > > > > > > > >
> > > > > > > > > > > MET 7.0 is still available on our machine (I
don't
> recall
> > > > when
> > > > > > that
> > > > > > > > was
> > > > > > > > > > > implemented), and I get the same result with MET
7.0
> > that I
> > > > do
> > > > > > > when I
> > > > > > > > > run
> > > > > > > > > > > with MET 8.1.  Therefore, I'm not sure I can get
back
> to
> > > what
> > > > > had
> > > > > > > > > worked.
> > > > > > > > > > >
> > > > > > > > > > > If anyone remembers helping with PM previously
and has
> > > access
> > > > > to
> > > > > > > > > WCOSS, I
> > > > > > > > > > > can point you to data/scripts/commands and see
if you
> can
> > > > help.
> > > > > > > > > > >
> > > > > > > > > > > Thanks!
> > > > > > > > > > >
> > > > > > > > > > > Perry
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Thu Jul 11 06:59:54 2019

Hi, John,

The sooner the better.  Folks here are beginning to get impatient for
AQ
verification.

Is the workaround just to use A0 in the config files instead of A1?
We can
use that as a temporary work-around, if need be.

Thanks!

Perry

On Wed, Jul 10, 2019 at 6:17 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Perry,
>
> I did some forensic work on the repository and found exactly where
the
> ANOWPM logic got dropped.  I added the details to this GitHub issue:
>    https://github.com/NCAR/MET/issues/1162
>
> Adding it back in to the master_v8.1 bugfix branch and the develop
branch
> (for future versions) is easy.  Figuring out when/how to get the
change to
> you is the hard part.  We just did the first met-8.1.1 bugfix
release on
> Monday.  This change will be included in met-8.1.2... but how soon
should
> we do that?
>
> Thanks,
> John
>
> On Wed, Jul 10, 2019 at 1:59 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >
> > Hi, John,
> >
> > Interesting.  I have no clue how that got dropped.
> >
> > Thanks for that info!
> >
> > Perry
> >
> > On Wed, Jul 10, 2019 at 3:57 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Perry,
> > >
> > > I grepped through the version control logs and found this
smoking gun:
> > >
> > > commit f88b9f36c3d73a9ed771e2d4995bc059b9b05df7
> > > Author: John Halley Gotway <johnhg at ucar.edu>
> > > Date:   Wed Mar 28 20:51:07 2018 +0000
> > >
> > >     Add the ANOWPM message type to the is_airnow boolean logic.
> > >
> > > Now the question is, how did that changed get dropped from the
code?
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Jul 10, 2019 at 1:38 PM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>
> > > >
> > > > Hi, John,
> > > >
> > > > Those old stat files have "A1" in them.
> > > >
> > > > Perry
> > > >
> > > > On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Perry,
> > > > >
> > > > > Can you please take a look at those older stat files in the
> "OBS_LEV"
> > > > > output column?  Does it say "A0" or "A1"?  Because using
"A0" will
> > > indeed
> > > > > work now.
> > > > >
> > > > > But searching through the source code for older versions of
pb2nc,
> I
> > > can
> > > > > find now reference to it ever including ANOWPM.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> > > > > >
> > > > > > Hi, John,
> > > > > >
> > > > > > I clearly have older .stat files with PM verification
lines
> listed
> > in
> > > > > > there, so it must have been working at some point.  (Which
is
> why I
> > > was
> > > > > > satisfied at the time to move on to something else.)
> > > > > >
> > > > > > Sounds like you are saying that I make that replacement
that the
> > > thing
> > > > > will
> > > > > > now work?  Where is that replacement being done?  I don't
see it
> in
> > > the
> > > > > > PB2NC config file.
> > > > > >
> > > > > > BTW - the full path to debufr is /nwprod/util/exec/debufr.
Try
> it
> > on
> > > > any
> > > > > > prepbufr file, you get lots of information on each
observations.
> > It
> > > > > > produces a large output file, though, since it goes
through each
> > > > > > observations individually.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Perry,
> > > > > > >
> > > > > > > I ran through the debugger and found that TPHR is only
being
> > > checked
> > > > > for
> > > > > > > AIRNOW observations:
> > > > > > >
> > > > > > >
> > > > > > > *      // Special handling for "AIRNOW"      bool
is_airnow =
> (0
> > ==
> > > > > > > strcmp("AIRNOW", hdr_typ));*
> > > > > > >
> > > > > > > And the message type for these obs is "ANOWPM".  I
tested this
> > > > change:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *      // Special handling for "AIRNOW" and "ANOWPM"
bool
> > > > > is_airnow
> > > > > > =
> > > > > > > (0 == strcmp("AIRNOW", hdr_typ) ||
> > >   0
> > > > > ==
> > > > > > > strcmp("ANOWPM", hdr_typ));*
> > > > > > >
> > > > > > > And that makes PB2NC correctly handle the 1-hour
averaging
> > period.
> > > > > > >
> > > > > > > Are you sure this was ever working?  I remember that we
got
> > things
> > > > > > working
> > > > > > > for ozone, but don't remember the PM.
> > > > > > >
> > > > > > > Howard, do you have any recollection of this?
> > > > > > >
> > > > > > > The debufr tool sounds useful.  Is that on wcoss?  I
don't have
> > it
> > > in
> > > > > my
> > > > > > > path and can't find it in a module.
> > > > > > >
> > > > > > > That Rscript can be found in
> > > > > > >
> > > > >
> > >
>
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov
via
> RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > >
> > > > > > > >
> > > > > > > > Hi, John,
> > > > > > > >
> > > > > > > > One more thing.  I don't know how you get the interval
you've
> > > been
> > > > > > > talking
> > > > > > > > about, but I've been using a local tool called debufr
(I
> don't
> > > know
> > > > > if
> > > > > > > you
> > > > > > > > have this tool), but it lists out what is in the
prepbufr
> file
> > > > > > directly.
> > > > > > > > Seems the interval is gotten from a bufr variable
called
> TPHR,
> > > > which
> > > > > is
> > > > > > > > listed in the debufr output as "Time Period or
Displacement".
> > > For
> > > > > the
> > > > > > > > ozone file these have values of -1.0 and -8.0 (note
the minus
> > > > signs).
> > > > > > > For
> > > > > > > > the PM file, they all have a value of -1.
> > > > > > > >
> > > > > > > > This is the same as in the 2018 file as it is for the
2019
> > file.
> > > > > > > >
> > > > > > > > I'm also going to run PM using the change from A1 to
A0 as
> you
> > > have
> > > > > > done.
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Perry,
> > > > > > > > >
> > > > > > > > > I found data to test this behavior out in this
directory:
> > > > > > > > >
> > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Here's the problem, pb2nc is reporting the
accumulation
> > > interval
> > > > > for
> > > > > > > the
> > > > > > > > > COPOPM observation as being 0.  Changing the Point-
Stat
> > config
> > > > file
> > > > > > > like
> > > > > > > > > this does indeed produce matched pairs:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *obs = {   convert(x) = x * 10^9;   field = [      {
name =
> > > > > "COPOPM";
> > > > > > > > level
> > > > > > > > > = "A0";' }   ];}*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > If I remember correctly though, these files should
contain
> > both
> > > > > > 1-hour
> > > > > > > > and
> > > > > > > > > 8-hour accumulations of COPOPM.  I do know that this
logic
> > used
> > > > to
> > > > > > > > work...
> > > > > > > > > and the fact that it's now NOT working with both
met-7.0
> and
> > > > > met-8.1
> > > > > > > > means
> > > > > > > > > that the most likely explanation is that the data
has
> > changed.
> > > > > > Perhaps
> > > > > > > > > something changed in the data files after the recent
GFS
> > > upgrade?
> > > > > > > > >
> > > > > > > > > We could test this theory like this...
> > > > > > > > > 1. Get a PREPBUFR file from a year ago
(prepda.2018070712).
> > > > > > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> > > > > > > > > 3. Run them both through PB2NC using the same
version of
> MET
> > > and
> > > > > the
> > > > > > > same
> > > > > > > > > config file.
> > > > > > > > > 4. Run both of them them through an Rscript to dump
the
> > NetCDF
> > > > obs
> > > > > to
> > > > > > > > ascii
> > > > > > > > >
> > > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > > prepda.nc.2018070712
> > > > > > > > > >prepda.nc.2018070712.txt
> > > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> > > > > prepda.nc.2019070712
> > > > > > > > > >prepda.nc.2019070712.txt
> > > > > > > > >
> > > > > > > > > Compare the 8-th column of the output.  The theory
is that
> > the
> > > > 2018
> > > > > > > data
> > > > > > > > > will have 1's and 8's there while the 2019 data will
have
> all
> > > > 0's.
> > > > > > > > >
> > > > > > > > > If that's the case, we'll need to fix pb2nc to parse
the
> > > > > accumulation
> > > > > > > > > interval correctly from the updated file format.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Jul 2, 2019 at 1:44 PM
perry.shafran at noaa.gov via
> > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > >
> > > > > > > > > >
> > > > > > > > > > Hi, John,
> > > > > > > > > >
> > > > > > > > > > OK, you can find things in the working directory:
> > > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> > > > > > > > > >
> > > > > > > > > > MET commands I've been using:
> > > > > > > > > >
> > > > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> > > > > > prepda.2019062912
> > > > > > > > > > prepda.nc.2019062912
> > > > > > > > > >
> /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
> > > -v 3
> > > > > > > > > >
> > > > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> > > > > > > AWIP3D06.tm00
> > > > > > > > > > prepda.nc.2019062912
> > > > > > > > > >
> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> > > > > > -v
> > > > > > > 3
> > > > > > > > > >
> > > > > > > > > > This is using 7.0 but I see the same problem with
the
> > current
> > > > > > version
> > > > > > > > 8.1
> > > > > > > > > > as well.
> > > > > > > > > >
> > > > > > > > > > Here is the output from point_stat:
> > > > > > > > > >
> > > > > > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> > > > > > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2: Reading data for PMTF/A1.
> > > > > > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() -
Found
> exact
> > > > match
> > > > > > for
> > > > > > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
> > > > "AWIP3D06.tm00"
> > > > > > > > > > DEBUG 3: Applying censor thresholds "NA" and
replacing
> with
> > > > > values
> > > > > > > > > "(nul)".
> > > > > > > > > > DEBUG 3: Censored values for 0 of 117130 grid
points.
> > > > > > > > > > DEBUG 3: Use the matching forecast and observation
grids.
> > > > > > > > > > DEBUG 3: Grid Definition: Projection: Lambert
Conformal
> Nx:
> > > 442
> > > > > Ny:
> > > > > > > 265
> > > > > > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000
Alpha:
> > > > 1037.975
> > > > > > > Cone:
> > > > > > > > > > 0.630 Bx: 208.6215 By: 7
> > > > > > > > > > 84.2266
> > > > > > > > > > DEBUG 2: Processing masking regions.
> > > > > > > > > > DEBUG 3: Processing grid mask: FULL
> > > > > > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
> climatology
> > > > mean
> > > > > > > > levels,
> > > > > > > > > > and 0 climatology standard deviation levels.
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2: Searching 21786 observations from 21786
> messages.
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
> > observation
> > > > > type
> > > > > > > > > ANOWPM,
> > > > > > > > > > over region FULL, for interpolation method
NEAREST(1),
> > using
> > > 0
> > > > > > pairs.
> > > > > > > > > > DEBUG 3: Number of matched pairs  = 0
> > > > > > > > > > DEBUG 3: Observations processed   = 21786
> > > > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > > > DEBUG 3: Rejected: obs type       = 0
> > > > > > > > > > DEBUG 3: Rejected: valid time     = 0
> > > > > > > > > > DEBUG 3: Rejected: bad obs value  = 0
> > > > > > > > > > DEBUG 3: Rejected: off the grid   = 1095
> > > > > > > > > > DEBUG 3: Rejected: level mismatch = 20691
> > > > > > > > > > DEBUG 3: Rejected: quality marker = 0
> > > > > > > > > > DEBUG 3: Rejected: message type   = 0
> > > > > > > > > > DEBUG 3: Rejected: masking region = 0
> > > > > > > > > > DEBUG 3: Rejected: bad fcst value = 0
> > > > > > > > > > DEBUG 3: Rejected: duplicates     = 0
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 1: Output file:
> > > > > > > > > >
./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> > > > > > > > > >
> > > > > > > > > > Perry
> > > > > > > > > >
> > > > > > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway
via RT
> <
> > > > > > > > > > met_help at ucar.edu>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Perry,
> > > > > > > > > > >
> > > > > > > > > > > I can log on and take a look today.  Can you
point me
> in
> > > the
> > > > > > right
> > > > > > > > > > > direction on WCOSS?
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > John
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Jul 2, 2019 at 8:13 AM
perry.shafran at noaa.gov
> > via
> > > > RT <
> > > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was
acted
> upon.
> > > > > > > > > > > > Transaction: Ticket created by
> perry.shafran at noaa.gov
> > > > > > > > > > > >        Queue: met_help
> > > > > > > > > > > >      Subject: Use of MET/METplus to verify
> Particulate
> > > > Matter
> > > > > > > (PM)
> > > > > > > > > > > >        Owner: Nobody
> > > > > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > > > > >       Status: new
> > > > > > > > > > > >  Ticket <URL:
> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Hi, there,
> > > > > > > > > > > >
> > > > > > > > > > > > I don't know if y'all recall, but you had
helped me
> > > > sometime
> > > > > > last
> > > > > > > > > year
> > > > > > > > > > to
> > > > > > > > > > > > verify Particulate Matter (PM) using MET,
which at
> one
> > > time
> > > > > had
> > > > > > > > > worked
> > > > > > > > > > > and
> > > > > > > > > > > > gave me output in the .stat file using
point_stat.
> > > > > > > > > > > >
> > > > > > > > > > > > Now, that seems to no longer work despite not
making
> > any
> > > > > > changes
> > > > > > > to
> > > > > > > > > any
> > > > > > > > > > > of
> > > > > > > > > > > > the configuration files.  Tara had asked me to
re-run
> > MET
> > > > > using
> > > > > > > an
> > > > > > > > > > older
> > > > > > > > > > > > version that had worked.
> > > > > > > > > > > >
> > > > > > > > > > > > MET 7.0 is still available on our machine (I
don't
> > recall
> > > > > when
> > > > > > > that
> > > > > > > > > was
> > > > > > > > > > > > implemented), and I get the same result with
MET 7.0
> > > that I
> > > > > do
> > > > > > > > when I
> > > > > > > > > > run
> > > > > > > > > > > > with MET 8.1.  Therefore, I'm not sure I can
get back
> > to
> > > > what
> > > > > > had
> > > > > > > > > > worked.
> > > > > > > > > > > >
> > > > > > > > > > > > If anyone remembers helping with PM previously
and
> has
> > > > access
> > > > > > to
> > > > > > > > > > WCOSS, I
> > > > > > > > > > > > can point you to data/scripts/commands and see
if you
> > can
> > > > > help.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks!
> > > > > > > > > > > >
> > > > > > > > > > > > Perry
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: perry.shafran at noaa.gov
Time: Thu Jul 11 09:05:46 2019

FYI - using A0 in the config files (for both the observation and the
forecast) instead of A1 does give output in the .stat file for PM
verification.

Therefore I can use this for PM verification and you don't need to do
anything quickly to get us 8.1.2.

Thanks!

Perry

On Thu, Jul 11, 2019 at 8:59 AM Perry Shafran - NOAA Affiliate <
perry.shafran at noaa.gov> wrote:

> Hi, John,
>
> The sooner the better.  Folks here are beginning to get impatient
for AQ
> verification.
>
> Is the workaround just to use A0 in the config files instead of A1?
We
> can use that as a temporary work-around, if need be.
>
> Thanks!
>
> Perry
>
> On Wed, Jul 10, 2019 at 6:17 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Perry,
>>
>> I did some forensic work on the repository and found exactly where
the
>> ANOWPM logic got dropped.  I added the details to this GitHub
issue:
>>    https://github.com/NCAR/MET/issues/1162
>>
>> Adding it back in to the master_v8.1 bugfix branch and the develop
branch
>> (for future versions) is easy.  Figuring out when/how to get the
change to
>> you is the hard part.  We just did the first met-8.1.1 bugfix
release on
>> Monday.  This change will be included in met-8.1.2... but how soon
should
>> we do that?
>>
>> Thanks,
>> John
>>
>> On Wed, Jul 10, 2019 at 1:59 PM perry.shafran at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>> >
>> > Hi, John,
>> >
>> > Interesting.  I have no clue how that got dropped.
>> >
>> > Thanks for that info!
>> >
>> > Perry
>> >
>> > On Wed, Jul 10, 2019 at 3:57 PM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Perry,
>> > >
>> > > I grepped through the version control logs and found this
smoking gun:
>> > >
>> > > commit f88b9f36c3d73a9ed771e2d4995bc059b9b05df7
>> > > Author: John Halley Gotway <johnhg at ucar.edu>
>> > > Date:   Wed Mar 28 20:51:07 2018 +0000
>> > >
>> > >     Add the ANOWPM message type to the is_airnow boolean logic.
>> > >
>> > > Now the question is, how did that changed get dropped from the
code?
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Wed, Jul 10, 2019 at 1:38 PM perry.shafran at noaa.gov via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>
>> > > >
>> > > > Hi, John,
>> > > >
>> > > > Those old stat files have "A1" in them.
>> > > >
>> > > > Perry
>> > > >
>> > > > On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT <
>> > > > met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Perry,
>> > > > >
>> > > > > Can you please take a look at those older stat files in the
>> "OBS_LEV"
>> > > > > output column?  Does it say "A0" or "A1"?  Because using
"A0" will
>> > > indeed
>> > > > > work now.
>> > > > >
>> > > > > But searching through the source code for older versions of
>> pb2nc, I
>> > > can
>> > > > > find now reference to it ever including ANOWPM.
>> > > > >
>> > > > > Thanks,
>> > > > > John
>> > > > >
>> > > > > On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov via
RT <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > > >
>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>> > > > > >
>> > > > > > Hi, John,
>> > > > > >
>> > > > > > I clearly have older .stat files with PM verification
lines
>> listed
>> > in
>> > > > > > there, so it must have been working at some point.
(Which is
>> why I
>> > > was
>> > > > > > satisfied at the time to move on to something else.)
>> > > > > >
>> > > > > > Sounds like you are saying that I make that replacement
that the
>> > > thing
>> > > > > will
>> > > > > > now work?  Where is that replacement being done?  I don't
see
>> it in
>> > > the
>> > > > > > PB2NC config file.
>> > > > > >
>> > > > > > BTW - the full path to debufr is
/nwprod/util/exec/debufr.  Try
>> it
>> > on
>> > > > any
>> > > > > > prepbufr file, you get lots of information on each
observations.
>> > It
>> > > > > > produces a large output file, though, since it goes
through each
>> > > > > > observations individually.
>> > > > > >
>> > > > > > Thanks!
>> > > > > >
>> > > > > > Perry
>> > > > > >
>> > > > > > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via RT
<
>> > > > > > met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Perry,
>> > > > > > >
>> > > > > > > I ran through the debugger and found that TPHR is only
being
>> > > checked
>> > > > > for
>> > > > > > > AIRNOW observations:
>> > > > > > >
>> > > > > > >
>> > > > > > > *      // Special handling for "AIRNOW"      bool
is_airnow =
>> (0
>> > ==
>> > > > > > > strcmp("AIRNOW", hdr_typ));*
>> > > > > > >
>> > > > > > > And the message type for these obs is "ANOWPM".  I
tested this
>> > > > change:
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > *      // Special handling for "AIRNOW" and "ANOWPM"
bool
>> > > > > is_airnow
>> > > > > > =
>> > > > > > > (0 == strcmp("AIRNOW", hdr_typ) ||
>> > >   0
>> > > > > ==
>> > > > > > > strcmp("ANOWPM", hdr_typ));*
>> > > > > > >
>> > > > > > > And that makes PB2NC correctly handle the 1-hour
averaging
>> > period.
>> > > > > > >
>> > > > > > > Are you sure this was ever working?  I remember that we
got
>> > things
>> > > > > > working
>> > > > > > > for ozone, but don't remember the PM.
>> > > > > > >
>> > > > > > > Howard, do you have any recollection of this?
>> > > > > > >
>> > > > > > > The debufr tool sounds useful.  Is that on wcoss?  I
don't
>> have
>> > it
>> > > in
>> > > > > my
>> > > > > > > path and can't find it in a module.
>> > > > > > >
>> > > > > > > That Rscript can be found in
>> > > > > > >
>> > > > >
>> > >
>>
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > John
>> > > > > > >
>> > > > > > > On Wed, Jul 10, 2019 at 12:43 PM perry.shafran at noaa.gov
via
>> RT <
>> > > > > > > met_help at ucar.edu> wrote:
>> > > > > > >
>> > > > > > > >
>> > > > > > > > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>> > >
>> > > > > > > >
>> > > > > > > > Hi, John,
>> > > > > > > >
>> > > > > > > > One more thing.  I don't know how you get the
interval
>> you've
>> > > been
>> > > > > > > talking
>> > > > > > > > about, but I've been using a local tool called debufr
(I
>> don't
>> > > know
>> > > > > if
>> > > > > > > you
>> > > > > > > > have this tool), but it lists out what is in the
prepbufr
>> file
>> > > > > > directly.
>> > > > > > > > Seems the interval is gotten from a bufr variable
called
>> TPHR,
>> > > > which
>> > > > > is
>> > > > > > > > listed in the debufr output as "Time Period or
>> Displacement".
>> > > For
>> > > > > the
>> > > > > > > > ozone file these have values of -1.0 and -8.0 (note
the
>> minus
>> > > > signs).
>> > > > > > > For
>> > > > > > > > the PM file, they all have a value of -1.
>> > > > > > > >
>> > > > > > > > This is the same as in the 2018 file as it is for the
2019
>> > file.
>> > > > > > > >
>> > > > > > > > I'm also going to run PM using the change from A1 to
A0 as
>> you
>> > > have
>> > > > > > done.
>> > > > > > > >
>> > > > > > > > Perry
>> > > > > > > >
>> > > > > > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway
via RT <
>> > > > > > > > met_help at ucar.edu>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Perry,
>> > > > > > > > >
>> > > > > > > > > I found data to test this behavior out in this
directory:
>> > > > > > > > >
>> > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Here's the problem, pb2nc is reporting the
accumulation
>> > > interval
>> > > > > for
>> > > > > > > the
>> > > > > > > > > COPOPM observation as being 0.  Changing the Point-
Stat
>> > config
>> > > > file
>> > > > > > > like
>> > > > > > > > > this does indeed produce matched pairs:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *obs = {   convert(x) = x * 10^9;   field = [
{ name
>> =
>> > > > > "COPOPM";
>> > > > > > > > level
>> > > > > > > > > = "A0";' }   ];}*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > If I remember correctly though, these files should
contain
>> > both
>> > > > > > 1-hour
>> > > > > > > > and
>> > > > > > > > > 8-hour accumulations of COPOPM.  I do know that
this logic
>> > used
>> > > > to
>> > > > > > > > work...
>> > > > > > > > > and the fact that it's now NOT working with both
met-7.0
>> and
>> > > > > met-8.1
>> > > > > > > > means
>> > > > > > > > > that the most likely explanation is that the data
has
>> > changed.
>> > > > > > Perhaps
>> > > > > > > > > something changed in the data files after the
recent GFS
>> > > upgrade?
>> > > > > > > > >
>> > > > > > > > > We could test this theory like this...
>> > > > > > > > > 1. Get a PREPBUFR file from a year ago
>> (prepda.2018070712).
>> > > > > > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
>> > > > > > > > > 3. Run them both through PB2NC using the same
version of
>> MET
>> > > and
>> > > > > the
>> > > > > > > same
>> > > > > > > > > config file.
>> > > > > > > > > 4. Run both of them them through an Rscript to dump
the
>> > NetCDF
>> > > > obs
>> > > > > to
>> > > > > > > > ascii
>> > > > > > > > >
>> > > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
>> > > > > prepda.nc.2018070712
>> > > > > > > > > >prepda.nc.2018070712.txt
>> > > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
>> > > > > prepda.nc.2019070712
>> > > > > > > > > >prepda.nc.2019070712.txt
>> > > > > > > > >
>> > > > > > > > > Compare the 8-th column of the output.  The theory
is that
>> > the
>> > > > 2018
>> > > > > > > data
>> > > > > > > > > will have 1's and 8's there while the 2019 data
will have
>> all
>> > > > 0's.
>> > > > > > > > >
>> > > > > > > > > If that's the case, we'll need to fix pb2nc to
parse the
>> > > > > accumulation
>> > > > > > > > > interval correctly from the updated file format.
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > John
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Tue, Jul 2, 2019 at 1:44 PM
perry.shafran at noaa.gov via
>> > RT <
>> > > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>> > > > >
>> > > > > > > > > >
>> > > > > > > > > > Hi, John,
>> > > > > > > > > >
>> > > > > > > > > > OK, you can find things in the working directory:
>> > > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
>> > > > > > > > > >
>> > > > > > > > > > MET commands I've been using:
>> > > > > > > > > >
>> > > > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
>> > > > > > prepda.2019062912
>> > > > > > > > > > prepda.nc.2019062912
>> > > > > > > > > >
>> /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
>> > > -v 3
>> > > > > > > > > >
>> > > > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
>> > > > > > > AWIP3D06.tm00
>> > > > > > > > > > prepda.nc.2019062912
>> > > > > > > > > >
>> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
>> > > > > > -v
>> > > > > > > 3
>> > > > > > > > > >
>> > > > > > > > > > This is using 7.0 but I see the same problem with
the
>> > current
>> > > > > > version
>> > > > > > > > 8.1
>> > > > > > > > > > as well.
>> > > > > > > > > >
>> > > > > > > > > > Here is the output from point_stat:
>> > > > > > > > > >
>> > > > > > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
>> > > > > > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2: Reading data for PMTF/A1.
>> > > > > > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() -
Found
>> exact
>> > > > match
>> > > > > > for
>> > > > > > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2 file
>> > > > "AWIP3D06.tm00"
>> > > > > > > > > > DEBUG 3: Applying censor thresholds "NA" and
replacing
>> with
>> > > > > values
>> > > > > > > > > "(nul)".
>> > > > > > > > > > DEBUG 3: Censored values for 0 of 117130 grid
points.
>> > > > > > > > > > DEBUG 3: Use the matching forecast and
observation
>> grids.
>> > > > > > > > > > DEBUG 3: Grid Definition: Projection: Lambert
Conformal
>> Nx:
>> > > 442
>> > > > > Ny:
>> > > > > > > 265
>> > > > > > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient: 97.000
Alpha:
>> > > > 1037.975
>> > > > > > > Cone:
>> > > > > > > > > > 0.630 Bx: 208.6215 By: 7
>> > > > > > > > > > 84.2266
>> > > > > > > > > > DEBUG 2: Processing masking regions.
>> > > > > > > > > > DEBUG 3: Processing grid mask: FULL
>> > > > > > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
>> climatology
>> > > > mean
>> > > > > > > > levels,
>> > > > > > > > > > and 0 climatology standard deviation levels.
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2: Searching 21786 observations from 21786
>> messages.
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1, for
>> > observation
>> > > > > type
>> > > > > > > > > ANOWPM,
>> > > > > > > > > > over region FULL, for interpolation method
NEAREST(1),
>> > using
>> > > 0
>> > > > > > pairs.
>> > > > > > > > > > DEBUG 3: Number of matched pairs  = 0
>> > > > > > > > > > DEBUG 3: Observations processed   = 21786
>> > > > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
>> > > > > > > > > > DEBUG 3: Rejected: obs type       = 0
>> > > > > > > > > > DEBUG 3: Rejected: valid time     = 0
>> > > > > > > > > > DEBUG 3: Rejected: bad obs value  = 0
>> > > > > > > > > > DEBUG 3: Rejected: off the grid   = 1095
>> > > > > > > > > > DEBUG 3: Rejected: level mismatch = 20691
>> > > > > > > > > > DEBUG 3: Rejected: quality marker = 0
>> > > > > > > > > > DEBUG 3: Rejected: message type   = 0
>> > > > > > > > > > DEBUG 3: Rejected: masking region = 0
>> > > > > > > > > > DEBUG 3: Rejected: bad fcst value = 0
>> > > > > > > > > > DEBUG 3: Rejected: duplicates     = 0
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------
>> > > > > > > > > > DEBUG 2:
>> > > > > > > > > > DEBUG 1: Output file:
>> > > > > > > > > >
./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
>> > > > > > > > > >
>> > > > > > > > > > Perry
>> > > > > > > > > >
>> > > > > > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley Gotway
via
>> RT <
>> > > > > > > > > > met_help at ucar.edu>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Hi Perry,
>> > > > > > > > > > >
>> > > > > > > > > > > I can log on and take a look today.  Can you
point me
>> in
>> > > the
>> > > > > > right
>> > > > > > > > > > > direction on WCOSS?
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks,
>> > > > > > > > > > > John
>> > > > > > > > > > >
>> > > > > > > > > > > On Tue, Jul 2, 2019 at 8:13 AM
perry.shafran at noaa.gov
>> > via
>> > > > RT <
>> > > > > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was
acted
>> upon.
>> > > > > > > > > > > > Transaction: Ticket created by
>> perry.shafran at noaa.gov
>> > > > > > > > > > > >        Queue: met_help
>> > > > > > > > > > > >      Subject: Use of MET/METplus to verify
>> Particulate
>> > > > Matter
>> > > > > > > (PM)
>> > > > > > > > > > > >        Owner: Nobody
>> > > > > > > > > > > >   Requestors: perry.shafran at noaa.gov
>> > > > > > > > > > > >       Status: new
>> > > > > > > > > > > >  Ticket <URL:
>> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > Hi, there,
>> > > > > > > > > > > >
>> > > > > > > > > > > > I don't know if y'all recall, but you had
helped me
>> > > > sometime
>> > > > > > last
>> > > > > > > > > year
>> > > > > > > > > > to
>> > > > > > > > > > > > verify Particulate Matter (PM) using MET,
which at
>> one
>> > > time
>> > > > > had
>> > > > > > > > > worked
>> > > > > > > > > > > and
>> > > > > > > > > > > > gave me output in the .stat file using
point_stat.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Now, that seems to no longer work despite not
making
>> > any
>> > > > > > changes
>> > > > > > > to
>> > > > > > > > > any
>> > > > > > > > > > > of
>> > > > > > > > > > > > the configuration files.  Tara had asked me
to
>> re-run
>> > MET
>> > > > > using
>> > > > > > > an
>> > > > > > > > > > older
>> > > > > > > > > > > > version that had worked.
>> > > > > > > > > > > >
>> > > > > > > > > > > > MET 7.0 is still available on our machine (I
don't
>> > recall
>> > > > > when
>> > > > > > > that
>> > > > > > > > > was
>> > > > > > > > > > > > implemented), and I get the same result with
MET 7.0
>> > > that I
>> > > > > do
>> > > > > > > > when I
>> > > > > > > > > > run
>> > > > > > > > > > > > with MET 8.1.  Therefore, I'm not sure I can
get
>> back
>> > to
>> > > > what
>> > > > > > had
>> > > > > > > > > > worked.
>> > > > > > > > > > > >
>> > > > > > > > > > > > If anyone remembers helping with PM
previously and
>> has
>> > > > access
>> > > > > > to
>> > > > > > > > > > WCOSS, I
>> > > > > > > > > > > > can point you to data/scripts/commands and
see if
>> you
>> > can
>> > > > > help.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thanks!
>> > > > > > > > > > > >
>> > > > > > > > > > > > Perry
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>

------------------------------------------------
Subject: Use of MET/METplus to verify Particulate Matter (PM)
From: John Halley Gotway
Time: Thu Jul 11 09:31:40 2019

Great, thanks.  I'll go ahead and resolve this ticket.

John

On Thu, Jul 11, 2019 at 9:05 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
>
> FYI - using A0 in the config files (for both the observation and the
> forecast) instead of A1 does give output in the .stat file for PM
> verification.
>
> Therefore I can use this for PM verification and you don't need to
do
> anything quickly to get us 8.1.2.
>
> Thanks!
>
> Perry
>
> On Thu, Jul 11, 2019 at 8:59 AM Perry Shafran - NOAA Affiliate <
> perry.shafran at noaa.gov> wrote:
>
> > Hi, John,
> >
> > The sooner the better.  Folks here are beginning to get impatient
for AQ
> > verification.
> >
> > Is the workaround just to use A0 in the config files instead of
A1?  We
> > can use that as a temporary work-around, if need be.
> >
> > Thanks!
> >
> > Perry
> >
> > On Wed, Jul 10, 2019 at 6:17 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Perry,
> >>
> >> I did some forensic work on the repository and found exactly
where the
> >> ANOWPM logic got dropped.  I added the details to this GitHub
issue:
> >>    https://github.com/NCAR/MET/issues/1162
> >>
> >> Adding it back in to the master_v8.1 bugfix branch and the
develop
> branch
> >> (for future versions) is easy.  Figuring out when/how to get the
change
> to
> >> you is the hard part.  We just did the first met-8.1.1 bugfix
release on
> >> Monday.  This change will be included in met-8.1.2... but how
soon
> should
> >> we do that?
> >>
> >> Thanks,
> >> John
> >>
> >> On Wed, Jul 10, 2019 at 1:59 PM perry.shafran at noaa.gov via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >> >
> >> > Hi, John,
> >> >
> >> > Interesting.  I have no clue how that got dropped.
> >> >
> >> > Thanks for that info!
> >> >
> >> > Perry
> >> >
> >> > On Wed, Jul 10, 2019 at 3:57 PM John Halley Gotway via RT <
> >> > met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Perry,
> >> > >
> >> > > I grepped through the version control logs and found this
smoking
> gun:
> >> > >
> >> > > commit f88b9f36c3d73a9ed771e2d4995bc059b9b05df7
> >> > > Author: John Halley Gotway <johnhg at ucar.edu>
> >> > > Date:   Wed Mar 28 20:51:07 2018 +0000
> >> > >
> >> > >     Add the ANOWPM message type to the is_airnow boolean
logic.
> >> > >
> >> > > Now the question is, how did that changed get dropped from
the code?
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > > On Wed, Jul 10, 2019 at 1:38 PM perry.shafran at noaa.gov via RT
<
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874 >
> >> > > >
> >> > > > Hi, John,
> >> > > >
> >> > > > Those old stat files have "A1" in them.
> >> > > >
> >> > > > Perry
> >> > > >
> >> > > > On Wed, Jul 10, 2019 at 3:35 PM John Halley Gotway via RT <
> >> > > > met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Perry,
> >> > > > >
> >> > > > > Can you please take a look at those older stat files in
the
> >> "OBS_LEV"
> >> > > > > output column?  Does it say "A0" or "A1"?  Because using
"A0"
> will
> >> > > indeed
> >> > > > > work now.
> >> > > > >
> >> > > > > But searching through the source code for older versions
of
> >> pb2nc, I
> >> > > can
> >> > > > > find now reference to it ever including ANOWPM.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > John
> >> > > > >
> >> > > > > On Wed, Jul 10, 2019 at 1:26 PM perry.shafran at noaa.gov
via RT <
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > > >
> >> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >
> >> > > > > >
> >> > > > > > Hi, John,
> >> > > > > >
> >> > > > > > I clearly have older .stat files with PM verification
lines
> >> listed
> >> > in
> >> > > > > > there, so it must have been working at some point.
(Which is
> >> why I
> >> > > was
> >> > > > > > satisfied at the time to move on to something else.)
> >> > > > > >
> >> > > > > > Sounds like you are saying that I make that replacement
that
> the
> >> > > thing
> >> > > > > will
> >> > > > > > now work?  Where is that replacement being done?  I
don't see
> >> it in
> >> > > the
> >> > > > > > PB2NC config file.
> >> > > > > >
> >> > > > > > BTW - the full path to debufr is
/nwprod/util/exec/debufr.
> Try
> >> it
> >> > on
> >> > > > any
> >> > > > > > prepbufr file, you get lots of information on each
> observations.
> >> > It
> >> > > > > > produces a large output file, though, since it goes
through
> each
> >> > > > > > observations individually.
> >> > > > > >
> >> > > > > > Thanks!
> >> > > > > >
> >> > > > > > Perry
> >> > > > > >
> >> > > > > > On Wed, Jul 10, 2019 at 3:19 PM John Halley Gotway via
RT <
> >> > > > > > met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Perry,
> >> > > > > > >
> >> > > > > > > I ran through the debugger and found that TPHR is
only being
> >> > > checked
> >> > > > > for
> >> > > > > > > AIRNOW observations:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *      // Special handling for "AIRNOW"      bool
is_airnow
> =
> >> (0
> >> > ==
> >> > > > > > > strcmp("AIRNOW", hdr_typ));*
> >> > > > > > >
> >> > > > > > > And the message type for these obs is "ANOWPM".  I
tested
> this
> >> > > > change:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > *      // Special handling for "AIRNOW" and "ANOWPM"
> bool
> >> > > > > is_airnow
> >> > > > > > =
> >> > > > > > > (0 == strcmp("AIRNOW", hdr_typ) ||
> >> > >   0
> >> > > > > ==
> >> > > > > > > strcmp("ANOWPM", hdr_typ));*
> >> > > > > > >
> >> > > > > > > And that makes PB2NC correctly handle the 1-hour
averaging
> >> > period.
> >> > > > > > >
> >> > > > > > > Are you sure this was ever working?  I remember that
we got
> >> > things
> >> > > > > > working
> >> > > > > > > for ozone, but don't remember the PM.
> >> > > > > > >
> >> > > > > > > Howard, do you have any recollection of this?
> >> > > > > > >
> >> > > > > > > The debufr tool sounds useful.  Is that on wcoss?  I
don't
> >> have
> >> > it
> >> > > in
> >> > > > > my
> >> > > > > > > path and can't find it in a module.
> >> > > > > > >
> >> > > > > > > That Rscript can be found in
> >> > > > > > >
> >> > > > >
> >> > >
> >>
>
/global/noscrub/Julie.Prestopnik/met/8.1/share/met/Rscripts/pntnc2ascii.R
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > John
> >> > > > > > >
> >> > > > > > > On Wed, Jul 10, 2019 at 12:43 PM
perry.shafran at noaa.gov via
> >> RT <
> >> > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >> > >
> >> > > > > > > >
> >> > > > > > > > Hi, John,
> >> > > > > > > >
> >> > > > > > > > One more thing.  I don't know how you get the
interval
> >> you've
> >> > > been
> >> > > > > > > talking
> >> > > > > > > > about, but I've been using a local tool called
debufr (I
> >> don't
> >> > > know
> >> > > > > if
> >> > > > > > > you
> >> > > > > > > > have this tool), but it lists out what is in the
prepbufr
> >> file
> >> > > > > > directly.
> >> > > > > > > > Seems the interval is gotten from a bufr variable
called
> >> TPHR,
> >> > > > which
> >> > > > > is
> >> > > > > > > > listed in the debufr output as "Time Period or
> >> Displacement".
> >> > > For
> >> > > > > the
> >> > > > > > > > ozone file these have values of -1.0 and -8.0 (note
the
> >> minus
> >> > > > signs).
> >> > > > > > > For
> >> > > > > > > > the PM file, they all have a value of -1.
> >> > > > > > > >
> >> > > > > > > > This is the same as in the 2018 file as it is for
the 2019
> >> > file.
> >> > > > > > > >
> >> > > > > > > > I'm also going to run PM using the change from A1
to A0 as
> >> you
> >> > > have
> >> > > > > > done.
> >> > > > > > > >
> >> > > > > > > > Perry
> >> > > > > > > >
> >> > > > > > > > On Wed, Jul 10, 2019 at 2:00 PM John Halley Gotway
via RT
> <
> >> > > > > > > > met_help at ucar.edu>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Perry,
> >> > > > > > > > >
> >> > > > > > > > > I found data to test this behavior out in this
> directory:
> >> > > > > > > > >
> >> > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwprd/verf_gridtobs_pm_12/12
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Here's the problem, pb2nc is reporting the
accumulation
> >> > > interval
> >> > > > > for
> >> > > > > > > the
> >> > > > > > > > > COPOPM observation as being 0.  Changing the
Point-Stat
> >> > config
> >> > > > file
> >> > > > > > > like
> >> > > > > > > > > this does indeed produce matched pairs:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *obs = {   convert(x) = x * 10^9;   field = [
{
> name
> >> =
> >> > > > > "COPOPM";
> >> > > > > > > > level
> >> > > > > > > > > = "A0";' }   ];}*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > If I remember correctly though, these files
should
> contain
> >> > both
> >> > > > > > 1-hour
> >> > > > > > > > and
> >> > > > > > > > > 8-hour accumulations of COPOPM.  I do know that
this
> logic
> >> > used
> >> > > > to
> >> > > > > > > > work...
> >> > > > > > > > > and the fact that it's now NOT working with both
met-7.0
> >> and
> >> > > > > met-8.1
> >> > > > > > > > means
> >> > > > > > > > > that the most likely explanation is that the data
has
> >> > changed.
> >> > > > > > Perhaps
> >> > > > > > > > > something changed in the data files after the
recent GFS
> >> > > upgrade?
> >> > > > > > > > >
> >> > > > > > > > > We could test this theory like this...
> >> > > > > > > > > 1. Get a PREPBUFR file from a year ago
> >> (prepda.2018070712).
> >> > > > > > > > > 2. Get a recent PREPBUFR file (prepda.201907012).
> >> > > > > > > > > 3. Run them both through PB2NC using the same
version of
> >> MET
> >> > > and
> >> > > > > the
> >> > > > > > > same
> >> > > > > > > > > config file.
> >> > > > > > > > > 4. Run both of them them through an Rscript to
dump the
> >> > NetCDF
> >> > > > obs
> >> > > > > to
> >> > > > > > > > ascii
> >> > > > > > > > >
> >> > > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> >> > > > > prepda.nc.2018070712
> >> > > > > > > > > >prepda.nc.2018070712.txt
> >> > > > > > > > > Rscript met/8.1/share/met/Rscripts/pntnc2ascii.R
> >> > > > > prepda.nc.2019070712
> >> > > > > > > > > >prepda.nc.2019070712.txt
> >> > > > > > > > >
> >> > > > > > > > > Compare the 8-th column of the output.  The
theory is
> that
> >> > the
> >> > > > 2018
> >> > > > > > > data
> >> > > > > > > > > will have 1's and 8's there while the 2019 data
will
> have
> >> all
> >> > > > 0's.
> >> > > > > > > > >
> >> > > > > > > > > If that's the case, we'll need to fix pb2nc to
parse the
> >> > > > > accumulation
> >> > > > > > > > > interval correctly from the updated file format.
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > John
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Tue, Jul 2, 2019 at 1:44 PM
perry.shafran at noaa.gov
> via
> >> > RT <
> >> > > > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > <URL:
> >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >> > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Hi, John,
> >> > > > > > > > > >
> >> > > > > > > > > > OK, you can find things in the working
directory:
> >> > > > > > > > > >
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pm_12/12
> >> > > > > > > > > >
> >> > > > > > > > > > MET commands I've been using:
> >> > > > > > > > > >
> >> > > > > > > > > >
/global/noscrub/Julie.Prestopnik/met/7.0/bin/pb2nc
> >> > > > > > prepda.2019062912
> >> > > > > > > > > > prepda.nc.2019062912
> >> > > > > > > > > >
> >> /meso/save/Perry.Shafran/verif/nwtest/parm/PB2NCConfig_aqm
> >> > > -v 3
> >> > > > > > > > > >
> >> > > > > > > > > >
> /global/noscrub/Julie.Prestopnik/met/7.0/bin/point_stat
> >> > > > > > > AWIP3D06.tm00
> >> > > > > > > > > > prepda.nc.2019062912
> >> > > > > > > > > >
> >> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM
> >> > > > > > -v
> >> > > > > > > 3
> >> > > > > > > > > >
> >> > > > > > > > > > This is using 7.0 but I see the same problem
with the
> >> > current
> >> > > > > > version
> >> > > > > > > > 8.1
> >> > > > > > > > > > as well.
> >> > > > > > > > > >
> >> > > > > > > > > > Here is the output from point_stat:
> >> > > > > > > > > >
> >> > > > > > > > > > DEBUG 1: Forecast File: AWIP3D06.tm00
> >> > > > > > > > > > DEBUG 1: Observation File: prepda.nc.2019062912
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2: Reading data for PMTF/A1.
> >> > > > > > > > > > DEBUG 3: MetGrib2DataFile::data_plane_array() -
Found
> >> exact
> >> > > > match
> >> > > > > > for
> >> > > > > > > > > > "PMTF/A1" in GRIB2 record 1 field 1 of GRIB2
file
> >> > > > "AWIP3D06.tm00"
> >> > > > > > > > > > DEBUG 3: Applying censor thresholds "NA" and
replacing
> >> with
> >> > > > > values
> >> > > > > > > > > "(nul)".
> >> > > > > > > > > > DEBUG 3: Censored values for 0 of 117130 grid
points.
> >> > > > > > > > > > DEBUG 3: Use the matching forecast and
observation
> >> grids.
> >> > > > > > > > > > DEBUG 3: Grid Definition: Projection: Lambert
> Conformal
> >> Nx:
> >> > > 442
> >> > > > > Ny:
> >> > > > > > > 265
> >> > > > > > > > > > Lat_LL: 21.821 Lon_LL: 120.628 Lon_orient:
97.000
> Alpha:
> >> > > > 1037.975
> >> > > > > > > Cone:
> >> > > > > > > > > > 0.630 Bx: 208.6215 By: 7
> >> > > > > > > > > > 84.2266
> >> > > > > > > > > > DEBUG 2: Processing masking regions.
> >> > > > > > > > > > DEBUG 3: Processing grid mask: FULL
> >> > > > > > > > > > DEBUG 2: For PMTF/A1 found 1 forecast levels, 0
> >> climatology
> >> > > > mean
> >> > > > > > > > levels,
> >> > > > > > > > > > and 0 climatology standard deviation levels.
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2: Searching 21786 observations from
21786
> >> messages.
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2: Processing PMTF/A1 versus COPOPM/A1,
for
> >> > observation
> >> > > > > type
> >> > > > > > > > > ANOWPM,
> >> > > > > > > > > > over region FULL, for interpolation method
NEAREST(1),
> >> > using
> >> > > 0
> >> > > > > > pairs.
> >> > > > > > > > > > DEBUG 3: Number of matched pairs  = 0
> >> > > > > > > > > > DEBUG 3: Observations processed   = 21786
> >> > > > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> >> > > > > > > > > > DEBUG 3: Rejected: obs type       = 0
> >> > > > > > > > > > DEBUG 3: Rejected: valid time     = 0
> >> > > > > > > > > > DEBUG 3: Rejected: bad obs value  = 0
> >> > > > > > > > > > DEBUG 3: Rejected: off the grid   = 1095
> >> > > > > > > > > > DEBUG 3: Rejected: level mismatch = 20691
> >> > > > > > > > > > DEBUG 3: Rejected: quality marker = 0
> >> > > > > > > > > > DEBUG 3: Rejected: message type   = 0
> >> > > > > > > > > > DEBUG 3: Rejected: masking region = 0
> >> > > > > > > > > > DEBUG 3: Rejected: bad fcst value = 0
> >> > > > > > > > > > DEBUG 3: Rejected: duplicates     = 0
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------
> >> > > > > > > > > > DEBUG 2:
> >> > > > > > > > > > DEBUG 1: Output file:
> >> > > > > > > > > >
./point_stat_CMAQ_ANOWPM_060000L_20190629_120000V.stat
> >> > > > > > > > > >
> >> > > > > > > > > > Perry
> >> > > > > > > > > >
> >> > > > > > > > > > On Tue, Jul 2, 2019 at 3:04 PM John Halley
Gotway via
> >> RT <
> >> > > > > > > > > > met_help at ucar.edu>
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > Hi Perry,
> >> > > > > > > > > > >
> >> > > > > > > > > > > I can log on and take a look today.  Can you
point
> me
> >> in
> >> > > the
> >> > > > > > right
> >> > > > > > > > > > > direction on WCOSS?
> >> > > > > > > > > > >
> >> > > > > > > > > > > Thanks,
> >> > > > > > > > > > > John
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Tue, Jul 2, 2019 at 8:13 AM
> perry.shafran at noaa.gov
> >> > via
> >> > > > RT <
> >> > > > > > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Tue Jul 02 08:13:23 2019: Request 90874 was
acted
> >> upon.
> >> > > > > > > > > > > > Transaction: Ticket created by
> >> perry.shafran at noaa.gov
> >> > > > > > > > > > > >        Queue: met_help
> >> > > > > > > > > > > >      Subject: Use of MET/METplus to verify
> >> Particulate
> >> > > > Matter
> >> > > > > > > (PM)
> >> > > > > > > > > > > >        Owner: Nobody
> >> > > > > > > > > > > >   Requestors: perry.shafran at noaa.gov
> >> > > > > > > > > > > >       Status: new
> >> > > > > > > > > > > >  Ticket <URL:
> >> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90874
> >> > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Hi, there,
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > I don't know if y'all recall, but you had
helped
> me
> >> > > > sometime
> >> > > > > > last
> >> > > > > > > > > year
> >> > > > > > > > > > to
> >> > > > > > > > > > > > verify Particulate Matter (PM) using MET,
which at
> >> one
> >> > > time
> >> > > > > had
> >> > > > > > > > > worked
> >> > > > > > > > > > > and
> >> > > > > > > > > > > > gave me output in the .stat file using
point_stat.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Now, that seems to no longer work despite
not
> making
> >> > any
> >> > > > > > changes
> >> > > > > > > to
> >> > > > > > > > > any
> >> > > > > > > > > > > of
> >> > > > > > > > > > > > the configuration files.  Tara had asked me
to
> >> re-run
> >> > MET
> >> > > > > using
> >> > > > > > > an
> >> > > > > > > > > > older
> >> > > > > > > > > > > > version that had worked.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > MET 7.0 is still available on our machine
(I don't
> >> > recall
> >> > > > > when
> >> > > > > > > that
> >> > > > > > > > > was
> >> > > > > > > > > > > > implemented), and I get the same result
with MET
> 7.0
> >> > > that I
> >> > > > > do
> >> > > > > > > > when I
> >> > > > > > > > > > run
> >> > > > > > > > > > > > with MET 8.1.  Therefore, I'm not sure I
can get
> >> back
> >> > to
> >> > > > what
> >> > > > > > had
> >> > > > > > > > > > worked.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > If anyone remembers helping with PM
previously and
> >> has
> >> > > > access
> >> > > > > > to
> >> > > > > > > > > > WCOSS, I
> >> > > > > > > > > > > > can point you to data/scripts/commands and
see if
> >> you
> >> > can
> >> > > > > help.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Thanks!
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Perry
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
>
>

------------------------------------------------