[Met_help] [rt.rap.ucar.edu #86834] History for Re: PM daily max and mean verification

Mon Sep 24 08:38:55 MDT 2018

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi there,

I posted this 7 days ago and was wondering if we can get a response.  We
need to get AQM working so we can set up the AQ use case for METplus for
the workshop.

Thanks!

Perry

On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA Affiliate <
perry.shafran at noaa.gov> wrote:

> Hi, there,
>
> I am having some issues reading the max PM file.  The wgrib2 output looks
> like this:
>
> 37 (t10a1) /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 > wgrib2
> pm25_DAY1
> 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave fcst:
> 38 (t10a1) /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
>
> I have this in my point_stat config file:
>
> //
> // Forecast and observation fields to be verified
> //
>
> fcst = {
>    field = [
>       { name = "PDMAX1"; level = "A1"; valid_time="${CUR_VX_TIME1}"; }
>    ];
> }
>
> obs = {
>    convert(x) = x * 10^9;
>    field = [
>       { name = "COPOPM"; level = "A1"; message_type = "ANOWPM_MAX_230000";
> }
>    ];
> }
>
> But then I get this error:
>
> DEBUG 1: Forecast File: pm25_DAY1
> DEBUG 1: Observation File: prepda.2018082512.nc
> DEBUG 2:
> DEBUG 2:
> --------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for PDMAX1/A1.
> WARNING:
> WARNING: MetGrib2DataFile::data_plane_array() - No matching records found
> for 'PDMAX1/A1'
> WARNING:
> WARNING:
> WARNING: process_fcst_climo_files() -> no fields matching PDMAX1/A1 found
> in file: pm25_DAY1
> WARNING:
> ERROR  :
> ERROR  : process_fcst_climo_files() -> no requested forecast data found!
> Exiting...
> ERROR  :
>
> Might something be wrong with the CUR_VX_TIME1 in this particular case?
> You can find these files in the data already existing on Theia in, for
> example, the
> /scratch4/NCEPDEV/fv3-cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
>
> Thanks!
>
> Perry
>

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: PM daily max and mean verification
From: Julie Prestopnik
Time: Tue Sep 04 13:28:02 2018

Hi Perry.

I tried to access the data at
tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
unfortunately, I see it is no longer there.

Could you please do the following?

   1. Send me the full command line that you are running.
   2. Point me to your exact fcst, obs, and config files (machine and
   location information)
   3. Tell me what version of the code you are running - e.g. output
from
   "point_stat --version "

I will try to reproduce the error that you are getting and will follow
up.

Thanks,
Julie

On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> Transaction: Ticket created by perry.shafran at noaa.gov
>        Queue: met_help
>      Subject: Re: PM daily max and mean verification
>        Owner: Nobody
>   Requestors: perry.shafran at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
>
>
> Hi there,
>
> I posted this 7 days ago and was wondering if we can get a response.
We
> need to get AQM working so we can set up the AQ use case for METplus
for
> the workshop.
>
> Thanks!
>
> Perry
>
> On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA Affiliate <
> perry.shafran at noaa.gov> wrote:
>
> > Hi, there,
> >
> > I am having some issues reading the max PM file.  The wgrib2
output looks
> > like this:
> >
> > 37 (t10a1) /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
>
> wgrib2
> > pm25_DAY1
> > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave fcst:
> > 38 (t10a1) /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> >
> > I have this in my point_stat config file:
> >
> > //
> > // Forecast and observation fields to be verified
> > //
> >
> > fcst = {
> >    field = [
> >       { name = "PDMAX1"; level = "A1";
valid_time="${CUR_VX_TIME1}"; }
> >    ];
> > }
> >
> > obs = {
> >    convert(x) = x * 10^9;
> >    field = [
> >       { name = "COPOPM"; level = "A1"; message_type =
> "ANOWPM_MAX_230000";
> > }
> >    ];
> > }
> >
> > But then I get this error:
> >
> > DEBUG 1: Forecast File: pm25_DAY1
> > DEBUG 1: Observation File: prepda.2018082512.nc
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for PDMAX1/A1.
> > WARNING:
> > WARNING: MetGrib2DataFile::data_plane_array() - No matching
records found
> > for 'PDMAX1/A1'
> > WARNING:
> > WARNING:
> > WARNING: process_fcst_climo_files() -> no fields matching
PDMAX1/A1 found
> > in file: pm25_DAY1
> > WARNING:
> > ERROR  :
> > ERROR  : process_fcst_climo_files() -> no requested forecast data
found!
> > Exiting...
> > ERROR  :
> >
> > Might something be wrong with the CUR_VX_TIME1 in this particular
case?
> > You can find these files in the data already existing on Theia in,
for
> > example, the
> > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> >
> > Thanks!
> >
> > Perry
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Tue Sep 04 14:34:59 2018

Hi, Julie,

I will get on that tomorrow.  John had been helping me with the AQ
verification previously, which is a little tricky, but I will give you
the
information and see if you can help.

Thanks!

Perry

On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Hi Perry.
>
> I tried to access the data at
> tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
> unfortunately, I see it is no longer there.
>
> Could you please do the following?
>
>    1. Send me the full command line that you are running.
>    2. Point me to your exact fcst, obs, and config files (machine
and
>    location information)
>    3. Tell me what version of the code you are running - e.g. output
from
>    "point_stat --version "
>
> I will try to reproduce the error that you are getting and will
follow up.
>
> Thanks,
> Julie
>
> On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > Transaction: Ticket created by perry.shafran at noaa.gov
> >        Queue: met_help
> >      Subject: Re: PM daily max and mean verification
> >        Owner: Nobody
> >   Requestors: perry.shafran at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> >
> > Hi there,
> >
> > I posted this 7 days ago and was wondering if we can get a
response.  We
> > need to get AQM working so we can set up the AQ use case for
METplus for
> > the workshop.
> >
> > Thanks!
> >
> > Perry
> >
> > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA Affiliate <
> > perry.shafran at noaa.gov> wrote:
> >
> > > Hi, there,
> > >
> > > I am having some issues reading the max PM file.  The wgrib2
output
> looks
> > > like this:
> > >
> > > 37 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > wgrib2
> > > pm25_DAY1
> > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave fcst:
> > > 38 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > >
> > > I have this in my point_stat config file:
> > >
> > > //
> > > // Forecast and observation fields to be verified
> > > //
> > >
> > > fcst = {
> > >    field = [
> > >       { name = "PDMAX1"; level = "A1";
valid_time="${CUR_VX_TIME1}"; }
> > >    ];
> > > }
> > >
> > > obs = {
> > >    convert(x) = x * 10^9;
> > >    field = [
> > >       { name = "COPOPM"; level = "A1"; message_type =
> > "ANOWPM_MAX_230000";
> > > }
> > >    ];
> > > }
> > >
> > > But then I get this error:
> > >
> > > DEBUG 1: Forecast File: pm25_DAY1
> > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for PDMAX1/A1.
> > > WARNING:
> > > WARNING: MetGrib2DataFile::data_plane_array() - No matching
records
> found
> > > for 'PDMAX1/A1'
> > > WARNING:
> > > WARNING:
> > > WARNING: process_fcst_climo_files() -> no fields matching
PDMAX1/A1
> found
> > > in file: pm25_DAY1
> > > WARNING:
> > > ERROR  :
> > > ERROR  : process_fcst_climo_files() -> no requested forecast
data
> found!
> > > Exiting...
> > > ERROR  :
> > >
> > > Might something be wrong with the CUR_VX_TIME1 in this
particular case?
> > > You can find these files in the data already existing on Theia
in, for
> > > example, the
> > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Wed Sep 05 08:14:51 2018

Hi, Julie,

Maybe I should have put this on the original thread regarding AQ
verification with John, but that was not a ticket so I guess it's good
to
continue here - but you may not be aware of the genesis of AQ
verification
up to this point.

I re-ran my verification job so the directory was regenerated:
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12

Here's the command:

point_stat pm25_DAY1 prepda.2018090212.nc
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
-v 4

Average and max PM is weird because it uses 2 obs files but writes it
out
into one file as it sums up everything.  Here is the pb2nc command:

pb2nc prepda.2018090212 prepda.2018090212.nc
/meso/save/Perry.Shafran/verif/nwtest/
parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112

These commands tell you where the config files that I used are so you
may
look at them.

Here's the version.

4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat --version

MET Version:    V7.1
Repository:    https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
Revision:    5554
Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)

I had gotten things to work for the daily max ozone items but I need
to
work on the average and daily max PM.  I am cc'ing John so he has this
as
well and can take a look.

Thanks!

Perry

5 (t14a1) /ptmpp2/Perry.Shafran/output

On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Hi Perry.
>
> I tried to access the data at
> tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
> unfortunately, I see it is no longer there.
>
> Could you please do the following?
>
>    1. Send me the full command line that you are running.
>    2. Point me to your exact fcst, obs, and config files (machine
and
>    location information)
>    3. Tell me what version of the code you are running - e.g. output
from
>    "point_stat --version "
>
> I will try to reproduce the error that you are getting and will
follow up.
>
> Thanks,
> Julie
>
> On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > Transaction: Ticket created by perry.shafran at noaa.gov
> >        Queue: met_help
> >      Subject: Re: PM daily max and mean verification
> >        Owner: Nobody
> >   Requestors: perry.shafran at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> >
> > Hi there,
> >
> > I posted this 7 days ago and was wondering if we can get a
response.  We
> > need to get AQM working so we can set up the AQ use case for
METplus for
> > the workshop.
> >
> > Thanks!
> >
> > Perry
> >
> > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA Affiliate <
> > perry.shafran at noaa.gov> wrote:
> >
> > > Hi, there,
> > >
> > > I am having some issues reading the max PM file.  The wgrib2
output
> looks
> > > like this:
> > >
> > > 37 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > wgrib2
> > > pm25_DAY1
> > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave fcst:
> > > 38 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > >
> > > I have this in my point_stat config file:
> > >
> > > //
> > > // Forecast and observation fields to be verified
> > > //
> > >
> > > fcst = {
> > >    field = [
> > >       { name = "PDMAX1"; level = "A1";
valid_time="${CUR_VX_TIME1}"; }
> > >    ];
> > > }
> > >
> > > obs = {
> > >    convert(x) = x * 10^9;
> > >    field = [
> > >       { name = "COPOPM"; level = "A1"; message_type =
> > "ANOWPM_MAX_230000";
> > > }
> > >    ];
> > > }
> > >
> > > But then I get this error:
> > >
> > > DEBUG 1: Forecast File: pm25_DAY1
> > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for PDMAX1/A1.
> > > WARNING:
> > > WARNING: MetGrib2DataFile::data_plane_array() - No matching
records
> found
> > > for 'PDMAX1/A1'
> > > WARNING:
> > > WARNING:
> > > WARNING: process_fcst_climo_files() -> no fields matching
PDMAX1/A1
> found
> > > in file: pm25_DAY1
> > > WARNING:
> > > ERROR  :
> > > ERROR  : process_fcst_climo_files() -> no requested forecast
data
> found!
> > > Exiting...
> > > ERROR  :
> > >
> > > Might something be wrong with the CUR_VX_TIME1 in this
particular case?
> > > You can find these files in the data already existing on Theia
in, for
> > > example, the
> > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: Julie Prestopnik
Time: Wed Sep 05 12:39:13 2018

Hi Perry.

Thanks for the information.  I am definitely not familiar with any of
this
verification, nor do I know what AQ stands for.  I figured I'd enroll
John's help if necessary.  In the meantime, I thought I'd at least try
to
reproduce the problem you're experiencing.

Are you calling point_stat from a script or running it on the command
line?  If you are running from a script, can you please point me to
it?

I see a couple of environment variables in
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
Can you please tell me what I should set MODEL and CUR_VX_TIME1 to?

Thanks,
Julie

On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
>
> Hi, Julie,
>
> Maybe I should have put this on the original thread regarding AQ
> verification with John, but that was not a ticket so I guess it's
good to
> continue here - but you may not be aware of the genesis of AQ
verification
> up to this point.
>
> I re-ran my verification job so the directory was regenerated:
> /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
>
> Here's the command:
>
> point_stat pm25_DAY1 prepda.2018090212.nc
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
-v 4
>
> Average and max PM is weird because it uses 2 obs files but writes
it out
> into one file as it sums up everything.  Here is the pb2nc command:
>
> pb2nc prepda.2018090212 prepda.2018090212.nc
> /meso/save/Perry.Shafran/verif/nwtest/
> parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
>
> These commands tell you where the config files that I used are so
you may
> look at them.
>
> Here's the version.
>
> 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat --version
>
> MET Version:    V7.1
> Repository:    https://svn-met-dev.cgd.ucar.edu/tags/met/met-
7.1_beta6
> Revision:    5554
> Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)
>
> I had gotten things to work for the daily max ozone items but I need
to
> work on the average and daily max PM.  I am cc'ing John so he has
this as
> well and can take a look.
>
> Thanks!
>
> Perry
>
> 5 (t14a1) /ptmpp2/Perry.Shafran/output
>
> On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Perry.
> >
> > I tried to access the data at
> > tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
> > unfortunately, I see it is no longer there.
> >
> > Could you please do the following?
> >
> >    1. Send me the full command line that you are running.
> >    2. Point me to your exact fcst, obs, and config files (machine
and
> >    location information)
> >    3. Tell me what version of the code you are running - e.g.
output from
> >    "point_stat --version "
> >
> > I will try to reproduce the error that you are getting and will
follow
> up.
> >
> > Thanks,
> > Julie
> >
> > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > > Transaction: Ticket created by perry.shafran at noaa.gov
> > >        Queue: met_help
> > >      Subject: Re: PM daily max and mean verification
> > >        Owner: Nobody
> > >   Requestors: perry.shafran at noaa.gov
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> >
> > >
> > >
> > > Hi there,
> > >
> > > I posted this 7 days ago and was wondering if we can get a
response.
> We
> > > need to get AQM working so we can set up the AQ use case for
METplus
> for
> > > the workshop.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA Affiliate <
> > > perry.shafran at noaa.gov> wrote:
> > >
> > > > Hi, there,
> > > >
> > > > I am having some issues reading the max PM file.  The wgrib2
output
> > looks
> > > > like this:
> > > >
> > > > 37 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > wgrib2
> > > > pm25_DAY1
> > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave
fcst:
> > > > 38 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > >
> > > > I have this in my point_stat config file:
> > > >
> > > > //
> > > > // Forecast and observation fields to be verified
> > > > //
> > > >
> > > > fcst = {
> > > >    field = [
> > > >       { name = "PDMAX1"; level = "A1";
valid_time="${CUR_VX_TIME1}";
> }
> > > >    ];
> > > > }
> > > >
> > > > obs = {
> > > >    convert(x) = x * 10^9;
> > > >    field = [
> > > >       { name = "COPOPM"; level = "A1"; message_type =
> > > "ANOWPM_MAX_230000";
> > > > }
> > > >    ];
> > > > }
> > > >
> > > > But then I get this error:
> > > >
> > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > DEBUG 2:
> > > > DEBUG 2:
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > DEBUG 2:
> > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > WARNING:
> > > > WARNING: MetGrib2DataFile::data_plane_array() - No matching
records
> > found
> > > > for 'PDMAX1/A1'
> > > > WARNING:
> > > > WARNING:
> > > > WARNING: process_fcst_climo_files() -> no fields matching
PDMAX1/A1
> > found
> > > > in file: pm25_DAY1
> > > > WARNING:
> > > > ERROR  :
> > > > ERROR  : process_fcst_climo_files() -> no requested forecast
data
> > found!
> > > > Exiting...
> > > > ERROR  :
> > > >
> > > > Might something be wrong with the CUR_VX_TIME1 in this
particular
> case?
> > > > You can find these files in the data already existing on Theia
in,
> for
> > > > example, the
> > > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Wed Sep 05 12:48:38 2018

Hi, Julie,

Ah yes, I am setting it in a script.  Here are the environment
variables
you'll need:

CUR_VX_TIME1=20180902_05
MODEL=CMAQ
OBS=ANOWPM

On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Hi Perry.
>
> Thanks for the information.  I am definitely not familiar with any
of this
> verification, nor do I know what AQ stands for.  I figured I'd
enroll
> John's help if necessary.  In the meantime, I thought I'd at least
try to
> reproduce the problem you're experiencing.
>
> Are you calling point_stat from a script or running it on the
command
> line?  If you are running from a script, can you please point me to
it?
>
> I see a couple of environment variables in
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> Can you please tell me what I should set MODEL and CUR_VX_TIME1 to?
>
> Thanks,
> Julie
>
> On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> > Hi, Julie,
> >
> > Maybe I should have put this on the original thread regarding AQ
> > verification with John, but that was not a ticket so I guess it's
good to
> > continue here - but you may not be aware of the genesis of AQ
> verification
> > up to this point.
> >
> > I re-ran my verification job so the directory was regenerated:
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> >
> > Here's the command:
> >
> > point_stat pm25_DAY1 prepda.2018090212.nc
> >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
-v
> 4
> >
> > Average and max PM is weird because it uses 2 obs files but writes
it out
> > into one file as it sums up everything.  Here is the pb2nc
command:
> >
> > pb2nc prepda.2018090212 prepda.2018090212.nc
> > /meso/save/Perry.Shafran/verif/nwtest/
> > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> >
> > These commands tell you where the config files that I used are so
you may
> > look at them.
> >
> > Here's the version.
> >
> > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat --version
> >
> > MET Version:    V7.1
> > Repository:    https://svn-met-dev.cgd.ucar.edu/tags/met/met-
7.1_beta6
> > Revision:    5554
> > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)
> >
> > I had gotten things to work for the daily max ozone items but I
need to
> > work on the average and daily max PM.  I am cc'ing John so he has
this as
> > well and can take a look.
> >
> > Thanks!
> >
> > Perry
> >
> > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> >
> > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry.
> > >
> > > I tried to access the data at
> > > tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
> > > unfortunately, I see it is no longer there.
> > >
> > > Could you please do the following?
> > >
> > >    1. Send me the full command line that you are running.
> > >    2. Point me to your exact fcst, obs, and config files
(machine and
> > >    location information)
> > >    3. Tell me what version of the code you are running - e.g.
output
> from
> > >    "point_stat --version "
> > >
> > > I will try to reproduce the error that you are getting and will
follow
> > up.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > >        Queue: met_help
> > > >      Subject: Re: PM daily max and mean verification
> > > >        Owner: Nobody
> > > >   Requestors: perry.shafran at noaa.gov
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > >
> > > >
> > > >
> > > > Hi there,
> > > >
> > > > I posted this 7 days ago and was wondering if we can get a
response.
> > We
> > > > need to get AQM working so we can set up the AQ use case for
METplus
> > for
> > > > the workshop.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA Affiliate
<
> > > > perry.shafran at noaa.gov> wrote:
> > > >
> > > > > Hi, there,
> > > > >
> > > > > I am having some issues reading the max PM file.  The wgrib2
output
> > > looks
> > > > > like this:
> > > > >
> > > > > 37 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > wgrib2
> > > > > pm25_DAY1
> > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave
fcst:
> > > > > 38 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > >
> > > > > I have this in my point_stat config file:
> > > > >
> > > > > //
> > > > > // Forecast and observation fields to be verified
> > > > > //
> > > > >
> > > > > fcst = {
> > > > >    field = [
> > > > >       { name = "PDMAX1"; level = "A1";
> valid_time="${CUR_VX_TIME1}";
> > }
> > > > >    ];
> > > > > }
> > > > >
> > > > > obs = {
> > > > >    convert(x) = x * 10^9;
> > > > >    field = [
> > > > >       { name = "COPOPM"; level = "A1"; message_type =
> > > > "ANOWPM_MAX_230000";
> > > > > }
> > > > >    ];
> > > > > }
> > > > >
> > > > > But then I get this error:
> > > > >
> > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > DEBUG 2:
> > > > > DEBUG 2:
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > DEBUG 2:
> > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > WARNING:
> > > > > WARNING: MetGrib2DataFile::data_plane_array() - No matching
records
> > > found
> > > > > for 'PDMAX1/A1'
> > > > > WARNING:
> > > > > WARNING:
> > > > > WARNING: process_fcst_climo_files() -> no fields matching
PDMAX1/A1
> > > found
> > > > > in file: pm25_DAY1
> > > > > WARNING:
> > > > > ERROR  :
> > > > > ERROR  : process_fcst_climo_files() -> no requested forecast
data
> > > found!
> > > > > Exiting...
> > > > > ERROR  :
> > > > >
> > > > > Might something be wrong with the CUR_VX_TIME1 in this
particular
> > case?
> > > > > You can find these files in the data already existing on
Theia in,
> > for
> > > > > example, the
> > > > >
> /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: Julie Prestopnik
Time: Thu Sep 06 11:32:52 2018

Thank you, Perry.  Fortunately, I am able to reproduce the problem.
Unfortunately, I believe I need John's help.  He is working very hard
on
trying to get the met-8.0 release out, however.  Please know that we
haven't forgotten about you and will get back to you as soon as we
can.

Thank you!

Julie

On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
>
> Hi, Julie,
>
> Ah yes, I am setting it in a script.  Here are the environment
variables
> you'll need:
>
> CUR_VX_TIME1=20180902_05
> MODEL=CMAQ
> OBS=ANOWPM
>
>
>
> On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Perry.
> >
> > Thanks for the information.  I am definitely not familiar with any
of
> this
> > verification, nor do I know what AQ stands for.  I figured I'd
enroll
> > John's help if necessary.  In the meantime, I thought I'd at least
try to
> > reproduce the problem you're experiencing.
> >
> > Are you calling point_stat from a script or running it on the
command
> > line?  If you are running from a script, can you please point me
to it?
> >
> > I see a couple of environment variables in
> >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > Can you please tell me what I should set MODEL and CUR_VX_TIME1
to?
> >
> > Thanks,
> > Julie
> >
> > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > >
> > > Hi, Julie,
> > >
> > > Maybe I should have put this on the original thread regarding AQ
> > > verification with John, but that was not a ticket so I guess
it's good
> to
> > > continue here - but you may not be aware of the genesis of AQ
> > verification
> > > up to this point.
> > >
> > > I re-ran my verification job so the directory was regenerated:
> > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > >
> > > Here's the command:
> > >
> > > point_stat pm25_DAY1 prepda.2018090212.nc
> > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> -v
> > 4
> > >
> > > Average and max PM is weird because it uses 2 obs files but
writes it
> out
> > > into one file as it sums up everything.  Here is the pb2nc
command:
> > >
> > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > /meso/save/Perry.Shafran/verif/nwtest/
> > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > >
> > > These commands tell you where the config files that I used are
so you
> may
> > > look at them.
> > >
> > > Here's the version.
> > >
> > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat --version
> > >
> > > MET Version:    V7.1
> > > Repository:    https://svn-met-dev.cgd.ucar.edu/tags/met/met-
7.1_beta6
> > > Revision:    5554
> > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)
> > >
> > > I had gotten things to work for the daily max ozone items but I
need to
> > > work on the average and daily max PM.  I am cc'ing John so he
has this
> as
> > > well and can take a look.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > >
> > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Perry.
> > > >
> > > > I tried to access the data at
> > > > tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
but
> > > > unfortunately, I see it is no longer there.
> > > >
> > > > Could you please do the following?
> > > >
> > > >    1. Send me the full command line that you are running.
> > > >    2. Point me to your exact fcst, obs, and config files
(machine and
> > > >    location information)
> > > >    3. Tell me what version of the code you are running - e.g.
output
> > from
> > > >    "point_stat --version "
> > > >
> > > > I will try to reproduce the error that you are getting and
will
> follow
> > > up.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via RT
<
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > >        Queue: met_help
> > > > >      Subject: Re: PM daily max and mean verification
> > > > >        Owner: Nobody
> > > > >   Requestors: perry.shafran at noaa.gov
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > >
> > > > >
> > > > >
> > > > > Hi there,
> > > > >
> > > > > I posted this 7 days ago and was wondering if we can get a
> response.
> > > We
> > > > > need to get AQM working so we can set up the AQ use case for
> METplus
> > > for
> > > > > the workshop.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
Affiliate <
> > > > > perry.shafran at noaa.gov> wrote:
> > > > >
> > > > > > Hi, there,
> > > > > >
> > > > > > I am having some issues reading the max PM file.  The
wgrib2
> output
> > > > looks
> > > > > > like this:
> > > > > >
> > > > > > 37 (t10a1)
> /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > wgrib2
> > > > > > pm25_DAY1
> > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave fcst:
> > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour ave
fcst:
> > > > > > 38 (t10a1)
/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > >
> > > > > > I have this in my point_stat config file:
> > > > > >
> > > > > > //
> > > > > > // Forecast and observation fields to be verified
> > > > > > //
> > > > > >
> > > > > > fcst = {
> > > > > >    field = [
> > > > > >       { name = "PDMAX1"; level = "A1";
> > valid_time="${CUR_VX_TIME1}";
> > > }
> > > > > >    ];
> > > > > > }
> > > > > >
> > > > > > obs = {
> > > > > >    convert(x) = x * 10^9;
> > > > > >    field = [
> > > > > >       { name = "COPOPM"; level = "A1"; message_type =
> > > > > "ANOWPM_MAX_230000";
> > > > > > }
> > > > > >    ];
> > > > > > }
> > > > > >
> > > > > > But then I get this error:
> > > > > >
> > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > DEBUG 2:
> > > > > > DEBUG 2:
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > DEBUG 2:
> > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > WARNING:
> > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
matching
> records
> > > > found
> > > > > > for 'PDMAX1/A1'
> > > > > > WARNING:
> > > > > > WARNING:
> > > > > > WARNING: process_fcst_climo_files() -> no fields matching
> PDMAX1/A1
> > > > found
> > > > > > in file: pm25_DAY1
> > > > > > WARNING:
> > > > > > ERROR  :
> > > > > > ERROR  : process_fcst_climo_files() -> no requested
forecast data
> > > > found!
> > > > > > Exiting...
> > > > > > ERROR  :
> > > > > >
> > > > > > Might something be wrong with the CUR_VX_TIME1 in this
particular
> > > case?
> > > > > > You can find these files in the data already existing on
Theia
> in,
> > > for
> > > > > > example, the
> > > > > >
> > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Thu Sep 06 11:36:38 2018

Hi, Julie,

Thanks so much!

Perry

On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Thank you, Perry.  Fortunately, I am able to reproduce the problem.
> Unfortunately, I believe I need John's help.  He is working very
hard on
> trying to get the met-8.0 release out, however.  Please know that we
> haven't forgotten about you and will get back to you as soon as we
can.
>
> Thank you!
>
> Julie
>
> On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> > Hi, Julie,
> >
> > Ah yes, I am setting it in a script.  Here are the environment
variables
> > you'll need:
> >
> > CUR_VX_TIME1=20180902_05
> > MODEL=CMAQ
> > OBS=ANOWPM
> >
> >
> >
> > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry.
> > >
> > > Thanks for the information.  I am definitely not familiar with
any of
> > this
> > > verification, nor do I know what AQ stands for.  I figured I'd
enroll
> > > John's help if necessary.  In the meantime, I thought I'd at
least try
> to
> > > reproduce the problem you're experiencing.
> > >
> > > Are you calling point_stat from a script or running it on the
command
> > > line?  If you are running from a script, can you please point me
to it?
> > >
> > > I see a couple of environment variables in
> > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > Can you please tell me what I should set MODEL and CUR_VX_TIME1
to?
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
>
> > > >
> > > > Hi, Julie,
> > > >
> > > > Maybe I should have put this on the original thread regarding
AQ
> > > > verification with John, but that was not a ticket so I guess
it's
> good
> > to
> > > > continue here - but you may not be aware of the genesis of AQ
> > > verification
> > > > up to this point.
> > > >
> > > > I re-ran my verification job so the directory was regenerated:
> > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > >
> > > > Here's the command:
> > > >
> > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > -v
> > > 4
> > > >
> > > > Average and max PM is weird because it uses 2 obs files but
writes it
> > out
> > > > into one file as it sums up everything.  Here is the pb2nc
command:
> > > >
> > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > >
> > > > These commands tell you where the config files that I used are
so you
> > may
> > > > look at them.
> > > >
> > > > Here's the version.
> > > >
> > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat --version
> > > >
> > > > MET Version:    V7.1
> > > > Repository:
> https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > Revision:    5554
> > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)
> > > >
> > > > I had gotten things to work for the daily max ozone items but
I need
> to
> > > > work on the average and daily max PM.  I am cc'ing John so he
has
> this
> > as
> > > > well and can take a look.
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > >
> > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry.
> > > > >
> > > > > I tried to access the data at
> > > > > tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
but
> > > > > unfortunately, I see it is no longer there.
> > > > >
> > > > > Could you please do the following?
> > > > >
> > > > >    1. Send me the full command line that you are running.
> > > > >    2. Point me to your exact fcst, obs, and config files
(machine
> and
> > > > >    location information)
> > > > >    3. Tell me what version of the code you are running -
e.g.
> output
> > > from
> > > > >    "point_stat --version "
> > > > >
> > > > > I will try to reproduce the error that you are getting and
will
> > follow
> > > > up.
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > >        Queue: met_help
> > > > > >      Subject: Re: PM daily max and mean verification
> > > > > >        Owner: Nobody
> > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > >       Status: new
> > > > > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi there,
> > > > > >
> > > > > > I posted this 7 days ago and was wondering if we can get a
> > response.
> > > > We
> > > > > > need to get AQM working so we can set up the AQ use case
for
> > METplus
> > > > for
> > > > > > the workshop.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
Affiliate <
> > > > > > perry.shafran at noaa.gov> wrote:
> > > > > >
> > > > > > > Hi, there,
> > > > > > >
> > > > > > > I am having some issues reading the max PM file.  The
wgrib2
> > output
> > > > > looks
> > > > > > > like this:
> > > > > > >
> > > > > > > 37 (t10a1)
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > wgrib2
> > > > > > > pm25_DAY1
> > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave
fcst:
> > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour
ave fcst:
> > > > > > > 38 (t10a1)
> /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > >
> > > > > > > I have this in my point_stat config file:
> > > > > > >
> > > > > > > //
> > > > > > > // Forecast and observation fields to be verified
> > > > > > > //
> > > > > > >
> > > > > > > fcst = {
> > > > > > >    field = [
> > > > > > >       { name = "PDMAX1"; level = "A1";
> > > valid_time="${CUR_VX_TIME1}";
> > > > }
> > > > > > >    ];
> > > > > > > }
> > > > > > >
> > > > > > > obs = {
> > > > > > >    convert(x) = x * 10^9;
> > > > > > >    field = [
> > > > > > >       { name = "COPOPM"; level = "A1"; message_type =
> > > > > > "ANOWPM_MAX_230000";
> > > > > > > }
> > > > > > >    ];
> > > > > > > }
> > > > > > >
> > > > > > > But then I get this error:
> > > > > > >
> > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > DEBUG 2:
> > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > WARNING:
> > > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
matching
> > records
> > > > > found
> > > > > > > for 'PDMAX1/A1'
> > > > > > > WARNING:
> > > > > > > WARNING:
> > > > > > > WARNING: process_fcst_climo_files() -> no fields
matching
> > PDMAX1/A1
> > > > > found
> > > > > > > in file: pm25_DAY1
> > > > > > > WARNING:
> > > > > > > ERROR  :
> > > > > > > ERROR  : process_fcst_climo_files() -> no requested
forecast
> data
> > > > > found!
> > > > > > > Exiting...
> > > > > > > ERROR  :
> > > > > > >
> > > > > > > Might something be wrong with the CUR_VX_TIME1 in this
> particular
> > > > case?
> > > > > > > You can find these files in the data already existing on
Theia
> > in,
> > > > for
> > > > > > > example, the
> > > > > > >
> > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: John Halley Gotway
Time: Mon Sep 10 10:51:01 2018

Julie,

I tested this out.  The CUR_VX_TIME1 that Perry sent was wrong.  He
sent:
   CUR_VX_TIME1=20180902_05

But running wgrib2 shows that it's valid at 04Z:

   CUR_VX_TIME1=20180902_04

wgrib2 -vt pm25_DAY1

1:0:vt=2018090204

2:144904:vt=2018090304

John

On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
>
> Hi, Julie,
>
> Thanks so much!
>
> Perry
>
> On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > Unfortunately, I believe I need John's help.  He is working very
hard on
> > trying to get the met-8.0 release out, however.  Please know that
we
> > haven't forgotten about you and will get back to you as soon as we
can.
> >
> > Thank you!
> >
> > Julie
> >
> > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > >
> > > Hi, Julie,
> > >
> > > Ah yes, I am setting it in a script.  Here are the environment
> variables
> > > you'll need:
> > >
> > > CUR_VX_TIME1=20180902_05
> > > MODEL=CMAQ
> > > OBS=ANOWPM
> > >
> > >
> > >
> > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Perry.
> > > >
> > > > Thanks for the information.  I am definitely not familiar with
any of
> > > this
> > > > verification, nor do I know what AQ stands for.  I figured I'd
enroll
> > > > John's help if necessary.  In the meantime, I thought I'd at
least
> try
> > to
> > > > reproduce the problem you're experiencing.
> > > >
> > > > Are you calling point_stat from a script or running it on the
command
> > > > line?  If you are running from a script, can you please point
me to
> it?
> > > >
> > > > I see a couple of environment variables in
> > > >
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1 to?
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > >
> > > > > Hi, Julie,
> > > > >
> > > > > Maybe I should have put this on the original thread
regarding AQ
> > > > > verification with John, but that was not a ticket so I guess
it's
> > good
> > > to
> > > > > continue here - but you may not be aware of the genesis of
AQ
> > > > verification
> > > > > up to this point.
> > > > >
> > > > > I re-ran my verification job so the directory was
regenerated:
> > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > >
> > > > > Here's the command:
> > > > >
> > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > >
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > -v
> > > > 4
> > > > >
> > > > > Average and max PM is weird because it uses 2 obs files but
writes
> it
> > > out
> > > > > into one file as it sums up everything.  Here is the pb2nc
command:
> > > > >
> > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > >
> > > > > These commands tell you where the config files that I used
are so
> you
> > > may
> > > > > look at them.
> > > > >
> > > > > Here's the version.
> > > > >
> > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > >
> > > > > MET Version:    V7.1
> > > > > Repository:
> > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > Revision:    5554
> > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)
> > > > >
> > > > > I had gotten things to work for the daily max ozone items
but I
> need
> > to
> > > > > work on the average and daily max PM.  I am cc'ing John so
he has
> > this
> > > as
> > > > > well and can take a look.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > >
> > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Perry.
> > > > > >
> > > > > > I tried to access the data at
> > > > > >
tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
> > > > > > unfortunately, I see it is no longer there.
> > > > > >
> > > > > > Could you please do the following?
> > > > > >
> > > > > >    1. Send me the full command line that you are running.
> > > > > >    2. Point me to your exact fcst, obs, and config files
(machine
> > and
> > > > > >    location information)
> > > > > >    3. Tell me what version of the code you are running -
e.g.
> > output
> > > > from
> > > > > >    "point_stat --version "
> > > > > >
> > > > > > I will try to reproduce the error that you are getting and
will
> > > follow
> > > > > up.
> > > > > >
> > > > > > Thanks,
> > > > > > Julie
> > > > > >
> > > > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via
RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > > >        Queue: met_help
> > > > > > >      Subject: Re: PM daily max and mean verification
> > > > > > >        Owner: Nobody
> > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > >       Status: new
> > > > > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi there,
> > > > > > >
> > > > > > > I posted this 7 days ago and was wondering if we can get
a
> > > response.
> > > > > We
> > > > > > > need to get AQM working so we can set up the AQ use case
for
> > > METplus
> > > > > for
> > > > > > > the workshop.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
Affiliate
> <
> > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > >
> > > > > > > > Hi, there,
> > > > > > > >
> > > > > > > > I am having some issues reading the max PM file.  The
wgrib2
> > > output
> > > > > > looks
> > > > > > > > like this:
> > > > > > > >
> > > > > > > > 37 (t10a1)
> > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > wgrib2
> > > > > > > > pm25_DAY1
> > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave
fcst:
> > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour
ave
> fcst:
> > > > > > > > 38 (t10a1)
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > >
> > > > > > > > I have this in my point_stat config file:
> > > > > > > >
> > > > > > > > //
> > > > > > > > // Forecast and observation fields to be verified
> > > > > > > > //
> > > > > > > >
> > > > > > > > fcst = {
> > > > > > > >    field = [
> > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > valid_time="${CUR_VX_TIME1}";
> > > > > }
> > > > > > > >    ];
> > > > > > > > }
> > > > > > > >
> > > > > > > > obs = {
> > > > > > > >    convert(x) = x * 10^9;
> > > > > > > >    field = [
> > > > > > > >       { name = "COPOPM"; level = "A1"; message_type =
> > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > }
> > > > > > > >    ];
> > > > > > > > }
> > > > > > > >
> > > > > > > > But then I get this error:
> > > > > > > >
> > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > WARNING:
> > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
matching
> > > records
> > > > > > found
> > > > > > > > for 'PDMAX1/A1'
> > > > > > > > WARNING:
> > > > > > > > WARNING:
> > > > > > > > WARNING: process_fcst_climo_files() -> no fields
matching
> > > PDMAX1/A1
> > > > > > found
> > > > > > > > in file: pm25_DAY1
> > > > > > > > WARNING:
> > > > > > > > ERROR  :
> > > > > > > > ERROR  : process_fcst_climo_files() -> no requested
forecast
> > data
> > > > > > found!
> > > > > > > > Exiting...
> > > > > > > > ERROR  :
> > > > > > > >
> > > > > > > > Might something be wrong with the CUR_VX_TIME1 in this
> > particular
> > > > > case?
> > > > > > > > You can find these files in the data already existing
on
> Theia
> > > in,
> > > > > for
> > > > > > > > example, the
> > > > > > > >
> > > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: Julie Prestopnik
Time: Mon Sep 10 10:53:34 2018

Hi Perry.

I was able to get some help with this.

Perry, apologies for not getting to this issue sooner…  I took at look
at
> the data this morning and see the problem.
>
>
>    -
>
>    The real issue is “I’m having trouble getting MET to read a
particular
>    GRIB record”.  When in that position, I find the plot_data_plane
tool to be
>    a quick and easy way of testing.
>    -
>
>    So I ran the equivalent plot_data_plane command:
>
>
>   plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A1";'
>
>   ERROR  : plot_data_plane -> trouble getting field "name="PDMAX1";
> level="A1";" from file "pm25_DAY1"
>
>
>    -
>
>    Setting ‘level=”A1”;’ you’re telling MET to look for a 1-hour
>    accumulation of data.  But looking at the wgrib2 output you sent,
the
>    accumulations are over 23 hours (‘-7-16 hour ave fcst’ and ‘17-40
hour ave
>    fcst’)
>    -
>
>    So use “A23” instead of “A1” and we get a plot, but also a
warning
>    about multiple matches:
>
>
> plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1"; level="A23";'
>
> WARNING: MetGrib2DataFile::data_plane() - Multiple matching records
found
> for 'PDMAX1/A23' - 1, 2 - using 1
>
>
>    -
>
>    You see that warning when MET finds multiple matching records.
So you
>    need to specify more information to select the actual record you
want…
>    either “lead_time” or “valid_time”.  In the example below, I
selected the
>    40-hour forecast from the second record:
>
> plot_data_plane pm25_DAY1 pm25_DAY1_F40.ps 'name="PDMAX1";
level="A23";
> lead_time="40";'
>
>
>    -
>
>    And attached is the resulting image.
>
>
I modified A1 to A23 and ran point_stat on tide again.  I received
another
warning about no matching records.  I had CUR_VX_TIME1 set to
20180902_05,
but using wgrib2, it looks like I needed to set it to 20180902_04:

[Julie.Prestopnik at t20a1 shafran_20180904]$ $WGRIB2 -vt pm25_DAY1
> 1:0:vt=2018090204
> 2:144904:vt=2018090304

After doing that, I got a successful run.  Please try it out and let
us
know how it goes.

Thanks!

Julie

On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
>
> Hi, Julie,
>
> Thanks so much!
>
> Perry
>
> On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > Unfortunately, I believe I need John's help.  He is working very
hard on
> > trying to get the met-8.0 release out, however.  Please know that
we
> > haven't forgotten about you and will get back to you as soon as we
can.
> >
> > Thank you!
> >
> > Julie
> >
> > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > >
> > > Hi, Julie,
> > >
> > > Ah yes, I am setting it in a script.  Here are the environment
> variables
> > > you'll need:
> > >
> > > CUR_VX_TIME1=20180902_05
> > > MODEL=CMAQ
> > > OBS=ANOWPM
> > >
> > >
> > >
> > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Perry.
> > > >
> > > > Thanks for the information.  I am definitely not familiar with
any of
> > > this
> > > > verification, nor do I know what AQ stands for.  I figured I'd
enroll
> > > > John's help if necessary.  In the meantime, I thought I'd at
least
> try
> > to
> > > > reproduce the problem you're experiencing.
> > > >
> > > > Are you calling point_stat from a script or running it on the
command
> > > > line?  If you are running from a script, can you please point
me to
> it?
> > > >
> > > > I see a couple of environment variables in
> > > >
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1 to?
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > >
> > > > > Hi, Julie,
> > > > >
> > > > > Maybe I should have put this on the original thread
regarding AQ
> > > > > verification with John, but that was not a ticket so I guess
it's
> > good
> > > to
> > > > > continue here - but you may not be aware of the genesis of
AQ
> > > > verification
> > > > > up to this point.
> > > > >
> > > > > I re-ran my verification job so the directory was
regenerated:
> > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > >
> > > > > Here's the command:
> > > > >
> > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > >
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > -v
> > > > 4
> > > > >
> > > > > Average and max PM is weird because it uses 2 obs files but
writes
> it
> > > out
> > > > > into one file as it sums up everything.  Here is the pb2nc
command:
> > > > >
> > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > >
> > > > > These commands tell you where the config files that I used
are so
> you
> > > may
> > > > > look at them.
> > > > >
> > > > > Here's the version.
> > > > >
> > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > >
> > > > > MET Version:    V7.1
> > > > > Repository:
> > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > Revision:    5554
> > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul 2018)
> > > > >
> > > > > I had gotten things to work for the daily max ozone items
but I
> need
> > to
> > > > > work on the average and daily max PM.  I am cc'ing John so
he has
> > this
> > > as
> > > > > well and can take a look.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Perry
> > > > >
> > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > >
> > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Perry.
> > > > > >
> > > > > > I tried to access the data at
> > > > > >
tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12, but
> > > > > > unfortunately, I see it is no longer there.
> > > > > >
> > > > > > Could you please do the following?
> > > > > >
> > > > > >    1. Send me the full command line that you are running.
> > > > > >    2. Point me to your exact fcst, obs, and config files
(machine
> > and
> > > > > >    location information)
> > > > > >    3. Tell me what version of the code you are running -
e.g.
> > output
> > > > from
> > > > > >    "point_stat --version "
> > > > > >
> > > > > > I will try to reproduce the error that you are getting and
will
> > > follow
> > > > > up.
> > > > > >
> > > > > > Thanks,
> > > > > > Julie
> > > > > >
> > > > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov via
RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted upon.
> > > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > > >        Queue: met_help
> > > > > > >      Subject: Re: PM daily max and mean verification
> > > > > > >        Owner: Nobody
> > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > >       Status: new
> > > > > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi there,
> > > > > > >
> > > > > > > I posted this 7 days ago and was wondering if we can get
a
> > > response.
> > > > > We
> > > > > > > need to get AQM working so we can set up the AQ use case
for
> > > METplus
> > > > > for
> > > > > > > the workshop.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
Affiliate
> <
> > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > >
> > > > > > > > Hi, there,
> > > > > > > >
> > > > > > > > I am having some issues reading the max PM file.  The
wgrib2
> > > output
> > > > > > looks
> > > > > > > > like this:
> > > > > > > >
> > > > > > > > 37 (t10a1)
> > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > wgrib2
> > > > > > > > pm25_DAY1
> > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave
fcst:
> > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40 hour
ave
> fcst:
> > > > > > > > 38 (t10a1)
> > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > >
> > > > > > > > I have this in my point_stat config file:
> > > > > > > >
> > > > > > > > //
> > > > > > > > // Forecast and observation fields to be verified
> > > > > > > > //
> > > > > > > >
> > > > > > > > fcst = {
> > > > > > > >    field = [
> > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > valid_time="${CUR_VX_TIME1}";
> > > > > }
> > > > > > > >    ];
> > > > > > > > }
> > > > > > > >
> > > > > > > > obs = {
> > > > > > > >    convert(x) = x * 10^9;
> > > > > > > >    field = [
> > > > > > > >       { name = "COPOPM"; level = "A1"; message_type =
> > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > }
> > > > > > > >    ];
> > > > > > > > }
> > > > > > > >
> > > > > > > > But then I get this error:
> > > > > > > >
> > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > DEBUG 2:
> > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > WARNING:
> > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
matching
> > > records
> > > > > > found
> > > > > > > > for 'PDMAX1/A1'
> > > > > > > > WARNING:
> > > > > > > > WARNING:
> > > > > > > > WARNING: process_fcst_climo_files() -> no fields
matching
> > > PDMAX1/A1
> > > > > > found
> > > > > > > > in file: pm25_DAY1
> > > > > > > > WARNING:
> > > > > > > > ERROR  :
> > > > > > > > ERROR  : process_fcst_climo_files() -> no requested
forecast
> > data
> > > > > > found!
> > > > > > > > Exiting...
> > > > > > > > ERROR  :
> > > > > > > >
> > > > > > > > Might something be wrong with the CUR_VX_TIME1 in this
> > particular
> > > > > case?
> > > > > > > > You can find these files in the data already existing
on
> Theia
> > > in,
> > > > > for
> > > > > > > > example, the
> > > > > > > >
> > > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Wed Sep 12 07:17:12 2018

Hi, John,

Just to note, CUR_VX_TIME1=20180902_05 is what you sent for ozone, and
that
has worked (the _04 for ozone did not).  It would seem for PM that the
value should be 20180902_04, so we'll need to ensure that we have the
proper values for both ozone and for PM.

Thanks!

Perry

On Mon, Sep 10, 2018 at 12:51 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Julie,
>
> I tested this out.  The CUR_VX_TIME1 that Perry sent was wrong.  He
sent:
>    CUR_VX_TIME1=20180902_05
>
> But running wgrib2 shows that it's valid at 04Z:
>
>    CUR_VX_TIME1=20180902_04
>
>
> wgrib2 -vt pm25_DAY1
>
> 1:0:vt=2018090204
>
> 2:144904:vt=2018090304
>
>
> John
>
> On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> > Hi, Julie,
> >
> > Thanks so much!
> >
> > Perry
> >
> > On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > > Unfortunately, I believe I need John's help.  He is working very
hard
> on
> > > trying to get the met-8.0 release out, however.  Please know
that we
> > > haven't forgotten about you and will get back to you as soon as
we can.
> > >
> > > Thank you!
> > >
> > > Julie
> > >
> > > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
>
> > > >
> > > > Hi, Julie,
> > > >
> > > > Ah yes, I am setting it in a script.  Here are the environment
> > variables
> > > > you'll need:
> > > >
> > > > CUR_VX_TIME1=20180902_05
> > > > MODEL=CMAQ
> > > > OBS=ANOWPM
> > > >
> > > >
> > > >
> > > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry.
> > > > >
> > > > > Thanks for the information.  I am definitely not familiar
with any
> of
> > > > this
> > > > > verification, nor do I know what AQ stands for.  I figured
I'd
> enroll
> > > > > John's help if necessary.  In the meantime, I thought I'd at
least
> > try
> > > to
> > > > > reproduce the problem you're experiencing.
> > > > >
> > > > > Are you calling point_stat from a script or running it on
the
> command
> > > > > line?  If you are running from a script, can you please
point me to
> > it?
> > > > >
> > > > > I see a couple of environment variables in
> > > > >
> >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1 to?
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > > >
> > > > > > Hi, Julie,
> > > > > >
> > > > > > Maybe I should have put this on the original thread
regarding AQ
> > > > > > verification with John, but that was not a ticket so I
guess it's
> > > good
> > > > to
> > > > > > continue here - but you may not be aware of the genesis of
AQ
> > > > > verification
> > > > > > up to this point.
> > > > > >
> > > > > > I re-ran my verification job so the directory was
regenerated:
> > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > >
> > > > > > Here's the command:
> > > > > >
> > > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > > >
> >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > > -v
> > > > > 4
> > > > > >
> > > > > > Average and max PM is weird because it uses 2 obs files
but
> writes
> > it
> > > > out
> > > > > > into one file as it sums up everything.  Here is the pb2nc
> command:
> > > > > >
> > > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > > >
> > > > > > These commands tell you where the config files that I used
are so
> > you
> > > > may
> > > > > > look at them.
> > > > > >
> > > > > > Here's the version.
> > > > > >
> > > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > > >
> > > > > > MET Version:    V7.1
> > > > > > Repository:
> > > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > > Revision:    5554
> > > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul
2018)
> > > > > >
> > > > > > I had gotten things to work for the daily max ozone items
but I
> > need
> > > to
> > > > > > work on the average and daily max PM.  I am cc'ing John so
he has
> > > this
> > > > as
> > > > > > well and can take a look.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > > >
> > > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry.
> > > > > > >
> > > > > > > I tried to access the data at
> > > > > > >
tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
> but
> > > > > > > unfortunately, I see it is no longer there.
> > > > > > >
> > > > > > > Could you please do the following?
> > > > > > >
> > > > > > >    1. Send me the full command line that you are
running.
> > > > > > >    2. Point me to your exact fcst, obs, and config files
> (machine
> > > and
> > > > > > >    location information)
> > > > > > >    3. Tell me what version of the code you are running -
e.g.
> > > output
> > > > > from
> > > > > > >    "point_stat --version "
> > > > > > >
> > > > > > > I will try to reproduce the error that you are getting
and will
> > > > follow
> > > > > > up.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov
via RT
> <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted
upon.
> > > > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > > > >        Queue: met_help
> > > > > > > >      Subject: Re: PM daily max and mean verification
> > > > > > > >        Owner: Nobody
> > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > >       Status: new
> > > > > > > >  Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi there,
> > > > > > > >
> > > > > > > > I posted this 7 days ago and was wondering if we can
get a
> > > > response.
> > > > > > We
> > > > > > > > need to get AQM working so we can set up the AQ use
case for
> > > > METplus
> > > > > > for
> > > > > > > > the workshop.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
> Affiliate
> > <
> > > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > > >
> > > > > > > > > Hi, there,
> > > > > > > > >
> > > > > > > > > I am having some issues reading the max PM file.
The
> wgrib2
> > > > output
> > > > > > > looks
> > > > > > > > > like this:
> > > > > > > > >
> > > > > > > > > 37 (t10a1)
> > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > > wgrib2
> > > > > > > > > pm25_DAY1
> > > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave
fcst:
> > > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40
hour ave
> > fcst:
> > > > > > > > > 38 (t10a1)
> > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > > >
> > > > > > > > > I have this in my point_stat config file:
> > > > > > > > >
> > > > > > > > > //
> > > > > > > > > // Forecast and observation fields to be verified
> > > > > > > > > //
> > > > > > > > >
> > > > > > > > > fcst = {
> > > > > > > > >    field = [
> > > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > > valid_time="${CUR_VX_TIME1}";
> > > > > > }
> > > > > > > > >    ];
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > obs = {
> > > > > > > > >    convert(x) = x * 10^9;
> > > > > > > > >    field = [
> > > > > > > > >       { name = "COPOPM"; level = "A1"; message_type
=
> > > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > > }
> > > > > > > > >    ];
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > But then I get this error:
> > > > > > > > >
> > > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > > WARNING:
> > > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
matching
> > > > records
> > > > > > > found
> > > > > > > > > for 'PDMAX1/A1'
> > > > > > > > > WARNING:
> > > > > > > > > WARNING:
> > > > > > > > > WARNING: process_fcst_climo_files() -> no fields
matching
> > > > PDMAX1/A1
> > > > > > > found
> > > > > > > > > in file: pm25_DAY1
> > > > > > > > > WARNING:
> > > > > > > > > ERROR  :
> > > > > > > > > ERROR  : process_fcst_climo_files() -> no requested
> forecast
> > > data
> > > > > > > found!
> > > > > > > > > Exiting...
> > > > > > > > > ERROR  :
> > > > > > > > >
> > > > > > > > > Might something be wrong with the CUR_VX_TIME1 in
this
> > > particular
> > > > > > case?
> > > > > > > > > You can find these files in the data already
existing on
> > Theia
> > > > in,
> > > > > > for
> > > > > > > > > example, the
> > > > > > > > >
> > > > >
> /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > Perry
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Wed Sep 12 07:18:34 2018

Hi, Julie,

Thanks!  I do want to ask (and maybe John can help out as well).  I
also
want to plot the average PM as well.  How do I differentiate in the
point_stat file whether I plot the max or the average?

Thanks!

Perry

On Mon, Sep 10, 2018 at 12:53 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Hi Perry.
>
> I was able to get some help with this.
>
> Perry, apologies for not getting to this issue sooner…  I took at
look at
> > the data this morning and see the problem.
> >
> >
> >    -
> >
> >    The real issue is “I’m having trouble getting MET to read a
particular
> >    GRIB record”.  When in that position, I find the
plot_data_plane tool
> to be
> >    a quick and easy way of testing.
> >    -
> >
> >    So I ran the equivalent plot_data_plane command:
> >
> >
> >   plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A1";'
> >
> >   ERROR  : plot_data_plane -> trouble getting field
"name="PDMAX1";
> > level="A1";" from file "pm25_DAY1"
> >
> >
> >    -
> >
> >    Setting ‘level=”A1”;’ you’re telling MET to look for a 1-hour
> >    accumulation of data.  But looking at the wgrib2 output you
sent, the
> >    accumulations are over 23 hours (‘-7-16 hour ave fcst’ and ‘17-
40
> hour ave
> >    fcst’)
> >    -
> >
> >    So use “A23” instead of “A1” and we get a plot, but also a
warning
> >    about multiple matches:
> >
> >
> > plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A23";'
> >
> > WARNING: MetGrib2DataFile::data_plane() - Multiple matching
records found
> > for 'PDMAX1/A23' - 1, 2 - using 1
> >
> >
> >    -
> >
> >    You see that warning when MET finds multiple matching records.
So you
> >    need to specify more information to select the actual record
you want…
> >    either “lead_time” or “valid_time”.  In the example below, I
selected
> the
> >    40-hour forecast from the second record:
> >
> > plot_data_plane pm25_DAY1 pm25_DAY1_F40.ps 'name="PDMAX1";
level="A23";
> > lead_time="40";'
> >
> >
> >    -
> >
> >    And attached is the resulting image.
> >
> >
> I modified A1 to A23 and ran point_stat on tide again.  I received
another
> warning about no matching records.  I had CUR_VX_TIME1 set to
20180902_05,
> but using wgrib2, it looks like I needed to set it to 20180902_04:
>
> [Julie.Prestopnik at t20a1 shafran_20180904]$ $WGRIB2 -vt pm25_DAY1
> > 1:0:vt=2018090204
> > 2:144904:vt=2018090304
>
>
> After doing that, I got a successful run.  Please try it out and let
us
> know how it goes.
>
> Thanks!
>
> Julie
>
>
>
> On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> > Hi, Julie,
> >
> > Thanks so much!
> >
> > Perry
> >
> > On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > > Unfortunately, I believe I need John's help.  He is working very
hard
> on
> > > trying to get the met-8.0 release out, however.  Please know
that we
> > > haven't forgotten about you and will get back to you as soon as
we can.
> > >
> > > Thank you!
> > >
> > > Julie
> > >
> > > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
>
> > > >
> > > > Hi, Julie,
> > > >
> > > > Ah yes, I am setting it in a script.  Here are the environment
> > variables
> > > > you'll need:
> > > >
> > > > CUR_VX_TIME1=20180902_05
> > > > MODEL=CMAQ
> > > > OBS=ANOWPM
> > > >
> > > >
> > > >
> > > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Perry.
> > > > >
> > > > > Thanks for the information.  I am definitely not familiar
with any
> of
> > > > this
> > > > > verification, nor do I know what AQ stands for.  I figured
I'd
> enroll
> > > > > John's help if necessary.  In the meantime, I thought I'd at
least
> > try
> > > to
> > > > > reproduce the problem you're experiencing.
> > > > >
> > > > > Are you calling point_stat from a script or running it on
the
> command
> > > > > line?  If you are running from a script, can you please
point me to
> > it?
> > > > >
> > > > > I see a couple of environment variables in
> > > > >
> >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1 to?
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > > >
> > > > > > Hi, Julie,
> > > > > >
> > > > > > Maybe I should have put this on the original thread
regarding AQ
> > > > > > verification with John, but that was not a ticket so I
guess it's
> > > good
> > > > to
> > > > > > continue here - but you may not be aware of the genesis of
AQ
> > > > > verification
> > > > > > up to this point.
> > > > > >
> > > > > > I re-ran my verification job so the directory was
regenerated:
> > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > >
> > > > > > Here's the command:
> > > > > >
> > > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > > >
> >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > > -v
> > > > > 4
> > > > > >
> > > > > > Average and max PM is weird because it uses 2 obs files
but
> writes
> > it
> > > > out
> > > > > > into one file as it sums up everything.  Here is the pb2nc
> command:
> > > > > >
> > > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > > >
> > > > > > These commands tell you where the config files that I used
are so
> > you
> > > > may
> > > > > > look at them.
> > > > > >
> > > > > > Here's the version.
> > > > > >
> > > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > > >
> > > > > > MET Version:    V7.1
> > > > > > Repository:
> > > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > > Revision:    5554
> > > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul
2018)
> > > > > >
> > > > > > I had gotten things to work for the daily max ozone items
but I
> > need
> > > to
> > > > > > work on the average and daily max PM.  I am cc'ing John so
he has
> > > this
> > > > as
> > > > > > well and can take a look.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Perry
> > > > > >
> > > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > > >
> > > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry.
> > > > > > >
> > > > > > > I tried to access the data at
> > > > > > >
tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
> but
> > > > > > > unfortunately, I see it is no longer there.
> > > > > > >
> > > > > > > Could you please do the following?
> > > > > > >
> > > > > > >    1. Send me the full command line that you are
running.
> > > > > > >    2. Point me to your exact fcst, obs, and config files
> (machine
> > > and
> > > > > > >    location information)
> > > > > > >    3. Tell me what version of the code you are running -
e.g.
> > > output
> > > > > from
> > > > > > >    "point_stat --version "
> > > > > > >
> > > > > > > I will try to reproduce the error that you are getting
and will
> > > > follow
> > > > > > up.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov
via RT
> <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted
upon.
> > > > > > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > > > > >        Queue: met_help
> > > > > > > >      Subject: Re: PM daily max and mean verification
> > > > > > > >        Owner: Nobody
> > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > >       Status: new
> > > > > > > >  Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi there,
> > > > > > > >
> > > > > > > > I posted this 7 days ago and was wondering if we can
get a
> > > > response.
> > > > > > We
> > > > > > > > need to get AQM working so we can set up the AQ use
case for
> > > > METplus
> > > > > > for
> > > > > > > > the workshop.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
> Affiliate
> > <
> > > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > > >
> > > > > > > > > Hi, there,
> > > > > > > > >
> > > > > > > > > I am having some issues reading the max PM file.
The
> wgrib2
> > > > output
> > > > > > > looks
> > > > > > > > > like this:
> > > > > > > > >
> > > > > > > > > 37 (t10a1)
> > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > > wgrib2
> > > > > > > > > pm25_DAY1
> > > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour ave
fcst:
> > > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40
hour ave
> > fcst:
> > > > > > > > > 38 (t10a1)
> > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > > >
> > > > > > > > > I have this in my point_stat config file:
> > > > > > > > >
> > > > > > > > > //
> > > > > > > > > // Forecast and observation fields to be verified
> > > > > > > > > //
> > > > > > > > >
> > > > > > > > > fcst = {
> > > > > > > > >    field = [
> > > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > > valid_time="${CUR_VX_TIME1}";
> > > > > > }
> > > > > > > > >    ];
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > obs = {
> > > > > > > > >    convert(x) = x * 10^9;
> > > > > > > > >    field = [
> > > > > > > > >       { name = "COPOPM"; level = "A1"; message_type
=
> > > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > > }
> > > > > > > > >    ];
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > But then I get this error:
> > > > > > > > >
> > > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > DEBUG 2:
> > > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > > WARNING:
> > > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
matching
> > > > records
> > > > > > > found
> > > > > > > > > for 'PDMAX1/A1'
> > > > > > > > > WARNING:
> > > > > > > > > WARNING:
> > > > > > > > > WARNING: process_fcst_climo_files() -> no fields
matching
> > > > PDMAX1/A1
> > > > > > > found
> > > > > > > > > in file: pm25_DAY1
> > > > > > > > > WARNING:
> > > > > > > > > ERROR  :
> > > > > > > > > ERROR  : process_fcst_climo_files() -> no requested
> forecast
> > > data
> > > > > > > found!
> > > > > > > > > Exiting...
> > > > > > > > > ERROR  :
> > > > > > > > >
> > > > > > > > > Might something be wrong with the CUR_VX_TIME1 in
this
> > > particular
> > > > > > case?
> > > > > > > > > You can find these files in the data already
existing on
> > Theia
> > > > in,
> > > > > > for
> > > > > > > > > example, the
> > > > > > > > >
> > > > >
> /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > > directory, the file aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > Perry
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: John Halley Gotway
Time: Wed Sep 12 10:48:18 2018

Hi Perry,

Julie and I just talked about your question... how to differentiate in
the
output from Point-Stat between using MAX and MEAN observation
summaries?

This info will show up in the "OBTYPE" column of the Point-Stat output
files.  Here's the "obs" section from your Point-Stat config file:

obs = {
   convert(x) = x * 10^9;
   field = [
      { name = "COPOPM"; level = "A1"; message_type =
"ANOWPM_MAX_230000"; }
   ];
}

You are telling Point-Stat to verify against the "ANOWPM_MAX_230000"
message type.  And that message type string will be written to the
OBTYPE
output column.

FYI, you could also use the "desc" config file option to populate the
DESC
output column however you'd like.  For example, setting:
    desc = "DAILY_MAX";
Would set the DESC output column to that string.  So that's a second
option
if you find it more convenient.

The Point-Stat config file you sent us includes a single verification
task... compare forecast PDMAX1 data to observation message type
ANOWPM_MAX_230000.  If you'd also like to compare to the daily mean,
you
could add a second verification task but for observation message_type
=
ANOWPM_MEAN_230000.

Running ncdump on the output of PB2NC and look at the "hdr_typ_table"
variable, here's the message types in that file:

ncdump -v hdr_typ_table prepda.2018090212.nc
...
hdr_typ_table =
  "ANOWPM",
  "ANOWPM_MAX_230000",
  "ANOWPM_MEAN_230000" ;

Hope that helps.

Thanks,
John

On Wed, Sep 12, 2018 at 7:18 AM perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
>
> Hi, Julie,
>
> Thanks!  I do want to ask (and maybe John can help out as well).  I
also
> want to plot the average PM as well.  How do I differentiate in the
> point_stat file whether I plot the max or the average?
>
> Thanks!
>
> Perry
>
> On Mon, Sep 10, 2018 at 12:53 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Perry.
> >
> > I was able to get some help with this.
> >
> > Perry, apologies for not getting to this issue sooner…  I took at
look at
> > > the data this morning and see the problem.
> > >
> > >
> > >    -
> > >
> > >    The real issue is “I’m having trouble getting MET to read a
> particular
> > >    GRIB record”.  When in that position, I find the
plot_data_plane
> tool
> > to be
> > >    a quick and easy way of testing.
> > >    -
> > >
> > >    So I ran the equivalent plot_data_plane command:
> > >
> > >
> > >   plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A1";'
> > >
> > >   ERROR  : plot_data_plane -> trouble getting field
"name="PDMAX1";
> > > level="A1";" from file "pm25_DAY1"
> > >
> > >
> > >    -
> > >
> > >    Setting ‘level=”A1”;’ you’re telling MET to look for a 1-hour
> > >    accumulation of data.  But looking at the wgrib2 output you
sent,
> the
> > >    accumulations are over 23 hours (‘-7-16 hour ave fcst’ and
‘17-40
> > hour ave
> > >    fcst’)
> > >    -
> > >
> > >    So use “A23” instead of “A1” and we get a plot, but also a
warning
> > >    about multiple matches:
> > >
> > >
> > > plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A23";'
> > >
> > > WARNING: MetGrib2DataFile::data_plane() - Multiple matching
records
> found
> > > for 'PDMAX1/A23' - 1, 2 - using 1
> > >
> > >
> > >    -
> > >
> > >    You see that warning when MET finds multiple matching
records.  So
> you
> > >    need to specify more information to select the actual record
you
> want…
> > >    either “lead_time” or “valid_time”.  In the example below, I
> selected
> > the
> > >    40-hour forecast from the second record:
> > >
> > > plot_data_plane pm25_DAY1 pm25_DAY1_F40.ps 'name="PDMAX1";
level="A23";
> > > lead_time="40";'
> > >
> > >
> > >    -
> > >
> > >    And attached is the resulting image.
> > >
> > >
> > I modified A1 to A23 and ran point_stat on tide again.  I received
> another
> > warning about no matching records.  I had CUR_VX_TIME1 set to
> 20180902_05,
> > but using wgrib2, it looks like I needed to set it to 20180902_04:
> >
> > [Julie.Prestopnik at t20a1 shafran_20180904]$ $WGRIB2 -vt pm25_DAY1
> > > 1:0:vt=2018090204
> > > 2:144904:vt=2018090304
> >
> >
> > After doing that, I got a successful run.  Please try it out and
let us
> > know how it goes.
> >
> > Thanks!
> >
> > Julie
> >
> >
> >
> > On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > >
> > > Hi, Julie,
> > >
> > > Thanks so much!
> > >
> > > Perry
> > >
> > > On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > > > Unfortunately, I believe I need John's help.  He is working
very hard
> > on
> > > > trying to get the met-8.0 release out, however.  Please know
that we
> > > > haven't forgotten about you and will get back to you as soon
as we
> can.
> > > >
> > > > Thank you!
> > > >
> > > > Julie
> > > >
> > > > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > >
> > > > > Hi, Julie,
> > > > >
> > > > > Ah yes, I am setting it in a script.  Here are the
environment
> > > variables
> > > > > you'll need:
> > > > >
> > > > > CUR_VX_TIME1=20180902_05
> > > > > MODEL=CMAQ
> > > > > OBS=ANOWPM
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Perry.
> > > > > >
> > > > > > Thanks for the information.  I am definitely not familiar
with
> any
> > of
> > > > > this
> > > > > > verification, nor do I know what AQ stands for.  I figured
I'd
> > enroll
> > > > > > John's help if necessary.  In the meantime, I thought I'd
at
> least
> > > try
> > > > to
> > > > > > reproduce the problem you're experiencing.
> > > > > >
> > > > > > Are you calling point_stat from a script or running it on
the
> > command
> > > > > > line?  If you are running from a script, can you please
point me
> to
> > > it?
> > > > > >
> > > > > > I see a couple of environment variables in
> > > > > >
> > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1
> to?
> > > > > >
> > > > > > Thanks,
> > > > > > Julie
> > > > > >
> > > > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov via
RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> >
> > > > > > >
> > > > > > > Hi, Julie,
> > > > > > >
> > > > > > > Maybe I should have put this on the original thread
regarding
> AQ
> > > > > > > verification with John, but that was not a ticket so I
guess
> it's
> > > > good
> > > > > to
> > > > > > > continue here - but you may not be aware of the genesis
of AQ
> > > > > > verification
> > > > > > > up to this point.
> > > > > > >
> > > > > > > I re-ran my verification job so the directory was
regenerated:
> > > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > >
> > > > > > > Here's the command:
> > > > > > >
> > > > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > > > >
> > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > > > -v
> > > > > > 4
> > > > > > >
> > > > > > > Average and max PM is weird because it uses 2 obs files
but
> > writes
> > > it
> > > > > out
> > > > > > > into one file as it sums up everything.  Here is the
pb2nc
> > command:
> > > > > > >
> > > > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > > > >
> > > > > > > These commands tell you where the config files that I
used are
> so
> > > you
> > > > > may
> > > > > > > look at them.
> > > > > > >
> > > > > > > Here's the version.
> > > > > > >
> > > > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > > > >
> > > > > > > MET Version:    V7.1
> > > > > > > Repository:
> > > > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > > > Revision:    5554
> > > > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul
2018)
> > > > > > >
> > > > > > > I had gotten things to work for the daily max ozone
items but I
> > > need
> > > > to
> > > > > > > work on the average and daily max PM.  I am cc'ing John
so he
> has
> > > > this
> > > > > as
> > > > > > > well and can take a look.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Perry
> > > > > > >
> > > > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > > > >
> > > > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT <
> > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Perry.
> > > > > > > >
> > > > > > > > I tried to access the data at
> > > > > > > >
tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
> > but
> > > > > > > > unfortunately, I see it is no longer there.
> > > > > > > >
> > > > > > > > Could you please do the following?
> > > > > > > >
> > > > > > > >    1. Send me the full command line that you are
running.
> > > > > > > >    2. Point me to your exact fcst, obs, and config
files
> > (machine
> > > > and
> > > > > > > >    location information)
> > > > > > > >    3. Tell me what version of the code you are running
- e.g.
> > > > output
> > > > > > from
> > > > > > > >    "point_stat --version "
> > > > > > > >
> > > > > > > > I will try to reproduce the error that you are getting
and
> will
> > > > > follow
> > > > > > > up.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Julie
> > > > > > > >
> > > > > > > > On Tue, Sep 4, 2018 at 11:26 AM perry.shafran at noaa.gov
via
> RT
> > <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted
upon.
> > > > > > > > > Transaction: Ticket created by
perry.shafran at noaa.gov
> > > > > > > > >        Queue: met_help
> > > > > > > > >      Subject: Re: PM daily max and mean verification
> > > > > > > > >        Owner: Nobody
> > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > >       Status: new
> > > > > > > > >  Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi there,
> > > > > > > > >
> > > > > > > > > I posted this 7 days ago and was wondering if we can
get a
> > > > > response.
> > > > > > > We
> > > > > > > > > need to get AQM working so we can set up the AQ use
case
> for
> > > > > METplus
> > > > > > > for
> > > > > > > > > the workshop.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > Perry
> > > > > > > > >
> > > > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran - NOAA
> > Affiliate
> > > <
> > > > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > > > >
> > > > > > > > > > Hi, there,
> > > > > > > > > >
> > > > > > > > > > I am having some issues reading the max PM file.
The
> > wgrib2
> > > > > output
> > > > > > > > looks
> > > > > > > > > > like this:
> > > > > > > > > >
> > > > > > > > > > 37 (t10a1)
> > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > > > wgrib2
> > > > > > > > > > pm25_DAY1
> > > > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour
ave
> fcst:
> > > > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40
hour ave
> > > fcst:
> > > > > > > > > > 38 (t10a1)
> > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > > > >
> > > > > > > > > > I have this in my point_stat config file:
> > > > > > > > > >
> > > > > > > > > > //
> > > > > > > > > > // Forecast and observation fields to be verified
> > > > > > > > > > //
> > > > > > > > > >
> > > > > > > > > > fcst = {
> > > > > > > > > >    field = [
> > > > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > > > valid_time="${CUR_VX_TIME1}";
> > > > > > > }
> > > > > > > > > >    ];
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > obs = {
> > > > > > > > > >    convert(x) = x * 10^9;
> > > > > > > > > >    field = [
> > > > > > > > > >       { name = "COPOPM"; level = "A1";
message_type =
> > > > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > > > }
> > > > > > > > > >    ];
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > But then I get this error:
> > > > > > > > > >
> > > > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2:
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > DEBUG 2:
> > > > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > > > WARNING:
> > > > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() - No
> matching
> > > > > records
> > > > > > > > found
> > > > > > > > > > for 'PDMAX1/A1'
> > > > > > > > > > WARNING:
> > > > > > > > > > WARNING:
> > > > > > > > > > WARNING: process_fcst_climo_files() -> no fields
matching
> > > > > PDMAX1/A1
> > > > > > > > found
> > > > > > > > > > in file: pm25_DAY1
> > > > > > > > > > WARNING:
> > > > > > > > > > ERROR  :
> > > > > > > > > > ERROR  : process_fcst_climo_files() -> no
requested
> > forecast
> > > > data
> > > > > > > > found!
> > > > > > > > > > Exiting...
> > > > > > > > > > ERROR  :
> > > > > > > > > >
> > > > > > > > > > Might something be wrong with the CUR_VX_TIME1 in
this
> > > > particular
> > > > > > > case?
> > > > > > > > > > You can find these files in the data already
existing on
> > > Theia
> > > > > in,
> > > > > > > for
> > > > > > > > > > example, the
> > > > > > > > > >
> > > > > >
> > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > > > directory, the file
aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > >
> > > > > > > > > > Perry
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Wed Sep 12 10:51:38 2018

Hi, John,

Super - this info should help.  I assumed that this stuff had a lot to
do
with it, but I wanted to make sure from the experts.  Thanks so much!

Once I get this working, I'd like to start developing a use case for
METplus for CMAQ using all we've done for air quality, so we can show
it at
the upcoming workshop.

Thanks!

Perry

On Wed, Sep 12, 2018 at 12:48 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Hi Perry,
>
> Julie and I just talked about your question... how to differentiate
in the
> output from Point-Stat between using MAX and MEAN observation
summaries?
>
> This info will show up in the "OBTYPE" column of the Point-Stat
output
> files.  Here's the "obs" section from your Point-Stat config file:
>
> obs = {
>    convert(x) = x * 10^9;
>    field = [
>       { name = "COPOPM"; level = "A1"; message_type =
"ANOWPM_MAX_230000";
> }
>    ];
> }
>
> You are telling Point-Stat to verify against the "ANOWPM_MAX_230000"
> message type.  And that message type string will be written to the
OBTYPE
> output column.
>
> FYI, you could also use the "desc" config file option to populate
the DESC
> output column however you'd like.  For example, setting:
>     desc = "DAILY_MAX";
> Would set the DESC output column to that string.  So that's a second
option
> if you find it more convenient.
>
> The Point-Stat config file you sent us includes a single
verification
> task... compare forecast PDMAX1 data to observation message type
> ANOWPM_MAX_230000.  If you'd also like to compare to the daily mean,
you
> could add a second verification task but for observation
message_type =
> ANOWPM_MEAN_230000.
>
> Running ncdump on the output of PB2NC and look at the
"hdr_typ_table"
> variable, here's the message types in that file:
>
> ncdump -v hdr_typ_table prepda.2018090212.nc
> ...
> hdr_typ_table =
>   "ANOWPM",
>   "ANOWPM_MAX_230000",
>   "ANOWPM_MEAN_230000" ;
>
> Hope that helps.
>
> Thanks,
> John
>
>
>
> On Wed, Sep 12, 2018 at 7:18 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> > Hi, Julie,
> >
> > Thanks!  I do want to ask (and maybe John can help out as well).
I also
> > want to plot the average PM as well.  How do I differentiate in
the
> > point_stat file whether I plot the max or the average?
> >
> > Thanks!
> >
> > Perry
> >
> > On Mon, Sep 10, 2018 at 12:53 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry.
> > >
> > > I was able to get some help with this.
> > >
> > > Perry, apologies for not getting to this issue sooner…  I took
at look
> at
> > > > the data this morning and see the problem.
> > > >
> > > >
> > > >    -
> > > >
> > > >    The real issue is “I’m having trouble getting MET to read a
> > particular
> > > >    GRIB record”.  When in that position, I find the
plot_data_plane
> > tool
> > > to be
> > > >    a quick and easy way of testing.
> > > >    -
> > > >
> > > >    So I ran the equivalent plot_data_plane command:
> > > >
> > > >
> > > >   plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A1";'
> > > >
> > > >   ERROR  : plot_data_plane -> trouble getting field
"name="PDMAX1";
> > > > level="A1";" from file "pm25_DAY1"
> > > >
> > > >
> > > >    -
> > > >
> > > >    Setting ‘level=”A1”;’ you’re telling MET to look for a 1-
hour
> > > >    accumulation of data.  But looking at the wgrib2 output you
sent,
> > the
> > > >    accumulations are over 23 hours (‘-7-16 hour ave fcst’ and
‘17-40
> > > hour ave
> > > >    fcst’)
> > > >    -
> > > >
> > > >    So use “A23” instead of “A1” and we get a plot, but also a
warning
> > > >    about multiple matches:
> > > >
> > > >
> > > > plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A23";'
> > > >
> > > > WARNING: MetGrib2DataFile::data_plane() - Multiple matching
records
> > found
> > > > for 'PDMAX1/A23' - 1, 2 - using 1
> > > >
> > > >
> > > >    -
> > > >
> > > >    You see that warning when MET finds multiple matching
records.  So
> > you
> > > >    need to specify more information to select the actual
record you
> > want…
> > > >    either “lead_time” or “valid_time”.  In the example below,
I
> > selected
> > > the
> > > >    40-hour forecast from the second record:
> > > >
> > > > plot_data_plane pm25_DAY1 pm25_DAY1_F40.ps 'name="PDMAX1";
> level="A23";
> > > > lead_time="40";'
> > > >
> > > >
> > > >    -
> > > >
> > > >    And attached is the resulting image.
> > > >
> > > >
> > > I modified A1 to A23 and ran point_stat on tide again.  I
received
> > another
> > > warning about no matching records.  I had CUR_VX_TIME1 set to
> > 20180902_05,
> > > but using wgrib2, it looks like I needed to set it to
20180902_04:
> > >
> > > [Julie.Prestopnik at t20a1 shafran_20180904]$ $WGRIB2 -vt pm25_DAY1
> > > > 1:0:vt=2018090204
> > > > 2:144904:vt=2018090304
> > >
> > >
> > > After doing that, I got a successful run.  Please try it out and
let us
> > > know how it goes.
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > >
> > >
> > > On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
>
> > > >
> > > > Hi, Julie,
> > > >
> > > > Thanks so much!
> > > >
> > > > Perry
> > > >
> > > > On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > > > > Unfortunately, I believe I need John's help.  He is working
very
> hard
> > > on
> > > > > trying to get the met-8.0 release out, however.  Please know
that
> we
> > > > > haven't forgotten about you and will get back to you as soon
as we
> > can.
> > > > >
> > > > > Thank you!
> > > > >
> > > > > Julie
> > > > >
> > > > > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > > >
> > > > > > Hi, Julie,
> > > > > >
> > > > > > Ah yes, I am setting it in a script.  Here are the
environment
> > > > variables
> > > > > > you'll need:
> > > > > >
> > > > > > CUR_VX_TIME1=20180902_05
> > > > > > MODEL=CMAQ
> > > > > > OBS=ANOWPM
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry.
> > > > > > >
> > > > > > > Thanks for the information.  I am definitely not
familiar with
> > any
> > > of
> > > > > > this
> > > > > > > verification, nor do I know what AQ stands for.  I
figured I'd
> > > enroll
> > > > > > > John's help if necessary.  In the meantime, I thought
I'd at
> > least
> > > > try
> > > > > to
> > > > > > > reproduce the problem you're experiencing.
> > > > > > >
> > > > > > > Are you calling point_stat from a script or running it
on the
> > > command
> > > > > > > line?  If you are running from a script, can you please
point
> me
> > to
> > > > it?
> > > > > > >
> > > > > > > I see a couple of environment variables in
> > > > > > >
> > > >
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1
> > to?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov
via RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > >
> > > > > > > >
> > > > > > > > Hi, Julie,
> > > > > > > >
> > > > > > > > Maybe I should have put this on the original thread
regarding
> > AQ
> > > > > > > > verification with John, but that was not a ticket so I
guess
> > it's
> > > > > good
> > > > > > to
> > > > > > > > continue here - but you may not be aware of the
genesis of AQ
> > > > > > > verification
> > > > > > > > up to this point.
> > > > > > > >
> > > > > > > > I re-ran my verification job so the directory was
> regenerated:
> > > > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > >
> > > > > > > > Here's the command:
> > > > > > > >
> > > > > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > > > > >
> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > > > > -v
> > > > > > > 4
> > > > > > > >
> > > > > > > > Average and max PM is weird because it uses 2 obs
files but
> > > writes
> > > > it
> > > > > > out
> > > > > > > > into one file as it sums up everything.  Here is the
pb2nc
> > > command:
> > > > > > > >
> > > > > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > > > > >
> > > > > > > > These commands tell you where the config files that I
used
> are
> > so
> > > > you
> > > > > > may
> > > > > > > > look at them.
> > > > > > > >
> > > > > > > > Here's the version.
> > > > > > > >
> > > > > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > > > > >
> > > > > > > > MET Version:    V7.1
> > > > > > > > Repository:
> > > > > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > > > > Revision:    5554
> > > > > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul
2018)
> > > > > > > >
> > > > > > > > I had gotten things to work for the daily max ozone
items
> but I
> > > > need
> > > > > to
> > > > > > > > work on the average and daily max PM.  I am cc'ing
John so he
> > has
> > > > > this
> > > > > > as
> > > > > > > > well and can take a look.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > > > > >
> > > > > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Perry.
> > > > > > > > >
> > > > > > > > > I tried to access the data at
> > > > > > > > >
> tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
> > > but
> > > > > > > > > unfortunately, I see it is no longer there.
> > > > > > > > >
> > > > > > > > > Could you please do the following?
> > > > > > > > >
> > > > > > > > >    1. Send me the full command line that you are
running.
> > > > > > > > >    2. Point me to your exact fcst, obs, and config
files
> > > (machine
> > > > > and
> > > > > > > > >    location information)
> > > > > > > > >    3. Tell me what version of the code you are
running -
> e.g.
> > > > > output
> > > > > > > from
> > > > > > > > >    "point_stat --version "
> > > > > > > > >
> > > > > > > > > I will try to reproduce the error that you are
getting and
> > will
> > > > > > follow
> > > > > > > > up.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Julie
> > > > > > > > >
> > > > > > > > > On Tue, Sep 4, 2018 at 11:26 AM
perry.shafran at noaa.gov via
> > RT
> > > <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted
upon.
> > > > > > > > > > Transaction: Ticket created by
perry.shafran at noaa.gov
> > > > > > > > > >        Queue: met_help
> > > > > > > > > >      Subject: Re: PM daily max and mean
verification
> > > > > > > > > >        Owner: Nobody
> > > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > > >       Status: new
> > > > > > > > > >  Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi there,
> > > > > > > > > >
> > > > > > > > > > I posted this 7 days ago and was wondering if we
can get
> a
> > > > > > response.
> > > > > > > > We
> > > > > > > > > > need to get AQM working so we can set up the AQ
use case
> > for
> > > > > > METplus
> > > > > > > > for
> > > > > > > > > > the workshop.
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > >
> > > > > > > > > > Perry
> > > > > > > > > >
> > > > > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran -
NOAA
> > > Affiliate
> > > > <
> > > > > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi, there,
> > > > > > > > > > >
> > > > > > > > > > > I am having some issues reading the max PM file.
The
> > > wgrib2
> > > > > > output
> > > > > > > > > looks
> > > > > > > > > > > like this:
> > > > > > > > > > >
> > > > > > > > > > > 37 (t10a1)
> > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > > > > wgrib2
> > > > > > > > > > > pm25_DAY1
> > > > > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour
ave
> > fcst:
> > > > > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40
hour
> ave
> > > > fcst:
> > > > > > > > > > > 38 (t10a1)
> > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > > > > >
> > > > > > > > > > > I have this in my point_stat config file:
> > > > > > > > > > >
> > > > > > > > > > > //
> > > > > > > > > > > // Forecast and observation fields to be
verified
> > > > > > > > > > > //
> > > > > > > > > > >
> > > > > > > > > > > fcst = {
> > > > > > > > > > >    field = [
> > > > > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > > > > valid_time="${CUR_VX_TIME1}";
> > > > > > > > }
> > > > > > > > > > >    ];
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > obs = {
> > > > > > > > > > >    convert(x) = x * 10^9;
> > > > > > > > > > >    field = [
> > > > > > > > > > >       { name = "COPOPM"; level = "A1";
message_type =
> > > > > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > > > > }
> > > > > > > > > > >    ];
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > But then I get this error:
> > > > > > > > > > >
> > > > > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > > > > DEBUG 2:
> > > > > > > > > > > DEBUG 2:
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > > DEBUG 2:
> > > > > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > > > > WARNING:
> > > > > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() -
No
> > matching
> > > > > > records
> > > > > > > > > found
> > > > > > > > > > > for 'PDMAX1/A1'
> > > > > > > > > > > WARNING:
> > > > > > > > > > > WARNING:
> > > > > > > > > > > WARNING: process_fcst_climo_files() -> no fields
> matching
> > > > > > PDMAX1/A1
> > > > > > > > > found
> > > > > > > > > > > in file: pm25_DAY1
> > > > > > > > > > > WARNING:
> > > > > > > > > > > ERROR  :
> > > > > > > > > > > ERROR  : process_fcst_climo_files() -> no
requested
> > > forecast
> > > > > data
> > > > > > > > > found!
> > > > > > > > > > > Exiting...
> > > > > > > > > > > ERROR  :
> > > > > > > > > > >
> > > > > > > > > > > Might something be wrong with the CUR_VX_TIME1
in this
> > > > > particular
> > > > > > > > case?
> > > > > > > > > > > You can find these files in the data already
existing
> on
> > > > Theia
> > > > > > in,
> > > > > > > > for
> > > > > > > > > > > example, the
> > > > > > > > > > >
> > > > > > >
> > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > > > > directory, the file
aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > > > > >
> > > > > > > > > > > Thanks!
> > > > > > > > > > >
> > > > > > > > > > > Perry
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: Re: PM daily max and mean verification
From: perry.shafran at noaa.gov
Time: Thu Sep 13 12:43:14 2018

OK, we're in business.  I now have gotten to work both the daily max
and
mean for the PM.  This along with the earlier achievement of getting
the
max ozone means that we basically have everything that we need for
CMAQ
verification (ozone and PM).

I think that I'd like to start to build the METplus use case that
utilizes
all of these items.  I think that I can give a stab at it, but will
let you
know if I run into problems.  The main issue will be setting up the
regex
pointers to the observations and the models, which I am sure I will
need
your assistance with.

Do let me know how I can help you help me.

Thanks!

Perry

On Wed, Sep 12, 2018 at 12:48 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Hi Perry,
>
> Julie and I just talked about your question... how to differentiate
in the
> output from Point-Stat between using MAX and MEAN observation
summaries?
>
> This info will show up in the "OBTYPE" column of the Point-Stat
output
> files.  Here's the "obs" section from your Point-Stat config file:
>
> obs = {
>    convert(x) = x * 10^9;
>    field = [
>       { name = "COPOPM"; level = "A1"; message_type =
"ANOWPM_MAX_230000";
> }
>    ];
> }
>
> You are telling Point-Stat to verify against the "ANOWPM_MAX_230000"
> message type.  And that message type string will be written to the
OBTYPE
> output column.
>
> FYI, you could also use the "desc" config file option to populate
the DESC
> output column however you'd like.  For example, setting:
>     desc = "DAILY_MAX";
> Would set the DESC output column to that string.  So that's a second
option
> if you find it more convenient.
>
> The Point-Stat config file you sent us includes a single
verification
> task... compare forecast PDMAX1 data to observation message type
> ANOWPM_MAX_230000.  If you'd also like to compare to the daily mean,
you
> could add a second verification task but for observation
message_type =
> ANOWPM_MEAN_230000.
>
> Running ncdump on the output of PB2NC and look at the
"hdr_typ_table"
> variable, here's the message types in that file:
>
> ncdump -v hdr_typ_table prepda.2018090212.nc
> ...
> hdr_typ_table =
>   "ANOWPM",
>   "ANOWPM_MAX_230000",
>   "ANOWPM_MEAN_230000" ;
>
> Hope that helps.
>
> Thanks,
> John
>
>
>
> On Wed, Sep 12, 2018 at 7:18 AM perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> >
> > Hi, Julie,
> >
> > Thanks!  I do want to ask (and maybe John can help out as well).
I also
> > want to plot the average PM as well.  How do I differentiate in
the
> > point_stat file whether I plot the max or the average?
> >
> > Thanks!
> >
> > Perry
> >
> > On Mon, Sep 10, 2018 at 12:53 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Perry.
> > >
> > > I was able to get some help with this.
> > >
> > > Perry, apologies for not getting to this issue sooner…  I took
at look
> at
> > > > the data this morning and see the problem.
> > > >
> > > >
> > > >    -
> > > >
> > > >    The real issue is “I’m having trouble getting MET to read a
> > particular
> > > >    GRIB record”.  When in that position, I find the
plot_data_plane
> > tool
> > > to be
> > > >    a quick and easy way of testing.
> > > >    -
> > > >
> > > >    So I ran the equivalent plot_data_plane command:
> > > >
> > > >
> > > >   plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A1";'
> > > >
> > > >   ERROR  : plot_data_plane -> trouble getting field
"name="PDMAX1";
> > > > level="A1";" from file "pm25_DAY1"
> > > >
> > > >
> > > >    -
> > > >
> > > >    Setting ‘level=”A1”;’ you’re telling MET to look for a 1-
hour
> > > >    accumulation of data.  But looking at the wgrib2 output you
sent,
> > the
> > > >    accumulations are over 23 hours (‘-7-16 hour ave fcst’ and
‘17-40
> > > hour ave
> > > >    fcst’)
> > > >    -
> > > >
> > > >    So use “A23” instead of “A1” and we get a plot, but also a
warning
> > > >    about multiple matches:
> > > >
> > > >
> > > > plot_data_plane pm25_DAY1 pm25_DAY1.ps 'name="PDMAX1";
level="A23";'
> > > >
> > > > WARNING: MetGrib2DataFile::data_plane() - Multiple matching
records
> > found
> > > > for 'PDMAX1/A23' - 1, 2 - using 1
> > > >
> > > >
> > > >    -
> > > >
> > > >    You see that warning when MET finds multiple matching
records.  So
> > you
> > > >    need to specify more information to select the actual
record you
> > want…
> > > >    either “lead_time” or “valid_time”.  In the example below,
I
> > selected
> > > the
> > > >    40-hour forecast from the second record:
> > > >
> > > > plot_data_plane pm25_DAY1 pm25_DAY1_F40.ps 'name="PDMAX1";
> level="A23";
> > > > lead_time="40";'
> > > >
> > > >
> > > >    -
> > > >
> > > >    And attached is the resulting image.
> > > >
> > > >
> > > I modified A1 to A23 and ran point_stat on tide again.  I
received
> > another
> > > warning about no matching records.  I had CUR_VX_TIME1 set to
> > 20180902_05,
> > > but using wgrib2, it looks like I needed to set it to
20180902_04:
> > >
> > > [Julie.Prestopnik at t20a1 shafran_20180904]$ $WGRIB2 -vt pm25_DAY1
> > > > 1:0:vt=2018090204
> > > > 2:144904:vt=2018090304
> > >
> > >
> > > After doing that, I got a successful run.  Please try it out and
let us
> > > know how it goes.
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > >
> > >
> > > On Thu, Sep 6, 2018 at 11:36 AM perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
>
> > > >
> > > > Hi, Julie,
> > > >
> > > > Thanks so much!
> > > >
> > > > Perry
> > > >
> > > > On Thu, Sep 6, 2018 at 1:32 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Thank you, Perry.  Fortunately, I am able to reproduce the
problem.
> > > > > Unfortunately, I believe I need John's help.  He is working
very
> hard
> > > on
> > > > > trying to get the met-8.0 release out, however.  Please know
that
> we
> > > > > haven't forgotten about you and will get back to you as soon
as we
> > can.
> > > > >
> > > > > Thank you!
> > > > >
> > > > > Julie
> > > > >
> > > > > On Wed, Sep 5, 2018 at 2:12 PM perry.shafran at noaa.gov via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834 >
> > > > > >
> > > > > > Hi, Julie,
> > > > > >
> > > > > > Ah yes, I am setting it in a script.  Here are the
environment
> > > > variables
> > > > > > you'll need:
> > > > > >
> > > > > > CUR_VX_TIME1=20180902_05
> > > > > > MODEL=CMAQ
> > > > > > OBS=ANOWPM
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 5, 2018 at 2:39 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Perry.
> > > > > > >
> > > > > > > Thanks for the information.  I am definitely not
familiar with
> > any
> > > of
> > > > > > this
> > > > > > > verification, nor do I know what AQ stands for.  I
figured I'd
> > > enroll
> > > > > > > John's help if necessary.  In the meantime, I thought
I'd at
> > least
> > > > try
> > > > > to
> > > > > > > reproduce the problem you're experiencing.
> > > > > > >
> > > > > > > Are you calling point_stat from a script or running it
on the
> > > command
> > > > > > > line?  If you are running from a script, can you please
point
> me
> > to
> > > > it?
> > > > > > >
> > > > > > > I see a couple of environment variables in
> > > > > > >
> > > >
>
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max.
> > > > > > > Can you please tell me what I should set MODEL and
CUR_VX_TIME1
> > to?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Wed, Sep 5, 2018 at 8:14 AM perry.shafran at noaa.gov
via RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > >
> > > > > > > >
> > > > > > > > Hi, Julie,
> > > > > > > >
> > > > > > > > Maybe I should have put this on the original thread
regarding
> > AQ
> > > > > > > > verification with John, but that was not a ticket so I
guess
> > it's
> > > > > good
> > > > > > to
> > > > > > > > continue here - but you may not be aware of the
genesis of AQ
> > > > > > > verification
> > > > > > > > up to this point.
> > > > > > > >
> > > > > > > > I re-ran my verification job so the directory was
> regenerated:
> > > > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > >
> > > > > > > > Here's the command:
> > > > > > > >
> > > > > > > > point_stat pm25_DAY1 prepda.2018090212.nc
> > > > > > > >
> > > >
/meso/save/Perry.Shafran/verif/nwtest/parm/PointStatConfig_ANOWPM_max
> > > > > > -v
> > > > > > > 4
> > > > > > > >
> > > > > > > > Average and max PM is weird because it uses 2 obs
files but
> > > writes
> > > > it
> > > > > > out
> > > > > > > > into one file as it sums up everything.  Here is the
pb2nc
> > > command:
> > > > > > > >
> > > > > > > > pb2nc prepda.2018090212 prepda.2018090212.nc
> > > > > > > > /meso/save/Perry.Shafran/verif/nwtest/
> > > > > > > > parm/PB2NCConfig_aqmmax -pbfile prepda.2018090112
> > > > > > > >
> > > > > > > > These commands tell you where the config files that I
used
> are
> > so
> > > > you
> > > > > > may
> > > > > > > > look at them.
> > > > > > > >
> > > > > > > > Here's the version.
> > > > > > > >
> > > > > > > > 4 (t14a1) /ptmpp2/Perry.Shafran/output > point_stat
--version
> > > > > > > >
> > > > > > > > MET Version:    V7.1
> > > > > > > > Repository:
> > > > > https://svn-met-dev.cgd.ucar.edu/tags/met/met-7.1_beta6
> > > > > > > > Revision:    5554
> > > > > > > > Change Date:    2018-07-09 09:22:15 -0600 (Mon, 09 Jul
2018)
> > > > > > > >
> > > > > > > > I had gotten things to work for the daily max ozone
items
> but I
> > > > need
> > > > > to
> > > > > > > > work on the average and daily max PM.  I am cc'ing
John so he
> > has
> > > > > this
> > > > > > as
> > > > > > > > well and can take a look.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > Perry
> > > > > > > >
> > > > > > > > 5 (t14a1) /ptmpp2/Perry.Shafran/output
> > > > > > > >
> > > > > > > > On Tue, Sep 4, 2018 at 3:28 PM Julie Prestopnik via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Perry.
> > > > > > > > >
> > > > > > > > > I tried to access the data at
> > > > > > > > >
> tide:/stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12,
> > > but
> > > > > > > > > unfortunately, I see it is no longer there.
> > > > > > > > >
> > > > > > > > > Could you please do the following?
> > > > > > > > >
> > > > > > > > >    1. Send me the full command line that you are
running.
> > > > > > > > >    2. Point me to your exact fcst, obs, and config
files
> > > (machine
> > > > > and
> > > > > > > > >    location information)
> > > > > > > > >    3. Tell me what version of the code you are
running -
> e.g.
> > > > > output
> > > > > > > from
> > > > > > > > >    "point_stat --version "
> > > > > > > > >
> > > > > > > > > I will try to reproduce the error that you are
getting and
> > will
> > > > > > follow
> > > > > > > > up.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Julie
> > > > > > > > >
> > > > > > > > > On Tue, Sep 4, 2018 at 11:26 AM
perry.shafran at noaa.gov via
> > RT
> > > <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Tue Sep 04 11:26:02 2018: Request 86834 was acted
upon.
> > > > > > > > > > Transaction: Ticket created by
perry.shafran at noaa.gov
> > > > > > > > > >        Queue: met_help
> > > > > > > > > >      Subject: Re: PM daily max and mean
verification
> > > > > > > > > >        Owner: Nobody
> > > > > > > > > >   Requestors: perry.shafran at noaa.gov
> > > > > > > > > >       Status: new
> > > > > > > > > >  Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86834
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi there,
> > > > > > > > > >
> > > > > > > > > > I posted this 7 days ago and was wondering if we
can get
> a
> > > > > > response.
> > > > > > > > We
> > > > > > > > > > need to get AQM working so we can set up the AQ
use case
> > for
> > > > > > METplus
> > > > > > > > for
> > > > > > > > > > the workshop.
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > >
> > > > > > > > > > Perry
> > > > > > > > > >
> > > > > > > > > > On Tue, Aug 28, 2018 at 2:37 PM Perry Shafran -
NOAA
> > > Affiliate
> > > > <
> > > > > > > > > > perry.shafran at noaa.gov> wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi, there,
> > > > > > > > > > >
> > > > > > > > > > > I am having some issues reading the max PM file.
The
> > > wgrib2
> > > > > > output
> > > > > > > > > looks
> > > > > > > > > > > like this:
> > > > > > > > > > >
> > > > > > > > > > > 37 (t10a1)
> > > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12 >
> > > > > > > > > > wgrib2
> > > > > > > > > > > pm25_DAY1
> > > > > > > > > > > 1:0:d=2018082412:PDMAX1:1 sigma level:-7-16 hour
ave
> > fcst:
> > > > > > > > > > > 2:152934:d=2018082412:PDMAX1:1 sigma level:17-40
hour
> ave
> > > > fcst:
> > > > > > > > > > > 38 (t10a1)
> > > > > /stmpp2/Perry.Shafran/tmpnwtest/verf_gridtobs_pmmax_12
> > > > > > > > > > >
> > > > > > > > > > > I have this in my point_stat config file:
> > > > > > > > > > >
> > > > > > > > > > > //
> > > > > > > > > > > // Forecast and observation fields to be
verified
> > > > > > > > > > > //
> > > > > > > > > > >
> > > > > > > > > > > fcst = {
> > > > > > > > > > >    field = [
> > > > > > > > > > >       { name = "PDMAX1"; level = "A1";
> > > > > > > valid_time="${CUR_VX_TIME1}";
> > > > > > > > }
> > > > > > > > > > >    ];
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > obs = {
> > > > > > > > > > >    convert(x) = x * 10^9;
> > > > > > > > > > >    field = [
> > > > > > > > > > >       { name = "COPOPM"; level = "A1";
message_type =
> > > > > > > > > > "ANOWPM_MAX_230000";
> > > > > > > > > > > }
> > > > > > > > > > >    ];
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > But then I get this error:
> > > > > > > > > > >
> > > > > > > > > > > DEBUG 1: Forecast File: pm25_DAY1
> > > > > > > > > > > DEBUG 1: Observation File: prepda.2018082512.nc
> > > > > > > > > > > DEBUG 2:
> > > > > > > > > > > DEBUG 2:
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------
> > > > > > > > > > > DEBUG 2:
> > > > > > > > > > > DEBUG 2: Reading data for PDMAX1/A1.
> > > > > > > > > > > WARNING:
> > > > > > > > > > > WARNING: MetGrib2DataFile::data_plane_array() -
No
> > matching
> > > > > > records
> > > > > > > > > found
> > > > > > > > > > > for 'PDMAX1/A1'
> > > > > > > > > > > WARNING:
> > > > > > > > > > > WARNING:
> > > > > > > > > > > WARNING: process_fcst_climo_files() -> no fields
> matching
> > > > > > PDMAX1/A1
> > > > > > > > > found
> > > > > > > > > > > in file: pm25_DAY1
> > > > > > > > > > > WARNING:
> > > > > > > > > > > ERROR  :
> > > > > > > > > > > ERROR  : process_fcst_climo_files() -> no
requested
> > > forecast
> > > > > data
> > > > > > > > > found!
> > > > > > > > > > > Exiting...
> > > > > > > > > > > ERROR  :
> > > > > > > > > > >
> > > > > > > > > > > Might something be wrong with the CUR_VX_TIME1
in this
> > > > > particular
> > > > > > > > case?
> > > > > > > > > > > You can find these files in the data already
existing
> on
> > > > Theia
> > > > > > in,
> > > > > > > > for
> > > > > > > > > > > example, the
> > > > > > > > > > >
> > > > > > >
> > > /scratch4/NCEPDEV/fv3-
cam/noscrub/Perry.Shafran/for_john/aqm.20180705
> > > > > > > > > > > directory, the file
aqm.t12z.max_1hr_pm25.148.grib2.
> > > > > > > > > > >
> > > > > > > > > > > Thanks!
> > > > > > > > > > >
> > > > > > > > > > > Perry
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------