[Met_help] [rt.rap.ucar.edu #81723] History for pb_report_type in PB2NC
John Halley Gotway via RT
met_help at ucar.edu
Wed Sep 6 12:12:57 MDT 2017
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi, there,
I was going over the MET documentation and there really wasn't much
information about the variable pb_report_type in PB2NC. I tried the
following:
pb_report_type = ["120","220", "221", "122", "222", "223", "224", "133",
"233", "188", "288", "180", "280", "181", "182", "281", "282", "183",
"284", "187", "287", "330", "331", "333", "430", "431", "433", "530",
"531", "533"];
These numbers are associated with the prepbufr report types used in the
table below.
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_4.htm
In the table here, for example, 120, 122, 220, 221, 222, 232, and 132 are
associated with the prepbufr message type "ADPUPA", while there are several
in the 180s and 280s associated with "ADPSFC".
It seems that pb_report_type in PB2NC isn't associated with these numerical
report types, so I am curious what this variable is expecting. Is it just
"ADPSFC" or "ADPUPA" - what we call the prepbufr message type?
Thanks!
Perry
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: pb_report_type in PB2NC
From: John Halley Gotway
Time: Wed Aug 23 09:51:43 2017
Hi Perry,
I see you have a question about the "pb_report_type" option in the
PB2NC
configuration file. In practice, I've never really used that option
when
running MET on a variety of projects. We included it because we saw
potential for users wanting to filter by it. In practice though, we
have
only ever filtered observations by the message_type (i.e.
ADPSFC/ADPUPA and
so on).
Below, I've cut-and-pasted a description of the pb_report_type from
the
met-6.0/share/met/config/README file which explains it. That README
is the
best place to look for a description of config file options:
//
// The "pb_report_type" entry is an array of PrepBufr report types to
be
// retained. The numeric "pb_report_type" entry allows for further
stratification
// within message types. An empty list indicates that all should be
retained.
//
//
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_4.htm
//
// e.g.
// Report Type 120 is for message type ADPUPA but is only RAWINSONDE
// Report Type 132 is for message type ADPUPA but is only FLIGHT-
LEVEL
RECON
// and PROFILE DROPSONDE
//
pb_report_type = [];
The important thing to note is that you do not need to provide a list
of
numbers. An empty list tells PB2NC to use all report types.
But looking at that URL listed above, I see for example that 220 and
221
are two different report types that are both included in the ADPUPA
message
type. By way of testing, I ran pb2nc on a sample NDAS file twice,
once for
220 and a second time for 221. The first time, it kept 250 of 69833
messages and the second time it kept 153 of them.
I ran the output of PB2NC through plot_point_obs to show the locations
and
have attached PNG versions the result.
So as you can see, the pb_report_type config file option is doing some
filtering, but I doubt you actually need or want to use it for the
work
you're doing.
Hope that helps clarify.
Thanks,
John
On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> Transaction: Ticket created by perry.shafran at noaa.gov
> Queue: met_help
> Subject: pb_report_type in PB2NC
> Owner: Nobody
> Requestors: perry.shafran at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>
>
> Hi, there,
>
> I was going over the MET documentation and there really wasn't much
> information about the variable pb_report_type in PB2NC. I tried the
> following:
>
> pb_report_type = ["120","220", "221", "122", "222", "223", "224",
"133",
> "233", "188", "288", "180", "280", "181", "182", "281", "282",
"183",
> "284", "187", "287", "330", "331", "333", "430", "431", "433",
"530",
> "531", "533"];
>
> These numbers are associated with the prepbufr report types used in
the
> table below.
>
>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_4.htm
>
> In the table here, for example, 120, 122, 220, 221, 222, 232, and
132 are
> associated with the prepbufr message type "ADPUPA", while there are
several
> in the 180s and 280s associated with "ADPSFC".
>
> It seems that pb_report_type in PB2NC isn't associated with these
numerical
> report types, so I am curious what this variable is expecting. Is
it just
> "ADPSFC" or "ADPUPA" - what we call the prepbufr message type?
>
> Thanks!
>
> Perry
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: perry.shafran at noaa.gov
Time: Wed Aug 23 09:55:53 2017
Hi, John,
The problem I am asking about is how to properly use this particular
variable. The way I have written it above yields an error:
ERROR :
ERROR : Dictionary::lookup_num_array() -> array "pb_report_type" does
not
contain numeric entries.
If I am to compare my VSDB verification to MET verification, I do need
to
list these numeric report types explicitly if I want to have the most
apples-to-apples comparison.
Perry
On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hi Perry,
>
> I see you have a question about the "pb_report_type" option in the
PB2NC
> configuration file. In practice, I've never really used that option
when
> running MET on a variety of projects. We included it because we saw
> potential for users wanting to filter by it. In practice though, we
have
> only ever filtered observations by the message_type (i.e.
ADPSFC/ADPUPA and
> so on).
>
> Below, I've cut-and-pasted a description of the pb_report_type from
the
> met-6.0/share/met/config/README file which explains it. That README
is
> the
> best place to look for a description of config file options:
>
> //
> // The "pb_report_type" entry is an array of PrepBufr report types
to be
> // retained. The numeric "pb_report_type" entry allows for further
> stratification
> // within message types. An empty list indicates that all should be
> retained.
> //
> // http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> doc/table_4.htm
> //
> // e.g.
> // Report Type 120 is for message type ADPUPA but is only
RAWINSONDE
> // Report Type 132 is for message type ADPUPA but is only FLIGHT-
LEVEL
> RECON
> // and PROFILE DROPSONDE
> //
> pb_report_type = [];
>
> The important thing to note is that you do not need to provide a
list of
> numbers. An empty list tells PB2NC to use all report types.
>
> But looking at that URL listed above, I see for example that 220 and
221
> are two different report types that are both included in the ADPUPA
message
> type. By way of testing, I ran pb2nc on a sample NDAS file twice,
once for
> 220 and a second time for 221. The first time, it kept 250 of 69833
> messages and the second time it kept 153 of them.
>
> I ran the output of PB2NC through plot_point_obs to show the
locations and
> have attached PNG versions the result.
>
> So as you can see, the pb_report_type config file option is doing
some
> filtering, but I doubt you actually need or want to use it for the
work
> you're doing.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
>
> On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> > Transaction: Ticket created by perry.shafran at noaa.gov
> > Queue: met_help
> > Subject: pb_report_type in PB2NC
> > Owner: Nobody
> > Requestors: perry.shafran at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> >
> >
> > Hi, there,
> >
> > I was going over the MET documentation and there really wasn't
much
> > information about the variable pb_report_type in PB2NC. I tried
the
> > following:
> >
> > pb_report_type = ["120","220", "221", "122", "222", "223", "224",
"133",
> > "233", "188", "288", "180", "280", "181", "182", "281", "282",
"183",
> > "284", "187", "287", "330", "331", "333", "430", "431", "433",
"530",
> > "531", "533"];
> >
> > These numbers are associated with the prepbufr report types used
in the
> > table below.
> >
> > http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> doc/table_4.htm
> >
> > In the table here, for example, 120, 122, 220, 221, 222, 232, and
132 are
> > associated with the prepbufr message type "ADPUPA", while there
are
> several
> > in the 180s and 280s associated with "ADPSFC".
> >
> > It seems that pb_report_type in PB2NC isn't associated with these
> numerical
> > report types, so I am curious what this variable is expecting. Is
it
> just
> > "ADPSFC" or "ADPUPA" - what we call the prepbufr message type?
> >
> > Thanks!
> >
> > Perry
> >
> >
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: John Halley Gotway
Time: Wed Aug 23 10:15:19 2017
Perry,
Ah, ok. That's a much easier question to answer. Remove the double
quotes. pb_report_type expects a list of integers, not strings.
That's
what this error message is telling you:
ERROR : Dictionary::lookup_num_array() -> array "pb_report_type" does
not
contain numeric entries.
Thanks,
John
On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>
> Hi, John,
>
> The problem I am asking about is how to properly use this particular
> variable. The way I have written it above yields an error:
>
> ERROR :
> ERROR : Dictionary::lookup_num_array() -> array "pb_report_type"
does not
> contain numeric entries.
>
> If I am to compare my VSDB verification to MET verification, I do
need to
> list these numeric report types explicitly if I want to have the
most
> apples-to-apples comparison.
>
> Perry
>
> On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Hi Perry,
> >
> > I see you have a question about the "pb_report_type" option in the
PB2NC
> > configuration file. In practice, I've never really used that
option when
> > running MET on a variety of projects. We included it because we
saw
> > potential for users wanting to filter by it. In practice though,
we have
> > only ever filtered observations by the message_type (i.e.
ADPSFC/ADPUPA
> and
> > so on).
> >
> > Below, I've cut-and-pasted a description of the pb_report_type
from the
> > met-6.0/share/met/config/README file which explains it. That
README is
> > the
> > best place to look for a description of config file options:
> >
> > //
> > // The "pb_report_type" entry is an array of PrepBufr report types
to be
> > // retained. The numeric "pb_report_type" entry allows for
further
> > stratification
> > // within message types. An empty list indicates that all should
be
> > retained.
> > //
> > // http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > doc/table_4.htm
> > //
> > // e.g.
> > // Report Type 120 is for message type ADPUPA but is only
RAWINSONDE
> > // Report Type 132 is for message type ADPUPA but is only
FLIGHT-LEVEL
> > RECON
> > // and PROFILE DROPSONDE
> > //
> > pb_report_type = [];
> >
> > The important thing to note is that you do not need to provide a
list of
> > numbers. An empty list tells PB2NC to use all report types.
> >
> > But looking at that URL listed above, I see for example that 220
and 221
> > are two different report types that are both included in the
ADPUPA
> message
> > type. By way of testing, I ran pb2nc on a sample NDAS file twice,
once
> for
> > 220 and a second time for 221. The first time, it kept 250 of
69833
> > messages and the second time it kept 153 of them.
> >
> > I ran the output of PB2NC through plot_point_obs to show the
locations
> and
> > have attached PNG versions the result.
> >
> > So as you can see, the pb_report_type config file option is doing
some
> > filtering, but I doubt you actually need or want to use it for the
work
> > you're doing.
> >
> > Hope that helps clarify.
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > Queue: met_help
> > > Subject: pb_report_type in PB2NC
> > > Owner: Nobody
> > > Requestors: perry.shafran at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723
> >
> > >
> > >
> > > Hi, there,
> > >
> > > I was going over the MET documentation and there really wasn't
much
> > > information about the variable pb_report_type in PB2NC. I tried
the
> > > following:
> > >
> > > pb_report_type = ["120","220", "221", "122", "222", "223",
"224",
> "133",
> > > "233", "188", "288", "180", "280", "181", "182", "281", "282",
"183",
> > > "284", "187", "287", "330", "331", "333", "430", "431", "433",
"530",
> > > "531", "533"];
> > >
> > > These numbers are associated with the prepbufr report types used
in the
> > > table below.
> > >
> > > http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > doc/table_4.htm
> > >
> > > In the table here, for example, 120, 122, 220, 221, 222, 232,
and 132
> are
> > > associated with the prepbufr message type "ADPUPA", while there
are
> > several
> > > in the 180s and 280s associated with "ADPSFC".
> > >
> > > It seems that pb_report_type in PB2NC isn't associated with
these
> > numerical
> > > report types, so I am curious what this variable is expecting.
Is it
> > just
> > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message type?
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: perry.shafran at noaa.gov
Time: Wed Aug 23 10:18:13 2017
Hi, John,
OK, cool, thanks!
Perry
On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Perry,
>
> Ah, ok. That's a much easier question to answer. Remove the double
> quotes. pb_report_type expects a list of integers, not strings.
That's
> what this error message is telling you:
>
> ERROR : Dictionary::lookup_num_array() -> array "pb_report_type"
does not
> contain numeric entries.
>
> Thanks,
> John
>
> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> >
> > Hi, John,
> >
> > The problem I am asking about is how to properly use this
particular
> > variable. The way I have written it above yields an error:
> >
> > ERROR :
> > ERROR : Dictionary::lookup_num_array() -> array "pb_report_type"
does
> not
> > contain numeric entries.
> >
> > If I am to compare my VSDB verification to MET verification, I do
need to
> > list these numeric report types explicitly if I want to have the
most
> > apples-to-apples comparison.
> >
> > Perry
> >
> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Hi Perry,
> > >
> > > I see you have a question about the "pb_report_type" option in
the
> PB2NC
> > > configuration file. In practice, I've never really used that
option
> when
> > > running MET on a variety of projects. We included it because we
saw
> > > potential for users wanting to filter by it. In practice
though, we
> have
> > > only ever filtered observations by the message_type (i.e.
ADPSFC/ADPUPA
> > and
> > > so on).
> > >
> > > Below, I've cut-and-pasted a description of the pb_report_type
from the
> > > met-6.0/share/met/config/README file which explains it. That
README
> is
> > > the
> > > best place to look for a description of config file options:
> > >
> > > //
> > > // The "pb_report_type" entry is an array of PrepBufr report
types to
> be
> > > // retained. The numeric "pb_report_type" entry allows for
further
> > > stratification
> > > // within message types. An empty list indicates that all
should be
> > > retained.
> > > //
> > > // http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > > doc/table_4.htm
> > > //
> > > // e.g.
> > > // Report Type 120 is for message type ADPUPA but is only
RAWINSONDE
> > > // Report Type 132 is for message type ADPUPA but is only
> FLIGHT-LEVEL
> > > RECON
> > > // and PROFILE DROPSONDE
> > > //
> > > pb_report_type = [];
> > >
> > > The important thing to note is that you do not need to provide a
list
> of
> > > numbers. An empty list tells PB2NC to use all report types.
> > >
> > > But looking at that URL listed above, I see for example that 220
and
> 221
> > > are two different report types that are both included in the
ADPUPA
> > message
> > > type. By way of testing, I ran pb2nc on a sample NDAS file
twice, once
> > for
> > > 220 and a second time for 221. The first time, it kept 250 of
69833
> > > messages and the second time it kept 153 of them.
> > >
> > > I ran the output of PB2NC through plot_point_obs to show the
locations
> > and
> > > have attached PNG versions the result.
> > >
> > > So as you can see, the pb_report_type config file option is
doing some
> > > filtering, but I doubt you actually need or want to use it for
the work
> > > you're doing.
> > >
> > > Hope that helps clarify.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > > > Queue: met_help
> > > > Subject: pb_report_type in PB2NC
> > > > Owner: Nobody
> > > > Requestors: perry.shafran at noaa.gov
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=81723
> > >
> > > >
> > > >
> > > > Hi, there,
> > > >
> > > > I was going over the MET documentation and there really wasn't
much
> > > > information about the variable pb_report_type in PB2NC. I
tried the
> > > > following:
> > > >
> > > > pb_report_type = ["120","220", "221", "122", "222", "223",
"224",
> > "133",
> > > > "233", "188", "288", "180", "280", "181", "182", "281", "282",
"183",
> > > > "284", "187", "287", "330", "331", "333", "430", "431", "433",
"530",
> > > > "531", "533"];
> > > >
> > > > These numbers are associated with the prepbufr report types
used in
> the
> > > > table below.
> > > >
> > > > http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > > doc/table_4.htm
> > > >
> > > > In the table here, for example, 120, 122, 220, 221, 222, 232,
and 132
> > are
> > > > associated with the prepbufr message type "ADPUPA", while
there are
> > > several
> > > > in the 180s and 280s associated with "ADPSFC".
> > > >
> > > > It seems that pb_report_type in PB2NC isn't associated with
these
> > > numerical
> > > > report types, so I am curious what this variable is expecting.
Is it
> > > just
> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message type?
> > > >
> > > > Thanks!
> > > >
> > > > Perry
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: perry.shafran at noaa.gov
Time: Wed Aug 23 11:01:40 2017
Hi, John,
Related to this, is this error statement:
ERROR : PB2NCConfInfo::process_config() -> the "pb_report_type"
entries
(330) must be set between 100 and 300.
I would like to request that you increase the level of allowable
report
types to 500. We have added 300- and 400- level items to the bufr
file.
These are aircraft ascent/descent profiles, with the nearest airport
surface observation attached to them.
If anyone has access to WCOSS, you can find an example of this file in
the
/com2/nam/prod directories, specifically files that look like this:
nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are appended to
the
regular prepbufr files for verification here at EMC as another source
of
profiles.
Also - these have the type AIRCAR, similar to the regular AIRCAR data.
The
original AIRCAR data are single level observations, and we run a code
here
to make ascent and descent profiles from this single-level data. It
would
be nice if we were able to have a designation for these obs rather
than
AIRCAR, to differentiate between these profiles and regular AIRCAR
data.
Please let me know if you want to discuss this further. We can add
this as
a topic to the next MET issues telecon.
Thanks!
Perry
On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA Affiliate <
perry.shafran at noaa.gov> wrote:
> Hi, John,
>
> OK, cool, thanks!
>
> Perry
>
> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Perry,
>>
>> Ah, ok. That's a much easier question to answer. Remove the
double
>> quotes. pb_report_type expects a list of integers, not strings.
That's
>> what this error message is telling you:
>>
>> ERROR : Dictionary::lookup_num_array() -> array "pb_report_type"
does not
>> contain numeric entries.
>>
>> Thanks,
>> John
>>
>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>> >
>> > Hi, John,
>> >
>> > The problem I am asking about is how to properly use this
particular
>> > variable. The way I have written it above yields an error:
>> >
>> > ERROR :
>> > ERROR : Dictionary::lookup_num_array() -> array "pb_report_type"
does
>> not
>> > contain numeric entries.
>> >
>> > If I am to compare my VSDB verification to MET verification, I do
need
>> to
>> > list these numeric report types explicitly if I want to have the
most
>> > apples-to-apples comparison.
>> >
>> > Perry
>> >
>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > > Hi Perry,
>> > >
>> > > I see you have a question about the "pb_report_type" option in
the
>> PB2NC
>> > > configuration file. In practice, I've never really used that
option
>> when
>> > > running MET on a variety of projects. We included it because
we saw
>> > > potential for users wanting to filter by it. In practice
though, we
>> have
>> > > only ever filtered observations by the message_type (i.e.
>> ADPSFC/ADPUPA
>> > and
>> > > so on).
>> > >
>> > > Below, I've cut-and-pasted a description of the pb_report_type
from
>> the
>> > > met-6.0/share/met/config/README file which explains it. That
README
>> is
>> > > the
>> > > best place to look for a description of config file options:
>> > >
>> > > //
>> > > // The "pb_report_type" entry is an array of PrepBufr report
types to
>> be
>> > > // retained. The numeric "pb_report_type" entry allows for
further
>> > > stratification
>> > > // within message types. An empty list indicates that all
should be
>> > > retained.
>> > > //
>> > > // http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
>> > > doc/table_4.htm
>> > > //
>> > > // e.g.
>> > > // Report Type 120 is for message type ADPUPA but is only
RAWINSONDE
>> > > // Report Type 132 is for message type ADPUPA but is only
>> FLIGHT-LEVEL
>> > > RECON
>> > > // and PROFILE DROPSONDE
>> > > //
>> > > pb_report_type = [];
>> > >
>> > > The important thing to note is that you do not need to provide
a list
>> of
>> > > numbers. An empty list tells PB2NC to use all report types.
>> > >
>> > > But looking at that URL listed above, I see for example that
220 and
>> 221
>> > > are two different report types that are both included in the
ADPUPA
>> > message
>> > > type. By way of testing, I ran pb2nc on a sample NDAS file
twice,
>> once
>> > for
>> > > 220 and a second time for 221. The first time, it kept 250 of
69833
>> > > messages and the second time it kept 153 of them.
>> > >
>> > > I ran the output of PB2NC through plot_point_obs to show the
locations
>> > and
>> > > have attached PNG versions the result.
>> > >
>> > > So as you can see, the pb_report_type config file option is
doing some
>> > > filtering, but I doubt you actually need or want to use it for
the
>> work
>> > > you're doing.
>> > >
>> > > Hope that helps clarify.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > >
>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via RT
<
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
>> > > > Queue: met_help
>> > > > Subject: pb_report_type in PB2NC
>> > > > Owner: Nobody
>> > > > Requestors: perry.shafran at noaa.gov
>> > > > Status: new
>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>> ket/Display.html?id=81723
>> > >
>> > > >
>> > > >
>> > > > Hi, there,
>> > > >
>> > > > I was going over the MET documentation and there really
wasn't much
>> > > > information about the variable pb_report_type in PB2NC. I
tried the
>> > > > following:
>> > > >
>> > > > pb_report_type = ["120","220", "221", "122", "222", "223",
"224",
>> > "133",
>> > > > "233", "188", "288", "180", "280", "181", "182", "281",
"282",
>> "183",
>> > > > "284", "187", "287", "330", "331", "333", "430", "431",
"433",
>> "530",
>> > > > "531", "533"];
>> > > >
>> > > > These numbers are associated with the prepbufr report types
used in
>> the
>> > > > table below.
>> > > >
>> > > > http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
>> > > doc/table_4.htm
>> > > >
>> > > > In the table here, for example, 120, 122, 220, 221, 222, 232,
and
>> 132
>> > are
>> > > > associated with the prepbufr message type "ADPUPA", while
there are
>> > > several
>> > > > in the 180s and 280s associated with "ADPSFC".
>> > > >
>> > > > It seems that pb_report_type in PB2NC isn't associated with
these
>> > > numerical
>> > > > report types, so I am curious what this variable is
expecting. Is
>> it
>> > > just
>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message
type?
>> > > >
>> > > > Thanks!
>> > > >
>> > > > Perry
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: perry.shafran at noaa.gov
Time: Wed Aug 23 11:02:51 2017
BTW - make that making the max 600. We have 500-level report types as
well
I think.
Perry
On Wed, Aug 23, 2017 at 1:01 PM, Perry Shafran - NOAA Affiliate <
perry.shafran at noaa.gov> wrote:
> Hi, John,
>
> Related to this, is this error statement:
>
> ERROR : PB2NCConfInfo::process_config() -> the "pb_report_type"
entries
> (330) must be set between 100 and 300.
>
> I would like to request that you increase the level of allowable
report
> types to 500. We have added 300- and 400- level items to the bufr
file.
> These are aircraft ascent/descent profiles, with the nearest airport
> surface observation attached to them.
>
> If anyone has access to WCOSS, you can find an example of this file
in the
> /com2/nam/prod directories, specifically files that look like this:
> nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are appended
to
> the regular prepbufr files for verification here at EMC as another
source
> of profiles.
>
> Also - these have the type AIRCAR, similar to the regular AIRCAR
data.
> The original AIRCAR data are single level observations, and we run a
code
> here to make ascent and descent profiles from this single-level
data. It
> would be nice if we were able to have a designation for these obs
rather
> than AIRCAR, to differentiate between these profiles and regular
AIRCAR
> data.
>
> Please let me know if you want to discuss this further. We can add
this
> as a topic to the next MET issues telecon.
>
> Thanks!
>
> Perry
>
> On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA Affiliate <
> perry.shafran at noaa.gov> wrote:
>
>> Hi, John,
>>
>> OK, cool, thanks!
>>
>> Perry
>>
>> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> Perry,
>>>
>>> Ah, ok. That's a much easier question to answer. Remove the
double
>>> quotes. pb_report_type expects a list of integers, not strings.
That's
>>> what this error message is telling you:
>>>
>>> ERROR : Dictionary::lookup_num_array() -> array "pb_report_type"
does
>>> not
>>> contain numeric entries.
>>>
>>> Thanks,
>>> John
>>>
>>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>>> >
>>> > Hi, John,
>>> >
>>> > The problem I am asking about is how to properly use this
particular
>>> > variable. The way I have written it above yields an error:
>>> >
>>> > ERROR :
>>> > ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type" does
>>> not
>>> > contain numeric entries.
>>> >
>>> > If I am to compare my VSDB verification to MET verification, I
do need
>>> to
>>> > list these numeric report types explicitly if I want to have the
most
>>> > apples-to-apples comparison.
>>> >
>>> > Perry
>>> >
>>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
>>> > met_help at ucar.edu> wrote:
>>> >
>>> > > Hi Perry,
>>> > >
>>> > > I see you have a question about the "pb_report_type" option in
the
>>> PB2NC
>>> > > configuration file. In practice, I've never really used that
option
>>> when
>>> > > running MET on a variety of projects. We included it because
we saw
>>> > > potential for users wanting to filter by it. In practice
though, we
>>> have
>>> > > only ever filtered observations by the message_type (i.e.
>>> ADPSFC/ADPUPA
>>> > and
>>> > > so on).
>>> > >
>>> > > Below, I've cut-and-pasted a description of the pb_report_type
from
>>> the
>>> > > met-6.0/share/met/config/README file which explains it. That
>>> README is
>>> > > the
>>> > > best place to look for a description of config file options:
>>> > >
>>> > > //
>>> > > // The "pb_report_type" entry is an array of PrepBufr report
types
>>> to be
>>> > > // retained. The numeric "pb_report_type" entry allows for
further
>>> > > stratification
>>> > > // within message types. An empty list indicates that all
should be
>>> > > retained.
>>> > > //
>>> > > // http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
>>> > > doc/table_4.htm
>>> > > //
>>> > > // e.g.
>>> > > // Report Type 120 is for message type ADPUPA but is only
>>> RAWINSONDE
>>> > > // Report Type 132 is for message type ADPUPA but is only
>>> FLIGHT-LEVEL
>>> > > RECON
>>> > > // and PROFILE DROPSONDE
>>> > > //
>>> > > pb_report_type = [];
>>> > >
>>> > > The important thing to note is that you do not need to provide
a
>>> list of
>>> > > numbers. An empty list tells PB2NC to use all report types.
>>> > >
>>> > > But looking at that URL listed above, I see for example that
220 and
>>> 221
>>> > > are two different report types that are both included in the
ADPUPA
>>> > message
>>> > > type. By way of testing, I ran pb2nc on a sample NDAS file
twice,
>>> once
>>> > for
>>> > > 220 and a second time for 221. The first time, it kept 250 of
69833
>>> > > messages and the second time it kept 153 of them.
>>> > >
>>> > > I ran the output of PB2NC through plot_point_obs to show the
>>> locations
>>> > and
>>> > > have attached PNG versions the result.
>>> > >
>>> > > So as you can see, the pb_report_type config file option is
doing
>>> some
>>> > > filtering, but I doubt you actually need or want to use it for
the
>>> work
>>> > > you're doing.
>>> > >
>>> > > Hope that helps clarify.
>>> > >
>>> > > Thanks,
>>> > > John
>>> > >
>>> > >
>>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via RT
<
>>> > > met_help at ucar.edu> wrote:
>>> > >
>>> > > >
>>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
>>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
>>> > > > Queue: met_help
>>> > > > Subject: pb_report_type in PB2NC
>>> > > > Owner: Nobody
>>> > > > Requestors: perry.shafran at noaa.gov
>>> > > > Status: new
>>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>>> ket/Display.html?id=81723
>>> > >
>>> > > >
>>> > > >
>>> > > > Hi, there,
>>> > > >
>>> > > > I was going over the MET documentation and there really
wasn't much
>>> > > > information about the variable pb_report_type in PB2NC. I
tried
>>> the
>>> > > > following:
>>> > > >
>>> > > > pb_report_type = ["120","220", "221", "122", "222", "223",
"224",
>>> > "133",
>>> > > > "233", "188", "288", "180", "280", "181", "182", "281",
"282",
>>> "183",
>>> > > > "284", "187", "287", "330", "331", "333", "430", "431",
"433",
>>> "530",
>>> > > > "531", "533"];
>>> > > >
>>> > > > These numbers are associated with the prepbufr report types
used
>>> in the
>>> > > > table below.
>>> > > >
>>> > > > http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
>>> > > doc/table_4.htm
>>> > > >
>>> > > > In the table here, for example, 120, 122, 220, 221, 222,
232, and
>>> 132
>>> > are
>>> > > > associated with the prepbufr message type "ADPUPA", while
there are
>>> > > several
>>> > > > in the 180s and 280s associated with "ADPSFC".
>>> > > >
>>> > > > It seems that pb_report_type in PB2NC isn't associated with
these
>>> > > numerical
>>> > > > report types, so I am curious what this variable is
expecting. Is
>>> it
>>> > > just
>>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message
type?
>>> > > >
>>> > > > Thanks!
>>> > > >
>>> > > > Perry
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> >
>>> >
>>>
>>>
>>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: John Halley Gotway
Time: Wed Aug 23 16:09:41 2017
Perry,
There really is no need to error check these values at all. The
reason for
doing so was just a sanity check on them to help users avoid mistakes.
But
they are subject to change, there's no need to do any sanity checking.
We are currently checking that:
100 <= pb_report_type <= 300
0 <= level_category <= 7
0 <= quality_mark_thresh <= 15
But if there's no good reason to do so, we can easily remove those
checks
from the file pb2nc_conf_info.cc.
What do you think and when do you need this? As a patch for met-6.0
or in
the met-6.1 release?
Thanks,
John
On Wed, Aug 23, 2017 at 11:02 AM, perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>
> BTW - make that making the max 600. We have 500-level report types
as well
> I think.
>
> Perry
>
> On Wed, Aug 23, 2017 at 1:01 PM, Perry Shafran - NOAA Affiliate <
> perry.shafran at noaa.gov> wrote:
>
> > Hi, John,
> >
> > Related to this, is this error statement:
> >
> > ERROR : PB2NCConfInfo::process_config() -> the "pb_report_type"
entries
> > (330) must be set between 100 and 300.
> >
> > I would like to request that you increase the level of allowable
report
> > types to 500. We have added 300- and 400- level items to the bufr
file.
> > These are aircraft ascent/descent profiles, with the nearest
airport
> > surface observation attached to them.
> >
> > If anyone has access to WCOSS, you can find an example of this
file in
> the
> > /com2/nam/prod directories, specifically files that look like
this:
> > nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are
appended to
> > the regular prepbufr files for verification here at EMC as another
source
> > of profiles.
> >
> > Also - these have the type AIRCAR, similar to the regular AIRCAR
data.
> > The original AIRCAR data are single level observations, and we run
a code
> > here to make ascent and descent profiles from this single-level
data. It
> > would be nice if we were able to have a designation for these obs
rather
> > than AIRCAR, to differentiate between these profiles and regular
AIRCAR
> > data.
> >
> > Please let me know if you want to discuss this further. We can
add this
> > as a topic to the next MET issues telecon.
> >
> > Thanks!
> >
> > Perry
> >
> > On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA Affiliate <
> > perry.shafran at noaa.gov> wrote:
> >
> >> Hi, John,
> >>
> >> OK, cool, thanks!
> >>
> >> Perry
> >>
> >> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >>> Perry,
> >>>
> >>> Ah, ok. That's a much easier question to answer. Remove the
double
> >>> quotes. pb_report_type expects a list of integers, not strings.
> That's
> >>> what this error message is telling you:
> >>>
> >>> ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type" does
> >>> not
> >>> contain numeric entries.
> >>>
> >>> Thanks,
> >>> John
> >>>
> >>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT <
> >>> met_help at ucar.edu> wrote:
> >>>
> >>> >
> >>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723
>
> >>> >
> >>> > Hi, John,
> >>> >
> >>> > The problem I am asking about is how to properly use this
particular
> >>> > variable. The way I have written it above yields an error:
> >>> >
> >>> > ERROR :
> >>> > ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
> does
> >>> not
> >>> > contain numeric entries.
> >>> >
> >>> > If I am to compare my VSDB verification to MET verification, I
do
> need
> >>> to
> >>> > list these numeric report types explicitly if I want to have
the most
> >>> > apples-to-apples comparison.
> >>> >
> >>> > Perry
> >>> >
> >>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
> >>> > met_help at ucar.edu> wrote:
> >>> >
> >>> > > Hi Perry,
> >>> > >
> >>> > > I see you have a question about the "pb_report_type" option
in the
> >>> PB2NC
> >>> > > configuration file. In practice, I've never really used
that
> option
> >>> when
> >>> > > running MET on a variety of projects. We included it
because we
> saw
> >>> > > potential for users wanting to filter by it. In practice
though,
> we
> >>> have
> >>> > > only ever filtered observations by the message_type (i.e.
> >>> ADPSFC/ADPUPA
> >>> > and
> >>> > > so on).
> >>> > >
> >>> > > Below, I've cut-and-pasted a description of the
pb_report_type from
> >>> the
> >>> > > met-6.0/share/met/config/README file which explains it.
That
> >>> README is
> >>> > > the
> >>> > > best place to look for a description of config file options:
> >>> > >
> >>> > > //
> >>> > > // The "pb_report_type" entry is an array of PrepBufr report
types
> >>> to be
> >>> > > // retained. The numeric "pb_report_type" entry allows for
further
> >>> > > stratification
> >>> > > // within message types. An empty list indicates that all
should
> be
> >>> > > retained.
> >>> > > //
> >>> > > //
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> >>> > > doc/table_4.htm
> >>> > > //
> >>> > > // e.g.
> >>> > > // Report Type 120 is for message type ADPUPA but is only
> >>> RAWINSONDE
> >>> > > // Report Type 132 is for message type ADPUPA but is only
> >>> FLIGHT-LEVEL
> >>> > > RECON
> >>> > > // and PROFILE DROPSONDE
> >>> > > //
> >>> > > pb_report_type = [];
> >>> > >
> >>> > > The important thing to note is that you do not need to
provide a
> >>> list of
> >>> > > numbers. An empty list tells PB2NC to use all report types.
> >>> > >
> >>> > > But looking at that URL listed above, I see for example that
220
> and
> >>> 221
> >>> > > are two different report types that are both included in the
ADPUPA
> >>> > message
> >>> > > type. By way of testing, I ran pb2nc on a sample NDAS file
twice,
> >>> once
> >>> > for
> >>> > > 220 and a second time for 221. The first time, it kept 250
of
> 69833
> >>> > > messages and the second time it kept 153 of them.
> >>> > >
> >>> > > I ran the output of PB2NC through plot_point_obs to show the
> >>> locations
> >>> > and
> >>> > > have attached PNG versions the result.
> >>> > >
> >>> > > So as you can see, the pb_report_type config file option is
doing
> >>> some
> >>> > > filtering, but I doubt you actually need or want to use it
for the
> >>> work
> >>> > > you're doing.
> >>> > >
> >>> > > Hope that helps clarify.
> >>> > >
> >>> > > Thanks,
> >>> > > John
> >>> > >
> >>> > >
> >>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via
RT <
> >>> > > met_help at ucar.edu> wrote:
> >>> > >
> >>> > > >
> >>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> >>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> >>> > > > Queue: met_help
> >>> > > > Subject: pb_report_type in PB2NC
> >>> > > > Owner: Nobody
> >>> > > > Requestors: perry.shafran at noaa.gov
> >>> > > > Status: new
> >>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> >>> ket/Display.html?id=81723
> >>> > >
> >>> > > >
> >>> > > >
> >>> > > > Hi, there,
> >>> > > >
> >>> > > > I was going over the MET documentation and there really
wasn't
> much
> >>> > > > information about the variable pb_report_type in PB2NC. I
tried
> >>> the
> >>> > > > following:
> >>> > > >
> >>> > > > pb_report_type = ["120","220", "221", "122", "222",
"223",
> "224",
> >>> > "133",
> >>> > > > "233", "188", "288", "180", "280", "181", "182", "281",
"282",
> >>> "183",
> >>> > > > "284", "187", "287", "330", "331", "333", "430", "431",
"433",
> >>> "530",
> >>> > > > "531", "533"];
> >>> > > >
> >>> > > > These numbers are associated with the prepbufr report
types used
> >>> in the
> >>> > > > table below.
> >>> > > >
> >>> > > > http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> >>> > > doc/table_4.htm
> >>> > > >
> >>> > > > In the table here, for example, 120, 122, 220, 221, 222,
232, and
> >>> 132
> >>> > are
> >>> > > > associated with the prepbufr message type "ADPUPA", while
there
> are
> >>> > > several
> >>> > > > in the 180s and 280s associated with "ADPSFC".
> >>> > > >
> >>> > > > It seems that pb_report_type in PB2NC isn't associated
with these
> >>> > > numerical
> >>> > > > report types, so I am curious what this variable is
expecting.
> Is
> >>> it
> >>> > > just
> >>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message
type?
> >>> > > >
> >>> > > > Thanks!
> >>> > > >
> >>> > > > Perry
> >>> > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> >
> >>> >
> >>>
> >>>
> >>
> >
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: John Halley Gotway
Time: Wed Aug 23 16:29:27 2017
Perry,
I'm thinking about the AIRCAR issue you mentioned. So for example,
let's
say you have two different PREPBUFR files that came from different
sources. When you process the first file, you'd like AIRCAR
observations
to be written out as AIRCAR. But when you process the second file,
you'd
like AIRCAR observations to be written out as something else... maybe
AIRCAR_PROFILES or something.
I can think of two ways we might accomplish this... both by adding a
new
config file option.
Option 1:
Add new config file option for:
message_type_suffix = "_PROFILES";
That string will be appended to all message types that are read from
the
current file.
Option 2:
Add new config file option for:
message_type_map = [
{ key = "AIRCAR"; val = "AIRCAR_PROFILES"; },
{ key = "ADPUPA"; val = "UppperAir"; }
];
I prefer option 2 because it's consistent with the an "obs_var_map"
option
that we're already adding for met-6.1. It also gives you much finer
control over which message types should be changed an exactly how.
Do you think this solution would meet your needs?
Thanks,
John
On Wed, Aug 23, 2017 at 4:09 PM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Perry,
>
> There really is no need to error check these values at all. The
reason
> for doing so was just a sanity check on them to help users avoid
mistakes.
> But they are subject to change, there's no need to do any sanity
checking.
>
> We are currently checking that:
> 100 <= pb_report_type <= 300
> 0 <= level_category <= 7
> 0 <= quality_mark_thresh <= 15
>
> But if there's no good reason to do so, we can easily remove those
checks
> from the file pb2nc_conf_info.cc.
>
> What do you think and when do you need this? As a patch for met-6.0
or in
> the met-6.1 release?
>
> Thanks,
> John
>
>
> On Wed, Aug 23, 2017 at 11:02 AM, perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>>
>> BTW - make that making the max 600. We have 500-level report types
as
>> well
>> I think.
>>
>> Perry
>>
>> On Wed, Aug 23, 2017 at 1:01 PM, Perry Shafran - NOAA Affiliate <
>> perry.shafran at noaa.gov> wrote:
>>
>> > Hi, John,
>> >
>> > Related to this, is this error statement:
>> >
>> > ERROR : PB2NCConfInfo::process_config() -> the "pb_report_type"
>> entries
>> > (330) must be set between 100 and 300.
>> >
>> > I would like to request that you increase the level of allowable
report
>> > types to 500. We have added 300- and 400- level items to the
bufr file.
>> > These are aircraft ascent/descent profiles, with the nearest
airport
>> > surface observation attached to them.
>> >
>> > If anyone has access to WCOSS, you can find an example of this
file in
>> the
>> > /com2/nam/prod directories, specifically files that look like
this:
>> > nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are
appended to
>> > the regular prepbufr files for verification here at EMC as
another
>> source
>> > of profiles.
>> >
>> > Also - these have the type AIRCAR, similar to the regular AIRCAR
data.
>> > The original AIRCAR data are single level observations, and we
run a
>> code
>> > here to make ascent and descent profiles from this single-level
data.
>> It
>> > would be nice if we were able to have a designation for these obs
rather
>> > than AIRCAR, to differentiate between these profiles and regular
AIRCAR
>> > data.
>> >
>> > Please let me know if you want to discuss this further. We can
add this
>> > as a topic to the next MET issues telecon.
>> >
>> > Thanks!
>> >
>> > Perry
>> >
>> > On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA Affiliate
<
>> > perry.shafran at noaa.gov> wrote:
>> >
>> >> Hi, John,
>> >>
>> >> OK, cool, thanks!
>> >>
>> >> Perry
>> >>
>> >> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
>> >> met_help at ucar.edu> wrote:
>> >>
>> >>> Perry,
>> >>>
>> >>> Ah, ok. That's a much easier question to answer. Remove the
double
>> >>> quotes. pb_report_type expects a list of integers, not
strings.
>> That's
>> >>> what this error message is telling you:
>> >>>
>> >>> ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type" does
>> >>> not
>> >>> contain numeric entries.
>> >>>
>> >>> Thanks,
>> >>> John
>> >>>
>> >>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT
<
>> >>> met_help at ucar.edu> wrote:
>> >>>
>> >>> >
>> >>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723
>
>> >>> >
>> >>> > Hi, John,
>> >>> >
>> >>> > The problem I am asking about is how to properly use this
particular
>> >>> > variable. The way I have written it above yields an error:
>> >>> >
>> >>> > ERROR :
>> >>> > ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
>> does
>> >>> not
>> >>> > contain numeric entries.
>> >>> >
>> >>> > If I am to compare my VSDB verification to MET verification,
I do
>> need
>> >>> to
>> >>> > list these numeric report types explicitly if I want to have
the
>> most
>> >>> > apples-to-apples comparison.
>> >>> >
>> >>> > Perry
>> >>> >
>> >>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT <
>> >>> > met_help at ucar.edu> wrote:
>> >>> >
>> >>> > > Hi Perry,
>> >>> > >
>> >>> > > I see you have a question about the "pb_report_type" option
in the
>> >>> PB2NC
>> >>> > > configuration file. In practice, I've never really used
that
>> option
>> >>> when
>> >>> > > running MET on a variety of projects. We included it
because we
>> saw
>> >>> > > potential for users wanting to filter by it. In practice
though,
>> we
>> >>> have
>> >>> > > only ever filtered observations by the message_type (i.e.
>> >>> ADPSFC/ADPUPA
>> >>> > and
>> >>> > > so on).
>> >>> > >
>> >>> > > Below, I've cut-and-pasted a description of the
pb_report_type
>> from
>> >>> the
>> >>> > > met-6.0/share/met/config/README file which explains it.
That
>> >>> README is
>> >>> > > the
>> >>> > > best place to look for a description of config file
options:
>> >>> > >
>> >>> > > //
>> >>> > > // The "pb_report_type" entry is an array of PrepBufr
report types
>> >>> to be
>> >>> > > // retained. The numeric "pb_report_type" entry allows for
>> further
>> >>> > > stratification
>> >>> > > // within message types. An empty list indicates that all
should
>> be
>> >>> > > retained.
>> >>> > > //
>> >>> > > //
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
>> >>> > > doc/table_4.htm
>> >>> > > //
>> >>> > > // e.g.
>> >>> > > // Report Type 120 is for message type ADPUPA but is only
>> >>> RAWINSONDE
>> >>> > > // Report Type 132 is for message type ADPUPA but is only
>> >>> FLIGHT-LEVEL
>> >>> > > RECON
>> >>> > > // and PROFILE DROPSONDE
>> >>> > > //
>> >>> > > pb_report_type = [];
>> >>> > >
>> >>> > > The important thing to note is that you do not need to
provide a
>> >>> list of
>> >>> > > numbers. An empty list tells PB2NC to use all report
types.
>> >>> > >
>> >>> > > But looking at that URL listed above, I see for example
that 220
>> and
>> >>> 221
>> >>> > > are two different report types that are both included in
the
>> ADPUPA
>> >>> > message
>> >>> > > type. By way of testing, I ran pb2nc on a sample NDAS file
twice,
>> >>> once
>> >>> > for
>> >>> > > 220 and a second time for 221. The first time, it kept 250
of
>> 69833
>> >>> > > messages and the second time it kept 153 of them.
>> >>> > >
>> >>> > > I ran the output of PB2NC through plot_point_obs to show
the
>> >>> locations
>> >>> > and
>> >>> > > have attached PNG versions the result.
>> >>> > >
>> >>> > > So as you can see, the pb_report_type config file option is
doing
>> >>> some
>> >>> > > filtering, but I doubt you actually need or want to use it
for the
>> >>> work
>> >>> > > you're doing.
>> >>> > >
>> >>> > > Hope that helps clarify.
>> >>> > >
>> >>> > > Thanks,
>> >>> > > John
>> >>> > >
>> >>> > >
>> >>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov via
RT <
>> >>> > > met_help at ucar.edu> wrote:
>> >>> > >
>> >>> > > >
>> >>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
>> >>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
>> >>> > > > Queue: met_help
>> >>> > > > Subject: pb_report_type in PB2NC
>> >>> > > > Owner: Nobody
>> >>> > > > Requestors: perry.shafran at noaa.gov
>> >>> > > > Status: new
>> >>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>> >>> ket/Display.html?id=81723
>> >>> > >
>> >>> > > >
>> >>> > > >
>> >>> > > > Hi, there,
>> >>> > > >
>> >>> > > > I was going over the MET documentation and there really
wasn't
>> much
>> >>> > > > information about the variable pb_report_type in PB2NC.
I tried
>> >>> the
>> >>> > > > following:
>> >>> > > >
>> >>> > > > pb_report_type = ["120","220", "221", "122", "222",
"223",
>> "224",
>> >>> > "133",
>> >>> > > > "233", "188", "288", "180", "280", "181", "182", "281",
"282",
>> >>> "183",
>> >>> > > > "284", "187", "287", "330", "331", "333", "430", "431",
"433",
>> >>> "530",
>> >>> > > > "531", "533"];
>> >>> > > >
>> >>> > > > These numbers are associated with the prepbufr report
types used
>> >>> in the
>> >>> > > > table below.
>> >>> > > >
>> >>> > > >
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
>> >>> > > doc/table_4.htm
>> >>> > > >
>> >>> > > > In the table here, for example, 120, 122, 220, 221, 222,
232,
>> and
>> >>> 132
>> >>> > are
>> >>> > > > associated with the prepbufr message type "ADPUPA", while
there
>> are
>> >>> > > several
>> >>> > > > in the 180s and 280s associated with "ADPSFC".
>> >>> > > >
>> >>> > > > It seems that pb_report_type in PB2NC isn't associated
with
>> these
>> >>> > > numerical
>> >>> > > > report types, so I am curious what this variable is
expecting.
>> Is
>> >>> it
>> >>> > > just
>> >>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message
type?
>> >>> > > >
>> >>> > > > Thanks!
>> >>> > > >
>> >>> > > > Perry
>> >>> > > >
>> >>> > > >
>> >>> > >
>> >>> > >
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>
>> >
>>
>>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: perry.shafran at noaa.gov
Time: Thu Aug 24 06:40:01 2017
Hi, John,
There is no rush here - 6.1 release would be just fine.
I think that keeping the error checks might not be a bad idea though,
so
people don't put in the wrong values for these.
Perry
On Wed, Aug 23, 2017 at 6:09 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Perry,
>
> There really is no need to error check these values at all. The
reason for
> doing so was just a sanity check on them to help users avoid
mistakes. But
> they are subject to change, there's no need to do any sanity
checking.
>
> We are currently checking that:
> 100 <= pb_report_type <= 300
> 0 <= level_category <= 7
> 0 <= quality_mark_thresh <= 15
>
> But if there's no good reason to do so, we can easily remove those
checks
> from the file pb2nc_conf_info.cc.
>
> What do you think and when do you need this? As a patch for met-6.0
or in
> the met-6.1 release?
>
> Thanks,
> John
>
>
> On Wed, Aug 23, 2017 at 11:02 AM, perry.shafran at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> >
> > BTW - make that making the max 600. We have 500-level report
types as
> well
> > I think.
> >
> > Perry
> >
> > On Wed, Aug 23, 2017 at 1:01 PM, Perry Shafran - NOAA Affiliate <
> > perry.shafran at noaa.gov> wrote:
> >
> > > Hi, John,
> > >
> > > Related to this, is this error statement:
> > >
> > > ERROR : PB2NCConfInfo::process_config() -> the "pb_report_type"
> entries
> > > (330) must be set between 100 and 300.
> > >
> > > I would like to request that you increase the level of allowable
report
> > > types to 500. We have added 300- and 400- level items to the
bufr
> file.
> > > These are aircraft ascent/descent profiles, with the nearest
airport
> > > surface observation attached to them.
> > >
> > > If anyone has access to WCOSS, you can find an example of this
file in
> > the
> > > /com2/nam/prod directories, specifically files that look like
this:
> > > nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are
appended to
> > > the regular prepbufr files for verification here at EMC as
another
> source
> > > of profiles.
> > >
> > > Also - these have the type AIRCAR, similar to the regular AIRCAR
data.
> > > The original AIRCAR data are single level observations, and we
run a
> code
> > > here to make ascent and descent profiles from this single-level
data.
> It
> > > would be nice if we were able to have a designation for these
obs
> rather
> > > than AIRCAR, to differentiate between these profiles and regular
AIRCAR
> > > data.
> > >
> > > Please let me know if you want to discuss this further. We can
add
> this
> > > as a topic to the next MET issues telecon.
> > >
> > > Thanks!
> > >
> > > Perry
> > >
> > > On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA Affiliate
<
> > > perry.shafran at noaa.gov> wrote:
> > >
> > >> Hi, John,
> > >>
> > >> OK, cool, thanks!
> > >>
> > >> Perry
> > >>
> > >> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >>> Perry,
> > >>>
> > >>> Ah, ok. That's a much easier question to answer. Remove the
double
> > >>> quotes. pb_report_type expects a list of integers, not
strings.
> > That's
> > >>> what this error message is telling you:
> > >>>
> > >>> ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
> does
> > >>> not
> > >>> contain numeric entries.
> > >>>
> > >>> Thanks,
> > >>> John
> > >>>
> > >>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via RT
<
> > >>> met_help at ucar.edu> wrote:
> > >>>
> > >>> >
> > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> > >>> >
> > >>> > Hi, John,
> > >>> >
> > >>> > The problem I am asking about is how to properly use this
> particular
> > >>> > variable. The way I have written it above yields an error:
> > >>> >
> > >>> > ERROR :
> > >>> > ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
> > does
> > >>> not
> > >>> > contain numeric entries.
> > >>> >
> > >>> > If I am to compare my VSDB verification to MET verification,
I do
> > need
> > >>> to
> > >>> > list these numeric report types explicitly if I want to have
the
> most
> > >>> > apples-to-apples comparison.
> > >>> >
> > >>> > Perry
> > >>> >
> > >>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT
<
> > >>> > met_help at ucar.edu> wrote:
> > >>> >
> > >>> > > Hi Perry,
> > >>> > >
> > >>> > > I see you have a question about the "pb_report_type"
option in
> the
> > >>> PB2NC
> > >>> > > configuration file. In practice, I've never really used
that
> > option
> > >>> when
> > >>> > > running MET on a variety of projects. We included it
because we
> > saw
> > >>> > > potential for users wanting to filter by it. In practice
though,
> > we
> > >>> have
> > >>> > > only ever filtered observations by the message_type (i.e.
> > >>> ADPSFC/ADPUPA
> > >>> > and
> > >>> > > so on).
> > >>> > >
> > >>> > > Below, I've cut-and-pasted a description of the
pb_report_type
> from
> > >>> the
> > >>> > > met-6.0/share/met/config/README file which explains it.
That
> > >>> README is
> > >>> > > the
> > >>> > > best place to look for a description of config file
options:
> > >>> > >
> > >>> > > //
> > >>> > > // The "pb_report_type" entry is an array of PrepBufr
report
> types
> > >>> to be
> > >>> > > // retained. The numeric "pb_report_type" entry allows
for
> further
> > >>> > > stratification
> > >>> > > // within message types. An empty list indicates that all
should
> > be
> > >>> > > retained.
> > >>> > > //
> > >>> > > //
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > >>> > > doc/table_4.htm
> > >>> > > //
> > >>> > > // e.g.
> > >>> > > // Report Type 120 is for message type ADPUPA but is
only
> > >>> RAWINSONDE
> > >>> > > // Report Type 132 is for message type ADPUPA but is
only
> > >>> FLIGHT-LEVEL
> > >>> > > RECON
> > >>> > > // and PROFILE DROPSONDE
> > >>> > > //
> > >>> > > pb_report_type = [];
> > >>> > >
> > >>> > > The important thing to note is that you do not need to
provide a
> > >>> list of
> > >>> > > numbers. An empty list tells PB2NC to use all report
types.
> > >>> > >
> > >>> > > But looking at that URL listed above, I see for example
that 220
> > and
> > >>> 221
> > >>> > > are two different report types that are both included in
the
> ADPUPA
> > >>> > message
> > >>> > > type. By way of testing, I ran pb2nc on a sample NDAS
file
> twice,
> > >>> once
> > >>> > for
> > >>> > > 220 and a second time for 221. The first time, it kept
250 of
> > 69833
> > >>> > > messages and the second time it kept 153 of them.
> > >>> > >
> > >>> > > I ran the output of PB2NC through plot_point_obs to show
the
> > >>> locations
> > >>> > and
> > >>> > > have attached PNG versions the result.
> > >>> > >
> > >>> > > So as you can see, the pb_report_type config file option
is doing
> > >>> some
> > >>> > > filtering, but I doubt you actually need or want to use it
for
> the
> > >>> work
> > >>> > > you're doing.
> > >>> > >
> > >>> > > Hope that helps clarify.
> > >>> > >
> > >>> > > Thanks,
> > >>> > > John
> > >>> > >
> > >>> > >
> > >>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov
via RT <
> > >>> > > met_help at ucar.edu> wrote:
> > >>> > >
> > >>> > > >
> > >>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> > >>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > >>> > > > Queue: met_help
> > >>> > > > Subject: pb_report_type in PB2NC
> > >>> > > > Owner: Nobody
> > >>> > > > Requestors: perry.shafran at noaa.gov
> > >>> > > > Status: new
> > >>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> > >>> ket/Display.html?id=81723
> > >>> > >
> > >>> > > >
> > >>> > > >
> > >>> > > > Hi, there,
> > >>> > > >
> > >>> > > > I was going over the MET documentation and there really
wasn't
> > much
> > >>> > > > information about the variable pb_report_type in PB2NC.
I
> tried
> > >>> the
> > >>> > > > following:
> > >>> > > >
> > >>> > > > pb_report_type = ["120","220", "221", "122", "222",
"223",
> > "224",
> > >>> > "133",
> > >>> > > > "233", "188", "288", "180", "280", "181", "182", "281",
"282",
> > >>> "183",
> > >>> > > > "284", "187", "287", "330", "331", "333", "430", "431",
"433",
> > >>> "530",
> > >>> > > > "531", "533"];
> > >>> > > >
> > >>> > > > These numbers are associated with the prepbufr report
types
> used
> > >>> in the
> > >>> > > > table below.
> > >>> > > >
> > >>> > > >
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > >>> > > doc/table_4.htm
> > >>> > > >
> > >>> > > > In the table here, for example, 120, 122, 220, 221, 222,
232,
> and
> > >>> 132
> > >>> > are
> > >>> > > > associated with the prepbufr message type "ADPUPA",
while there
> > are
> > >>> > > several
> > >>> > > > in the 180s and 280s associated with "ADPSFC".
> > >>> > > >
> > >>> > > > It seems that pb_report_type in PB2NC isn't associated
with
> these
> > >>> > > numerical
> > >>> > > > report types, so I am curious what this variable is
expecting.
> > Is
> > >>> it
> > >>> > > just
> > >>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr message
type?
> > >>> > > >
> > >>> > > > Thanks!
> > >>> > > >
> > >>> > > > Perry
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>
> > >
> >
> >
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: perry.shafran at noaa.gov
Time: Thu Aug 24 06:42:07 2017
Hi, John,
That sounds like a great idea.
I think I also like option 2 better.
Perry
On Wed, Aug 23, 2017 at 6:29 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Perry,
>
> I'm thinking about the AIRCAR issue you mentioned. So for example,
let's
> say you have two different PREPBUFR files that came from different
> sources. When you process the first file, you'd like AIRCAR
observations
> to be written out as AIRCAR. But when you process the second file,
you'd
> like AIRCAR observations to be written out as something else...
maybe
> AIRCAR_PROFILES or something.
>
> I can think of two ways we might accomplish this... both by adding a
new
> config file option.
>
> Option 1:
> Add new config file option for:
> message_type_suffix = "_PROFILES";
> That string will be appended to all message types that are read from
the
> current file.
>
> Option 2:
> Add new config file option for:
> message_type_map = [
> { key = "AIRCAR"; val = "AIRCAR_PROFILES"; },
> { key = "ADPUPA"; val = "UppperAir"; }
> ];
>
>
> I prefer option 2 because it's consistent with the an "obs_var_map"
option
> that we're already adding for met-6.1. It also gives you much finer
> control over which message types should be changed an exactly how.
>
> Do you think this solution would meet your needs?
>
> Thanks,
> John
>
>
> On Wed, Aug 23, 2017 at 4:09 PM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Perry,
> >
> > There really is no need to error check these values at all. The
reason
> > for doing so was just a sanity check on them to help users avoid
> mistakes.
> > But they are subject to change, there's no need to do any sanity
> checking.
> >
> > We are currently checking that:
> > 100 <= pb_report_type <= 300
> > 0 <= level_category <= 7
> > 0 <= quality_mark_thresh <= 15
> >
> > But if there's no good reason to do so, we can easily remove those
checks
> > from the file pb2nc_conf_info.cc.
> >
> > What do you think and when do you need this? As a patch for met-
6.0 or
> in
> > the met-6.1 release?
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Aug 23, 2017 at 11:02 AM, perry.shafran at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> >>
> >> BTW - make that making the max 600. We have 500-level report
types as
> >> well
> >> I think.
> >>
> >> Perry
> >>
> >> On Wed, Aug 23, 2017 at 1:01 PM, Perry Shafran - NOAA Affiliate <
> >> perry.shafran at noaa.gov> wrote:
> >>
> >> > Hi, John,
> >> >
> >> > Related to this, is this error statement:
> >> >
> >> > ERROR : PB2NCConfInfo::process_config() -> the
"pb_report_type"
> >> entries
> >> > (330) must be set between 100 and 300.
> >> >
> >> > I would like to request that you increase the level of
allowable
> report
> >> > types to 500. We have added 300- and 400- level items to the
bufr
> file.
> >> > These are aircraft ascent/descent profiles, with the nearest
airport
> >> > surface observation attached to them.
> >> >
> >> > If anyone has access to WCOSS, you can find an example of this
file in
> >> the
> >> > /com2/nam/prod directories, specifically files that look like
this:
> >> > nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are
appended
> to
> >> > the regular prepbufr files for verification here at EMC as
another
> >> source
> >> > of profiles.
> >> >
> >> > Also - these have the type AIRCAR, similar to the regular
AIRCAR data.
> >> > The original AIRCAR data are single level observations, and we
run a
> >> code
> >> > here to make ascent and descent profiles from this single-level
data.
> >> It
> >> > would be nice if we were able to have a designation for these
obs
> rather
> >> > than AIRCAR, to differentiate between these profiles and
regular
> AIRCAR
> >> > data.
> >> >
> >> > Please let me know if you want to discuss this further. We can
add
> this
> >> > as a topic to the next MET issues telecon.
> >> >
> >> > Thanks!
> >> >
> >> > Perry
> >> >
> >> > On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA
Affiliate <
> >> > perry.shafran at noaa.gov> wrote:
> >> >
> >> >> Hi, John,
> >> >>
> >> >> OK, cool, thanks!
> >> >>
> >> >> Perry
> >> >>
> >> >> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT <
> >> >> met_help at ucar.edu> wrote:
> >> >>
> >> >>> Perry,
> >> >>>
> >> >>> Ah, ok. That's a much easier question to answer. Remove the
double
> >> >>> quotes. pb_report_type expects a list of integers, not
strings.
> >> That's
> >> >>> what this error message is telling you:
> >> >>>
> >> >>> ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
> does
> >> >>> not
> >> >>> contain numeric entries.
> >> >>>
> >> >>> Thanks,
> >> >>> John
> >> >>>
> >> >>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via
RT <
> >> >>> met_help at ucar.edu> wrote:
> >> >>>
> >> >>> >
> >> >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> >> >>> >
> >> >>> > Hi, John,
> >> >>> >
> >> >>> > The problem I am asking about is how to properly use this
> particular
> >> >>> > variable. The way I have written it above yields an error:
> >> >>> >
> >> >>> > ERROR :
> >> >>> > ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
> >> does
> >> >>> not
> >> >>> > contain numeric entries.
> >> >>> >
> >> >>> > If I am to compare my VSDB verification to MET
verification, I do
> >> need
> >> >>> to
> >> >>> > list these numeric report types explicitly if I want to
have the
> >> most
> >> >>> > apples-to-apples comparison.
> >> >>> >
> >> >>> > Perry
> >> >>> >
> >> >>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via RT
<
> >> >>> > met_help at ucar.edu> wrote:
> >> >>> >
> >> >>> > > Hi Perry,
> >> >>> > >
> >> >>> > > I see you have a question about the "pb_report_type"
option in
> the
> >> >>> PB2NC
> >> >>> > > configuration file. In practice, I've never really used
that
> >> option
> >> >>> when
> >> >>> > > running MET on a variety of projects. We included it
because we
> >> saw
> >> >>> > > potential for users wanting to filter by it. In practice
> though,
> >> we
> >> >>> have
> >> >>> > > only ever filtered observations by the message_type (i.e.
> >> >>> ADPSFC/ADPUPA
> >> >>> > and
> >> >>> > > so on).
> >> >>> > >
> >> >>> > > Below, I've cut-and-pasted a description of the
pb_report_type
> >> from
> >> >>> the
> >> >>> > > met-6.0/share/met/config/README file which explains it.
That
> >> >>> README is
> >> >>> > > the
> >> >>> > > best place to look for a description of config file
options:
> >> >>> > >
> >> >>> > > //
> >> >>> > > // The "pb_report_type" entry is an array of PrepBufr
report
> types
> >> >>> to be
> >> >>> > > // retained. The numeric "pb_report_type" entry allows
for
> >> further
> >> >>> > > stratification
> >> >>> > > // within message types. An empty list indicates that
all
> should
> >> be
> >> >>> > > retained.
> >> >>> > > //
> >> >>> > > //
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> >> >>> > > doc/table_4.htm
> >> >>> > > //
> >> >>> > > // e.g.
> >> >>> > > // Report Type 120 is for message type ADPUPA but is
only
> >> >>> RAWINSONDE
> >> >>> > > // Report Type 132 is for message type ADPUPA but is
only
> >> >>> FLIGHT-LEVEL
> >> >>> > > RECON
> >> >>> > > // and PROFILE DROPSONDE
> >> >>> > > //
> >> >>> > > pb_report_type = [];
> >> >>> > >
> >> >>> > > The important thing to note is that you do not need to
provide a
> >> >>> list of
> >> >>> > > numbers. An empty list tells PB2NC to use all report
types.
> >> >>> > >
> >> >>> > > But looking at that URL listed above, I see for example
that 220
> >> and
> >> >>> 221
> >> >>> > > are two different report types that are both included in
the
> >> ADPUPA
> >> >>> > message
> >> >>> > > type. By way of testing, I ran pb2nc on a sample NDAS
file
> twice,
> >> >>> once
> >> >>> > for
> >> >>> > > 220 and a second time for 221. The first time, it kept
250 of
> >> 69833
> >> >>> > > messages and the second time it kept 153 of them.
> >> >>> > >
> >> >>> > > I ran the output of PB2NC through plot_point_obs to show
the
> >> >>> locations
> >> >>> > and
> >> >>> > > have attached PNG versions the result.
> >> >>> > >
> >> >>> > > So as you can see, the pb_report_type config file option
is
> doing
> >> >>> some
> >> >>> > > filtering, but I doubt you actually need or want to use
it for
> the
> >> >>> work
> >> >>> > > you're doing.
> >> >>> > >
> >> >>> > > Hope that helps clarify.
> >> >>> > >
> >> >>> > > Thanks,
> >> >>> > > John
> >> >>> > >
> >> >>> > >
> >> >>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov
via RT
> <
> >> >>> > > met_help at ucar.edu> wrote:
> >> >>> > >
> >> >>> > > >
> >> >>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted upon.
> >> >>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> >> >>> > > > Queue: met_help
> >> >>> > > > Subject: pb_report_type in PB2NC
> >> >>> > > > Owner: Nobody
> >> >>> > > > Requestors: perry.shafran at noaa.gov
> >> >>> > > > Status: new
> >> >>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> >> >>> ket/Display.html?id=81723
> >> >>> > >
> >> >>> > > >
> >> >>> > > >
> >> >>> > > > Hi, there,
> >> >>> > > >
> >> >>> > > > I was going over the MET documentation and there really
wasn't
> >> much
> >> >>> > > > information about the variable pb_report_type in PB2NC.
I
> tried
> >> >>> the
> >> >>> > > > following:
> >> >>> > > >
> >> >>> > > > pb_report_type = ["120","220", "221", "122", "222",
"223",
> >> "224",
> >> >>> > "133",
> >> >>> > > > "233", "188", "288", "180", "280", "181", "182", "281",
"282",
> >> >>> "183",
> >> >>> > > > "284", "187", "287", "330", "331", "333", "430", "431",
"433",
> >> >>> "530",
> >> >>> > > > "531", "533"];
> >> >>> > > >
> >> >>> > > > These numbers are associated with the prepbufr report
types
> used
> >> >>> in the
> >> >>> > > > table below.
> >> >>> > > >
> >> >>> > > >
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> >> >>> > > doc/table_4.htm
> >> >>> > > >
> >> >>> > > > In the table here, for example, 120, 122, 220, 221,
222, 232,
> >> and
> >> >>> 132
> >> >>> > are
> >> >>> > > > associated with the prepbufr message type "ADPUPA",
while
> there
> >> are
> >> >>> > > several
> >> >>> > > > in the 180s and 280s associated with "ADPSFC".
> >> >>> > > >
> >> >>> > > > It seems that pb_report_type in PB2NC isn't associated
with
> >> these
> >> >>> > > numerical
> >> >>> > > > report types, so I am curious what this variable is
expecting.
> >> Is
> >> >>> it
> >> >>> > > just
> >> >>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr
message type?
> >> >>> > > >
> >> >>> > > > Thanks!
> >> >>> > > >
> >> >>> > > > Perry
> >> >>> > > >
> >> >>> > > >
> >> >>> > >
> >> >>> > >
> >> >>> >
> >> >>> >
> >> >>>
> >> >>>
> >> >>
> >> >
> >>
> >>
> >
>
>
------------------------------------------------
Subject: pb_report_type in PB2NC
From: John Halley Gotway
Time: Thu Aug 24 10:10:21 2017
Perry,
Thanks for the feedback. I just made two changes:
(1) Increase acceptable report type values up to 600.
(2) Changed those three range checks in the config file from errors to
warnings. So we will still alert the user if they're using an
unexpected
value. But if you guys start using pb_report_type values in the
600's,
PB2NC will still run to completion... just with a warning message.
I also defined a development task in JIRA to add support for the
message_type_map we discussed in version 6.1.
Thanks,
John
On Thu, Aug 24, 2017 at 6:42 AM, perry.shafran at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
>
> Hi, John,
>
> That sounds like a great idea.
>
> I think I also like option 2 better.
>
> Perry
>
> On Wed, Aug 23, 2017 at 6:29 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Perry,
> >
> > I'm thinking about the AIRCAR issue you mentioned. So for
example, let's
> > say you have two different PREPBUFR files that came from different
> > sources. When you process the first file, you'd like AIRCAR
observations
> > to be written out as AIRCAR. But when you process the second
file, you'd
> > like AIRCAR observations to be written out as something else...
maybe
> > AIRCAR_PROFILES or something.
> >
> > I can think of two ways we might accomplish this... both by adding
a new
> > config file option.
> >
> > Option 1:
> > Add new config file option for:
> > message_type_suffix = "_PROFILES";
> > That string will be appended to all message types that are read
from the
> > current file.
> >
> > Option 2:
> > Add new config file option for:
> > message_type_map = [
> > { key = "AIRCAR"; val = "AIRCAR_PROFILES"; },
> > { key = "ADPUPA"; val = "UppperAir"; }
> > ];
> >
> >
> > I prefer option 2 because it's consistent with the an
"obs_var_map"
> option
> > that we're already adding for met-6.1. It also gives you much
finer
> > control over which message types should be changed an exactly how.
> >
> > Do you think this solution would meet your needs?
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Aug 23, 2017 at 4:09 PM, John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> >
> > > Perry,
> > >
> > > There really is no need to error check these values at all. The
reason
> > > for doing so was just a sanity check on them to help users avoid
> > mistakes.
> > > But they are subject to change, there's no need to do any sanity
> > checking.
> > >
> > > We are currently checking that:
> > > 100 <= pb_report_type <= 300
> > > 0 <= level_category <= 7
> > > 0 <= quality_mark_thresh <= 15
> > >
> > > But if there's no good reason to do so, we can easily remove
those
> checks
> > > from the file pb2nc_conf_info.cc.
> > >
> > > What do you think and when do you need this? As a patch for
met-6.0 or
> > in
> > > the met-6.1 release?
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Wed, Aug 23, 2017 at 11:02 AM, perry.shafran at noaa.gov via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> > >>
> > >> BTW - make that making the max 600. We have 500-level report
types as
> > >> well
> > >> I think.
> > >>
> > >> Perry
> > >>
> > >> On Wed, Aug 23, 2017 at 1:01 PM, Perry Shafran - NOAA Affiliate
<
> > >> perry.shafran at noaa.gov> wrote:
> > >>
> > >> > Hi, John,
> > >> >
> > >> > Related to this, is this error statement:
> > >> >
> > >> > ERROR : PB2NCConfInfo::process_config() -> the
"pb_report_type"
> > >> entries
> > >> > (330) must be set between 100 and 300.
> > >> >
> > >> > I would like to request that you increase the level of
allowable
> > report
> > >> > types to 500. We have added 300- and 400- level items to the
bufr
> > file.
> > >> > These are aircraft ascent/descent profiles, with the nearest
airport
> > >> > surface observation attached to them.
> > >> >
> > >> > If anyone has access to WCOSS, you can find an example of
this file
> in
> > >> the
> > >> > /com2/nam/prod directories, specifically files that look like
this:
> > >> > nam.t18z.prepbufr.acft_profiles_sfc.tm06. These files are
appended
> > to
> > >> > the regular prepbufr files for verification here at EMC as
another
> > >> source
> > >> > of profiles.
> > >> >
> > >> > Also - these have the type AIRCAR, similar to the regular
AIRCAR
> data.
> > >> > The original AIRCAR data are single level observations, and
we run a
> > >> code
> > >> > here to make ascent and descent profiles from this single-
level
> data.
> > >> It
> > >> > would be nice if we were able to have a designation for these
obs
> > rather
> > >> > than AIRCAR, to differentiate between these profiles and
regular
> > AIRCAR
> > >> > data.
> > >> >
> > >> > Please let me know if you want to discuss this further. We
can add
> > this
> > >> > as a topic to the next MET issues telecon.
> > >> >
> > >> > Thanks!
> > >> >
> > >> > Perry
> > >> >
> > >> > On Wed, Aug 23, 2017 at 12:18 PM, Perry Shafran - NOAA
Affiliate <
> > >> > perry.shafran at noaa.gov> wrote:
> > >> >
> > >> >> Hi, John,
> > >> >>
> > >> >> OK, cool, thanks!
> > >> >>
> > >> >> Perry
> > >> >>
> > >> >> On Wed, Aug 23, 2017 at 12:15 PM, John Halley Gotway via RT
<
> > >> >> met_help at ucar.edu> wrote:
> > >> >>
> > >> >>> Perry,
> > >> >>>
> > >> >>> Ah, ok. That's a much easier question to answer. Remove
the
> double
> > >> >>> quotes. pb_report_type expects a list of integers, not
strings.
> > >> That's
> > >> >>> what this error message is telling you:
> > >> >>>
> > >> >>> ERROR : Dictionary::lookup_num_array() -> array
"pb_report_type"
> > does
> > >> >>> not
> > >> >>> contain numeric entries.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> John
> > >> >>>
> > >> >>> On Wed, Aug 23, 2017 at 9:55 AM, perry.shafran at noaa.gov via
RT <
> > >> >>> met_help at ucar.edu> wrote:
> > >> >>>
> > >> >>> >
> > >> >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=81723 >
> > >> >>> >
> > >> >>> > Hi, John,
> > >> >>> >
> > >> >>> > The problem I am asking about is how to properly use this
> > particular
> > >> >>> > variable. The way I have written it above yields an
error:
> > >> >>> >
> > >> >>> > ERROR :
> > >> >>> > ERROR : Dictionary::lookup_num_array() -> array
> "pb_report_type"
> > >> does
> > >> >>> not
> > >> >>> > contain numeric entries.
> > >> >>> >
> > >> >>> > If I am to compare my VSDB verification to MET
verification, I
> do
> > >> need
> > >> >>> to
> > >> >>> > list these numeric report types explicitly if I want to
have the
> > >> most
> > >> >>> > apples-to-apples comparison.
> > >> >>> >
> > >> >>> > Perry
> > >> >>> >
> > >> >>> > On Wed, Aug 23, 2017 at 11:51 AM, John Halley Gotway via
RT <
> > >> >>> > met_help at ucar.edu> wrote:
> > >> >>> >
> > >> >>> > > Hi Perry,
> > >> >>> > >
> > >> >>> > > I see you have a question about the "pb_report_type"
option in
> > the
> > >> >>> PB2NC
> > >> >>> > > configuration file. In practice, I've never really
used that
> > >> option
> > >> >>> when
> > >> >>> > > running MET on a variety of projects. We included it
because
> we
> > >> saw
> > >> >>> > > potential for users wanting to filter by it. In
practice
> > though,
> > >> we
> > >> >>> have
> > >> >>> > > only ever filtered observations by the message_type
(i.e.
> > >> >>> ADPSFC/ADPUPA
> > >> >>> > and
> > >> >>> > > so on).
> > >> >>> > >
> > >> >>> > > Below, I've cut-and-pasted a description of the
pb_report_type
> > >> from
> > >> >>> the
> > >> >>> > > met-6.0/share/met/config/README file which explains it.
That
> > >> >>> README is
> > >> >>> > > the
> > >> >>> > > best place to look for a description of config file
options:
> > >> >>> > >
> > >> >>> > > //
> > >> >>> > > // The "pb_report_type" entry is an array of PrepBufr
report
> > types
> > >> >>> to be
> > >> >>> > > // retained. The numeric "pb_report_type" entry allows
for
> > >> further
> > >> >>> > > stratification
> > >> >>> > > // within message types. An empty list indicates that
all
> > should
> > >> be
> > >> >>> > > retained.
> > >> >>> > > //
> > >> >>> > > //
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > >> >>> > > doc/table_4.htm
> > >> >>> > > //
> > >> >>> > > // e.g.
> > >> >>> > > // Report Type 120 is for message type ADPUPA but is
only
> > >> >>> RAWINSONDE
> > >> >>> > > // Report Type 132 is for message type ADPUPA but is
only
> > >> >>> FLIGHT-LEVEL
> > >> >>> > > RECON
> > >> >>> > > // and PROFILE DROPSONDE
> > >> >>> > > //
> > >> >>> > > pb_report_type = [];
> > >> >>> > >
> > >> >>> > > The important thing to note is that you do not need to
> provide a
> > >> >>> list of
> > >> >>> > > numbers. An empty list tells PB2NC to use all report
types.
> > >> >>> > >
> > >> >>> > > But looking at that URL listed above, I see for example
that
> 220
> > >> and
> > >> >>> 221
> > >> >>> > > are two different report types that are both included
in the
> > >> ADPUPA
> > >> >>> > message
> > >> >>> > > type. By way of testing, I ran pb2nc on a sample NDAS
file
> > twice,
> > >> >>> once
> > >> >>> > for
> > >> >>> > > 220 and a second time for 221. The first time, it kept
250 of
> > >> 69833
> > >> >>> > > messages and the second time it kept 153 of them.
> > >> >>> > >
> > >> >>> > > I ran the output of PB2NC through plot_point_obs to
show the
> > >> >>> locations
> > >> >>> > and
> > >> >>> > > have attached PNG versions the result.
> > >> >>> > >
> > >> >>> > > So as you can see, the pb_report_type config file
option is
> > doing
> > >> >>> some
> > >> >>> > > filtering, but I doubt you actually need or want to use
it for
> > the
> > >> >>> work
> > >> >>> > > you're doing.
> > >> >>> > >
> > >> >>> > > Hope that helps clarify.
> > >> >>> > >
> > >> >>> > > Thanks,
> > >> >>> > > John
> > >> >>> > >
> > >> >>> > >
> > >> >>> > > On Wed, Aug 23, 2017 at 9:08 AM, perry.shafran at noaa.gov
via
> RT
> > <
> > >> >>> > > met_help at ucar.edu> wrote:
> > >> >>> > >
> > >> >>> > > >
> > >> >>> > > > Wed Aug 23 09:08:09 2017: Request 81723 was acted
upon.
> > >> >>> > > > Transaction: Ticket created by perry.shafran at noaa.gov
> > >> >>> > > > Queue: met_help
> > >> >>> > > > Subject: pb_report_type in PB2NC
> > >> >>> > > > Owner: Nobody
> > >> >>> > > > Requestors: perry.shafran at noaa.gov
> > >> >>> > > > Status: new
> > >> >>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> > >> >>> ket/Display.html?id=81723
> > >> >>> > >
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > > Hi, there,
> > >> >>> > > >
> > >> >>> > > > I was going over the MET documentation and there
really
> wasn't
> > >> much
> > >> >>> > > > information about the variable pb_report_type in
PB2NC. I
> > tried
> > >> >>> the
> > >> >>> > > > following:
> > >> >>> > > >
> > >> >>> > > > pb_report_type = ["120","220", "221", "122", "222",
"223",
> > >> "224",
> > >> >>> > "133",
> > >> >>> > > > "233", "188", "288", "180", "280", "181", "182",
"281",
> "282",
> > >> >>> "183",
> > >> >>> > > > "284", "187", "287", "330", "331", "333", "430",
"431",
> "433",
> > >> >>> "530",
> > >> >>> > > > "531", "533"];
> > >> >>> > > >
> > >> >>> > > > These numbers are associated with the prepbufr report
types
> > used
> > >> >>> in the
> > >> >>> > > > table below.
> > >> >>> > > >
> > >> >>> > > >
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.
> > >> >>> > > doc/table_4.htm
> > >> >>> > > >
> > >> >>> > > > In the table here, for example, 120, 122, 220, 221,
222,
> 232,
> > >> and
> > >> >>> 132
> > >> >>> > are
> > >> >>> > > > associated with the prepbufr message type "ADPUPA",
while
> > there
> > >> are
> > >> >>> > > several
> > >> >>> > > > in the 180s and 280s associated with "ADPSFC".
> > >> >>> > > >
> > >> >>> > > > It seems that pb_report_type in PB2NC isn't
associated with
> > >> these
> > >> >>> > > numerical
> > >> >>> > > > report types, so I am curious what this variable is
> expecting.
> > >> Is
> > >> >>> it
> > >> >>> > > just
> > >> >>> > > > "ADPSFC" or "ADPUPA" - what we call the prepbufr
message
> type?
> > >> >>> > > >
> > >> >>> > > > Thanks!
> > >> >>> > > >
> > >> >>> > > > Perry
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > >
> > >> >>> > >
> > >> >>> >
> > >> >>> >
> > >> >>>
> > >> >>>
> > >> >>
> > >> >
> > >>
> > >>
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list