[Met_help] [rt.rap.ucar.edu #59667] History for Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
John Halley Gotway via RT
met_help at ucar.edu
Fri Jan 4 11:39:00 MST 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi MET team.
I am trying to use Point Stat to verify GRIB forecasts with multiple forecast times per file - eg 20110102_0000 initialisation time, with hourly forecast data out to 72hrs.
I have found mention of the fcst_valid and fcst_lead command line options for previous versions of MET, and noticed that it is mentioned in the release notes of METv4.0 that this command lines options have been moved to the config files. However, I can't find any documentation regarding how/where to specify them in the config file in the v4.0 User's Guide or config/README. I would be very grateful if you could advise me or point me to it if I've overlooked it? Would love to avoid having to split up my data into separate lead times!
Also, I am slightly confused about the difference between fcst_valid and fcst_lead - for a given initialisation time (single forecast file) with multiple timesteps, should I be specifying fcst_lead to verify a particular time? ie, to verify the 12-hour forecast for a 20110102_0000 file, should I specify fcst_lead = 20110102_1200?
Thanks very much
Malcolm Nunn
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
From: John Halley Gotway
Time: Wed Dec 19 12:24:58 2012
Malcolm,
Yes, you are correct. They have been moved from the command line to
the config file. I apologize for the incomplete information in the
config README file. I'll make sure that we update the
documentation prior to the next release.
Basically, you can use the following settings in the config file:
lead_time = "HH[MMSS]";
valid_time = "YYYYMMDD[_HH[MMSS]]";
You should put this inside the "fcst" and/or "obs" settings in the
config file. You can use neither, one, or both of the settings. If
provided, Point-Stat will only match fields that meet this
additional timing criteria.
For example, here's how you could edit this config file used by the
point_stat test script:
METv4.0/scripts/config/PointStatConfig
fcst = {
wind_thresh = [ NA ];
message_type = [ "ADPUPA", "ADPSFC" ];
lead_time = "12";
valid_time = "20070331_12";
field = [
{
name = "TMP";
level = [ "P750-900" ];
cat_thresh = [ <=273, >273 ];
},
{
name = "UGRD";
level = [ "Z10" ];
cat_thresh = [ >=5 ];
},
{
name = "VGRD";
level = [ "Z10" ];
cat_thresh = [ >=5 ];
}
];
};
Note that in the above setting, this timing information will apply to
each of the 3 fields listed. If your GRIB file contains multiple
output times, you could actually verify all of them in a single
call like this:
fcst = {
wind_thresh = [ NA ];
message_type = [ "ADPUPA", "ADPSFC" ];
field = [
{
name = "TMP";
level = [ "P750-900" ];
cat_thresh = [ <=273, >273 ];
lead_time = "06";
},
{
name = "TMP";
level = [ "P750-900" ];
cat_thresh = [ <=273, >273 ];
lead_time = "12";
}
];
};
This will verify the 6 and 12-hour forecasts of 2-meter TMP from your
file. So you can specify the timing information for each "field"
entry. But you just need to be sure that you pass Point-Stat
observations that cover this time range as well.
Hope that helps.
John Halley Gotway
met_help at ucar.edu
On 12/18/2012 07:15 PM, Malcolm.Nunn at csiro.au via RT wrote:
>
> Tue Dec 18 19:15:58 2012: Request 59667 was acted upon.
> Transaction: Ticket created by Malcolm.Nunn at csiro.au
> Queue: met_help
> Subject: Point_Stat - specifying fcst_valid and fcst_lead in
METv4.0
> Owner: Nobody
> Requestors: Malcolm.Nunn at csiro.au
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667 >
>
>
> Hi MET team.
>
> I am trying to use Point Stat to verify GRIB forecasts with multiple
forecast times per file - eg 20110102_0000 initialisation time, with
hourly forecast data out to 72hrs.
>
> I have found mention of the fcst_valid and fcst_lead command line
options for previous versions of MET, and noticed that it is mentioned
in the release notes of METv4.0 that this command lines options have
been moved to the config files. However, I can't find any
documentation regarding how/where to specify them in the config file
in the v4.0 User's Guide or config/README. I would be very grateful if
you could advise me or point me to it if I've overlooked it? Would
love to avoid having to split up my data into separate lead times!
>
> Also, I am slightly confused about the difference between fcst_valid
and fcst_lead - for a given initialisation time (single forecast file)
with multiple timesteps, should I be specifying fcst_lead to verify a
particular time? ie, to verify the 12-hour forecast for a
20110102_0000 file, should I specify fcst_lead = 20110102_1200?
>
> Thanks very much
> Malcolm Nunn
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
From: Malcolm.Nunn at csiro.au
Time: Wed Dec 19 20:23:37 2012
Brilliant, thank you. Specifying multiple lead times within a single
config saves a lot of hassle too.
Very minor note - the file name
generated for point_stat output will only reflect a single lead time,
ie a config to verify at 24, 48 and 72 hr lead time still has an
output filename "[prefix]240000L_20110201_120000V.stat"
Not an issue
for me but thought I'd mention it.
Is there any way to aggregate
continuous statistics through MET or does that need to be done with an
external script? If, for example, I filter my output so I have a month
of daily CNT stat lines, is there a way to pool these with
stat_analysis? I know it can't aggregate CNT lines.
Thanks again
Malcolm
-----Original Message-----
From: John Halley Gotway via RT
[mailto:met_help at ucar.edu]
Sent: Thursday, 20 December 2012 6:25 AM
To: Nunn, Malcolm (CMAR, Aspendale)
Subject: Re: [rt.rap.ucar.edu
#59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
Malcolm,
Yes, you are correct. They have been moved from the
command line to the config file. I apologize for the incomplete
information in the config README file. I'll make sure that we update
the documentation prior to the next release.
Basically, you can use
the following settings in the config file:
lead_time =
"HH[MMSS]";
valid_time = "YYYYMMDD[_HH[MMSS]]";
You should put
this inside the "fcst" and/or "obs" settings in the config file. You
can use neither, one, or both of the settings. If provided, Point-
Stat will only match fields that meet this additional timing criteria.
For example, here's how you could edit this config file used by the
point_stat test script:
METv4.0/scripts/config/PointStatConfig
fcst = {
wind_thresh = [ NA ];
message_type = [ "ADPUPA",
"ADPSFC" ];
lead_time = "12";
valid_time = "20070331_12";
field = [
{
name = "TMP";
level
= [ "P750-900" ];
cat_thresh = [ <=273, >273 ];
},
{
name = "UGRD";
level = [ "Z10" ];
cat_thresh = [ >=5 ];
},
{
name =
"VGRD";
level = [ "Z10" ];
cat_thresh = [ >=5
];
}
];
};
Note that in the above setting, this
timing information will apply to each of the 3 fields listed. If your
GRIB file contains multiple output times, you could actually verify
all of them in a single call like this:
fcst = {
wind_thresh
= [ NA ];
message_type = [ "ADPUPA", "ADPSFC" ];
field = [
{
name = "TMP";
level = [ "P750-900" ];
cat_thresh = [ <=273, >273 ];
lead_time = "06";
},
{
name = "TMP";
level = [ "P750-900" ];
cat_thresh = [ <=273, >273 ];
lead_time = "12";
}
];
};
This will verify the 6 and 12-hour forecasts of 2-meter
TMP from your file. So you can specify the timing information for
each "field" entry. But you just need to be sure that you pass Point-
Stat observations that cover this time range as well.
Hope that
helps.
John Halley Gotway
met_help at ucar.edu
On 12/18/2012
07:15 PM, Malcolm.Nunn at csiro.au via RT wrote:
>
> Tue Dec 18
19:15:58 2012: Request 59667 was acted upon.
> Transaction: Ticket
created by Malcolm.Nunn at csiro.au
> Queue: met_help
>
Subject: Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
> Owner: Nobody
> Requestors: Malcolm.Nunn at csiro.au
>
Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667
> >
>
>
>
Hi MET team.
>
> I am trying to use Point Stat to verify GRIB
forecasts with multiple forecast times per file - eg 20110102_0000
initialisation time, with hourly forecast data out to 72hrs.
>
> I
have found mention of the fcst_valid and fcst_lead command line
options for previous versions of MET, and noticed that it is mentioned
in the release notes of METv4.0 that this command lines options have
been moved to the config files. However, I can't find any
documentation regarding how/where to specify them in the config file
in the v4.0 User's Guide or config/README. I would be very grateful if
you could advise me or point me to it if I've overlooked it? Would
love to avoid having to split up my data into separate lead times!
>
> Also, I am slightly confused about the difference between fcst_valid
and fcst_lead - for a given initialisation time (single forecast file)
with multiple timesteps, should I be specifying fcst_lead to verify a
particular time? ie, to verify the 12-hour forecast for a
20110102_0000 file, should I specify fcst_lead = 20110102_1200?
>
>
Thanks very much
> Malcolm Nunn
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
From: John Halley Gotway
Time: Wed Dec 19 22:30:18 2012
Malcolm,
Now that you mention the issue of the output file name, I'm not
entirely
sure how the code will handle that. Hmmm, that'll take a bit of
testing.
I'd say if you do try to verify multiple lead times in a single call
to
Point-Stat, proceed with caution. Run with a high level of verbosity
(-v
3 or -v 4), and inspect the log and output files to make sure it's
working
as you'd expect. None of the GRIB files I've worked with have been
organized in that way, but I know they're out there.
I'm out of the office for 2 weeks over the holidays, but if you
continue
to have issues/questions, I'll be able to assist you more in early
January.
Regarding aggregating the output of point-stat, yes, the answer is
stat-analysis.
When running point-stat, writing out the matched pair (MPR) line type
gives you the most amount of flexibility in stat-analysis. The
downside,
of course, is that a lot of MPR output for many matched pairs takes up
a
whole lot of space. A middle ground is writing out the SL1L2 partial
sums. While the CNT lines cannot be aggregated directly in stat-
analysis,
the SL1L2 lines can and then a subset of the CNT statistics can be
derived
from those aggregated partial sums. Really that's the whole point of
the
SL1L2 line type in the first place.
If you store MPR output from point-stat, the following stat-analysis
job
type will be most useful:
-job aggregate_stat -line_type MPR -out_line_type CNT (more
filtering
criteria to subset your data)
If you only store SL1L2 lines, use the following:
-job aggregate_stat -line_type SL1L2 -out_line_type CNT (more
filtering
criteria)
Many, but not all, of the CNT stats can be derived from the SL1L2
partial
sums. If you haven't already seen it, you may find the stat-analysis
section of the MET online tutorial to be helpful:
http://www.dtcenter.org/met/users/support/online_tutorial/METv4.0/stat_analysis/index.php
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667 >
>
> Brilliant, thank you. Specifying multiple lead times within a single
> config saves a lot of hassle too.
>
> Very minor note - the file name generated for point_stat output will
only
> reflect a single lead time, ie a config to verify at 24, 48 and 72
hr lead
> time still has an output filename
"[prefix]240000L_20110201_120000V.stat"
> Not an issue for me but thought I'd mention it.
>
> Is there any way to aggregate continuous statistics through MET or
does
> that need to be done with an external script? If, for example, I
filter my
> output so I have a month of daily CNT stat lines, is there a way to
pool
> these with stat_analysis? I know it can't aggregate CNT lines.
>
> Thanks again
> Malcolm
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, 20 December 2012 6:25 AM
> To: Nunn, Malcolm (CMAR, Aspendale)
> Subject: Re: [rt.rap.ucar.edu #59667] Point_Stat - specifying
fcst_valid
> and fcst_lead in METv4.0
>
> Malcolm,
>
> Yes, you are correct. They have been moved from the command line to
the
> config file. I apologize for the incomplete information in the
config
> README file. I'll make sure that we update the documentation prior
to the
> next release.
>
> Basically, you can use the following settings in the config file:
>
> lead_time = "HH[MMSS]";
> valid_time = "YYYYMMDD[_HH[MMSS]]";
>
> You should put this inside the "fcst" and/or "obs" settings in the
config
> file. You can use neither, one, or both of the settings. If
provided,
> Point-Stat will only match fields that meet this additional timing
> criteria.
>
> For example, here's how you could edit this config file used by the
> point_stat test script:
> METv4.0/scripts/config/PointStatConfig
>
> fcst = {
> wind_thresh = [ NA ];
> message_type = [ "ADPUPA", "ADPSFC" ];
>
> lead_time = "12";
> valid_time = "20070331_12";
>
> field = [
> {
> name = "TMP";
> level = [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
> },
>
> {
> name = "UGRD";
> level = [ "Z10" ];
> cat_thresh = [ >=5 ];
> },
>
> {
> name = "VGRD";
> level = [ "Z10" ];
> cat_thresh = [ >=5 ];
> }
> ];
>
> };
>
> Note that in the above setting, this timing information will apply
to each
> of the 3 fields listed. If your GRIB file contains multiple output
times,
> you could actually verify all of them in a single call like this:
>
> fcst = {
> wind_thresh = [ NA ];
> message_type = [ "ADPUPA", "ADPSFC" ];
>
> field = [
> {
> name = "TMP";
> level = [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
> lead_time = "06";
> },
>
> {
> name = "TMP";
> level = [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
> lead_time = "12";
> }
> ];
>
> };
>
> This will verify the 6 and 12-hour forecasts of 2-meter TMP from
your
> file. So you can specify the timing information for each "field"
entry.
> But you just need to be sure that you pass Point-Stat observations
that
> cover this time range as well.
>
> Hope that helps.
>
> John Halley Gotway
> met_help at ucar.edu
>
>
> On 12/18/2012 07:15 PM, Malcolm.Nunn at csiro.au via RT wrote:
>>
>> Tue Dec 18 19:15:58 2012: Request 59667 was acted upon.
>> Transaction: Ticket created by Malcolm.Nunn at csiro.au
>> Queue: met_help
>> Subject: Point_Stat - specifying fcst_valid and fcst_lead in
>> METv4.0
>> Owner: Nobody
>> Requestors: Malcolm.Nunn at csiro.au
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667
>> >
>>
>>
>> Hi MET team.
>>
>> I am trying to use Point Stat to verify GRIB forecasts with
multiple
>> forecast times per file - eg 20110102_0000 initialisation time,
with
>> hourly forecast data out to 72hrs.
>>
>> I have found mention of the fcst_valid and fcst_lead command line
>> options for previous versions of MET, and noticed that it is
mentioned
>> in the release notes of METv4.0 that this command lines options
have
>> been moved to the config files. However, I can't find any
documentation
>> regarding how/where to specify them in the config file in the v4.0
>> User's Guide or config/README. I would be very grateful if you
could
>> advise me or point me to it if I've overlooked it? Would love to
avoid
>> having to split up my data into separate lead times!
>>
>> Also, I am slightly confused about the difference between
fcst_valid and
>> fcst_lead - for a given initialisation time (single forecast file)
with
>> multiple timesteps, should I be specifying fcst_lead to verify a
>> particular time? ie, to verify the 12-hour forecast for a
20110102_0000
>> file, should I specify fcst_lead = 20110102_1200?
>>
>> Thanks very much
>> Malcolm Nunn
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
From: Malcolm.Nunn at csiro.au
Time: Thu Dec 20 17:04:26 2012
Excellent, this is just what I need. What's the difference between the
aggregated CNT output from SL1L2 and MPR lines? I can't find any
specific explanation in the User's Guide or tutorial.
I'll check
the output carefully, so far though it seems to be processing the
right lines though! So I'm guessing stat_analysis doesn't look at the
filenames, just the data?
I've noticed that when point_stat is run
there is no fcst_init time in the output - so a 20110101-00:00
initialised forecast verified at 24 hour lead time has a valid time of
20110102-00:00 (with a lead time of 24hr), and is named
<prefix>240000L_20110102_000000_[line_type extension]. Since there is
no explicit fcst_init time in the output data columns or the file
name, when stat_analysis performs a job with fcst_init time as
criteria does it just work it out from the data as fcst_valid -
fcst_lead?
I've tried aggregating CNT from SL1L2 and that works
fine. For some reason I'm getting a segmentation fault when I try to
aggregate CNT from MPR though:
DEBUG 4: Initializing Job 2 to
default job: "-fcst_lead 240000 -fcst_valid_hour 000000 -fcst_var TMP
-fcst_lev Z0 -obtype ADPSFC -line_type MPR "
DEBUG 4:
DEBUG 4:
Amending Job 2 with options: "-job aggregate_stat -out_line_type CNT
-dump_row data/stat_analysis/SL1L2_201101_MPR/job2_agg_stat_cnt.stat"
DEBUG 4:
DEBUG 4: Amending Job 2 with command line options: "(nul)"
Segmentation fault
Thanks very much for your assistance John, and
if you're already out of the office for the holidays, merry Christmas!
Cheers
Malcolm
-----Original Message-----
From: John Halley
Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, 20 December
2012 4:30 PM
To: Nunn, Malcolm (CMAR, Aspendale)
Subject: RE:
[rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and
fcst_lead in METv4.0
Malcolm,
Now that you mention the issue of
the output file name, I'm not entirely sure how the code will handle
that. Hmmm, that'll take a bit of testing.
I'd say if you do try
to verify multiple lead times in a single call to Point-Stat, proceed
with caution. Run with a high level of verbosity (-v
3 or -v 4), and
inspect the log and output files to make sure it's working as you'd
expect. None of the GRIB files I've worked with have been organized
in that way, but I know they're out there.
I'm out of the office
for 2 weeks over the holidays, but if you continue to have
issues/questions, I'll be able to assist you more in early January.
Regarding aggregating the output of point-stat, yes, the answer is
stat-analysis.
When running point-stat, writing out the matched
pair (MPR) line type gives you the most amount of flexibility in stat-
analysis. The downside, of course, is that a lot of MPR output for
many matched pairs takes up a whole lot of space. A middle ground is
writing out the SL1L2 partial sums. While the CNT lines cannot be
aggregated directly in stat-analysis, the SL1L2 lines can and then a
subset of the CNT statistics can be derived from those aggregated
partial sums. Really that's the whole point of the
SL1L2 line type
in the first place.
If you store MPR output from point-stat, the
following stat-analysis job type will be most useful:
-job
aggregate_stat -line_type MPR -out_line_type CNT (more filtering
criteria to subset your data)
If you only store SL1L2 lines, use
the following:
-job aggregate_stat -line_type SL1L2 -out_line_type
CNT (more filtering
criteria)
Many, but not all, of the CNT stats
can be derived from the SL1L2 partial sums. If you haven't already
seen it, you may find the stat-analysis section of the MET online
tutorial to be helpful:
http://www.dtcenter.org/met/users/support/online_tutorial/METv4.0/stat_analysis/index.php
Thanks,
John
>
> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667 >
>
>
Brilliant, thank you. Specifying multiple lead times within a single
> config saves a lot of hassle too.
>
> Very minor note - the file
name generated for point_stat output will
> only reflect a single
lead time, ie a config to verify at 24, 48 and
> 72 hr lead time
still has an output filename "[prefix]240000L_20110201_120000V.stat"
> Not an issue for me but thought I'd mention it.
>
> Is there any
way to aggregate continuous statistics through MET or
> does that
need to be done with an external script? If, for example, I
> filter
my output so I have a month of daily CNT stat lines, is there a
>
way to pool these with stat_analysis? I know it can't aggregate CNT
lines.
>
> Thanks again
> Malcolm
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent:
Thursday, 20 December 2012 6:25 AM
> To: Nunn, Malcolm (CMAR,
Aspendale)
> Subject: Re: [rt.rap.ucar.edu #59667] Point_Stat -
specifying
> fcst_valid and fcst_lead in METv4.0
>
> Malcolm,
>
> Yes, you are correct. They have been moved from the command line to
> the config file. I apologize for the incomplete information in the
> config README file. I'll make sure that we update the documentation
> prior to the next release.
>
> Basically, you can use the
following settings in the config file:
>
> lead_time =
"HH[MMSS]";
> valid_time = "YYYYMMDD[_HH[MMSS]]";
>
> You should
put this inside the "fcst" and/or "obs" settings in the
> config
file. You can use neither, one, or both of the settings. If
>
provided, Point-Stat will only match fields that meet this additional
> timing criteria.
>
> For example, here's how you could edit this
config file used by the
> point_stat test script:
>
METv4.0/scripts/config/PointStatConfig
>
> fcst = {
>
wind_thresh = [ NA ];
> message_type = [ "ADPUPA", "ADPSFC" ];
>
> lead_time = "12";
> valid_time = "20070331_12";
>
>
field = [
> {
> name = "TMP";
>
level = [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
> },
>
> {
> name = "UGRD";
>
level = [ "Z10" ];
> cat_thresh = [ >=5 ];
> },
>
> {
> name = "VGRD";
> level
= [ "Z10" ];
> cat_thresh = [ >=5 ];
> }
> ];
>
> };
>
> Note that in the above setting, this timing information
will apply to
> each of the 3 fields listed. If your GRIB file
contains multiple
> output times, you could actually verify all of
them in a single call like this:
>
> fcst = {
> wind_thresh =
[ NA ];
> message_type = [ "ADPUPA", "ADPSFC" ];
>
> field
= [
> {
> name = "TMP";
> level
= [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
>
lead_time = "06";
> },
>
> {
> name
= "TMP";
> level = [ "P750-900" ];
>
cat_thresh = [ <=273, >273 ];
> lead_time = "12";
>
}
> ];
>
> };
>
> This will verify the 6 and 12-hour
forecasts of 2-meter TMP from your
> file. So you can specify the
timing information for each "field" entry.
> But you just need to be
sure that you pass Point-Stat observations
> that cover this time
range as well.
>
> Hope that helps.
>
> John Halley Gotway
>
met_help at ucar.edu
>
>
> On 12/18/2012 07:15 PM,
Malcolm.Nunn at csiro.au via RT wrote:
>>
>> Tue Dec 18 19:15:58 2012:
Request 59667 was acted upon.
>> Transaction: Ticket created by
Malcolm.Nunn at csiro.au
>> Queue: met_help
>> Subject:
Point_Stat - specifying fcst_valid and fcst_lead in
>> METv4.0
>>
Owner: Nobody
>> Requestors: Malcolm.Nunn at csiro.au
>>
Status: new
>> Ticket <URL:
>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667
>> >
>>
>>
>> Hi MET team.
>>
>> I am trying to use Point Stat to verify GRIB
forecasts with multiple
>> forecast times per file - eg
20110102_0000 initialisation time, with
>> hourly forecast data out
to 72hrs.
>>
>> I have found mention of the fcst_valid and fcst_lead
command line
>> options for previous versions of MET, and noticed
that it is
>> mentioned in the release notes of METv4.0 that this
command lines
>> options have been moved to the config files.
However, I can't find
>> any documentation regarding how/where to
specify them in the config
>> file in the v4.0 User's Guide or
config/README. I would be very
>> grateful if you could advise me or
point me to it if I've overlooked
>> it? Would love to avoid having
to split up my data into separate lead times!
>>
>> Also, I am
slightly confused about the difference between fcst_valid
>> and
fcst_lead - for a given initialisation time (single forecast
>>
file) with multiple timesteps, should I be specifying fcst_lead to
>> verify a particular time? ie, to verify the 12-hour forecast for a
>> 20110102_0000 file, should I specify fcst_lead = 20110102_1200?
>>
>> Thanks very much
>> Malcolm Nunn
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
From: Malcolm.Nunn at csiro.au
Time: Thu Dec 20 22:03:53 2012
Whoops, please ignore my last query about the Stat_Analysis
segmentation fault. I just realised I hadn't applied the patch which
deals with that bug when aggregating from MPR. Aggregating CNT from
MPR works fine now!
Cheers (and happy holidays!)
Malcolm
-----Original Message-----
From: John Halley Gotway via RT
[mailto:met_help at ucar.edu]
Sent: Thursday, 20 December 2012 4:30 PM
To: Nunn, Malcolm (CMAR, Aspendale)
Subject: RE: [rt.rap.ucar.edu
#59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
Malcolm,
Now that you mention the issue of the output file name,
I'm not entirely sure how the code will handle that. Hmmm, that'll
take a bit of testing.
I'd say if you do try to verify multiple
lead times in a single call to Point-Stat, proceed with caution. Run
with a high level of verbosity (-v
3 or -v 4), and inspect the log
and output files to make sure it's working as you'd expect. None of
the GRIB files I've worked with have been organized in that way, but I
know they're out there.
I'm out of the office for 2 weeks over the
holidays, but if you continue to have issues/questions, I'll be able
to assist you more in early January.
Regarding aggregating the
output of point-stat, yes, the answer is stat-analysis.
When
running point-stat, writing out the matched pair (MPR) line type gives
you the most amount of flexibility in stat-analysis. The downside, of
course, is that a lot of MPR output for many matched pairs takes up a
whole lot of space. A middle ground is writing out the SL1L2 partial
sums. While the CNT lines cannot be aggregated directly in stat-
analysis, the SL1L2 lines can and then a subset of the CNT statistics
can be derived from those aggregated partial sums. Really that's the
whole point of the
SL1L2 line type in the first place.
If you
store MPR output from point-stat, the following stat-analysis job type
will be most useful:
-job aggregate_stat -line_type MPR
-out_line_type CNT (more filtering criteria to subset your data)
If
you only store SL1L2 lines, use the following:
-job aggregate_stat
-line_type SL1L2 -out_line_type CNT (more filtering
criteria)
Many, but not all, of the CNT stats can be derived from the SL1L2
partial sums. If you haven't already seen it, you may find the stat-
analysis section of the MET online tutorial to be helpful:
http://www.dtcenter.org/met/users/support/online_tutorial/METv4.0/stat_analysis/index.php
Thanks,
John
>
> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667 >
>
>
Brilliant, thank you. Specifying multiple lead times within a single
> config saves a lot of hassle too.
>
> Very minor note - the file
name generated for point_stat output will
> only reflect a single
lead time, ie a config to verify at 24, 48 and
> 72 hr lead time
still has an output filename "[prefix]240000L_20110201_120000V.stat"
> Not an issue for me but thought I'd mention it.
>
> Is there any
way to aggregate continuous statistics through MET or
> does that
need to be done with an external script? If, for example, I
> filter
my output so I have a month of daily CNT stat lines, is there a
>
way to pool these with stat_analysis? I know it can't aggregate CNT
lines.
>
> Thanks again
> Malcolm
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent:
Thursday, 20 December 2012 6:25 AM
> To: Nunn, Malcolm (CMAR,
Aspendale)
> Subject: Re: [rt.rap.ucar.edu #59667] Point_Stat -
specifying
> fcst_valid and fcst_lead in METv4.0
>
> Malcolm,
>
> Yes, you are correct. They have been moved from the command line to
> the config file. I apologize for the incomplete information in the
> config README file. I'll make sure that we update the documentation
> prior to the next release.
>
> Basically, you can use the
following settings in the config file:
>
> lead_time =
"HH[MMSS]";
> valid_time = "YYYYMMDD[_HH[MMSS]]";
>
> You should
put this inside the "fcst" and/or "obs" settings in the
> config
file. You can use neither, one, or both of the settings. If
>
provided, Point-Stat will only match fields that meet this additional
> timing criteria.
>
> For example, here's how you could edit this
config file used by the
> point_stat test script:
>
METv4.0/scripts/config/PointStatConfig
>
> fcst = {
>
wind_thresh = [ NA ];
> message_type = [ "ADPUPA", "ADPSFC" ];
>
> lead_time = "12";
> valid_time = "20070331_12";
>
>
field = [
> {
> name = "TMP";
>
level = [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
> },
>
> {
> name = "UGRD";
>
level = [ "Z10" ];
> cat_thresh = [ >=5 ];
> },
>
> {
> name = "VGRD";
> level
= [ "Z10" ];
> cat_thresh = [ >=5 ];
> }
> ];
>
> };
>
> Note that in the above setting, this timing information
will apply to
> each of the 3 fields listed. If your GRIB file
contains multiple
> output times, you could actually verify all of
them in a single call like this:
>
> fcst = {
> wind_thresh =
[ NA ];
> message_type = [ "ADPUPA", "ADPSFC" ];
>
> field
= [
> {
> name = "TMP";
> level
= [ "P750-900" ];
> cat_thresh = [ <=273, >273 ];
>
lead_time = "06";
> },
>
> {
> name
= "TMP";
> level = [ "P750-900" ];
>
cat_thresh = [ <=273, >273 ];
> lead_time = "12";
>
}
> ];
>
> };
>
> This will verify the 6 and 12-hour
forecasts of 2-meter TMP from your
> file. So you can specify the
timing information for each "field" entry.
> But you just need to be
sure that you pass Point-Stat observations
> that cover this time
range as well.
>
> Hope that helps.
>
> John Halley Gotway
>
met_help at ucar.edu
>
>
> On 12/18/2012 07:15 PM,
Malcolm.Nunn at csiro.au via RT wrote:
>>
>> Tue Dec 18 19:15:58 2012:
Request 59667 was acted upon.
>> Transaction: Ticket created by
Malcolm.Nunn at csiro.au
>> Queue: met_help
>> Subject:
Point_Stat - specifying fcst_valid and fcst_lead in
>> METv4.0
>>
Owner: Nobody
>> Requestors: Malcolm.Nunn at csiro.au
>>
Status: new
>> Ticket <URL:
>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667
>> >
>>
>>
>> Hi MET team.
>>
>> I am trying to use Point Stat to verify GRIB
forecasts with multiple
>> forecast times per file - eg
20110102_0000 initialisation time, with
>> hourly forecast data out
to 72hrs.
>>
>> I have found mention of the fcst_valid and fcst_lead
command line
>> options for previous versions of MET, and noticed
that it is
>> mentioned in the release notes of METv4.0 that this
command lines
>> options have been moved to the config files.
However, I can't find
>> any documentation regarding how/where to
specify them in the config
>> file in the v4.0 User's Guide or
config/README. I would be very
>> grateful if you could advise me or
point me to it if I've overlooked
>> it? Would love to avoid having
to split up my data into separate lead times!
>>
>> Also, I am
slightly confused about the difference between fcst_valid
>> and
fcst_lead - for a given initialisation time (single forecast
>>
file) with multiple timesteps, should I be specifying fcst_lead to
>> verify a particular time? ie, to verify the 12-hour forecast for a
>> 20110102_0000 file, should I specify fcst_lead = 20110102_1200?
>>
>> Thanks very much
>> Malcolm Nunn
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59667] Point_Stat - specifying fcst_valid and fcst_lead in METv4.0
From: John Halley Gotway
Time: Fri Jan 04 11:38:33 2013
Malcolm,
Great, glad you were able to figure that out.
I'll go ahead and resolve this ticket. Feel free to write us back if
more questions arise.
Thanks,
John
On 12/20/2012 10:03 PM, Malcolm.Nunn at csiro.au via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667 >
>
> Whoops, please ignore my last query about the Stat_Analysis
segmentation fault. I just realised I hadn't applied the patch which
deals with that bug when aggregating from MPR. Aggregating CNT from
MPR works fine now!
>
> Cheers (and happy holidays!)
> Malcolm
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, 20 December 2012 4:30 PM
> To: Nunn, Malcolm (CMAR, Aspendale)
> Subject: RE: [rt.rap.ucar.edu #59667] Point_Stat - specifying
fcst_valid and fcst_lead in METv4.0
>
> Malcolm,
>
> Now that you mention the issue of the output file name, I'm not
entirely sure how the code will handle that. Hmmm, that'll take a bit
of testing.
>
> I'd say if you do try to verify multiple lead times in a single call
to Point-Stat, proceed with caution. Run with a high level of
verbosity (-v
> 3 or -v 4), and inspect the log and output files to make sure it's
working as you'd expect. None of the GRIB files I've worked with have
been organized in that way, but I know they're out there.
>
> I'm out of the office for 2 weeks over the holidays, but if you
continue to have issues/questions, I'll be able to assist you more in
early January.
>
> Regarding aggregating the output of point-stat, yes, the answer is
stat-analysis.
>
> When running point-stat, writing out the matched pair (MPR) line
type gives you the most amount of flexibility in stat-analysis. The
downside, of course, is that a lot of MPR output for many matched
pairs takes up a whole lot of space. A middle ground is writing out
the SL1L2 partial sums. While the CNT lines cannot be aggregated
directly in stat-analysis, the SL1L2 lines can and then a subset of
the CNT statistics can be derived from those aggregated partial sums.
Really that's the whole point of the
> SL1L2 line type in the first place.
>
> If you store MPR output from point-stat, the following stat-analysis
job type will be most useful:
> -job aggregate_stat -line_type MPR -out_line_type CNT (more
filtering criteria to subset your data)
>
> If you only store SL1L2 lines, use the following:
> -job aggregate_stat -line_type SL1L2 -out_line_type CNT (more
filtering
> criteria)
>
> Many, but not all, of the CNT stats can be derived from the SL1L2
partial sums. If you haven't already seen it, you may find the stat-
analysis section of the MET online tutorial to be helpful:
>
http://www.dtcenter.org/met/users/support/online_tutorial/METv4.0/stat_analysis/index.php
>
> Thanks,
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667 >
>>
>> Brilliant, thank you. Specifying multiple lead times within a
single
>> config saves a lot of hassle too.
>>
>> Very minor note - the file name generated for point_stat output
will
>> only reflect a single lead time, ie a config to verify at 24, 48
and
>> 72 hr lead time still has an output filename
"[prefix]240000L_20110201_120000V.stat"
>> Not an issue for me but thought I'd mention it.
>>
>> Is there any way to aggregate continuous statistics through MET or
>> does that need to be done with an external script? If, for example,
I
>> filter my output so I have a month of daily CNT stat lines, is
there a
>> way to pool these with stat_analysis? I know it can't aggregate CNT
lines.
>>
>> Thanks again
>> Malcolm
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, 20 December 2012 6:25 AM
>> To: Nunn, Malcolm (CMAR, Aspendale)
>> Subject: Re: [rt.rap.ucar.edu #59667] Point_Stat - specifying
>> fcst_valid and fcst_lead in METv4.0
>>
>> Malcolm,
>>
>> Yes, you are correct. They have been moved from the command line
to
>> the config file. I apologize for the incomplete information in the
>> config README file. I'll make sure that we update the
documentation
>> prior to the next release.
>>
>> Basically, you can use the following settings in the config file:
>>
>> lead_time = "HH[MMSS]";
>> valid_time = "YYYYMMDD[_HH[MMSS]]";
>>
>> You should put this inside the "fcst" and/or "obs" settings in the
>> config file. You can use neither, one, or both of the settings.
If
>> provided, Point-Stat will only match fields that meet this
additional
>> timing criteria.
>>
>> For example, here's how you could edit this config file used by the
>> point_stat test script:
>> METv4.0/scripts/config/PointStatConfig
>>
>> fcst = {
>> wind_thresh = [ NA ];
>> message_type = [ "ADPUPA", "ADPSFC" ];
>>
>> lead_time = "12";
>> valid_time = "20070331_12";
>>
>> field = [
>> {
>> name = "TMP";
>> level = [ "P750-900" ];
>> cat_thresh = [ <=273, >273 ];
>> },
>>
>> {
>> name = "UGRD";
>> level = [ "Z10" ];
>> cat_thresh = [ >=5 ];
>> },
>>
>> {
>> name = "VGRD";
>> level = [ "Z10" ];
>> cat_thresh = [ >=5 ];
>> }
>> ];
>>
>> };
>>
>> Note that in the above setting, this timing information will apply
to
>> each of the 3 fields listed. If your GRIB file contains multiple
>> output times, you could actually verify all of them in a single
call like this:
>>
>> fcst = {
>> wind_thresh = [ NA ];
>> message_type = [ "ADPUPA", "ADPSFC" ];
>>
>> field = [
>> {
>> name = "TMP";
>> level = [ "P750-900" ];
>> cat_thresh = [ <=273, >273 ];
>> lead_time = "06";
>> },
>>
>> {
>> name = "TMP";
>> level = [ "P750-900" ];
>> cat_thresh = [ <=273, >273 ];
>> lead_time = "12";
>> }
>> ];
>>
>> };
>>
>> This will verify the 6 and 12-hour forecasts of 2-meter TMP from
your
>> file. So you can specify the timing information for each "field"
entry.
>> But you just need to be sure that you pass Point-Stat observations
>> that cover this time range as well.
>>
>> Hope that helps.
>>
>> John Halley Gotway
>> met_help at ucar.edu
>>
>>
>> On 12/18/2012 07:15 PM, Malcolm.Nunn at csiro.au via RT wrote:
>>>
>>> Tue Dec 18 19:15:58 2012: Request 59667 was acted upon.
>>> Transaction: Ticket created by Malcolm.Nunn at csiro.au
>>> Queue: met_help
>>> Subject: Point_Stat - specifying fcst_valid and fcst_lead
in
>>> METv4.0
>>> Owner: Nobody
>>> Requestors: Malcolm.Nunn at csiro.au
>>> Status: new
>>> Ticket <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59667
>>>>
>>>
>>>
>>> Hi MET team.
>>>
>>> I am trying to use Point Stat to verify GRIB forecasts with
multiple
>>> forecast times per file - eg 20110102_0000 initialisation time,
with
>>> hourly forecast data out to 72hrs.
>>>
>>> I have found mention of the fcst_valid and fcst_lead command line
>>> options for previous versions of MET, and noticed that it is
>>> mentioned in the release notes of METv4.0 that this command lines
>>> options have been moved to the config files. However, I can't find
>>> any documentation regarding how/where to specify them in the
config
>>> file in the v4.0 User's Guide or config/README. I would be very
>>> grateful if you could advise me or point me to it if I've
overlooked
>>> it? Would love to avoid having to split up my data into separate
lead times!
>>>
>>> Also, I am slightly confused about the difference between
fcst_valid
>>> and fcst_lead - for a given initialisation time (single forecast
>>> file) with multiple timesteps, should I be specifying fcst_lead to
>>> verify a particular time? ie, to verify the 12-hour forecast for a
>>> 20110102_0000 file, should I specify fcst_lead = 20110102_1200?
>>>
>>> Thanks very much
>>> Malcolm Nunn
>>>
>>
>>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list