[Met_help] [rt.rap.ucar.edu #59926] History for Verification with a persistence reference forecast

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi John

I was looking to use a persistence forecast as a reference for comparison alongside my forecast verification. From what I can determine, there is no in-built feature in METv4.0 for this? I just wanted to check that I haven't missed anything before implementing a work-around. I know there is the option to specify a climatology file in Point-Stat but that's not quite what I'm after.

If there is no feature in METv4.0 for it, I am thinking the best way to do it would be to specify a forecast from the previous day for a given set of observations - however so that MET would match up the times it would require altering the init_time and valid_time attributes so that MET believes it was the correct day. This is certainly doable but a little clunky. Do you have any other suggestions for producing verification scores for a persistence forecast?

Cheers
Malcolm Nunn


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #59926] Verification with a persistence reference forecast
From: John Halley Gotway
Time: Wed Jan 23 09:01:29 2013

Malcolm,

I apologize for the delay in getting back to you.  I overlooked your
email and just ran across it today.

The answer to your question is no, there is not anything in particular
in the MET that's set up for verifying persistence forecasts.  There
are any number of ways a "persistence" forecast could be
defined, so it's not clear what enhancements would be generally useful
for them.  However, you do not need to actually modify the date/time
stamps within the input files themselves.  When your run the
MET stat tools, you typically pass them a forecast file, an
observation file, and a configuration file to tell the tool which
fields/methods you'd like to verify.

The Grid-Stat and MODE tools will retrieve the fields from the input
files and then print a warning if their valid times do not match.
That will be the case for a persistence forecast.  So you'll see
a warning message about the times not matching, but you should still
get meaningful output.

The Point-Stat tool retrieves the gridded fields from the input
forecast file.  Then it defines a matching observation time window
around that forecast valid time.  It uses the "beg" and "end" values
from the "obs_window" portion of the config file to define the time
window around the forecast valid time.  For a persistence forecast,
you'd want that time window defined a little differently.  So
you could modify the beg and end values to set the window how you'd
like.  Alternatively, you could use the "-obs_valid_beg" and "-
obs_valid_end" command line options to explicitly set that window in
YYYYMMDD[_HH[MMSS]] format.  The command line options override the
config file values.  I'd probably do it the later way, but either
would work.

So the fact that the valid times don't match up for a persistence
forecast really is fine.  There are ways of getting around it without
actually modifying the times in the input files.

One last word of warning - when verifying a persistence forecast, be
sure to set the "model" value in the config file to something
descriptive.  Otherwise, you won't be able to distinguish the regular
forecast output from the persistence forecast output.

Hope that helps, and my apologizes again for the delay.

John Halley Gotway
met_help at ucar.edu

On 01/16/2013 06:32 PM, Malcolm.Nunn at csiro.au via RT wrote:
>
> Wed Jan 16 18:32:57 2013: Request 59926 was acted upon.
> Transaction: Ticket created by Malcolm.Nunn at csiro.au
>         Queue: met_help
>       Subject: Verification with a persistence reference forecast
>         Owner: Nobody
>    Requestors: Malcolm.Nunn at csiro.au
>        Status: new
>   Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59926 >
>
>
> Hi John
>
> I was looking to use a persistence forecast as a reference for
comparison alongside my forecast verification. From what I can
determine, there is no in-built feature in METv4.0 for this? I just
wanted to check that I haven't missed anything before implementing a
work-around. I know there is the option to specify a climatology file
in Point-Stat but that's not quite what I'm after.
>
> If there is no feature in METv4.0 for it, I am thinking the best way
to do it would be to specify a forecast from the previous day for a
given set of observations - however so that MET would match up the
times it would require altering the init_time and valid_time
attributes so that MET believes it was the correct day. This is
certainly doable but a little clunky. Do you have any other
suggestions for producing verification scores for a persistence
forecast?
>
> Cheers
> Malcolm Nunn
>

------------------------------------------------