[Met_help] [rt.rap.ucar.edu #47285] History for point stat question

[RAL HelpDesk {for John Halley Gotway}]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,
I am just starting to use MET package and would like to use it plot time series of standard statistics like bias, RMSE, correlation etc for
wrf forecasts against prepbufr date. I produced ncdf obs files for the length of forecast chunks (say each 6h). In the config file
I have put 

beg_ds = -1799;
end_ds =  30000;


fcst_field[] = [  "TT(0-5,1,*,*)" ];

obs_field[]  = [  "TMP/P925"];

but that still produces statistics for the initial hour only. Is there a method to have separate statistics output for every hour in the forecasts file?
Also,  in *cnt file is it spatial correlation between forecast/obs at the specified time? 
Thanks,
Mariusz




----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #47285] point stat question
From: John Halley Gotway
Time: Thu Jun 02 13:24:42 2011

Mariusz,

The MET statistics tools, like point_stat, grid_stat, and mode, for
example, are designed to be run once for each valid time you're
evaluating.  It looks like you are interested in verifying
temperature for forecast hours 0 through 5.  In that case, you'll need
to call point_stat once for each hour.

You may find the following Point-Stat config file settings to be of
use:

  beg_ds = -300;
  end_ds =  300;
  fcst_field[] = [  "TT(${ITIME},6,*,*)" ];
  obs_field[]  = [  "TMP/P925"];

  - I've set the matching time window to be +/- 5 minutes.  So only
observations falling within that time window around the forecast valid
time will be used.  Set this to whatever you'd like.
  - I've set the first element of the TT array to be ${ITIME}.  This
assumes that you're calling Point-Stat from a script which sets an
environment variable named ITIME to be 0, 1, 2, ..., 5.
  - I've set the third element of the TT array to be 6.  You'll need
take a look at your data to see what the index for pressure level 925
is in your output.  In my sample data, 925 is the 7 pressure
level defined - so it has an index value of 6 (zero-based, not one-
based).  Try doing "ncdump -v pressure pinterp_file.nc" to see what
the pressure levels are in your p_interp output.

Once you have output for each individual hour, you can plot it to make
a time series.  All of the stats you listed are contained in the
continuous statistics line type (CNT).  So it would suffice to
set the output_flag in the Point-Stat config file so that only
continuous statistics are output.  Although you may want the partial
sums (SL1L2) and/or matched pair (MPR) lines as well.  Just refer to
comments in the config file.

Once you have output for several times, you may find it useful to use
the STAT-Analysis tool to aggregate your results through time.  For
example, you could use STAT-Analysis to look at the
performance of the temperature forecast for all 12 hour lead times
over a whole season.  Just let us know if more questions arise.

Hope that helps.

Thanks,
John Halley Gotway
met_help at ucar.edu


On 06/02/2011 12:12 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>
> Thu Jun 02 12:12:33 2011: Request 47285 was acted upon.
> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>        Queue: met_help
>      Subject: point stat question
>        Owner: Nobody
>   Requestors: Mariusz.Pagowski at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47285 >
>
>
> Hello,
> I am just starting to use MET package and would like to use it plot
time series of standard statistics like bias, RMSE, correlation etc
for
> wrf forecasts against prepbufr date. I produced ncdf obs files for
the length of forecast chunks (say each 6h). In the config file
> I have put
>
> beg_ds = -1799;
> end_ds =  30000;
>
>
> fcst_field[] = [  "TT(0-5,1,*,*)" ];
>
> obs_field[]  = [  "TMP/P925"];
>
> but that still produces statistics for the initial hour only. Is
there a method to have separate statistics output for every hour in
the forecasts file?
> Also,  in *cnt file is it spatial correlation between forecast/obs
at the specified time?
> Thanks,
> Mariusz
>
>

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47285] point stat question
From: Mariusz Pagowski
Time: Thu Jun 02 14:25:39 2011

Thakns a lot John, very helpful reply,
Mariusz

On Jun 2, 2011, at 1:24 PM, RAL HelpDesk {for John Halley Gotway}
wrote:

> Mariusz,
>
> The MET statistics tools, like point_stat, grid_stat, and mode, for
example, are designed to be run once for each valid time you're
evaluating.  It looks like you are interested in verifying
> temperature for forecast hours 0 through 5.  In that case, you'll
need to call point_stat once for each hour.
>
> You may find the following Point-Stat config file settings to be of
use:
>
>  beg_ds = -300;
>  end_ds =  300;
>  fcst_field[] = [  "TT(${ITIME},6,*,*)" ];
>  obs_field[]  = [  "TMP/P925"];
>
>  - I've set the matching time window to be +/- 5 minutes.  So only
observations falling within that time window around the forecast valid
time will be used.  Set this to whatever you'd like.
>  - I've set the first element of the TT array to be ${ITIME}.  This
assumes that you're calling Point-Stat from a script which sets an
environment variable named ITIME to be 0, 1, 2, ..., 5.
>  - I've set the third element of the TT array to be 6.  You'll need
take a look at your data to see what the index for pressure level 925
is in your output.  In my sample data, 925 is the 7 pressure
> level defined - so it has an index value of 6 (zero-based, not one-
based).  Try doing "ncdump -v pressure pinterp_file.nc" to see what
the pressure levels are in your p_interp output.
>
> Once you have output for each individual hour, you can plot it to
make a time series.  All of the stats you listed are contained in the
continuous statistics line type (CNT).  So it would suffice to
> set the output_flag in the Point-Stat config file so that only
continuous statistics are output.  Although you may want the partial
sums (SL1L2) and/or matched pair (MPR) lines as well.  Just refer to
> comments in the config file.
>
> Once you have output for several times, you may find it useful to
use the STAT-Analysis tool to aggregate your results through time.
For example, you could use STAT-Analysis to look at the
> performance of the temperature forecast for all 12 hour lead times
over a whole season.  Just let us know if more questions arise.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> On 06/02/2011 12:12 PM, RAL HelpDesk {for Mariusz Pagowski} wrote:
>>
>> Thu Jun 02 12:12:33 2011: Request 47285 was acted upon.
>> Transaction: Ticket created by Mariusz.Pagowski at noaa.gov
>>       Queue: met_help
>>     Subject: point stat question
>>       Owner: Nobody
>>  Requestors: Mariusz.Pagowski at noaa.gov
>>      Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47285 >
>>
>>
>> Hello,
>> I am just starting to use MET package and would like to use it plot
time series of standard statistics like bias, RMSE, correlation etc
for
>> wrf forecasts against prepbufr date. I produced ncdf obs files for
the length of forecast chunks (say each 6h). In the config file
>> I have put
>>
>> beg_ds = -1799;
>> end_ds =  30000;
>>
>>
>> fcst_field[] = [  "TT(0-5,1,*,*)" ];
>>
>> obs_field[]  = [  "TMP/P925"];
>>
>> but that still produces statistics for the initial hour only. Is
there a method to have separate statistics output for every hour in
the forecasts file?
>> Also,  in *cnt file is it spatial correlation between forecast/obs
at the specified time?
>> Thanks,
>> Mariusz
>>
>>
>


------------------------------------------------