[Met_help] [rt.rap.ucar.edu #61729] History for Evaluating WRF output fields on pressure level with METv4.1

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

Right now I am using METv4.1 to evaluate our WRF forecasting system. I want to use Grid-Stat to compare our 12-hour forecast with the NCEP reanalysis data (processed by WRF as the 00-hr from wrfout). Both 12-hr forecast and 00-hr observation are post-processed with UPPV2.0 to obtain the GRIB format. I encounter some problems when trying to use Grid-Stat to compare the pressure surface (1000mb, 925mb ... etc) HGT, TMP, U and V. As we know, some grid points on these surfaces, such as 1000mb, are actually below ground. Furthermore, some grid points on a surface may be above ground in FCST file but below ground in OBS file. I want to get rid of these grid points that are below ground either in the FCST or the OBS file when calculating the RMSE and correlation. I looked through the MET manual and it seems it is not described explicitly, though I believed there is a very simple way dealing with the GridStatConfig file. How could I do so? Or should I rather go back to UPP?

Another issue is - the time taken to compute all the scores listed in the CNT file is very long (~4 minutes for a 250x200 domain). In fact I just want the RMSE and correlation. I cannot get from the MET manual how I could turn off calculating all other scores.

Hope you could help.

Best regards,

LO Chi On Andrew * Research Officer * Centre for Climate Research Singapore * Tel : +65 6488 1857 * Fax : +65 6289 9313 * Email : andrew_lo at nea.gov.sg<mailto:Felicia_Shaw at nea.gov.sg>

[cid:image003.jpg at 01CE62C6.492A5200]

This message may contain confidential information under the purview of the Official Secrets Act. Unauthorized communication or disclosure of such information is an offence under the Official Secrets Act. If you are not the intended recipient of this message, please notify the sender and delete it. Do not retain it or disclose the contents to any person, as it may be an offence under the Official Secrets Act.

Please consider the environment before printing this email






----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #61729] Evaluating WRF output fields on pressure level with METv4.1
From: John Halley Gotway
Time: Thu Jun 06 11:11:58 2013

LO Chi On Andrew,

I've tried sending this message a couple of times, but our support
system seems not to be working well.  So I'm sending it directly to
your email address this time...

The answer to your second question is easier.  In the Grid-Stat
configuration file, in the "boot" section, make sure that "n_rep" is
set to 0.  That disables the computation of bootstrap confidence
intervals, which is time consuming.  Also, set "rank_corr_flag =
FALSE;".  That disables the computation of rank correlations, which is
also time consuming.  Lastly, if you're really only interested
in RMSE and correlation, you only need the continuous (CNT) statistics
output line.  So in the output flag section, you could set "cnt" to
"STAT" or "BOTH", and the rest of the options to "NONE".
However, I'd suggest also outputting the "sl1l2" partial sums line
type.  That will enable you to use stat-analysis to aggregate the
partial sums across multiple runs and compute aggregated statistics.

Regarding the first question, you can do something in MET called "data
masking".  Basically, you select a single 2D field and supply a
threshold for it.  And grid-stat will only use points where that
threshold criteria is MET.  Do you have a field in your WRF output
that indicates which 1000mb grid points are below ground?

Here are two options:
(1) You could pass in a topography field and only verify grid points
whose elevation is less than a certain value.  You could choose the
elevation conservatively so that none of the grid points at
1000mb would be below ground.  And when you run multiple times, you'd
be using the same masking region throughout - which is good.

(2) You could pass in surface pressure and only verify grid points
whose surface pressure is >= 1000mb.  The downside is that this will
change from model run to model run.

Here's how you'd do both of these types of masks:

mask = {
    grid = [];
    poly = ["WRFPRS_d01_12.tm00 { name=\"HGT\";  level=\"Z0\"; }
<1500",
            "WRFPRS_d01_12.tm00 { name=\"PRES\"; level=\"Z0\"; }
>=100000" ];
};

The first masking region is only those grid points whose surface
height is less-than 1500 meters are used.  The second is only those
grid points whose surface pressure is >= 1000 mb.

Hope that helps.

Thanks,
John Halley Gotway
met_help at ucar.edu


------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #61729] Evaluating WRF output fields on pressure level with METv4.1
From: Andrew LO
Time: Thu Jun 06 21:03:02 2013

Hi John,

Thanks very much for your help. Just after sending out the first email
I already realized turning on "rank_corr_flag" will make the
computation so time-consuming. Now it takes only a second or two to
calculate tens of variables.

The syntax for the second mask is going to be very useful. Just one
last question - I would like to read the surface pressure from both
FCST and OBS files. What is the correct syntax to mean something like
this
    poly = ["FCST_file { name=\"PRES\"; level=\"Z0\"; } >=100000" AND
            "OBS_file  { name=\"PRES\"; level=\"Z0\"; } >=100000" ];
With the comma instead of the 'AND', MET produces separate mask for
each file and gives me two different sets of RMSE and corr using each
mask. This is not yet what I expect to get because I would like to
mask out all the points at which P1000 is below ground no matter in
the FCST or OBS or both. Basically I would like to know how the Config
can read from two (or more) files to generate one single mask.

Will be happy to know from where I can read more about syntax writing
the MET Config.

Cheers,

LO Chi On Andrew • Research Officer • Centre for Climate Research
Singapore • Tel : +65 6488 1857 • Fax : +65 6289 9313 • Email :
andrew_lo at nea.gov.sg

This message may contain confidential information under the purview of
the Official Secrets Act. Unauthorized communication or disclosure of
such information is an offence under the Official Secrets Act. If you
are not the intended recipient of this message, please notify the
sender and delete it. Do not retain it or disclose the contents to any
person, as it may be an offence under the Official Secrets Act.

Please consider the environment before printing this email








-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, 7 June, 2013 1:12 AM
To: Andrew LO (NEA)
Subject: Re: [rt.rap.ucar.edu #61729] Evaluating WRF output fields on
pressure level with METv4.1

LO Chi On Andrew,

I've tried sending this message a couple of times, but our support
system seems not to be working well.  So I'm sending it directly to
your email address this time...

The answer to your second question is easier.  In the Grid-Stat
configuration file, in the "boot" section, make sure that "n_rep" is
set to 0.  That disables the computation of bootstrap confidence
intervals, which is time consuming.  Also, set "rank_corr_flag =
FALSE;".  That disables the computation of rank correlations, which is
also time consuming.  Lastly, if you're really only interested in RMSE
and correlation, you only need the continuous (CNT) statistics output
line.  So in the output flag section, you could set "cnt" to "STAT" or
"BOTH", and the rest of the options to "NONE".
However, I'd suggest also outputting the "sl1l2" partial sums line
type.  That will enable you to use stat-analysis to aggregate the
partial sums across multiple runs and compute aggregated statistics.

Regarding the first question, you can do something in MET called "data
masking".  Basically, you select a single 2D field and supply a
threshold for it.  And grid-stat will only use points where that
threshold criteria is MET.  Do you have a field in your WRF output
that indicates which 1000mb grid points are below ground?

Here are two options:
(1) You could pass in a topography field and only verify grid points
whose elevation is less than a certain value.  You could choose the
elevation conservatively so that none of the grid points at 1000mb
would be below ground.  And when you run multiple times, you'd be
using the same masking region throughout - which is good.

(2) You could pass in surface pressure and only verify grid points
whose surface pressure is >= 1000mb.  The downside is that this will
change from model run to model run.

Here's how you'd do both of these types of masks:

mask = {
    grid = [];
    poly = ["WRFPRS_d01_12.tm00 { name=\"HGT\";  level=\"Z0\"; }
<1500",
            "WRFPRS_d01_12.tm00 { name=\"PRES\"; level=\"Z0\"; }
>=100000" ]; };

The first masking region is only those grid points whose surface
height is less-than 1500 meters are used.  The second is only those
grid points whose surface pressure is >= 1000 mb.

Hope that helps.

Thanks,
John Halley Gotway
met_help at ucar.edu




------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #61729] Evaluating WRF output fields on pressure level with METv4.1
From: John Halley Gotway
Time: Fri Jun 07 09:37:03 2013

Andrew,

I thought you might ask about that.  Unfortunately no, MET does not
currently support that type of data masking.  It's only set up to
handle a single field and a single threshold.  But that certainly
would be a reasonable extension of the functionality.

If you need to do that at this point, you'd need to generate a single
field that contains the maximum of the two fields outside of MET.

Thanks,
John

On 06/06/2013 09:03 PM, Andrew LO via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61729 >
>
> Hi John,
>
> Thanks very much for your help. Just after sending out the first
email I already realized turning on "rank_corr_flag" will make the
computation so time-consuming. Now it takes only a second or two to
calculate tens of variables.
>
> The syntax for the second mask is going to be very useful. Just one
last question - I would like to read the surface pressure from both
FCST and OBS files. What is the correct syntax to mean something like
this
>      poly = ["FCST_file { name=\"PRES\"; level=\"Z0\"; } >=100000"
AND
>              "OBS_file  { name=\"PRES\"; level=\"Z0\"; } >=100000"
];
> With the comma instead of the 'AND', MET produces separate mask for
each file and gives me two different sets of RMSE and corr using each
mask. This is not yet what I expect to get because I would like to
mask out all the points at which P1000 is below ground no matter in
the FCST or OBS or both. Basically I would like to know how the Config
can read from two (or more) files to generate one single mask.
>
> Will be happy to know from where I can read more about syntax
writing the MET Config.
>
> Cheers,
>
> LO Chi On Andrew • Research Officer • Centre for Climate Research
Singapore • Tel : +65 6488 1857 • Fax : +65 6289 9313 • Email :
andrew_lo at nea.gov.sg
>
> This message may contain confidential information under the purview
of the Official Secrets Act. Unauthorized communication or disclosure
of such information is an offence under the Official Secrets Act. If
you are not the intended recipient of this message, please notify the
sender and delete it. Do not retain it or disclose the contents to any
person, as it may be an offence under the Official Secrets Act.
>
> Please consider the environment before printing this email
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, 7 June, 2013 1:12 AM
> To: Andrew LO (NEA)
> Subject: Re: [rt.rap.ucar.edu #61729] Evaluating WRF output fields
on pressure level with METv4.1
>
> LO Chi On Andrew,
>
> I've tried sending this message a couple of times, but our support
system seems not to be working well.  So I'm sending it directly to
your email address this time...
>
> The answer to your second question is easier.  In the Grid-Stat
configuration file, in the "boot" section, make sure that "n_rep" is
set to 0.  That disables the computation of bootstrap confidence
intervals, which is time consuming.  Also, set "rank_corr_flag =
FALSE;".  That disables the computation of rank correlations, which is
also time consuming.  Lastly, if you're really only interested in RMSE
and correlation, you only need the continuous (CNT) statistics output
line.  So in the output flag section, you could set "cnt" to "STAT" or
"BOTH", and the rest of the options to "NONE".
> However, I'd suggest also outputting the "sl1l2" partial sums line
type.  That will enable you to use stat-analysis to aggregate the
partial sums across multiple runs and compute aggregated statistics.
>
> Regarding the first question, you can do something in MET called
"data masking".  Basically, you select a single 2D field and supply a
threshold for it.  And grid-stat will only use points where that
threshold criteria is MET.  Do you have a field in your WRF output
that indicates which 1000mb grid points are below ground?
>
> Here are two options:
> (1) You could pass in a topography field and only verify grid points
whose elevation is less than a certain value.  You could choose the
elevation conservatively so that none of the grid points at 1000mb
would be below ground.  And when you run multiple times, you'd be
using the same masking region throughout - which is good.
>
> (2) You could pass in surface pressure and only verify grid points
whose surface pressure is >= 1000mb.  The downside is that this will
change from model run to model run.
>
> Here's how you'd do both of these types of masks:
>
> mask = {
>      grid = [];
>      poly = ["WRFPRS_d01_12.tm00 { name=\"HGT\";  level=\"Z0\"; }
<1500",
>              "WRFPRS_d01_12.tm00 { name=\"PRES\"; level=\"Z0\"; }
>=100000" ]; };
>
> The first masking region is only those grid points whose surface
height is less-than 1500 meters are used.  The second is only those
grid points whose surface pressure is >= 1000 mb.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
>

------------------------------------------------