[Met_help] [rt.rap.ucar.edu #40984] History for Verifying 5-Minute Winds in WRF
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Mon Sep 27 13:26:56 MDT 2010
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Greetings Met_help:
I've added code to WRF to create 5-minute winds at both 10 meters and 100 meters, and I'd like to do verification in MET using some Texas Tech Reese Tower obs available to me. I've created unique 2-d arrays for each 5-minute wind component, e.g., U10MIN05/V10MIN05, U10MIN10/V10MIN10, U10MIN15/V10MIN15, etc. These are the U and V component 10-meter 5-minute winds valid at 5 minutes, 10 minutes, and 15 minutes, respectively. I have similar arrays for the U and V components at 100 meters. These arrays are output in hourly WRF histories.
My first question is this ... how can I get MET to verify all of the 10-meter (and 100-meter) 5-minute winds given that they exist in unique arrays? In other words, I would like to verify these special U and V wind components as I would the standard U10 and V10 arrays, but in this case I have 12 U/V component 10-meter 5-minute wind arrays in each hourly history file. Is there some simple way to bin these unique arrays together in MET so I get stats for "5-minute winds"?
Similarly, how can I get MET to match these WRF fields with the obs? I assume MET will consider each of these WRF fields to be valid at the top-of-the-hour time given that they're written in the hourly wrfout files, while the obs are valid at 5-minutes past the hour, 10-minutes past the hour, etc. I will need to convert the ASCII format Reese Tower obs to netCDF for input into MET, so I suppose I could assign the 5-minute obs to fields that match the WRF field names (U10MIN05/V10MIN05, U10MIN10/V10MIN10, etc), but that still leaves the issue of verifying all 12 5-minute fields available each hour.
Any guidance you can offer would be greatly appreciated. Many thanks!
Regards,
Scott
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in WRF
From: John Halley Gotway
Time: Mon Sep 20 12:31:50 2010
Scott,
OK, I think there are several issues to address here. I'll try to go
through them one by one...
(1) Version of MET:
We're currently testing the next release of MET, version 3.0, and
expect to be releasing it in the next week or so. The last big thing
to do is finish up the documentation. In METv3.0, we're able to
read the output of the ARW-pinterp tool, which I'll discuss below. If
it'd be helpful, I'd be happy to send you a beta release of METv3.0
you could use with your data. Just let me know.
(2) Format of the point observations:
This should be pretty straight-forward. I'd suggest reformatting your
5 minute point observations into ASCII, and then run them through the
ASCII2NC program. You're welcome to group them however
you'd like - breaking them out by time, or putting them all the times
in the same NetCDF file. If they're all in the same file, you can use
command line arguments in Point-Stat to set the matching
time window. For now, I'd suggest setting the observation message
type as MSONET.
(3) Format of the gridded forecasts:
Ultimately, METv3.0 is able to read gridded data in GRIB format or the
NetCDF output of the pinterp tool (for ARW). It does not read the
NetCDF output of WRF directly. So you'll need to postprocess
your WRF output either using the WRF-PostProcessor (for GRIB,
recommended!) or pinterp (for NetCDF output, new for v3.0).
However, since you've added new variables to your WRF output files, I
have no idea how WPP or pinterp will handle them. I'm guess that WPP
will NOT handle them well, and I have no idea about pinterp.
Can you tell me, is there a reason you modified the WRF code rather
than just setting your history interval to be every 5-minutes
(history_interval 5)? If you did the later, you'd be able to run your
WRF output through WPP to get GRIB files with 5-minute winds.
(4) Verifying your 5-minutes winds in MET:
So let's assume you have your observations as the NetCDF output of
ASCII2NC, and let's assume you have your gridded data in a format MET
can read. Next, you'd run Point-Stat once for each valid time
you'd like to verify. On the command line, you can use the "-
obs_valid_beg" and "-obs_valid_end" to explicitly set the matching
observation time window for Point-Stat. You'd verify the 10-meter
winds and the 100-meter winds separately. And since you probably
don't have tons of data, you could write out the MPR (matched pair)
line type from each run of Point-Stat.
After you've run Point-Stat once for valid time, you could use the
STAT-Analysis tool to perform whatever types of aggregations you want.
With STAT-Analysis, you can group together the matched pairs
from multiple valid times or levels (to combine the 10m and 100m
verification).
I suspect you'll need more help to get this up and running. It seems
to me that the following steps may be difficult:
- Getting your gridded data into a format MET can read.
- Setting the message type and level information correctly for Point-
Stat to verify at 10-meters and 100-meters.
- Setting up the STAT-Analysis commands to perform the types of
aggregations you want.
Let me know if you get stuck, and need some help. If you do, please
send me some sample data: a gridded forecast file, ASCII point
observation file, and Point-Stat configuration file.
Hope that helps.
Thanks,
John Halley Gotway
met_help at ucar.edu
RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
> Mon Sep 20 09:43:18 2010: Request 40984 was acted upon.
> Transaction: Ticket created by scott.r.dembek at nasa.gov
> Queue: met_help
> Subject: Verifying 5-Minute Winds in WRF
> Owner: Nobody
> Requestors: scott.r.dembek at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>
>
> Greetings Met_help:
>
> I've added code to WRF to create 5-minute winds at both 10 meters
and 100 meters, and I'd like to do verification in MET using some
Texas Tech Reese Tower obs available to me. I've created unique 2-d
arrays for each 5-minute wind component, e.g., U10MIN05/V10MIN05,
U10MIN10/V10MIN10, U10MIN15/V10MIN15, etc. These are the U and V
component 10-meter 5-minute winds valid at 5 minutes, 10 minutes, and
15 minutes, respectively. I have similar arrays for the U and V
components at 100 meters. These arrays are output in hourly WRF
histories.
>
> My first question is this ... how can I get MET to verify all of the
10-meter (and 100-meter) 5-minute winds given that they exist in
unique arrays? In other words, I would like to verify these special U
and V wind components as I would the standard U10 and V10 arrays, but
in this case I have 12 U/V component 10-meter 5-minute wind arrays in
each hourly history file. Is there some simple way to bin these
unique arrays together in MET so I get stats for "5-minute winds"?
>
> Similarly, how can I get MET to match these WRF fields with the obs?
I assume MET will consider each of these WRF fields to be valid at the
top-of-the-hour time given that they're written in the hourly wrfout
files, while the obs are valid at 5-minutes past the hour, 10-minutes
past the hour, etc. I will need to convert the ASCII format Reese
Tower obs to netCDF for input into MET, so I suppose I could assign
the 5-minute obs to fields that match the WRF field names
(U10MIN05/V10MIN05, U10MIN10/V10MIN10, etc), but that still leaves the
issue of verifying all 12 5-minute fields available each hour.
>
> Any guidance you can offer would be greatly appreciated. Many
thanks!
>
> Regards,
> Scott
>
------------------------------------------------
Subject: Verifying 5-Minute Winds in WRF
From: Dembek, Scott Robert[Universities Space Research Association]
Time: Mon Sep 20 13:16:22 2010
Hi John,
Thanks for your quick reply! I'll try to tackle this one piece at a
time, and will follow up with you as things get confusing. Before I
get started...
(1) Version of MET:
I suspect I can start formatting the obs using ASCII2NC with MET v2.0
which I have already installed locally. By the time I get that worked
out I can come back to the issue of v3.0 vs. v2.0. If you think it's
to my advantage to use V3.0, I'll definitely give it a try. Or will I
have to use v3.0 beginning with ASCII2NC if I want to continue to use
it later in the process?
(2) Formatting the point obs:
I think I understand how this works in principle, but I want to be
sure I understand your comments. Are you saying it's OK for me to
generate a single hourly NetCDF file and I could use the same names as
I used in the WRF histories ... call the 5-min obs U10MIN05/V10MIN05,
call the 10-min obs U10MIN10/V10MIN10, etc? They would all have the
same valid time if I did it that way, is that correct?
(3) Format of the gridded forecasts:
Two reasons for doing the winds the way I did. First, these are 5-min
average winds I'm generating in WRF so I've coded WRF to make the 5-
min average calculation as it iterates through a forecast, and the
vertical interpolation to 100m is done in WRF as well. Second, I'm
running the code in our daily NSSL WRF "real-time" forecasts which
write hourly histories, and I didn't have the freedom to modify the
WRF output frequency but I was allowed to add fields to the histories.
That's why I created 12 U/V 10m wind fields each hour and gave them
each a unique name (same for 100m).
I'm very familiar with WRFPOST so I don't foresee a problem modifying
it to handle my unique fields, but if I understand your comments
that's only necessary if I decide to use MET v3.0? If I stick with
v2.0 the fields can be read directly from the existing NetCDF files I
have on hand, correct? Or do I have to use v3.0 in order to do the
verification as you've described it?
(4) Verifying the 5-minute winds in MET:
I'll leave this alone for now as I suspect it's a long way off and
I'll have a lot to manage before I get to that point.
Thanks again!
Regards,
Scott
________________________________
From: RAL HelpDesk {for John Halley Gotway} [met_help at ucar.edu]
Sent: Monday, September 20, 2010 2:31 PM
To: Dembek, Scott Robert (MSFC-VP61)[Universities Space Research
Association (USRA)]
Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in WRF
Scott,
OK, I think there are several issues to address here. I'll try to go
through them one by one...
(1) Version of MET:
We're currently testing the next release of MET, version 3.0, and
expect to be releasing it in the next week or so. The last big thing
to do is finish up the documentation. In METv3.0, we're able to
read the output of the ARW-pinterp tool, which I'll discuss below. If
it'd be helpful, I'd be happy to send you a beta release of METv3.0
you could use with your data. Just let me know.
(2) Format of the point observations:
This should be pretty straight-forward. I'd suggest reformatting your
5 minute point observations into ASCII, and then run them through the
ASCII2NC program. You're welcome to group them however
you'd like - breaking them out by time, or putting them all the times
in the same NetCDF file. If they're all in the same file, you can use
command line arguments in Point-Stat to set the matching
time window. For now, I'd suggest setting the observation message
type as MSONET.
(3) Format of the gridded forecasts:
Ultimately, METv3.0 is able to read gridded data in GRIB format or the
NetCDF output of the pinterp tool (for ARW). It does not read the
NetCDF output of WRF directly. So you'll need to postprocess
your WRF output either using the WRF-PostProcessor (for GRIB,
recommended!) or pinterp (for NetCDF output, new for v3.0).
However, since you've added new variables to your WRF output files, I
have no idea how WPP or pinterp will handle them. I'm guess that WPP
will NOT handle them well, and I have no idea about pinterp.
Can you tell me, is there a reason you modified the WRF code rather
than just setting your history interval to be every 5-minutes
(history_interval 5)? If you did the later, you'd be able to run your
WRF output through WPP to get GRIB files with 5-minute winds.
(4) Verifying your 5-minutes winds in MET:
So let's assume you have your observations as the NetCDF output of
ASCII2NC, and let's assume you have your gridded data in a format MET
can read. Next, you'd run Point-Stat once for each valid time
you'd like to verify. On the command line, you can use the "-
obs_valid_beg" and "-obs_valid_end" to explicitly set the matching
observation time window for Point-Stat. You'd verify the 10-meter
winds and the 100-meter winds separately. And since you probably
don't have tons of data, you could write out the MPR (matched pair)
line type from each run of Point-Stat.
After you've run Point-Stat once for valid time, you could use the
STAT-Analysis tool to perform whatever types of aggregations you want.
With STAT-Analysis, you can group together the matched pairs
from multiple valid times or levels (to combine the 10m and 100m
verification).
I suspect you'll need more help to get this up and running. It seems
to me that the following steps may be difficult:
- Getting your gridded data into a format MET can read.
- Setting the message type and level information correctly for Point-
Stat to verify at 10-meters and 100-meters.
- Setting up the STAT-Analysis commands to perform the types of
aggregations you want.
Let me know if you get stuck, and need some help. If you do, please
send me some sample data: a gridded forecast file, ASCII point
observation file, and Point-Stat configuration file.
Hope that helps.
Thanks,
John Halley Gotway
met_help at ucar.edu
RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
> Mon Sep 20 09:43:18 2010: Request 40984 was acted upon.
> Transaction: Ticket created by scott.r.dembek at nasa.gov
> Queue: met_help
> Subject: Verifying 5-Minute Winds in WRF
> Owner: Nobody
> Requestors: scott.r.dembek at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>
>
> Greetings Met_help:
>
> I've added code to WRF to create 5-minute winds at both 10 meters
and 100 meters, and I'd like to do verification in MET using some
Texas Tech Reese Tower obs available to me. I've created unique 2-d
arrays for each 5-minute wind component, e.g., U10MIN05/V10MIN05,
U10MIN10/V10MIN10, U10MIN15/V10MIN15, etc. These are the U and V
component 10-meter 5-minute winds valid at 5 minutes, 10 minutes, and
15 minutes, respectively. I have similar arrays for the U and V
components at 100 meters. These arrays are output in hourly WRF
histories.
>
> My first question is this ... how can I get MET to verify all of the
10-meter (and 100-meter) 5-minute winds given that they exist in
unique arrays? In other words, I would like to verify these special U
and V wind components as I would the standard U10 and V10 arrays, but
in this case I have 12 U/V component 10-meter 5-minute wind arrays in
each hourly history file. Is there some simple way to bin these
unique arrays together in MET so I get stats for "5-minute winds"?
>
> Similarly, how can I get MET to match these WRF fields with the obs?
I assume MET will consider each of these WRF fields to be valid at the
top-of-the-hour time given that they're written in the hourly wrfout
files, while the obs are valid at 5-minutes past the hour, 10-minutes
past the hour, etc. I will need to convert the ASCII format Reese
Tower obs to netCDF for input into MET, so I suppose I could assign
the 5-minute obs to fields that match the WRF field names
(U10MIN05/V10MIN05, U10MIN10/V10MIN10, etc), but that still leaves the
issue of verifying all 12 5-minute fields available each hour.
>
> Any guidance you can offer would be greatly appreciated. Many
thanks!
>
> Regards,
> Scott
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in WRF
From: John Halley Gotway
Time: Mon Sep 20 14:56:45 2010
Scott,
(1) Version of MET:
I think it'd be fine to use ASCII2NC from METv2.0. In METv3.0, we've
added a couple of global attributes, but I don't think Point-Stat
actually reads them. So ASCII2NC from METv2.0 should be fine.
(2) Formatting the point obs:
The ASCII point observation format to be used for as input to ASCII2NC
is pretty simple:
The only currently supported ASCII format is "met_point" in
which each row of observation data consists of 10 columns:
Message_Type Station_ID Valid_Time(YYYYMMDD_HHMMSS)
Lat(Deg North) Lon(Deg East) Elevation(msl)
Grib_Code Level Height(msl or agl) Observation_Value
where "Level" is the pressure level (hPa) or
accumulation interval (hh[_mmss]).
"Height" is meters above sea level or above
ground level for the observation (msl or agl).
Use a value of "-9999" to indicate missing
data.
You don't name any NetCDF variables yourself. Instead, for each
observation value you list out the 10 ASCII columns listed above. The
GRIB code would be 33 for U-component, 34 for V-component, or 32
for wind speed. You'd set "Level" to -9999 for NA, and you'd set
height in meters for the location of the observation. The height (AGL
or MSL) should be defined in a way that's consistent with how
your forecast data is output. You'd set the valid time for each
observation in YYYYMMDD_HHMMSS format. Once you have the ASCII
observations formatted, you'd run them through the ASCII2NC tool.
(3) Format of the gridded forecasts:
I was actually making the opposite point about METv2.0 vs METv3.0.
They both handle the GRIB output of WPP. However, in METv3.0, we've
added support for the NetCDF output of the pinterp tool (not
WRFOUT files directly). If you're able to run your WRF output through
WPP and generate GRIB output files, that'd be ideal. I don't know
exactly how you'll do that, but I'll leave that to you. No
matter what, I'd recommend using METv3.0. That way, if issues come up
and changes are required, it'll be easier to provide you with support.
Hope that helps.
John
RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>
> Hi John,
>
> Thanks for your quick reply! I'll try to tackle this one piece at a
time, and will follow up with you as things get confusing. Before I
get started...
>
> (1) Version of MET:
>
> I suspect I can start formatting the obs using ASCII2NC with MET
v2.0 which I have already installed locally. By the time I get that
worked out I can come back to the issue of v3.0 vs. v2.0. If you
think it's to my advantage to use V3.0, I'll definitely give it a try.
Or will I have to use v3.0 beginning with ASCII2NC if I want to
continue to use it later in the process?
>
> (2) Formatting the point obs:
>
> I think I understand how this works in principle, but I want to be
sure I understand your comments. Are you saying it's OK for me to
generate a single hourly NetCDF file and I could use the same names as
I used in the WRF histories ... call the 5-min obs U10MIN05/V10MIN05,
call the 10-min obs U10MIN10/V10MIN10, etc? They would all have the
same valid time if I did it that way, is that correct?
>
> (3) Format of the gridded forecasts:
>
> Two reasons for doing the winds the way I did. First, these are 5-
min average winds I'm generating in WRF so I've coded WRF to make the
5-min average calculation as it iterates through a forecast, and the
vertical interpolation to 100m is done in WRF as well. Second, I'm
running the code in our daily NSSL WRF "real-time" forecasts which
write hourly histories, and I didn't have the freedom to modify the
WRF output frequency but I was allowed to add fields to the histories.
That's why I created 12 U/V 10m wind fields each hour and gave them
each a unique name (same for 100m).
>
> I'm very familiar with WRFPOST so I don't foresee a problem
modifying it to handle my unique fields, but if I understand your
comments that's only necessary if I decide to use MET v3.0? If I
stick with v2.0 the fields can be read directly from the existing
NetCDF files I have on hand, correct? Or do I have to use v3.0 in
order to do the verification as you've described it?
>
> (4) Verifying the 5-minute winds in MET:
>
> I'll leave this alone for now as I suspect it's a long way off and
I'll have a lot to manage before I get to that point.
>
> Thanks again!
>
> Regards,
> Scott
>
>
>
> ________________________________
> From: RAL HelpDesk {for John Halley Gotway} [met_help at ucar.edu]
> Sent: Monday, September 20, 2010 2:31 PM
> To: Dembek, Scott Robert (MSFC-VP61)[Universities Space Research
Association (USRA)]
> Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in
WRF
>
> Scott,
>
> OK, I think there are several issues to address here. I'll try to
go through them one by one...
>
> (1) Version of MET:
>
> We're currently testing the next release of MET, version 3.0, and
expect to be releasing it in the next week or so. The last big thing
to do is finish up the documentation. In METv3.0, we're able to
> read the output of the ARW-pinterp tool, which I'll discuss below.
If it'd be helpful, I'd be happy to send you a beta release of METv3.0
you could use with your data. Just let me know.
>
> (2) Format of the point observations:
>
> This should be pretty straight-forward. I'd suggest reformatting
your 5 minute point observations into ASCII, and then run them through
the ASCII2NC program. You're welcome to group them however
> you'd like - breaking them out by time, or putting them all the
times in the same NetCDF file. If they're all in the same file, you
can use command line arguments in Point-Stat to set the matching
> time window. For now, I'd suggest setting the observation message
type as MSONET.
>
> (3) Format of the gridded forecasts:
>
> Ultimately, METv3.0 is able to read gridded data in GRIB format or
the NetCDF output of the pinterp tool (for ARW). It does not read the
NetCDF output of WRF directly. So you'll need to postprocess
> your WRF output either using the WRF-PostProcessor (for GRIB,
recommended!) or pinterp (for NetCDF output, new for v3.0).
>
> However, since you've added new variables to your WRF output files,
I have no idea how WPP or pinterp will handle them. I'm guess that
WPP will NOT handle them well, and I have no idea about pinterp.
>
> Can you tell me, is there a reason you modified the WRF code rather
than just setting your history interval to be every 5-minutes
(history_interval 5)? If you did the later, you'd be able to run your
> WRF output through WPP to get GRIB files with 5-minute winds.
>
> (4) Verifying your 5-minutes winds in MET:
>
> So let's assume you have your observations as the NetCDF output of
ASCII2NC, and let's assume you have your gridded data in a format MET
can read. Next, you'd run Point-Stat once for each valid time
> you'd like to verify. On the command line, you can use the "-
obs_valid_beg" and "-obs_valid_end" to explicitly set the matching
observation time window for Point-Stat. You'd verify the 10-meter
> winds and the 100-meter winds separately. And since you probably
don't have tons of data, you could write out the MPR (matched pair)
line type from each run of Point-Stat.
>
> After you've run Point-Stat once for valid time, you could use the
STAT-Analysis tool to perform whatever types of aggregations you want.
With STAT-Analysis, you can group together the matched pairs
> from multiple valid times or levels (to combine the 10m and 100m
verification).
>
> I suspect you'll need more help to get this up and running. It
seems to me that the following steps may be difficult:
> - Getting your gridded data into a format MET can read.
> - Setting the message type and level information correctly for
Point-Stat to verify at 10-meters and 100-meters.
> - Setting up the STAT-Analysis commands to perform the types of
aggregations you want.
>
> Let me know if you get stuck, and need some help. If you do, please
send me some sample data: a gridded forecast file, ASCII point
observation file, and Point-Stat configuration file.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
>> Mon Sep 20 09:43:18 2010: Request 40984 was acted upon.
>> Transaction: Ticket created by scott.r.dembek at nasa.gov
>> Queue: met_help
>> Subject: Verifying 5-Minute Winds in WRF
>> Owner: Nobody
>> Requestors: scott.r.dembek at nasa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>>
>>
>> Greetings Met_help:
>>
>> I've added code to WRF to create 5-minute winds at both 10 meters
and 100 meters, and I'd like to do verification in MET using some
Texas Tech Reese Tower obs available to me. I've created unique 2-d
arrays for each 5-minute wind component, e.g., U10MIN05/V10MIN05,
U10MIN10/V10MIN10, U10MIN15/V10MIN15, etc. These are the U and V
component 10-meter 5-minute winds valid at 5 minutes, 10 minutes, and
15 minutes, respectively. I have similar arrays for the U and V
components at 100 meters. These arrays are output in hourly WRF
histories.
>>
>> My first question is this ... how can I get MET to verify all of
the 10-meter (and 100-meter) 5-minute winds given that they exist in
unique arrays? In other words, I would like to verify these special U
and V wind components as I would the standard U10 and V10 arrays, but
in this case I have 12 U/V component 10-meter 5-minute wind arrays in
each hourly history file. Is there some simple way to bin these
unique arrays together in MET so I get stats for "5-minute winds"?
>>
>> Similarly, how can I get MET to match these WRF fields with the
obs? I assume MET will consider each of these WRF fields to be valid
at the top-of-the-hour time given that they're written in the hourly
wrfout files, while the obs are valid at 5-minutes past the hour, 10-
minutes past the hour, etc. I will need to convert the ASCII format
Reese Tower obs to netCDF for input into MET, so I suppose I could
assign the 5-minute obs to fields that match the WRF field names
(U10MIN05/V10MIN05, U10MIN10/V10MIN10, etc), but that still leaves the
issue of verifying all 12 5-minute fields available each hour.
>>
>> Any guidance you can offer would be greatly appreciated. Many
thanks!
>>
>> Regards,
>> Scott
>>
>
------------------------------------------------
Subject: Verifying 5-Minute Winds in WRF
From: Dembek, Scott Robert[Universities Space Research Association]
Time: Mon Sep 27 13:09:33 2010
Hi John,
Just a quick follow up to say thanks, and to let you know where I
stand right now. The amount of time I've been able to devote to this
has been limited recently, but I believe I have the obs portion
working and am generating output through ASCII2NC. Soon I'll begin
working with WPP/WRFPOST to reformat the 5-min WRF winds into GRIB
files. At that point I'll probably call on you again to assist with
Point-Stat and Stat-Analysis if that's OK with you. Thanks!
Regards,
Scott
________________________________
From: RAL HelpDesk {for John Halley Gotway} [met_help at ucar.edu]
Sent: Monday, September 20, 2010 4:56 PM
To: Dembek, Scott Robert (MSFC-VP61)[Universities Space Research
Association (USRA)]
Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in WRF
Scott,
(1) Version of MET:
I think it'd be fine to use ASCII2NC from METv2.0. In METv3.0, we've
added a couple of global attributes, but I don't think Point-Stat
actually reads them. So ASCII2NC from METv2.0 should be fine.
(2) Formatting the point obs:
The ASCII point observation format to be used for as input to ASCII2NC
is pretty simple:
The only currently supported ASCII format is "met_point" in
which each row of observation data consists of 10 columns:
Message_Type Station_ID Valid_Time(YYYYMMDD_HHMMSS)
Lat(Deg North) Lon(Deg East) Elevation(msl)
Grib_Code Level Height(msl or agl) Observation_Value
where "Level" is the pressure level (hPa) or
accumulation interval (hh[_mmss]).
"Height" is meters above sea level or above
ground level for the observation (msl or agl).
Use a value of "-9999" to indicate missing
data.
You don't name any NetCDF variables yourself. Instead, for each
observation value you list out the 10 ASCII columns listed above. The
GRIB code would be 33 for U-component, 34 for V-component, or 32
for wind speed. You'd set "Level" to -9999 for NA, and you'd set
height in meters for the location of the observation. The height (AGL
or MSL) should be defined in a way that's consistent with how
your forecast data is output. You'd set the valid time for each
observation in YYYYMMDD_HHMMSS format. Once you have the ASCII
observations formatted, you'd run them through the ASCII2NC tool.
(3) Format of the gridded forecasts:
I was actually making the opposite point about METv2.0 vs METv3.0.
They both handle the GRIB output of WPP. However, in METv3.0, we've
added support for the NetCDF output of the pinterp tool (not
WRFOUT files directly). If you're able to run your WRF output through
WPP and generate GRIB output files, that'd be ideal. I don't know
exactly how you'll do that, but I'll leave that to you. No
matter what, I'd recommend using METv3.0. That way, if issues come up
and changes are required, it'll be easier to provide you with support.
Hope that helps.
John
RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>
> Hi John,
>
> Thanks for your quick reply! I'll try to tackle this one piece at a
time, and will follow up with you as things get confusing. Before I
get started...
>
> (1) Version of MET:
>
> I suspect I can start formatting the obs using ASCII2NC with MET
v2.0 which I have already installed locally. By the time I get that
worked out I can come back to the issue of v3.0 vs. v2.0. If you
think it's to my advantage to use V3.0, I'll definitely give it a try.
Or will I have to use v3.0 beginning with ASCII2NC if I want to
continue to use it later in the process?
>
> (2) Formatting the point obs:
>
> I think I understand how this works in principle, but I want to be
sure I understand your comments. Are you saying it's OK for me to
generate a single hourly NetCDF file and I could use the same names as
I used in the WRF histories ... call the 5-min obs U10MIN05/V10MIN05,
call the 10-min obs U10MIN10/V10MIN10, etc? They would all have the
same valid time if I did it that way, is that correct?
>
> (3) Format of the gridded forecasts:
>
> Two reasons for doing the winds the way I did. First, these are 5-
min average winds I'm generating in WRF so I've coded WRF to make the
5-min average calculation as it iterates through a forecast, and the
vertical interpolation to 100m is done in WRF as well. Second, I'm
running the code in our daily NSSL WRF "real-time" forecasts which
write hourly histories, and I didn't have the freedom to modify the
WRF output frequency but I was allowed to add fields to the histories.
That's why I created 12 U/V 10m wind fields each hour and gave them
each a unique name (same for 100m).
>
> I'm very familiar with WRFPOST so I don't foresee a problem
modifying it to handle my unique fields, but if I understand your
comments that's only necessary if I decide to use MET v3.0? If I
stick with v2.0 the fields can be read directly from the existing
NetCDF files I have on hand, correct? Or do I have to use v3.0 in
order to do the verification as you've described it?
>
> (4) Verifying the 5-minute winds in MET:
>
> I'll leave this alone for now as I suspect it's a long way off and
I'll have a lot to manage before I get to that point.
>
> Thanks again!
>
> Regards,
> Scott
>
>
>
> ________________________________
> From: RAL HelpDesk {for John Halley Gotway} [met_help at ucar.edu]
> Sent: Monday, September 20, 2010 2:31 PM
> To: Dembek, Scott Robert (MSFC-VP61)[Universities Space Research
Association (USRA)]
> Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in
WRF
>
> Scott,
>
> OK, I think there are several issues to address here. I'll try to
go through them one by one...
>
> (1) Version of MET:
>
> We're currently testing the next release of MET, version 3.0, and
expect to be releasing it in the next week or so. The last big thing
to do is finish up the documentation. In METv3.0, we're able to
> read the output of the ARW-pinterp tool, which I'll discuss below.
If it'd be helpful, I'd be happy to send you a beta release of METv3.0
you could use with your data. Just let me know.
>
> (2) Format of the point observations:
>
> This should be pretty straight-forward. I'd suggest reformatting
your 5 minute point observations into ASCII, and then run them through
the ASCII2NC program. You're welcome to group them however
> you'd like - breaking them out by time, or putting them all the
times in the same NetCDF file. If they're all in the same file, you
can use command line arguments in Point-Stat to set the matching
> time window. For now, I'd suggest setting the observation message
type as MSONET.
>
> (3) Format of the gridded forecasts:
>
> Ultimately, METv3.0 is able to read gridded data in GRIB format or
the NetCDF output of the pinterp tool (for ARW). It does not read the
NetCDF output of WRF directly. So you'll need to postprocess
> your WRF output either using the WRF-PostProcessor (for GRIB,
recommended!) or pinterp (for NetCDF output, new for v3.0).
>
> However, since you've added new variables to your WRF output files,
I have no idea how WPP or pinterp will handle them. I'm guess that
WPP will NOT handle them well, and I have no idea about pinterp.
>
> Can you tell me, is there a reason you modified the WRF code rather
than just setting your history interval to be every 5-minutes
(history_interval 5)? If you did the later, you'd be able to run your
> WRF output through WPP to get GRIB files with 5-minute winds.
>
> (4) Verifying your 5-minutes winds in MET:
>
> So let's assume you have your observations as the NetCDF output of
ASCII2NC, and let's assume you have your gridded data in a format MET
can read. Next, you'd run Point-Stat once for each valid time
> you'd like to verify. On the command line, you can use the "-
obs_valid_beg" and "-obs_valid_end" to explicitly set the matching
observation time window for Point-Stat. You'd verify the 10-meter
> winds and the 100-meter winds separately. And since you probably
don't have tons of data, you could write out the MPR (matched pair)
line type from each run of Point-Stat.
>
> After you've run Point-Stat once for valid time, you could use the
STAT-Analysis tool to perform whatever types of aggregations you want.
With STAT-Analysis, you can group together the matched pairs
> from multiple valid times or levels (to combine the 10m and 100m
verification).
>
> I suspect you'll need more help to get this up and running. It
seems to me that the following steps may be difficult:
> - Getting your gridded data into a format MET can read.
> - Setting the message type and level information correctly for
Point-Stat to verify at 10-meters and 100-meters.
> - Setting up the STAT-Analysis commands to perform the types of
aggregations you want.
>
> Let me know if you get stuck, and need some help. If you do, please
send me some sample data: a gridded forecast file, ASCII point
observation file, and Point-Stat configuration file.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
>> Mon Sep 20 09:43:18 2010: Request 40984 was acted upon.
>> Transaction: Ticket created by scott.r.dembek at nasa.gov
>> Queue: met_help
>> Subject: Verifying 5-Minute Winds in WRF
>> Owner: Nobody
>> Requestors: scott.r.dembek at nasa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>>
>>
>> Greetings Met_help:
>>
>> I've added code to WRF to create 5-minute winds at both 10 meters
and 100 meters, and I'd like to do verification in MET using some
Texas Tech Reese Tower obs available to me. I've created unique 2-d
arrays for each 5-minute wind component, e.g., U10MIN05/V10MIN05,
U10MIN10/V10MIN10, U10MIN15/V10MIN15, etc. These are the U and V
component 10-meter 5-minute winds valid at 5 minutes, 10 minutes, and
15 minutes, respectively. I have similar arrays for the U and V
components at 100 meters. These arrays are output in hourly WRF
histories.
>>
>> My first question is this ... how can I get MET to verify all of
the 10-meter (and 100-meter) 5-minute winds given that they exist in
unique arrays? In other words, I would like to verify these special U
and V wind components as I would the standard U10 and V10 arrays, but
in this case I have 12 U/V component 10-meter 5-minute wind arrays in
each hourly history file. Is there some simple way to bin these
unique arrays together in MET so I get stats for "5-minute winds"?
>>
>> Similarly, how can I get MET to match these WRF fields with the
obs? I assume MET will consider each of these WRF fields to be valid
at the top-of-the-hour time given that they're written in the hourly
wrfout files, while the obs are valid at 5-minutes past the hour, 10-
minutes past the hour, etc. I will need to convert the ASCII format
Reese Tower obs to netCDF for input into MET, so I suppose I could
assign the 5-minute obs to fields that match the WRF field names
(U10MIN05/V10MIN05, U10MIN10/V10MIN10, etc), but that still leaves the
issue of verifying all 12 5-minute fields available each hour.
>>
>> Any guidance you can offer would be greatly appreciated. Many
thanks!
>>
>> Regards,
>> Scott
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in WRF
From: John Halley Gotway
Time: Mon Sep 27 13:26:48 2010
Scott,
Thanks for following up. I'll go ahead and resolve this help ticket.
When/if more questions come up, just write us at met_help at ucar.edu,
and we'll be happy to help.
Thanks,
John
RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>
> Hi John,
>
> Just a quick follow up to say thanks, and to let you know where I
stand right now. The amount of time I've been able to devote to this
has been limited recently, but I believe I have the obs portion
working and am generating output through ASCII2NC. Soon I'll begin
working with WPP/WRFPOST to reformat the 5-min WRF winds into GRIB
files. At that point I'll probably call on you again to assist with
Point-Stat and Stat-Analysis if that's OK with you. Thanks!
>
> Regards,
> Scott
>
>
> ________________________________
> From: RAL HelpDesk {for John Halley Gotway} [met_help at ucar.edu]
> Sent: Monday, September 20, 2010 4:56 PM
> To: Dembek, Scott Robert (MSFC-VP61)[Universities Space Research
Association (USRA)]
> Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in
WRF
>
> Scott,
>
> (1) Version of MET:
> I think it'd be fine to use ASCII2NC from METv2.0. In METv3.0,
we've added a couple of global attributes, but I don't think Point-
Stat actually reads them. So ASCII2NC from METv2.0 should be fine.
>
> (2) Formatting the point obs:
> The ASCII point observation format to be used for as input to
ASCII2NC is pretty simple:
>
> The only currently supported ASCII format is "met_point" in
which each row of observation data consists of 10 columns:
> Message_Type Station_ID Valid_Time(YYYYMMDD_HHMMSS)
> Lat(Deg North) Lon(Deg East) Elevation(msl)
> Grib_Code Level Height(msl or agl) Observation_Value
>
> where "Level" is the pressure level (hPa) or
accumulation interval (hh[_mmss]).
> "Height" is meters above sea level or above
ground level for the observation (msl or agl).
>
> Use a value of "-9999" to indicate missing
data.
>
> You don't name any NetCDF variables yourself. Instead, for each
observation value you list out the 10 ASCII columns listed above. The
GRIB code would be 33 for U-component, 34 for V-component, or 32
> for wind speed. You'd set "Level" to -9999 for NA, and you'd set
height in meters for the location of the observation. The height (AGL
or MSL) should be defined in a way that's consistent with how
> your forecast data is output. You'd set the valid time for each
observation in YYYYMMDD_HHMMSS format. Once you have the ASCII
observations formatted, you'd run them through the ASCII2NC tool.
>
> (3) Format of the gridded forecasts:
> I was actually making the opposite point about METv2.0 vs METv3.0.
They both handle the GRIB output of WPP. However, in METv3.0, we've
added support for the NetCDF output of the pinterp tool (not
> WRFOUT files directly). If you're able to run your WRF output
through WPP and generate GRIB output files, that'd be ideal. I don't
know exactly how you'll do that, but I'll leave that to you. No
> matter what, I'd recommend using METv3.0. That way, if issues come
up and changes are required, it'll be easier to provide you with
support.
>
> Hope that helps.
>
> John
>
>
> RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>>
>> Hi John,
>>
>> Thanks for your quick reply! I'll try to tackle this one piece at
a time, and will follow up with you as things get confusing. Before I
get started...
>>
>> (1) Version of MET:
>>
>> I suspect I can start formatting the obs using ASCII2NC with MET
v2.0 which I have already installed locally. By the time I get that
worked out I can come back to the issue of v3.0 vs. v2.0. If you
think it's to my advantage to use V3.0, I'll definitely give it a try.
Or will I have to use v3.0 beginning with ASCII2NC if I want to
continue to use it later in the process?
>>
>> (2) Formatting the point obs:
>>
>> I think I understand how this works in principle, but I want to be
sure I understand your comments. Are you saying it's OK for me to
generate a single hourly NetCDF file and I could use the same names as
I used in the WRF histories ... call the 5-min obs U10MIN05/V10MIN05,
call the 10-min obs U10MIN10/V10MIN10, etc? They would all have the
same valid time if I did it that way, is that correct?
>>
>> (3) Format of the gridded forecasts:
>>
>> Two reasons for doing the winds the way I did. First, these are 5-
min average winds I'm generating in WRF so I've coded WRF to make the
5-min average calculation as it iterates through a forecast, and the
vertical interpolation to 100m is done in WRF as well. Second, I'm
running the code in our daily NSSL WRF "real-time" forecasts which
write hourly histories, and I didn't have the freedom to modify the
WRF output frequency but I was allowed to add fields to the histories.
That's why I created 12 U/V 10m wind fields each hour and gave them
each a unique name (same for 100m).
>>
>> I'm very familiar with WRFPOST so I don't foresee a problem
modifying it to handle my unique fields, but if I understand your
comments that's only necessary if I decide to use MET v3.0? If I
stick with v2.0 the fields can be read directly from the existing
NetCDF files I have on hand, correct? Or do I have to use v3.0 in
order to do the verification as you've described it?
>>
>> (4) Verifying the 5-minute winds in MET:
>>
>> I'll leave this alone for now as I suspect it's a long way off and
I'll have a lot to manage before I get to that point.
>>
>> Thanks again!
>>
>> Regards,
>> Scott
>>
>>
>>
>> ________________________________
>> From: RAL HelpDesk {for John Halley Gotway} [met_help at ucar.edu]
>> Sent: Monday, September 20, 2010 2:31 PM
>> To: Dembek, Scott Robert (MSFC-VP61)[Universities Space Research
Association (USRA)]
>> Subject: Re: [rt.rap.ucar.edu #40984] Verifying 5-Minute Winds in
WRF
>>
>> Scott,
>>
>> OK, I think there are several issues to address here. I'll try to
go through them one by one...
>>
>> (1) Version of MET:
>>
>> We're currently testing the next release of MET, version 3.0, and
expect to be releasing it in the next week or so. The last big thing
to do is finish up the documentation. In METv3.0, we're able to
>> read the output of the ARW-pinterp tool, which I'll discuss below.
If it'd be helpful, I'd be happy to send you a beta release of METv3.0
you could use with your data. Just let me know.
>>
>> (2) Format of the point observations:
>>
>> This should be pretty straight-forward. I'd suggest reformatting
your 5 minute point observations into ASCII, and then run them through
the ASCII2NC program. You're welcome to group them however
>> you'd like - breaking them out by time, or putting them all the
times in the same NetCDF file. If they're all in the same file, you
can use command line arguments in Point-Stat to set the matching
>> time window. For now, I'd suggest setting the observation message
type as MSONET.
>>
>> (3) Format of the gridded forecasts:
>>
>> Ultimately, METv3.0 is able to read gridded data in GRIB format or
the NetCDF output of the pinterp tool (for ARW). It does not read the
NetCDF output of WRF directly. So you'll need to postprocess
>> your WRF output either using the WRF-PostProcessor (for GRIB,
recommended!) or pinterp (for NetCDF output, new for v3.0).
>>
>> However, since you've added new variables to your WRF output files,
I have no idea how WPP or pinterp will handle them. I'm guess that
WPP will NOT handle them well, and I have no idea about pinterp.
>>
>> Can you tell me, is there a reason you modified the WRF code rather
than just setting your history interval to be every 5-minutes
(history_interval 5)? If you did the later, you'd be able to run your
>> WRF output through WPP to get GRIB files with 5-minute winds.
>>
>> (4) Verifying your 5-minutes winds in MET:
>>
>> So let's assume you have your observations as the NetCDF output of
ASCII2NC, and let's assume you have your gridded data in a format MET
can read. Next, you'd run Point-Stat once for each valid time
>> you'd like to verify. On the command line, you can use the "-
obs_valid_beg" and "-obs_valid_end" to explicitly set the matching
observation time window for Point-Stat. You'd verify the 10-meter
>> winds and the 100-meter winds separately. And since you probably
don't have tons of data, you could write out the MPR (matched pair)
line type from each run of Point-Stat.
>>
>> After you've run Point-Stat once for valid time, you could use the
STAT-Analysis tool to perform whatever types of aggregations you want.
With STAT-Analysis, you can group together the matched pairs
>> from multiple valid times or levels (to combine the 10m and 100m
verification).
>>
>> I suspect you'll need more help to get this up and running. It
seems to me that the following steps may be difficult:
>> - Getting your gridded data into a format MET can read.
>> - Setting the message type and level information correctly for
Point-Stat to verify at 10-meters and 100-meters.
>> - Setting up the STAT-Analysis commands to perform the types of
aggregations you want.
>>
>> Let me know if you get stuck, and need some help. If you do,
please send me some sample data: a gridded forecast file, ASCII point
observation file, and Point-Stat configuration file.
>>
>> Hope that helps.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>>
>> RAL HelpDesk {for Dembek, Scott Robert[Universities Space Research
Association]} wrote:
>>> Mon Sep 20 09:43:18 2010: Request 40984 was acted upon.
>>> Transaction: Ticket created by scott.r.dembek at nasa.gov
>>> Queue: met_help
>>> Subject: Verifying 5-Minute Winds in WRF
>>> Owner: Nobody
>>> Requestors: scott.r.dembek at nasa.gov
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40984 >
>>>
>>>
>>> Greetings Met_help:
>>>
>>> I've added code to WRF to create 5-minute winds at both 10 meters
and 100 meters, and I'd like to do verification in MET using some
Texas Tech Reese Tower obs available to me. I've created unique 2-d
arrays for each 5-minute wind component, e.g., U10MIN05/V10MIN05,
U10MIN10/V10MIN10, U10MIN15/V10MIN15, etc. These are the U and V
component 10-meter 5-minute winds valid at 5 minutes, 10 minutes, and
15 minutes, respectively. I have similar arrays for the U and V
components at 100 meters. These arrays are output in hourly WRF
histories.
>>>
>>> My first question is this ... how can I get MET to verify all of
the 10-meter (and 100-meter) 5-minute winds given that they exist in
unique arrays? In other words, I would like to verify these special U
and V wind components as I would the standard U10 and V10 arrays, but
in this case I have 12 U/V component 10-meter 5-minute wind arrays in
each hourly history file. Is there some simple way to bin these
unique arrays together in MET so I get stats for "5-minute winds"?
>>>
>>> Similarly, how can I get MET to match these WRF fields with the
obs? I assume MET will consider each of these WRF fields to be valid
at the top-of-the-hour time given that they're written in the hourly
wrfout files, while the obs are valid at 5-minutes past the hour, 10-
minutes past the hour, etc. I will need to convert the ASCII format
Reese Tower obs to netCDF for input into MET, so I suppose I could
assign the 5-minute obs to fields that match the WRF field names
(U10MIN05/V10MIN05, U10MIN10/V10MIN10, etc), but that still leaves the
issue of verifying all 12 5-minute fields available each hour.
>>>
>>> Any guidance you can offer would be greatly appreciated. Many
thanks!
>>>
>>> Regards,
>>> Scott
>>>
>
------------------------------------------------
More information about the Met_help
mailing list