[Met_help] [rt.rap.ucar.edu #48043] History for formatting ascii2nc
RAL HelpDesk {for Paul Oldenburg}
met_help at ucar.edu
Fri Jul 22 13:50:37 MDT 2011
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Could you clarify "8. Level as the pressure level in hPa or accumulation
interval in hours" when formatting point observations. My point observation is
an anemometer at a fixed height. What value would I put for this column?
Thanks,
David Bryan
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: Paul Oldenburg
Time: Tue Jul 05 09:37:51 2011
David,
This column does not apply to your data, since you are only measuring
wind speed at a fixed height above the ground. I
would recommend putting -9999 in that column, since that is the value
that MET uses for bad data or N/A.
You should use column 6 to enter the elevation of your sensor, but be
advised that if you use a "surface" message type
(ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless of
what you enter in column 6. If you want MET to
interpolate model data to the height of your sensor, use another
message type. I hope this is clear. Please let me
know if you have any questions.
Thanks,
Paul
On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>
> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
> Transaction: Ticket created by davidstephenbryan at yahoo.com
> Queue: met_help
> Subject: formatting ascii2nc
> Owner: Nobody
> Requestors: davidstephenbryan at yahoo.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
>
> Could you clarify "8. Level as the pressure level in hPa or
accumulation
> interval in hours" when formatting point observations. My point
observation is
> an anemometer at a fixed height. What value would I put for this
column?
>
> Thanks,
> David Bryan
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: David Bryan
Time: Tue Jul 05 15:24:56 2011
Thanks Paul,
That's very helpful.
I do want to interpolate to the height of the sensor, so could you
recommend a
message type? Frankly, I'm not familiar with these message types, so
could you
refer me to a link or reference to these message types?
Also, I had assumed column 6 would get the elevation of the terrain at
that
point and column nine would be the sensor's height about ground. But
it sounds
like column 6 should be the sensor's total height above sea level,
right? If
so, what should column 9 have?
Thanks again,
David
----- Original Message ----
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Tue, July 5, 2011 1:07:52 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
This column does not apply to your data, since you are only measuring
wind speed
at a fixed height above the ground. I
would recommend putting -9999 in that column, since that is the value
that MET
uses for bad data or N/A.
You should use column 6 to enter the elevation of your sensor, but be
advised
that if you use a "surface" message type
(ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless of
what you
enter in column 6. If you want MET to
interpolate model data to the height of your sensor, use another
message type.
I hope this is clear. Please let me
know if you have any questions.
Thanks,
Paul
On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>
> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
> Transaction: Ticket created by davidstephenbryan at yahoo.com
> Queue: met_help
> Subject: formatting ascii2nc
> Owner: Nobody
> Requestors: davidstephenbryan at yahoo.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
>
> Could you clarify "8. Level as the pressure level in hPa or
accumulation
> interval in hours" when formatting point observations. My point
observation
>is
>
> an anemometer at a fixed height. What value would I put for this
column?
>
> Thanks,
> David Bryan
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: David Bryan
Time: Wed Jul 06 04:04:06 2011
Paul,
One other question. The netcdf file I'll be creating with ascii2nc
covers six
months. So I'll have one six-month file for observations. I want to
use
Point-Stat to compare that file to a series of 48-hour forecasts. My
question--will Point-Stat match the observation time to the model time
and
calculate its statistics from that? In other words, Point-Stat's
output for
each forecast is not based on periods for which there is no forecast
data,
correct?
Thanks,
David
----- Original Message ----
From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Tue, July 5, 2011 6:54:56 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
Thanks Paul,
That's very helpful.
I do want to interpolate to the height of the
sensor, so could you recommend a
message type? Frankly, I'm not familiar with these message types, so
could you
refer me to a link or reference to these message types?
Also, I had assumed column 6 would get the elevation of the terrain at
that
point and column nine would be the sensor's height about ground. But
it sounds
like column 6 should be the sensor's total height above sea level,
right? If
so, what should column 9 have?
Thanks again,
David
----- Original Message ----
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Tue, July 5, 2011 1:07:52 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
This column does not apply to your data, since you are only measuring
wind speed
at a fixed height above the ground. I
would recommend putting -9999 in that column, since that is the value
that MET
uses for bad data or N/A.
You should use column 6 to enter the elevation of your sensor, but be
advised
that if you use a "surface" message type
(ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless of
what you
enter in column 6. If you want MET to
interpolate model data to the height of your sensor, use another
message type.
I hope this is clear. Please let me
know if you have any questions.
Thanks,
Paul
On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>
> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
> Transaction: Ticket created by davidstephenbryan at yahoo.com
> Queue: met_help
> Subject: formatting ascii2nc
> Owner: Nobody
> Requestors: davidstephenbryan at yahoo.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
>
> Could you clarify "8. Level as the pressure level in hPa or
accumulation
> interval in hours" when formatting point observations. My point
observation
>is
>
> an anemometer at a fixed height. What value would I put for this
column?
>
> Thanks,
> David Bryan
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: Paul Oldenburg
Time: Wed Jul 06 08:55:50 2011
David,
> I do want to interpolate to the height of the sensor, so could you
recommend a
> message type? Frankly, I'm not familiar with these message types,
so could you
> refer me to a link or reference to these message types?
I think PROFLR is the appropriate message type for your situation.
Here is a reference on the message types:
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
Message types are a concept that is associated with PrepBUFR point
observations produced by NCEP. These observations
are often used in MET. The logic that MET uses for these message
types are applied to all observations, regardless of
source.
> Also, I had assumed column 6 would get the elevation of the terrain
at that
> point and column nine would be the sensor's height about ground.
But it sounds
> like column 6 should be the sensor's total height above sea level,
right? If
> so, what should column 9 have?
No, you are correct. I should have been more clear in my previous
email.
> will Point-Stat match the observation time to the model time and
> calculate its statistics from that?
Point-Stat was designed to be run once for each forecast valid time.
If your model data files contain single 48 hour
forecasts, run Point-Stat once for each model file. Each time, you
will pass the model data file you want to verify,
the single observation file and the configuration file. In the
configuration file, you should decide the time window
inside which observations "match" the model valid time. The settings
for this are beg_ds and end_ds. You could also
use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
If you have multiple forecasts in a single model data file, you will
need to use the command line parameter fcst_valid
time to specify which forecast Point-Stat should verify. In this
case, Point-Stat must still be run once for each valid
time, where each call differs only by the value passed to the
fcst_valid parameter.
Does that make sense? Please let me know if you have questions.
Thanks,
Paul
On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> One other question. The netcdf file I'll be creating with ascii2nc
covers six
> months. So I'll have one six-month file for observations. I want
to use
> Point-Stat to compare that file to a series of 48-hour forecasts.
My
> question--will Point-Stat match the observation time to the model
time and
> calculate its statistics from that? In other words, Point-Stat's
output for
> each forecast is not based on periods for which there is no forecast
data,
> correct?
>
> Thanks,
> David
>
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tue, July 5, 2011 6:54:56 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> Thanks Paul,
>
> That's very helpful.
>
> I do want to interpolate to the height of the sensor, so could you
recommend a
> message type? Frankly, I'm not familiar with these message types,
so could you
> refer me to a link or reference to these message types?
>
> Also, I had assumed column 6 would get the elevation of the terrain
at that
> point and column nine would be the sensor's height about ground.
But it sounds
> like column 6 should be the sensor's total height above sea level,
right? If
> so, what should column 9 have?
>
> Thanks again,
> David
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tue, July 5, 2011 1:07:52 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> This column does not apply to your data, since you are only
measuring wind speed
>
> at a fixed height above the ground. I
> would recommend putting -9999 in that column, since that is the
value that MET
> uses for bad data or N/A.
>
> You should use column 6 to enter the elevation of your sensor, but
be advised
> that if you use a "surface" message type
> (ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless
of what you
> enter in column 6. If you want MET to
> interpolate model data to the height of your sensor, use another
message type.
> I hope this is clear. Please let me
> know if you have any questions.
>
> Thanks,
>
> Paul
>
>
> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>
>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>> Queue: met_help
>> Subject: formatting ascii2nc
>> Owner: Nobody
>> Requestors: davidstephenbryan at yahoo.com
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>>
>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>> interval in hours" when formatting point observations. My point
observation
>> is
>>
>> an anemometer at a fixed height. What value would I put for this
column?
>>
>> Thanks,
>> David Bryan
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: David Bryan
Time: Wed Jul 06 15:09:49 2011
Paul,
Thanks again for your help. Now, I'm on to creating the config file
for Point
Stat.
My model output is for 48-hour runs of WRF with hourly output. The
output is
concatenated into a single file, 48 time steps in a single file.
(Perhaps
that's not practical.)
My observations are for two met towers, each with two anemometers at
different
heights. When I set up ASCII2NC, I broke each set of anemometer data
into its
own netcdf file, each to serve as its own point source.
So when running PointStat, here's my understanding:
PointStat will only analyze one model time per run.
In the command line arguments, I will have to specify -fcst_valid.
For
instance, if my WRF run is initiated with NAM data produced on the
20110625_1200
(and my model is initiated on the same time), that's my valid time.
I will also have to specify -fcst_lead, which indicates where
(temporally) in
the model's forecast I want to analyze. So with that run I mentioned,
if I
wanted to analyze five hours into my 48-hour run, my lead time would
be
20110625_1700.
The purpose of specifying a temporal range of observation values is
for
interpolation across the temporal dimension.
The model time that's compared to observation time is the lead time.
If any of my understanding of how time specification in Point Stat
works is
incorrect, please let me know.
My other main question concerns specifying the levels for each
fcst_field. The
most attractive to me is the ZNNN for a straightforward vertical
level. Is that
numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be more
than three
digits? (My MSL+AGL is ~1700m.) Also the provided default config
file includes
the following comment: "GC/ZNNN-NNN for a range of vertical levels
(MSL or
AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
eta-levels which is more related to pressure levels. Does that
matter?
Also, I assume that the forecast and observation thresholds serve as a
quality
control.
Thanks again for all your assistance!
David
----- Original Message ----
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Wed, July 6, 2011 12:25:51 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
> I do want to interpolate to the height of the sensor, so could you
recommend a
> message type? Frankly, I'm not familiar with these message types,
so could
you
> refer me to a link or reference to these message types?
I think PROFLR is the appropriate message type for your situation.
Here is a
reference on the message types:
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
Message types are a concept that is associated with PrepBUFR point
observations
produced by NCEP. These observations
are often used in MET. The logic that MET uses for these message
types are
applied to all observations, regardless of
source.
> Also, I had assumed column 6 would get the elevation of the terrain
at that
> point and column nine would be the sensor's height about ground.
But it
sounds
> like column 6 should be the sensor's total height above sea level,
right? If
> so, what should column 9 have?
No, you are correct. I should have been more clear in my previous
email.
> will Point-Stat match the observation time to the model time and
> calculate its statistics from that?
Point-Stat was designed to be run once for each forecast valid time.
If your
model data files contain single 48 hour
forecasts, run Point-Stat once for each model file. Each time, you
will pass
the model data file you want to verify,
the single observation file and the configuration file. In the
configuration
file, you should decide the time window
inside which observations "match" the model valid time. The settings
for this
are beg_ds and end_ds. You could also
use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
If you have multiple forecasts in a single model data file, you will
need to use
the command line parameter fcst_valid
time to specify which forecast Point-Stat should verify. In this
case,
Point-Stat must still be run once for each valid
time, where each call differs only by the value passed to the
fcst_valid
parameter.
Does that make sense? Please let me know if you have questions.
Thanks,
Paul
On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> One other question. The netcdf file I'll be creating with ascii2nc
covers six
> months. So I'll have one six-month file for observations. I want
to use
> Point-Stat to compare that file to a series of 48-hour forecasts.
My
> question--will Point-Stat match the observation time to the model
time and
> calculate its statistics from that? In other words, Point-Stat's
output for
> each forecast is not based on periods for which there is no forecast
data,
> correct?
>
> Thanks,
> David
>
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tue, July 5, 2011 6:54:56 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> Thanks Paul,
>
> That's very helpful.
>
> I do want to interpolate to the height of the sensor, so could you
recommend a
> message type? Frankly, I'm not familiar with these message types,
so could you
>
> refer me to a link or reference to these message types?
>
> Also, I had assumed column 6 would get the elevation of the terrain
at that
> point and column nine would be the sensor's height about ground.
But it sounds
>
> like column 6 should be the sensor's total height above sea level,
right? If
> so, what should column 9 have?
>
> Thanks again,
> David
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tue, July 5, 2011 1:07:52 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> This column does not apply to your data, since you are only
measuring wind
>speed
>
>
> at a fixed height above the ground. I
> would recommend putting -9999 in that column, since that is the
value that MET
> uses for bad data or N/A.
>
> You should use column 6 to enter the elevation of your sensor, but
be advised
> that if you use a "surface" message type
> (ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless
of what you
>
> enter in column 6. If you want MET to
> interpolate model data to the height of your sensor, use another
message type.
>
> I hope this is clear. Please let me
> know if you have any questions.
>
> Thanks,
>
> Paul
>
>
> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>
>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>> Queue: met_help
>> Subject: formatting ascii2nc
>> Owner: Nobody
>> Requestors: davidstephenbryan at yahoo.com
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>>
>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>> interval in hours" when formatting point observations. My point
observation
>> is
>>
>> an anemometer at a fixed height. What value would I put for this
column?
>>
>> Thanks,
>> David Bryan
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: Paul Oldenburg
Time: Thu Jul 07 09:55:06 2011
David,
Regarding your observations, it might be easiest to include all of
your observations into a single NetCDF file. You can
denote the two different sensor locations using Station_ID (column 2)
in ascii2nc. Then, you can use the mask_sid
option in the point_stat configuration file to specify which stations
obs you want to use. Otherwise, you can use both
of your current NetCDF obs files in a single point_stat call by
passing one as the "obs_file" and the other one using
the -point_obs optional argument.
Regarding the forecast times, the following is from the point_stat
usage statement, which you can see by running
point_stat with no arguments:
"-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the forecast
valid time to be verified (optional).
"-fcst_lead time" in HH[MMSS] format sets the forecast lead time to
be verified (optional).
If your model is initiated (we call it initialized) at 20110625_1200,
then that is your fcst_init time, not your
fcst_valid time. If you are interested in verifying the forecast five
hours into your 48-hour run, then your fcst_lead
is 05 and your fcst_valid is 20110625_1700. Is that clear?
Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long. Typically, for wind ground-based
measurements, the fcst_field level looks something like "Z10" for a
sensor at 10m above the ground. Unfortunately, MET
does not read model data on eta levels. In order to feed your WRF
data to MET, it must be post-processed using either
the WRF Post Processor tool (WPP) or p_interp. This process will put
the data onto levels in meters and/or pressure,
both of which MET can read.
Please let me know if you have any more questions.
Paul
On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> Thanks again for your help. Now, I'm on to creating the config file
for Point
> Stat.
>
> My model output is for 48-hour runs of WRF with hourly output. The
output is
> concatenated into a single file, 48 time steps in a single file.
(Perhaps
> that's not practical.)
>
> My observations are for two met towers, each with two anemometers at
different
> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
> own netcdf file, each to serve as its own point source.
>
>
> So when running PointStat, here's my understanding:
>
> PointStat will only analyze one model time per run.
> In the command line arguments, I will have to specify -fcst_valid.
For
> instance, if my WRF run is initiated with NAM data produced on the
20110625_1200
> (and my model is initiated on the same time), that's my valid time.
> I will also have to specify -fcst_lead, which indicates where
(temporally) in
> the model's forecast I want to analyze. So with that run I
mentioned, if I
> wanted to analyze five hours into my 48-hour run, my lead time would
be
> 20110625_1700.
> The purpose of specifying a temporal range of observation values is
for
> interpolation across the temporal dimension.
> The model time that's compared to observation time is the lead time.
> If any of my understanding of how time specification in Point Stat
works is
> incorrect, please let me know.
>
> My other main question concerns specifying the levels for each
fcst_field. The
> most attractive to me is the ZNNN for a straightforward vertical
level. Is that
> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than three
> digits? (My MSL+AGL is ~1700m.) Also the provided default config
file includes
> the following comment: "GC/ZNNN-NNN for a range of vertical levels
(MSL or
> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
> eta-levels which is more related to pressure levels. Does that
matter?
>
> Also, I assume that the forecast and observation thresholds serve as
a quality
> control.
>
> Thanks again for all your assistance!
>
> David
>
>
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Wed, July 6, 2011 12:25:51 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
>> I do want to interpolate to the height of the sensor, so could you
recommend a
>> message type? Frankly, I'm not familiar with these message types,
so could
> you
>> refer me to a link or reference to these message types?
>
> I think PROFLR is the appropriate message type for your situation.
Here is a
> reference on the message types:
>
>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>
> Message types are a concept that is associated with PrepBUFR point
observations
> produced by NCEP. These observations
> are often used in MET. The logic that MET uses for these message
types are
> applied to all observations, regardless of
> source.
>
>
>> Also, I had assumed column 6 would get the elevation of the terrain
at that
>> point and column nine would be the sensor's height about ground.
But it
> sounds
>> like column 6 should be the sensor's total height above sea level,
right? If
>> so, what should column 9 have?
>
> No, you are correct. I should have been more clear in my previous
email.
>
>
>> will Point-Stat match the observation time to the model time and
>> calculate its statistics from that?
>
> Point-Stat was designed to be run once for each forecast valid time.
If your
> model data files contain single 48 hour
> forecasts, run Point-Stat once for each model file. Each time, you
will pass
> the model data file you want to verify,
> the single observation file and the configuration file. In the
configuration
> file, you should decide the time window
> inside which observations "match" the model valid time. The
settings for this
> are beg_ds and end_ds. You could also
> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>
> If you have multiple forecasts in a single model data file, you will
need to use
> the command line parameter fcst_valid
> time to specify which forecast Point-Stat should verify. In this
case,
> Point-Stat must still be run once for each valid
> time, where each call differs only by the value passed to the
fcst_valid
> parameter.
>
> Does that make sense? Please let me know if you have questions.
>
> Thanks,
>
> Paul
>
>
> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> One other question. The netcdf file I'll be creating with ascii2nc
covers six
>
>> months. So I'll have one six-month file for observations. I want
to use
>> Point-Stat to compare that file to a series of 48-hour forecasts.
My
>> question--will Point-Stat match the observation time to the model
time and
>> calculate its statistics from that? In other words, Point-Stat's
output for
>> each forecast is not based on periods for which there is no
forecast data,
>> correct?
>>
>> Thanks,
>> David
>>
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tue, July 5, 2011 6:54:56 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> Thanks Paul,
>>
>> That's very helpful.
>>
>> I do want to interpolate to the height of the sensor, so could you
recommend a
>
>> message type? Frankly, I'm not familiar with these message types,
so could you
>>
>> refer me to a link or reference to these message types?
>>
>> Also, I had assumed column 6 would get the elevation of the terrain
at that
>> point and column nine would be the sensor's height about ground.
But it sounds
>>
>> like column 6 should be the sensor's total height above sea level,
right? If
>> so, what should column 9 have?
>>
>> Thanks again,
>> David
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tue, July 5, 2011 1:07:52 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> This column does not apply to your data, since you are only
measuring wind
>> speed
>>
>>
>> at a fixed height above the ground. I
>> would recommend putting -9999 in that column, since that is the
value that MET
>
>> uses for bad data or N/A.
>>
>> You should use column 6 to enter the elevation of your sensor, but
be advised
>> that if you use a "surface" message type
>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless
of what you
>>
>> enter in column 6. If you want MET to
>> interpolate model data to the height of your sensor, use another
message type.
>>
>> I hope this is clear. Please let me
>> know if you have any questions.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>> Queue: met_help
>>> Subject: formatting ascii2nc
>>> Owner: Nobody
>>> Requestors: davidstephenbryan at yahoo.com
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>>
>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>> interval in hours" when formatting point observations. My point
observation
>
>>> is
>>>
>>> an anemometer at a fixed height. What value would I put for this
column?
>>>
>>> Thanks,
>>> David Bryan
>>>
>>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Thu Jul 07 12:20:14 2011
Paul,
I've attached my config file.
When I tried running Point Stat, I got the error message below. I'm
not clear
if it's coming from the config file or the grib file. I will point
out, though,
that the grib files are produced by the WRF post processor (though
they are
concatenated, using Linux cat). The text in the message isn't in the
config
file.
Any suggestions would be welcomed.
Thanks,
David
[amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
pment/David_Bryan/ASCII2NC/PointStat/ -v 9
config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
grib"
line = 1
econfig_column = 7
text = "^"
GRIB
____
----- Original Message ----
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Thu, July 7, 2011 1:25:07 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
Regarding your observations, it might be easiest to include all of
your
observations into a single NetCDF file. You can
denote the two different sensor locations using Station_ID (column 2)
in
ascii2nc. Then, you can use the mask_sid
option in the point_stat configuration file to specify which stations
obs you
want to use. Otherwise, you can use both
of your current NetCDF obs files in a single point_stat call by
passing one as
the "obs_file" and the other one using
the -point_obs optional argument.
Regarding the forecast times, the following is from the point_stat
usage
statement, which you can see by running
point_stat with no arguments:
"-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the forecast
valid time
to be verified (optional).
"-fcst_lead time" in HH[MMSS] format sets the forecast lead time to
be
verified (optional).
If your model is initiated (we call it initialized) at 20110625_1200,
then that
is your fcst_init time, not your
fcst_valid time. If you are interested in verifying the forecast five
hours
into your 48-hour run, then your fcst_lead
is 05 and your fcst_valid is 20110625_1700. Is that clear?
Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long.
Typically, for wind ground-based
measurements, the fcst_field level looks something like "Z10" for a
sensor at
10m above the ground. Unfortunately, MET
does not read model data on eta levels. In order to feed your WRF
data to MET,
it must be post-processed using either
the WRF Post Processor tool (WPP) or p_interp. This process will put
the data
onto levels in meters and/or pressure,
both of which MET can read.
Please let me know if you have any more questions.
Paul
On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> Thanks again for your help. Now, I'm on to creating the config file
for Point
> Stat.
>
> My model output is for 48-hour runs of WRF with hourly output. The
output is
> concatenated into a single file, 48 time steps in a single file.
(Perhaps
> that's not practical.)
>
> My observations are for two met towers, each with two anemometers at
different
> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
> own netcdf file, each to serve as its own point source.
>
>
> So when running PointStat, here's my understanding:
>
> PointStat will only analyze one model time per run.
> In the command line arguments, I will have to specify -fcst_valid.
For
> instance, if my WRF run is initiated with NAM data produced on the
>20110625_1200
>
> (and my model is initiated on the same time), that's my valid time.
> I will also have to specify -fcst_lead, which indicates where
(temporally) in
> the model's forecast I want to analyze. So with that run I
mentioned, if I
> wanted to analyze five hours into my 48-hour run, my lead time would
be
> 20110625_1700.
> The purpose of specifying a temporal range of observation values is
for
> interpolation across the temporal dimension.
> The model time that's compared to observation time is the lead time.
> If any of my understanding of how time specification in Point Stat
works is
> incorrect, please let me know.
>
> My other main question concerns specifying the levels for each
fcst_field. The
>
> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>that
>
> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>three
>
> digits? (My MSL+AGL is ~1700m.) Also the provided default config
file
>includes
>
> the following comment: "GC/ZNNN-NNN for a range of vertical levels
(MSL or
> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
> eta-levels which is more related to pressure levels. Does that
matter?
>
> Also, I assume that the forecast and observation thresholds serve as
a quality
> control.
>
> Thanks again for all your assistance!
>
> David
>
>
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Wed, July 6, 2011 12:25:51 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
>> I do want to interpolate to the height of the sensor, so could you
recommend
a
>> message type? Frankly, I'm not familiar with these message types,
so could
> you
>> refer me to a link or reference to these message types?
>
> I think PROFLR is the appropriate message type for your situation.
Here is a
> reference on the message types:
>
>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>
> Message types are a concept that is associated with PrepBUFR point
observations
>
> produced by NCEP. These observations
> are often used in MET. The logic that MET uses for these message
types are
> applied to all observations, regardless of
> source.
>
>
>> Also, I had assumed column 6 would get the elevation of the terrain
at that
>> point and column nine would be the sensor's height about ground.
But it
> sounds
>> like column 6 should be the sensor's total height above sea level,
right? If
>> so, what should column 9 have?
>
> No, you are correct. I should have been more clear in my previous
email.
>
>
>> will Point-Stat match the observation time to the model time and
>> calculate its statistics from that?
>
> Point-Stat was designed to be run once for each forecast valid time.
If your
> model data files contain single 48 hour
> forecasts, run Point-Stat once for each model file. Each time, you
will pass
> the model data file you want to verify,
> the single observation file and the configuration file. In the
configuration
> file, you should decide the time window
> inside which observations "match" the model valid time. The
settings for this
> are beg_ds and end_ds. You could also
> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>
> If you have multiple forecasts in a single model data file, you will
need to
>use
>
> the command line parameter fcst_valid
> time to specify which forecast Point-Stat should verify. In this
case,
> Point-Stat must still be run once for each valid
> time, where each call differs only by the value passed to the
fcst_valid
> parameter.
>
> Does that make sense? Please let me know if you have questions.
>
> Thanks,
>
> Paul
>
>
> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> One other question. The netcdf file I'll be creating with ascii2nc
covers six
>
>
>> months. So I'll have one six-month file for observations. I want
to use
>> Point-Stat to compare that file to a series of 48-hour forecasts.
My
>> question--will Point-Stat match the observation time to the model
time and
>> calculate its statistics from that? In other words, Point-Stat's
output for
>> each forecast is not based on periods for which there is no
forecast data,
>> correct?
>>
>> Thanks,
>> David
>>
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tue, July 5, 2011 6:54:56 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> Thanks Paul,
>>
>> That's very helpful.
>>
>> I do want to interpolate to the height of the sensor, so could you
recommend a
>
>
>> message type? Frankly, I'm not familiar with these message types,
so could you
>>
>>
>> refer me to a link or reference to these message types?
>>
>> Also, I had assumed column 6 would get the elevation of the terrain
at that
>> point and column nine would be the sensor's height about ground.
But it sounds
>>
>>
>> like column 6 should be the sensor's total height above sea level,
right? If
>> so, what should column 9 have?
>>
>> Thanks again,
>> David
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tue, July 5, 2011 1:07:52 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> This column does not apply to your data, since you are only
measuring wind
>> speed
>>
>>
>> at a fixed height above the ground. I
>> would recommend putting -9999 in that column, since that is the
value that MET
>
>
>> uses for bad data or N/A.
>>
>> You should use column 6 to enter the elevation of your sensor, but
be advised
>> that if you use a "surface" message type
>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m regardless
of what you
>>
>>
>> enter in column 6. If you want MET to
>> interpolate model data to the height of your sensor, use another
message type.
>>
>>
>> I hope this is clear. Please let me
>> know if you have any questions.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>> Queue: met_help
>>> Subject: formatting ascii2nc
>>> Owner: Nobody
>>> Requestors: davidstephenbryan at yahoo.com
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>>
>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>> interval in hours" when formatting point observations. My point
observation
>
>
>>> is
>>>
>>> an anemometer at a fixed height. What value would I put for this
column?
>>>
>>> Thanks,
>>> David Bryan
>>>
>>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Thu Jul 07 12:20:14 2011
model = "WRF";
beg_ds = -600;
end_ds = 600;
fcst_field[] = [
"PRES/Z3","TMP/Z3","WIND/Z50","WDIR/Z50","WIND/Z80","WDIR/Z80" ];
obs_field[] = [];
fcst_thresh[] = [ "ge87000 le108330", "ge233.15
le322.15", "ge0 le141.33", "ge0 le360", "ge233.15 le322.15", "ge0
le141.33", "ge0 le360" ];
obs_thresh[] = [];
fcst_wind_thresh[]
= [ "NA" ];
obs_wind_thresh[] = [];
message_type[] = [ "PROFLR"
];
mask_grid[] = [ "FULL" ];
mask_poly[] = [];
mask_sid = "";
ci_alpha[] = [ 0.05 ];
boot_interval = 1;
boot_rep_prop = 1.0;
n_boot_rep = 1000;
boot_rng = "mt19937";
boot_seed = "";
interp_method[] = [ "UW_MEAN" ];
interp_width[] = [ 1 ];
interp_thresh = 1.0;
output_flag[] = [ 2, 2, 2, 0, 0, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2 ];
rank_corr_flag = 1;
grib_ptv = 2;
tmp_dir
= "/home/WRF/Development/David_Bryan/ASCII2NC/tmp/";
output_prefix
= "";
version = "V3.0";
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: Paul Oldenburg
Time: Thu Jul 07 12:24:37 2011
David,
The point_stat usage statement indicates that the arguments should be
specified in the following order. Please try this
(you should be able just copy/paste):
./point_stat \
/ext/models/WRF/NREL/10061712/NREL10061712.grib \
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
-fcst_valid 20100617_17 \
-outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
-v 9
I think that may be the cause of your problem.
Paul
On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> I've attached my config file.
>
> When I tried running Point Stat, I got the error message below. I'm
not clear
> if it's coming from the config file or the grib file. I will point
out, though,
> that the grib files are produced by the WRF post processor (though
they are
> concatenated, using Linux cat). The text in the message isn't in
the config
> file.
>
> Any suggestions would be welcomed.
>
> Thanks,
> David
>
>
>
> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>
>
> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
> grib"
>
> line = 1
>
> econfig_column = 7
>
> text = "^"
>
>
>
> GRIB
> ____
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Thu, July 7, 2011 1:25:07 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> Regarding your observations, it might be easiest to include all of
your
> observations into a single NetCDF file. You can
> denote the two different sensor locations using Station_ID (column
2) in
> ascii2nc. Then, you can use the mask_sid
> option in the point_stat configuration file to specify which
stations obs you
> want to use. Otherwise, you can use both
> of your current NetCDF obs files in a single point_stat call by
passing one as
> the "obs_file" and the other one using
> the -point_obs optional argument.
>
> Regarding the forecast times, the following is from the point_stat
usage
> statement, which you can see by running
> point_stat with no arguments:
>
> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the forecast
valid time
> to be verified (optional).
> "-fcst_lead time" in HH[MMSS] format sets the forecast lead time
to be
> verified (optional).
>
> If your model is initiated (we call it initialized) at
20110625_1200, then that
> is your fcst_init time, not your
> fcst_valid time. If you are interested in verifying the forecast
five hours
> into your 48-hour run, then your fcst_lead
> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>
> Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long.
> Typically, for wind ground-based
> measurements, the fcst_field level looks something like "Z10" for a
sensor at
> 10m above the ground. Unfortunately, MET
> does not read model data on eta levels. In order to feed your WRF
data to MET,
> it must be post-processed using either
> the WRF Post Processor tool (WPP) or p_interp. This process will
put the data
> onto levels in meters and/or pressure,
> both of which MET can read.
>
> Please let me know if you have any more questions.
>
> Paul
>
>
> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>
>> Stat.
>>
>> My model output is for 48-hour runs of WRF with hourly output. The
output is
>> concatenated into a single file, 48 time steps in a single file.
(Perhaps
>> that's not practical.)
>>
>> My observations are for two met towers, each with two anemometers
at different
>
>> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
>
>> own netcdf file, each to serve as its own point source.
>>
>>
>> So when running PointStat, here's my understanding:
>>
>> PointStat will only analyze one model time per run.
>> In the command line arguments, I will have to specify -fcst_valid.
For
>> instance, if my WRF run is initiated with NAM data produced on the
>> 20110625_1200
>>
>> (and my model is initiated on the same time), that's my valid time.
>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>> 20110625_1700.
>> The purpose of specifying a temporal range of observation values is
for
>> interpolation across the temporal dimension.
>> The model time that's compared to observation time is the lead
time.
>> If any of my understanding of how time specification in Point Stat
works is
>> incorrect, please let me know.
>>
>> My other main question concerns specifying the levels for each
fcst_field. The
>>
>> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>> that
>>
>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>> three
>>
>> digits? (My MSL+AGL is ~1700m.) Also the provided default config
file
>> includes
>>
>> the following comment: "GC/ZNNN-NNN for a range of vertical levels
(MSL or
>> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
>
>> eta-levels which is more related to pressure levels. Does that
matter?
>>
>> Also, I assume that the forecast and observation thresholds serve
as a quality
>
>> control.
>>
>> Thanks again for all your assistance!
>>
>> David
>>
>>
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Wed, July 6, 2011 12:25:51 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>>> I do want to interpolate to the height of the sensor, so could you
recommend
> a
>>> message type? Frankly, I'm not familiar with these message types,
so could
>> you
>>> refer me to a link or reference to these message types?
>>
>> I think PROFLR is the appropriate message type for your situation.
Here is a
>> reference on the message types:
>>
>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>
>> Message types are a concept that is associated with PrepBUFR point
observations
>>
>> produced by NCEP. These observations
>> are often used in MET. The logic that MET uses for these message
types are
>> applied to all observations, regardless of
>> source.
>>
>>
>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>> point and column nine would be the sensor's height about ground.
But it
>> sounds
>>> like column 6 should be the sensor's total height above sea level,
right? If
>>> so, what should column 9 have?
>>
>> No, you are correct. I should have been more clear in my previous
email.
>>
>>
>>> will Point-Stat match the observation time to the model time and
>>> calculate its statistics from that?
>>
>> Point-Stat was designed to be run once for each forecast valid
time. If your
>> model data files contain single 48 hour
>> forecasts, run Point-Stat once for each model file. Each time, you
will pass
>> the model data file you want to verify,
>> the single observation file and the configuration file. In the
configuration
>> file, you should decide the time window
>> inside which observations "match" the model valid time. The
settings for this
>
>> are beg_ds and end_ds. You could also
>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>
>> If you have multiple forecasts in a single model data file, you
will need to
>> use
>>
>> the command line parameter fcst_valid
>> time to specify which forecast Point-Stat should verify. In this
case,
>> Point-Stat must still be run once for each valid
>> time, where each call differs only by the value passed to the
fcst_valid
>> parameter.
>>
>> Does that make sense? Please let me know if you have questions.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>
>>
>>> months. So I'll have one six-month file for observations. I want
to use
>>> Point-Stat to compare that file to a series of 48-hour forecasts.
My
>>> question--will Point-Stat match the observation time to the model
time and
>>> calculate its statistics from that? In other words, Point-Stat's
output for
>>> each forecast is not based on periods for which there is no
forecast data,
>>> correct?
>>>
>>> Thanks,
>>> David
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> Thanks Paul,
>>>
>>> That's very helpful.
>>>
>>> I do want to interpolate to the height of the sensor, so could you
recommend a
>>
>>
>>> message type? Frankly, I'm not familiar with these message types,
so could you
>>>
>>>
>>> refer me to a link or reference to these message types?
>>>
>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>> point and column nine would be the sensor's height about ground.
But it sounds
>>>
>>>
>>> like column 6 should be the sensor's total height above sea level,
right? If
>
>>> so, what should column 9 have?
>>>
>>> Thanks again,
>>> David
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> This column does not apply to your data, since you are only
measuring wind
>>> speed
>>>
>>>
>>> at a fixed height above the ground. I
>>> would recommend putting -9999 in that column, since that is the
value that MET
>>
>>
>>> uses for bad data or N/A.
>>>
>>> You should use column 6 to enter the elevation of your sensor, but
be advised
>
>>> that if you use a "surface" message type
>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>
>>>
>>> enter in column 6. If you want MET to
>>> interpolate model data to the height of your sensor, use another
message type.
>>>
>>>
>>> I hope this is clear. Please let me
>>> know if you have any questions.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>> Queue: met_help
>>>> Subject: formatting ascii2nc
>>>> Owner: Nobody
>>>> Requestors: davidstephenbryan at yahoo.com
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>>
>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>> interval in hours" when formatting point observations. My point
observation
>>
>>
>>>> is
>>>>
>>>> an anemometer at a fixed height. What value would I put for this
column?
>>>>
>>>> Thanks,
>>>> David Bryan
>>>>
>>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: David Bryan
Time: Fri Jul 08 11:52:49 2011
Paul,
Below are outputs from two attempts to run Point_Stat.
For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
For the second, it appears the "MISSING/Z3" applies to the pressure
field (or maybe temperature; both were at 3 m). I'm not sure why this
is a problem. Could it have to do with a mismatch between fields
created with ASCII2NC and those in the grib file? For instance, for
the pressure value I used the field PRES (001) in ASCII2NC. But as I
examine the grib file in IDV, I don't see a simple pressure field. In
IDV under 2D grid, I see: Mean_sea_level_pressure_NAM_model @ msl,
Pressure @ cloud_base, Pressure @ cloud_tops, Pressure @ surface, and
Pressure_reduced_to_MSL @ msl. Under 3D grid, I see Pressure @
hybrid. The only field the grib file has in common with table of grib
parameters that could be used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
Similarly, I had assumed, if the netcdf file produced by ASCII2NC had
wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
Thanks,
David
[amec at WRF-master bin]$ ./point_stat \
> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
> -fcst_valid 20100617_17 \
> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
> -v 9
ERROR: PointStatConfInfo::process_config() -> the wind direction field
may not b
e verified using point_stat.
[amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
/WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072421513038
Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
Climatology File: none
Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
Observation File: /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
--------------------------------------------------------------------------------
Reading records for MISSING/Z3.
ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3 found
in GRIB f
ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
----- Original Message -----
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Cc:
Sent: Thursday, July 7, 2011 3:54 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
The point_stat usage statement indicates that the arguments should be
specified in the following order. Please try this
(you should be able just copy/paste):
./point_stat \
/ext/models/WRF/NREL/10061712/NREL10061712.grib \
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
-fcst_valid 20100617_17 \
-outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
-v 9
I think that may be the cause of your problem.
Paul
On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> I've attached my config file.
>
> When I tried running Point Stat, I got the error message below. I'm
not clear
> if it's coming from the config file or the grib file. I will point
out, though,
> that the grib files are produced by the WRF post processor (though
they are
> concatenated, using Linux cat). The text in the message isn't in
the config
> file.
>
> Any suggestions would be welcomed.
>
> Thanks,
> David
>
>
>
> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>
>
> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
> grib"
>
> line = 1
>
> econfig_column = 7
>
> text = "^"
>
>
>
> GRIB
> ____
>
>
> ----- Original Message ----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Thu, July 7, 2011 1:25:07 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> Regarding your observations, it might be easiest to include all of
your
> observations into a single NetCDF file. You can
> denote the two different sensor locations using Station_ID (column
2) in
> ascii2nc. Then, you can use the mask_sid
> option in the point_stat configuration file to specify which
stations obs you
> want to use. Otherwise, you can use both
> of your current NetCDF obs files in a single point_stat call by
passing one as
> the "obs_file" and the other one using
> the -point_obs optional argument.
>
> Regarding the forecast times, the following is from the point_stat
usage
> statement, which you can see by running
> point_stat with no arguments:
>
> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the forecast
valid time
> to be verified (optional).
> "-fcst_lead time" in HH[MMSS] format sets the forecast lead time
to be
> verified (optional).
>
> If your model is initiated (we call it initialized) at
20110625_1200, then that
> is your fcst_init time, not your
> fcst_valid time. If you are interested in verifying the forecast
five hours
> into your 48-hour run, then your fcst_lead
> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>
> Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long.
> Typically, for wind ground-based
> measurements, the fcst_field level looks something like "Z10" for a
sensor at
> 10m above the ground. Unfortunately, MET
> does not read model data on eta levels. In order to feed your WRF
data to MET,
> it must be post-processed using either
> the WRF Post Processor tool (WPP) or p_interp. This process will
put the data
> onto levels in meters and/or pressure,
> both of which MET can read.
>
> Please let me know if you have any more questions.
>
> Paul
>
>
> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>
>> Stat.
>>
>> My model output is for 48-hour runs of WRF with hourly output. The
output is
>> concatenated into a single file, 48 time steps in a single file.
(Perhaps
>> that's not practical.)
>>
>> My observations are for two met towers, each with two anemometers
at different
>
>> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
>
>> own netcdf file, each to serve as its own point source.
>>
>>
>> So when running PointStat, here's my understanding:
>>
>> PointStat will only analyze one model time per run.
>> In the command line arguments, I will have to specify -fcst_valid.
For
>> instance, if my WRF run is initiated with NAM data produced on the
>> 20110625_1200
>>
>> (and my model is initiated on the same time), that's my valid time.
>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>> 20110625_1700.
>> The purpose of specifying a temporal range of observation values is
for
>> interpolation across the temporal dimension.
>> The model time that's compared to observation time is the lead
time.
>> If any of my understanding of how time specification in Point Stat
works is
>> incorrect, please let me know.
>>
>> My other main question concerns specifying the levels for each
fcst_field. The
>>
>> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>> that
>>
>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>> three
>>
>> digits? (My MSL+AGL is ~1700m.) Also the provided default config
file
>> includes
>>
>> the following comment: "GC/ZNNN-NNN for a range of vertical levels
(MSL or
>> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
>
>> eta-levels which is more related to pressure levels. Does that
matter?
>>
>> Also, I assume that the forecast and observation thresholds serve
as a quality
>
>> control.
>>
>> Thanks again for all your assistance!
>>
>> David
>>
>>
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Wed, July 6, 2011 12:25:51 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>>> I do want to interpolate to the height of the sensor, so could you
recommend
> a
>>> message type? Frankly, I'm not familiar with these message types,
so could
>> you
>>> refer me to a link or reference to these message types?
>>
>> I think PROFLR is the appropriate message type for your situation.
Here is a
>> reference on the message types:
>>
>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>
>> Message types are a concept that is associated with PrepBUFR point
observations
>>
>> produced by NCEP. These observations
>> are often used in MET. The logic that MET uses for these message
types are
>> applied to all observations, regardless of
>> source.
>>
>>
>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>> point and column nine would be the sensor's height about ground.
But it
>> sounds
>>> like column 6 should be the sensor's total height above sea level,
right? If
>>> so, what should column 9 have?
>>
>> No, you are correct. I should have been more clear in my previous
email.
>>
>>
>>> will Point-Stat match the observation time to the model time and
>>> calculate its statistics from that?
>>
>> Point-Stat was designed to be run once for each forecast valid
time. If your
>> model data files contain single 48 hour
>> forecasts, run Point-Stat once for each model file. Each time, you
will pass
>> the model data file you want to verify,
>> the single observation file and the configuration file. In the
configuration
>> file, you should decide the time window
>> inside which observations "match" the model valid time. The
settings for this
>
>> are beg_ds and end_ds. You could also
>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>
>> If you have multiple forecasts in a single model data file, you
will need to
>> use
>>
>> the command line parameter fcst_valid
>> time to specify which forecast Point-Stat should verify. In this
case,
>> Point-Stat must still be run once for each valid
>> time, where each call differs only by the value passed to the
fcst_valid
>> parameter.
>>
>> Does that make sense? Please let me know if you have questions.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>
>>
>>> months. So I'll have one six-month file for observations. I want
to use
>>> Point-Stat to compare that file to a series of 48-hour forecasts.
My
>>> question--will Point-Stat match the observation time to the model
time and
>>> calculate its statistics from that? In other words, Point-Stat's
output for
>>> each forecast is not based on periods for which there is no
forecast data,
>>> correct?
>>>
>>> Thanks,
>>> David
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> Thanks Paul,
>>>
>>> That's very helpful.
>>>
>>> I do want to interpolate to the height of the sensor, so could you
recommend a
>>
>>
>>> message type? Frankly, I'm not familiar with these message types,
so could you
>>>
>>>
>>> refer me to a link or reference to these message types?
>>>
>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>> point and column nine would be the sensor's height about ground.
But it sounds
>>>
>>>
>>> like column 6 should be the sensor's total height above sea level,
right? If
>
>>> so, what should column 9 have?
>>>
>>> Thanks again,
>>> David
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> This column does not apply to your data, since you are only
measuring wind
>>> speed
>>>
>>>
>>> at a fixed height above the ground. I
>>> would recommend putting -9999 in that column, since that is the
value that MET
>>
>>
>>> uses for bad data or N/A.
>>>
>>> You should use column 6 to enter the elevation of your sensor, but
be advised
>
>>> that if you use a "surface" message type
>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>
>>>
>>> enter in column 6. If you want MET to
>>> interpolate model data to the height of your sensor, use another
message type.
>>>
>>>
>>> I hope this is clear. Please let me
>>> know if you have any questions.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>> Queue: met_help
>>>> Subject: formatting ascii2nc
>>>> Owner: Nobody
>>>> Requestors: davidstephenbryan at yahoo.com
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>>
>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>> interval in hours" when formatting point observations. My point
observation
>>
>>
>>>> is
>>>>
>>>> an anemometer at a fixed height. What value would I put for this
column?
>>>>
>>>> Thanks,
>>>> David Bryan
>>>>
>>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: Paul Oldenburg
Time: Fri Jul 08 14:07:28 2011
David,
Can you please bundle up your model data, obs and config files using
"tar xzvf" and upload them to our FTP site using
the following instructions?
http://www.dtcenter.org/met/users/support/met_help.php#ftp
That will make it easier for me to analyze your situation and figure
out the best course of action with your data.
Thanks,
Paul
On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> Below are outputs from two attempts to run Point_Stat.
>
> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>
> For the second, it appears the "MISSING/Z3" applies to the pressure
field (or maybe temperature; both were at 3 m). I'm not sure why this
is a problem. Could it have to do with a mismatch between fields
created with ASCII2NC and those in the grib file? For instance, for
the pressure value I used the field PRES (001) in ASCII2NC. But as I
examine the grib file in IDV, I don't see a simple pressure field. In
IDV under 2D grid, I see: Mean_sea_level_pressure_NAM_model @ msl,
Pressure @ cloud_base, Pressure @ cloud_tops, Pressure @ surface, and
Pressure_reduced_to_MSL @ msl. Under 3D grid, I see Pressure @
hybrid. The only field the grib file has in common with table of grib
parameters that could be used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>
> Similarly, I had assumed, if the netcdf file produced by ASCII2NC
had wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
>
> Thanks,
> David
>
>
> [amec at WRF-master bin]$ ./point_stat \
>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>> -fcst_valid 20100617_17 \
>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>> -v 9
>
>
> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
> e verified using point_stat.
>
> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744072421513038
> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
> Climatology File: none
> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
> Observation File: /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>
>
--------------------------------------------------------------------------------
>
> Reading records for MISSING/Z3.
>
>
> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>
>
> ----- Original Message -----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Cc:
> Sent: Thursday, July 7, 2011 3:54 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> The point_stat usage statement indicates that the arguments should
be specified in the following order. Please try this
> (you should be able just copy/paste):
>
> ./point_stat \
> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
> -fcst_valid 20100617_17 \
> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
> -v 9
>
> I think that may be the cause of your problem.
>
> Paul
>
>
> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> I've attached my config file.
>>
>> When I tried running Point Stat, I got the error message below.
I'm not clear
>> if it's coming from the config file or the grib file. I will point
out, though,
>> that the grib files are produced by the WRF post processor (though
they are
>> concatenated, using Linux cat). The text in the message isn't in
the config
>> file.
>>
>> Any suggestions would be welcomed.
>>
>> Thanks,
>> David
>>
>>
>>
>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>
>>
>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>> grib"
>>
>> line = 1
>>
>> econfig_column = 7
>>
>> text = "^"
>>
>>
>>
>> GRIB
>> ____
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Thu, July 7, 2011 1:25:07 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> Regarding your observations, it might be easiest to include all of
your
>> observations into a single NetCDF file. You can
>> denote the two different sensor locations using Station_ID (column
2) in
>> ascii2nc. Then, you can use the mask_sid
>> option in the point_stat configuration file to specify which
stations obs you
>> want to use. Otherwise, you can use both
>> of your current NetCDF obs files in a single point_stat call by
passing one as
>> the "obs_file" and the other one using
>> the -point_obs optional argument.
>>
>> Regarding the forecast times, the following is from the point_stat
usage
>> statement, which you can see by running
>> point_stat with no arguments:
>>
>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>> to be verified (optional).
>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead time
to be
>> verified (optional).
>>
>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>> is your fcst_init time, not your
>> fcst_valid time. If you are interested in verifying the forecast
five hours
>> into your 48-hour run, then your fcst_lead
>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>
>> Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long.
>> Typically, for wind ground-based
>> measurements, the fcst_field level looks something like "Z10" for a
sensor at
>> 10m above the ground. Unfortunately, MET
>> does not read model data on eta levels. In order to feed your WRF
data to MET,
>> it must be post-processed using either
>> the WRF Post Processor tool (WPP) or p_interp. This process will
put the data
>> onto levels in meters and/or pressure,
>> both of which MET can read.
>>
>> Please let me know if you have any more questions.
>>
>> Paul
>>
>>
>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>>
>>> Stat.
>>>
>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>> concatenated into a single file, 48 time steps in a single file.
(Perhaps
>>> that's not practical.)
>>>
>>> My observations are for two met towers, each with two anemometers
at different
>>
>>> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
>>
>>> own netcdf file, each to serve as its own point source.
>>>
>>>
>>> So when running PointStat, here's my understanding:
>>>
>>> PointStat will only analyze one model time per run.
>>> In the command line arguments, I will have to specify -fcst_valid.
For
>>> instance, if my WRF run is initiated with NAM data produced on the
>>> 20110625_1200
>>>
>>> (and my model is initiated on the same time), that's my valid
time.
>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>> 20110625_1700.
>>> The purpose of specifying a temporal range of observation values
is for
>>> interpolation across the temporal dimension.
>>> The model time that's compared to observation time is the lead
time.
>>> If any of my understanding of how time specification in Point Stat
works is
>>> incorrect, please let me know.
>>>
>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>
>>> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>>> that
>>>
>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>>> three
>>>
>>> digits? (My MSL+AGL is ~1700m.) Also the provided default config
file
>>> includes
>>>
>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
>>
>>> eta-levels which is more related to pressure levels. Does that
matter?
>>>
>>> Also, I assume that the forecast and observation thresholds serve
as a quality
>>
>>> control.
>>>
>>> Thanks again for all your assistance!
>>>
>>> David
>>>
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>>> I do want to interpolate to the height of the sensor, so could
you recommend
>> a
>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>> you
>>>> refer me to a link or reference to these message types?
>>>
>>> I think PROFLR is the appropriate message type for your situation.
Here is a
>>> reference on the message types:
>>>
>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>
>>> Message types are a concept that is associated with PrepBUFR point
observations
>>>
>>> produced by NCEP. These observations
>>> are often used in MET. The logic that MET uses for these message
types are
>>> applied to all observations, regardless of
>>> source.
>>>
>>>
>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>> point and column nine would be the sensor's height about ground.
But it
>>> sounds
>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>> so, what should column 9 have?
>>>
>>> No, you are correct. I should have been more clear in my previous
email.
>>>
>>>
>>>> will Point-Stat match the observation time to the model time and
>>>> calculate its statistics from that?
>>>
>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>> model data files contain single 48 hour
>>> forecasts, run Point-Stat once for each model file. Each time,
you will pass
>>> the model data file you want to verify,
>>> the single observation file and the configuration file. In the
configuration
>>> file, you should decide the time window
>>> inside which observations "match" the model valid time. The
settings for this
>>
>>> are beg_ds and end_ds. You could also
>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>
>>> If you have multiple forecasts in a single model data file, you
will need to
>>> use
>>>
>>> the command line parameter fcst_valid
>>> time to specify which forecast Point-Stat should verify. In this
case,
>>> Point-Stat must still be run once for each valid
>>> time, where each call differs only by the value passed to the
fcst_valid
>>> parameter.
>>>
>>> Does that make sense? Please let me know if you have questions.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>
>>>
>>>> months. So I'll have one six-month file for observations. I
want to use
>>>> Point-Stat to compare that file to a series of 48-hour forecasts.
My
>>>> question--will Point-Stat match the observation time to the model
time and
>>>> calculate its statistics from that? In other words, Point-Stat's
output for
>>>> each forecast is not based on periods for which there is no
forecast data,
>>>> correct?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>
>>>>
>>>> ----- Original Message ----
>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> Thanks Paul,
>>>>
>>>> That's very helpful.
>>>>
>>>> I do want to interpolate to the height of the sensor, so could
you recommend a
>>>
>>>
>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>
>>>>
>>>> refer me to a link or reference to these message types?
>>>>
>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>> point and column nine would be the sensor's height about ground.
But it sounds
>>>>
>>>>
>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>
>>>> so, what should column 9 have?
>>>>
>>>> Thanks again,
>>>> David
>>>>
>>>>
>>>> ----- Original Message ----
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> This column does not apply to your data, since you are only
measuring wind
>>>> speed
>>>>
>>>>
>>>> at a fixed height above the ground. I
>>>> would recommend putting -9999 in that column, since that is the
value that MET
>>>
>>>
>>>> uses for bad data or N/A.
>>>>
>>>> You should use column 6 to enter the elevation of your sensor,
but be advised
>>
>>>> that if you use a "surface" message type
>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>
>>>>
>>>> enter in column 6. If you want MET to
>>>> interpolate model data to the height of your sensor, use another
message type.
>>>>
>>>>
>>>> I hope this is clear. Please let me
>>>> know if you have any questions.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>> Queue: met_help
>>>>> Subject: formatting ascii2nc
>>>>> Owner: Nobody
>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>> Status: new
>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>>
>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>> interval in hours" when formatting point observations. My
point observation
>>>
>>>
>>>>> is
>>>>>
>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>
>>>>> Thanks,
>>>>> David Bryan
>>>>>
>>>>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Mon Jul 11 13:10:39 2011
Paul,
I've posted the data on your ftp site under the directory,
Bryan_data.
The contents should be self explanatory, largely. I included two
versions of the config file I tried. Also, I included the original
source of the data, met2010.csv.
Thanks,
David Bryan
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Friday, July 8, 2011 5:37 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
Can you please bundle up your model data, obs and config files using
"tar xzvf" and upload them to our FTP site using
the following instructions?
http://www.dtcenter.org/met/users/support/met_help.php#ftp
That will make it easier for me to analyze your situation and figure
out the best course of action with your data.
Thanks,
Paul
On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> Below are outputs from two attempts to run Point_Stat.
>
> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>
> For the second, it appears the "MISSING/Z3" applies to the pressure
field (or maybe temperature; both were at 3 m). I'm not sure why this
is a problem. Could it have to do with a mismatch between fields
created with ASCII2NC and those in the grib file? For instance, for
the pressure value I used the field PRES (001) in ASCII2NC. But as I
examine the grib file in IDV, I don't see a simple pressure field. In
IDV under 2D grid, I see: Mean_sea_level_pressure_NAM_model @ msl,
Pressure @ cloud_base, Pressure @ cloud_tops, Pressure @ surface, and
Pressure_reduced_to_MSL @ msl. Under 3D grid, I see Pressure @
hybrid. The only field the grib file has in common with table of grib
parameters that could be used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>
> Similarly, I had assumed, if the netcdf file produced by ASCII2NC
had wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
>
> Thanks,
> David
>
>
> [amec at WRF-master bin]$ ./point_stat \
>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>> -fcst_valid 20100617_17 \
>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>> -v 9
>
>
> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
> e verified using point_stat.
>
> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744072421513038
> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
> Climatology File: none
> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
> Observation File: /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>
>
--------------------------------------------------------------------------------
>
> Reading records for MISSING/Z3.
>
>
> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>
>
> ----- Original Message -----
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Cc:
> Sent: Thursday, July 7, 2011 3:54 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> The point_stat usage statement indicates that the arguments should
be specified in the following order. Please try this
> (you should be able just copy/paste):
>
> ./point_stat \
> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
> -fcst_valid 20100617_17 \
> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
> -v 9
>
> I think that may be the cause of your problem.
>
> Paul
>
>
> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> I've attached my config file.
>>
>> When I tried running Point Stat, I got the error message below.
I'm not clear
>> if it's coming from the config file or the grib file. I will point
out, though,
>> that the grib files are produced by the WRF post processor (though
they are
>> concatenated, using Linux cat). The text in the message isn't in
the config
>> file.
>>
>> Any suggestions would be welcomed.
>>
>> Thanks,
>> David
>>
>>
>>
>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>
>>
>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>> grib"
>>
>> line = 1
>>
>> econfig_column = 7
>>
>> text = "^"
>>
>>
>>
>> GRIB
>> ____
>>
>>
>> ----- Original Message ----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Thu, July 7, 2011 1:25:07 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> Regarding your observations, it might be easiest to include all of
your
>> observations into a single NetCDF file. You can
>> denote the two different sensor locations using Station_ID (column
2) in
>> ascii2nc. Then, you can use the mask_sid
>> option in the point_stat configuration file to specify which
stations obs you
>> want to use. Otherwise, you can use both
>> of your current NetCDF obs files in a single point_stat call by
passing one as
>> the "obs_file" and the other one using
>> the -point_obs optional argument.
>>
>> Regarding the forecast times, the following is from the point_stat
usage
>> statement, which you can see by running
>> point_stat with no arguments:
>>
>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>> to be verified (optional).
>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead time
to be
>> verified (optional).
>>
>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>> is your fcst_init time, not your
>> fcst_valid time. If you are interested in verifying the forecast
five hours
>> into your 48-hour run, then your fcst_lead
>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>
>> Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long.
>> Typically, for wind ground-based
>> measurements, the fcst_field level looks something like "Z10" for a
sensor at
>> 10m above the ground. Unfortunately, MET
>> does not read model data on eta levels. In order to feed your WRF
data to MET,
>> it must be post-processed using either
>> the WRF Post Processor tool (WPP) or p_interp. This process will
put the data
>> onto levels in meters and/or pressure,
>> both of which MET can read.
>>
>> Please let me know if you have any more questions.
>>
>> Paul
>>
>>
>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>>
>>> Stat.
>>>
>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>> concatenated into a single file, 48 time steps in a single file.
(Perhaps
>>> that's not practical.)
>>>
>>> My observations are for two met towers, each with two anemometers
at different
>>
>>> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
>>
>>> own netcdf file, each to serve as its own point source.
>>>
>>>
>>> So when running PointStat, here's my understanding:
>>>
>>> PointStat will only analyze one model time per run.
>>> In the command line arguments, I will have to specify -fcst_valid.
For
>>> instance, if my WRF run is initiated with NAM data produced on the
>>> 20110625_1200
>>>
>>> (and my model is initiated on the same time), that's my valid
time.
>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>> 20110625_1700.
>>> The purpose of specifying a temporal range of observation values
is for
>>> interpolation across the temporal dimension.
>>> The model time that's compared to observation time is the lead
time.
>>> If any of my understanding of how time specification in Point Stat
works is
>>> incorrect, please let me know.
>>>
>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>
>>> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>>> that
>>>
>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>>> three
>>>
>>> digits? (My MSL+AGL is ~1700m.) Also the provided default config
file
>>> includes
>>>
>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
>>
>>> eta-levels which is more related to pressure levels. Does that
matter?
>>>
>>> Also, I assume that the forecast and observation thresholds serve
as a quality
>>
>>> control.
>>>
>>> Thanks again for all your assistance!
>>>
>>> David
>>>
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>>> I do want to interpolate to the height of the sensor, so could
you recommend
>> a
>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>> you
>>>> refer me to a link or reference to these message types?
>>>
>>> I think PROFLR is the appropriate message type for your situation.
Here is a
>>> reference on the message types:
>>>
>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>
>>> Message types are a concept that is associated with PrepBUFR point
observations
>>>
>>> produced by NCEP. These observations
>>> are often used in MET. The logic that MET uses for these message
types are
>>> applied to all observations, regardless of
>>> source.
>>>
>>>
>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>> point and column nine would be the sensor's height about ground.
But it
>>> sounds
>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>> so, what should column 9 have?
>>>
>>> No, you are correct. I should have been more clear in my previous
email.
>>>
>>>
>>>> will Point-Stat match the observation time to the model time and
>>>> calculate its statistics from that?
>>>
>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>> model data files contain single 48 hour
>>> forecasts, run Point-Stat once for each model file. Each time,
you will pass
>>> the model data file you want to verify,
>>> the single observation file and the configuration file. In the
configuration
>>> file, you should decide the time window
>>> inside which observations "match" the model valid time. The
settings for this
>>
>>> are beg_ds and end_ds. You could also
>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>
>>> If you have multiple forecasts in a single model data file, you
will need to
>>> use
>>>
>>> the command line parameter fcst_valid
>>> time to specify which forecast Point-Stat should verify. In this
case,
>>> Point-Stat must still be run once for each valid
>>> time, where each call differs only by the value passed to the
fcst_valid
>>> parameter.
>>>
>>> Does that make sense? Please let me know if you have questions.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>
>>>
>>>> months. So I'll have one six-month file for observations. I
want to use
>>>> Point-Stat to compare that file to a series of 48-hour forecasts.
My
>>>> question--will Point-Stat match the observation time to the model
time and
>>>> calculate its statistics from that? In other words, Point-Stat's
output for
>>>> each forecast is not based on periods for which there is no
forecast data,
>>>> correct?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>
>>>>
>>>> ----- Original Message ----
>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> Thanks Paul,
>>>>
>>>> That's very helpful.
>>>>
>>>> I do want to interpolate to the height of the sensor, so could
you recommend a
>>>
>>>
>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>
>>>>
>>>> refer me to a link or reference to these message types?
>>>>
>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>> point and column nine would be the sensor's height about ground.
But it sounds
>>>>
>>>>
>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>
>>>> so, what should column 9 have?
>>>>
>>>> Thanks again,
>>>> David
>>>>
>>>>
>>>> ----- Original Message ----
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> This column does not apply to your data, since you are only
measuring wind
>>>> speed
>>>>
>>>>
>>>> at a fixed height above the ground. I
>>>> would recommend putting -9999 in that column, since that is the
value that MET
>>>
>>>
>>>> uses for bad data or N/A.
>>>>
>>>> You should use column 6 to enter the elevation of your sensor,
but be advised
>>
>>>> that if you use a "surface" message type
>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>
>>>>
>>>> enter in column 6. If you want MET to
>>>> interpolate model data to the height of your sensor, use another
message type.
>>>>
>>>>
>>>> I hope this is clear. Please let me
>>>> know if you have any questions.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>> Queue: met_help
>>>>> Subject: formatting ascii2nc
>>>>> Owner: Nobody
>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>> Status: new
>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>>
>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>> interval in hours" when formatting point observations. My
point observation
>>>
>>>
>>>>> is
>>>>>
>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>
>>>>> Thanks,
>>>>> David Bryan
>>>>>
>>>>
------------------------------------------------
Subject: formatting ascii2nc
From: Paul Oldenburg
Time: Mon Jul 11 15:09:52 2011
David,
I'm still working on your wind verification problem.
Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
Point_stat can only match forecast/obs pairs in three ways:
1. by height in meters for non-"surface" message type obs and model
records
2. by pressure in hPa for non-"surface" message type obs and model
records
3. by pairing "surface" message type obs (i.e. ADPSFC and SFCSHP) to
surface model records
For surface level matching, you must specify the "surface" height in
the fcst_field of the config file: 0m for PRES, 2m
for TMP and 10m for winds. Surface GRIB records have 'sfc' in the
12th wgrib field:
$ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":"); print
$_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
...
To generate matched pairs with with your obs, I modified your obs
files in the following way:
$ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
$ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
version that I used. The point_stat output is shown below. I realize
that this is a bit confusing, but the setup of
the observation level is really dictated by the information in the
model GRIB file. In this case, the level there is
surface.
Unfortunately, if this is all the obs that you have, you only get 3
matched pairs for each field. You really only have
1 exact match if you reduce the beg_ds and end_ds time window to less
than 10 min (+/- 600s), since your obs are valid
every 10 minutes. In your case, I think you may want to dump out only
the MPR data which contains matched pair data.
Then, once you run point_stat across all of your valid_time data, you
can use the stat_analysis tool to read your MPRs
and generate aggregated statistics.
If you have any questions, please let me know. Once I have something
to give you for verifying your wind data, I'll
send you an update.
Thanks,
Paul
$ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt -fcst_valid
20100617_13 -outdir . -v 2
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744071635082062
Forecast File: NREL10061712.grib
Climatology File: none
Configuration File: PSconfig2.txt
Observation File: A9_sfc.nc
--------------------------------------------------------------------------------
Reading records for PRES/Z0.
For PRES/Z0 found 1 forecast levels and 0 climatology levels.
--------------------------------------------------------------------------------
Reading records for TMP/Z2.
For TMP/Z2 found 1 forecast levels and 0 climatology levels.
--------------------------------------------------------------------------------
Searching 184440 observations from 30740 PrepBufr messages.
--------------------------------------------------------------------------------
Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
using 3 pairs.
Computing Categorical Statistics.
Computing Continuous Statistics.
Computing Scalar Partial Sums.
--------------------------------------------------------------------------------
Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
using 3 pairs.
Computing Categorical Statistics.
Computing Continuous Statistics.
Computing Scalar Partial Sums.
--------------------------------------------------------------------------------
Output file: ./point_stat_010000L_20100617_130000V.stat
On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> Paul,
>
> I've posted the data on your ftp site under the directory,
Bryan_data.
>
> The contents should be self explanatory, largely. I included two
versions of the config file I tried. Also, I included the original
source of the data, met2010.csv.
>
> Thanks,
> David Bryan
>
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Friday, July 8, 2011 5:37 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> Can you please bundle up your model data, obs and config files using
"tar xzvf" and upload them to our FTP site using
> the following instructions?
>
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> That will make it easier for me to analyze your situation and figure
out the best course of action with your data.
>
> Thanks,
>
> Paul
>
>
> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> Below are outputs from two attempts to run Point_Stat.
>>
>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>
>> For the second, it appears the "MISSING/Z3" applies to the pressure
field (or maybe temperature; both were at 3 m). I'm not sure why this
is a problem. Could it have to do with a mismatch between fields
created with ASCII2NC and those in the grib file? For instance, for
the pressure value I used the field PRES (001) in ASCII2NC. But as I
examine the grib file in IDV, I don't see a simple pressure field. In
IDV under 2D grid, I see: Mean_sea_level_pressure_NAM_model @ msl,
Pressure @ cloud_base, Pressure @ cloud_tops, Pressure @ surface, and
Pressure_reduced_to_MSL @ msl. Under 3D grid, I see Pressure @
hybrid. The only field the grib file has in common with table of grib
parameters that could be used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>
>> Similarly, I had assumed, if the netcdf file produced by ASCII2NC
had wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
>>
>> Thanks,
>> David
>>
>>
>> [amec at WRF-master bin]$ ./point_stat \
>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>> -fcst_valid 20100617_17 \
>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>> -v 9
>>
>>
>> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
>> e verified using point_stat.
>>
>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=18446744072421513038
>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>> Climatology File: none
>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>> Observation File: /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>
>>
--------------------------------------------------------------------------------
>>
>> Reading records for MISSING/Z3.
>>
>>
>> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>
>>
>> ----- Original Message -----
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Cc:
>> Sent: Thursday, July 7, 2011 3:54 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> The point_stat usage statement indicates that the arguments should
be specified in the following order. Please try this
>> (you should be able just copy/paste):
>>
>> ./point_stat \
>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>> -fcst_valid 20100617_17 \
>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>> -v 9
>>
>> I think that may be the cause of your problem.
>>
>> Paul
>>
>>
>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> I've attached my config file.
>>>
>>> When I tried running Point Stat, I got the error message below.
I'm not clear
>>> if it's coming from the config file or the grib file. I will
point out, though,
>>> that the grib files are produced by the WRF post processor (though
they are
>>> concatenated, using Linux cat). The text in the message isn't in
the config
>>> file.
>>>
>>> Any suggestions would be welcomed.
>>>
>>> Thanks,
>>> David
>>>
>>>
>>>
>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>
>>>
>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>> grib"
>>>
>>> line = 1
>>>
>>> econfig_column = 7
>>>
>>> text = "^"
>>>
>>>
>>>
>>> GRIB
>>> ____
>>>
>>>
>>> ----- Original Message ----
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> Regarding your observations, it might be easiest to include all of
your
>>> observations into a single NetCDF file. You can
>>> denote the two different sensor locations using Station_ID (column
2) in
>>> ascii2nc. Then, you can use the mask_sid
>>> option in the point_stat configuration file to specify which
stations obs you
>>> want to use. Otherwise, you can use both
>>> of your current NetCDF obs files in a single point_stat call by
passing one as
>>> the "obs_file" and the other one using
>>> the -point_obs optional argument.
>>>
>>> Regarding the forecast times, the following is from the point_stat
usage
>>> statement, which you can see by running
>>> point_stat with no arguments:
>>>
>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>> to be verified (optional).
>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>> verified (optional).
>>>
>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>> is your fcst_init time, not your
>>> fcst_valid time. If you are interested in verifying the forecast
five hours
>>> into your 48-hour run, then your fcst_lead
>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>
>>> Regarding levels, ZNNN is AGL and it can be an arbitrary number of
digits long.
>>> Typically, for wind ground-based
>>> measurements, the fcst_field level looks something like "Z10" for
a sensor at
>>> 10m above the ground. Unfortunately, MET
>>> does not read model data on eta levels. In order to feed your WRF
data to MET,
>>> it must be post-processed using either
>>> the WRF Post Processor tool (WPP) or p_interp. This process will
put the data
>>> onto levels in meters and/or pressure,
>>> both of which MET can read.
>>>
>>> Please let me know if you have any more questions.
>>>
>>> Paul
>>>
>>>
>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>>>
>>>> Stat.
>>>>
>>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>>> concatenated into a single file, 48 time steps in a single file.
(Perhaps
>>>> that's not practical.)
>>>>
>>>> My observations are for two met towers, each with two anemometers
at different
>>>
>>>> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
>>>
>>>> own netcdf file, each to serve as its own point source.
>>>>
>>>>
>>>> So when running PointStat, here's my understanding:
>>>>
>>>> PointStat will only analyze one model time per run.
>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>> instance, if my WRF run is initiated with NAM data produced on
the
>>>> 20110625_1200
>>>>
>>>> (and my model is initiated on the same time), that's my valid
time.
>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>>> 20110625_1700.
>>>> The purpose of specifying a temporal range of observation values
is for
>>>> interpolation across the temporal dimension.
>>>> The model time that's compared to observation time is the lead
time.
>>>> If any of my understanding of how time specification in Point
Stat works is
>>>> incorrect, please let me know.
>>>>
>>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>>
>>>> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>>>> that
>>>>
>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>>>> three
>>>>
>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>> includes
>>>>
>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>> AGL)." How would one specify whether it's MSL or AGL? Also, WRF
output is in
>>>
>>>> eta-levels which is more related to pressure levels. Does that
matter?
>>>>
>>>> Also, I assume that the forecast and observation thresholds serve
as a quality
>>>
>>>> control.
>>>>
>>>> Thanks again for all your assistance!
>>>>
>>>> David
>>>>
>>>>
>>>>
>>>>
>>>> ----- Original Message ----
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>>> I do want to interpolate to the height of the sensor, so could
you recommend
>>> a
>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>> you
>>>>> refer me to a link or reference to these message types?
>>>>
>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>> reference on the message types:
>>>>
>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>
>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>
>>>> produced by NCEP. These observations
>>>> are often used in MET. The logic that MET uses for these message
types are
>>>> applied to all observations, regardless of
>>>> source.
>>>>
>>>>
>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>> point and column nine would be the sensor's height about ground.
But it
>>>> sounds
>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>> so, what should column 9 have?
>>>>
>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>
>>>>
>>>>> will Point-Stat match the observation time to the model time and
>>>>> calculate its statistics from that?
>>>>
>>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>>> model data files contain single 48 hour
>>>> forecasts, run Point-Stat once for each model file. Each time,
you will pass
>>>> the model data file you want to verify,
>>>> the single observation file and the configuration file. In the
configuration
>>>> file, you should decide the time window
>>>> inside which observations "match" the model valid time. The
settings for this
>>>
>>>> are beg_ds and end_ds. You could also
>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>
>>>> If you have multiple forecasts in a single model data file, you
will need to
>>>> use
>>>>
>>>> the command line parameter fcst_valid
>>>> time to specify which forecast Point-Stat should verify. In this
case,
>>>> Point-Stat must still be run once for each valid
>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>> parameter.
>>>>
>>>> Does that make sense? Please let me know if you have questions.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>
>>>>
>>>>> months. So I'll have one six-month file for observations. I
want to use
>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>> correct?
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message ----
>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> Thanks Paul,
>>>>>
>>>>> That's very helpful.
>>>>>
>>>>> I do want to interpolate to the height of the sensor, so could
you recommend a
>>>>
>>>>
>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>
>>>>>
>>>>> refer me to a link or reference to these message types?
>>>>>
>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>> point and column nine would be the sensor's height about ground.
But it sounds
>>>>>
>>>>>
>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>
>>>>> so, what should column 9 have?
>>>>>
>>>>> Thanks again,
>>>>> David
>>>>>
>>>>>
>>>>> ----- Original Message ----
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>> speed
>>>>>
>>>>>
>>>>> at a fixed height above the ground. I
>>>>> would recommend putting -9999 in that column, since that is the
value that MET
>>>>
>>>>
>>>>> uses for bad data or N/A.
>>>>>
>>>>> You should use column 6 to enter the elevation of your sensor,
but be advised
>>>
>>>>> that if you use a "surface" message type
>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>
>>>>>
>>>>> enter in column 6. If you want MET to
>>>>> interpolate model data to the height of your sensor, use another
message type.
>>>>>
>>>>>
>>>>> I hope this is clear. Please let me
>>>>> know if you have any questions.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>> Queue: met_help
>>>>>> Subject: formatting ascii2nc
>>>>>> Owner: Nobody
>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>> Status: new
>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>>
>>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>
>>>>
>>>>>> is
>>>>>>
>>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>>
>>>>>> Thanks,
>>>>>> David Bryan
>>>>>>
>>>>>
------------------------------------------------
Subject: formatting ascii2nc
From: Paul Oldenburg
Time: Mon Jul 11 15:09:52 2011
model = "WRF";
beg_ds = -600;
end_ds = 600;
//fcst_field[] = [ "PRES/Z3","TMP/Z3","WIND/Z50","WIND/Z80"];
fcst_field[] = [ "PRES/Z0", "TMP/Z2" ];
obs_field[] = [ "PRES/Z3", "TMP/Z3" ];
//fcst_thresh[] = [ "ge87000 le108330", "ge233.15 le322.15", "ge0
le141.33", "ge0 le141.33" ];
fcst_thresh[] = [ "ge87000 le108330", "ge233.15 le322.15" ];
obs_thresh[] = [];
fcst_wind_thresh[] = [ "NA" ];
obs_wind_thresh[] = [];
message_type[] = [ "PROFLR" ];
//message_type[] = [ "ADPSFC" ];
mask_grid[] = [ "FULL" ];
mask_poly[] = [];
mask_sid = "";
ci_alpha[] = [ 0.05 ];
boot_interval = 1;
boot_rep_prop = 1.0;
n_boot_rep = 1000;
boot_rng = "mt19937";
boot_seed = "";
interp_method[] = [ "UW_MEAN" ];
interp_width[] = [ 1 ];
interp_thresh = 1.0;
output_flag[] = [ 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ];
rank_corr_flag = 1;
grib_ptv = 2;
//tmp_dir = "/home/WRF/Development/David_Bryan/ASCII2NC/tmp/";
tmp_dir = "/tmp";
output_prefix = "";
version = "V3.0";
------------------------------------------------
Subject: formatting ascii2nc
From: Paul Oldenburg
Time: Mon Jul 11 15:13:10 2011
David,
I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
obs, by the way. It matches the "surface" fields in the model data
file against your original PROFLR obs. I attached
the version I did intend to send with this email.
Paul
On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
> David,
>
> I'm still working on your wind verification problem.
>
> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>
>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>
> Point_stat can only match forecast/obs pairs in three ways:
>
> 1. by height in meters for non-"surface" message type obs and model
records
> 2. by pressure in hPa for non-"surface" message type obs and model
records
> 3. by pairing "surface" message type obs (i.e. ADPSFC and SFCSHP)
to surface model records
>
> For surface level matching, you must specify the "surface" height in
the fcst_field of the config file: 0m for PRES, 2m
> for TMP and 10m for winds. Surface GRIB records have 'sfc' in the
12th wgrib field:
>
> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
> ...
>
> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>
> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>
> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
> the observation level is really dictated by the information in the
model GRIB file. In this case, the level there is
> surface.
>
> Unfortunately, if this is all the obs that you have, you only get 3
matched pairs for each field. You really only have
> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
> and generate aggregated statistics.
>
> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
> send you an update.
>
> Thanks,
>
> Paul
>
>
> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt -fcst_valid
20100617_13 -outdir . -v 2
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744071635082062
> Forecast File: NREL10061712.grib
> Climatology File: none
> Configuration File: PSconfig2.txt
> Observation File: A9_sfc.nc
>
>
--------------------------------------------------------------------------------
>
> Reading records for PRES/Z0.
> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>
>
--------------------------------------------------------------------------------
>
> Reading records for TMP/Z2.
> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>
>
--------------------------------------------------------------------------------
>
> Searching 184440 observations from 30740 PrepBufr messages.
>
>
--------------------------------------------------------------------------------
>
> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
> using 3 pairs.
> Computing Categorical Statistics.
> Computing Continuous Statistics.
> Computing Scalar Partial Sums.
>
>
--------------------------------------------------------------------------------
>
> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
> using 3 pairs.
> Computing Categorical Statistics.
> Computing Continuous Statistics.
> Computing Scalar Partial Sums.
>
>
--------------------------------------------------------------------------------
>
> Output file: ./point_stat_010000L_20100617_130000V.stat
>
>
>
>
>
> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> Paul,
>>
>> I've posted the data on your ftp site under the directory,
Bryan_data.
>>
>> The contents should be self explanatory, largely. I included two
versions of the config file I tried. Also, I included the original
source of the data, met2010.csv.
>>
>> Thanks,
>> David Bryan
>>
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Friday, July 8, 2011 5:37 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>> the following instructions?
>>
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> Below are outputs from two attempts to run Point_Stat.
>>>
>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>
>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>
>>> Similarly, I had assumed, if the netcdf file produced by ASCII2NC
had wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
>>>
>>> Thanks,
>>> David
>>>
>>>
>>> [amec at WRF-master bin]$ ./point_stat \
>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>> -fcst_valid 20100617_17 \
>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>> -v 9
>>>
>>>
>>> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
>>> e verified using point_stat.
>>>
>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>> GSL_RNG_TYPE=mt19937
>>> GSL_RNG_SEED=18446744072421513038
>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>> Climatology File: none
>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>> Observation File: /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Reading records for MISSING/Z3.
>>>
>>>
>>> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>
>>>
>>> ----- Original Message -----
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Cc:
>>> Sent: Thursday, July 7, 2011 3:54 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> The point_stat usage statement indicates that the arguments should
be specified in the following order. Please try this
>>> (you should be able just copy/paste):
>>>
>>> ./point_stat \
>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>> -fcst_valid 20100617_17 \
>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>> -v 9
>>>
>>> I think that may be the cause of your problem.
>>>
>>> Paul
>>>
>>>
>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> I've attached my config file.
>>>>
>>>> When I tried running Point Stat, I got the error message below.
I'm not clear
>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>> that the grib files are produced by the WRF post processor
(though they are
>>>> concatenated, using Linux cat). The text in the message isn't in
the config
>>>> file.
>>>>
>>>> Any suggestions would be welcomed.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>
>>>>
>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>
>>>>
>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>> grib"
>>>>
>>>> line = 1
>>>>
>>>> econfig_column = 7
>>>>
>>>> text = "^"
>>>>
>>>>
>>>>
>>>> GRIB
>>>> ____
>>>>
>>>>
>>>> ----- Original Message ----
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> Regarding your observations, it might be easiest to include all
of your
>>>> observations into a single NetCDF file. You can
>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>> ascii2nc. Then, you can use the mask_sid
>>>> option in the point_stat configuration file to specify which
stations obs you
>>>> want to use. Otherwise, you can use both
>>>> of your current NetCDF obs files in a single point_stat call by
passing one as
>>>> the "obs_file" and the other one using
>>>> the -point_obs optional argument.
>>>>
>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>> statement, which you can see by running
>>>> point_stat with no arguments:
>>>>
>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>> to be verified (optional).
>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>>> verified (optional).
>>>>
>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>> is your fcst_init time, not your
>>>> fcst_valid time. If you are interested in verifying the forecast
five hours
>>>> into your 48-hour run, then your fcst_lead
>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>
>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary number
of digits long.
>>>> Typically, for wind ground-based
>>>> measurements, the fcst_field level looks something like "Z10" for
a sensor at
>>>> 10m above the ground. Unfortunately, MET
>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>> it must be post-processed using either
>>>> the WRF Post Processor tool (WPP) or p_interp. This process will
put the data
>>>> onto levels in meters and/or pressure,
>>>> both of which MET can read.
>>>>
>>>> Please let me know if you have any more questions.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>>>>
>>>>> Stat.
>>>>>
>>>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>>>> concatenated into a single file, 48 time steps in a single file.
(Perhaps
>>>>> that's not practical.)
>>>>>
>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>
>>>>> heights. When I set up ASCII2NC, I broke each set of anemometer
data into its
>>>>
>>>>> own netcdf file, each to serve as its own point source.
>>>>>
>>>>>
>>>>> So when running PointStat, here's my understanding:
>>>>>
>>>>> PointStat will only analyze one model time per run.
>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>> instance, if my WRF run is initiated with NAM data produced on
the
>>>>> 20110625_1200
>>>>>
>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>>>> 20110625_1700.
>>>>> The purpose of specifying a temporal range of observation values
is for
>>>>> interpolation across the temporal dimension.
>>>>> The model time that's compared to observation time is the lead
time.
>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>> incorrect, please let me know.
>>>>>
>>>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>>>
>>>>> most attractive to me is the ZNNN for a straightforward vertical
level. Is
>>>>> that
>>>>>
>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it be
more than
>>>>> three
>>>>>
>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>> includes
>>>>>
>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>
>>>>> eta-levels which is more related to pressure levels. Does that
matter?
>>>>>
>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>
>>>>> control.
>>>>>
>>>>> Thanks again for all your assistance!
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message ----
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>>> I do want to interpolate to the height of the sensor, so could
you recommend
>>>> a
>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>> you
>>>>>> refer me to a link or reference to these message types?
>>>>>
>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>> reference on the message types:
>>>>>
>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>
>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>
>>>>> produced by NCEP. These observations
>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>> applied to all observations, regardless of
>>>>> source.
>>>>>
>>>>>
>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>> sounds
>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>> so, what should column 9 have?
>>>>>
>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>
>>>>>
>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>> calculate its statistics from that?
>>>>>
>>>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>>>> model data files contain single 48 hour
>>>>> forecasts, run Point-Stat once for each model file. Each time,
you will pass
>>>>> the model data file you want to verify,
>>>>> the single observation file and the configuration file. In the
configuration
>>>>> file, you should decide the time window
>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>
>>>>> are beg_ds and end_ds. You could also
>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>
>>>>> If you have multiple forecasts in a single model data file, you
will need to
>>>>> use
>>>>>
>>>>> the command line parameter fcst_valid
>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>> Point-Stat must still be run once for each valid
>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>> parameter.
>>>>>
>>>>> Does that make sense? Please let me know if you have questions.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>
>>>>>
>>>>>> months. So I'll have one six-month file for observations. I
want to use
>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>> correct?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message ----
>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> Thanks Paul,
>>>>>>
>>>>>> That's very helpful.
>>>>>>
>>>>>> I do want to interpolate to the height of the sensor, so could
you recommend a
>>>>>
>>>>>
>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>
>>>>>>
>>>>>> refer me to a link or reference to these message types?
>>>>>>
>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>
>>>>>>
>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>
>>>>>> so, what should column 9 have?
>>>>>>
>>>>>> Thanks again,
>>>>>> David
>>>>>>
>>>>>>
>>>>>> ----- Original Message ----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>> speed
>>>>>>
>>>>>>
>>>>>> at a fixed height above the ground. I
>>>>>> would recommend putting -9999 in that column, since that is the
value that MET
>>>>>
>>>>>
>>>>>> uses for bad data or N/A.
>>>>>>
>>>>>> You should use column 6 to enter the elevation of your sensor,
but be advised
>>>>
>>>>>> that if you use a "surface" message type
>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>
>>>>>>
>>>>>> enter in column 6. If you want MET to
>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>
>>>>>>
>>>>>> I hope this is clear. Please let me
>>>>>> know if you have any questions.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>> Queue: met_help
>>>>>>> Subject: formatting ascii2nc
>>>>>>> Owner: Nobody
>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>> Status: new
>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>
>>>>>>>
>>>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>
>>>>>
>>>>>>> is
>>>>>>>
>>>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David Bryan
>>>>>>>
>>>>>>
>
------------------------------------------------
Subject: formatting ascii2nc
From: Paul Oldenburg
Time: Mon Jul 11 15:13:10 2011
model = "WRF";
beg_ds = -600;
end_ds = 600;
//fcst_field[] = [ "PRES/Z3","TMP/Z3","WIND/Z50","WIND/Z80"];
fcst_field[] = [ "PRES/Z0", "TMP/Z2" ];
obs_field[] = [ ];
//fcst_thresh[] = [ "ge87000 le108330", "ge233.15 le322.15", "ge0
le141.33", "ge0 le141.33" ];
fcst_thresh[] = [ "ge87000 le108330", "ge233.15 le322.15" ];
obs_thresh[] = [];
fcst_wind_thresh[] = [ "NA" ];
obs_wind_thresh[] = [];
//message_type[] = [ "PROFLR" ];
message_type[] = [ "ADPSFC" ];
mask_grid[] = [ "FULL" ];
mask_poly[] = [];
mask_sid = "";
ci_alpha[] = [ 0.05 ];
boot_interval = 1;
boot_rep_prop = 1.0;
n_boot_rep = 1000;
boot_rng = "mt19937";
boot_seed = "";
interp_method[] = [ "UW_MEAN" ];
interp_width[] = [ 1 ];
interp_thresh = 1.0;
output_flag[] = [ 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 ];
rank_corr_flag = 1;
grib_ptv = 2;
//tmp_dir = "/home/WRF/Development/David_Bryan/ASCII2NC/tmp/";
tmp_dir = "/tmp";
output_prefix = "";
version = "V3.0";
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: John Halley Gotway
Time: Mon Jul 11 15:36:25 2011
Hello David,
My name is John Halley Gotway. I work on MET development and support
with Paul, and he and I were just discussing your questions about
running Point-Stat. Paul's helped you with verifying surface
temperature and pressure, and it sounds like you're also interested in
verifying winds. Looking at the sample data you sent Paul, I noticed
two things:
(1) You wind observations appear to be at 50 and 80 meters - I
assume that's on a wind turbine.
(2) Using wgrib to look at your model output, I see that you have
forecasts of U and V, but not at vertical heights above ground.
We usually have the opposite problem - plenty of forecasts but no
observations for verification. But in your case we have observations
at 50 and 80 meters but no forecasts to verify! I assume that
you're using the WRF PostProcessor to post-process your WRF output.
Unfortunately, the released version of WPP does not create output for
winds at the AGL heights you'd need to for winds. However, I
worked on an extension for WPP to output these fields. We're in a bit
of flux right now as we transition from WPP to the Unified
PostProcessor (UPP). But for now, if you'd like, I could point you to
a version of WPP that does generate this AGL output:
ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
However, please be aware that this is development code, as-is, and is
not part of the officially supported release. But I have passed this
along to the UPP developers for possible inclusion in a
future release.
So, assuming that you're able to generate U and V forecast output at
50-meters and 80-meters, here's what you might try in you Point-Stat
config file:
fcst_field[] = [ "WIND/Z50", "WIND/Z80" ];
obs_field[] = [];
message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it just
needs to match what you put in the ASCII observation data. And it
probably shouldn't be ADPSFC since these are not "surface" obs
This will verify wind speed directly at 50 and 80 meters.
Unfortunately, Point-Stat can't verify wind direction directly.
Instead, you could use it to dump out VL1L2 lines and verify it in the
STAT-Analysis tool. But to do so, you'd need you observations to be U
and V vectors (instead of wind speed and direction).
So one option would be for you to back out the U and V vectors from
wind speed and direction (it's just some simple trig to do so). And
then include the U and V observations in your ASCII observation
files. Then you could verify the following in Point-Stat:
fcst_field[] = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
"WIND/Z50", "WIND/Z80" ];
obs_field[] = [];
message_type[] = [ "PROFLR" ];
This will verify U, V, and wind speed directly. And, if you have the
VL1L2 flag turned on in the output_flag in the Point-Stat config file,
you'll also get VL1L2 lines in your output. Then, you
could use STAT-Analysis to look at wind direction errors. If you
decide to go that route, and need help, just let us know. Here's an
example of this from the recent MET tutorial:
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
Clear as mud?
Just let us know if more issues arise. We're happy to help.
Thanks,
John
On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> David,
>
> I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
> obs, by the way. It matches the "surface" fields in the model data
file against your original PROFLR obs. I attached
> the version I did intend to send with this email.
>
> Paul
>
>
> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>> David,
>>
>> I'm still working on your wind verification problem.
>>
>> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
>> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>>
>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>
>> Point_stat can only match forecast/obs pairs in three ways:
>>
>> 1. by height in meters for non-"surface" message type obs and
model records
>> 2. by pressure in hPa for non-"surface" message type obs and model
records
>> 3. by pairing "surface" message type obs (i.e. ADPSFC and SFCSHP)
to surface model records
>>
>> For surface level matching, you must specify the "surface" height
in the fcst_field of the config file: 0m for PRES, 2m
>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in the
12th wgrib field:
>>
>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
>> ...
>>
>> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>>
>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>
>> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
>> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
>> the observation level is really dictated by the information in the
model GRIB file. In this case, the level there is
>> surface.
>>
>> Unfortunately, if this is all the obs that you have, you only get 3
matched pairs for each field. You really only have
>> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
>> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
>> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
>> and generate aggregated statistics.
>>
>> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
>> send you an update.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt -fcst_valid
20100617_13 -outdir . -v 2
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=18446744071635082062
>> Forecast File: NREL10061712.grib
>> Climatology File: none
>> Configuration File: PSconfig2.txt
>> Observation File: A9_sfc.nc
>>
>>
--------------------------------------------------------------------------------
>>
>> Reading records for PRES/Z0.
>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>
>>
--------------------------------------------------------------------------------
>>
>> Reading records for TMP/Z2.
>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>
>>
--------------------------------------------------------------------------------
>>
>> Searching 184440 observations from 30740 PrepBufr messages.
>>
>>
--------------------------------------------------------------------------------
>>
>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>> using 3 pairs.
>> Computing Categorical Statistics.
>> Computing Continuous Statistics.
>> Computing Scalar Partial Sums.
>>
>>
--------------------------------------------------------------------------------
>>
>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
>> using 3 pairs.
>> Computing Categorical Statistics.
>> Computing Continuous Statistics.
>> Computing Scalar Partial Sums.
>>
>>
--------------------------------------------------------------------------------
>>
>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>
>>
>>
>>
>>
>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> I've posted the data on your ftp site under the directory,
Bryan_data.
>>>
>>> The contents should be self explanatory, largely. I included two
versions of the config file I tried. Also, I included the original
source of the data, met2010.csv.
>>>
>>> Thanks,
>>> David Bryan
>>>
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Friday, July 8, 2011 5:37 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>>> the following instructions?
>>>
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> Below are outputs from two attempts to run Point_Stat.
>>>>
>>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>>
>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>>
>>>> Similarly, I had assumed, if the netcdf file produced by ASCII2NC
had wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>
>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>> -fcst_valid 20100617_17 \
>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>> -v 9
>>>>
>>>>
>>>> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
>>>> e verified using point_stat.
>>>>
>>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>> GSL_RNG_TYPE=mt19937
>>>> GSL_RNG_SEED=18446744072421513038
>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>> Climatology File: none
>>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Reading records for MISSING/Z3.
>>>>
>>>>
>>>> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Cc:
>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> The point_stat usage statement indicates that the arguments
should be specified in the following order. Please try this
>>>> (you should be able just copy/paste):
>>>>
>>>> ./point_stat \
>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>> -fcst_valid 20100617_17 \
>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>> -v 9
>>>>
>>>> I think that may be the cause of your problem.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> I've attached my config file.
>>>>>
>>>>> When I tried running Point Stat, I got the error message below.
I'm not clear
>>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>>> that the grib files are produced by the WRF post processor
(though they are
>>>>> concatenated, using Linux cat). The text in the message isn't
in the config
>>>>> file.
>>>>>
>>>>> Any suggestions would be welcomed.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>
>>>>>
>>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>> grib"
>>>>>
>>>>> line = 1
>>>>>
>>>>> econfig_column = 7
>>>>>
>>>>> text = "^"
>>>>>
>>>>>
>>>>>
>>>>> GRIB
>>>>> ____
>>>>>
>>>>>
>>>>> ----- Original Message ----
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> Regarding your observations, it might be easiest to include all
of your
>>>>> observations into a single NetCDF file. You can
>>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>>> ascii2nc. Then, you can use the mask_sid
>>>>> option in the point_stat configuration file to specify which
stations obs you
>>>>> want to use. Otherwise, you can use both
>>>>> of your current NetCDF obs files in a single point_stat call by
passing one as
>>>>> the "obs_file" and the other one using
>>>>> the -point_obs optional argument.
>>>>>
>>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>>> statement, which you can see by running
>>>>> point_stat with no arguments:
>>>>>
>>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>>> to be verified (optional).
>>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>>>> verified (optional).
>>>>>
>>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>>> is your fcst_init time, not your
>>>>> fcst_valid time. If you are interested in verifying the
forecast five hours
>>>>> into your 48-hour run, then your fcst_lead
>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>
>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary number
of digits long.
>>>>> Typically, for wind ground-based
>>>>> measurements, the fcst_field level looks something like "Z10"
for a sensor at
>>>>> 10m above the ground. Unfortunately, MET
>>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>>> it must be post-processed using either
>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will put the data
>>>>> onto levels in meters and/or pressure,
>>>>> both of which MET can read.
>>>>>
>>>>> Please let me know if you have any more questions.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>>>>>
>>>>>> Stat.
>>>>>>
>>>>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>>>>> concatenated into a single file, 48 time steps in a single
file. (Perhaps
>>>>>> that's not practical.)
>>>>>>
>>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>>
>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer data into its
>>>>>
>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>
>>>>>>
>>>>>> So when running PointStat, here's my understanding:
>>>>>>
>>>>>> PointStat will only analyze one model time per run.
>>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>>> instance, if my WRF run is initiated with NAM data produced on
the
>>>>>> 20110625_1200
>>>>>>
>>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>>>>> 20110625_1700.
>>>>>> The purpose of specifying a temporal range of observation
values is for
>>>>>> interpolation across the temporal dimension.
>>>>>> The model time that's compared to observation time is the lead
time.
>>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>>> incorrect, please let me know.
>>>>>>
>>>>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>>>>
>>>>>> most attractive to me is the ZNNN for a straightforward
vertical level. Is
>>>>>> that
>>>>>>
>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it
be more than
>>>>>> three
>>>>>>
>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>>> includes
>>>>>>
>>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>>
>>>>>> eta-levels which is more related to pressure levels. Does that
matter?
>>>>>>
>>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>>
>>>>>> control.
>>>>>>
>>>>>> Thanks again for all your assistance!
>>>>>>
>>>>>> David
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message ----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>>> I do want to interpolate to the height of the sensor, so could
you recommend
>>>>> a
>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>>> you
>>>>>>> refer me to a link or reference to these message types?
>>>>>>
>>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>>> reference on the message types:
>>>>>>
>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>
>>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>>
>>>>>> produced by NCEP. These observations
>>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>>> applied to all observations, regardless of
>>>>>> source.
>>>>>>
>>>>>>
>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>>> sounds
>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>> so, what should column 9 have?
>>>>>>
>>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>>
>>>>>>
>>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>>> calculate its statistics from that?
>>>>>>
>>>>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>>>>> model data files contain single 48 hour
>>>>>> forecasts, run Point-Stat once for each model file. Each time,
you will pass
>>>>>> the model data file you want to verify,
>>>>>> the single observation file and the configuration file. In the
configuration
>>>>>> file, you should decide the time window
>>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>>
>>>>>> are beg_ds and end_ds. You could also
>>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>>
>>>>>> If you have multiple forecasts in a single model data file, you
will need to
>>>>>> use
>>>>>>
>>>>>> the command line parameter fcst_valid
>>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>>> Point-Stat must still be run once for each valid
>>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>>> parameter.
>>>>>>
>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>>
>>>>>>
>>>>>>> months. So I'll have one six-month file for observations. I
want to use
>>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>>> correct?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> Thanks Paul,
>>>>>>>
>>>>>>> That's very helpful.
>>>>>>>
>>>>>>> I do want to interpolate to the height of the sensor, so could
you recommend a
>>>>>>
>>>>>>
>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>>
>>>>>>>
>>>>>>> refer me to a link or reference to these message types?
>>>>>>>
>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>>
>>>>>>>
>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>
>>>>>>> so, what should column 9 have?
>>>>>>>
>>>>>>> Thanks again,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>>> speed
>>>>>>>
>>>>>>>
>>>>>>> at a fixed height above the ground. I
>>>>>>> would recommend putting -9999 in that column, since that is
the value that MET
>>>>>>
>>>>>>
>>>>>>> uses for bad data or N/A.
>>>>>>>
>>>>>>> You should use column 6 to enter the elevation of your sensor,
but be advised
>>>>>
>>>>>>> that if you use a "surface" message type
>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>>
>>>>>>>
>>>>>>> enter in column 6. If you want MET to
>>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>>
>>>>>>>
>>>>>>> I hope this is clear. Please let me
>>>>>>> know if you have any questions.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>> Queue: met_help
>>>>>>>> Subject: formatting ascii2nc
>>>>>>>> Owner: Nobody
>>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>>> Status: new
>>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>
>>>>>>>>
>>>>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>>
>>>>>>
>>>>>>>> is
>>>>>>>>
>>>>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David Bryan
>>>>>>>>
>>>>>>>
>>
>
>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Tue Jul 12 13:03:03 2011
John & Paul,
Thanks for your reply and attention.
MET Tools will accept forecasts in NetCDF format (ie, normal wrfout
files), correct? And such files should include wind above ground,
right?
If so, I'll simply quit doing any post-processing grib conversion
and use WRF's output.
Once I have some output in NetCDF, I'll look into Paul's scripting
suggestions.
Thanks,
David Bryan
From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Monday, July 11, 2011 7:06 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
Hello David,
My name is John Halley Gotway. I work on MET development and support
with Paul, and he and I were just discussing your questions about
running Point-Stat. Paul's helped you with verifying surface
temperature and pressure, and it sounds like you're also interested in
verifying winds. Looking at the sample data you sent Paul, I noticed
two things:
(1) You wind observations appear to be at 50 and 80 meters - I
assume that's on a wind turbine.
(2) Using wgrib to look at your model output, I see that you have
forecasts of U and V, but not at vertical heights above ground.
We usually have the opposite problem - plenty of forecasts but no
observations for verification. But in your case we have observations
at 50 and 80 meters but no forecasts to verify! I assume that
you're using the WRF PostProcessor to post-process your WRF output.
Unfortunately, the released version of WPP does not create output for
winds at the AGL heights you'd need to for winds. However, I
worked on an extension for WPP to output these fields. We're in a bit
of flux right now as we transition from WPP to the Unified
PostProcessor (UPP). But for now, if you'd like, I could point you to
a version of WPP that does generate this AGL output:
ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
However, please be aware that this is development code, as-is, and is
not part of the officially supported release. But I have passed this
along to the UPP developers for possible inclusion in a
future release.
So, assuming that you're able to generate U and V forecast output at
50-meters and 80-meters, here's what you might try in you Point-Stat
config file:
fcst_field[] = [ "WIND/Z50", "WIND/Z80" ];
obs_field[] = [];
message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it just
needs to match what you put in the ASCII observation data. And it
probably shouldn't be ADPSFC since these are not "surface" obs
This will verify wind speed directly at 50 and 80 meters.
Unfortunately, Point-Stat can't verify wind direction directly.
Instead, you could use it to dump out VL1L2 lines and verify it in the
STAT-Analysis tool. But to do so, you'd need you observations to be U
and V vectors (instead of wind speed and direction).
So one option would be for you to back out the U and V vectors from
wind speed and direction (it's just some simple trig to do so). And
then include the U and V observations in your ASCII observation
files. Then you could verify the following in Point-Stat:
fcst_field[] = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
"WIND/Z50", "WIND/Z80" ];
obs_field[] = [];
message_type[] = [ "PROFLR" ];
This will verify U, V, and wind speed directly. And, if you have the
VL1L2 flag turned on in the output_flag in the Point-Stat config file,
you'll also get VL1L2 lines in your output. Then, you
could use STAT-Analysis to look at wind direction errors. If you
decide to go that route, and need help, just let us know. Here's an
example of this from the recent MET tutorial:
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
Clear as mud?
Just let us know if more issues arise. We're happy to help.
Thanks,
John
On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> David,
>
> I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
> obs, by the way. It matches the "surface" fields in the model data
file against your original PROFLR obs. I attached
> the version I did intend to send with this email.
>
> Paul
>
>
> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>> David,
>>
>> I'm still working on your wind verification problem.
>>
>> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
>> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>>
>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>
>> Point_stat can only match forecast/obs pairs in three ways:
>>
>> 1. by height in meters for non-"surface" message type obs and
model records
>> 2. by pressure in hPa for non-"surface" message type obs and model
records
>> 3. by pairing "surface" message type obs (i.e. ADPSFC and SFCSHP)
to surface model records
>>
>> For surface level matching, you must specify the "surface" height
in the fcst_field of the config file: 0m for PRES, 2m
>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in the
12th wgrib field:
>>
>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
>> ...
>>
>> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>>
>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>
>> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
>> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
>> the observation level is really dictated by the information in the
model GRIB file. In this case, the level there is
>> surface.
>>
>> Unfortunately, if this is all the obs that you have, you only get 3
matched pairs for each field. You really only have
>> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
>> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
>> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
>> and generate aggregated statistics.
>>
>> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
>> send you an update.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt -fcst_valid
20100617_13 -outdir . -v 2
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=18446744071635082062
>> Forecast File: NREL10061712.grib
>> Climatology File: none
>> Configuration File: PSconfig2.txt
>> Observation File: A9_sfc.nc
>>
>>
--------------------------------------------------------------------------------
>>
>> Reading records for PRES/Z0.
>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>
>>
--------------------------------------------------------------------------------
>>
>> Reading records for TMP/Z2.
>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>
>>
--------------------------------------------------------------------------------
>>
>> Searching 184440 observations from 30740 PrepBufr messages.
>>
>>
--------------------------------------------------------------------------------
>>
>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>> using 3 pairs.
>> Computing Categorical Statistics.
>> Computing Continuous Statistics.
>> Computing Scalar Partial Sums.
>>
>>
--------------------------------------------------------------------------------
>>
>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
>> using 3 pairs.
>> Computing Categorical Statistics.
>> Computing Continuous Statistics.
>> Computing Scalar Partial Sums.
>>
>>
--------------------------------------------------------------------------------
>>
>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>
>>
>>
>>
>>
>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> Paul,
>>>
>>> I've posted the data on your ftp site under the directory,
Bryan_data.
>>>
>>> The contents should be self explanatory, largely. I
included two versions of the config file I tried. Also, I included
the original source of the data, met2010.csv.
>>>
>>> Thanks,
>>> David Bryan
>>>
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Friday, July 8, 2011 5:37 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>>> the following instructions?
>>>
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> Below are outputs from two attempts to run Point_Stat.
>>>>
>>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>>
>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>>
>>>> Similarly, I had assumed, if the netcdf file produced by ASCII2NC
had wind speed and direction and the grib file had U and V wind
components, that POINT_STAT would sort that out. Is that correct?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>
>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>> -fcst_valid 20100617_17 \
>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>> -v 9
>>>>
>>>>
>>>> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
>>>> e verified using point_stat.
>>>>
>>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>> GSL_RNG_TYPE=mt19937
>>>> GSL_RNG_SEED=18446744072421513038
>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>> Climatology File: none
>>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Reading records for MISSING/Z3.
>>>>
>>>>
>>>> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Cc:
>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> The point_stat usage statement indicates that the arguments
should be specified in the following order. Please try this
>>>> (you should be able just copy/paste):
>>>>
>>>> ./point_stat \
>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>> -fcst_valid 20100617_17 \
>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>> -v 9
>>>>
>>>> I think that may be the cause of your problem.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> I've attached my config file.
>>>>>
>>>>> When I tried running Point Stat, I got the error message below.
I'm not clear
>>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>>> that the grib files are produced by the WRF post processor
(though they are
>>>>> concatenated, using Linux cat). The text in the message isn't
in the config
>>>>> file.
>>>>>
>>>>> Any suggestions would be welcomed.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>
>>>>>
>>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>> grib"
>>>>>
>>>>> line = 1
>>>>>
>>>>> econfig_column = 7
>>>>>
>>>>> text = "^"
>>>>>
>>>>>
>>>>>
>>>>> GRIB
>>>>> ____
>>>>>
>>>>>
>>>>> ----- Original Message ----
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> Regarding your observations, it might be easiest to include all
of your
>>>>> observations into a single NetCDF file. You can
>>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>>> ascii2nc. Then, you can use the mask_sid
>>>>> option in the point_stat configuration file to specify which
stations obs you
>>>>> want to use. Otherwise, you can use both
>>>>> of your current NetCDF obs files in a single point_stat call by
passing one as
>>>>> the "obs_file" and the other one using
>>>>> the -point_obs optional argument.
>>>>>
>>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>>> statement, which you can see by running
>>>>> point_stat with no arguments:
>>>>>
>>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>>> to be verified (optional).
>>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>>>> verified (optional).
>>>>>
>>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>>> is your fcst_init time, not your
>>>>> fcst_valid time. If you are interested in verifying the
forecast five hours
>>>>> into your 48-hour run, then your fcst_lead
>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>
>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary number
of digits long.
>>>>> Typically, for wind ground-based
>>>>> measurements, the fcst_field level looks something like "Z10"
for a sensor at
>>>>> 10m above the ground. Unfortunately, MET
>>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>>> it must be post-processed using either
>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will put the data
>>>>> onto levels in meters and/or pressure,
>>>>> both of which MET can read.
>>>>>
>>>>> Please let me know if you have any more questions.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> Thanks again for your help. Now, I'm on to creating the config
file for Point
>>>>>
>>>>>> Stat.
>>>>>>
>>>>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>>>>> concatenated into a single file, 48 time steps in a single
file. (Perhaps
>>>>>> that's not practical.)
>>>>>>
>>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>>
>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer data into its
>>>>>
>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>
>>>>>>
>>>>>> So when running PointStat, here's my understanding:
>>>>>>
>>>>>> PointStat will only analyze one model time per run.
>>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>>> instance, if my WRF run is initiated with NAM data produced on
the
>>>>>> 20110625_1200
>>>>>>
>>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>>>>> 20110625_1700.
>>>>>> The purpose of specifying a temporal range of observation
values is for
>>>>>> interpolation across the temporal dimension.
>>>>>> The model time that's compared to observation time is the lead
time.
>>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>>> incorrect, please let me know.
>>>>>>
>>>>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>>>>
>>>>>> most attractive to me is the ZNNN for a straightforward
vertical level. Is
>>>>>> that
>>>>>>
>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it
be more than
>>>>>> three
>>>>>>
>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>>> includes
>>>>>>
>>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>>
>>>>>> eta-levels which is more related to pressure levels. Does that
matter?
>>>>>>
>>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>>
>>>>>> control.
>>>>>>
>>>>>> Thanks again for all your assistance!
>>>>>>
>>>>>> David
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message ----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>>> I do want to interpolate to the height of the sensor, so could
you recommend
>>>>> a
>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>>> you
>>>>>>> refer me to a link or reference to these message types?
>>>>>>
>>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>>> reference on the message types:
>>>>>>
>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>
>>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>>
>>>>>> produced by NCEP. These observations
>>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>>> applied to all observations, regardless of
>>>>>> source.
>>>>>>
>>>>>>
>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>>> sounds
>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>> so, what should column 9 have?
>>>>>>
>>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>>
>>>>>>
>>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>>> calculate its statistics from that?
>>>>>>
>>>>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>>>>> model data files contain single 48 hour
>>>>>> forecasts, run Point-Stat once for each model file. Each time,
you will pass
>>>>>> the model data file you want to verify,
>>>>>> the single observation file and the configuration file. In the
configuration
>>>>>> file, you should decide the time window
>>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>>
>>>>>> are beg_ds and end_ds. You could also
>>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>>
>>>>>> If you have multiple forecasts in a single model data file, you
will need to
>>>>>> use
>>>>>>
>>>>>> the command line parameter fcst_valid
>>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>>> Point-Stat must still be run once for each valid
>>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>>> parameter.
>>>>>>
>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>>
>>>>>>
>>>>>>> months. So I'll have one six-month file for observations. I
want to use
>>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>>> correct?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> Thanks Paul,
>>>>>>>
>>>>>>> That's very helpful.
>>>>>>>
>>>>>>> I do want to interpolate to the height of the sensor, so could
you recommend a
>>>>>>
>>>>>>
>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>>
>>>>>>>
>>>>>>> refer me to a link or reference to these message types?
>>>>>>>
>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>>
>>>>>>>
>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>
>>>>>>> so, what should column 9 have?
>>>>>>>
>>>>>>> Thanks again,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>>> speed
>>>>>>>
>>>>>>>
>>>>>>> at a fixed height above the ground. I
>>>>>>> would recommend putting -9999 in that column, since that is
the value that MET
>>>>>>
>>>>>>
>>>>>>> uses for bad data or N/A.
>>>>>>>
>>>>>>> You should use column 6 to enter the elevation of your sensor,
but be advised
>>>>>
>>>>>>> that if you use a "surface" message type
>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>>
>>>>>>>
>>>>>>> enter in column 6. If you want MET to
>>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>>
>>>>>>>
>>>>>>> I hope this is clear. Please let me
>>>>>>> know if you have any questions.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>> Queue: met_help
>>>>>>>> Subject: formatting ascii2nc
>>>>>>>> Owner: Nobody
>>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>>> Status: new
>>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>
>>>>>>>>
>>>>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>>
>>>>>>
>>>>>>>> is
>>>>>>>>
>>>>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David Bryan
>>>>>>>>
>>>>>>>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: Paul Oldenburg
Time: Tue Jul 12 13:11:23 2011
David,
MET cannot read the NetCDF output of WRF directly. The reason is that
WRF NetCDF data is on staggered Arakawa
horizontal grids and on hybrid vertical levels. Observation data
always uses pressure, elevation or height for vertical
levels. In your case, I think the easiest path is to continue to
post-process using WPP as you have been doing (I
assume), and then use the module that John sent to generate wind data
at 50m and 80m. Does that make sense?
Paul
On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> John & Paul,
>
> Thanks for your reply and attention.
>
> MET Tools will accept forecasts in NetCDF format (ie, normal wrfout
files), correct? And such files should include wind above ground,
right?
>
> If so, I'll simply quit doing any post-processing grib conversion
and use WRF's output.
>
> Once I have some output in NetCDF, I'll look into Paul's scripting
suggestions.
>
> Thanks,
> David Bryan
>
>
>
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Monday, July 11, 2011 7:06 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> Hello David,
>
> My name is John Halley Gotway. I work on MET development and
support with Paul, and he and I were just discussing your questions
about running Point-Stat. Paul's helped you with verifying surface
> temperature and pressure, and it sounds like you're also interested
in verifying winds. Looking at the sample data you sent Paul, I
noticed two things:
> (1) You wind observations appear to be at 50 and 80 meters - I
assume that's on a wind turbine.
> (2) Using wgrib to look at your model output, I see that you have
forecasts of U and V, but not at vertical heights above ground.
>
> We usually have the opposite problem - plenty of forecasts but no
observations for verification. But in your case we have observations
at 50 and 80 meters but no forecasts to verify! I assume that
> you're using the WRF PostProcessor to post-process your WRF output.
Unfortunately, the released version of WPP does not create output for
winds at the AGL heights you'd need to for winds. However, I
> worked on an extension for WPP to output these fields. We're in a
bit of flux right now as we transition from WPP to the Unified
PostProcessor (UPP). But for now, if you'd like, I could point you to
> a version of WPP that does generate this AGL output:
> ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>
> However, please be aware that this is development code, as-is, and
is not part of the officially supported release. But I have passed
this along to the UPP developers for possible inclusion in a
> future release.
>
> So, assuming that you're able to generate U and V forecast output at
50-meters and 80-meters, here's what you might try in you Point-Stat
config file:
>
> fcst_field[] = [ "WIND/Z50", "WIND/Z80" ];
> obs_field[] = [];
> message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just needs to match what you put in the ASCII observation data. And
it probably shouldn't be ADPSFC since these are not "surface" obs
>
> This will verify wind speed directly at 50 and 80 meters.
Unfortunately, Point-Stat can't verify wind direction directly.
Instead, you could use it to dump out VL1L2 lines and verify it in the
> STAT-Analysis tool. But to do so, you'd need you observations to be
U and V vectors (instead of wind speed and direction).
>
> So one option would be for you to back out the U and V vectors from
wind speed and direction (it's just some simple trig to do so). And
then include the U and V observations in your ASCII observation
> files. Then you could verify the following in Point-Stat:
>
> fcst_field[] = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
"WIND/Z50", "WIND/Z80" ];
> obs_field[] = [];
> message_type[] = [ "PROFLR" ];
>
> This will verify U, V, and wind speed directly. And, if you have
the VL1L2 flag turned on in the output_flag in the Point-Stat config
file, you'll also get VL1L2 lines in your output. Then, you
> could use STAT-Analysis to look at wind direction errors. If you
decide to go that route, and need help, just let us know. Here's an
example of this from the recent MET tutorial:
>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>
> Clear as mud?
>
> Just let us know if more issues arise. We're happy to help.
>
> Thanks,
> John
>
>
>
> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> David,
>>
>> I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
>> obs, by the way. It matches the "surface" fields in the model data
file against your original PROFLR obs. I attached
>> the version I did intend to send with this email.
>>
>> Paul
>>
>>
>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>> David,
>>>
>>> I'm still working on your wind verification problem.
>>>
>>> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
>>> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>>>
>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>
>>> Point_stat can only match forecast/obs pairs in three ways:
>>>
>>> 1. by height in meters for non-"surface" message type obs and
model records
>>> 2. by pressure in hPa for non-"surface" message type obs and
model records
>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and SFCSHP)
to surface model records
>>>
>>> For surface level matching, you must specify the "surface" height
in the fcst_field of the config file: 0m for PRES, 2m
>>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in the
12th wgrib field:
>>>
>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
>>> ...
>>>
>>> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>>>
>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>
>>> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
>>> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
>>> the observation level is really dictated by the information in the
model GRIB file. In this case, the level there is
>>> surface.
>>>
>>> Unfortunately, if this is all the obs that you have, you only get
3 matched pairs for each field. You really only have
>>> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
>>> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
>>> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
>>> and generate aggregated statistics.
>>>
>>> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
>>> send you an update.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt -fcst_valid
20100617_13 -outdir . -v 2
>>> GSL_RNG_TYPE=mt19937
>>> GSL_RNG_SEED=18446744071635082062
>>> Forecast File: NREL10061712.grib
>>> Climatology File: none
>>> Configuration File: PSconfig2.txt
>>> Observation File: A9_sfc.nc
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Reading records for PRES/Z0.
>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Reading records for TMP/Z2.
>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>>> using 3 pairs.
>>> Computing Categorical Statistics.
>>> Computing Continuous Statistics.
>>> Computing Scalar Partial Sums.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
>>> using 3 pairs.
>>> Computing Categorical Statistics.
>>> Computing Continuous Statistics.
>>> Computing Scalar Partial Sums.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>>
>>>
>>>
>>>
>>>
>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> I've posted the data on your ftp site under the directory,
Bryan_data.
>>>>
>>>> The contents should be self explanatory, largely. I included two
versions of the config file I tried. Also, I included the original
source of the data, met2010.csv.
>>>>
>>>> Thanks,
>>>> David Bryan
>>>>
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>>>> the following instructions?
>>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>
>>>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>>>
>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>>>
>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC had wind speed and direction and the grib file had U and V
wind components, that POINT_STAT would sort that out. Is that
correct?
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>
>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>> -fcst_valid 20100617_17 \
>>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>>> -v 9
>>>>>
>>>>>
>>>>> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
>>>>> e verified using point_stat.
>>>>>
>>>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>> GSL_RNG_TYPE=mt19937
>>>>> GSL_RNG_SEED=18446744072421513038
>>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>> Climatology File: none
>>>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Reading records for MISSING/Z3.
>>>>>
>>>>>
>>>>> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Cc:
>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> The point_stat usage statement indicates that the arguments
should be specified in the following order. Please try this
>>>>> (you should be able just copy/paste):
>>>>>
>>>>> ./point_stat \
>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>> -fcst_valid 20100617_17 \
>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>> -v 9
>>>>>
>>>>> I think that may be the cause of your problem.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> I've attached my config file.
>>>>>>
>>>>>> When I tried running Point Stat, I got the error message below.
I'm not clear
>>>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>>>> that the grib files are produced by the WRF post processor
(though they are
>>>>>> concatenated, using Linux cat). The text in the message isn't
in the config
>>>>>> file.
>>>>>>
>>>>>> Any suggestions would be welcomed.
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>
>>>>>>
>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>
>>>>>>
>>>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>> grib"
>>>>>>
>>>>>> line = 1
>>>>>>
>>>>>> econfig_column = 7
>>>>>>
>>>>>> text = "^"
>>>>>>
>>>>>>
>>>>>>
>>>>>> GRIB
>>>>>> ____
>>>>>>
>>>>>>
>>>>>> ----- Original Message ----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> Regarding your observations, it might be easiest to include all
of your
>>>>>> observations into a single NetCDF file. You can
>>>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>> option in the point_stat configuration file to specify which
stations obs you
>>>>>> want to use. Otherwise, you can use both
>>>>>> of your current NetCDF obs files in a single point_stat call by
passing one as
>>>>>> the "obs_file" and the other one using
>>>>>> the -point_obs optional argument.
>>>>>>
>>>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>>>> statement, which you can see by running
>>>>>> point_stat with no arguments:
>>>>>>
>>>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>>>> to be verified (optional).
>>>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>>>>> verified (optional).
>>>>>>
>>>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>>>> is your fcst_init time, not your
>>>>>> fcst_valid time. If you are interested in verifying the
forecast five hours
>>>>>> into your 48-hour run, then your fcst_lead
>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>
>>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary number
of digits long.
>>>>>> Typically, for wind ground-based
>>>>>> measurements, the fcst_field level looks something like "Z10"
for a sensor at
>>>>>> 10m above the ground. Unfortunately, MET
>>>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>>>> it must be post-processed using either
>>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will put the data
>>>>>> onto levels in meters and/or pressure,
>>>>>> both of which MET can read.
>>>>>>
>>>>>> Please let me know if you have any more questions.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> Thanks again for your help. Now, I'm on to creating the
config file for Point
>>>>>>
>>>>>>> Stat.
>>>>>>>
>>>>>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>>>>>> concatenated into a single file, 48 time steps in a single
file. (Perhaps
>>>>>>> that's not practical.)
>>>>>>>
>>>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>>>
>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer data into its
>>>>>>
>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>
>>>>>>>
>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>
>>>>>>> PointStat will only analyze one model time per run.
>>>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>>>> instance, if my WRF run is initiated with NAM data produced on
the
>>>>>>> 20110625_1200
>>>>>>>
>>>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>>>>>> 20110625_1700.
>>>>>>> The purpose of specifying a temporal range of observation
values is for
>>>>>>> interpolation across the temporal dimension.
>>>>>>> The model time that's compared to observation time is the lead
time.
>>>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>>>> incorrect, please let me know.
>>>>>>>
>>>>>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>>>>>
>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical level. Is
>>>>>>> that
>>>>>>>
>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it
be more than
>>>>>>> three
>>>>>>>
>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>>>> includes
>>>>>>>
>>>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>>>
>>>>>>> eta-levels which is more related to pressure levels. Does
that matter?
>>>>>>>
>>>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>>>
>>>>>>> control.
>>>>>>>
>>>>>>> Thanks again for all your assistance!
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend
>>>>>> a
>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>>>> you
>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>
>>>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>>>> reference on the message types:
>>>>>>>
>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>
>>>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>>>
>>>>>>> produced by NCEP. These observations
>>>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>>>> applied to all observations, regardless of
>>>>>>> source.
>>>>>>>
>>>>>>>
>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>>>> sounds
>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>>> so, what should column 9 have?
>>>>>>>
>>>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>>>
>>>>>>>
>>>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>>>> calculate its statistics from that?
>>>>>>>
>>>>>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>>>>>> model data files contain single 48 hour
>>>>>>> forecasts, run Point-Stat once for each model file. Each
time, you will pass
>>>>>>> the model data file you want to verify,
>>>>>>> the single observation file and the configuration file. In
the configuration
>>>>>>> file, you should decide the time window
>>>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>>>
>>>>>>> are beg_ds and end_ds. You could also
>>>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>>>
>>>>>>> If you have multiple forecasts in a single model data file,
you will need to
>>>>>>> use
>>>>>>>
>>>>>>> the command line parameter fcst_valid
>>>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>>>> Point-Stat must still be run once for each valid
>>>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>>>> parameter.
>>>>>>>
>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>>>
>>>>>>>
>>>>>>>> months. So I'll have one six-month file for observations. I
want to use
>>>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>>>> correct?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> Thanks Paul,
>>>>>>>>
>>>>>>>> That's very helpful.
>>>>>>>>
>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend a
>>>>>>>
>>>>>>>
>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>>>
>>>>>>>>
>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>
>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>>>
>>>>>>>>
>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>
>>>>>>>> so, what should column 9 have?
>>>>>>>>
>>>>>>>> Thanks again,
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>>>> speed
>>>>>>>>
>>>>>>>>
>>>>>>>> at a fixed height above the ground. I
>>>>>>>> would recommend putting -9999 in that column, since that is
the value that MET
>>>>>>>
>>>>>>>
>>>>>>>> uses for bad data or N/A.
>>>>>>>>
>>>>>>>> You should use column 6 to enter the elevation of your
sensor, but be advised
>>>>>>
>>>>>>>> that if you use a "surface" message type
>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>>>
>>>>>>>>
>>>>>>>> enter in column 6. If you want MET to
>>>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>>>
>>>>>>>>
>>>>>>>> I hope this is clear. Please let me
>>>>>>>> know if you have any questions.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>>> Queue: met_help
>>>>>>>>> Subject: formatting ascii2nc
>>>>>>>>> Owner: Nobody
>>>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>>>> Status: new
>>>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>>>
>>>>>>>
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David Bryan
>>>>>>>>>
>>>>>>>>
>>>
>>
>>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Tue Jul 12 13:22:30 2011
OK, but will those results at 50 and 80 m be as accurate as they were
before WPP? What about using the other WRF grib converter?
Thanks,
David Bryan
From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Tuesday, July 12, 2011 4:41 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
MET cannot read the NetCDF output of WRF directly. The reason is that
WRF NetCDF data is on staggered Arakawa
horizontal grids and on hybrid vertical levels. Observation data
always uses pressure, elevation or height for vertical
levels. In your case, I think the easiest path is to continue to
post-process using WPP as you have been doing (I
assume), and then use the module that John sent to generate wind data
at 50m and 80m. Does that make sense?
Paul
On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> John & Paul,
>
> Thanks for your reply and attention.
>
> MET Tools will accept forecasts in NetCDF format (ie, normal
wrfout files), correct? And such files should include wind above
ground, right?
>
> If so, I'll simply quit doing any post-processing grib
conversion and use WRF's output.
>
> Once I have some output in NetCDF, I'll look into Paul's
scripting suggestions.
>
> Thanks,
> David Bryan
>
>
>
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Monday, July 11, 2011 7:06 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> Hello David,
>
> My name is John Halley Gotway. I work on MET development and
support with Paul, and he and I were just discussing your questions
about running Point-Stat. Paul's helped you with verifying surface
> temperature and pressure, and it sounds like you're also interested
in verifying winds. Looking at the sample data you sent Paul, I
noticed two things:
> (1) You wind observations appear to be at 50 and 80 meters - I
assume that's on a wind turbine.
> (2) Using wgrib to look at your model output, I see that you have
forecasts of U and V, but not at vertical heights above ground.
>
> We usually have the opposite problem - plenty of forecasts but no
observations for verification. But in your case we have observations
at 50 and 80 meters but no forecasts to verify! I assume that
> you're using the WRF PostProcessor to post-process your WRF output.
Unfortunately, the released version of WPP does not create output for
winds at the AGL heights you'd need to for winds. However, I
> worked on an extension for WPP to output these fields. We're in a
bit of flux right now as we transition from WPP to the Unified
PostProcessor (UPP). But for now, if you'd like, I could point you to
> a version of WPP that does generate this AGL output:
> ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>
> However, please be aware that this is development code, as-is, and
is not part of the officially supported release. But I have passed
this along to the UPP developers for possible inclusion in a
> future release.
>
> So, assuming that you're able to generate U and V forecast output at
50-meters and 80-meters, here's what you might try in you Point-Stat
config file:
>
> fcst_field[] = [ "WIND/Z50", "WIND/Z80" ];
> obs_field[] = [];
> message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just needs to match what you put in the ASCII observation data. And
it probably shouldn't be ADPSFC since these are not "surface" obs
>
> This will verify wind speed directly at 50 and 80 meters.
Unfortunately, Point-Stat can't verify wind direction directly.
Instead, you could use it to dump out VL1L2 lines and verify it in the
> STAT-Analysis tool. But to do so, you'd need you observations to be
U and V vectors (instead of wind speed and direction).
>
> So one option would be for you to back out the U and V vectors from
wind speed and direction (it's just some simple trig to do so). And
then include the U and V observations in your ASCII observation
> files. Then you could verify the following in Point-Stat:
>
> fcst_field[] = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
"WIND/Z50", "WIND/Z80" ];
> obs_field[] = [];
> message_type[] = [ "PROFLR" ];
>
> This will verify U, V, and wind speed directly. And, if you have
the VL1L2 flag turned on in the output_flag in the Point-Stat config
file, you'll also get VL1L2 lines in your output. Then, you
> could use STAT-Analysis to look at wind direction errors. If you
decide to go that route, and need help, just let us know. Here's an
example of this from the recent MET tutorial:
>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>
> Clear as mud?
>
> Just let us know if more issues arise. We're happy to help.
>
> Thanks,
> John
>
>
>
> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> David,
>>
>> I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
>> obs, by the way. It matches the "surface" fields in the model data
file against your original PROFLR obs. I attached
>> the version I did intend to send with this email.
>>
>> Paul
>>
>>
>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>> David,
>>>
>>> I'm still working on your wind verification problem.
>>>
>>> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
>>> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>>>
>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>
>>> Point_stat can only match forecast/obs pairs in three ways:
>>>
>>> 1. by height in meters for non-"surface" message type obs and
model records
>>> 2. by pressure in hPa for non-"surface" message type obs and
model records
>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and SFCSHP)
to surface model records
>>>
>>> For surface level matching, you must specify the "surface" height
in the fcst_field of the config file: 0m for PRES, 2m
>>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in the
12th wgrib field:
>>>
>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
>>> ...
>>>
>>> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>>>
>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>
>>> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
>>> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
>>> the observation level is really dictated by the information in the
model GRIB file. In this case, the level there is
>>> surface.
>>>
>>> Unfortunately, if this is all the obs that you have, you only get
3 matched pairs for each field. You really only have
>>> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
>>> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
>>> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
>>> and generate aggregated statistics.
>>>
>>> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
>>> send you an update.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt -fcst_valid
20100617_13 -outdir . -v 2
>>> GSL_RNG_TYPE=mt19937
>>> GSL_RNG_SEED=18446744071635082062
>>> Forecast File: NREL10061712.grib
>>> Climatology File: none
>>> Configuration File: PSconfig2.txt
>>> Observation File: A9_sfc.nc
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Reading records for PRES/Z0.
>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Reading records for TMP/Z2.
>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>>> using 3 pairs.
>>> Computing Categorical Statistics.
>>> Computing Continuous Statistics.
>>> Computing Scalar Partial Sums.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method UW_MEAN(1),
>>> using 3 pairs.
>>> Computing Categorical Statistics.
>>> Computing Continuous Statistics.
>>> Computing Scalar Partial Sums.
>>>
>>>
--------------------------------------------------------------------------------
>>>
>>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>>
>>>
>>>
>>>
>>>
>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> Paul,
>>>>
>>>> I've posted the data on your ftp site under the directory,
Bryan_data.
>>>>
>>>> The contents should be self explanatory, largely. I
included two versions of the config file I tried. Also, I included
the original source of the data, met2010.csv.
>>>>
>>>> Thanks,
>>>> David Bryan
>>>>
>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> David,
>>>>
>>>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>>>> the following instructions?
>>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>
>>>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>>>
>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>>>
>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC had wind speed and direction and the grib file had U and V
wind components, that POINT_STAT would sort that out. Is that
correct?
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>
>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>> -fcst_valid 20100617_17 \
>>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>>> -v 9
>>>>>
>>>>>
>>>>> ERROR: PointStatConfInfo::process_config() -> the wind direction
field may not b
>>>>> e verified using point_stat.
>>>>>
>>>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>> GSL_RNG_TYPE=mt19937
>>>>> GSL_RNG_SEED=18446744072421513038
>>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>> Climatology File: none
>>>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Reading records for MISSING/Z3.
>>>>>
>>>>>
>>>>> ERROR: read_levels_grib() -> no GRIB records matching MISSING/Z3
found in GRIB f
>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Cc:
>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> The point_stat usage statement indicates that the arguments
should be specified in the following order. Please try this
>>>>> (you should be able just copy/paste):
>>>>>
>>>>> ./point_stat \
>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>> -fcst_valid 20100617_17 \
>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>> -v 9
>>>>>
>>>>> I think that may be the cause of your problem.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> I've attached my config file.
>>>>>>
>>>>>> When I tried running Point Stat, I got the error message below.
I'm not clear
>>>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>>>> that the grib files are produced by the WRF post processor
(though they are
>>>>>> concatenated, using Linux cat). The text in the message isn't
in the config
>>>>>> file.
>>>>>>
>>>>>> Any suggestions would be welcomed.
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>
>>>>>>
>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>
>>>>>>
>>>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>> grib"
>>>>>>
>>>>>> line = 1
>>>>>>
>>>>>> econfig_column = 7
>>>>>>
>>>>>> text = "^"
>>>>>>
>>>>>>
>>>>>>
>>>>>> GRIB
>>>>>> ____
>>>>>>
>>>>>>
>>>>>> ----- Original Message ----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> Regarding your observations, it might be easiest to include all
of your
>>>>>> observations into a single NetCDF file. You can
>>>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>> option in the point_stat configuration file to specify which
stations obs you
>>>>>> want to use. Otherwise, you can use both
>>>>>> of your current NetCDF obs files in a single point_stat call by
passing one as
>>>>>> the "obs_file" and the other one using
>>>>>> the -point_obs optional argument.
>>>>>>
>>>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>>>> statement, which you can see by running
>>>>>> point_stat with no arguments:
>>>>>>
>>>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>>>> to be verified (optional).
>>>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>>>>> verified (optional).
>>>>>>
>>>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>>>> is your fcst_init time, not your
>>>>>> fcst_valid time. If you are interested in verifying the
forecast five hours
>>>>>> into your 48-hour run, then your fcst_lead
>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>
>>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary number
of digits long.
>>>>>> Typically, for wind ground-based
>>>>>> measurements, the fcst_field level looks something like "Z10"
for a sensor at
>>>>>> 10m above the ground. Unfortunately, MET
>>>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>>>> it must be post-processed using either
>>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will put the data
>>>>>> onto levels in meters and/or pressure,
>>>>>> both of which MET can read.
>>>>>>
>>>>>> Please let me know if you have any more questions.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> Thanks again for your help. Now, I'm on to creating the
config file for Point
>>>>>>
>>>>>>> Stat.
>>>>>>>
>>>>>>> My model output is for 48-hour runs of WRF with hourly output.
The output is
>>>>>>> concatenated into a single file, 48 time steps in a single
file. (Perhaps
>>>>>>> that's not practical.)
>>>>>>>
>>>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>>>
>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer data into its
>>>>>>
>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>
>>>>>>>
>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>
>>>>>>> PointStat will only analyze one model time per run.
>>>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>>>> instance, if my WRF run is initiated with NAM data produced on
the
>>>>>>> 20110625_1200
>>>>>>>
>>>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>>>> wanted to analyze five hours into my 48-hour run, my lead time
would be
>>>>>>> 20110625_1700.
>>>>>>> The purpose of specifying a temporal range of observation
values is for
>>>>>>> interpolation across the temporal dimension.
>>>>>>> The model time that's compared to observation time is the lead
time.
>>>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>>>> incorrect, please let me know.
>>>>>>>
>>>>>>> My other main question concerns specifying the levels for each
fcst_field. The
>>>>>>>
>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical level. Is
>>>>>>> that
>>>>>>>
>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it
be more than
>>>>>>> three
>>>>>>>
>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>>>> includes
>>>>>>>
>>>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>>>
>>>>>>> eta-levels which is more related to pressure levels. Does
that matter?
>>>>>>>
>>>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>>>
>>>>>>> control.
>>>>>>>
>>>>>>> Thanks again for all your assistance!
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend
>>>>>> a
>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>>>> you
>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>
>>>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>>>> reference on the message types:
>>>>>>>
>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>
>>>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>>>
>>>>>>> produced by NCEP. These observations
>>>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>>>> applied to all observations, regardless of
>>>>>>> source.
>>>>>>>
>>>>>>>
>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>>>> sounds
>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>>> so, what should column 9 have?
>>>>>>>
>>>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>>>
>>>>>>>
>>>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>>>> calculate its statistics from that?
>>>>>>>
>>>>>>> Point-Stat was designed to be run once for each forecast valid
time. If your
>>>>>>> model data files contain single 48 hour
>>>>>>> forecasts, run Point-Stat once for each model file. Each
time, you will pass
>>>>>>> the model data file you want to verify,
>>>>>>> the single observation file and the configuration file. In
the configuration
>>>>>>> file, you should decide the time window
>>>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>>>
>>>>>>> are beg_ds and end_ds. You could also
>>>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>>>
>>>>>>> If you have multiple forecasts in a single model data file,
you will need to
>>>>>>> use
>>>>>>>
>>>>>>> the command line parameter fcst_valid
>>>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>>>> Point-Stat must still be run once for each valid
>>>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>>>> parameter.
>>>>>>>
>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>>>
>>>>>>>
>>>>>>>> months. So I'll have one six-month file for observations. I
want to use
>>>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>>>> correct?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> Thanks Paul,
>>>>>>>>
>>>>>>>> That's very helpful.
>>>>>>>>
>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend a
>>>>>>>
>>>>>>>
>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>>>
>>>>>>>>
>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>
>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>>>
>>>>>>>>
>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>
>>>>>>>> so, what should column 9 have?
>>>>>>>>
>>>>>>>> Thanks again,
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>>>> speed
>>>>>>>>
>>>>>>>>
>>>>>>>> at a fixed height above the ground. I
>>>>>>>> would recommend putting -9999 in that column, since that is
the value that MET
>>>>>>>
>>>>>>>
>>>>>>>> uses for bad data or N/A.
>>>>>>>>
>>>>>>>> You should use column 6 to enter the elevation of your
sensor, but be advised
>>>>>>
>>>>>>>> that if you use a "surface" message type
>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>>>
>>>>>>>>
>>>>>>>> enter in column 6. If you want MET to
>>>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>>>
>>>>>>>>
>>>>>>>> I hope this is clear. Please let me
>>>>>>>> know if you have any questions.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>>> Queue: met_help
>>>>>>>>> Subject: formatting ascii2nc
>>>>>>>>> Owner: Nobody
>>>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>>>> Status: new
>>>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Could you clarify "8. Level as the pressure level in hPa or
accumulation
>>>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>>>
>>>>>>>
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> an anemometer at a fixed height. What value would I put for
this column?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David Bryan
>>>>>>>>>
>>>>>>>>
>>>
>>
>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: John Halley Gotway
Time: Tue Jul 12 15:47:05 2011
David,
This is John - I wanted to pipe in here.
There is no other "WRF grib" converter available. The other post-
processing option is the WRF-ARW pinterp tool:
http://www.mmm.ucar.edu/wrf/users/download/get_sources.html
It reads WRF-ARW NetCDF out files, destaggers the grid, and
interpolates to pressure levels. However, I took a look at the WRF
registry file for ARW and don't think that WRF itself is able to dump
out winds at vertical levels like 50m and 80m. The vertical
coordinate is the model level, and I don't think it's set up to derive
vertical levels (other than 10m) like that. But I could be wrong.
Given that, I think your only real option for obtaining winds at
vertical AGL levels is using the version of WPP I sent. How much
error is introduced by post-processing is a valid question, and I
don't know the answer. I'm generally of the opinion that
interpolation errors are small relative to the model errors, but it's
worth investigating. I don't see any other real options for you
though.
Hope that helps.
John
On 07/12/2011 01:22 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> OK, but will those results at 50 and 80 m be as accurate as they
were before WPP? What about using the other WRF grib converter?
>
> Thanks,
> David Bryan
>
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tuesday, July 12, 2011 4:41 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> MET cannot read the NetCDF output of WRF directly. The reason is
that WRF NetCDF data is on staggered Arakawa
> horizontal grids and on hybrid vertical levels. Observation data
always uses pressure, elevation or height for vertical
> levels. In your case, I think the easiest path is to continue to
post-process using WPP as you have been doing (I
> assume), and then use the module that John sent to generate wind
data at 50m and 80m. Does that make sense?
>
> Paul
>
>
>
> On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> John & Paul,
>>
>> Thanks for your reply and attention.
>>
>> MET Tools will accept forecasts in NetCDF format (ie, normal wrfout
files), correct? And such files should include wind above ground,
right?
>>
>> If so, I'll simply quit doing any post-processing grib conversion
and use WRF's output.
>>
>> Once I have some output in NetCDF, I'll look into Paul's scripting
suggestions.
>>
>> Thanks,
>> David Bryan
>>
>>
>>
>> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Monday, July 11, 2011 7:06 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> Hello David,
>>
>> My name is John Halley Gotway. I work on MET development and
support with Paul, and he and I were just discussing your questions
about running Point-Stat. Paul's helped you with verifying surface
>> temperature and pressure, and it sounds like you're also interested
in verifying winds. Looking at the sample data you sent Paul, I
noticed two things:
>> (1) You wind observations appear to be at 50 and 80 meters - I
assume that's on a wind turbine.
>> (2) Using wgrib to look at your model output, I see that you have
forecasts of U and V, but not at vertical heights above ground.
>>
>> We usually have the opposite problem - plenty of forecasts but no
observations for verification. But in your case we have observations
at 50 and 80 meters but no forecasts to verify! I assume that
>> you're using the WRF PostProcessor to post-process your WRF output.
Unfortunately, the released version of WPP does not create output for
winds at the AGL heights you'd need to for winds. However, I
>> worked on an extension for WPP to output these fields. We're in a
bit of flux right now as we transition from WPP to the Unified
PostProcessor (UPP). But for now, if you'd like, I could point you to
>> a version of WPP that does generate this AGL output:
>>
ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>>
>> However, please be aware that this is development code, as-is, and
is not part of the officially supported release. But I have passed
this along to the UPP developers for possible inclusion in a
>> future release.
>>
>> So, assuming that you're able to generate U and V forecast output
at 50-meters and 80-meters, here's what you might try in you Point-
Stat config file:
>>
>> fcst_field[] = [ "WIND/Z50", "WIND/Z80" ];
>> obs_field[] = [];
>> message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just needs to match what you put in the ASCII observation data. And
it probably shouldn't be ADPSFC since these are not "surface" obs
>>
>> This will verify wind speed directly at 50 and 80 meters.
Unfortunately, Point-Stat can't verify wind direction directly.
Instead, you could use it to dump out VL1L2 lines and verify it in the
>> STAT-Analysis tool. But to do so, you'd need you observations to
be U and V vectors (instead of wind speed and direction).
>>
>> So one option would be for you to back out the U and V vectors from
wind speed and direction (it's just some simple trig to do so). And
then include the U and V observations in your ASCII observation
>> files. Then you could verify the following in Point-Stat:
>>
>> fcst_field[] = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
"WIND/Z50", "WIND/Z80" ];
>> obs_field[] = [];
>> message_type[] = [ "PROFLR" ];
>>
>> This will verify U, V, and wind speed directly. And, if you have
the VL1L2 flag turned on in the output_flag in the Point-Stat config
file, you'll also get VL1L2 lines in your output. Then, you
>> could use STAT-Analysis to look at wind direction errors. If you
decide to go that route, and need help, just let us know. Here's an
example of this from the recent MET tutorial:
>>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>>
>> Clear as mud?
>>
>> Just let us know if more issues arise. We're happy to help.
>>
>> Thanks,
>> John
>>
>>
>>
>> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> David,
>>>
>>> I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
>>> obs, by the way. It matches the "surface" fields in the model
data file against your original PROFLR obs. I attached
>>> the version I did intend to send with this email.
>>>
>>> Paul
>>>
>>>
>>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>>> David,
>>>>
>>>> I'm still working on your wind verification problem.
>>>>
>>>> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
>>>> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>>>>
>>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>>
>>>> Point_stat can only match forecast/obs pairs in three ways:
>>>>
>>>> 1. by height in meters for non-"surface" message type obs and
model records
>>>> 2. by pressure in hPa for non-"surface" message type obs and
model records
>>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and
SFCSHP) to surface model records
>>>>
>>>> For surface level matching, you must specify the "surface" height
in the fcst_field of the config file: 0m for PRES, 2m
>>>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in
the 12th wgrib field:
>>>>
>>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
>>>> ...
>>>>
>>>> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>>>>
>>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>>
>>>> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
>>>> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
>>>> the observation level is really dictated by the information in
the model GRIB file. In this case, the level there is
>>>> surface.
>>>>
>>>> Unfortunately, if this is all the obs that you have, you only get
3 matched pairs for each field. You really only have
>>>> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
>>>> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
>>>> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
>>>> and generate aggregated statistics.
>>>>
>>>> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
>>>> send you an update.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt
-fcst_valid 20100617_13 -outdir . -v 2
>>>> GSL_RNG_TYPE=mt19937
>>>> GSL_RNG_SEED=18446744071635082062
>>>> Forecast File: NREL10061712.grib
>>>> Climatology File: none
>>>> Configuration File: PSconfig2.txt
>>>> Observation File: A9_sfc.nc
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Reading records for PRES/Z0.
>>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Reading records for TMP/Z2.
>>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>>>> using 3 pairs.
>>>> Computing Categorical Statistics.
>>>> Computing Continuous Statistics.
>>>> Computing Scalar Partial Sums.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>>>> using 3 pairs.
>>>> Computing Categorical Statistics.
>>>> Computing Continuous Statistics.
>>>> Computing Scalar Partial Sums.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> I've posted the data on your ftp site under the directory,
Bryan_data.
>>>>>
>>>>> The contents should be self explanatory, largely. I included
two versions of the config file I tried. Also, I included the
original source of the data, met2010.csv.
>>>>>
>>>>> Thanks,
>>>>> David Bryan
>>>>>
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>>>>> the following instructions?
>>>>>
>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>>
>>>>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>>>>
>>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>>>>
>>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC had wind speed and direction and the grib file had U and V
wind components, that POINT_STAT would sort that out. Is that
correct?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>
>>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>> -fcst_valid 20100617_17 \
>>>>>>> -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>> -v 9
>>>>>>
>>>>>>
>>>>>> ERROR: PointStatConfInfo::process_config() -> the wind
direction field may not b
>>>>>> e verified using point_stat.
>>>>>>
>>>>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>> GSL_RNG_TYPE=mt19937
>>>>>> GSL_RNG_SEED=18446744072421513038
>>>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>> Climatology File: none
>>>>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Reading records for MISSING/Z3.
>>>>>>
>>>>>>
>>>>>> ERROR: read_levels_grib() -> no GRIB records matching
MISSING/Z3 found in GRIB f
>>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Cc:
>>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> The point_stat usage statement indicates that the arguments
should be specified in the following order. Please try this
>>>>>> (you should be able just copy/paste):
>>>>>>
>>>>>> ./point_stat \
>>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>> -fcst_valid 20100617_17 \
>>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>>> -v 9
>>>>>>
>>>>>> I think that may be the cause of your problem.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> I've attached my config file.
>>>>>>>
>>>>>>> When I tried running Point Stat, I got the error message
below. I'm not clear
>>>>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>>>>> that the grib files are produced by the WRF post processor
(though they are
>>>>>>> concatenated, using Linux cat). The text in the message isn't
in the config
>>>>>>> file.
>>>>>>>
>>>>>>> Any suggestions would be welcomed.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>>
>>>>>>>
>>>>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>>> grib"
>>>>>>>
>>>>>>> line = 1
>>>>>>>
>>>>>>> econfig_column = 7
>>>>>>>
>>>>>>> text = "^"
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> GRIB
>>>>>>> ____
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> Regarding your observations, it might be easiest to include
all of your
>>>>>>> observations into a single NetCDF file. You can
>>>>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>>> option in the point_stat configuration file to specify which
stations obs you
>>>>>>> want to use. Otherwise, you can use both
>>>>>>> of your current NetCDF obs files in a single point_stat call
by passing one as
>>>>>>> the "obs_file" and the other one using
>>>>>>> the -point_obs optional argument.
>>>>>>>
>>>>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>>>>> statement, which you can see by running
>>>>>>> point_stat with no arguments:
>>>>>>>
>>>>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>>>>> to be verified (optional).
>>>>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast
lead time to be
>>>>>>> verified (optional).
>>>>>>>
>>>>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>>>>> is your fcst_init time, not your
>>>>>>> fcst_valid time. If you are interested in verifying the
forecast five hours
>>>>>>> into your 48-hour run, then your fcst_lead
>>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>>
>>>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary
number of digits long.
>>>>>>> Typically, for wind ground-based
>>>>>>> measurements, the fcst_field level looks something like "Z10"
for a sensor at
>>>>>>> 10m above the ground. Unfortunately, MET
>>>>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>>>>> it must be post-processed using either
>>>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will put the data
>>>>>>> onto levels in meters and/or pressure,
>>>>>>> both of which MET can read.
>>>>>>>
>>>>>>> Please let me know if you have any more questions.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> Thanks again for your help. Now, I'm on to creating the
config file for Point
>>>>>>>
>>>>>>>> Stat.
>>>>>>>>
>>>>>>>> My model output is for 48-hour runs of WRF with hourly
output. The output is
>>>>>>>> concatenated into a single file, 48 time steps in a single
file. (Perhaps
>>>>>>>> that's not practical.)
>>>>>>>>
>>>>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>>>>
>>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer data into its
>>>>>>>
>>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>>
>>>>>>>>
>>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>>
>>>>>>>> PointStat will only analyze one model time per run.
>>>>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>>>>> instance, if my WRF run is initiated with NAM data produced
on the
>>>>>>>> 20110625_1200
>>>>>>>>
>>>>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>>>>> wanted to analyze five hours into my 48-hour run, my lead
time would be
>>>>>>>> 20110625_1700.
>>>>>>>> The purpose of specifying a temporal range of observation
values is for
>>>>>>>> interpolation across the temporal dimension.
>>>>>>>> The model time that's compared to observation time is the
lead time.
>>>>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>>>>> incorrect, please let me know.
>>>>>>>>
>>>>>>>> My other main question concerns specifying the levels for
each fcst_field. The
>>>>>>>>
>>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical level. Is
>>>>>>>> that
>>>>>>>>
>>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it
be more than
>>>>>>>> three
>>>>>>>>
>>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>>>>> includes
>>>>>>>>
>>>>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>>>>
>>>>>>>> eta-levels which is more related to pressure levels. Does
that matter?
>>>>>>>>
>>>>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>>>>
>>>>>>>> control.
>>>>>>>>
>>>>>>>> Thanks again for all your assistance!
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend
>>>>>>> a
>>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>>>>> you
>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>
>>>>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>>>>> reference on the message types:
>>>>>>>>
>>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>>
>>>>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>>>>
>>>>>>>> produced by NCEP. These observations
>>>>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>>>>> applied to all observations, regardless of
>>>>>>>> source.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>>>>> sounds
>>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>>>> so, what should column 9 have?
>>>>>>>>
>>>>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>>>>
>>>>>>>>
>>>>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>>>>> calculate its statistics from that?
>>>>>>>>
>>>>>>>> Point-Stat was designed to be run once for each forecast
valid time. If your
>>>>>>>> model data files contain single 48 hour
>>>>>>>> forecasts, run Point-Stat once for each model file. Each
time, you will pass
>>>>>>>> the model data file you want to verify,
>>>>>>>> the single observation file and the configuration file. In
the configuration
>>>>>>>> file, you should decide the time window
>>>>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>>>>
>>>>>>>> are beg_ds and end_ds. You could also
>>>>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>>>>
>>>>>>>> If you have multiple forecasts in a single model data file,
you will need to
>>>>>>>> use
>>>>>>>>
>>>>>>>> the command line parameter fcst_valid
>>>>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>>>>> Point-Stat must still be run once for each valid
>>>>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>>>>> parameter.
>>>>>>>>
>>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>> Paul,
>>>>>>>>>
>>>>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>>>>
>>>>>>>>
>>>>>>>>> months. So I'll have one six-month file for observations.
I want to use
>>>>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>>>>> correct?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> Thanks Paul,
>>>>>>>>>
>>>>>>>>> That's very helpful.
>>>>>>>>>
>>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend a
>>>>>>>>
>>>>>>>>
>>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>>
>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>>
>>>>>>>>> so, what should column 9 have?
>>>>>>>>>
>>>>>>>>> Thanks again,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> David,
>>>>>>>>>
>>>>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>>>>> speed
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> at a fixed height above the ground. I
>>>>>>>>> would recommend putting -9999 in that column, since that is
the value that MET
>>>>>>>>
>>>>>>>>
>>>>>>>>> uses for bad data or N/A.
>>>>>>>>>
>>>>>>>>> You should use column 6 to enter the elevation of your
sensor, but be advised
>>>>>>>
>>>>>>>>> that if you use a "surface" message type
>>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> enter in column 6. If you want MET to
>>>>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I hope this is clear. Please let me
>>>>>>>>> know if you have any questions.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>
>>>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>>>> Queue: met_help
>>>>>>>>>> Subject: formatting ascii2nc
>>>>>>>>>> Owner: Nobody
>>>>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>>>>> Status: new
>>>>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Could you clarify "8. Level as the pressure level in hPa
or accumulation
>>>>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>>>>
>>>>>>>>
>>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>> an anemometer at a fixed height. What value would I put
for this column?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> David Bryan
>>>>>>>>>>
>>>>>>>>>
>>>>
>>>
>>>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Tue Jul 12 20:36:00 2011
OK, so if I understand you, you're saying that WRF does NOT put out
wind values at heights other than 10 m? Really?!
I thought that,
at the boundary of each and every cell, is a U or V vector. Are you
saying that wind values exist in the model at higher levels but, due
to the registry file, they're not output?
If you only get wind speed at 10 m, what would any interpolation be
based on? Typically when interpolating, you need values at two
extremes (ie, 10 m and a much higher value) to interpolate an
intermediate value, but we're not given a higher value. If it's based
on V1/V2 = (H1/H2)^a, that formula is dubious at best.
If I'm understanding you, this is really a revelation. Do you think
the registry could be changed to give us wind at desired heights?
Thanks,
David
From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Tuesday, July 12, 2011 7:17 PM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
This is John - I wanted to pipe in here.
There is no other "WRF grib" converter available. The other post-
processing option is the WRF-ARW pinterp tool:
http://www.mmm.ucar.edu/wrf/users/download/get_sources.html
It reads WRF-ARW NetCDF out files, destaggers the grid, and
interpolates to pressure levels. However, I took a look at the WRF
registry file for ARW and don't think that WRF itself is able to dump
out winds at vertical levels like 50m and 80m. The vertical
coordinate is the model level, and I don't think it's set up to derive
vertical levels (other than 10m) like that. But I could be wrong.
Given that, I think your only real option for obtaining winds at
vertical AGL levels is using the version of WPP I sent. How much
error is introduced by post-processing is a valid question, and I
don't know the answer. I'm generally of the opinion that
interpolation errors are small relative to the model errors, but it's
worth investigating. I don't see any other real options for you
though.
Hope that helps.
John
On 07/12/2011 01:22 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> OK, but will those results at 50 and 80 m be as accurate as they
were before WPP? What about using the other WRF grib converter?
>
> Thanks,
> David Bryan
>
> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tuesday, July 12, 2011 4:41 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> MET cannot read the NetCDF output of WRF directly. The reason is
that WRF NetCDF data is on staggered Arakawa
> horizontal grids and on hybrid vertical levels. Observation data
always uses pressure, elevation or height for vertical
> levels. In your case, I think the easiest path is to continue to
post-process using WPP as you have been doing (I
> assume), and then use the module that John sent to generate wind
data at 50m and 80m. Does that make sense?
>
> Paul
>
>
>
> On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> John & Paul,
>>
>> Thanks for your reply and attention.
>>
>> MET Tools will accept forecasts in NetCDF format (ie, normal
wrfout files), correct? And such files should include wind above
ground, right?
>>
>> If so, I'll simply quit doing any post-processing grib
conversion and use WRF's output.
>>
>> Once I have some output in NetCDF, I'll look into Paul's
scripting suggestions.
>>
>> Thanks,
>> David Bryan
>>
>>
>>
>> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Monday, July 11, 2011 7:06 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> Hello David,
>>
>> My name is John Halley Gotway. I work on MET development and
support with Paul, and he and I were just discussing your questions
about running Point-Stat. Paul's helped you with verifying surface
>> temperature and pressure, and it sounds like you're also interested
in verifying winds. Looking at the sample data you sent Paul, I
noticed two things:
>> (1) You wind observations appear to be at 50 and 80 meters - I
assume that's on a wind turbine.
>> (2) Using wgrib to look at your model output, I see that you have
forecasts of U and V, but not at vertical heights above ground.
>>
>> We usually have the opposite problem - plenty of forecasts but no
observations for verification. But in your case we have observations
at 50 and 80 meters but no forecasts to verify! I assume that
>> you're using the WRF PostProcessor to post-process your WRF output.
Unfortunately, the released version of WPP does not create output for
winds at the AGL heights you'd need to for winds. However, I
>> worked on an extension for WPP to output these fields. We're in a
bit of flux right now as we transition from WPP to the Unified
PostProcessor (UPP). But for now, if you'd like, I could point you to
>> a version of WPP that does generate this AGL output:
>> ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>>
>> However, please be aware that this is development code, as-is, and
is not part of the officially supported release. But I have passed
this along to the UPP developers for possible inclusion in a
>> future release.
>>
>> So, assuming that you're able to generate U and V forecast output
at 50-meters and 80-meters, here's what you might try in you Point-
Stat config file:
>>
>> fcst_field[] = [ "WIND/Z50", "WIND/Z80" ];
>> obs_field[] = [];
>> message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just needs to match what you put in the ASCII observation data. And
it probably shouldn't be ADPSFC since these are not "surface" obs
>>
>> This will verify wind speed directly at 50 and 80 meters.
Unfortunately, Point-Stat can't verify wind direction directly.
Instead, you could use it to dump out VL1L2 lines and verify it in the
>> STAT-Analysis tool. But to do so, you'd need you observations to
be U and V vectors (instead of wind speed and direction).
>>
>> So one option would be for you to back out the U and V vectors from
wind speed and direction (it's just some simple trig to do so). And
then include the U and V observations in your ASCII observation
>> files. Then you could verify the following in Point-Stat:
>>
>> fcst_field[] = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
"WIND/Z50", "WIND/Z80" ];
>> obs_field[] = [];
>> message_type[] = [ "PROFLR" ];
>>
>> This will verify U, V, and wind speed directly. And, if you have
the VL1L2 flag turned on in the output_flag in the Point-Stat config
file, you'll also get VL1L2 lines in your output. Then, you
>> could use STAT-Analysis to look at wind direction errors. If you
decide to go that route, and need help, just let us know. Here's an
example of this from the recent MET tutorial:
>>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>>
>> Clear as mud?
>>
>> Just let us know if more issues arise. We're happy to help.
>>
>> Thanks,
>> John
>>
>>
>>
>> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> David,
>>>
>>> I accidentally sent you the wrong version of the config file. The
version I did send would work with PROFLR message
>>> obs, by the way. It matches the "surface" fields in the model
data file against your original PROFLR obs. I attached
>>> the version I did intend to send with this email.
>>>
>>> Paul
>>>
>>>
>>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>>> David,
>>>>
>>>> I'm still working on your wind verification problem.
>>>>
>>>> Regarding PRES and TMP, I may have not made it clear how exactly
point_stat treats surface fields. There is more
>>>> information on what I'm about to tell you in the METv3.0.2 User's
Guide pg. 4-1 here:
>>>>
>>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>>
>>>> Point_stat can only match forecast/obs pairs in three ways:
>>>>
>>>> 1. by height in meters for non-"surface" message type obs and
model records
>>>> 2. by pressure in hPa for non-"surface" message type obs and
model records
>>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and
SFCSHP) to surface model records
>>>>
>>>> For surface level matching, you must specify the "surface" height
in the fcst_field of the config file: 0m for PRES, 2m
>>>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in
the 12th wgrib field:
>>>>
>>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
fcst:NAve=0
>>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
fcst:NAve=0
>>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
fcst:NAve=0
>>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
fcst:NAve=0
>>>> ...
>>>>
>>>> To generate matched pairs with with your obs, I modified your obs
files in the following way:
>>>>
>>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>>
>>>> Then, I changed the fcst_field and message_type fields in the
point_stat config file accordingly. I attached the
>>>> version that I used. The point_stat output is shown below. I
realize that this is a bit confusing, but the setup of
>>>> the observation level is really dictated by the information in
the model GRIB file. In this case, the level there is
>>>> surface.
>>>>
>>>> Unfortunately, if this is all the obs that you have, you only get
3 matched pairs for each field. You really only have
>>>> 1 exact match if you reduce the beg_ds and end_ds time window to
less than 10 min (+/- 600s), since your obs are valid
>>>> every 10 minutes. In your case, I think you may want to dump out
only the MPR data which contains matched pair data.
>>>> Then, once you run point_stat across all of your valid_time data,
you can use the stat_analysis tool to read your MPRs
>>>> and generate aggregated statistics.
>>>>
>>>> If you have any questions, please let me know. Once I have
something to give you for verifying your wind data, I'll
>>>> send you an update.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt
-fcst_valid 20100617_13 -outdir . -v 2
>>>> GSL_RNG_TYPE=mt19937
>>>> GSL_RNG_SEED=18446744071635082062
>>>> Forecast File: NREL10061712.grib
>>>> Climatology File: none
>>>> Configuration File: PSconfig2.txt
>>>> Observation File: A9_sfc.nc
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Reading records for PRES/Z0.
>>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Reading records for TMP/Z2.
>>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>>>> using 3 pairs.
>>>> Computing Categorical Statistics.
>>>> Computing Continuous Statistics.
>>>> Computing Scalar Partial Sums.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(1),
>>>> using 3 pairs.
>>>> Computing Categorical Statistics.
>>>> Computing Continuous Statistics.
>>>> Computing Scalar Partial Sums.
>>>>
>>>>
--------------------------------------------------------------------------------
>>>>
>>>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> Paul,
>>>>>
>>>>> I've posted the data on your ftp site under the
directory, Bryan_data.
>>>>>
>>>>> The contents should be self explanatory, largely. I
included two versions of the config file I tried. Also, I included
the original source of the data, met2010.csv.
>>>>>
>>>>> Thanks,
>>>>> David Bryan
>>>>>
>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>> To: davidstephenbryan at yahoo.com
>>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>
>>>>> David,
>>>>>
>>>>> Can you please bundle up your model data, obs and config files
using "tar xzvf" and upload them to our FTP site using
>>>>> the following instructions?
>>>>>
>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> That will make it easier for me to analyze your situation and
figure out the best course of action with your data.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>>
>>>>>> For the first, it doesn't quite make sense to me that the wind
direction field could not be verified with point_stat. But I took
that out of the second config file just to move forward.
>>>>>>
>>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure field (or maybe temperature; both were at 3 m). I'm not sure
why this is a problem. Could it have to do with a mismatch between
fields created with ASCII2NC and those in the grib file? For
instance, for the pressure value I used the field PRES (001) in
ASCII2NC. But as I examine the grib file in IDV, I don't see a simple
pressure field. In IDV under 2D grid, I see:
Mean_sea_level_pressure_NAM_model @ msl, Pressure @ cloud_base,
Pressure @ cloud_tops, Pressure @ surface, and Pressure_reduced_to_MSL
@ msl. Under 3D grid, I see Pressure @ hybrid. The only field the
grib file has in common with table of grib parameters that could be
used in ASCII2NC
(http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is Pressure
reduced to MSL, though of course, my pressure observations are not
close to MSL. Temperature seems subject to the same circumstances.
>>>>>>
>>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC had wind speed and direction and the grib file had U and V
wind components, that POINT_STAT would sort that out. Is that
correct?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>
>>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>> -fcst_valid 20100617_17 \
>>>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>>>> -v 9
>>>>>>
>>>>>>
>>>>>> ERROR: PointStatConfInfo::process_config() -> the wind
direction field may not b
>>>>>> e verified using point_stat.
>>>>>>
>>>>>> [amec at WRF-master bin]$ ./point_stat
/ext/models/WRF/NREL/10061712/NREL1006171
>>>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
/home/WRF/Developme
>>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid 20100617_17
-outdir /home
>>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>> GSL_RNG_TYPE=mt19937
>>>>>> GSL_RNG_SEED=18446744072421513038
>>>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>> Climatology File: none
>>>>>> Configuration File:
/home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Reading records for MISSING/Z3.
>>>>>>
>>>>>>
>>>>>> ERROR: read_levels_grib() -> no GRIB records matching
MISSING/Z3 found in GRIB f
>>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Cc:
>>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> The point_stat usage statement indicates that the arguments
should be specified in the following order. Please try this
>>>>>> (you should be able just copy/paste):
>>>>>>
>>>>>> ./point_stat \
>>>>>> /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>> -fcst_valid 20100617_17 \
>>>>>> -outdir /home/WRF/Development/David_Bryan/ASCII2NC/PointStat/
\
>>>>>> -v 9
>>>>>>
>>>>>> I think that may be the cause of your problem.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> I've attached my config file.
>>>>>>>
>>>>>>> When I tried running Point Stat, I got the error message
below. I'm not clear
>>>>>>> if it's coming from the config file or the grib file. I will
point out, though,
>>>>>>> that the grib files are produced by the WRF post processor
(though they are
>>>>>>> concatenated, using Linux cat). The text in the message isn't
in the config
>>>>>>> file.
>>>>>>>
>>>>>>> Any suggestions would be welcomed.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
/ext/models/WRF/NRE
>>>>>>> L/10061712/NREL10061712.grib
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt -outdir
/home/WRF/Develo
>>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>>
>>>>>>>
>>>>>>> config() -> syntax error in file
"/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>>> grib"
>>>>>>>
>>>>>>> line = 1
>>>>>>>
>>>>>>> econfig_column = 7
>>>>>>>
>>>>>>> text = "^"
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> GRIB
>>>>>>> ____
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message ----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> Regarding your observations, it might be easiest to include
all of your
>>>>>>> observations into a single NetCDF file. You can
>>>>>>> denote the two different sensor locations using Station_ID
(column 2) in
>>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>>> option in the point_stat configuration file to specify which
stations obs you
>>>>>>> want to use. Otherwise, you can use both
>>>>>>> of your current NetCDF obs files in a single point_stat call
by passing one as
>>>>>>> the "obs_file" and the other one using
>>>>>>> the -point_obs optional argument.
>>>>>>>
>>>>>>> Regarding the forecast times, the following is from the
point_stat usage
>>>>>>> statement, which you can see by running
>>>>>>> point_stat with no arguments:
>>>>>>>
>>>>>>> "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets the
forecast valid time
>>>>>>> to be verified (optional).
>>>>>>> "-fcst_lead time" in HH[MMSS] format sets the forecast lead
time to be
>>>>>>> verified (optional).
>>>>>>>
>>>>>>> If your model is initiated (we call it initialized) at
20110625_1200, then that
>>>>>>> is your fcst_init time, not your
>>>>>>> fcst_valid time. If you are interested in verifying the
forecast five hours
>>>>>>> into your 48-hour run, then your fcst_lead
>>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>>
>>>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary
number of digits long.
>>>>>>> Typically, for wind ground-based
>>>>>>> measurements, the fcst_field level looks something like "Z10"
for a sensor at
>>>>>>> 10m above the ground. Unfortunately, MET
>>>>>>> does not read model data on eta levels. In order to feed your
WRF data to MET,
>>>>>>> it must be post-processed using either
>>>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will put the data
>>>>>>> onto levels in meters and/or pressure,
>>>>>>> both of which MET can read.
>>>>>>>
>>>>>>> Please let me know if you have any more questions.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> Thanks again for your help. Now, I'm on to creating the
config file for Point
>>>>>>>
>>>>>>>> Stat.
>>>>>>>>
>>>>>>>> My model output is for 48-hour runs of WRF with hourly
output. The output is
>>>>>>>> concatenated into a single file, 48 time steps in a single
file. (Perhaps
>>>>>>>> that's not practical.)
>>>>>>>>
>>>>>>>> My observations are for two met towers, each with two
anemometers at different
>>>>>>>
>>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer data into its
>>>>>>>
>>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>>
>>>>>>>>
>>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>>
>>>>>>>> PointStat will only analyze one model time per run.
>>>>>>>> In the command line arguments, I will have to specify
-fcst_valid. For
>>>>>>>> instance, if my WRF run is initiated with NAM data produced
on the
>>>>>>>> 20110625_1200
>>>>>>>>
>>>>>>>> (and my model is initiated on the same time), that's my valid
time.
>>>>>>>> I will also have to specify -fcst_lead, which indicates where
(temporally) in
>>>>>>>> the model's forecast I want to analyze. So with that run I
mentioned, if I
>>>>>>>> wanted to analyze five hours into my 48-hour run, my lead
time would be
>>>>>>>> 20110625_1700.
>>>>>>>> The purpose of specifying a temporal range of observation
values is for
>>>>>>>> interpolation across the temporal dimension.
>>>>>>>> The model time that's compared to observation time is the
lead time.
>>>>>>>> If any of my understanding of how time specification in Point
Stat works is
>>>>>>>> incorrect, please let me know.
>>>>>>>>
>>>>>>>> My other main question concerns specifying the levels for
each fcst_field. The
>>>>>>>>
>>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical level. Is
>>>>>>>> that
>>>>>>>>
>>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can it
be more than
>>>>>>>> three
>>>>>>>>
>>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided default
config file
>>>>>>>> includes
>>>>>>>>
>>>>>>>> the following comment: "GC/ZNNN-NNN for a range of vertical
levels (MSL or
>>>>>>>> AGL)." How would one specify whether it's MSL or AGL? Also,
WRF output is in
>>>>>>>
>>>>>>>> eta-levels which is more related to pressure levels. Does
that matter?
>>>>>>>>
>>>>>>>> Also, I assume that the forecast and observation thresholds
serve as a quality
>>>>>>>
>>>>>>>> control.
>>>>>>>>
>>>>>>>> Thanks again for all your assistance!
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend
>>>>>>> a
>>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could
>>>>>>>> you
>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>
>>>>>>>> I think PROFLR is the appropriate message type for your
situation. Here is a
>>>>>>>> reference on the message types:
>>>>>>>>
>>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>>
>>>>>>>> Message types are a concept that is associated with PrepBUFR
point observations
>>>>>>>>
>>>>>>>> produced by NCEP. These observations
>>>>>>>> are often used in MET. The logic that MET uses for these
message types are
>>>>>>>> applied to all observations, regardless of
>>>>>>>> source.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>>> point and column nine would be the sensor's height about
ground. But it
>>>>>>>> sounds
>>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>>>> so, what should column 9 have?
>>>>>>>>
>>>>>>>> No, you are correct. I should have been more clear in my
previous email.
>>>>>>>>
>>>>>>>>
>>>>>>>>> will Point-Stat match the observation time to the model time
and
>>>>>>>>> calculate its statistics from that?
>>>>>>>>
>>>>>>>> Point-Stat was designed to be run once for each forecast
valid time. If your
>>>>>>>> model data files contain single 48 hour
>>>>>>>> forecasts, run Point-Stat once for each model file. Each
time, you will pass
>>>>>>>> the model data file you want to verify,
>>>>>>>> the single observation file and the configuration file. In
the configuration
>>>>>>>> file, you should decide the time window
>>>>>>>> inside which observations "match" the model valid time. The
settings for this
>>>>>>>
>>>>>>>> are beg_ds and end_ds. You could also
>>>>>>>> use the Point-Stat command line parameters obs_valid_beg and
obs_valid_end.
>>>>>>>>
>>>>>>>> If you have multiple forecasts in a single model data file,
you will need to
>>>>>>>> use
>>>>>>>>
>>>>>>>> the command line parameter fcst_valid
>>>>>>>> time to specify which forecast Point-Stat should verify. In
this case,
>>>>>>>> Point-Stat must still be run once for each valid
>>>>>>>> time, where each call differs only by the value passed to the
fcst_valid
>>>>>>>> parameter.
>>>>>>>>
>>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>> Paul,
>>>>>>>>>
>>>>>>>>> One other question. The netcdf file I'll be creating with
ascii2nc covers six
>>>>>>>>
>>>>>>>>
>>>>>>>>> months. So I'll have one six-month file for observations.
I want to use
>>>>>>>>> Point-Stat to compare that file to a series of 48-hour
forecasts. My
>>>>>>>>> question--will Point-Stat match the observation time to the
model time and
>>>>>>>>> calculate its statistics from that? In other words, Point-
Stat's output for
>>>>>>>>> each forecast is not based on periods for which there is no
forecast data,
>>>>>>>>> correct?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> Thanks Paul,
>>>>>>>>>
>>>>>>>>> That's very helpful.
>>>>>>>>>
>>>>>>>>> I do want to interpolate to the height of the sensor, so
could you recommend a
>>>>>>>>
>>>>>>>>
>>>>>>>>> message type? Frankly, I'm not familiar with these message
types, so could you
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>>
>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
terrain at that
>>>>>>>>> point and column nine would be the sensor's height about
ground. But it sounds
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> like column 6 should be the sensor's total height above sea
level, right? If
>>>>>>>
>>>>>>>>> so, what should column 9 have?
>>>>>>>>>
>>>>>>>>> Thanks again,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> David,
>>>>>>>>>
>>>>>>>>> This column does not apply to your data, since you are only
measuring wind
>>>>>>>>> speed
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> at a fixed height above the ground. I
>>>>>>>>> would recommend putting -9999 in that column, since that is
the value that MET
>>>>>>>>
>>>>>>>>
>>>>>>>>> uses for bad data or N/A.
>>>>>>>>>
>>>>>>>>> You should use column 6 to enter the elevation of your
sensor, but be advised
>>>>>>>
>>>>>>>>> that if you use a "surface" message type
>>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
regardless of what you
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> enter in column 6. If you want MET to
>>>>>>>>> interpolate model data to the height of your sensor, use
another message type.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I hope this is clear. Please let me
>>>>>>>>> know if you have any questions.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>
>>>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>>>> Queue: met_help
>>>>>>>>>> Subject: formatting ascii2nc
>>>>>>>>>> Owner: Nobody
>>>>>>>>>> Requestors: davidstephenbryan at yahoo.com
>>>>>>>>>> Status: new
>>>>>>>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Could you clarify "8. Level as the pressure level in hPa
or accumulation
>>>>>>>>>> interval in hours" when formatting point observations. My
point observation
>>>>>>>>
>>>>>>>>
>>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>> an anemometer at a fixed height. What value would I put
for this column?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> David Bryan
>>>>>>>>>>
>>>>>>>>>
>>>>
>>>
>>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: John Halley Gotway
Time: Tue Jul 12 21:07:09 2011
David,
My understanding is that the vertical coordinate in WRF is defined on
eta
levels, and those are the levels over which the model integration is
performed. The wrfout files contains output defined on those eta
levels
as well as some derived variables - including things like 2m
temperature
and 10 meter winds.
The WRF PostProcessor code derives many additional fields as wells as
interpolating to pressure levels in the vertical and destaggering the
horizontal grid.
Having said that, I don't work directly with WRF on a regular basis,
and
it you may find it helpful to refer to the WRF users's guide, online
tutorial, and/or write wrfhelp at ucar.edu for more information.
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> OK, so if I understand you, you're saying that WRF does NOT put out
wind
> values at heights other than 10 m? Really?!Â
> Â
> I thought
that, at the boundary of each and every cell, is a U or V
> vector. Are you saying that wind values exist in the model at
higher
> levels but, due to the registry file, they're not output?
> Â
> If you only get wind speed at 10 m, what would any
interpolation be based
> on? Typically when interpolating, you need values at two extremes
(ie,
> 10 m and a much higher value) to interpolate an intermediate value,
but
> we're not given a higher value. If it's based on V1/V2 =
(H1/H2)^a, that
> formula is dubious at best.
> Â
> If I'm understanding you, this is really a revelation. Do you
think the
> registry could be changed to give us wind at desired heights?
> Â
> Thanks,
> David
>
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tuesday, July 12, 2011 7:17 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> This is John - I wanted to pipe in here.
>
> There is no other "WRF grib" converter available. The other
> post-processing option is the WRF-ARW pinterp tool:
> Â http://www.mmm.ucar.edu/wrf/users/download/get_sources.html
>
> It reads WRF-ARW NetCDF out files, destaggers the grid, and
interpolates
> to pressure levels. However, I took a look at the WRF registry
file for
> ARW and don't think that WRF itself is able to dump
> out winds at vertical levels like 50m and 80m. The vertical
coordinate
> is the model level, and I don't think it's set up to derive vertical
> levels (other than 10m) like that. But I could be wrong.
>
> Given that, I think your only real option for obtaining winds at
vertical
> AGL levels is using the version of WPP I sent. How much error is
> introduced by post-processing is a valid question, and I
> don't know the answer. I'm generally of the opinion that
interpolation
> errors are small relative to the model errors, but it's worth
> investigating. I don't see any other real options for you though.
>
> Hope that helps.
>
> John
>
> On 07/12/2011 01:22 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> OK, but will those results at 50 and 80 m be as accurate as they
were
>> before WPP? What about using the other WRF grib converter?
>>Â
>> Thanks,
>> David Bryan
>>
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tuesday, July 12, 2011 4:41 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> MET cannot read the NetCDF output of WRF directly. The reason is
that
>> WRF NetCDF data is on staggered Arakawa
>> horizontal grids and on hybrid vertical levels. Observation data
>> always uses pressure, elevation or height for vertical
>> levels. In your case, I think the easiest path is to continue to
>> post-process using WPP as you have been doing (I
>> assume), and then use the module that John sent to generate wind
data at
>> 50m and 80m. Does that make sense?
>>
>> Paul
>>
>>
>>
>> On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> John & Paul,
>>>Â
>>> Thanks for your reply and attention.
>>>Â
>>> MET Tools will accept forecasts in NetCDF format (ie, normal
wrfout
>>> files), correct? And such files should include wind above
ground,
>>> right?
>>>Â
>>> If so, I'll simply quit doing any post-processing grib
conversion and
>>> use WRF's output.
>>>Â
>>> Once I have some output in NetCDF, I'll look into Paul's
scripting
>>> suggestions.
>>>Â
>>> Thanks,
>>> David Bryan
>>>Â
>>>Â
>>>
>>> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Monday, July 11, 2011 7:06 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> Hello David,
>>>
>>> My name is John Halley Gotway. I work on MET development and
support
>>> with Paul, and he and I were just discussing your questions about
>>> running Point-Stat. Paul's helped you with verifying surface
>>> temperature and pressure, and it sounds like you're also
interested in
>>> verifying winds. Looking at the sample data you sent Paul, I
noticed
>>> two things:
>>>Â (1) You wind observations appear to be at 50 and 80 meters - I
assume
>>> that's on a wind turbine.
>>>Â (2) Using wgrib to look at your model output, I see that you
have
>>> forecasts of U and V, but not at vertical heights above ground.
>>>
>>> We usually have the opposite problem - plenty of forecasts but no
>>> observations for verification. But in your case we have
observations
>>> at 50 and 80 meters but no forecasts to verify! I assume that
>>> you're using the WRF PostProcessor to post-process your WRF
output.Â
>>> Unfortunately, the released version of WPP does not
create output for
>>> winds at the AGL heights you'd need to for winds. However, I
>>> worked on an extension for WPP to output these fields. We're in
a bit
>>> of flux right now as we transition from WPP to the Unified
>>> PostProcessor (UPP). But for now, if you'd like, I could point
you to
>>> a version of WPP that does generate this AGL output:
>>>Â
ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>>>
>>> However, please be aware that this is development code, as-is, and
is
>>> not part of the officially supported release. But I have passed
this
>>> along to the UPP developers for possible inclusion in a
>>> future release.
>>>
>>> So, assuming that you're able to generate U and V forecast output
at
>>> 50-meters and 80-meters, here's what you might try in you Point-
Stat
>>> config file:
>>>
>>> fcst_field[]Â = [ "WIND/Z50", "WIND/Z80" ];
>>> obs_field[]Â Â = [];
>>> message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just
>>> needs to match what you put in the ASCII observation data. And
it
>>> probably shouldn't be ADPSFC since these are not "surface" obs
>>>
>>> This will verify wind speed directly at 50 and 80 meters.Â
>>>
Unfortunately, Point-Stat can't verify wind direction directly.Â
>>>
Instead, you could use it to dump out VL1L2 lines and verify it in the
>>> STAT-Analysis tool. But to do so, you'd need you observations to
be U
>>> and V vectors (instead of wind speed and direction).
>>>
>>> So one option would be for you to back out the U and V vectors
from
>>> wind speed and direction (it's just some simple trig to do so).Â
And
>>> then include the U and V observations in your ASCII observation
>>> files. Then you could verify the following in Point-Stat:
>>>
>>> fcst_field[]Â = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
>>> "WIND/Z50", "WIND/Z80" ];
>>> obs_field[]Â Â = [];
>>> message_type[] = [ "PROFLR" ];
>>>
>>> This will verify U, V, and wind speed directly. And, if you have
the
>>> VL1L2 flag turned on in the output_flag in the Point-Stat config
file,
>>> you'll also get VL1L2 lines in your output. Then, you
>>> could use STAT-Analysis to look at wind direction errors. If you
>>> decide to go that route, and need help, just let us know. Here's
an
>>> example of this from the recent MET tutorial:
>>>Â
>>>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>>>
>>> Clear as mud?
>>>
>>> Just let us know if more issues arise. We're happy to help.
>>>
>>> Thanks,
>>> John
>>>
>>>
>>>
>>> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> David,
>>>>
>>>> I accidentally sent you the wrong version of the config file.Â
The
>>>> version I did send would work with PROFLR message
>>>> obs, by the way. It matches the "surface" fields in the model
data
>>>> file against your original PROFLR obs. I attached
>>>> the version I did intend to send with this email.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>>>> David,
>>>>>
>>>>> I'm still working on your wind verification problem.
>>>>>
>>>>> Regarding PRES and TMP, I may have not made it clear how exactly
>>>>> point_stat treats surface fields. There is more
>>>>> information on what I'm about to tell you in the METv3.0.2
User's
>>>>> Guide pg. 4-1 here:
>>>>>
>>>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>>>
>>>>> Point_stat can only match forecast/obs pairs in three ways:
>>>>>
>>>>> 1. by height in meters for non-"surface" message type obs and
model
>>>>> records
>>>>> 2. by pressure in hPa for non-"surface" message type obs and
model
>>>>> records
>>>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and
SFCSHP)
>>>>> to surface model records
>>>>>
>>>>> For surface level matching, you must specify the "surface"
height in
>>>>> the fcst_field of the config file: 0m for PRES, 2m
>>>>> for TMP and 10m for winds. Surface GRIB records have 'sfc' in
the
>>>>> 12th wgrib field:
>>>>>
>>>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print
>>>>> $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
>>>>> fcst:NAve=0
>>>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
>>>>> fcst:NAve=0
>>>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
>>>>> fcst:NAve=0
>>>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
>>>>> fcst:NAve=0
>>>>> ...
>>>>>
>>>>> To generate matched pairs with with your obs, I modified your
obs
>>>>> files in the following way:
>>>>>
>>>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>>>
>>>>> Then, I changed the fcst_field and message_type fields in the
>>>>> point_stat config file accordingly. I attached the
>>>>> version that I used. The point_stat output is shown below. I
>>>>> realize that this is a bit confusing, but the setup of
>>>>> the observation level is really dictated by the information in
the
>>>>> model GRIB file. In this case, the level there is
>>>>> surface.
>>>>>
>>>>> Unfortunately, if this is all the obs that you have, you only
get 3
>>>>> matched pairs for each field. You really only have
>>>>> 1 exact match if you reduce the beg_ds and end_ds time window to
less
>>>>> than 10 min (+/- 600s), since your obs are valid
>>>>> every 10 minutes. In your case, I think you may want to dump
out
>>>>> only the MPR data which contains matched pair data.
>>>>> Then, once you run point_stat across all of your valid_time
data, you
>>>>> can use the stat_analysis tool to read your MPRs
>>>>> and generate aggregated statistics.
>>>>>
>>>>> If you have any questions, please let me know. Once I have
>>>>> something to give you for verifying your wind data, I'll
>>>>> send you an update.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt
-fcst_valid
>>>>> 20100617_13 -outdir . -v 2
>>>>> GSL_RNG_TYPE=mt19937
>>>>> GSL_RNG_SEED=18446744071635082062
>>>>> Forecast File: NREL10061712.grib
>>>>> Climatology File: none
>>>>> Configuration File: PSconfig2.txt
>>>>> Observation File: A9_sfc.nc
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Reading records for PRES/Z0.
>>>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Reading records for TMP/Z2.
>>>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over
>>>>> region FULL, for interpolation method UW_MEAN(1),
>>>>> using 3 pairs.
>>>>> Computing Categorical Statistics.
>>>>> Computing Continuous Statistics.
>>>>> Computing Scalar Partial Sums.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over
>>>>> region FULL, for interpolation method UW_MEAN(1),
>>>>> using 3 pairs.
>>>>> Computing Categorical Statistics.
>>>>> Computing Continuous Statistics.
>>>>> Computing Scalar Partial Sums.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Output file: ./point_stat_010000L_20100617_130000V.stat
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>Â
>>>>>> I've posted the data on your ftp site under the
directory,
>>>>>> Bryan_data.
>>>>>>Â
>>>>>> The contents should be self explanatory, largely. I
included two
>>>>>> versions of the config file I tried. Also, I included the
original
>>>>>> source of the data, met2010.csv.
>>>>>>Â
>>>>>> Thanks,
>>>>>> David Bryan
>>>>>>
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> Can you please bundle up your model data, obs and config files
using
>>>>>> "tar xzvf" and upload them to our FTP site using
>>>>>> the following instructions?
>>>>>>
>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>
>>>>>> That will make it easier for me to analyze your situation and
figure
>>>>>> out the best course of action with your data.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>>>
>>>>>>> For the first, it doesn't quite make sense to me that the wind
>>>>>>> direction field could not be verified with point_stat. But I
took
>>>>>>> that out of the second config file just to move forward.
>>>>>>>
>>>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure
>>>>>>> field (or maybe temperature; both were at 3 m). I'm not sure
why
>>>>>>> this is a problem. Could it have to do with a mismatch
between
>>>>>>> fields created with ASCII2NC and those in the grib file? For
>>>>>>> instance, for the pressure value I used the field PRES (001)
in
>>>>>>> ASCII2NC. But as I examine the grib file in IDV, I don't see
a
>>>>>>> simple pressure field. In IDV under 2D grid, I see:Â
>>>>>>> Mean_sea_level_pressure_NAM_model @ msl, Pressure @
cloud_base,
>>>>>>> Pressure @ cloud_tops, Pressure @ surface, and
>>>>>>> Pressure_reduced_to_MSL @ msl. Under 3D grid, I see Pressure
@
>>>>>>> hybrid. The only field the grib file has in common with
table of
>>>>>>> grib parameters that could be used in ASCII2NC
>>>>>>> (http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is
>>>>>>> Pressure reduced to MSL, though of course, my pressure
observations
>>>>>>> are not close to MSL. Temperature seems subject to the same
>>>>>>> circumstances.
>>>>>>>
>>>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC
>>>>>>> had wind speed and direction and the grib file had U and V
wind
>>>>>>> components, that POINT_STAT would sort that out. Is that
correct?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>>>>Â /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>>>Â -fcst_valid 20100617_17 \
>>>>>>>>Â -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>>>Â -v 9
>>>>>>>
>>>>>>>
>>>>>>> ERROR: PointStatConfInfo::process_config() -> the wind
direction
>>>>>>> field may not b
>>>>>>> e verified using point_stat.
>>>>>>>
>>>>>>> [amec at WRF-master bin]$ ./point_statÂ
>>>>>>>
/ext/models/WRF/NREL/10061712/NREL1006171
>>>>>>> 2.grib /home/WRF/Development/David_Bryan/ASCII2NC/A9.ncÂ
>>>>>>> /home/WRF/Developme
>>>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid
20100617_17Â
>>>>>>> -outdir /home
>>>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/Â -v 9
>>>>>>> GSL_RNG_TYPE=mt19937
>>>>>>> GSL_RNG_SEED=18446744072421513038
>>>>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>> Climatology File: none
>>>>>>> Configuration File:
>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>>>
>>>>>>>
--------------------------------------------------------------------------------
>>>>>>>
>>>>>>> Reading records for MISSING/Z3.
>>>>>>>
>>>>>>>
>>>>>>> ERROR: read_levels_grib() -> no GRIB records matching
MISSING/Z3
>>>>>>> found in GRIB f
>>>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Cc:
>>>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> The point_stat usage statement indicates that the arguments
should
>>>>>>> be specified in the following order. Please try this
>>>>>>> (you should be able just copy/paste):
>>>>>>>
>>>>>>> ./point_stat \
>>>>>>>Â /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>>Â -fcst_valid 20100617_17 \
>>>>>>>Â -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>>Â -v 9
>>>>>>>
>>>>>>> I think that may be the cause of your problem.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> I've attached my config file.
>>>>>>>>
>>>>>>>> When I tried running Point Stat, I got the error message
below.Â
>>>>>>>> I'm not clear
>>>>>>>> if it's coming from the config file or the grib file. I
will
>>>>>>>> point out, though,
>>>>>>>> that the grib files are produced by the WRF post processor
(though
>>>>>>>> they are
>>>>>>>> concatenated, using Linux cat). The text in the message
isn't in
>>>>>>>> the config
>>>>>>>> file.
>>>>>>>>
>>>>>>>> Any suggestions would be welcomed.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
>>>>>>>> /ext/models/WRF/NRE
>>>>>>>> L/10061712/NREL10061712.grib
>>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt
-outdir
>>>>>>>> /home/WRF/Develo
>>>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>>>
>>>>>>>>
>>>>>>>>Â Â config() -> syntax error in file
>>>>>>>> "/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>>>> grib"
>>>>>>>>
>>>>>>>>    line = 1
>>>>>>>>
>>>>>>>>Â Â Â Â econfig_column = 7
>>>>>>>>
>>>>>>>>    text = "^"
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> GRIB
>>>>>>>> ____
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>> Regarding your observations, it might be easiest to include
all of
>>>>>>>> your
>>>>>>>> observations into a single NetCDF file. You can
>>>>>>>> denote the two different sensor locations using Station_ID
(column
>>>>>>>> 2) in
>>>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>>>> option in the point_stat configuration file to specify which
>>>>>>>> stations obs you
>>>>>>>> want to use. Otherwise, you can use both
>>>>>>>> of your current NetCDF obs files in a single point_stat call
by
>>>>>>>> passing one as
>>>>>>>> the "obs_file" and the other one using
>>>>>>>> the -point_obs optional argument.
>>>>>>>>
>>>>>>>> Regarding the forecast times, the following is from the
point_stat
>>>>>>>> usage
>>>>>>>> statement, which you can see by running
>>>>>>>> point_stat with no arguments:
>>>>>>>>
>>>>>>>>Â Â "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets
the
>>>>>>>> forecast valid time
>>>>>>>> to be verified (optional).
>>>>>>>>Â Â "-fcst_lead time" in HH[MMSS] format sets the forecast
lead
>>>>>>>> time to be
>>>>>>>> verified (optional).
>>>>>>>>
>>>>>>>> If your model is initiated (we call it initialized) at
>>>>>>>> 20110625_1200, then that
>>>>>>>> is your fcst_init time, not your
>>>>>>>> fcst_valid time. If you are interested in verifying the
forecast
>>>>>>>> five hours
>>>>>>>> into your 48-hour run, then your fcst_lead
>>>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>>>
>>>>>>>> Regarding levels, ZNNN is AGL and it can be an arbitrary
number of
>>>>>>>> digits long.Â
>>>>>>>> Typically, for wind ground-based
>>>>>>>> measurements, the fcst_field level looks something like "Z10"
for
>>>>>>>> a sensor at
>>>>>>>> 10m above the ground. Unfortunately, MET
>>>>>>>> does not read model data on eta levels. In order to feed
your
>>>>>>>> WRF data to MET,
>>>>>>>> it must be post-processed using either
>>>>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will
>>>>>>>> put the data
>>>>>>>> onto levels in meters and/or pressure,
>>>>>>>> both of which MET can read.
>>>>>>>>
>>>>>>>> Please let me know if you have any more questions.
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>> Paul,
>>>>>>>>>
>>>>>>>>> Thanks again for your help. Now, I'm on to creating the
config
>>>>>>>>> file for Point
>>>>>>>>
>>>>>>>>> Stat.
>>>>>>>>>
>>>>>>>>> My model output is for 48-hour runs of WRF with hourly
output.Â
>>>>>>>>> The output is
>>>>>>>>> concatenated into a single file, 48 time steps in a single
>>>>>>>>> file. (Perhaps
>>>>>>>>> that's not practical.)
>>>>>>>>>
>>>>>>>>> My observations are for two met towers, each with two
anemometers
>>>>>>>>> at different
>>>>>>>>
>>>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer
>>>>>>>>> data into its
>>>>>>>>
>>>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>>>
>>>>>>>>> PointStat will only analyze one model time per run.
>>>>>>>>> In the command line arguments, I will have to specify
>>>>>>>>> -fcst_valid. For
>>>>>>>>> instance, if my WRF run is initiated with NAM data produced
on
>>>>>>>>> the
>>>>>>>>> 20110625_1200
>>>>>>>>>
>>>>>>>>> (and my model is initiated on the same time), that's my
valid
>>>>>>>>> time.
>>>>>>>>> I will also have to specify -fcst_lead, which indicates
where
>>>>>>>>> (temporally) in
>>>>>>>>> the model's forecast I want to analyze. So with that run I
>>>>>>>>> mentioned, if I
>>>>>>>>> wanted to analyze five hours into my 48-hour run, my lead
time
>>>>>>>>> would be
>>>>>>>>> 20110625_1700.
>>>>>>>>> The purpose of specifying a temporal range of observation
values
>>>>>>>>> is for
>>>>>>>>> interpolation across the temporal dimension.
>>>>>>>>> The model time that's compared to observation time is the
lead
>>>>>>>>> time.
>>>>>>>>> If any of my understanding of how time specification in
Point
>>>>>>>>> Stat works is
>>>>>>>>> incorrect, please let me know.
>>>>>>>>>
>>>>>>>>> My other main question concerns specifying the levels for
each
>>>>>>>>> fcst_field. The
>>>>>>>>>
>>>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical
>>>>>>>>> level. Is
>>>>>>>>> that
>>>>>>>>>
>>>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can
it
>>>>>>>>> be more than
>>>>>>>>> three
>>>>>>>>>
>>>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided
default
>>>>>>>>> config file
>>>>>>>>> includes
>>>>>>>>>
>>>>>>>>> the following comment:Â "GC/ZNNN-NNN for a range of
vertical
>>>>>>>>> levels (MSL or
>>>>>>>>> AGL)."Â How would one specify whether it's MSL or AGL?Â
Also,
>>>>>>>>> WRF output is in
>>>>>>>>
>>>>>>>>> eta-levels which is more related to pressure levels. Does
that
>>>>>>>>> matter?
>>>>>>>>>
>>>>>>>>> Also, I assume that the forecast and observation thresholds
serve
>>>>>>>>> as a quality
>>>>>>>>
>>>>>>>>> control.
>>>>>>>>>
>>>>>>>>> Thanks again for all your assistance!
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>Â
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> David,
>>>>>>>>>
>>>>>>>>>> I do want to interpolate to the height of the sensor, so
could
>>>>>>>>>> you recommend
>>>>>>>> a
>>>>>>>>>> message type? Frankly, I'm not familiar with these
message
>>>>>>>>>> types, so could
>>>>>>>>> you
>>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>>
>>>>>>>>> I think PROFLR is the appropriate message type for your
>>>>>>>>> situation. Here is a
>>>>>>>>> reference on the message types:
>>>>>>>>>
>>>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>>>
>>>>>>>>> Message types are a concept that is associated with PrepBUFR
>>>>>>>>> point observations
>>>>>>>>>
>>>>>>>>> produced by NCEP. These observations
>>>>>>>>> are often used in MET. The logic that MET uses for these
>>>>>>>>> message types are
>>>>>>>>> applied to all observations, regardless of
>>>>>>>>> source.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
>>>>>>>>>> terrain at that
>>>>>>>>>> point and column nine would be the sensor's height about
>>>>>>>>>> ground. But it
>>>>>>>>> sounds
>>>>>>>>>> like column 6 should be the sensor's total height above sea
>>>>>>>>>> level, right? If
>>>>>>>>>> so, what should column 9 have?
>>>>>>>>>
>>>>>>>>> No, you are correct. I should have been more clear in my
>>>>>>>>> previous email.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> will Point-Stat match the observation time to the model
time and
>>>>>>>>>> calculate its statistics from that?
>>>>>>>>>
>>>>>>>>> Point-Stat was designed to be run once for each forecast
valid
>>>>>>>>> time. If your
>>>>>>>>> model data files contain single 48 hour
>>>>>>>>> forecasts, run Point-Stat once for each model file. Each
time,
>>>>>>>>> you will pass
>>>>>>>>> the model data file you want to verify,
>>>>>>>>> the single observation file and the configuration file. In
the
>>>>>>>>> configuration
>>>>>>>>> file, you should decide the time window
>>>>>>>>> inside which observations "match" the model valid time.Â
The
>>>>>>>>> settings for this
>>>>>>>>
>>>>>>>>> are beg_ds and end_ds. You could also
>>>>>>>>> use the Point-Stat command line parameters obs_valid_beg and
>>>>>>>>> obs_valid_end.
>>>>>>>>>
>>>>>>>>> If you have multiple forecasts in a single model data file,
you
>>>>>>>>> will need to
>>>>>>>>> use
>>>>>>>>>
>>>>>>>>> the command line parameter fcst_valid
>>>>>>>>> time to specify which forecast Point-Stat should verify.Â
In
>>>>>>>>> this case,
>>>>>>>>> Point-Stat must still be run once for each valid
>>>>>>>>> time, where each call differs only by the value passed to
the
>>>>>>>>> fcst_valid
>>>>>>>>> parameter.
>>>>>>>>>
>>>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>
>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>
>>>>>>>>>> Paul,
>>>>>>>>>>
>>>>>>>>>> One other question. The netcdf file I'll be creating with
>>>>>>>>>> ascii2nc covers six
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> months. So I'll have one six-month file for
observations. I
>>>>>>>>>> want to use
>>>>>>>>>> Point-Stat to compare that file to a series of 48-hour
>>>>>>>>>> forecasts. My
>>>>>>>>>> question--will Point-Stat match the observation time to the
>>>>>>>>>> model time and
>>>>>>>>>> calculate its statistics from that? In other words,
>>>>>>>>>> Point-Stat's output for
>>>>>>>>>> each forecast is not based on periods for which there is no
>>>>>>>>>> forecast data,
>>>>>>>>>> correct?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Original Message ----
>>>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>
>>>>>>>>>> Thanks Paul,
>>>>>>>>>>
>>>>>>>>>> That's very helpful.Â
>>>>>>>>>>
>>>>>>>>>> I do want to interpolate to the height of the sensor, so
could
>>>>>>>>>> you recommend a
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> message type? Frankly, I'm not familiar with these
message
>>>>>>>>>> types, so could you
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>>>
>>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
>>>>>>>>>> terrain at that
>>>>>>>>>> point and column nine would be the sensor's height about
>>>>>>>>>> ground. But it sounds
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> like column 6 should be the sensor's total height above sea
>>>>>>>>>> level, right? If
>>>>>>>>
>>>>>>>>>> so, what should column 9 have?
>>>>>>>>>>
>>>>>>>>>> Thanks again,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Original Message ----
>>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>
>>>>>>>>>> David,
>>>>>>>>>>
>>>>>>>>>> This column does not apply to your data, since you are only
>>>>>>>>>> measuring wind
>>>>>>>>>> speed
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> at a fixed height above the ground. I
>>>>>>>>>> would recommend putting -9999 in that column, since that is
the
>>>>>>>>>> value that MET
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> uses for bad data or N/A.
>>>>>>>>>>
>>>>>>>>>> You should use column 6 to enter the elevation of your
sensor,
>>>>>>>>>> but be advised
>>>>>>>>
>>>>>>>>>> that if you use a "surface" message type
>>>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
>>>>>>>>>> regardless of what you
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> enter in column 6. If you want MET to
>>>>>>>>>> interpolate model data to the height of your sensor, use
another
>>>>>>>>>> message type.Â
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I hope this is clear. Please let me
>>>>>>>>>> know if you have any questions.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Paul
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>>
>>>>>>>>>>> Tue Jul 05 06:25:54 2011: Request 48043 was acted upon.
>>>>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>>>>>Â Â Â Â Â Queue: met_help
>>>>>>>>>>>Â Â Â Â Subject: formatting ascii2nc
>>>>>>>>>>>Â Â Â Â Â Owner: Nobody
>>>>>>>>>>>Â Â Requestors: davidstephenbryan at yahoo.com
>>>>>>>>>>>Â Â Â Â Status: new
>>>>>>>>>>>Â Â Ticket <URL:
>>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Could you clarify "8. Level as the pressure level in hPa
or
>>>>>>>>>>> accumulation
>>>>>>>>>>> interval in hours"Â when formatting point observations.Â
My
>>>>>>>>>>> point observation
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> is
>>>>>>>>>>>
>>>>>>>>>>> an anemometer at a fixed height. What value would I put
for
>>>>>>>>>>> this column?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> David Bryan
>>>>>>>>>>>
>>>>>>>>>>
>>>>>
>>>>
>>>>
>
------------------------------------------------
Subject: formatting ascii2nc
From: David Bryan
Time: Thu Jul 14 12:26:55 2011
John & Paul,
I happen to have been at UCAR this past week, doing WRF training.
During this period, I asked one of our instructions on how to relate
the wind values at an eta levels with an AGL height.
Basically, we
got two values related to geopotential height, added them and divided
the sum by 9.81 m/s^2. I made two videos of his assistance here:
http://www.youtube.com/watch?v=GbetuH5pmnQ
http://www.youtube.com/watch?v=usOhip7Anoo
And I've attached the program he wrote (for NCL post processing
utility).
I would have imagined that MET could do the type of interpolation that
we're doing here.
Thanks,
David Bryan
________________________________
From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
To: davidstephenbryan at yahoo.com
Sent: Wednesday, July 13, 2011 12:37 AM
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
David,
My understanding is that the vertical coordinate in WRF is defined on
eta
levels, and those are the levels over which the model integration is
performed. The wrfout files contains output defined on those eta
levels
as well as some derived variables - including things like 2m
temperature
and 10 meter winds.
The WRF PostProcessor code derives many additional fields as wells as
interpolating to pressure levels in the vertical and destaggering the
horizontal grid.
Having said that, I don't work directly with WRF on a regular basis,
and
it you may find it helpful to refer to the WRF users's guide, online
tutorial, and/or write wrfhelp at ucar.edu for more information.
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> OK, so if I understand you, you're saying that WRF does NOT put out
wind
> values at heights other than 10 m? Really?!Â
> Â
> I thought
that, at the boundary of each and every cell, is a U or V
> vector. Are you saying that wind values exist in the model at
higher
> levels but, due to the registry file, they're not output?
> Â
> If you only get wind speed at 10 m, what would any
interpolation be based
> on? Typically when interpolating, you need values at two extremes
(ie,
> 10 m and a much higher value) to interpolate an intermediate value,
but
> we're not given a higher value. If it's based on V1/V2 =
(H1/H2)^a, that
> formula is dubious at best.
>
Â
> If I'm understanding you, this is really a revelation. Do you
think the
> registry could be changed to give us wind at desired heights?
> Â
> Thanks,
> David
>
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Tuesday, July 12, 2011 7:17 PM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> This is John - I wanted to pipe in here.
>
> There is no other "WRF grib" converter available. The other
> post-processing option is the WRF-ARW pinterp tool:
> Â http://www.mmm.ucar.edu/wrf/users/download/get_sources.html
>
> It reads WRF-ARW NetCDF out files, destaggers the grid, and
interpolates
> to pressure levels. However, I took a look at the WRF registry
file for
> ARW and don't think that WRF itself is able to dump
> out winds at vertical levels like 50m and 80m. The vertical
coordinate
> is the model level, and I don't think it's set up to derive vertical
> levels (other than 10m) like that. But I could be wrong.
>
> Given that, I think your only real option for obtaining winds at
vertical
> AGL levels is using the version of WPP I sent. How much error is
> introduced by post-processing is a valid question, and I
> don't know the answer. I'm generally of the opinion that
interpolation
> errors are small relative to the model errors, but it's worth
>
investigating. I don't see any other real options for you though.
>
> Hope that helps.
>
> John
>
> On 07/12/2011 01:22 PM, RAL HelpDesk {for David Bryan} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> OK, but will those results at 50 and 80 m be as accurate as they
were
>> before WPP? What about using the other WRF grib converter?
>>Â
>> Thanks,
>> David Bryan
>>
>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tuesday, July 12, 2011
4:41 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> MET cannot read the NetCDF output of WRF directly. The reason is
that
>> WRF NetCDF data is on staggered Arakawa
>> horizontal grids and on hybrid vertical levels. Observation data
>> always uses pressure, elevation or height for vertical
>> levels. In your case, I think the easiest path is to continue to
>> post-process using WPP as you have been doing (I
>> assume), and then use the module that John sent to generate wind
data at
>> 50m and 80m. Does that make sense?
>>
>> Paul
>>
>>
>>
>> On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> John & Paul,
>>>Â
>>> Thanks for your reply and attention.
>>>Â
>>> MET Tools will accept forecasts in NetCDF format (ie, normal
wrfout
>>> files), correct? And such files should include wind above
ground,
>>> right?
>>>Â
>>> If so, I'll simply quit doing any post-processing grib
conversion and
>>> use WRF's output.
>>>Â
>>> Once I have some output in NetCDF, I'll look into Paul's
scripting
>>> suggestions.
>>>Â
>>> Thanks,
>>> David Bryan
>>>Â
>>>Â
>>>
>>> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Monday, July 11, 2011 7:06 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> Hello David,
>>>
>>> My name is John Halley Gotway. I work on MET development and
support
>>> with Paul, and he and I were just discussing your questions about
>>> running Point-Stat. Paul's helped you with verifying surface
>>> temperature and pressure, and it sounds like you're also
interested in
>>> verifying winds. Looking at the sample data you sent Paul, I
noticed
>>> two things:
>>>Â (1) You wind observations appear to be at 50 and 80 meters - I
assume
>>>
that's on a wind turbine.
>>>Â (2) Using wgrib to look at your model output, I see that you
have
>>> forecasts of U and V, but not at vertical heights above ground.
>>>
>>> We usually have the opposite problem - plenty of forecasts but no
>>> observations for verification. But in your case we have
observations
>>> at 50 and 80 meters but no forecasts to verify! I assume that
>>> you're using the WRF PostProcessor to post-process your WRF
output.Â
>>> Unfortunately, the released version of WPP does not
create output for
>>> winds at the AGL heights you'd need to for winds. However, I
>>> worked on an extension for WPP to output these fields. We're in
a bit
>>> of flux right now as we transition from WPP to the Unified
>>> PostProcessor (UPP). But for now, if
you'd like, I could point you to
>>> a version of WPP that does generate this AGL output:
>>>Â
ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>>>
>>> However, please be aware that this is development code, as-is, and
is
>>> not part of the officially supported release. But I have passed
this
>>> along to the UPP developers for possible inclusion in a
>>> future release.
>>>
>>> So, assuming that you're able to generate U and V forecast output
at
>>> 50-meters and 80-meters, here's what you might try in you Point-
Stat
>>> config file:
>>>
>>> fcst_field[]Â = [ "WIND/Z50", "WIND/Z80" ];
>>> obs_field[]Â Â = [];
>>>
message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just
>>> needs to match what you put in the ASCII observation data. And
it
>>> probably shouldn't be ADPSFC since these are not "surface" obs
>>>
>>> This will verify wind speed directly at 50 and 80 meters.Â
>>>
Unfortunately, Point-Stat can't verify wind direction directly.Â
>>>
Instead, you could use it to dump out VL1L2 lines and verify it in the
>>> STAT-Analysis tool. But to do so, you'd need you observations to
be U
>>> and V vectors (instead of wind speed and direction).
>>>
>>> So one option would be for you to back out the U and V vectors
from
>>> wind speed and direction (it's just some simple trig to do so).Â
And
>>> then include the U and V observations in your ASCII observation
>>>
files. Then you could verify the following in Point-Stat:
>>>
>>> fcst_field[]Â = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80", "VGRD/Z80",
>>> "WIND/Z50", "WIND/Z80" ];
>>> obs_field[]Â Â = [];
>>> message_type[] = [ "PROFLR" ];
>>>
>>> This will verify U, V, and wind speed directly. And, if you have
the
>>> VL1L2 flag turned on in the output_flag in the Point-Stat config
file,
>>> you'll also get VL1L2 lines in your output. Then, you
>>> could use STAT-Analysis to look at wind direction errors. If you
>>> decide to go that route, and need help, just let us know. Here's
an
>>> example of this from the recent MET tutorial:
>>>Â
>>>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>>>
>>> Clear as mud?
>>>
>>> Just let us know if more issues arise. We're happy to help.
>>>
>>> Thanks,
>>> John
>>>
>>>
>>>
>>> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> David,
>>>>
>>>> I accidentally sent you the wrong version of the config file.Â
The
>>>> version I did send
would work with PROFLR message
>>>> obs, by the way. It matches the "surface" fields in the model
data
>>>> file against your original PROFLR obs. I attached
>>>> the version I did intend to send with this email.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>>>> David,
>>>>>
>>>>> I'm still working on your wind verification problem.
>>>>>
>>>>> Regarding PRES and TMP, I may have not made it clear how exactly
>>>>> point_stat treats surface fields. There is more
>>>>> information on what I'm about to tell you in the METv3.0.2
User's
>>>>> Guide pg. 4-1 here:
>>>>>
>>>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>>>
>>>>> Point_stat can only match forecast/obs pairs in three ways:
>>>>>
>>>>> 1. by height in meters for non-"surface" message type obs and
model
>>>>> records
>>>>> 2. by pressure in hPa for non-"surface" message type obs and
model
>>>>> records
>>>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and
SFCSHP)
>>>>> to surface model records
>>>>>
>>>>> For surface level matching, you must specify the "surface"
height in
>>>>> the fcst_field of the config file: 0m for PRES, 2m
>>>>> for TMP and 10m for
winds. Surface GRIB records have 'sfc' in the
>>>>> 12th wgrib field:
>>>>>
>>>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print
>>>>> $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
>>>>> fcst:NAve=0
>>>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
>>>>> fcst:NAve=0
>>>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
>>>>> fcst:NAve=0
>>>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
>>>>> fcst:NAve=0
>>>>> ...
>>>>>
>>>>> To generate matched pairs
with with your obs, I modified your obs
>>>>> files in the following way:
>>>>>
>>>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>>>
>>>>> Then, I changed the fcst_field and message_type fields in the
>>>>> point_stat config file accordingly. I attached the
>>>>> version that I used. The point_stat output is shown below. I
>>>>> realize that this is a bit confusing, but the setup of
>>>>> the observation level is really dictated by the information in
the
>>>>> model GRIB file. In this case, the level there is
>>>>> surface.
>>>>>
>>>>> Unfortunately, if this is all the obs that you have, you only
get
3
>>>>> matched pairs for each field. You really only have
>>>>> 1 exact match if you reduce the beg_ds and end_ds time window to
less
>>>>> than 10 min (+/- 600s), since your obs are valid
>>>>> every 10 minutes. In your case, I think you may want to dump
out
>>>>> only the MPR data which contains matched pair data.
>>>>> Then, once you run point_stat across all of your valid_time
data, you
>>>>> can use the stat_analysis tool to read your MPRs
>>>>> and generate aggregated statistics.
>>>>>
>>>>> If you have any questions, please let me know. Once I have
>>>>> something to give you for verifying your wind data, I'll
>>>>> send you an update.
>>>>>
>>>>>
Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt
-fcst_valid
>>>>> 20100617_13 -outdir . -v 2
>>>>> GSL_RNG_TYPE=mt19937
>>>>> GSL_RNG_SEED=18446744071635082062
>>>>> Forecast File: NREL10061712.grib
>>>>> Climatology File: none
>>>>> Configuration File: PSconfig2.txt
>>>>> Observation File: A9_sfc.nc
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Reading records for PRES/Z0.
>>>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Reading records for TMP/Z2.
>>>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over
>>>>> region FULL, for interpolation method UW_MEAN(1),
>>>>> using 3 pairs.
>>>>> Computing Categorical Statistics.
>>>>> Computing Continuous
Statistics.
>>>>> Computing Scalar Partial Sums.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over
>>>>> region FULL, for interpolation method UW_MEAN(1),
>>>>> using 3 pairs.
>>>>> Computing Categorical Statistics.
>>>>> Computing Continuous Statistics.
>>>>> Computing Scalar Partial Sums.
>>>>>
>>>>>
--------------------------------------------------------------------------------
>>>>>
>>>>> Output file:
./point_stat_010000L_20100617_130000V.stat
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>
>>>>>> Paul,
>>>>>>Â
>>>>>> I've posted the data on your ftp site under the
directory,
>>>>>> Bryan_data.
>>>>>>Â
>>>>>> The contents should be self explanatory, largely. I
included two
>>>>>> versions of the config file I tried. Also, I included the
original
>>>>>> source of the
data, met2010.csv.
>>>>>>Â
>>>>>> Thanks,
>>>>>> David Bryan
>>>>>>
>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>> To: davidstephenbryan at yahoo.com
>>>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>
>>>>>> David,
>>>>>>
>>>>>> Can you please bundle up your model data, obs and config files
using
>>>>>> "tar xzvf" and upload them to our FTP site using
>>>>>> the following
instructions?
>>>>>>
>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>
>>>>>> That will make it easier for me to analyze your situation and
figure
>>>>>> out the best course of action with your data.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>>>
>>>>>>> For the first, it doesn't quite make sense to me that the wind
>>>>>>> direction field could not be verified with point_stat. But I
took
>>>>>>> that out of the second config file just to move forward.
>>>>>>>
>>>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure
>>>>>>> field (or maybe temperature; both were at 3 m). I'm not sure
why
>>>>>>> this is a problem. Could it have to do with a mismatch
between
>>>>>>> fields created with ASCII2NC and those in the grib file?Â
For
>>>>>>> instance, for the pressure value I used the field PRES (001)
in
>>>>>>> ASCII2NC. But as I examine the grib file in IDV, I don't see
a
>>>>>>> simple pressure field. In IDV under 2D grid, I see:Â
>>>>>>> Mean_sea_level_pressure_NAM_model @ msl, Pressure @
cloud_base,
>>>>>>> Pressure @ cloud_tops, Pressure @ surface, and
>>>>>>> Pressure_reduced_to_MSL @ msl. Under 3D grid, I see Pressure
@
>>>>>>> hybrid. The only field the grib file has in common with
table of
>>>>>>> grib parameters that could be used in ASCII2NC
>>>>>>> (http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is
>>>>>>> Pressure reduced to MSL, though of course, my pressure
observations
>>>>>>> are not close to MSL. Temperature seems subject to the same
>>>>>>> circumstances.
>>>>>>>
>>>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC
>>>>>>> had wind speed and direction and the grib file had U and V
wind
>>>>>>> components, that POINT_STAT would sort that out. Is that
correct?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>>>>Â /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>>>Â
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>>>Â -fcst_valid 20100617_17 \
>>>>>>>>Â -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>>>Â -v 9
>>>>>>>
>>>>>>>
>>>>>>> ERROR: PointStatConfInfo::process_config() -> the wind
direction
>>>>>>> field may not b
>>>>>>> e verified using point_stat.
>>>>>>>
>>>>>>> [amec at WRF-master bin]$ ./point_statÂ
>>>>>>>
/ext/models/WRF/NREL/10061712/NREL1006171
>>>>>>> 2.gribÂ
/home/WRF/Development/David_Bryan/ASCII2NC/A9.ncÂ
>>>>>>> /home/WRF/Developme
>>>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid
20100617_17Â
>>>>>>> -outdir /home
>>>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/Â -v 9
>>>>>>> GSL_RNG_TYPE=mt19937
>>>>>>> GSL_RNG_SEED=18446744072421513038
>>>>>>> Forecast File: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>> Climatology File: none
>>>>>>> Configuration File:
>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>>>
>>>>>>>
--------------------------------------------------------------------------------
>>>>>>>
>>>>>>> Reading records for MISSING/Z3.
>>>>>>>
>>>>>>>
>>>>>>> ERROR: read_levels_grib() -> no GRIB records matching
MISSING/Z3
>>>>>>> found in GRIB f
>>>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>
Cc:
>>>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> The point_stat usage statement indicates that the arguments
should
>>>>>>> be specified in the following order. Please try this
>>>>>>> (you should be able just copy/paste):
>>>>>>>
>>>>>>> ./point_stat \
>>>>>>>Â /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>>Â /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>>Â -fcst_valid 20100617_17
\
>>>>>>>Â -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>>Â -v 9
>>>>>>>
>>>>>>> I think that may be the cause of your problem.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> I've attached my config
file.
>>>>>>>>
>>>>>>>> When I tried running Point Stat, I got the error message
below.Â
>>>>>>>> I'm not clear
>>>>>>>> if it's coming from the config file or the grib file. I
will
>>>>>>>> point out, though,
>>>>>>>> that the grib files are produced by the WRF post processor
(though
>>>>>>>> they are
>>>>>>>> concatenated, using Linux cat). The text in the message
isn't in
>>>>>>>> the config
>>>>>>>> file.
>>>>>>>>
>>>>>>>> Any suggestions would be welcomed.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
David
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
>>>>>>>> /ext/models/WRF/NRE
>>>>>>>> L/10061712/NREL10061712.grib
>>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt
-outdir
>>>>>>>> /home/WRF/Develo
>>>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>>>
>>>>>>>>
>>>>>>>>Â Â config() -> syntax error in file
>>>>>>>> "/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>>>>
grib"
>>>>>>>>
>>>>>>>>    line = 1
>>>>>>>>
>>>>>>>>Â Â Â Â econfig_column = 7
>>>>>>>>
>>>>>>>>    text = "^"
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> GRIB
>>>>>>>> ____
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message ----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>> Regarding your observations, it might be easiest to include
all of
>>>>>>>> your
>>>>>>>> observations into a single NetCDF file. You can
>>>>>>>> denote the two different sensor locations using Station_ID
(column
>>>>>>>> 2) in
>>>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>>>> option in the point_stat configuration file to specify
which
>>>>>>>> stations obs you
>>>>>>>> want to use. Otherwise, you can use both
>>>>>>>> of your current NetCDF obs files in a single point_stat call
by
>>>>>>>> passing one as
>>>>>>>> the "obs_file" and the other one using
>>>>>>>> the -point_obs optional argument.
>>>>>>>>
>>>>>>>> Regarding the forecast times, the following is from the
point_stat
>>>>>>>> usage
>>>>>>>> statement, which you can see by running
>>>>>>>> point_stat with no arguments:
>>>>>>>>
>>>>>>>>Â Â "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets
the
>>>>>>>> forecast valid
time
>>>>>>>> to be verified (optional).
>>>>>>>>Â Â "-fcst_lead time" in HH[MMSS] format sets the forecast
lead
>>>>>>>> time to be
>>>>>>>> verified (optional).
>>>>>>>>
>>>>>>>> If your model is initiated (we call it initialized) at
>>>>>>>> 20110625_1200, then that
>>>>>>>> is your fcst_init time, not your
>>>>>>>> fcst_valid time. If you are interested in verifying the
forecast
>>>>>>>> five hours
>>>>>>>> into your 48-hour run, then your fcst_lead
>>>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>>>
>>>>>>>> Regarding
levels, ZNNN is AGL and it can be an arbitrary number of
>>>>>>>> digits long.Â
>>>>>>>> Typically, for wind ground-based
>>>>>>>> measurements, the fcst_field level looks something like "Z10"
for
>>>>>>>> a sensor at
>>>>>>>> 10m above the ground. Unfortunately, MET
>>>>>>>> does not read model data on eta levels. In order to feed
your
>>>>>>>> WRF data to MET,
>>>>>>>> it must be post-processed using either
>>>>>>>> the WRF Post Processor tool (WPP) or p_interp. This process
will
>>>>>>>> put the data
>>>>>>>> onto levels in meters and/or pressure,
>>>>>>>> both of which MET can
read.
>>>>>>>>
>>>>>>>> Please let me know if you have any more questions.
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>> Paul,
>>>>>>>>>
>>>>>>>>> Thanks again for your help. Now, I'm on to creating the
config
>>>>>>>>> file for
Point
>>>>>>>>
>>>>>>>>> Stat.
>>>>>>>>>
>>>>>>>>> My model output is for 48-hour runs of WRF with hourly
output.Â
>>>>>>>>> The output is
>>>>>>>>> concatenated into a single file, 48 time steps in a single
>>>>>>>>> file. (Perhaps
>>>>>>>>> that's not practical.)
>>>>>>>>>
>>>>>>>>> My observations are for two met towers, each with two
anemometers
>>>>>>>>> at different
>>>>>>>>
>>>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer
>>>>>>>>> data into
its
>>>>>>>>
>>>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>>>
>>>>>>>>> PointStat will only analyze one model time per run.
>>>>>>>>> In the command line arguments, I will have to specify
>>>>>>>>> -fcst_valid. For
>>>>>>>>> instance, if my WRF run is initiated with NAM data produced
on
>>>>>>>>> the
>>>>>>>>> 20110625_1200
>>>>>>>>>
>>>>>>>>> (and my model is initiated on the same time), that's my
valid
>>>>>>>>> time.
>>>>>>>>> I will also have to specify -fcst_lead, which indicates
where
>>>>>>>>> (temporally) in
>>>>>>>>> the model's forecast I want to analyze. So with that run I
>>>>>>>>> mentioned, if I
>>>>>>>>> wanted to analyze five hours into my 48-hour run, my lead
time
>>>>>>>>> would be
>>>>>>>>> 20110625_1700.
>>>>>>>>> The purpose of specifying a temporal range of observation
values
>>>>>>>>> is for
>>>>>>>>> interpolation across the temporal dimension.
>>>>>>>>> The model time that's compared to observation time is the
lead
>>>>>>>>>
time.
>>>>>>>>> If any of my understanding of how time specification in
Point
>>>>>>>>> Stat works is
>>>>>>>>> incorrect, please let me know.
>>>>>>>>>
>>>>>>>>> My other main question concerns specifying the levels for
each
>>>>>>>>> fcst_field. The
>>>>>>>>>
>>>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical
>>>>>>>>> level. Is
>>>>>>>>> that
>>>>>>>>>
>>>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL, can
it
>>>>>>>>> be more than
>>>>>>>>>
three
>>>>>>>>>
>>>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided
default
>>>>>>>>> config file
>>>>>>>>> includes
>>>>>>>>>
>>>>>>>>> the following comment:Â "GC/ZNNN-NNN for a range of
vertical
>>>>>>>>> levels (MSL or
>>>>>>>>> AGL)."Â How would one specify whether it's MSL or AGL?Â
Also,
>>>>>>>>> WRF output is in
>>>>>>>>
>>>>>>>>> eta-levels which is more related to pressure levels. Does
that
>>>>>>>>> matter?
>>>>>>>>>
>>>>>>>>> Also, I assume that the forecast and observation
thresholds serve
>>>>>>>>> as a quality
>>>>>>>>
>>>>>>>>> control.
>>>>>>>>>
>>>>>>>>> Thanks again for all your assistance!
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>Â
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> David,
>>>>>>>>>
>>>>>>>>>> I do want to interpolate to the height of the sensor, so
could
>>>>>>>>>> you recommend
>>>>>>>> a
>>>>>>>>>> message type? Frankly, I'm not familiar with these
message
>>>>>>>>>> types, so could
>>>>>>>>> you
>>>>>>>>>> refer me to a link or reference to these message
types?
>>>>>>>>>
>>>>>>>>> I think PROFLR is the appropriate message type for your
>>>>>>>>> situation. Here is a
>>>>>>>>> reference on the message types:
>>>>>>>>>
>>>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>>>
>>>>>>>>> Message types are a concept that is associated with PrepBUFR
>>>>>>>>> point observations
>>>>>>>>>
>>>>>>>>> produced by NCEP. These observations
>>>>>>>>> are often used in MET. The logic that MET uses for
these
>>>>>>>>> message types are
>>>>>>>>> applied to all observations, regardless of
>>>>>>>>> source.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
>>>>>>>>>> terrain at that
>>>>>>>>>> point and column nine would be the sensor's height about
>>>>>>>>>> ground. But it
>>>>>>>>> sounds
>>>>>>>>>> like column 6 should be the sensor's total height above sea
>>>>>>>>>> level, right? If
>>>>>>>>>> so, what should column 9
have?
>>>>>>>>>
>>>>>>>>> No, you are correct. I should have been more clear in my
>>>>>>>>> previous email.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> will Point-Stat match the observation time to the model
time and
>>>>>>>>>> calculate its statistics from that?
>>>>>>>>>
>>>>>>>>> Point-Stat was designed to be run once for each forecast
valid
>>>>>>>>> time. If your
>>>>>>>>> model data files contain single 48 hour
>>>>>>>>> forecasts, run Point-Stat once for each model file. Each
time,
>>>>>>>>> you will pass
>>>>>>>>>
the model data file you want to verify,
>>>>>>>>> the single observation file and the configuration file. In
the
>>>>>>>>> configuration
>>>>>>>>> file, you should decide the time window
>>>>>>>>> inside which observations "match" the model valid time.Â
The
>>>>>>>>> settings for this
>>>>>>>>
>>>>>>>>> are beg_ds and end_ds. You could also
>>>>>>>>> use the Point-Stat command line parameters obs_valid_beg and
>>>>>>>>> obs_valid_end.
>>>>>>>>>
>>>>>>>>> If you have multiple forecasts in a single model data file,
you
>>>>>>>>> will need to
>>>>>>>>>
use
>>>>>>>>>
>>>>>>>>> the command line parameter fcst_valid
>>>>>>>>> time to specify which forecast Point-Stat should verify.Â
In
>>>>>>>>> this case,
>>>>>>>>> Point-Stat must still be run once for each valid
>>>>>>>>> time, where each call differs only by the value passed to
the
>>>>>>>>> fcst_valid
>>>>>>>>> parameter.
>>>>>>>>>
>>>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>>
Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>
>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>
>>>>>>>>>> Paul,
>>>>>>>>>>
>>>>>>>>>> One other question. The netcdf file I'll be creating with
>>>>>>>>>> ascii2nc covers six
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> months. So I'll have one six-month file for
observations.Â
I
>>>>>>>>>> want to use
>>>>>>>>>> Point-Stat to compare that file to a series of 48-hour
>>>>>>>>>> forecasts. My
>>>>>>>>>> question--will Point-Stat match the observation time to the
>>>>>>>>>> model time and
>>>>>>>>>> calculate its statistics from that? In other words,
>>>>>>>>>> Point-Stat's output for
>>>>>>>>>> each forecast is not based on periods for which there is no
>>>>>>>>>> forecast data,
>>>>>>>>>> correct?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
David
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Original Message ----
>>>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>
>>>>>>>>>> Thanks Paul,
>>>>>>>>>>
>>>>>>>>>> That's very
helpful.Â
>>>>>>>>>>
>>>>>>>>>> I do want to interpolate to the height of the sensor, so
could
>>>>>>>>>> you recommend a
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> message type? Frankly, I'm not familiar with these
message
>>>>>>>>>> types, so could you
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>>>
>>>>>>>>>> Also, I had assumed column 6 would get the elevation of the
>>>>>>>>>> terrain at that
>>>>>>>>>> point and column nine would be the sensor's
height about
>>>>>>>>>> ground. But it sounds
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> like column 6 should be the sensor's total height above sea
>>>>>>>>>> level, right? If
>>>>>>>>
>>>>>>>>>> so, what should column 9 have?
>>>>>>>>>>
>>>>>>>>>> Thanks again,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Original Message ----
>>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>
>>>>>>>>>> David,
>>>>>>>>>>
>>>>>>>>>> This column does not apply to your data, since you are only
>>>>>>>>>> measuring wind
>>>>>>>>>> speed
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> at a fixed height above the ground.Â
I
>>>>>>>>>> would recommend putting -9999 in that column, since that is
the
>>>>>>>>>> value that MET
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> uses for bad data or N/A.
>>>>>>>>>>
>>>>>>>>>> You should use column 6 to enter the elevation of your
sensor,
>>>>>>>>>> but be advised
>>>>>>>>
>>>>>>>>>> that if you use a "surface" message type
>>>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
>>>>>>>>>> regardless of what you
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> enter in column
6. If you want MET to
>>>>>>>>>> interpolate model data to the height of your sensor, use
another
>>>>>>>>>> message type.Â
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I hope this is clear. Please let me
>>>>>>>>>> know if you have any questions.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Paul
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>>
>>>>>>>>>>> Tue Jul 05 06:25:54 2011:
Request 48043 was acted upon.
>>>>>>>>>>> Transaction: Ticket created by davidstephenbryan at yahoo.com
>>>>>>>>>>>Â Â Â Â Â Queue: met_help
>>>>>>>>>>>Â Â Â Â Subject: formatting ascii2nc
>>>>>>>>>>>Â Â Â Â Â Owner: Nobody
>>>>>>>>>>>Â Â Requestors: davidstephenbryan at yahoo.com
>>>>>>>>>>>Â Â Â Â Status: new
>>>>>>>>>>>Â Â Ticket
<URL:
>>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Could you clarify "8. Level as the pressure level in hPa
or
>>>>>>>>>>> accumulation
>>>>>>>>>>> interval in hours"Â when formatting point observations.Â
My
>>>>>>>>>>> point observation
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> is
>>>>>>>>>>>
>>>>>>>>>>> an anemometer at a fixed height. What value would I put
for
>>>>>>>>>>> this column?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> David Bryan
>>>>>>>>>>>
>>>>>>>>>>
>>>>>
>>>>
>>>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
From: John Halley Gotway
Time: Thu Jul 14 15:40:26 2011
David,
Thanks for the info. Hope you visit to NCAR went well.
While MET was written primarily with users of WRF in mind, it was not
done entirely so. So while many users of MET are verifying WRF
output, a good number are verifying output from other NWP models.
In that sense, tying MET too closely to the NetCDF output of WRF
would have been limiting.
And one of our design principles for MET is to focus specifically on
the area of verification and not let MET sway into other areas - we
don't have time or funding to do so. I mention this in regards
to model post-processing. I sit in on the WRF Post Processing working
group - when the group meets - and know that model post-processing is
quite a can of worms. We don't want to be in the business
of post-processing raw model output prior to verification. Instead,
we rely on the output of the model post-processors that are designed
specifically for that purpose.
Other areas that we try to stay out of are ensemble post-processing
and providing robust graphical verification output.
Regarding post-processing to AGL levels - I definitely agree. That
interpolation should be handled in the model post-processing, and
we've already discussed the status of that.
Thanks,
John
On 07/14/2011 12:26 PM, RAL HelpDesk {for David Bryan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>
> John & Paul,
>
> I happen to have been at UCAR this past week, doing WRF training.
During this period, I asked one of our instructions on how to relate
the wind values at an eta levels with an AGL height.
>
> Basically, we got two values related to geopotential height, added
them and divided the sum by 9.81 m/s^2. I made two videos of his
assistance here:
>
> http://www.youtube.com/watch?v=GbetuH5pmnQ
>
>
> http://www.youtube.com/watch?v=usOhip7Anoo
>
>
> And I've attached the program he wrote (for NCL post processing
utility).
>
> I would have imagined that MET could do the type of interpolation
that we're doing here.
>
> Thanks,
> David Bryan
>
>
>
>
> ________________________________
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> To: davidstephenbryan at yahoo.com
> Sent: Wednesday, July 13, 2011 12:37 AM
> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>
> David,
>
> My understanding is that the vertical coordinate in WRF is defined
on eta
> levels, and those are the levels over which the model integration is
> performed. The wrfout files contains output defined on those eta
levels
> as well as some derived variables - including things like 2m
temperature
> and 10 meter winds.
>
> The WRF PostProcessor code derives many additional fields as wells
as
> interpolating to pressure levels in the vertical and destaggering
the
> horizontal grid.
>
> Having said that, I don't work directly with WRF on a regular basis,
and
> it you may find it helpful to refer to the WRF users's guide, online
> tutorial, and/or write wrfhelp at ucar.edu for more information.
>
> Thanks,
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>
>> OK, so if I understand you, you're saying that WRF does NOT put out
wind
>> values at heights other than 10 m? Really?!Â
>> Â
>> I thought that, at the boundary of each and every cell, is a U
or V
>> vector. Are you saying that wind values exist in the model at
higher
>> levels but, due to the registry file, they're not output?
>> Â
>> If you only get wind speed at 10 m, what would any interpolation be
based
>> on? Typically when interpolating, you need values at two extremes
(ie,
>> 10 m and a much higher value) to interpolate an intermediate value,
but
>> we're not given a higher value. If it's based on V1/V2 =
(H1/H2)^a, that
>> formula is dubious at best.
>>
> Â
>> If I'm understanding you, this is really a revelation. Do you
think the
>> registry could be changed to give us wind at desired heights?
>> Â
>> Thanks,
>> David
>>
>> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
>> To: davidstephenbryan at yahoo.com
>> Sent: Tuesday, July 12, 2011 7:17 PM
>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>
>> David,
>>
>> This is John - I wanted to pipe in here.
>>
>> There is no other "WRF grib" converter available. The other
>> post-processing option is the WRF-ARW pinterp tool:
>> Â http://www.mmm.ucar.edu/wrf/users/download/get_sources.html
>>
>> It reads WRF-ARW NetCDF out files, destaggers the grid, and
interpolates
>> to pressure levels. However, I took a look at the WRF registry
file for
>> ARW and don't think that WRF itself is able to dump
>> out winds at vertical levels like 50m and 80m. The vertical
coordinate
>> is the model level, and I don't think it's set up to derive
vertical
>> levels (other than 10m) like that. But I could be wrong.
>>
>> Given that, I think your only real option for obtaining winds at
vertical
>> AGL levels is using the version of WPP I sent. How much error is
>> introduced by post-processing is a valid question, and I
>> don't know the answer. I'm generally of the opinion that
interpolation
>> errors are small relative to the model errors, but it's worth
>>
> investigating. I don't see any other real options for you though.
>>
>> Hope that helps.
>>
>> John
>>
>> On 07/12/2011 01:22 PM, RAL HelpDesk {for David Bryan} wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>
>>> OK, but will those results at 50 and 80 m be as accurate as they
were
>>> before WPP? What about using the other WRF grib converter?
>>> Â
>>> Thanks,
>>> David Bryan
>>>
>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>> To: davidstephenbryan at yahoo.com
>>> Sent: Tuesday, July 12, 2011
> 4:41 PM
>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>
>>> David,
>>>
>>> MET cannot read the NetCDF output of WRF directly. The reason is
that
>>> WRF NetCDF data is on staggered Arakawa
>>> horizontal grids and on hybrid vertical levels. Observation data
>>> always uses pressure, elevation or height for vertical
>>> levels. In your case, I think the easiest path is to continue to
>>> post-process using WPP as you have been doing (I
>>> assume), and then use the module that John sent to generate wind
data at
>>> 50m and 80m. Does that make sense?
>>>
>>> Paul
>>>
>>>
>>>
>>> On 07/12/2011 01:03 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>
>>>> John & Paul,
>>>> Â
>>>> Thanks for your reply and attention.
>>>> Â
>>>> MET Tools will accept forecasts in NetCDF format (ie, normal
wrfout
>>>> files), correct? And such files should include wind above
ground,
>>>> right?
>>>> Â
>>>> If so, I'll simply quit doing any post-processing grib conversion
and
>>>> use WRF's output.
>>>> Â
>>>> Once I have some output in NetCDF, I'll look into Paul's
scripting
>>>> suggestions.
>>>> Â
>>>> Thanks,
>>>> David Bryan
>>>> Â
>>>> Â
>>>>
>>>> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
>>>> To: davidstephenbryan at yahoo.com
>>>> Sent: Monday, July 11, 2011 7:06 PM
>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>
>>>> Hello David,
>>>>
>>>> My name is John Halley Gotway. I work on MET development and
support
>>>> with Paul, and he and I were just discussing your questions about
>>>> running Point-Stat. Paul's helped you with verifying surface
>>>> temperature and pressure, and it sounds like you're also
interested in
>>>> verifying winds. Looking at the sample data you sent Paul, I
noticed
>>>> two things:
>>>> Â (1) You wind observations appear to be at 50 and 80 meters - I
assume
>>>>
> that's on a wind turbine.
>>>> Â (2) Using wgrib to look at your model output, I see that you
have
>>>> forecasts of U and V, but not at vertical heights above ground.
>>>>
>>>> We usually have the opposite problem - plenty of forecasts but no
>>>> observations for verification. But in your case we have
observations
>>>> at 50 and 80 meters but no forecasts to verify! I assume that
>>>> you're using the WRF PostProcessor to post-process your WRF
output.Â
>>>> Unfortunately, the released version of WPP does not create output
for
>>>> winds at the AGL heights you'd need to for winds. However, I
>>>> worked on an extension for WPP to output these fields. We're in
a bit
>>>> of flux right now as we transition from WPP to the Unified
>>>> PostProcessor (UPP). But for now, if
> you'd like, I could point you to
>>>> a version of WPP that does generate this AGL output:
>>>> Â
ftp://ftp.rap.ucar.edu/incoming/irap/johnhg/WPPV3_AGL_WINDS.tar.gz
>>>>
>>>> However, please be aware that this is development code, as-is,
and is
>>>> not part of the officially supported release. But I have passed
this
>>>> along to the UPP developers for possible inclusion in a
>>>> future release.
>>>>
>>>> So, assuming that you're able to generate U and V forecast output
at
>>>> 50-meters and 80-meters, here's what you might try in you Point-
Stat
>>>> config file:
>>>>
>>>> fcst_field[]Â = [ "WIND/Z50", "WIND/Z80" ];
>>>> obs_field[]Â Â = [];
>>>>
> message_type[] = [ "PROFLR" ]; // Really this doesn't matter - it
just
>>>> needs to match what you put in the ASCII observation data. And
it
>>>> probably shouldn't be ADPSFC since these are not "surface" obs
>>>>
>>>> This will verify wind speed directly at 50 and 80 meters.Â
>>>> Unfortunately, Point-Stat can't verify wind direction directly.Â
>>>> Instead, you could use it to dump out VL1L2 lines and verify it
in the
>>>> STAT-Analysis tool. But to do so, you'd need you observations
to be U
>>>> and V vectors (instead of wind speed and direction).
>>>>
>>>> So one option would be for you to back out the U and V vectors
from
>>>> wind speed and direction (it's just some simple trig to do so).Â
And
>>>> then include the U and V observations in your ASCII observation
>>>>
> files. Then you could verify the following in Point-Stat:
>>>>
>>>> fcst_field[]Â = [ "UGRD/Z50", "VGRD/Z50", "UGRD/Z80",
"VGRD/Z80",
>>>> "WIND/Z50", "WIND/Z80" ];
>>>> obs_field[]Â Â = [];
>>>> message_type[] = [ "PROFLR" ];
>>>>
>>>> This will verify U, V, and wind speed directly. And, if you
have the
>>>> VL1L2 flag turned on in the output_flag in the Point-Stat config
file,
>>>> you'll also get VL1L2 lines in your output. Then, you
>>>> could use STAT-Analysis to look at wind direction errors. If
you
>>>> decide to go that route, and need help, just let us know.Â
Here's an
>>>> example of this from the recent MET tutorial:
>>>> Â
>>>>
http://www.dtcenter.org/met/users/support/OnLinePractical/OnLinePractical_611/point_stat/wind_dir.php
>>>>
>>>> Clear as mud?
>>>>
>>>> Just let us know if more issues arise. We're happy to help.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>>
>>>>
>>>> On 07/11/2011 03:13 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>
>>>>> David,
>>>>>
>>>>> I accidentally sent you the wrong version of the config file.Â
The
>>>>> version I did send
> would work with PROFLR message
>>>>> obs, by the way. It matches the "surface" fields in the model
data
>>>>> file against your original PROFLR obs. I attached
>>>>> the version I did intend to send with this email.
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 07/11/2011 03:09 PM, Paul Oldenburg wrote:
>>>>>> David,
>>>>>>
>>>>>> I'm still working on your wind verification problem.
>>>>>>
>>>>>> Regarding PRES and TMP, I may have not made it clear how
exactly
>>>>>> point_stat treats surface fields. There is more
>>>>>> information on what I'm about to tell you in the METv3.0.2
User's
>>>>>> Guide pg. 4-1 here:
>>>>>>
>>>>>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v3.0.2.pdf
>>>>>>
>>>>>> Point_stat can only match forecast/obs pairs in three ways:
>>>>>>
>>>>>> 1. by height in meters for non-"surface" message type obs and
model
>>>>>> records
>>>>>> 2. by pressure in hPa for non-"surface" message type obs and
model
>>>>>> records
>>>>>> 3. by pairing "surface" message type obs (i.e. ADPSFC and
SFCSHP)
>>>>>> to surface model records
>>>>>>
>>>>>> For surface level matching, you must specify the "surface"
height in
>>>>>> the fcst_field of the config file: 0m for PRES, 2m
>>>>>> for TMP and 10m for
> winds. Surface GRIB records have 'sfc' in the
>>>>>> 12th wgrib field:
>>>>>>
>>>>>> $ wgrib NREL10061712.grib | perl -e'while(<>){ @l = split(":");
print
>>>>>> $_ if $l[11] =~ /^sfc$/; }' | egrep ':TMP:'
>>>>>>
357:1366904:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=1:P2=0:TimeU=1:sfc:1hr
>>>>>> fcst:NAve=0
>>>>>>
818:2928104:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=2:P2=0:TimeU=1:sfc:2hr
>>>>>> fcst:NAve=0
>>>>>>
1279:4479536:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=3:P2=0:TimeU=1:sfc:3hr
>>>>>> fcst:NAve=0
>>>>>>
1740:6027152:d=10061712:TMP:kpds5=11:kpds6=1:kpds7=0:TR=0:P1=4:P2=0:TimeU=1:sfc:4hr
>>>>>> fcst:NAve=0
>>>>>> ...
>>>>>>
>>>>>> To generate matched pairs
> with with your obs, I modified your obs
>>>>>> files in the following way:
>>>>>>
>>>>>> $ cat A9.txt | sed -r 's/^PROFLR/ADPSFC/' > A9_sfc.txt
>>>>>> $ ascii2nc A9_sfc.txt A9_sfc.nc -v 3
>>>>>>
>>>>>> Then, I changed the fcst_field and message_type fields in the
>>>>>> point_stat config file accordingly. I attached the
>>>>>> version that I used. The point_stat output is shown below.Â
I
>>>>>> realize that this is a bit confusing, but the setup of
>>>>>> the observation level is really dictated by the information in
the
>>>>>> model GRIB file. In this case, the level there is
>>>>>> surface.
>>>>>>
>>>>>> Unfortunately, if this is all the obs that you have, you only
get
> 3
>>>>>> matched pairs for each field. You really only have
>>>>>> 1 exact match if you reduce the beg_ds and end_ds time window
to less
>>>>>> than 10 min (+/- 600s), since your obs are valid
>>>>>> every 10 minutes. In your case, I think you may want to dump
out
>>>>>> only the MPR data which contains matched pair data.
>>>>>> Then, once you run point_stat across all of your valid_time
data, you
>>>>>> can use the stat_analysis tool to read your MPRs
>>>>>> and generate aggregated statistics.
>>>>>>
>>>>>> If you have any questions, please let me know. Once I have
>>>>>> something to give you for verifying your wind data, I'll
>>>>>> send you an update.
>>>>>>
>>>>>>
> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> $ point_stat NREL10061712.grib A9_sfc.nc PSconfig2.txt
-fcst_valid
>>>>>> 20100617_13 -outdir . -v 2
>>>>>> GSL_RNG_TYPE=mt19937
>>>>>> GSL_RNG_SEED=18446744071635082062
>>>>>> Forecast File: NREL10061712.grib
>>>>>> Climatology File: none
>>>>>> Configuration File: PSconfig2.txt
>>>>>> Observation File: A9_sfc.nc
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Reading records for PRES/Z0.
>>>>>> For PRES/Z0 found 1 forecast levels and 0 climatology levels.
>>>>>>
>>>>>>
>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Reading records for TMP/Z2.
>>>>>> For TMP/Z2 found 1 forecast levels and 0 climatology levels.
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Searching 184440 observations from 30740 PrepBufr messages.
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Processing PRES/Z0 versus PRES/Z0, for observation type ADPSFC,
over
>>>>>> region FULL, for interpolation method UW_MEAN(1),
>>>>>> using 3 pairs.
>>>>>> Computing Categorical Statistics.
>>>>>> Computing Continuous
> Statistics.
>>>>>> Computing Scalar Partial Sums.
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over
>>>>>> region FULL, for interpolation method UW_MEAN(1),
>>>>>> using 3 pairs.
>>>>>> Computing Categorical Statistics.
>>>>>> Computing Continuous Statistics.
>>>>>> Computing Scalar Partial Sums.
>>>>>>
>>>>>>
--------------------------------------------------------------------------------
>>>>>>
>>>>>> Output file:
> ./point_stat_010000L_20100617_130000V.stat
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 07/11/2011 01:10 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>
>>>>>>> Paul,
>>>>>>> Â
>>>>>>> I've posted the data on your ftp site under the directory,
>>>>>>> Bryan_data.
>>>>>>> Â
>>>>>>> The contents should be self explanatory, largely. I included
two
>>>>>>> versions of the config file I tried. Also, I included the
original
>>>>>>> source of the
> data, met2010.csv.
>>>>>>> Â
>>>>>>> Thanks,
>>>>>>> David Bryan
>>>>>>>
>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>> Sent: Friday, July 8, 2011 5:37 PM
>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>
>>>>>>> David,
>>>>>>>
>>>>>>> Can you please bundle up your model data, obs and config files
using
>>>>>>> "tar xzvf" and upload them to our FTP site using
>>>>>>> the following
> instructions?
>>>>>>>
>>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>>
>>>>>>> That will make it easier for me to analyze your situation and
figure
>>>>>>> out the best course of action with your data.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 07/08/2011 11:52 AM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043
>
>>>>>>>>
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> Below are outputs from two attempts to run Point_Stat.
>>>>>>>>
>>>>>>>> For the first, it doesn't quite make sense to me that the
wind
>>>>>>>> direction field could not be verified with point_stat. But
I took
>>>>>>>> that out of the second config file just to move forward.
>>>>>>>>
>>>>>>>> For the second, it appears the "MISSING/Z3" applies to the
pressure
>>>>>>>> field (or maybe temperature; both were at 3 m). I'm not
sure why
>>>>>>>> this is a problem. Could it have to do with a mismatch
between
>>>>>>>> fields created with ASCII2NC and those in the grib file?Â
> For
>>>>>>>> instance, for the pressure value I used the field PRES (001)
in
>>>>>>>> ASCII2NC. But as I examine the grib file in IDV, I don't
see a
>>>>>>>> simple pressure field. In IDV under 2D grid, I see:Â
>>>>>>>> Mean_sea_level_pressure_NAM_model @ msl, Pressure @
cloud_base,
>>>>>>>> Pressure @ cloud_tops, Pressure @ surface, and
>>>>>>>> Pressure_reduced_to_MSL @ msl. Under 3D grid, I see
Pressure @
>>>>>>>> hybrid. The only field the grib file has in common with
table of
>>>>>>>> grib parameters that could be used in ASCII2NC
>>>>>>>> (http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html) is
>>>>>>>> Pressure reduced to MSL, though of course, my pressure
observations
>>>>>>>> are not close to MSL. Temperature seems subject to the same
>>>>>>>> circumstances.
>>>>>>>>
>>>>>>>> Similarly, I had assumed, if the netcdf file produced by
ASCII2NC
>>>>>>>> had wind speed and direction and the grib file had U and V
wind
>>>>>>>> components, that POINT_STAT would sort that out. Is that
correct?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>
>>>>>>>> [amec at WRF-master bin]$ ./point_stat \
>>>>>>>>> Â /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>>>> Â
> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>>>> Â /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>>>> Â -fcst_valid 20100617_17 \
>>>>>>>>> Â -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>>>> Â -v 9
>>>>>>>>
>>>>>>>>
>>>>>>>> ERROR: PointStatConfInfo::process_config() -> the wind
direction
>>>>>>>> field may not b
>>>>>>>> e verified using point_stat.
>>>>>>>>
>>>>>>>> [amec at WRF-master bin]$ ./point_statÂ
>>>>>>>> /ext/models/WRF/NREL/10061712/NREL1006171
>>>>>>>> 2.gribÂ
> /home/WRF/Development/David_Bryan/ASCII2NC/A9.ncÂ
>>>>>>>> /home/WRF/Developme
>>>>>>>> nt/David_Bryan/ASCII2NC/PSconfig2.txt -fcst_valid
20100617_17Â
>>>>>>>> -outdir /home
>>>>>>>> /WRF/Development/David_Bryan/ASCII2NC/PointStat/Â -v 9
>>>>>>>> GSL_RNG_TYPE=mt19937
>>>>>>>> GSL_RNG_SEED=18446744072421513038
>>>>>>>> Forecast File:
/ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>>> Climatology File: none
>>>>>>>> Configuration File:
>>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig2.txt
>>>>>>>> Observation File:
/home/WRF/Development/David_Bryan/ASCII2NC/A9.nc
>>>>>>>>
>>>>>>>>
>
--------------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Reading records for MISSING/Z3.
>>>>>>>>
>>>>>>>>
>>>>>>>> ERROR: read_levels_grib() -> no GRIB records matching
MISSING/Z3
>>>>>>>> found in GRIB f
>>>>>>>> ile: /ext/models/WRF/NREL/10061712/NREL10061712.grib
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>
> Cc:
>>>>>>>> Sent: Thursday, July 7, 2011 3:54 PM
>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>
>>>>>>>> David,
>>>>>>>>
>>>>>>>> The point_stat usage statement indicates that the arguments
should
>>>>>>>> be specified in the following order. Please try this
>>>>>>>> (you should be able just copy/paste):
>>>>>>>>
>>>>>>>> ./point_stat \
>>>>>>>> Â /ext/models/WRF/NREL/10061712/NREL10061712.grib \
>>>>>>>> Â /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc \
>>>>>>>> Â /home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt \
>>>>>>>> Â -fcst_valid 20100617_17
> \
>>>>>>>> Â -outdir
/home/WRF/Development/David_Bryan/ASCII2NC/PointStat/ \
>>>>>>>> Â -v 9
>>>>>>>>
>>>>>>>> I think that may be the cause of your problem.
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/07/2011 12:20 PM, RAL HelpDesk {for David Bryan} wrote:
>>>>>>>>>
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>
>>>>>>>>> Paul,
>>>>>>>>>
>>>>>>>>> I've attached my config
> file.
>>>>>>>>>
>>>>>>>>> When I tried running Point Stat, I got the error message
below.Â
>>>>>>>>> I'm not clear
>>>>>>>>> if it's coming from the config file or the grib file. I
will
>>>>>>>>> point out, though,
>>>>>>>>> that the grib files are produced by the WRF post processor
(though
>>>>>>>>> they are
>>>>>>>>> concatenated, using Linux cat). The text in the message
isn't in
>>>>>>>>> the config
>>>>>>>>> file.
>>>>>>>>>
>>>>>>>>> Any suggestions would be welcomed.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
> David
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [amec at WRF-master bin]$ ./point_stat -fcst_valid 20100617_17
>>>>>>>>> /ext/models/WRF/NRE
>>>>>>>>> L/10061712/NREL10061712.grib
>>>>>>>>> /home/WRF/Development/David_Bryan/ASCII2NC/A9.nc /
>>>>>>>>> home/WRF/Development/David_Bryan/ASCII2NC/PSconfig.txt
-outdir
>>>>>>>>> /home/WRF/Develo
>>>>>>>>> pment/David_Bryan/ASCII2NC/PointStat/ -v 9
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Â Â config() -> syntax error in file
>>>>>>>>> "/ext/models/WRF/NREL/10061712/NREL10061712.
>>>>>>>>>
> grib"
>>>>>>>>>
>>>>>>>>>     line = 1
>>>>>>>>>
>>>>>>>>> Â Â Â Â econfig_column = 7
>>>>>>>>>
>>>>>>>>>     text = "^"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> GRIB
>>>>>>>>> ____
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ----- Original Message ----
>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>> Sent: Thu, July 7, 2011 1:25:07 PM
>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>
>>>>>>>>> David,
>>>>>>>>>
>>>>>>>>> Regarding your observations, it might be easiest to include
all of
>>>>>>>>> your
>>>>>>>>> observations into a single NetCDF file. You can
>>>>>>>>> denote the two different sensor locations using Station_ID
(column
>>>>>>>>> 2) in
>>>>>>>>> ascii2nc. Then, you can use the mask_sid
>>>>>>>>> option in the point_stat configuration file to specify
> which
>>>>>>>>> stations obs you
>>>>>>>>> want to use. Otherwise, you can use both
>>>>>>>>> of your current NetCDF obs files in a single point_stat call
by
>>>>>>>>> passing one as
>>>>>>>>> the "obs_file" and the other one using
>>>>>>>>> the -point_obs optional argument.
>>>>>>>>>
>>>>>>>>> Regarding the forecast times, the following is from the
point_stat
>>>>>>>>> usage
>>>>>>>>> statement, which you can see by running
>>>>>>>>> point_stat with no arguments:
>>>>>>>>>
>>>>>>>>> Â Â "-fcst_valid time" in YYYYMMDD[_HH[MMSS]] format sets
the
>>>>>>>>> forecast valid
> time
>>>>>>>>> to be verified (optional).
>>>>>>>>> Â Â "-fcst_lead time" in HH[MMSS] format sets the forecast
lead
>>>>>>>>> time to be
>>>>>>>>> verified (optional).
>>>>>>>>>
>>>>>>>>> If your model is initiated (we call it initialized) at
>>>>>>>>> 20110625_1200, then that
>>>>>>>>> is your fcst_init time, not your
>>>>>>>>> fcst_valid time. If you are interested in verifying the
forecast
>>>>>>>>> five hours
>>>>>>>>> into your 48-hour run, then your fcst_lead
>>>>>>>>> is 05 and your fcst_valid is 20110625_1700. Is that clear?
>>>>>>>>>
>>>>>>>>> Regarding
> levels, ZNNN is AGL and it can be an arbitrary number of
>>>>>>>>> digits long.Â
>>>>>>>>> Typically, for wind ground-based
>>>>>>>>> measurements, the fcst_field level looks something like
"Z10" for
>>>>>>>>> a sensor at
>>>>>>>>> 10m above the ground. Unfortunately, MET
>>>>>>>>> does not read model data on eta levels. In order to feed
your
>>>>>>>>> WRF data to MET,
>>>>>>>>> it must be post-processed using either
>>>>>>>>> the WRF Post Processor tool (WPP) or p_interp. This
process will
>>>>>>>>> put the data
>>>>>>>>> onto levels in meters and/or pressure,
>>>>>>>>> both of which MET can
> read.
>>>>>>>>>
>>>>>>>>> Please let me know if you have any more questions.
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/06/2011 03:09 PM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>
>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>
>>>>>>>>>> Paul,
>>>>>>>>>>
>>>>>>>>>> Thanks again for your help. Now, I'm on to creating the
config
>>>>>>>>>> file for
> Point
>>>>>>>>>
>>>>>>>>>> Stat.
>>>>>>>>>>
>>>>>>>>>> My model output is for 48-hour runs of WRF with hourly
output.Â
>>>>>>>>>> The output is
>>>>>>>>>> concatenated into a single file, 48 time steps in a single
>>>>>>>>>> file. (Perhaps
>>>>>>>>>> that's not practical.)
>>>>>>>>>>
>>>>>>>>>> My observations are for two met towers, each with two
anemometers
>>>>>>>>>> at different
>>>>>>>>>
>>>>>>>>>> heights. When I set up ASCII2NC, I broke each set of
anemometer
>>>>>>>>>> data into
> its
>>>>>>>>>
>>>>>>>>>> own netcdf file, each to serve as its own point source.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So when running PointStat, here's my understanding:
>>>>>>>>>>
>>>>>>>>>> PointStat will only analyze one model time per run.
>>>>>>>>>> In the command line arguments, I will have to specify
>>>>>>>>>> -fcst_valid. For
>>>>>>>>>> instance, if my WRF run is initiated with NAM data produced
on
>>>>>>>>>> the
>>>>>>>>>> 20110625_1200
>>>>>>>>>>
>>>>>>>>>> (and my model is initiated on the same time), that's my
> valid
>>>>>>>>>> time.
>>>>>>>>>> I will also have to specify -fcst_lead, which indicates
where
>>>>>>>>>> (temporally) in
>>>>>>>>>> the model's forecast I want to analyze. So with that run
I
>>>>>>>>>> mentioned, if I
>>>>>>>>>> wanted to analyze five hours into my 48-hour run, my lead
time
>>>>>>>>>> would be
>>>>>>>>>> 20110625_1700.
>>>>>>>>>> The purpose of specifying a temporal range of observation
values
>>>>>>>>>> is for
>>>>>>>>>> interpolation across the temporal dimension.
>>>>>>>>>> The model time that's compared to observation time is the
lead
>>>>>>>>>>
> time.
>>>>>>>>>> If any of my understanding of how time specification in
Point
>>>>>>>>>> Stat works is
>>>>>>>>>> incorrect, please let me know.
>>>>>>>>>>
>>>>>>>>>> My other main question concerns specifying the levels for
each
>>>>>>>>>> fcst_field. The
>>>>>>>>>>
>>>>>>>>>> most attractive to me is the ZNNN for a straightforward
vertical
>>>>>>>>>> level. Is
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>> numeric "NNN" in meters? Is it AGL? If it's AGL+MSL,
can it
>>>>>>>>>> be more than
>>>>>>>>>>
> three
>>>>>>>>>>
>>>>>>>>>> digits? (My MSL+AGL is ~1700m.) Also the provided
default
>>>>>>>>>> config file
>>>>>>>>>> includes
>>>>>>>>>>
>>>>>>>>>> the following comment:Â "GC/ZNNN-NNN for a range of
vertical
>>>>>>>>>> levels (MSL or
>>>>>>>>>> AGL)."Â How would one specify whether it's MSL or AGL?Â
Also,
>>>>>>>>>> WRF output is in
>>>>>>>>>
>>>>>>>>>> eta-levels which is more related to pressure levels. Does
that
>>>>>>>>>> matter?
>>>>>>>>>>
>>>>>>>>>> Also, I assume that the forecast and observation
> thresholds serve
>>>>>>>>>> as a quality
>>>>>>>>>
>>>>>>>>>> control.
>>>>>>>>>>
>>>>>>>>>> Thanks again for all your assistance!
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>> Â
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Original Message ----
>>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg} <met_help at ucar.edu>
>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>> Sent: Wed, July 6, 2011 12:25:51 PM
>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>
>>>>>>>>>> David,
>>>>>>>>>>
>>>>>>>>>>> I do want to interpolate to the height of the sensor, so
could
>>>>>>>>>>> you recommend
>>>>>>>>> a
>>>>>>>>>>> message type? Frankly, I'm not familiar with these
message
>>>>>>>>>>> types, so could
>>>>>>>>>> you
>>>>>>>>>>> refer me to a link or reference to these message
> types?
>>>>>>>>>>
>>>>>>>>>> I think PROFLR is the appropriate message type for your
>>>>>>>>>> situation. Here is a
>>>>>>>>>> reference on the message types:
>>>>>>>>>>
>>>>>>>>>>
http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>>>>>>>>>>
>>>>>>>>>> Message types are a concept that is associated with
PrepBUFR
>>>>>>>>>> point observations
>>>>>>>>>>
>>>>>>>>>> produced by NCEP. These observations
>>>>>>>>>> are often used in MET. The logic that MET uses for
> these
>>>>>>>>>> message types are
>>>>>>>>>> applied to all observations, regardless of
>>>>>>>>>> source.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Also, I had assumed column 6 would get the elevation of
the
>>>>>>>>>>> terrain at that
>>>>>>>>>>> point and column nine would be the sensor's height about
>>>>>>>>>>> ground. But it
>>>>>>>>>> sounds
>>>>>>>>>>> like column 6 should be the sensor's total height above
sea
>>>>>>>>>>> level, right? If
>>>>>>>>>>> so, what should column 9
> have?
>>>>>>>>>>
>>>>>>>>>> No, you are correct. I should have been more clear in my
>>>>>>>>>> previous email.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> will Point-Stat match the observation time to the model
time and
>>>>>>>>>>> calculate its statistics from that?
>>>>>>>>>>
>>>>>>>>>> Point-Stat was designed to be run once for each forecast
valid
>>>>>>>>>> time. If your
>>>>>>>>>> model data files contain single 48 hour
>>>>>>>>>> forecasts, run Point-Stat once for each model file. Each
time,
>>>>>>>>>> you will pass
>>>>>>>>>>
> the model data file you want to verify,
>>>>>>>>>> the single observation file and the configuration file.Â
In the
>>>>>>>>>> configuration
>>>>>>>>>> file, you should decide the time window
>>>>>>>>>> inside which observations "match" the model valid time.Â
The
>>>>>>>>>> settings for this
>>>>>>>>>
>>>>>>>>>> are beg_ds and end_ds. You could also
>>>>>>>>>> use the Point-Stat command line parameters obs_valid_beg
and
>>>>>>>>>> obs_valid_end.
>>>>>>>>>>
>>>>>>>>>> If you have multiple forecasts in a single model data file,
you
>>>>>>>>>> will need to
>>>>>>>>>>
> use
>>>>>>>>>>
>>>>>>>>>> the command line parameter fcst_valid
>>>>>>>>>> time to specify which forecast Point-Stat should verify.Â
In
>>>>>>>>>> this case,
>>>>>>>>>> Point-Stat must still be run once for each valid
>>>>>>>>>> time, where each call differs only by the value passed to
the
>>>>>>>>>> fcst_valid
>>>>>>>>>> parameter.
>>>>>>>>>>
>>>>>>>>>> Does that make sense? Please let me know if you have
questions.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>>
> Paul
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 07/06/2011 04:04 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>>
>>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>>
>>>>>>>>>>> Paul,
>>>>>>>>>>>
>>>>>>>>>>> One other question. The netcdf file I'll be creating
with
>>>>>>>>>>> ascii2nc covers six
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> months. So I'll have one six-month file for
observations.Â
> I
>>>>>>>>>>> want to use
>>>>>>>>>>> Point-Stat to compare that file to a series of 48-hour
>>>>>>>>>>> forecasts. My
>>>>>>>>>>> question--will Point-Stat match the observation time to
the
>>>>>>>>>>> model time and
>>>>>>>>>>> calculate its statistics from that? In other words,
>>>>>>>>>>> Point-Stat's output for
>>>>>>>>>>> each forecast is not based on periods for which there is
no
>>>>>>>>>>> forecast data,
>>>>>>>>>>> correct?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
> David
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message ----
>>>>>>>>>>> From: RAL HelpDesk {for David Bryan} <met_help at ucar.edu>
>>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>>> Sent: Tue, July 5, 2011 6:54:56 PM
>>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>>
>>>>>>>>>>> Thanks Paul,
>>>>>>>>>>>
>>>>>>>>>>> That's very
> helpful.Â
>>>>>>>>>>>
>>>>>>>>>>> I do want to interpolate to the height of the sensor, so
could
>>>>>>>>>>> you recommend a
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> message type? Frankly, I'm not familiar with these
message
>>>>>>>>>>> types, so could you
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> refer me to a link or reference to these message types?
>>>>>>>>>>>
>>>>>>>>>>> Also, I had assumed column 6 would get the elevation of
the
>>>>>>>>>>> terrain at that
>>>>>>>>>>> point and column nine would be the sensor's
> height about
>>>>>>>>>>> ground. But it sounds
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> like column 6 should be the sensor's total height above
sea
>>>>>>>>>>> level, right? If
>>>>>>>>>
>>>>>>>>>>> so, what should column 9 have?
>>>>>>>>>>>
>>>>>>>>>>> Thanks again,
>>>>>>>>>>> David
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message ----
>>>>>>>>>>> From: RAL HelpDesk {for Paul Oldenburg}
<met_help at ucar.edu>
>>>>>>>>>>> To: davidstephenbryan at yahoo.com
>>>>>>>>>>> Sent: Tue, July 5, 2011 1:07:52 PM
>>>>>>>>>>> Subject: Re: [rt.rap.ucar.edu #48043] formatting ascii2nc
>>>>>>>>>>>
>>>>>>>>>>> David,
>>>>>>>>>>>
>>>>>>>>>>> This column does not apply to your data, since you are
only
>>>>>>>>>>> measuring wind
>>>>>>>>>>> speed
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> at a fixed height above the ground.Â
> I
>>>>>>>>>>> would recommend putting -9999 in that column, since that
is the
>>>>>>>>>>> value that MET
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> uses for bad data or N/A.
>>>>>>>>>>>
>>>>>>>>>>> You should use column 6 to enter the elevation of your
sensor,
>>>>>>>>>>> but be advised
>>>>>>>>>
>>>>>>>>>>> that if you use a "surface" message type
>>>>>>>>>>> (ADPSFC or SFCSHP), MET will assume the elevation is 10m
>>>>>>>>>>> regardless of what you
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> enter in column
> 6. If you want MET to
>>>>>>>>>>> interpolate model data to the height of your sensor, use
another
>>>>>>>>>>> message type.Â
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I hope this is clear. Please let me
>>>>>>>>>>> know if you have any questions.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Paul
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 07/05/2011 06:25 AM, RAL HelpDesk {for David Bryan}
wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Tue Jul 05 06:25:54 2011:
> Request 48043 was acted upon.
>>>>>>>>>>>> Transaction: Ticket created by
davidstephenbryan at yahoo.com
>>>>>>>>>>>> Â Â Â Â Â Queue: met_help
>>>>>>>>>>>> Â Â Â Â Subject: formatting ascii2nc
>>>>>>>>>>>> Â Â Â Â Â Owner: Nobody
>>>>>>>>>>>> Â Â Requestors: davidstephenbryan at yahoo.com
>>>>>>>>>>>> Â Â Â Â Status: new
>>>>>>>>>>>> Â Â Ticket
> <URL:
>>>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=48043 >
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Could you clarify "8. Level as the pressure level in
hPa or
>>>>>>>>>>>> accumulation
>>>>>>>>>>>> interval in hours"Â when formatting point observations.Â
My
>>>>>>>>>>>> point observation
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> is
>>>>>>>>>>>>
>>>>>>>>>>>> an anemometer at a fixed height. What value would I put
> for
>>>>>>>>>>>> this column?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> David Bryan
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>
------------------------------------------------
More information about the Met_help
mailing list