[Met_help] [rt.rap.ucar.edu #68300] History for using point-stat tool for daily averaged observations
John Halley Gotway via RT
met_help at ucar.edu
Fri Oct 3 12:28:59 MDT 2014
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I am a new METv4.1 user trying to use the point-stat tool to verify my
WRF-ARWv3.5 model output.
I have daily averaged observations and WRF fields output every hour. I am
using the WRF Unified Post Processor to convert my netcdf files to grib1
format. The message type would be ADPSFC. My fields are Temp, Wind speed,
Pressure, RH, and Precip.
A couple of questions:
1) Under the Level column of the ASCII point obs file, I have been putting
"24" as the observation accumulation interval (since they are daily
averages). Did I understand that correctly?
2) Each wrf output file is 30 hours long. I have been using UPP to split it
into 30 GRIB files, 1 for each hour. But this is a problem, because what I
actually want to do is compare forecast daily averages with obs daily
averages ... and it appears what I am currently set up to do is compare
forecast hourly averages with obs daily averages.
Any input would be greatly appreciated.
Thank you!
Archana
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: using point-stat tool for daily averaged observations
From: John Halley Gotway
Time: Wed Jul 23 12:53:45 2014
Archana,
I see that you'd like to use Point-Stat to verify your WRF output.
That's
great that you've succeeded in running WRF and UPP and have GRIB1
output
files. And I understand that you're wondering how you can compare
daily
averaged observations to hourly model output.
You're correct in realizing that you first need to make your forecast
and
observations comparable before verifying them. Is there a reason why
your
observations are limited to daily averages? Is the hourly observation
data
not available to you? Or is there some other reason why you're
choosing to
use daily averages? Getting hourly observations and comparing them to
your
hourly model output would obviously be the best solution. But I'll
assume
that's not possible or not what you want to do.
So how can you get 24-hour averages of your WRF model output? I asked
around and it may be possible for WRF to dump "output diagnostic"
fields
directly that are averaged over some time interval. I couldn't find
any
information about this in the user's guide, but the WRF Registry file
contains entries for 2m and 10m output diagnostics. It may be
possible to
dump out 24-hourly averages of those fields directly from WRF. If
that
works, the next step would be to make sure that UPP is passing them
through
the GRIB1 output files.
I'd suggest writing wrfhelp at ucar.edu to ask for help in dumping these
output diagnostic fields. And then you could compare WRF's 24-hour
averages to your daily average observations, making sure the averages
are
defined with the same beginning/ending times. That'd be the second
best
solution.
If all else fails, you could just compute the average temperature over
24
hourly files yourself. You could do that using a scripting language,
like
R or python or whatever other tool you prefer for reading NetCDF data.
Alternatively, you could use the MET ensemble_stat tool. You could
pass
the tool those 24 hourly files, list the fields you want, and computed
the
ensemble mean field. The mean field would be the average over those
24
files. I haven't done this in the past myself. So if you go that
route,
please let me know what questions come up.
Regarding your first question, the "accumulation interval" is meant
for
fields like precipitation. It'd be the time interval of the
accumulated
precip. For 2-m temperature, I'd just put in the level value as fill
data
(-9999) and the height as 2 (for 2 meters).
Hope that helps.
Thanks,
John Halley Gotway
met_help at ucar.edu
On Tue, Jul 22, 2014 at 3:41 PM, Archana Dayalu via RT
<met_help at ucar.edu>
wrote:
>
> Tue Jul 22 15:41:16 2014: Request 68300 was acted upon.
> Transaction: Ticket created by adayalu at seas.harvard.edu
> Queue: met_help
> Subject: using point-stat tool for daily averaged observations
> Owner: Nobody
> Requestors: adayalu at seas.harvard.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300 >
>
>
> Hi,
> I am a new METv4.1 user trying to use the point-stat tool to verify
my
> WRF-ARWv3.5 model output.
> I have daily averaged observations and WRF fields output every hour.
I am
> using the WRF Unified Post Processor to convert my netcdf files to
grib1
> format. The message type would be ADPSFC. My fields are Temp, Wind
speed,
> Pressure, RH, and Precip.
> A couple of questions:
> 1) Under the Level column of the ASCII point obs file, I have been
putting
> "24" as the observation accumulation interval (since they are daily
> averages). Did I understand that correctly?
> 2) Each wrf output file is 30 hours long. I have been using UPP to
split it
> into 30 GRIB files, 1 for each hour. But this is a problem, because
what I
> actually want to do is compare forecast daily averages with obs
daily
> averages ... and it appears what I am currently set up to do is
compare
> forecast hourly averages with obs daily averages.
> Any input would be greatly appreciated.
> Thank you!
> Archana
>
>
------------------------------------------------
Subject: using point-stat tool for daily averaged observations
From: Archana Dayalu
Time: Thu Jul 24 19:03:01 2014
Hi John,
Thanks for the speedy response. That's definitely helpful to know.
Unfortunately I only have access to the 24-hour observation files.
Couple
of follow-up questions:
1) I will try the WRF auxdata output of 24 hours ... and maybe I will
try
directly outputting my WRF to GRIB format. However in the event that I
choose to go the manual route of 24-h averaging using R, I would
eventually
need to output my 24h files back to GRIB. So, rewriting to netcdf
using R
and then converting to GRIB from UPP would be the way to go for that,
correct? Or am I missing a step?
2) What happens if I leave the level and height variables as is for
non-precip data? Will that harm in anyway? Just curious.
Thanks much!
Archana
On Wed, Jul 23, 2014 at 2:53 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Archana,
>
> I see that you'd like to use Point-Stat to verify your WRF output.
That's
> great that you've succeeded in running WRF and UPP and have GRIB1
output
> files. And I understand that you're wondering how you can compare
daily
> averaged observations to hourly model output.
>
> You're correct in realizing that you first need to make your
forecast and
> observations comparable before verifying them. Is there a reason
why your
> observations are limited to daily averages? Is the hourly
observation data
> not available to you? Or is there some other reason why you're
choosing to
> use daily averages? Getting hourly observations and comparing them
to your
> hourly model output would obviously be the best solution. But I'll
assume
> that's not possible or not what you want to do.
>
> So how can you get 24-hour averages of your WRF model output? I
asked
> around and it may be possible for WRF to dump "output diagnostic"
fields
> directly that are averaged over some time interval. I couldn't find
any
> information about this in the user's guide, but the WRF Registry
file
> contains entries for 2m and 10m output diagnostics. It may be
possible to
> dump out 24-hourly averages of those fields directly from WRF. If
that
> works, the next step would be to make sure that UPP is passing them
through
> the GRIB1 output files.
>
> I'd suggest writing wrfhelp at ucar.edu to ask for help in dumping
these
> output diagnostic fields. And then you could compare WRF's 24-hour
> averages to your daily average observations, making sure the
averages are
> defined with the same beginning/ending times. That'd be the second
best
> solution.
>
> If all else fails, you could just compute the average temperature
over 24
> hourly files yourself. You could do that using a scripting
language, like
> R or python or whatever other tool you prefer for reading NetCDF
data.
> Alternatively, you could use the MET ensemble_stat tool. You could
pass
> the tool those 24 hourly files, list the fields you want, and
computed the
> ensemble mean field. The mean field would be the average over those
24
> files. I haven't done this in the past myself. So if you go that
route,
> please let me know what questions come up.
>
> Regarding your first question, the "accumulation interval" is meant
for
> fields like precipitation. It'd be the time interval of the
accumulated
> precip. For 2-m temperature, I'd just put in the level value as
fill data
> (-9999) and the height as 2 (for 2 meters).
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> On Tue, Jul 22, 2014 at 3:41 PM, Archana Dayalu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Tue Jul 22 15:41:16 2014: Request 68300 was acted upon.
> > Transaction: Ticket created by adayalu at seas.harvard.edu
> > Queue: met_help
> > Subject: using point-stat tool for daily averaged
observations
> > Owner: Nobody
> > Requestors: adayalu at seas.harvard.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300 >
> >
> >
> > Hi,
> > I am a new METv4.1 user trying to use the point-stat tool to
verify my
> > WRF-ARWv3.5 model output.
> > I have daily averaged observations and WRF fields output every
hour. I am
> > using the WRF Unified Post Processor to convert my netcdf files to
grib1
> > format. The message type would be ADPSFC. My fields are Temp, Wind
speed,
> > Pressure, RH, and Precip.
> > A couple of questions:
> > 1) Under the Level column of the ASCII point obs file, I have been
> putting
> > "24" as the observation accumulation interval (since they are
daily
> > averages). Did I understand that correctly?
> > 2) Each wrf output file is 30 hours long. I have been using UPP to
split
> it
> > into 30 GRIB files, 1 for each hour. But this is a problem,
because what
> I
> > actually want to do is compare forecast daily averages with obs
daily
> > averages ... and it appears what I am currently set up to do is
compare
> > forecast hourly averages with obs daily averages.
> > Any input would be greatly appreciated.
> > Thank you!
> > Archana
> >
> >
>
>
--
____________________________________________________
*Archana Dayalu*
Graduate Student
Dept. of Earth and Planetary Sciences
Harvard University
24 Oxford Street #402
Cambridge, MA 02138
(617) 384-8206
------------------------------------------------
Subject: using point-stat tool for daily averaged observations
From: John Halley Gotway
Time: Fri Jul 25 09:07:58 2014
Archana,
I'm not aware of a way to have WRF write directly to GRIB. WRF can
output
a binary format directly, but I'm pretty sure that's not GRIB. So
you'll
likely need to continue writing WRF NetCDF output and then running it
through the Unified Post Processor, which dumps GRIB.
As for averaging the data, there are many different options...
As I mentioned, you could try using the MET ensemble-stat tool. It
has the
ability to compute an ensemble mean, which would be the average of
your
24-hourly files. It's output is a flavor of NetCDF that the other MET
tools can read directly. But setting up the config file for ensemble-
stat
might take some work.
I've never read GRIB data directly using R. I'm not aware of an R
package
for doing so, although there may be something out there. If you
happen to
be familiar with NCL, I believe it could easily read in that GRIB data
and
compute an average. The trick would be getting its output in a flavor
of
NetCDF that MET can read.
There are also the NetCDF operators (nco) which can be used to compute
averages of NetCDF files. I wonder if it'd be possible to compute the
average of your NetCDF wrfout files directly prior to running the
averaged
file through UPP. I'm just speculating here - I've never done this in
the
past.
Regarding your last question, if you leave the level and height
variables
as is for non-precip data, you may or may not have problems matching
the
observations up to the forecast. Since you're just doing surface
verification using the ADPSFC message type, I suspect it wouldn't be a
problem. When verifying surface variables, like 2-m temperature,
against a
surface message type, like ADPSFC, all the observations falling within
your
geographic masking region are used, regardless of their height or
level
value. The height and level values come into play when verifying
upper-air
pressure levels or higher vertical level data, like hub-height winds
at 60
msl.
Hope that helps give you some ideas.
Thanks,
John
On Thu, Jul 24, 2014 at 7:03 PM, Archana Dayalu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300 >
>
> Hi John,
> Thanks for the speedy response. That's definitely helpful to know.
> Unfortunately I only have access to the 24-hour observation files.
Couple
> of follow-up questions:
> 1) I will try the WRF auxdata output of 24 hours ... and maybe I
will try
> directly outputting my WRF to GRIB format. However in the event that
I
> choose to go the manual route of 24-h averaging using R, I would
eventually
> need to output my 24h files back to GRIB. So, rewriting to netcdf
using R
> and then converting to GRIB from UPP would be the way to go for
that,
> correct? Or am I missing a step?
> 2) What happens if I leave the level and height variables as is for
> non-precip data? Will that harm in anyway? Just curious.
> Thanks much!
> Archana
>
>
> On Wed, Jul 23, 2014 at 2:53 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Archana,
> >
> > I see that you'd like to use Point-Stat to verify your WRF output.
> That's
> > great that you've succeeded in running WRF and UPP and have GRIB1
output
> > files. And I understand that you're wondering how you can compare
daily
> > averaged observations to hourly model output.
> >
> > You're correct in realizing that you first need to make your
forecast and
> > observations comparable before verifying them. Is there a reason
why
> your
> > observations are limited to daily averages? Is the hourly
observation
> data
> > not available to you? Or is there some other reason why you're
choosing
> to
> > use daily averages? Getting hourly observations and comparing
them to
> your
> > hourly model output would obviously be the best solution. But
I'll
> assume
> > that's not possible or not what you want to do.
> >
> > So how can you get 24-hour averages of your WRF model output? I
asked
> > around and it may be possible for WRF to dump "output diagnostic"
fields
> > directly that are averaged over some time interval. I couldn't
find any
> > information about this in the user's guide, but the WRF Registry
file
> > contains entries for 2m and 10m output diagnostics. It may be
possible
> to
> > dump out 24-hourly averages of those fields directly from WRF. If
that
> > works, the next step would be to make sure that UPP is passing
them
> through
> > the GRIB1 output files.
> >
> > I'd suggest writing wrfhelp at ucar.edu to ask for help in dumping
these
> > output diagnostic fields. And then you could compare WRF's 24-
hour
> > averages to your daily average observations, making sure the
averages are
> > defined with the same beginning/ending times. That'd be the
second best
> > solution.
> >
> > If all else fails, you could just compute the average temperature
over 24
> > hourly files yourself. You could do that using a scripting
language,
> like
> > R or python or whatever other tool you prefer for reading NetCDF
data.
> > Alternatively, you could use the MET ensemble_stat tool. You
could pass
> > the tool those 24 hourly files, list the fields you want, and
computed
> the
> > ensemble mean field. The mean field would be the average over
those 24
> > files. I haven't done this in the past myself. So if you go that
route,
> > please let me know what questions come up.
> >
> > Regarding your first question, the "accumulation interval" is
meant for
> > fields like precipitation. It'd be the time interval of the
accumulated
> > precip. For 2-m temperature, I'd just put in the level value as
fill
> data
> > (-9999) and the height as 2 (for 2 meters).
> >
> > Hope that helps.
> >
> > Thanks,
> > John Halley Gotway
> > met_help at ucar.edu
> >
> >
> > On Tue, Jul 22, 2014 at 3:41 PM, Archana Dayalu via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue Jul 22 15:41:16 2014: Request 68300 was acted upon.
> > > Transaction: Ticket created by adayalu at seas.harvard.edu
> > > Queue: met_help
> > > Subject: using point-stat tool for daily averaged
observations
> > > Owner: Nobody
> > > Requestors: adayalu at seas.harvard.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300
> >
> > >
> > >
> > > Hi,
> > > I am a new METv4.1 user trying to use the point-stat tool to
verify my
> > > WRF-ARWv3.5 model output.
> > > I have daily averaged observations and WRF fields output every
hour. I
> am
> > > using the WRF Unified Post Processor to convert my netcdf files
to
> grib1
> > > format. The message type would be ADPSFC. My fields are Temp,
Wind
> speed,
> > > Pressure, RH, and Precip.
> > > A couple of questions:
> > > 1) Under the Level column of the ASCII point obs file, I have
been
> > putting
> > > "24" as the observation accumulation interval (since they are
daily
> > > averages). Did I understand that correctly?
> > > 2) Each wrf output file is 30 hours long. I have been using UPP
to
> split
> > it
> > > into 30 GRIB files, 1 for each hour. But this is a problem,
because
> what
> > I
> > > actually want to do is compare forecast daily averages with obs
daily
> > > averages ... and it appears what I am currently set up to do is
compare
> > > forecast hourly averages with obs daily averages.
> > > Any input would be greatly appreciated.
> > > Thank you!
> > > Archana
> > >
> > >
> >
> >
>
>
> --
> ____________________________________________________
> *Archana Dayalu*
> Graduate Student
> Dept. of Earth and Planetary Sciences
> Harvard University
> 24 Oxford Street #402
> Cambridge, MA 02138
> (617) 384-8206
>
>
------------------------------------------------
Subject: using point-stat tool for daily averaged observations
From: Archana Dayalu
Time: Wed Jul 30 12:33:14 2014
John,
Thanks very much for all your help! I went a manual route, and it
seems to
be working fine.
Regards
Archana
On Fri, Jul 25, 2014 at 11:07 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Archana,
>
> I'm not aware of a way to have WRF write directly to GRIB. WRF can
output
> a binary format directly, but I'm pretty sure that's not GRIB. So
you'll
> likely need to continue writing WRF NetCDF output and then running
it
> through the Unified Post Processor, which dumps GRIB.
>
> As for averaging the data, there are many different options...
>
> As I mentioned, you could try using the MET ensemble-stat tool. It
has the
> ability to compute an ensemble mean, which would be the average of
your
> 24-hourly files. It's output is a flavor of NetCDF that the other
MET
> tools can read directly. But setting up the config file for
ensemble-stat
> might take some work.
>
> I've never read GRIB data directly using R. I'm not aware of an R
package
> for doing so, although there may be something out there. If you
happen to
> be familiar with NCL, I believe it could easily read in that GRIB
data and
> compute an average. The trick would be getting its output in a
flavor of
> NetCDF that MET can read.
>
> There are also the NetCDF operators (nco) which can be used to
compute
> averages of NetCDF files. I wonder if it'd be possible to compute
the
> average of your NetCDF wrfout files directly prior to running the
averaged
> file through UPP. I'm just speculating here - I've never done this
in the
> past.
>
> Regarding your last question, if you leave the level and height
variables
> as is for non-precip data, you may or may not have problems matching
the
> observations up to the forecast. Since you're just doing surface
> verification using the ADPSFC message type, I suspect it wouldn't be
a
> problem. When verifying surface variables, like 2-m temperature,
against a
> surface message type, like ADPSFC, all the observations falling
within your
> geographic masking region are used, regardless of their height or
level
> value. The height and level values come into play when verifying
upper-air
> pressure levels or higher vertical level data, like hub-height winds
at 60
> msl.
>
> Hope that helps give you some ideas.
>
> Thanks,
> John
>
>
> On Thu, Jul 24, 2014 at 7:03 PM, Archana Dayalu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300 >
> >
> > Hi John,
> > Thanks for the speedy response. That's definitely helpful to know.
> > Unfortunately I only have access to the 24-hour observation files.
Couple
> > of follow-up questions:
> > 1) I will try the WRF auxdata output of 24 hours ... and maybe I
will try
> > directly outputting my WRF to GRIB format. However in the event
that I
> > choose to go the manual route of 24-h averaging using R, I would
> eventually
> > need to output my 24h files back to GRIB. So, rewriting to netcdf
using R
> > and then converting to GRIB from UPP would be the way to go for
that,
> > correct? Or am I missing a step?
> > 2) What happens if I leave the level and height variables as is
for
> > non-precip data? Will that harm in anyway? Just curious.
> > Thanks much!
> > Archana
> >
> >
> > On Wed, Jul 23, 2014 at 2:53 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Archana,
> > >
> > > I see that you'd like to use Point-Stat to verify your WRF
output.
> > That's
> > > great that you've succeeded in running WRF and UPP and have
GRIB1
> output
> > > files. And I understand that you're wondering how you can
compare
> daily
> > > averaged observations to hourly model output.
> > >
> > > You're correct in realizing that you first need to make your
forecast
> and
> > > observations comparable before verifying them. Is there a
reason why
> > your
> > > observations are limited to daily averages? Is the hourly
observation
> > data
> > > not available to you? Or is there some other reason why you're
> choosing
> > to
> > > use daily averages? Getting hourly observations and comparing
them to
> > your
> > > hourly model output would obviously be the best solution. But
I'll
> > assume
> > > that's not possible or not what you want to do.
> > >
> > > So how can you get 24-hour averages of your WRF model output? I
asked
> > > around and it may be possible for WRF to dump "output
diagnostic"
> fields
> > > directly that are averaged over some time interval. I couldn't
find
> any
> > > information about this in the user's guide, but the WRF Registry
file
> > > contains entries for 2m and 10m output diagnostics. It may be
possible
> > to
> > > dump out 24-hourly averages of those fields directly from WRF.
If that
> > > works, the next step would be to make sure that UPP is passing
them
> > through
> > > the GRIB1 output files.
> > >
> > > I'd suggest writing wrfhelp at ucar.edu to ask for help in dumping
these
> > > output diagnostic fields. And then you could compare WRF's 24-
hour
> > > averages to your daily average observations, making sure the
averages
> are
> > > defined with the same beginning/ending times. That'd be the
second
> best
> > > solution.
> > >
> > > If all else fails, you could just compute the average
temperature over
> 24
> > > hourly files yourself. You could do that using a scripting
language,
> > like
> > > R or python or whatever other tool you prefer for reading NetCDF
data.
> > > Alternatively, you could use the MET ensemble_stat tool. You
could
> pass
> > > the tool those 24 hourly files, list the fields you want, and
computed
> > the
> > > ensemble mean field. The mean field would be the average over
those 24
> > > files. I haven't done this in the past myself. So if you go
that
> route,
> > > please let me know what questions come up.
> > >
> > > Regarding your first question, the "accumulation interval" is
meant for
> > > fields like precipitation. It'd be the time interval of the
> accumulated
> > > precip. For 2-m temperature, I'd just put in the level value as
fill
> > data
> > > (-9999) and the height as 2 (for 2 meters).
> > >
> > > Hope that helps.
> > >
> > > Thanks,
> > > John Halley Gotway
> > > met_help at ucar.edu
> > >
> > >
> > > On Tue, Jul 22, 2014 at 3:41 PM, Archana Dayalu via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Tue Jul 22 15:41:16 2014: Request 68300 was acted upon.
> > > > Transaction: Ticket created by adayalu at seas.harvard.edu
> > > > Queue: met_help
> > > > Subject: using point-stat tool for daily averaged
observations
> > > > Owner: Nobody
> > > > Requestors: adayalu at seas.harvard.edu
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300
> > >
> > > >
> > > >
> > > > Hi,
> > > > I am a new METv4.1 user trying to use the point-stat tool to
verify
> my
> > > > WRF-ARWv3.5 model output.
> > > > I have daily averaged observations and WRF fields output every
hour.
> I
> > am
> > > > using the WRF Unified Post Processor to convert my netcdf
files to
> > grib1
> > > > format. The message type would be ADPSFC. My fields are Temp,
Wind
> > speed,
> > > > Pressure, RH, and Precip.
> > > > A couple of questions:
> > > > 1) Under the Level column of the ASCII point obs file, I have
been
> > > putting
> > > > "24" as the observation accumulation interval (since they are
daily
> > > > averages). Did I understand that correctly?
> > > > 2) Each wrf output file is 30 hours long. I have been using
UPP to
> > split
> > > it
> > > > into 30 GRIB files, 1 for each hour. But this is a problem,
because
> > what
> > > I
> > > > actually want to do is compare forecast daily averages with
obs daily
> > > > averages ... and it appears what I am currently set up to do
is
> compare
> > > > forecast hourly averages with obs daily averages.
> > > > Any input would be greatly appreciated.
> > > > Thank you!
> > > > Archana
> > > >
> > > >
> > >
> > >
> >
> >
> > --
> > ____________________________________________________
> > *Archana Dayalu*
> > Graduate Student
> > Dept. of Earth and Planetary Sciences
> > Harvard University
> > 24 Oxford Street #402
> > Cambridge, MA 02138
> > (617) 384-8206
> >
> >
>
>
--
____________________________________________________
*Archana Dayalu*
Graduate Student
Dept. of Earth and Planetary Sciences
Harvard University
24 Oxford Street #402
Cambridge, MA 02138
(617) 384-8206
------------------------------------------------
Subject: using point-stat tool for daily averaged observations
From: John Halley Gotway
Time: Wed Jul 30 13:32:16 2014
Great, glad you were able to figure it out.
Thanks,
John
On Wed, Jul 30, 2014 at 12:33 PM, Archana Dayalu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300 >
>
> John,
> Thanks very much for all your help! I went a manual route, and it
seems to
> be working fine.
> Regards
> Archana
>
>
> On Fri, Jul 25, 2014 at 11:07 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Archana,
> >
> > I'm not aware of a way to have WRF write directly to GRIB. WRF
can
> output
> > a binary format directly, but I'm pretty sure that's not GRIB. So
you'll
> > likely need to continue writing WRF NetCDF output and then running
it
> > through the Unified Post Processor, which dumps GRIB.
> >
> > As for averaging the data, there are many different options...
> >
> > As I mentioned, you could try using the MET ensemble-stat tool.
It has
> the
> > ability to compute an ensemble mean, which would be the average of
your
> > 24-hourly files. It's output is a flavor of NetCDF that the other
MET
> > tools can read directly. But setting up the config file for
> ensemble-stat
> > might take some work.
> >
> > I've never read GRIB data directly using R. I'm not aware of an R
> package
> > for doing so, although there may be something out there. If you
happen
> to
> > be familiar with NCL, I believe it could easily read in that GRIB
data
> and
> > compute an average. The trick would be getting its output in a
flavor of
> > NetCDF that MET can read.
> >
> > There are also the NetCDF operators (nco) which can be used to
compute
> > averages of NetCDF files. I wonder if it'd be possible to compute
the
> > average of your NetCDF wrfout files directly prior to running the
> averaged
> > file through UPP. I'm just speculating here - I've never done
this in
> the
> > past.
> >
> > Regarding your last question, if you leave the level and height
variables
> > as is for non-precip data, you may or may not have problems
matching the
> > observations up to the forecast. Since you're just doing surface
> > verification using the ADPSFC message type, I suspect it wouldn't
be a
> > problem. When verifying surface variables, like 2-m temperature,
> against a
> > surface message type, like ADPSFC, all the observations falling
within
> your
> > geographic masking region are used, regardless of their height or
level
> > value. The height and level values come into play when verifying
> upper-air
> > pressure levels or higher vertical level data, like hub-height
winds at
> 60
> > msl.
> >
> > Hope that helps give you some ideas.
> >
> > Thanks,
> > John
> >
> >
> > On Thu, Jul 24, 2014 at 7:03 PM, Archana Dayalu via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300 >
> > >
> > > Hi John,
> > > Thanks for the speedy response. That's definitely helpful to
know.
> > > Unfortunately I only have access to the 24-hour observation
files.
> Couple
> > > of follow-up questions:
> > > 1) I will try the WRF auxdata output of 24 hours ... and maybe I
will
> try
> > > directly outputting my WRF to GRIB format. However in the event
that I
> > > choose to go the manual route of 24-h averaging using R, I would
> > eventually
> > > need to output my 24h files back to GRIB. So, rewriting to
netcdf
> using R
> > > and then converting to GRIB from UPP would be the way to go for
that,
> > > correct? Or am I missing a step?
> > > 2) What happens if I leave the level and height variables as is
for
> > > non-precip data? Will that harm in anyway? Just curious.
> > > Thanks much!
> > > Archana
> > >
> > >
> > > On Wed, Jul 23, 2014 at 2:53 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Archana,
> > > >
> > > > I see that you'd like to use Point-Stat to verify your WRF
output.
> > > That's
> > > > great that you've succeeded in running WRF and UPP and have
GRIB1
> > output
> > > > files. And I understand that you're wondering how you can
compare
> > daily
> > > > averaged observations to hourly model output.
> > > >
> > > > You're correct in realizing that you first need to make your
forecast
> > and
> > > > observations comparable before verifying them. Is there a
reason why
> > > your
> > > > observations are limited to daily averages? Is the hourly
> observation
> > > data
> > > > not available to you? Or is there some other reason why
you're
> > choosing
> > > to
> > > > use daily averages? Getting hourly observations and comparing
them
> to
> > > your
> > > > hourly model output would obviously be the best solution. But
I'll
> > > assume
> > > > that's not possible or not what you want to do.
> > > >
> > > > So how can you get 24-hour averages of your WRF model output?
I
> asked
> > > > around and it may be possible for WRF to dump "output
diagnostic"
> > fields
> > > > directly that are averaged over some time interval. I
couldn't find
> > any
> > > > information about this in the user's guide, but the WRF
Registry file
> > > > contains entries for 2m and 10m output diagnostics. It may be
> possible
> > > to
> > > > dump out 24-hourly averages of those fields directly from WRF.
If
> that
> > > > works, the next step would be to make sure that UPP is passing
them
> > > through
> > > > the GRIB1 output files.
> > > >
> > > > I'd suggest writing wrfhelp at ucar.edu to ask for help in
dumping
> these
> > > > output diagnostic fields. And then you could compare WRF's
24-hour
> > > > averages to your daily average observations, making sure the
averages
> > are
> > > > defined with the same beginning/ending times. That'd be the
second
> > best
> > > > solution.
> > > >
> > > > If all else fails, you could just compute the average
temperature
> over
> > 24
> > > > hourly files yourself. You could do that using a scripting
language,
> > > like
> > > > R or python or whatever other tool you prefer for reading
NetCDF
> data.
> > > > Alternatively, you could use the MET ensemble_stat tool. You
could
> > pass
> > > > the tool those 24 hourly files, list the fields you want, and
> computed
> > > the
> > > > ensemble mean field. The mean field would be the average over
those
> 24
> > > > files. I haven't done this in the past myself. So if you go
that
> > route,
> > > > please let me know what questions come up.
> > > >
> > > > Regarding your first question, the "accumulation interval" is
meant
> for
> > > > fields like precipitation. It'd be the time interval of the
> > accumulated
> > > > precip. For 2-m temperature, I'd just put in the level value
as fill
> > > data
> > > > (-9999) and the height as 2 (for 2 meters).
> > > >
> > > > Hope that helps.
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > > met_help at ucar.edu
> > > >
> > > >
> > > > On Tue, Jul 22, 2014 at 3:41 PM, Archana Dayalu via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue Jul 22 15:41:16 2014: Request 68300 was acted upon.
> > > > > Transaction: Ticket created by adayalu at seas.harvard.edu
> > > > > Queue: met_help
> > > > > Subject: using point-stat tool for daily averaged
observations
> > > > > Owner: Nobody
> > > > > Requestors: adayalu at seas.harvard.edu
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=68300
> > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > > I am a new METv4.1 user trying to use the point-stat tool to
verify
> > my
> > > > > WRF-ARWv3.5 model output.
> > > > > I have daily averaged observations and WRF fields output
every
> hour.
> > I
> > > am
> > > > > using the WRF Unified Post Processor to convert my netcdf
files to
> > > grib1
> > > > > format. The message type would be ADPSFC. My fields are
Temp, Wind
> > > speed,
> > > > > Pressure, RH, and Precip.
> > > > > A couple of questions:
> > > > > 1) Under the Level column of the ASCII point obs file, I
have been
> > > > putting
> > > > > "24" as the observation accumulation interval (since they
are daily
> > > > > averages). Did I understand that correctly?
> > > > > 2) Each wrf output file is 30 hours long. I have been using
UPP to
> > > split
> > > > it
> > > > > into 30 GRIB files, 1 for each hour. But this is a problem,
because
> > > what
> > > > I
> > > > > actually want to do is compare forecast daily averages with
obs
> daily
> > > > > averages ... and it appears what I am currently set up to do
is
> > compare
> > > > > forecast hourly averages with obs daily averages.
> > > > > Any input would be greatly appreciated.
> > > > > Thank you!
> > > > > Archana
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > ____________________________________________________
> > > *Archana Dayalu*
> > > Graduate Student
> > > Dept. of Earth and Planetary Sciences
> > > Harvard University
> > > 24 Oxford Street #402
> > > Cambridge, MA 02138
> > > (617) 384-8206
> > >
> > >
> >
> >
>
>
> --
> ____________________________________________________
> *Archana Dayalu*
> Graduate Student
> Dept. of Earth and Planetary Sciences
> Harvard University
> 24 Oxford Street #402
> Cambridge, MA 02138
> (617) 384-8206
>
>
------------------------------------------------
More information about the Met_help
mailing list