[Met_help] [rt.rap.ucar.edu #94792] History for Reading in Modified WRF Files
John Halley Gotway via RT
met_help at ucar.edu
Mon Apr 6 12:09:03 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am trying to read in a modified netCDF file in the point_stat command, along with preprocessed observations from pb2nc, and my verification statistics appear to be slightly wrong. I believe my problem is with the projection within my forecast file, which is Lambert conformal.
The attributes from the original WRF file are below, and I copy these into a smaller netCDF file (only has the variables I want to verify) and use the proper metadata format that the point_stat command can read, which is also printed out below:
Original WRF file:
// global attributes:
:TRUELAT1 = 38.5f ;
:TRUELAT2 = 38.5f ;
:STAND_LON = -97.5f ;
:CEN_LON = -97.48276f ;
:CEN_LAT = 38.51349f ;
:MOAD_CEN_LAT = 38.446f ;
:POLE_LAT = 90.f ;
:POLE_LON = 0.f ;
:MAP_PROJ = 1 ;
:MAP_PROJ_CHAR = "Lambert Conformal" ;
:START_DATE = "2017-04-22_12:00:00" ;
:SIMULATION_START_DATE = "2017-04-22_12:00:00” ;
Modified WRF file:
// global attributes:
:FileOrigins = "NA" ;
:MET_version = "V9.0" ;
:MET_tool = "pcp_combine" ;
:Projection = "Lambert Conformal" ;
:hemisphere = "N" ;
:scale_lat_1 = "38.500000" ;
:scale_lat_2 = "38.500000" ;
:lat_pin = "21.138123" ;
:lon_pin = "-122.71953" ;
:x_pin = "0.00000" ;
:y_pin = "0.00000" ;
:lon_orient = "-97.500000" ;
:d_km = "3.000000" ;
:r_km = "6371.200000" ;
:nx = "1800" ;
:ny = "1060 grid_points" ;
:FCST_VALID_BEG = "20170422_12” ;
My main concern is that I am not including the correct metadata in the modified netCDF file for the projection, which conversely impacts the point verification, i.e., the observation is not being mapped to the correct grid points. Are there any examples of the point_stat command reading in modified netCDF files, and in particular, for Lambert conformal gridded data? This is a fairly standard projection for WRF research, so I have to imagine someone has run into this issue before.
Thank you for any assistance you can provide
-Aaron Hill
________________________________
Aaron Hill, Ph.D.
Postdoctoral Fellow
Department of Atmospheric Science
Colorado State University
Email: aaron.hill at colostate.edu<mailto:aaron.hill at colostate.edu>
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Reading in Modified WRF Files
From: John Halley Gotway
Time: Tue Mar 31 17:42:53 2020
Aaron,
I see you have a question about preparing gridded data for use in
Point-Stat. So you'd like to run NetCDF output from WRF using Point-
Stat
and are wondering how to prepare the metadata to define the projection
information well.
MET reads a few different variants of NetCDF output, including the
NetCDF
output from a post-processing utility called wrf_interp (which used to
be
called p_interp). If all of the variables you extracted are at the
surface
and non-staggered (meaning, not winds), MET can be configured to read
the
data directly without adding additional metadata. Just try specifying
"file_type = NETCDF_PINT;" in the config file.
So let's say you've including WRF's "t2" variable containing 2-m
temperature in your output file named sample_wrf.nc. Try running
MET's
plot_data_plane tool on it like this:
plot_data_plane sample_wrf.nc plot_t2.ps 'name="t2"; level="(0,*,*)";
file_type=NETCDF_PINT;'
Just adapt that command using your file name and variable name.
Does that work? If so, just include the "file_type=NETCDF_PINT;"
inside
the "fcst" dictionary of the Point-Stat config file.
Thanks,
John Halley Gotway
On Tue, Mar 31, 2020 at 3:17 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Tue Mar 31 15:16:58 2020: Request 94792 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: Reading in Modified WRF Files
> Owner: johnhg
> Requestors: Aaron.Hill at colostate.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94792 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #94792] Reading in Modified WRF Files
From: Hill,Aaron
Time: Wed Apr 01 11:23:27 2020
Thank you John for the reply.
I was able to successfully run plot_data_plane as well as point_stat,
using the adjusted configuration setting of file_type=NETCDF_PINT, on
some of my WRF output; I have other output that is modified WRF and
doesn’t work because it has missing metadata. However, if I copy the
metadata that is written to the terminal window after executing
plot_data_plane to these modified WRF files, point_stat works on that
data as well.
With that being said, I am still curious why my verification
statistics output from point_stat would be incorrect. My problem is
this: I have two gridded forecasts (in netCDF) that are about 1 degree
different at an observation location; I can confirm this by ncdiff’ing
the two files. However, when I run point_stat with each gridded
forecast, verifying against that observation, the differences in the
point_stat output is less than 0.1 degrees. This is why I thought my
projection information was incorrect; if the projection is off,
point_stat would not be mapping the lat/lon of the observation
location correctly onto the grid.
So maybe I am misunderstanding how point_stat works? If I have gridded
2m temperature and want to verification against a 2m temperature
observation, there is only a horizontal interpolation, correct?
-Aaron
On Mar 31, 2020, at 5:42 PM, John Halley Gotway via RT
<met_help at ucar.edu> wrote:
Aaron,
I see you have a question about preparing gridded data for use in
Point-Stat. So you'd like to run NetCDF output from WRF using Point-
Stat
and are wondering how to prepare the metadata to define the projection
information well.
MET reads a few different variants of NetCDF output, including the
NetCDF
output from a post-processing utility called wrf_interp (which used to
be
called p_interp). If all of the variables you extracted are at the
surface
and non-staggered (meaning, not winds), MET can be configured to read
the
data directly without adding additional metadata. Just try specifying
"file_type = NETCDF_PINT;" in the config file.
So let's say you've including WRF's "t2" variable containing 2-m
temperature in your output file named sample_wrf.nc. Try running
MET's
plot_data_plane tool on it like this:
plot_data_plane sample_wrf.nc plot_t2.ps 'name="t2"; level="(0,*,*)";
file_type=NETCDF_PINT;'
Just adapt that command using your file name and variable name.
Does that work? If so, just include the "file_type=NETCDF_PINT;"
inside
the "fcst" dictionary of the Point-Stat config file.
Thanks,
John Halley Gotway
On Tue, Mar 31, 2020 at 3:17 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Tue Mar 31 15:16:58 2020: Request 94792 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: Reading in Modified WRF Files
> Owner: johnhg
> Requestors: Aaron.Hill at colostate.edu
> Status: new
> Ticket <URL:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D94792&data=02%7C01%7CAaron.Hill%40colostate.edu%7C58b6f747e22544321f3508d7d5cd3d01%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637212949782845476&sdata=ohoQPGMhBAkkXsy8Gm1If7ehHWG%2BW69qZ%2FPCgB5TiXo%3D&reserved=0
>
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: Reading in Modified WRF Files
From: John Halley Gotway
Time: Wed Apr 01 12:26:26 2020
Aaron,
Yes, that is correct. The processing steps for what you describe in
Point-Stat are basically:
(1) Read gridded data from the input NetCDF file and use the metadata
to
instantiate the grid.
(2) Loop through the point observations and assign them to the
verification
tasks requested in the config file (e.g. verify 2-m temperature).
(3) For each obs, interpolate the nearby forecast values to the
lat/lon
location of that obs.
There are many interpolation options here and you can configure
point-stat to use any many of them as you'd like.
To understand the details, here's only thing you try... configure
point-stat to use many interp methods. Search for "method" in the
README
file:
https://github.com/NCAR/MET/blob/master_v9.0/met/data/config/README
interp = { vld_thresh = 1.0; shape = SQUARE; type = [ { method =
NEAREST;
width = 1; }, { method = UPPER_LEFT; width = 1; }, { method =
UPPER_RIGHT;
width = 1; }, { method = LOWER_LEFT; width = 1; }, { method =
LOWER_RIGHT;
width = 1; }, { method = BILIN; width = 2; } ]; } Also make sure that
the
matched pair (MPR) output line type is requested in the "output_flag"
dictionary. Then your output will contain 6 versions of interpolating:
one
for the nearest forecast grid point value, four for each of the
corners,
and one using bilinear interpolation of those those corners. The
latter is
the what I typically see used for temperature.
Does that shed any more light on what Point-Stat is doing?
Thanks,
John
On Wed, Apr 1, 2020 at 11:24 AM Hill,Aaron via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94792 >
>
> Thank you John for the reply.
>
> I was able to successfully run plot_data_plane as well as
point_stat,
> using the adjusted configuration setting of file_type=NETCDF_PINT,
on some
> of my WRF output; I have other output that is modified WRF and
doesn’t work
> because it has missing metadata. However, if I copy the metadata
that is
> written to the terminal window after executing plot_data_plane to
these
> modified WRF files, point_stat works on that data as well.
>
> With that being said, I am still curious why my verification
statistics
> output from point_stat would be incorrect. My problem is this: I
have two
> gridded forecasts (in netCDF) that are about 1 degree different at
an
> observation location; I can confirm this by ncdiff’ing the two
files.
> However, when I run point_stat with each gridded forecast, verifying
> against that observation, the differences in the point_stat output
is less
> than 0.1 degrees. This is why I thought my projection information
was
> incorrect; if the projection is off, point_stat would not be mapping
the
> lat/lon of the observation location correctly onto the grid.
>
> So maybe I am misunderstanding how point_stat works? If I have
gridded 2m
> temperature and want to verification against a 2m temperature
observation,
> there is only a horizontal interpolation, correct?
>
> -Aaron
>
> On Mar 31, 2020, at 5:42 PM, John Halley Gotway via RT
<met_help at ucar.edu>
> wrote:
>
> Aaron,
>
> I see you have a question about preparing gridded data for use in
> Point-Stat. So you'd like to run NetCDF output from WRF using
Point-Stat
> and are wondering how to prepare the metadata to define the
projection
> information well.
>
> MET reads a few different variants of NetCDF output, including the
NetCDF
> output from a post-processing utility called wrf_interp (which used
to be
> called p_interp). If all of the variables you extracted are at the
surface
> and non-staggered (meaning, not winds), MET can be configured to
read the
> data directly without adding additional metadata. Just try
specifying
> "file_type = NETCDF_PINT;" in the config file.
>
> So let's say you've including WRF's "t2" variable containing 2-m
> temperature in your output file named sample_wrf.nc. Try running
MET's
> plot_data_plane tool on it like this:
>
> plot_data_plane sample_wrf.nc plot_t2.ps 'name="t2";
level="(0,*,*)";
> file_type=NETCDF_PINT;'
>
> Just adapt that command using your file name and variable name.
>
> Does that work? If so, just include the "file_type=NETCDF_PINT;"
inside
> the "fcst" dictionary of the Point-Stat config file.
>
> Thanks,
> John Halley Gotway
>
>
>
> On Tue, Mar 31, 2020 at 3:17 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Tue Mar 31 15:16:58 2020: Request 94792 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > Queue: met_help
> > Subject: Reading in Modified WRF Files
> > Owner: johnhg
> > Requestors: Aaron.Hill at colostate.edu
> > Status: new
> > Ticket <URL:
>
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D94792&data=02%7C01%7CAaron.Hill%40colostate.edu%7C58b6f747e22544321f3508d7d5cd3d01%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637212949782845476&sdata=ohoQPGMhBAkkXsy8Gm1If7ehHWG%2BW69qZ%2FPCgB5TiXo%3D&reserved=0
> >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
>
>
------------------------------------------------
Subject: Reading in Modified WRF Files
From: Hill,Aaron
Time: Mon Apr 06 10:11:07 2020
Thank you, John, for all the help. I think I have finally solved my
issues, thanks to more verbose output from point_stat. It turns out I
needed to be using the center lat/lon from my grid and not the lower
left lat/lon. This change essentially solved the gridding issues I was
seeing.
I appreciate the help!
-Aaron
On Apr 1, 2020, at 12:26 PM, John Halley Gotway via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>> wrote:
Aaron,
Yes, that is correct. The processing steps for what you describe in
Point-Stat are basically:
(1) Read gridded data from the input NetCDF file and use the metadata
to
instantiate the grid.
(2) Loop through the point observations and assign them to the
verification
tasks requested in the config file (e.g. verify 2-m temperature).
(3) For each obs, interpolate the nearby forecast values to the
lat/lon
location of that obs.
There are many interpolation options here and you can configure
point-stat to use any many of them as you'd like.
To understand the details, here's only thing you try... configure
point-stat to use many interp methods. Search for "method" in the
README
file:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNCAR%2FMET%2Fblob%2Fmaster_v9.0%2Fmet%2Fdata%2Fconfig%2FREADME&data=02%7C01%7CAaron.Hill%40colostate.edu%7Cab5713bc1e634a37ff0608d7d66a3216%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637213623912062923&sdata=6pjJtmEiPb%2FK%2FEXZ8UPhGBtWAdZg5nTDLR8DEgiP8f8%3D&reserved=0
interp = { vld_thresh = 1.0; shape = SQUARE; type = [ { method =
NEAREST;
width = 1; }, { method = UPPER_LEFT; width = 1; }, { method =
UPPER_RIGHT;
width = 1; }, { method = LOWER_LEFT; width = 1; }, { method =
LOWER_RIGHT;
width = 1; }, { method = BILIN; width = 2; } ]; } Also make sure that
the
matched pair (MPR) output line type is requested in the "output_flag"
dictionary. Then your output will contain 6 versions of interpolating:
one
for the nearest forecast grid point value, four for each of the
corners,
and one using bilinear interpolation of those those corners. The
latter is
the what I typically see used for temperature.
Does that shed any more light on what Point-Stat is doing?
Thanks,
John
On Wed, Apr 1, 2020 at 11:24 AM Hill,Aaron via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>> wrote:
<URL:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D94792&data=02%7C01%7CAaron.Hill%40colostate.edu%7Cab5713bc1e634a37ff0608d7d66a3216%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637213623912062923&sdata=acGPYGM92gGTD7EttC10%2Btqf9G503gM0NMY0Nzl3RDo%3D&reserved=0
>
Thank you John for the reply.
I was able to successfully run plot_data_plane as well as point_stat,
using the adjusted configuration setting of file_type=NETCDF_PINT, on
some
of my WRF output; I have other output that is modified WRF and doesn’t
work
because it has missing metadata. However, if I copy the metadata that
is
written to the terminal window after executing plot_data_plane to
these
modified WRF files, point_stat works on that data as well.
With that being said, I am still curious why my verification
statistics
output from point_stat would be incorrect. My problem is this: I have
two
gridded forecasts (in netCDF) that are about 1 degree different at an
observation location; I can confirm this by ncdiff’ing the two files.
However, when I run point_stat with each gridded forecast, verifying
against that observation, the differences in the point_stat output is
less
than 0.1 degrees. This is why I thought my projection information was
incorrect; if the projection is off, point_stat would not be mapping
the
lat/lon of the observation location correctly onto the grid.
So maybe I am misunderstanding how point_stat works? If I have gridded
2m
temperature and want to verification against a 2m temperature
observation,
there is only a horizontal interpolation, correct?
-Aaron
On Mar 31, 2020, at 5:42 PM, John Halley Gotway via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>>
wrote:
Aaron,
I see you have a question about preparing gridded data for use in
Point-Stat. So you'd like to run NetCDF output from WRF using Point-
Stat
and are wondering how to prepare the metadata to define the projection
information well.
MET reads a few different variants of NetCDF output, including the
NetCDF
output from a post-processing utility called wrf_interp (which used to
be
called p_interp). If all of the variables you extracted are at the
surface
and non-staggered (meaning, not winds), MET can be configured to read
the
data directly without adding additional metadata. Just try specifying
"file_type = NETCDF_PINT;" in the config file.
So let's say you've including WRF's "t2" variable containing 2-m
temperature in your output file named sample_wrf.nc. Try running
MET's
plot_data_plane tool on it like this:
plot_data_plane sample_wrf.nc plot_t2.ps 'name="t2"; level="(0,*,*)";
file_type=NETCDF_PINT;'
Just adapt that command using your file name and variable name.
Does that work? If so, just include the "file_type=NETCDF_PINT;"
inside
the "fcst" dictionary of the Point-Stat config file.
Thanks,
John Halley Gotway
On Tue, Mar 31, 2020 at 3:17 PM George McCabe via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>>
wrote:
Tue Mar 31 15:16:58 2020: Request 94792 was acted upon.
Transaction: Given to johnhg (John Halley Gotway) by mccabe
Queue: met_help
Subject: Reading in Modified WRF Files
Owner: johnhg
Requestors: Aaron.Hill at colostate.edu<mailto:Aaron.Hill at colostate.edu>
Status: new
Ticket <URL:
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D94792&data=02%7C01%7CAaron.Hill%40colostate.edu%7Cab5713bc1e634a37ff0608d7d66a3216%7Cafb58802ff7a4bb1ab21367ff2ecfc8b%7C0%7C0%7C637213623912062923&sdata=acGPYGM92gGTD7EttC10%2Btqf9G503gM0NMY0Nzl3RDo%3D&reserved=0
This transaction appears to have no content
------------------------------------------------
More information about the Met_help
mailing list