[Met_help] [rt.rap.ucar.edu #92934] History for Question about using MET with grid-stat, copygb
John Halley Gotway via RT
met_help at ucar.edu
Thu Nov 7 09:50:23 MST 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
To Whom It May Concern,
I'm looking to do a comparison of two forecasts to the same analysis file. Namely, HRRR forecast vs. HRRR analysis and WRF forecast vs. HRRR analysis. The WRF forecasts cover just California while the HRRR forecasts/analysis is over the CONUS. The WRF files underwent UPP to GRIB1 format, while the HRRR forecasts/analysis are GRIB2 format.
I've read that grid-stat is useful for comparing gridded forecasts to gridded observations, so I figured I might be able to use this tool for my comparison. I know that grid-stat requires the two files to be on the same grid; which explains why grid-stat worked for when I did HRRR forecast vs HRRR analysis, but not for when I did WRF forecast vs HRRR analysis. So I've been trying to figure out how to get the grid-stat to work for WRF forecasts vs HRRR analysis.
I saw that the tool copygb can be used for regridding datasets to be on the same grid. However, when I tried using it for the HRRR analysis to convert to the grid of WRF, it did not work (produced an empty file). I presume this is because the HRRR file is GRIB2 format, and the copygb only works for GRIB1 formats. But when I tried it going from WRF to the HRRR analysis grid, it also did not work, which I presume is because the WRF forecast domain is a subset of HRRR's domain and thus cannot be put onto the same grid?
I saw something about copygb2 online, which maybe is what I need to convert a GRIB2 file. I also read online about wgrib2 and the options "-set_grib_type same -new_grid" (see my test command below) but I then got a message about the "IPOLATES" package not being installed, which I have no clue about.
>> wgrib2 /shared/cbam/DATA/HRRR_SFCA/HRRR_SFCA_2019102103_f000.grb2 -set_grib_type same -new_grid lambert:260.300000:26.000000 254.353000:600:2000 25.157000:600:2000 tr.grb2
>> IPOLATES package is not installed
So my question which I'm seeking help for is:
(1) Do you think the approach I'm using for comparing HRRR forecasts vs HRRR analysis and WRF forecasts vs HRRR analysis is correct to be using the grid-stat tool? Or should I be using a different tool? My goal is to see whether the HRRR or WRF forecasts can best predict the 80m wind speeds over California using the HRRR analysis as a reference.
(2) If grid-stat is the way to go, do you know if I need to be using copygb2 or installing the IPOLATES tool, to accomplish the required regridding? Or is there a better way to go about it?
If there is a sftp or ftp portal, I can send the WRF file and example HRRR analysis file if needed, since they are too big to send via email. Although if someone has an example WRF file or HRRR file they can probably replicate what I'm doing.
Any insights or help would be extremely helpful. Thank you in advance for your time!!
Best,
Jeremy
P.S. Here are the grid information for HRRR analysis and my WRF forecast, in case it is useful:
HRRR:
132:46093348:vt=2019102103:275 mb:anl:VVEL Vertical Velocity (Pressure) [Pa/s]:
ndata=1905141:undef=0:mean=0.00790637:min=-0.0613893:max=0.438611
grid_template=30:winds(grid):
Lambert Conformal: (1799 x 1059) input WE:SN output WE:SN res 8
Lat1 21.138123 Lon1 237.280472 LoV 262.500000
LatD 38.500000 Latin1 38.500000 Latin2 38.500000
LatSP 0.000000 LonSP 0.000000
North Pole (1799 x 1059) Dx 3000.000000 m Dy 3000.000000 m mode 8
WRF:
rec 401:214487150:date 2019102912 CIN kpds5=156 kpds6=116 kpds7=46080 levels=(180,0) grid=255 180-0 mb above gnd 3hr fcst:
CIN=Convective inhibition [J/kg]
timerange 0 P1 3 P2 0 TimeU 1 nx 600 ny 600 GDS grid 3 num_in_ave 0 missing 0
center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
Lambert Conf: Lat1 25.157000 Lon1 254.353000 Lov 260.300000
Latin1 26.000000 Latin2 36.000000 LatSP 0.000000 LonSP 0.000000
North Pole (600 x 600) Dx 2.000000 Dy 2.000000 scan 64 mode 136
min/max data -509.727 0.273438 num bits 9 BDS_Ref -509.727 DecScale 0 BinScale 0
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Question about using MET with grid-stat, copygb
From: John Halley Gotway
Time: Mon Nov 04 14:40:57 2019
Jeremy,
Good news. There is functionality for handling this regridding
directly in
MET without the need to regrid using copygb. And yes, Grid-Stat is a
good
choice for this comparison.
So you're comparing HRRR to HRRR-Analysis and then WRF to HRRR-
Analysis.
And then presumably, you'd like to see how those stats compare. I
presume
that the WRF domain is a subset of the HRRR domain. I do see that WRF
is
2km grid spacing while HRRR is 3km grid spacing. In order to make a
fair
comparison, I'd recommend choosing a single domain on which you do the
verification.
One choice would be using the 2km WRF domain. However, that is a
little
unfair since you'd be "creating" new HRRR data when you regrid from 3
to 2
km resolution. For the sake of this example, though, let's say you
have a
WRF output file named "wrf.grb" that lives on that 2km CA domain.
Here's the Grid-Stat config file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/GridStatConfig_default
In there, you'll find a section named "regrid" which defines the grid
on
which the verification should be done. In the example below, I set
"to_grid" to the path to the GRIB file that's on the 2km WRF domain:
// Verification grid
// May be set separately in each "field" entry
//
regrid = {
to_grid = "/path/to/wrf.grb";
method = NEAREST;
width = 1;
vld_thresh = 0.5;
shape = SQUARE;
}
Note that the default regridding method is nearest neighbor. You may
want
to use bilinear interpolation (or something else) instead:
method = BILIN; width = 2;
Hope that helps.
Thanks,
John Halley Gotway
On Mon, Nov 4, 2019 at 1:00 PM Berman, Jeremy D via RT
<met_help at ucar.edu>
wrote:
>
> Mon Nov 04 13:00:13 2019: Request 92934 was acted upon.
> Transaction: Ticket created by jdberman at albany.edu
> Queue: met_help
> Subject: Question about using MET with grid-stat, copygb
> Owner: Nobody
> Requestors: jdberman at albany.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92934 >
>
>
> To Whom It May Concern,
>
> I'm looking to do a comparison of two forecasts to the same analysis
file.
> Namely, HRRR forecast vs. HRRR analysis and WRF forecast vs. HRRR
analysis.
> The WRF forecasts cover just California while the HRRR
forecasts/analysis
> is over the CONUS. The WRF files underwent UPP to GRIB1 format,
while the
> HRRR forecasts/analysis are GRIB2 format.
>
> I've read that grid-stat is useful for comparing gridded forecasts
to
> gridded observations, so I figured I might be able to use this tool
for my
> comparison. I know that grid-stat requires the two files to be on
the same
> grid; which explains why grid-stat worked for when I did HRRR
forecast vs
> HRRR analysis, but not for when I did WRF forecast vs HRRR analysis.
So
> I've been trying to figure out how to get the grid-stat to work for
WRF
> forecasts vs HRRR analysis.
>
> I saw that the tool copygb can be used for regridding datasets to be
on
> the same grid. However, when I tried using it for the HRRR analysis
to
> convert to the grid of WRF, it did not work (produced an empty
file). I
> presume this is because the HRRR file is GRIB2 format, and the
copygb only
> works for GRIB1 formats. But when I tried it going from WRF to the
HRRR
> analysis grid, it also did not work, which I presume is because the
WRF
> forecast domain is a subset of HRRR's domain and thus cannot be put
onto
> the same grid?
>
> I saw something about copygb2 online, which maybe is what I need to
> convert a GRIB2 file. I also read online about wgrib2 and the
options
> "-set_grib_type same -new_grid" (see my test command below) but I
then got
> a message about the "IPOLATES" package not being installed, which I
have no
> clue about.
>
>
> >> wgrib2 /shared/cbam/DATA/HRRR_SFCA/HRRR_SFCA_2019102103_f000.grb2
> -set_grib_type same -new_grid lambert:260.300000:26.000000
> 254.353000:600:2000 25.157000:600:2000 tr.grb2
>
> >> IPOLATES package is not installed
>
>
> So my question which I'm seeking help for is:
>
> (1) Do you think the approach I'm using for comparing HRRR forecasts
vs
> HRRR analysis and WRF forecasts vs HRRR analysis is correct to be
using the
> grid-stat tool? Or should I be using a different tool? My goal is to
see
> whether the HRRR or WRF forecasts can best predict the 80m wind
speeds over
> California using the HRRR analysis as a reference.
>
> (2) If grid-stat is the way to go, do you know if I need to be using
> copygb2 or installing the IPOLATES tool, to accomplish the required
> regridding? Or is there a better way to go about it?
>
>
> If there is a sftp or ftp portal, I can send the WRF file and
example HRRR
> analysis file if needed, since they are too big to send via email.
Although
> if someone has an example WRF file or HRRR file they can probably
replicate
> what I'm doing.
>
> Any insights or help would be extremely helpful. Thank you in
advance for
> your time!!
>
> Best,
> Jeremy
>
>
>
>
> P.S. Here are the grid information for HRRR analysis and my WRF
forecast,
> in case it is useful:
>
> HRRR:
>
>
> 132:46093348:vt=2019102103:275 mb:anl:VVEL Vertical Velocity
(Pressure)
> [Pa/s]:
>
> ndata=1905141:undef=0:mean=0.00790637:min=-
0.0613893:max=0.438611
>
> grid_template=30:winds(grid):
>
> Lambert Conformal: (1799 x 1059) input WE:SN output WE:SN res 8
>
> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
>
> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
>
> LatSP 0.000000 LonSP 0.000000
>
> North Pole (1799 x 1059) Dx 3000.000000 m Dy 3000.000000 m mode 8
>
>
> WRF:
>
>
> rec 401:214487150:date 2019102912 CIN kpds5=156 kpds6=116
kpds7=46080
> levels=(180,0) grid=255 180-0 mb above gnd 3hr fcst:
>
> CIN=Convective inhibition [J/kg]
>
> timerange 0 P1 3 P2 0 TimeU 1 nx 600 ny 600 GDS grid 3 num_in_ave
0
> missing 0
>
> center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
>
> Lambert Conf: Lat1 25.157000 Lon1 254.353000 Lov 260.300000
>
> Latin1 26.000000 Latin2 36.000000 LatSP 0.000000 LonSP
0.000000
>
> North Pole (600 x 600) Dx 2.000000 Dy 2.000000 scan 64 mode
136
>
> min/max data -509.727 0.273438 num bits 9 BDS_Ref -509.727
DecScale 0
> BinScale 0
>
>
>
>
------------------------------------------------
Subject: Question about using MET with grid-stat, copygb
From: Berman, Jeremy D
Time: Tue Nov 05 14:42:41 2019
Hi John,
Thank you for the quick response! That regridding tool within MET is
so much easier to use than copygb!
You're correct that the WRF domain is a subset of the HRRR domain
(just over California), and the WRF is 2km while HRRR is 3km. I'm also
using a "mask.poly" over just northern California, which is within the
WRF and HRRR domains. I tried out your suggested modifications and it
worked!
Just a follow-up question: you suggested using the 2km WRF domain as
the "verification domain" for regridding using the "regrid" option, so
that when I'm comparing WRF forecasts to HRRR analysis, it will regrid
the 3km HRRR analysis to 2km for that grid_stat calculation. If I'm
then computing HRRR forecasts compared to HRRR analysis, would I want
to turn off this regrid option, since they are already on the same
grid? Or is it best to keep it on, for comparisons? And finally, is
there any benefit to doing the reverse, and regrid the 2km WRF grid to
the 3km HRRR analysis when doing the comparison (or can you only
regrid the verification grid and not the model forecasts)? Just
looking for the most fair option haha.
Thanks for all your help! This helped a lot!
Jeremy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, November 4, 2019 4:40 PM
To: Berman, Jeremy D <jdberman at albany.edu>
Subject: Re: [rt.rap.ucar.edu #92934] Question about using MET with
grid-stat, copygb
Jeremy,
Good news. There is functionality for handling this regridding
directly in
MET without the need to regrid using copygb. And yes, Grid-Stat is a
good
choice for this comparison.
So you're comparing HRRR to HRRR-Analysis and then WRF to HRRR-
Analysis.
And then presumably, you'd like to see how those stats compare. I
presume
that the WRF domain is a subset of the HRRR domain. I do see that WRF
is
2km grid spacing while HRRR is 3km grid spacing. In order to make a
fair
comparison, I'd recommend choosing a single domain on which you do the
verification.
One choice would be using the 2km WRF domain. However, that is a
little
unfair since you'd be "creating" new HRRR data when you regrid from 3
to 2
km resolution. For the sake of this example, though, let's say you
have a
WRF output file named "wrf.grb" that lives on that 2km CA domain.
Here's the Grid-Stat config file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/GridStatConfig_default
In there, you'll find a section named "regrid" which defines the grid
on
which the verification should be done. In the example below, I set
"to_grid" to the path to the GRIB file that's on the 2km WRF domain:
// Verification grid
// May be set separately in each "field" entry
//
regrid = {
to_grid = "/path/to/wrf.grb";
method = NEAREST;
width = 1;
vld_thresh = 0.5;
shape = SQUARE;
}
Note that the default regridding method is nearest neighbor. You may
want
to use bilinear interpolation (or something else) instead:
method = BILIN; width = 2;
Hope that helps.
Thanks,
John Halley Gotway
On Mon, Nov 4, 2019 at 1:00 PM Berman, Jeremy D via RT
<met_help at ucar.edu>
wrote:
>
> Mon Nov 04 13:00:13 2019: Request 92934 was acted upon.
> Transaction: Ticket created by jdberman at albany.edu
> Queue: met_help
> Subject: Question about using MET with grid-stat, copygb
> Owner: Nobody
> Requestors: jdberman at albany.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92934 >
>
>
> To Whom It May Concern,
>
> I'm looking to do a comparison of two forecasts to the same analysis
file.
> Namely, HRRR forecast vs. HRRR analysis and WRF forecast vs. HRRR
analysis.
> The WRF forecasts cover just California while the HRRR
forecasts/analysis
> is over the CONUS. The WRF files underwent UPP to GRIB1 format,
while the
> HRRR forecasts/analysis are GRIB2 format.
>
> I've read that grid-stat is useful for comparing gridded forecasts
to
> gridded observations, so I figured I might be able to use this tool
for my
> comparison. I know that grid-stat requires the two files to be on
the same
> grid; which explains why grid-stat worked for when I did HRRR
forecast vs
> HRRR analysis, but not for when I did WRF forecast vs HRRR analysis.
So
> I've been trying to figure out how to get the grid-stat to work for
WRF
> forecasts vs HRRR analysis.
>
> I saw that the tool copygb can be used for regridding datasets to be
on
> the same grid. However, when I tried using it for the HRRR analysis
to
> convert to the grid of WRF, it did not work (produced an empty
file). I
> presume this is because the HRRR file is GRIB2 format, and the
copygb only
> works for GRIB1 formats. But when I tried it going from WRF to the
HRRR
> analysis grid, it also did not work, which I presume is because the
WRF
> forecast domain is a subset of HRRR's domain and thus cannot be put
onto
> the same grid?
>
> I saw something about copygb2 online, which maybe is what I need to
> convert a GRIB2 file. I also read online about wgrib2 and the
options
> "-set_grib_type same -new_grid" (see my test command below) but I
then got
> a message about the "IPOLATES" package not being installed, which I
have no
> clue about.
>
>
> >> wgrib2 /shared/cbam/DATA/HRRR_SFCA/HRRR_SFCA_2019102103_f000.grb2
> -set_grib_type same -new_grid lambert:260.300000:26.000000
> 254.353000:600:2000 25.157000:600:2000 tr.grb2
>
> >> IPOLATES package is not installed
>
>
> So my question which I'm seeking help for is:
>
> (1) Do you think the approach I'm using for comparing HRRR forecasts
vs
> HRRR analysis and WRF forecasts vs HRRR analysis is correct to be
using the
> grid-stat tool? Or should I be using a different tool? My goal is to
see
> whether the HRRR or WRF forecasts can best predict the 80m wind
speeds over
> California using the HRRR analysis as a reference.
>
> (2) If grid-stat is the way to go, do you know if I need to be using
> copygb2 or installing the IPOLATES tool, to accomplish the required
> regridding? Or is there a better way to go about it?
>
>
> If there is a sftp or ftp portal, I can send the WRF file and
example HRRR
> analysis file if needed, since they are too big to send via email.
Although
> if someone has an example WRF file or HRRR file they can probably
replicate
> what I'm doing.
>
> Any insights or help would be extremely helpful. Thank you in
advance for
> your time!!
>
> Best,
> Jeremy
>
>
>
>
> P.S. Here are the grid information for HRRR analysis and my WRF
forecast,
> in case it is useful:
>
> HRRR:
>
>
> 132:46093348:vt=2019102103:275 mb:anl:VVEL Vertical Velocity
(Pressure)
> [Pa/s]:
>
> ndata=1905141:undef=0:mean=0.00790637:min=-
0.0613893:max=0.438611
>
> grid_template=30:winds(grid):
>
> Lambert Conformal: (1799 x 1059) input WE:SN output WE:SN res 8
>
> Lat1 21.138123 Lon1 237.280472 LoV 262.500000
>
> LatD 38.500000 Latin1 38.500000 Latin2 38.500000
>
> LatSP 0.000000 LonSP 0.000000
>
> North Pole (1799 x 1059) Dx 3000.000000 m Dy 3000.000000 m mode 8
>
>
> WRF:
>
>
> rec 401:214487150:date 2019102912 CIN kpds5=156 kpds6=116
kpds7=46080
> levels=(180,0) grid=255 180-0 mb above gnd 3hr fcst:
>
> CIN=Convective inhibition [J/kg]
>
> timerange 0 P1 3 P2 0 TimeU 1 nx 600 ny 600 GDS grid 3 num_in_ave
0
> missing 0
>
> center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
>
> Lambert Conf: Lat1 25.157000 Lon1 254.353000 Lov 260.300000
>
> Latin1 26.000000 Latin2 36.000000 LatSP 0.000000 LonSP
0.000000
>
> North Pole (600 x 600) Dx 2.000000 Dy 2.000000 scan 64 mode
136
>
> min/max data -509.727 0.273438 num bits 9 BDS_Ref -509.727
DecScale 0
> BinScale 0
>
>
>
>
------------------------------------------------
Subject: Question about using MET with grid-stat, copygb
From: John Halley Gotway
Time: Tue Nov 05 16:01:04 2019
Jeremy,
Great, I'm glad to hear that you were able to make progress.
As to your question on verification grids, there is no obvious right
or
wrong answer. Generally, I do think it's best practice to pick a
single
domain on which to perform the analysis. Whether that grid is the 2km
WRF
domain, the 3km HRRR domain, or some other domain, I'm not sure. My
recommendation on using the 2km WRF domain was just to make it as easy
as
possible for you. Since the WRF domain is a subset of the HRRR
domain,
doing the vx on that domain would result in the same number of grid
points
being used... without needing to define a polyline.
So another option would be verifying on the 3km HRRR domain, using the
polyline region to make sure you're using the same set of grid points
in
the computation of statistics.
Since the HRRR domain is very dense, please be sure to run your
"mask.poly"
file through gen_vx_mask and use the NetCDF output of that in the
Grid-Stat
config file. That'll make Grid-Stat run faster without needing to
regenerate the mask every time you run Grid-Stat.
Thanks,
John
On Tue, Nov 5, 2019 at 2:43 PM Berman, Jeremy D via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92934 >
>
> Hi John,
>
> Thank you for the quick response! That regridding tool within MET is
so
> much easier to use than copygb!
>
> You're correct that the WRF domain is a subset of the HRRR domain
(just
> over California), and the WRF is 2km while HRRR is 3km. I'm also
using a
> "mask.poly" over just northern California, which is within the WRF
and HRRR
> domains. I tried out your suggested modifications and it worked!
>
> Just a follow-up question: you suggested using the 2km WRF domain as
the
> "verification domain" for regridding using the "regrid" option, so
that
> when I'm comparing WRF forecasts to HRRR analysis, it will regrid
the 3km
> HRRR analysis to 2km for that grid_stat calculation. If I'm then
computing
> HRRR forecasts compared to HRRR analysis, would I want to turn off
this
> regrid option, since they are already on the same grid? Or is it
best to
> keep it on, for comparisons? And finally, is there any benefit to
doing the
> reverse, and regrid the 2km WRF grid to the 3km HRRR analysis when
doing
> the comparison (or can you only regrid the verification grid and not
the
> model forecasts)? Just looking for the most fair option haha.
>
>
> Thanks for all your help! This helped a lot!
>
> Jeremy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, November 4, 2019 4:40 PM
> To: Berman, Jeremy D <jdberman at albany.edu>
> Subject: Re: [rt.rap.ucar.edu #92934] Question about using MET with
> grid-stat, copygb
>
> Jeremy,
>
> Good news. There is functionality for handling this regridding
directly in
> MET without the need to regrid using copygb. And yes, Grid-Stat is
a good
> choice for this comparison.
>
> So you're comparing HRRR to HRRR-Analysis and then WRF to HRRR-
Analysis.
> And then presumably, you'd like to see how those stats compare. I
presume
> that the WRF domain is a subset of the HRRR domain. I do see that
WRF is
> 2km grid spacing while HRRR is 3km grid spacing. In order to make a
fair
> comparison, I'd recommend choosing a single domain on which you do
the
> verification.
>
> One choice would be using the 2km WRF domain. However, that is a
little
> unfair since you'd be "creating" new HRRR data when you regrid from
3 to 2
> km resolution. For the sake of this example, though, let's say you
have a
> WRF output file named "wrf.grb" that lives on that 2km CA domain.
>
> Here's the Grid-Stat config file:
>
>
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/GridStatConfig_default
>
> In there, you'll find a section named "regrid" which defines the
grid on
> which the verification should be done. In the example below, I set
> "to_grid" to the path to the GRIB file that's on the 2km WRF domain:
>
> // Verification grid
> // May be set separately in each "field" entry
> //
> regrid = {
> to_grid = "/path/to/wrf.grb";
> method = NEAREST;
> width = 1;
> vld_thresh = 0.5;
> shape = SQUARE;
> }
>
> Note that the default regridding method is nearest neighbor. You
may want
> to use bilinear interpolation (or something else) instead:
> method = BILIN; width = 2;
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
>
> On Mon, Nov 4, 2019 at 1:00 PM Berman, Jeremy D via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Mon Nov 04 13:00:13 2019: Request 92934 was acted upon.
> > Transaction: Ticket created by jdberman at albany.edu
> > Queue: met_help
> > Subject: Question about using MET with grid-stat, copygb
> > Owner: Nobody
> > Requestors: jdberman at albany.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92934 >
> >
> >
> > To Whom It May Concern,
> >
> > I'm looking to do a comparison of two forecasts to the same
analysis
> file.
> > Namely, HRRR forecast vs. HRRR analysis and WRF forecast vs. HRRR
> analysis.
> > The WRF forecasts cover just California while the HRRR
forecasts/analysis
> > is over the CONUS. The WRF files underwent UPP to GRIB1 format,
while the
> > HRRR forecasts/analysis are GRIB2 format.
> >
> > I've read that grid-stat is useful for comparing gridded forecasts
to
> > gridded observations, so I figured I might be able to use this
tool for
> my
> > comparison. I know that grid-stat requires the two files to be on
the
> same
> > grid; which explains why grid-stat worked for when I did HRRR
forecast vs
> > HRRR analysis, but not for when I did WRF forecast vs HRRR
analysis. So
> > I've been trying to figure out how to get the grid-stat to work
for WRF
> > forecasts vs HRRR analysis.
> >
> > I saw that the tool copygb can be used for regridding datasets to
be on
> > the same grid. However, when I tried using it for the HRRR
analysis to
> > convert to the grid of WRF, it did not work (produced an empty
file). I
> > presume this is because the HRRR file is GRIB2 format, and the
copygb
> only
> > works for GRIB1 formats. But when I tried it going from WRF to the
HRRR
> > analysis grid, it also did not work, which I presume is because
the WRF
> > forecast domain is a subset of HRRR's domain and thus cannot be
put onto
> > the same grid?
> >
> > I saw something about copygb2 online, which maybe is what I need
to
> > convert a GRIB2 file. I also read online about wgrib2 and the
options
> > "-set_grib_type same -new_grid" (see my test command below) but I
then
> got
> > a message about the "IPOLATES" package not being installed, which
I have
> no
> > clue about.
> >
> >
> > >> wgrib2
/shared/cbam/DATA/HRRR_SFCA/HRRR_SFCA_2019102103_f000.grb2
> > -set_grib_type same -new_grid lambert:260.300000:26.000000
> > 254.353000:600:2000 25.157000:600:2000 tr.grb2
> >
> > >> IPOLATES package is not installed
> >
> >
> > So my question which I'm seeking help for is:
> >
> > (1) Do you think the approach I'm using for comparing HRRR
forecasts vs
> > HRRR analysis and WRF forecasts vs HRRR analysis is correct to be
using
> the
> > grid-stat tool? Or should I be using a different tool? My goal is
to see
> > whether the HRRR or WRF forecasts can best predict the 80m wind
speeds
> over
> > California using the HRRR analysis as a reference.
> >
> > (2) If grid-stat is the way to go, do you know if I need to be
using
> > copygb2 or installing the IPOLATES tool, to accomplish the
required
> > regridding? Or is there a better way to go about it?
> >
> >
> > If there is a sftp or ftp portal, I can send the WRF file and
example
> HRRR
> > analysis file if needed, since they are too big to send via email.
> Although
> > if someone has an example WRF file or HRRR file they can probably
> replicate
> > what I'm doing.
> >
> > Any insights or help would be extremely helpful. Thank you in
advance for
> > your time!!
> >
> > Best,
> > Jeremy
> >
> >
> >
> >
> > P.S. Here are the grid information for HRRR analysis and my WRF
forecast,
> > in case it is useful:
> >
> > HRRR:
> >
> >
> > 132:46093348:vt=2019102103:275 mb:anl:VVEL Vertical Velocity
(Pressure)
> > [Pa/s]:
> >
> > ndata=1905141:undef=0:mean=0.00790637:min=-
0.0613893:max=0.438611
> >
> > grid_template=30:winds(grid):
> >
> > Lambert Conformal: (1799 x 1059) input WE:SN output WE:SN res 8
> >
> > Lat1 21.138123 Lon1 237.280472 LoV 262.500000
> >
> > LatD 38.500000 Latin1 38.500000 Latin2 38.500000
> >
> > LatSP 0.000000 LonSP 0.000000
> >
> > North Pole (1799 x 1059) Dx 3000.000000 m Dy 3000.000000 m mode 8
> >
> >
> > WRF:
> >
> >
> > rec 401:214487150:date 2019102912 CIN kpds5=156 kpds6=116
kpds7=46080
> > levels=(180,0) grid=255 180-0 mb above gnd 3hr fcst:
> >
> > CIN=Convective inhibition [J/kg]
> >
> > timerange 0 P1 3 P2 0 TimeU 1 nx 600 ny 600 GDS grid 3
num_in_ave 0
> > missing 0
> >
> > center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
> >
> > Lambert Conf: Lat1 25.157000 Lon1 254.353000 Lov 260.300000
> >
> > Latin1 26.000000 Latin2 36.000000 LatSP 0.000000 LonSP
0.000000
> >
> > North Pole (600 x 600) Dx 2.000000 Dy 2.000000 scan 64 mode
136
> >
> > min/max data -509.727 0.273438 num bits 9 BDS_Ref -509.727
DecScale
> 0
> > BinScale 0
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Question about using MET with grid-stat, copygb
From: Berman, Jeremy D
Time: Wed Nov 06 19:48:45 2019
Great, thank you John for the insight and tips! This is extremely
helpful and satisfied my question. Thanks!
Best wishes,
Jeremy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, November 5, 2019 6:01 PM
To: Berman, Jeremy D <jdberman at albany.edu>
Subject: Re: [rt.rap.ucar.edu #92934] Question about using MET with
grid-stat, copygb
Jeremy,
Great, I'm glad to hear that you were able to make progress.
As to your question on verification grids, there is no obvious right
or
wrong answer. Generally, I do think it's best practice to pick a
single
domain on which to perform the analysis. Whether that grid is the 2km
WRF
domain, the 3km HRRR domain, or some other domain, I'm not sure. My
recommendation on using the 2km WRF domain was just to make it as easy
as
possible for you. Since the WRF domain is a subset of the HRRR
domain,
doing the vx on that domain would result in the same number of grid
points
being used... without needing to define a polyline.
So another option would be verifying on the 3km HRRR domain, using the
polyline region to make sure you're using the same set of grid points
in
the computation of statistics.
Since the HRRR domain is very dense, please be sure to run your
"mask.poly"
file through gen_vx_mask and use the NetCDF output of that in the
Grid-Stat
config file. That'll make Grid-Stat run faster without needing to
regenerate the mask every time you run Grid-Stat.
Thanks,
John
On Tue, Nov 5, 2019 at 2:43 PM Berman, Jeremy D via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92934 >
>
> Hi John,
>
> Thank you for the quick response! That regridding tool within MET is
so
> much easier to use than copygb!
>
> You're correct that the WRF domain is a subset of the HRRR domain
(just
> over California), and the WRF is 2km while HRRR is 3km. I'm also
using a
> "mask.poly" over just northern California, which is within the WRF
and HRRR
> domains. I tried out your suggested modifications and it worked!
>
> Just a follow-up question: you suggested using the 2km WRF domain as
the
> "verification domain" for regridding using the "regrid" option, so
that
> when I'm comparing WRF forecasts to HRRR analysis, it will regrid
the 3km
> HRRR analysis to 2km for that grid_stat calculation. If I'm then
computing
> HRRR forecasts compared to HRRR analysis, would I want to turn off
this
> regrid option, since they are already on the same grid? Or is it
best to
> keep it on, for comparisons? And finally, is there any benefit to
doing the
> reverse, and regrid the 2km WRF grid to the 3km HRRR analysis when
doing
> the comparison (or can you only regrid the verification grid and not
the
> model forecasts)? Just looking for the most fair option haha.
>
>
> Thanks for all your help! This helped a lot!
>
> Jeremy
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, November 4, 2019 4:40 PM
> To: Berman, Jeremy D <jdberman at albany.edu>
> Subject: Re: [rt.rap.ucar.edu #92934] Question about using MET with
> grid-stat, copygb
>
> Jeremy,
>
> Good news. There is functionality for handling this regridding
directly in
> MET without the need to regrid using copygb. And yes, Grid-Stat is
a good
> choice for this comparison.
>
> So you're comparing HRRR to HRRR-Analysis and then WRF to HRRR-
Analysis.
> And then presumably, you'd like to see how those stats compare. I
presume
> that the WRF domain is a subset of the HRRR domain. I do see that
WRF is
> 2km grid spacing while HRRR is 3km grid spacing. In order to make a
fair
> comparison, I'd recommend choosing a single domain on which you do
the
> verification.
>
> One choice would be using the 2km WRF domain. However, that is a
little
> unfair since you'd be "creating" new HRRR data when you regrid from
3 to 2
> km resolution. For the sake of this example, though, let's say you
have a
> WRF output file named "wrf.grb" that lives on that 2km CA domain.
>
> Here's the Grid-Stat config file:
>
>
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/GridStatConfig_default
>
> In there, you'll find a section named "regrid" which defines the
grid on
> which the verification should be done. In the example below, I set
> "to_grid" to the path to the GRIB file that's on the 2km WRF domain:
>
> // Verification grid
> // May be set separately in each "field" entry
> //
> regrid = {
> to_grid = "/path/to/wrf.grb";
> method = NEAREST;
> width = 1;
> vld_thresh = 0.5;
> shape = SQUARE;
> }
>
> Note that the default regridding method is nearest neighbor. You
may want
> to use bilinear interpolation (or something else) instead:
> method = BILIN; width = 2;
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
>
> On Mon, Nov 4, 2019 at 1:00 PM Berman, Jeremy D via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Mon Nov 04 13:00:13 2019: Request 92934 was acted upon.
> > Transaction: Ticket created by jdberman at albany.edu
> > Queue: met_help
> > Subject: Question about using MET with grid-stat, copygb
> > Owner: Nobody
> > Requestors: jdberman at albany.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92934 >
> >
> >
> > To Whom It May Concern,
> >
> > I'm looking to do a comparison of two forecasts to the same
analysis
> file.
> > Namely, HRRR forecast vs. HRRR analysis and WRF forecast vs. HRRR
> analysis.
> > The WRF forecasts cover just California while the HRRR
forecasts/analysis
> > is over the CONUS. The WRF files underwent UPP to GRIB1 format,
while the
> > HRRR forecasts/analysis are GRIB2 format.
> >
> > I've read that grid-stat is useful for comparing gridded forecasts
to
> > gridded observations, so I figured I might be able to use this
tool for
> my
> > comparison. I know that grid-stat requires the two files to be on
the
> same
> > grid; which explains why grid-stat worked for when I did HRRR
forecast vs
> > HRRR analysis, but not for when I did WRF forecast vs HRRR
analysis. So
> > I've been trying to figure out how to get the grid-stat to work
for WRF
> > forecasts vs HRRR analysis.
> >
> > I saw that the tool copygb can be used for regridding datasets to
be on
> > the same grid. However, when I tried using it for the HRRR
analysis to
> > convert to the grid of WRF, it did not work (produced an empty
file). I
> > presume this is because the HRRR file is GRIB2 format, and the
copygb
> only
> > works for GRIB1 formats. But when I tried it going from WRF to the
HRRR
> > analysis grid, it also did not work, which I presume is because
the WRF
> > forecast domain is a subset of HRRR's domain and thus cannot be
put onto
> > the same grid?
> >
> > I saw something about copygb2 online, which maybe is what I need
to
> > convert a GRIB2 file. I also read online about wgrib2 and the
options
> > "-set_grib_type same -new_grid" (see my test command below) but I
then
> got
> > a message about the "IPOLATES" package not being installed, which
I have
> no
> > clue about.
> >
> >
> > >> wgrib2
/shared/cbam/DATA/HRRR_SFCA/HRRR_SFCA_2019102103_f000.grb2
> > -set_grib_type same -new_grid lambert:260.300000:26.000000
> > 254.353000:600:2000 25.157000:600:2000 tr.grb2
> >
> > >> IPOLATES package is not installed
> >
> >
> > So my question which I'm seeking help for is:
> >
> > (1) Do you think the approach I'm using for comparing HRRR
forecasts vs
> > HRRR analysis and WRF forecasts vs HRRR analysis is correct to be
using
> the
> > grid-stat tool? Or should I be using a different tool? My goal is
to see
> > whether the HRRR or WRF forecasts can best predict the 80m wind
speeds
> over
> > California using the HRRR analysis as a reference.
> >
> > (2) If grid-stat is the way to go, do you know if I need to be
using
> > copygb2 or installing the IPOLATES tool, to accomplish the
required
> > regridding? Or is there a better way to go about it?
> >
> >
> > If there is a sftp or ftp portal, I can send the WRF file and
example
> HRRR
> > analysis file if needed, since they are too big to send via email.
> Although
> > if someone has an example WRF file or HRRR file they can probably
> replicate
> > what I'm doing.
> >
> > Any insights or help would be extremely helpful. Thank you in
advance for
> > your time!!
> >
> > Best,
> > Jeremy
> >
> >
> >
> >
> > P.S. Here are the grid information for HRRR analysis and my WRF
forecast,
> > in case it is useful:
> >
> > HRRR:
> >
> >
> > 132:46093348:vt=2019102103:275 mb:anl:VVEL Vertical Velocity
(Pressure)
> > [Pa/s]:
> >
> > ndata=1905141:undef=0:mean=0.00790637:min=-
0.0613893:max=0.438611
> >
> > grid_template=30:winds(grid):
> >
> > Lambert Conformal: (1799 x 1059) input WE:SN output WE:SN res 8
> >
> > Lat1 21.138123 Lon1 237.280472 LoV 262.500000
> >
> > LatD 38.500000 Latin1 38.500000 Latin2 38.500000
> >
> > LatSP 0.000000 LonSP 0.000000
> >
> > North Pole (1799 x 1059) Dx 3000.000000 m Dy 3000.000000 m mode 8
> >
> >
> > WRF:
> >
> >
> > rec 401:214487150:date 2019102912 CIN kpds5=156 kpds6=116
kpds7=46080
> > levels=(180,0) grid=255 180-0 mb above gnd 3hr fcst:
> >
> > CIN=Convective inhibition [J/kg]
> >
> > timerange 0 P1 3 P2 0 TimeU 1 nx 600 ny 600 GDS grid 3
num_in_ave 0
> > missing 0
> >
> > center 7 subcenter 0 process 125 Table 2 scan: WE:SN winds(grid)
> >
> > Lambert Conf: Lat1 25.157000 Lon1 254.353000 Lov 260.300000
> >
> > Latin1 26.000000 Latin2 36.000000 LatSP 0.000000 LonSP
0.000000
> >
> > North Pole (600 x 600) Dx 2.000000 Dy 2.000000 scan 64 mode
136
> >
> > min/max data -509.727 0.273438 num bits 9 BDS_Ref -509.727
DecScale
> 0
> > BinScale 0
> >
> >
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list