[Met_help] [rt.rap.ucar.edu #70019] History for Upscaling
Julie Prestopnik via RT
met_help at ucar.edu
Tue Jan 6 11:10:55 MST 2015
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi John. I have been reading about one of the methods of forecast verification -Neighborhood approach which is UPSCALING.
So what i understand is that the FCST and OBSNs are brought to the same resolution (Eg. in my case I have MODEL O/P at 9km and gridded OBSERVATIONS at 50km). My question is How will I go about doing that???
geeta
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Upscaling
From: Julie Prestopnik
Time: Tue Dec 09 09:11:12 2014
Hi Geeta.
I believe John is out of the office today.
The MET tools which compare gridded forecasts to gridded observations
(Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require that the
input
forecast and observation data be on the same grid. Therefore the
COPYGB
tool is very useful in preparing your gridded data for use in MET. The
COPYGB tool was developed by the NOAA Environmental Modeling Center,
is
distributed by the National Weather Service, Climate
Prediction Center, and is also distributed as part of the Unified
PostProcessor (UPP). It may be run on GRIB files to horizontally
interpolate the data from one grid to another.
There is a section in the MET Tutorial with example on how to use
copygb.
Here is the link to the METv5.0 tutorial:
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
Unfortunately, I don't have any experience with copygb, but you can
contact
wrfhelp at ucar.edu for support in using copygb. I hope this helps.
Thanks,
Julie
On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT <met_help at ucar.edu>
wrote:
>
> Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> Transaction: Ticket created by geeta124 at hotmail.com
> Queue: met_help
> Subject: Upscaling
> Owner: Nobody
> Requestors: geeta124 at hotmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
>
>
> Hi John. I have been reading about one of the methods of forecast
> verification -Neighborhood approach which is UPSCALING.
> So what i understand is that the FCST and OBSNs are brought to the
same
> resolution (Eg. in my case I have MODEL O/P at 9km and gridded
OBSERVATIONS
> at 50km). My question is How will I go about doing that???
>
> geeta
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: Upscaling
From: John Halley Gotway
Time: Tue Dec 09 09:47:21 2014
Hi Geeta,
Just wanted to chime in here about this. Julie is correct to point
out
that the forecast and observation data needs to be on the same grid
before
comparing them with the MET grid-to-grid tools. You can use copygb to
either put them both the 9km model grid or the 50km observation grid.
And yes, upscaling is one type of "neighborhood" verification method.
Generally speaking, smoother, low-resolution models perform better
than
more realistic looking, high-resolution models when judged using
traditional verification metrics, like RMSE. Smoother, low-resolution
models have fewer large errors which typically lead to better overall
RMSE
values. Upscaling is really just smoothing of the forecast field.
You'd
could apply this method in Grid-Stat by using the following in the
configuration file:
interp = {
field = FCST;
vld_thresh = 1.0;
method = UW_MEAN;
type = [
{ width = 1; },
{ width = 3; },
{ width = 5; },
{ width = 7; },
{ width = 9; }
];
};
We call this section "interp" to be consistent with the settings in
Point-Stat, but in the context of grid-to-grid comparisons, the
interpolation options are really just smoothing operators. Here, I've
said
that we should apply 5 different smoothing operators, taking the
simple
un-weighted average (UW_MEAN) around each grid point using boxes of
widths
1, 3, 5, 7, and 9. A box of width 1 really means no smoothing. A box
of
width 5 means to take the simple average of the 5x5 box = 25 grid
points.
I've specified the field as the "FCST" to only apply the smoothing to
the
forecast field, not the observation field.
Typically larger amounts of smoothing lead to better scores, such as
RMSE.
Now how you interpret these results is up to you. If you have
questions
about that part of it, I'd probably refer you to one of the
statisticians
here.
One other thing to point out is that if you turn on the NetCDF output
of
Grid-Stat, it will contain the smoothed fields.
Hope that helps.
Thanks,
John
On Tue, Dec 9, 2014 at 9:11 AM, Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
>
> Hi Geeta.
>
> I believe John is out of the office today.
>
> The MET tools which compare gridded forecasts to gridded
observations
> (Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require that the
input
> forecast and observation data be on the same grid. Therefore the
COPYGB
> tool is very useful in preparing your gridded data for use in MET.
The
> COPYGB tool was developed by the NOAA Environmental Modeling Center,
is
> distributed by the National Weather Service, Climate
> Prediction Center, and is also distributed as part of the Unified
> PostProcessor (UPP). It may be run on GRIB files to horizontally
> interpolate the data from one grid to another.
>
> There is a section in the MET Tutorial with example on how to use
copygb.
> Here is the link to the METv5.0 tutorial:
>
>
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
>
> Unfortunately, I don't have any experience with copygb, but you can
contact
> wrfhelp at ucar.edu for support in using copygb. I hope this helps.
>
> Thanks,
> Julie
>
> On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> > Transaction: Ticket created by geeta124 at hotmail.com
> > Queue: met_help
> > Subject: Upscaling
> > Owner: Nobody
> > Requestors: geeta124 at hotmail.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> >
> >
> > Hi John. I have been reading about one of the methods of forecast
> > verification -Neighborhood approach which is UPSCALING.
> > So what i understand is that the FCST and OBSNs are brought to the
same
> > resolution (Eg. in my case I have MODEL O/P at 9km and gridded
> OBSERVATIONS
> > at 50km). My question is How will I go about doing that???
> >
> > geeta
> >
>
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: Upscaling
From: Geeta Geeta
Time: Wed Dec 10 04:42:55 2014
Thanks John. I read that there are many fuzzy verification methods.
Simplest one being Upscaling. I want to know what are the other
methods available in MET..Or put it this way........how can I use
Multi-event contingency Table /intensity scale etc using MET.
Another thing, when I upscale my FCST by 25 points (5x5 grid) then the
resolution of OBS (50km) will remain different from the FCST. So how
will the grid-grid matching take place?????????????Hope U will guide
again. thanksgeeta
> Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> Date: Tue, 9 Dec 2014 09:47:22 -0700
>
> Hi Geeta,
>
> Just wanted to chime in here about this. Julie is correct to point
out
> that the forecast and observation data needs to be on the same grid
before
> comparing them with the MET grid-to-grid tools. You can use copygb
to
> either put them both the 9km model grid or the 50km observation
grid.
>
> And yes, upscaling is one type of "neighborhood" verification
method.
> Generally speaking, smoother, low-resolution models perform better
than
> more realistic looking, high-resolution models when judged using
> traditional verification metrics, like RMSE. Smoother, low-
resolution
> models have fewer large errors which typically lead to better
overall RMSE
> values. Upscaling is really just smoothing of the forecast field.
You'd
> could apply this method in Grid-Stat by using the following in the
> configuration file:
>
> interp = {
> field = FCST;
> vld_thresh = 1.0;
> method = UW_MEAN;
> type = [
> { width = 1; },
> { width = 3; },
> { width = 5; },
> { width = 7; },
> { width = 9; }
> ];
> };
>
> We call this section "interp" to be consistent with the settings in
> Point-Stat, but in the context of grid-to-grid comparisons, the
> interpolation options are really just smoothing operators. Here,
I've said
> that we should apply 5 different smoothing operators, taking the
simple
> un-weighted average (UW_MEAN) around each grid point using boxes of
widths
> 1, 3, 5, 7, and 9. A box of width 1 really means no smoothing. A
box of
> width 5 means to take the simple average of the 5x5 box = 25 grid
points.
> I've specified the field as the "FCST" to only apply the smoothing
to the
> forecast field, not the observation field.
>
> Typically larger amounts of smoothing lead to better scores, such as
RMSE.
>
> Now how you interpret these results is up to you. If you have
questions
> about that part of it, I'd probably refer you to one of the
statisticians
> here.
>
> One other thing to point out is that if you turn on the NetCDF
output of
> Grid-Stat, it will contain the smoothed fields.
>
> Hope that helps.
>
> Thanks,
> John
>
> On Tue, Dec 9, 2014 at 9:11 AM, Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> >
> > Hi Geeta.
> >
> > I believe John is out of the office today.
> >
> > The MET tools which compare gridded forecasts to gridded
observations
> > (Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require that
the input
> > forecast and observation data be on the same grid. Therefore the
COPYGB
> > tool is very useful in preparing your gridded data for use in MET.
The
> > COPYGB tool was developed by the NOAA Environmental Modeling
Center, is
> > distributed by the National Weather Service, Climate
> > Prediction Center, and is also distributed as part of the Unified
> > PostProcessor (UPP). It may be run on GRIB files to horizontally
> > interpolate the data from one grid to another.
> >
> > There is a section in the MET Tutorial with example on how to use
copygb.
> > Here is the link to the METv5.0 tutorial:
> >
> >
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
> >
> > Unfortunately, I don't have any experience with copygb, but you
can contact
> > wrfhelp at ucar.edu for support in using copygb. I hope this helps.
> >
> > Thanks,
> > Julie
> >
> > On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> > > Transaction: Ticket created by geeta124 at hotmail.com
> > > Queue: met_help
> > > Subject: Upscaling
> > > Owner: Nobody
> > > Requestors: geeta124 at hotmail.com
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > >
> > >
> > > Hi John. I have been reading about one of the methods of
forecast
> > > verification -Neighborhood approach which is UPSCALING.
> > > So what i understand is that the FCST and OBSNs are brought to
the same
> > > resolution (Eg. in my case I have MODEL O/P at 9km and gridded
> > OBSERVATIONS
> > > at 50km). My question is How will I go about doing that???
> > >
> > > geeta
> > >
> >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
>
------------------------------------------------
Subject: Upscaling
From: John Halley Gotway
Time: Wed Dec 10 09:17:33 2014
Geeta,
To be clear, the forecast and observation data passed to grid_stat
must be
on exactly the same grid. In your case, either your 9km or 50km grid.
The
statistics from grid_stat are computed over the points in that input
grid.
The "interp" options in the grid_stat configuration file really
perform a
smoothing operation. Using the interpolation method of UW_MEAN and
interpolation width of 5, the value at each grid point is replaced by
the
average value of the 5x5 box centered on that grid point. You still
end up
with data on the same grid as the input grid, but by doing this
spatial
averaging, the data is much smoother. This typically yield higher
traditional verification scores. Please just try running this and
then use
ncview to visualize the NetCDF output from grid_stat. Flip through
the
different forecast fields, and it'll be obvious that larger
interpolation
widths result in more smoothing being applied.
I don't know if that should be called "upscaling" or "smoothing" but
it's
pretty similar.
The Fractions Skill Score (FSS) is another very popular neighborhood
verification metric and is available in the NBRCNT output line from
Grid-Stat. To use it...
- turn on the nbrcnt output line type by setting it to BOTH or STAT
in the
configuration file
- edit the nbrhd section of the config file to specify what
neighborhood
widths to use. I'd suggest 3, 5, 7, 9, 11, 13 and so on.
For each raw threshold specified in the "cat_thresh" setting,
grid_stat
will compute FSS for each of the neighborhood sizes you specified.
Just
like smoothing the data, the FSS values will improve the larger the
size of
the neighborhood width.
You also asked about multi-category contingency tables. That output
is
available in grid_stat in the MCTC and MCTS output line types.
However,
there are relatively few statistics available for multi-category
contingency tables. This is not a neighborhood verification measure,
it's
a traditional method which uses exact grid point to grid point
matching.
To use it...
- turn on the mctc and mcts output line types by setting them to BOTH
or
STAT in the configuration file
- specify a list of thresholds in the cat_thresh setting in the
"fcst" and
"obs" sections which define the multi-category contingency table in
the way
you want. The thresholds must all be of the same type, meaning, for
example, you can't mix >= with <. They must all be the same type of
threshold.
Since you're applying multiple methods in grid_stat, you may find it
more
convenient to run grid_stat multiple times. For example, the
thresholds
you choose for neighborhood verification methods may differ from those
you'd choose for multi-category contingency tables. That's fine.
Just
create two separate config files and run grid_stat multiple times.
When
you do, I'd suggest setting "output_prefix" to some descriptive string
so
that you can distinguish the different output files.
Thanks,
John
On Wed, Dec 10, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
>
> Thanks John. I read that there are many fuzzy verification methods.
> Simplest one being Upscaling. I want to know what are the other
methods
> available in MET..Or put it this way........how can I use Multi-
event
> contingency Table /intensity scale etc using MET.
> Another thing, when I upscale my FCST by 25 points (5x5 grid) then
the
> resolution of OBS (50km) will remain different from the FCST. So how
will
> the grid-grid matching take place?????????????Hope U will guide
again.
> thanksgeeta
>
> > Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> > From: met_help at ucar.edu
> > To: geeta124 at hotmail.com
> > Date: Tue, 9 Dec 2014 09:47:22 -0700
> >
> > Hi Geeta,
> >
> > Just wanted to chime in here about this. Julie is correct to
point out
> > that the forecast and observation data needs to be on the same
grid
> before
> > comparing them with the MET grid-to-grid tools. You can use
copygb to
> > either put them both the 9km model grid or the 50km observation
grid.
> >
> > And yes, upscaling is one type of "neighborhood" verification
method.
> > Generally speaking, smoother, low-resolution models perform better
than
> > more realistic looking, high-resolution models when judged using
> > traditional verification metrics, like RMSE. Smoother, low-
resolution
> > models have fewer large errors which typically lead to better
overall
> RMSE
> > values. Upscaling is really just smoothing of the forecast field.
You'd
> > could apply this method in Grid-Stat by using the following in the
> > configuration file:
> >
> > interp = {
> > field = FCST;
> > vld_thresh = 1.0;
> > method = UW_MEAN;
> > type = [
> > { width = 1; },
> > { width = 3; },
> > { width = 5; },
> > { width = 7; },
> > { width = 9; }
> > ];
> > };
> >
> > We call this section "interp" to be consistent with the settings
in
> > Point-Stat, but in the context of grid-to-grid comparisons, the
> > interpolation options are really just smoothing operators. Here,
I've
> said
> > that we should apply 5 different smoothing operators, taking the
simple
> > un-weighted average (UW_MEAN) around each grid point using boxes
of
> widths
> > 1, 3, 5, 7, and 9. A box of width 1 really means no smoothing. A
box of
> > width 5 means to take the simple average of the 5x5 box = 25 grid
points.
> > I've specified the field as the "FCST" to only apply the smoothing
to the
> > forecast field, not the observation field.
> >
> > Typically larger amounts of smoothing lead to better scores, such
as
> RMSE.
> >
> > Now how you interpret these results is up to you. If you have
questions
> > about that part of it, I'd probably refer you to one of the
statisticians
> > here.
> >
> > One other thing to point out is that if you turn on the NetCDF
output of
> > Grid-Stat, it will contain the smoothed fields.
> >
> > Hope that helps.
> >
> > Thanks,
> > John
> >
> > On Tue, Dec 9, 2014 at 9:11 AM, Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > >
> > > Hi Geeta.
> > >
> > > I believe John is out of the office today.
> > >
> > > The MET tools which compare gridded forecasts to gridded
observations
> > > (Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require that
the
> input
> > > forecast and observation data be on the same grid. Therefore
the
> COPYGB
> > > tool is very useful in preparing your gridded data for use in
MET. The
> > > COPYGB tool was developed by the NOAA Environmental Modeling
Center, is
> > > distributed by the National Weather Service, Climate
> > > Prediction Center, and is also distributed as part of the
Unified
> > > PostProcessor (UPP). It may be run on GRIB files to horizontally
> > > interpolate the data from one grid to another.
> > >
> > > There is a section in the MET Tutorial with example on how to
use
> copygb.
> > > Here is the link to the METv5.0 tutorial:
> > >
> > >
>
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
> > >
> > > Unfortunately, I don't have any experience with copygb, but you
can
> contact
> > > wrfhelp at ucar.edu for support in using copygb. I hope this
helps.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> > > > Transaction: Ticket created by geeta124 at hotmail.com
> > > > Queue: met_help
> > > > Subject: Upscaling
> > > > Owner: Nobody
> > > > Requestors: geeta124 at hotmail.com
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > > >
> > > >
> > > > Hi John. I have been reading about one of the methods of
forecast
> > > > verification -Neighborhood approach which is UPSCALING.
> > > > So what i understand is that the FCST and OBSNs are brought to
the
> same
> > > > resolution (Eg. in my case I have MODEL O/P at 9km and gridded
> > > OBSERVATIONS
> > > > at 50km). My question is How will I go about doing that???
> > > >
> > > > geeta
> > > >
> > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> >
>
>
------------------------------------------------
Subject: Upscaling
From: Geeta Geeta
Time: Wed Dec 10 23:07:57 2014
Hi John.
I am now trying to run grid stat tool.
-bash-3.2$ ./test_grid_stat.sh
*** Running Grid-Stat on APCP using netCDF input for both forecast and
observation ***
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744073645754708
Forecast File: ./fcst_nc/2011060100_WRFPRS_day1_003Z.nc
Observation File: ../trmm_nc_data/02june2011.nc
Configuration File: GridStatConfig_APCP_12
NetCDF: Attribute not found
-bash-3.2$
I am sending you both the NC files + configuration file. (I copied the
2011060100_WRFPRS_day1_003Z.nc as test_fcst.nc)
But when I look at the header of nc files using the ncdump -h all
looks fine.
netcdf test {
dimensions:
lon = 53 ;
lat = 53 ;
variables:
double lon(lon) ;
lon:units = "degrees_east" ;
double lat(lat) ;
lat:units = "degrees_north" ;
float APCP_03(lat, lon) ;
APCP_03:units = "kg/m^2" ;
APCP_03:missing_value = -9999.f ;
APCP_03:long_name = "Total precipitation" ;
APCP_03:name = "APCP" ;
APCP_03:level = "A3" ;
APCP_03:grib_code = 61.f ;
APCP_03:_FillValue = -9999.f ;
APCP_03:init_time = "20110602_000000" ;
APCP_03:init_time_ut = 1306972800. ;
APCP_03:valid_time = "20110602_030000" ;
APCP_03:valid_time_ut = 1306983600. ;
APCP_03:accum_time = "030000" ;
APCP_03:accum_time_sec = 10800.f ;
// global attributes:
:FileOrigins = "File ../../../vpt/geeta/02june2011.nc
generated 20140123_163031 on host ncmr0102 by the Rscript trmm2nc.R" ;
:MET_version = "V3.0.1" ;
:Projection = "LatLon" ;
:lat_ll = "9 degrees_north" ;
:lon_ll = "74 degrees_east" ;
:delta_lat = "0.25 degrees" ;
:delta_lon = "0.25 degrees" ;
:Nlat = "53 grid_points" ;
:Nlon = "53 grid_points" ;
}
kindly suggest. what is the problem.
> Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> Date: Wed, 10 Dec 2014 09:17:33 -0700
>
> Geeta,
>
> To be clear, the forecast and observation data passed to grid_stat
must be
> on exactly the same grid. In your case, either your 9km or 50km
grid. The
> statistics from grid_stat are computed over the points in that input
grid.
>
> The "interp" options in the grid_stat configuration file really
perform a
> smoothing operation. Using the interpolation method of UW_MEAN and
> interpolation width of 5, the value at each grid point is replaced
by the
> average value of the 5x5 box centered on that grid point. You still
end up
> with data on the same grid as the input grid, but by doing this
spatial
> averaging, the data is much smoother. This typically yield higher
> traditional verification scores. Please just try running this and
then use
> ncview to visualize the NetCDF output from grid_stat. Flip through
the
> different forecast fields, and it'll be obvious that larger
interpolation
> widths result in more smoothing being applied.
>
> I don't know if that should be called "upscaling" or "smoothing" but
it's
> pretty similar.
>
> The Fractions Skill Score (FSS) is another very popular neighborhood
> verification metric and is available in the NBRCNT output line from
> Grid-Stat. To use it...
> - turn on the nbrcnt output line type by setting it to BOTH or STAT
in the
> configuration file
> - edit the nbrhd section of the config file to specify what
neighborhood
> widths to use. I'd suggest 3, 5, 7, 9, 11, 13 and so on.
>
> For each raw threshold specified in the "cat_thresh" setting,
grid_stat
> will compute FSS for each of the neighborhood sizes you specified.
Just
> like smoothing the data, the FSS values will improve the larger the
size of
> the neighborhood width.
>
> You also asked about multi-category contingency tables. That output
is
> available in grid_stat in the MCTC and MCTS output line types.
However,
> there are relatively few statistics available for multi-category
> contingency tables. This is not a neighborhood verification
measure, it's
> a traditional method which uses exact grid point to grid point
matching.
> To use it...
> - turn on the mctc and mcts output line types by setting them to
BOTH or
> STAT in the configuration file
> - specify a list of thresholds in the cat_thresh setting in the
"fcst" and
> "obs" sections which define the multi-category contingency table in
the way
> you want. The thresholds must all be of the same type, meaning, for
> example, you can't mix >= with <. They must all be the same type of
> threshold.
>
> Since you're applying multiple methods in grid_stat, you may find it
more
> convenient to run grid_stat multiple times. For example, the
thresholds
> you choose for neighborhood verification methods may differ from
those
> you'd choose for multi-category contingency tables. That's fine.
Just
> create two separate config files and run grid_stat multiple times.
When
> you do, I'd suggest setting "output_prefix" to some descriptive
string so
> that you can distinguish the different output files.
>
> Thanks,
> John
>
> On Wed, Dec 10, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> >
> > Thanks John. I read that there are many fuzzy verification
methods.
> > Simplest one being Upscaling. I want to know what are the other
methods
> > available in MET..Or put it this way........how can I use Multi-
event
> > contingency Table /intensity scale etc using MET.
> > Another thing, when I upscale my FCST by 25 points (5x5 grid) then
the
> > resolution of OBS (50km) will remain different from the FCST. So
how will
> > the grid-grid matching take place?????????????Hope U will guide
again.
> > thanksgeeta
> >
> > > Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> > > From: met_help at ucar.edu
> > > To: geeta124 at hotmail.com
> > > Date: Tue, 9 Dec 2014 09:47:22 -0700
> > >
> > > Hi Geeta,
> > >
> > > Just wanted to chime in here about this. Julie is correct to
point out
> > > that the forecast and observation data needs to be on the same
grid
> > before
> > > comparing them with the MET grid-to-grid tools. You can use
copygb to
> > > either put them both the 9km model grid or the 50km observation
grid.
> > >
> > > And yes, upscaling is one type of "neighborhood" verification
method.
> > > Generally speaking, smoother, low-resolution models perform
better than
> > > more realistic looking, high-resolution models when judged using
> > > traditional verification metrics, like RMSE. Smoother, low-
resolution
> > > models have fewer large errors which typically lead to better
overall
> > RMSE
> > > values. Upscaling is really just smoothing of the forecast
field. You'd
> > > could apply this method in Grid-Stat by using the following in
the
> > > configuration file:
> > >
> > > interp = {
> > > field = FCST;
> > > vld_thresh = 1.0;
> > > method = UW_MEAN;
> > > type = [
> > > { width = 1; },
> > > { width = 3; },
> > > { width = 5; },
> > > { width = 7; },
> > > { width = 9; }
> > > ];
> > > };
> > >
> > > We call this section "interp" to be consistent with the settings
in
> > > Point-Stat, but in the context of grid-to-grid comparisons, the
> > > interpolation options are really just smoothing operators.
Here, I've
> > said
> > > that we should apply 5 different smoothing operators, taking the
simple
> > > un-weighted average (UW_MEAN) around each grid point using boxes
of
> > widths
> > > 1, 3, 5, 7, and 9. A box of width 1 really means no smoothing.
A box of
> > > width 5 means to take the simple average of the 5x5 box = 25
grid points.
> > > I've specified the field as the "FCST" to only apply the
smoothing to the
> > > forecast field, not the observation field.
> > >
> > > Typically larger amounts of smoothing lead to better scores,
such as
> > RMSE.
> > >
> > > Now how you interpret these results is up to you. If you have
questions
> > > about that part of it, I'd probably refer you to one of the
statisticians
> > > here.
> > >
> > > One other thing to point out is that if you turn on the NetCDF
output of
> > > Grid-Stat, it will contain the smoothed fields.
> > >
> > > Hope that helps.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Dec 9, 2014 at 9:11 AM, Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019
>
> > > >
> > > > Hi Geeta.
> > > >
> > > > I believe John is out of the office today.
> > > >
> > > > The MET tools which compare gridded forecasts to gridded
observations
> > > > (Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require
that the
> > input
> > > > forecast and observation data be on the same grid. Therefore
the
> > COPYGB
> > > > tool is very useful in preparing your gridded data for use in
MET. The
> > > > COPYGB tool was developed by the NOAA Environmental Modeling
Center, is
> > > > distributed by the National Weather Service, Climate
> > > > Prediction Center, and is also distributed as part of the
Unified
> > > > PostProcessor (UPP). It may be run on GRIB files to
horizontally
> > > > interpolate the data from one grid to another.
> > > >
> > > > There is a section in the MET Tutorial with example on how to
use
> > copygb.
> > > > Here is the link to the METv5.0 tutorial:
> > > >
> > > >
> >
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
> > > >
> > > > Unfortunately, I don't have any experience with copygb, but
you can
> > contact
> > > > wrfhelp at ucar.edu for support in using copygb. I hope this
helps.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> > > > > Transaction: Ticket created by geeta124 at hotmail.com
> > > > > Queue: met_help
> > > > > Subject: Upscaling
> > > > > Owner: Nobody
> > > > > Requestors: geeta124 at hotmail.com
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > > > >
> > > > >
> > > > > Hi John. I have been reading about one of the methods of
forecast
> > > > > verification -Neighborhood approach which is UPSCALING.
> > > > > So what i understand is that the FCST and OBSNs are brought
to the
> > same
> > > > > resolution (Eg. in my case I have MODEL O/P at 9km and
gridded
> > > > OBSERVATIONS
> > > > > at 50km). My question is How will I go about doing that???
> > > > >
> > > > > geeta
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > >
> >
> >
>
------------------------------------------------
Subject: Upscaling
From: Geeta Geeta
Time: Thu Dec 11 22:14:29 2014
Hi John.
A. I have a couple of questions.
It is written in the MET Mannual that there are 3 approaches to
verification.
1 Treating the variables as continuous variable,
2. Categorical
3. Probabilistic.
I understand the 1 and 2 give me CNT an CTC statistics (MAE, RMSE,
POD, FAR etc).
In my case, I verified the model o/p with point observations
(deterministic FCST). I computed the CNT & CTC scores.
Now I wish to know some more details about the probabilistic FCST.
DOES it refer to the FCST by EPS???
IN which cases, the probabilistic FCST is valid. I mean when are we
computing these scores. ???. In which case these will be valid? I mean
How are we treating the FCST as probabilistic???
B. I have another question pertaining to the ATTRIBUTES of FCST"". I
wish to know what are the measures of Reliability and Resolution and
Uncertainity of the model.
How Can I calculate them. ???
I have GRIDDED an POINT OBSERVATIONS as the TrUTH data sets.
C. PLs solve my problem reg the running of GRID STAT. I have sent U my
datasets.
Kindly guide me. I shall be looking for Ur response eagerly.
Thanks
Geeta
From: geeta124 at hotmail.com
To: met_help at ucar.edu
Subject: RE: [rt.rap.ucar.edu #70019] Upscaling
Date: Thu, 11 Dec 2014 11:37:53 +0530
Hi John.
I am now trying to run grid stat tool.
-bash-3.2$ ./test_grid_stat.sh
*** Running Grid-Stat on APCP using netCDF input for both forecast and
observation ***
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744073645754708
Forecast File: ./fcst_nc/2011060100_WRFPRS_day1_003Z.nc
Observation File: ../trmm_nc_data/02june2011.nc
Configuration File: GridStatConfig_APCP_12
NetCDF: Attribute not found
-bash-3.2$
I am sending you both the NC files + configuration file. (I copied the
2011060100_WRFPRS_day1_003Z.nc as test_fcst.nc)
But when I look at the header of nc files using the ncdump -h all
looks fine.
netcdf test {
dimensions:
lon = 53 ;
lat = 53 ;
variables:
double lon(lon) ;
lon:units = "degrees_east" ;
double lat(lat) ;
lat:units = "degrees_north" ;
float APCP_03(lat, lon) ;
APCP_03:units = "kg/m^2" ;
APCP_03:missing_value = -9999.f ;
APCP_03:long_name = "Total precipitation" ;
APCP_03:name = "APCP" ;
APCP_03:level = "A3" ;
APCP_03:grib_code = 61.f ;
APCP_03:_FillValue = -9999.f ;
APCP_03:init_time = "20110602_000000" ;
APCP_03:init_time_ut = 1306972800. ;
APCP_03:valid_time = "20110602_030000" ;
APCP_03:valid_time_ut = 1306983600. ;
APCP_03:accum_time = "030000" ;
APCP_03:accum_time_sec = 10800.f ;
// global attributes:
:FileOrigins = "File ../../../vpt/geeta/02june2011.nc
generated 20140123_163031 on host ncmr0102 by the Rscript trmm2nc.R" ;
:MET_version = "V3.0.1" ;
:Projection = "LatLon" ;
:lat_ll = "9 degrees_north" ;
:lon_ll = "74 degrees_east" ;
:delta_lat = "0.25 degrees" ;
:delta_lon = "0.25 degrees" ;
:Nlat = "53 grid_points" ;
:Nlon = "53 grid_points" ;
}
kindly suggest. what is the problem.
> Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> Date: Wed, 10 Dec 2014 09:17:33 -0700
>
> Geeta,
>
> To be clear, the forecast and observation data passed to grid_stat
must be
> on exactly the same grid. In your case, either your 9km or 50km
grid. The
> statistics from grid_stat are computed over the points in that input
grid.
>
> The "interp" options in the grid_stat configuration file really
perform a
> smoothing operation. Using the interpolation method of UW_MEAN and
> interpolation width of 5, the value at each grid point is replaced
by the
> average value of the 5x5 box centered on that grid point. You still
end up
> with data on the same grid as the input grid, but by doing this
spatial
> averaging, the data is much smoother. This typically yield higher
> traditional verification scores. Please just try running this and
then use
> ncview to visualize the NetCDF output from grid_stat. Flip through
the
> different forecast fields, and it'll be obvious that larger
interpolation
> widths result in more smoothing being applied.
>
> I don't know if that should be called "upscaling" or "smoothing" but
it's
> pretty similar.
>
> The Fractions Skill Score (FSS) is another very popular neighborhood
> verification metric and is available in the NBRCNT output line from
> Grid-Stat. To use it...
> - turn on the nbrcnt output line type by setting it to BOTH or STAT
in the
> configuration file
> - edit the nbrhd section of the config file to specify what
neighborhood
> widths to use. I'd suggest 3, 5, 7, 9, 11, 13 and so on.
>
> For each raw threshold specified in the "cat_thresh" setting,
grid_stat
> will compute FSS for each of the neighborhood sizes you specified.
Just
> like smoothing the data, the FSS values will improve the larger the
size of
> the neighborhood width.
>
> You also asked about multi-category contingency tables. That output
is
> available in grid_stat in the MCTC and MCTS output line types.
However,
> there are relatively few statistics available for multi-category
> contingency tables. This is not a neighborhood verification
measure, it's
> a traditional method which uses exact grid point to grid point
matching.
> To use it...
> - turn on the mctc and mcts output line types by setting them to
BOTH or
> STAT in the configuration file
> - specify a list of thresholds in the cat_thresh setting in the
"fcst" and
> "obs" sections which define the multi-category contingency table in
the way
> you want. The thresholds must all be of the same type, meaning, for
> example, you can't mix >= with <. They must all be the same type of
> threshold.
>
> Since you're applying multiple methods in grid_stat, you may find it
more
> convenient to run grid_stat multiple times. For example, the
thresholds
> you choose for neighborhood verification methods may differ from
those
> you'd choose for multi-category contingency tables. That's fine.
Just
> create two separate config files and run grid_stat multiple times.
When
> you do, I'd suggest setting "output_prefix" to some descriptive
string so
> that you can distinguish the different output files.
>
> Thanks,
> John
>
> On Wed, Dec 10, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> >
> > Thanks John. I read that there are many fuzzy verification
methods.
> > Simplest one being Upscaling. I want to know what are the other
methods
> > available in MET..Or put it this way........how can I use Multi-
event
> > contingency Table /intensity scale etc using MET.
> > Another thing, when I upscale my FCST by 25 points (5x5 grid) then
the
> > resolution of OBS (50km) will remain different from the FCST. So
how will
> > the grid-grid matching take place?????????????Hope U will guide
again.
> > thanksgeeta
> >
> > > Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> > > From: met_help at ucar.edu
> > > To: geeta124 at hotmail.com
> > > Date: Tue, 9 Dec 2014 09:47:22 -0700
> > >
> > > Hi Geeta,
> > >
> > > Just wanted to chime in here about this. Julie is correct to
point out
> > > that the forecast and observation data needs to be on the same
grid
> > before
> > > comparing them with the MET grid-to-grid tools. You can use
copygb to
> > > either put them both the 9km model grid or the 50km observation
grid.
> > >
> > > And yes, upscaling is one type of "neighborhood" verification
method.
> > > Generally speaking, smoother, low-resolution models perform
better than
> > > more realistic looking, high-resolution models when judged using
> > > traditional verification metrics, like RMSE. Smoother, low-
resolution
> > > models have fewer large errors which typically lead to better
overall
> > RMSE
> > > values. Upscaling is really just smoothing of the forecast
field. You'd
> > > could apply this method in Grid-Stat by using the following in
the
> > > configuration file:
> > >
> > > interp = {
> > > field = FCST;
> > > vld_thresh = 1.0;
> > > method = UW_MEAN;
> > > type = [
> > > { width = 1; },
> > > { width = 3; },
> > > { width = 5; },
> > > { width = 7; },
> > > { width = 9; }
> > > ];
> > > };
> > >
> > > We call this section "interp" to be consistent with the settings
in
> > > Point-Stat, but in the context of grid-to-grid comparisons, the
> > > interpolation options are really just smoothing operators.
Here, I've
> > said
> > > that we should apply 5 different smoothing operators, taking the
simple
> > > un-weighted average (UW_MEAN) around each grid point using boxes
of
> > widths
> > > 1, 3, 5, 7, and 9. A box of width 1 really means no smoothing.
A box of
> > > width 5 means to take the simple average of the 5x5 box = 25
grid points.
> > > I've specified the field as the "FCST" to only apply the
smoothing to the
> > > forecast field, not the observation field.
> > >
> > > Typically larger amounts of smoothing lead to better scores,
such as
> > RMSE.
> > >
> > > Now how you interpret these results is up to you. If you have
questions
> > > about that part of it, I'd probably refer you to one of the
statisticians
> > > here.
> > >
> > > One other thing to point out is that if you turn on the NetCDF
output of
> > > Grid-Stat, it will contain the smoothed fields.
> > >
> > > Hope that helps.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Dec 9, 2014 at 9:11 AM, Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019
>
> > > >
> > > > Hi Geeta.
> > > >
> > > > I believe John is out of the office today.
> > > >
> > > > The MET tools which compare gridded forecasts to gridded
observations
> > > > (Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require
that the
> > input
> > > > forecast and observation data be on the same grid. Therefore
the
> > COPYGB
> > > > tool is very useful in preparing your gridded data for use in
MET. The
> > > > COPYGB tool was developed by the NOAA Environmental Modeling
Center, is
> > > > distributed by the National Weather Service, Climate
> > > > Prediction Center, and is also distributed as part of the
Unified
> > > > PostProcessor (UPP). It may be run on GRIB files to
horizontally
> > > > interpolate the data from one grid to another.
> > > >
> > > > There is a section in the MET Tutorial with example on how to
use
> > copygb.
> > > > Here is the link to the METv5.0 tutorial:
> > > >
> > > >
> >
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
> > > >
> > > > Unfortunately, I don't have any experience with copygb, but
you can
> > contact
> > > > wrfhelp at ucar.edu for support in using copygb. I hope this
helps.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> > > > > Transaction: Ticket created by geeta124 at hotmail.com
> > > > > Queue: met_help
> > > > > Subject: Upscaling
> > > > > Owner: Nobody
> > > > > Requestors: geeta124 at hotmail.com
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > > > >
> > > > >
> > > > > Hi John. I have been reading about one of the methods of
forecast
> > > > > verification -Neighborhood approach which is UPSCALING.
> > > > > So what i understand is that the FCST and OBSNs are brought
to the
> > same
> > > > > resolution (Eg. in my case I have MODEL O/P at 9km and
gridded
> > > > OBSERVATIONS
> > > > > at 50km). My question is How will I go about doing that???
> > > > >
> > > > > geeta
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > >
> >
> >
>
------------------------------------------------
Subject: Upscaling
From: John Halley Gotway
Time: Fri Dec 12 15:01:08 2014
Geeta,
I tried running the NetCDF files you sent through grid_stat. Looking
at
the config file, I see that you're still using MET version 3.0. When
I run
with METv3.0, including all posted bugfixes for that build, I get the
following error:
ERROR: parse_poly_mask() -> the dimensions of the masking region
(185,
129) must match the dimensions of the data (53, 53).
So I'm not able to reproduce the behavior you're describing. If you'd
like
me to look more closely, I'll need to know the exact version of MET
that
you're running.
Also, I'd strongly recommend upgrading to the latest version, version
5.0.
There have been many bugfixes and enhancements since METv3.0.
I see that you understand that grid_stat can be used to evaluate
scalar
fields using both continuous and categorical statistics. The tool
also has
methods for computing probabilistic results, but only when you have
probabilistic fields to evaluate. MET does not create probability
fields
for you, but if you have a probabilistic data from some other source,
you
can use MET to evaluate it. One common type of probabilistic field is
the
"probability of precipitation".
You say that you have sent me your datasets. You sent me two NetCDF
files,
"test_fcst.nc" and "test.nc". Both appear to contain 24-hour
accumulated
precipitation and are the output of the pcp_combine tool. Neither of
those
is a probability field, so probabilistic evaluation would not apply.
Thanks,
John
On Thu, Dec 11, 2014 at 10:14 PM, Geeta Geeta via RT
<met_help at ucar.edu>
wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
>
> Hi John.
> A. I have a couple of questions.
> It is written in the MET Mannual that there are 3 approaches to
> verification.
> 1 Treating the variables as continuous variable,
> 2. Categorical
> 3. Probabilistic.
> I understand the 1 and 2 give me CNT an CTC statistics (MAE, RMSE,
POD,
> FAR etc).
> In my case, I verified the model o/p with point observations
> (deterministic FCST). I computed the CNT & CTC scores.
> Now I wish to know some more details about the probabilistic FCST.
> DOES it refer to the FCST by EPS???
> IN which cases, the probabilistic FCST is valid. I mean when are we
> computing these scores. ???. In which case these will be valid? I
mean How
> are we treating the FCST as probabilistic???
>
> B. I have another question pertaining to the ATTRIBUTES of FCST"". I
wish
> to know what are the measures of Reliability and Resolution and
> Uncertainity of the model.
> How Can I calculate them. ???
> I have GRIDDED an POINT OBSERVATIONS as the TrUTH data sets.
>
> C. PLs solve my problem reg the running of GRID STAT. I have sent U
my
> datasets.
>
> Kindly guide me. I shall be looking for Ur response eagerly.
> Thanks
> Geeta
>
> From: geeta124 at hotmail.com
> To: met_help at ucar.edu
> Subject: RE: [rt.rap.ucar.edu #70019] Upscaling
> Date: Thu, 11 Dec 2014 11:37:53 +0530
>
>
>
>
> Hi John.
> I am now trying to run grid stat tool.
> -bash-3.2$ ./test_grid_stat.sh
> *** Running Grid-Stat on APCP using netCDF input for both forecast
and
> observation ***
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744073645754708
> Forecast File: ./fcst_nc/2011060100_WRFPRS_day1_003Z.nc
> Observation File: ../trmm_nc_data/02june2011.nc
> Configuration File: GridStatConfig_APCP_12
> NetCDF: Attribute not found
> -bash-3.2$
> I am sending you both the NC files + configuration file. (I copied
the
> 2011060100_WRFPRS_day1_003Z.nc as test_fcst.nc)
>
> But when I look at the header of nc files using the ncdump -h all
looks
> fine.
> netcdf test {
> dimensions:
> lon = 53 ;
> lat = 53 ;
> variables:
> double lon(lon) ;
> lon:units = "degrees_east" ;
> double lat(lat) ;
> lat:units = "degrees_north" ;
> float APCP_03(lat, lon) ;
> APCP_03:units = "kg/m^2" ;
> APCP_03:missing_value = -9999.f ;
> APCP_03:long_name = "Total precipitation" ;
> APCP_03:name = "APCP" ;
> APCP_03:level = "A3" ;
> APCP_03:grib_code = 61.f ;
> APCP_03:_FillValue = -9999.f ;
> APCP_03:init_time = "20110602_000000" ;
> APCP_03:init_time_ut = 1306972800. ;
> APCP_03:valid_time = "20110602_030000" ;
> APCP_03:valid_time_ut = 1306983600. ;
> APCP_03:accum_time = "030000" ;
> APCP_03:accum_time_sec = 10800.f ;
>
> // global attributes:
> :FileOrigins = "File
../../../vpt/geeta/02june2011.nc
> generated 20140123_163031 on host ncmr0102 by the Rscript trmm2nc.R"
;
> :MET_version = "V3.0.1" ;
> :Projection = "LatLon" ;
> :lat_ll = "9 degrees_north" ;
> :lon_ll = "74 degrees_east" ;
> :delta_lat = "0.25 degrees" ;
> :delta_lon = "0.25 degrees" ;
> :Nlat = "53 grid_points" ;
> :Nlon = "53 grid_points" ;
> }
> kindly suggest. what is the problem.
>
> > Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> > From: met_help at ucar.edu
> > To: geeta124 at hotmail.com
> > Date: Wed, 10 Dec 2014 09:17:33 -0700
> >
> > Geeta,
> >
> > To be clear, the forecast and observation data passed to grid_stat
must
> be
> > on exactly the same grid. In your case, either your 9km or 50km
grid.
> The
> > statistics from grid_stat are computed over the points in that
input
> grid.
> >
> > The "interp" options in the grid_stat configuration file really
perform a
> > smoothing operation. Using the interpolation method of UW_MEAN
and
> > interpolation width of 5, the value at each grid point is replaced
by the
> > average value of the 5x5 box centered on that grid point. You
still end
> up
> > with data on the same grid as the input grid, but by doing this
spatial
> > averaging, the data is much smoother. This typically yield higher
> > traditional verification scores. Please just try running this and
then
> use
> > ncview to visualize the NetCDF output from grid_stat. Flip
through the
> > different forecast fields, and it'll be obvious that larger
interpolation
> > widths result in more smoothing being applied.
> >
> > I don't know if that should be called "upscaling" or "smoothing"
but it's
> > pretty similar.
> >
> > The Fractions Skill Score (FSS) is another very popular
neighborhood
> > verification metric and is available in the NBRCNT output line
from
> > Grid-Stat. To use it...
> > - turn on the nbrcnt output line type by setting it to BOTH or
STAT in
> the
> > configuration file
> > - edit the nbrhd section of the config file to specify what
neighborhood
> > widths to use. I'd suggest 3, 5, 7, 9, 11, 13 and so on.
> >
> > For each raw threshold specified in the "cat_thresh" setting,
grid_stat
> > will compute FSS for each of the neighborhood sizes you specified.
Just
> > like smoothing the data, the FSS values will improve the larger
the size
> of
> > the neighborhood width.
> >
> > You also asked about multi-category contingency tables. That
output is
> > available in grid_stat in the MCTC and MCTS output line types.
However,
> > there are relatively few statistics available for multi-category
> > contingency tables. This is not a neighborhood verification
measure,
> it's
> > a traditional method which uses exact grid point to grid point
matching.
> > To use it...
> > - turn on the mctc and mcts output line types by setting them to
BOTH or
> > STAT in the configuration file
> > - specify a list of thresholds in the cat_thresh setting in the
"fcst"
> and
> > "obs" sections which define the multi-category contingency table
in the
> way
> > you want. The thresholds must all be of the same type, meaning,
for
> > example, you can't mix >= with <. They must all be the same type
of
> > threshold.
> >
> > Since you're applying multiple methods in grid_stat, you may find
it more
> > convenient to run grid_stat multiple times. For example, the
thresholds
> > you choose for neighborhood verification methods may differ from
those
> > you'd choose for multi-category contingency tables. That's fine.
Just
> > create two separate config files and run grid_stat multiple times.
When
> > you do, I'd suggest setting "output_prefix" to some descriptive
string so
> > that you can distinguish the different output files.
> >
> > Thanks,
> > John
> >
> > On Wed, Dec 10, 2014 at 4:42 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > >
> > > Thanks John. I read that there are many fuzzy verification
methods.
> > > Simplest one being Upscaling. I want to know what are the other
methods
> > > available in MET..Or put it this way........how can I use Multi-
event
> > > contingency Table /intensity scale etc using MET.
> > > Another thing, when I upscale my FCST by 25 points (5x5 grid)
then the
> > > resolution of OBS (50km) will remain different from the FCST. So
how
> will
> > > the grid-grid matching take place?????????????Hope U will guide
again.
> > > thanksgeeta
> > >
> > > > Subject: Re: [rt.rap.ucar.edu #70019] Upscaling
> > > > From: met_help at ucar.edu
> > > > To: geeta124 at hotmail.com
> > > > Date: Tue, 9 Dec 2014 09:47:22 -0700
> > > >
> > > > Hi Geeta,
> > > >
> > > > Just wanted to chime in here about this. Julie is correct to
point
> out
> > > > that the forecast and observation data needs to be on the same
grid
> > > before
> > > > comparing them with the MET grid-to-grid tools. You can use
copygb
> to
> > > > either put them both the 9km model grid or the 50km
observation grid.
> > > >
> > > > And yes, upscaling is one type of "neighborhood" verification
method.
> > > > Generally speaking, smoother, low-resolution models perform
better
> than
> > > > more realistic looking, high-resolution models when judged
using
> > > > traditional verification metrics, like RMSE. Smoother,
> low-resolution
> > > > models have fewer large errors which typically lead to better
overall
> > > RMSE
> > > > values. Upscaling is really just smoothing of the forecast
field.
> You'd
> > > > could apply this method in Grid-Stat by using the following in
the
> > > > configuration file:
> > > >
> > > > interp = {
> > > > field = FCST;
> > > > vld_thresh = 1.0;
> > > > method = UW_MEAN;
> > > > type = [
> > > > { width = 1; },
> > > > { width = 3; },
> > > > { width = 5; },
> > > > { width = 7; },
> > > > { width = 9; }
> > > > ];
> > > > };
> > > >
> > > > We call this section "interp" to be consistent with the
settings in
> > > > Point-Stat, but in the context of grid-to-grid comparisons,
the
> > > > interpolation options are really just smoothing operators.
Here,
> I've
> > > said
> > > > that we should apply 5 different smoothing operators, taking
the
> simple
> > > > un-weighted average (UW_MEAN) around each grid point using
boxes of
> > > widths
> > > > 1, 3, 5, 7, and 9. A box of width 1 really means no
smoothing. A
> box of
> > > > width 5 means to take the simple average of the 5x5 box = 25
grid
> points.
> > > > I've specified the field as the "FCST" to only apply the
smoothing
> to the
> > > > forecast field, not the observation field.
> > > >
> > > > Typically larger amounts of smoothing lead to better scores,
such as
> > > RMSE.
> > > >
> > > > Now how you interpret these results is up to you. If you have
> questions
> > > > about that part of it, I'd probably refer you to one of the
> statisticians
> > > > here.
> > > >
> > > > One other thing to point out is that if you turn on the NetCDF
> output of
> > > > Grid-Stat, it will contain the smoothed fields.
> > > >
> > > > Hope that helps.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Dec 9, 2014 at 9:11 AM, Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > > > >
> > > > > Hi Geeta.
> > > > >
> > > > > I believe John is out of the office today.
> > > > >
> > > > > The MET tools which compare gridded forecasts to gridded
> observations
> > > > > (Grid-Stat, Wavelet-Stat, Ensemble-Stat and, MODE) require
that the
> > > input
> > > > > forecast and observation data be on the same grid.
Therefore the
> > > COPYGB
> > > > > tool is very useful in preparing your gridded data for use
in MET.
> The
> > > > > COPYGB tool was developed by the NOAA Environmental Modeling
> Center, is
> > > > > distributed by the National Weather Service, Climate
> > > > > Prediction Center, and is also distributed as part of the
Unified
> > > > > PostProcessor (UPP). It may be run on GRIB files to
horizontally
> > > > > interpolate the data from one grid to another.
> > > > >
> > > > > There is a section in the MET Tutorial with example on how
to use
> > > copygb.
> > > > > Here is the link to the METv5.0 tutorial:
> > > > >
> > > > >
> > >
>
http://www.dtcenter.org/met/users/support/online_tutorial/METv5.0/copygb/index.php
> > > > >
> > > > > Unfortunately, I don't have any experience with copygb, but
you can
> > > contact
> > > > > wrfhelp at ucar.edu for support in using copygb. I hope this
helps.
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Tue, Dec 9, 2014 at 4:42 AM, Geeta Geeta via RT <
> met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Tue Dec 09 04:42:54 2014: Request 70019 was acted upon.
> > > > > > Transaction: Ticket created by geeta124 at hotmail.com
> > > > > > Queue: met_help
> > > > > > Subject: Upscaling
> > > > > > Owner: Nobody
> > > > > > Requestors: geeta124 at hotmail.com
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70019 >
> > > > > >
> > > > > >
> > > > > > Hi John. I have been reading about one of the methods of
forecast
> > > > > > verification -Neighborhood approach which is UPSCALING.
> > > > > > So what i understand is that the FCST and OBSNs are
brought to
> the
> > > same
> > > > > > resolution (Eg. in my case I have MODEL O/P at 9km and
gridded
> > > > > OBSERVATIONS
> > > > > > at 50km). My question is How will I go about doing
that???
> > > > > >
> > > > > > geeta
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list