[Met_help] [rt.rap.ucar.edu #79255] History for Point Stat Anomaly Correlation
John Halley Gotway via RT
met_help at ucar.edu
Wed Jan 25 13:32:09 MST 2017
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John or Julie, I am setting up MET to calculate 500mb HGT anomaly correlation. I will be using prep buffer for the observations. The question I have is what format does the climatology file have to be in? I have month specific climo files from NCEP in binary format but not sure what MET expects.
Thanks
Bob Craig
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Point Stat Anomaly Correlation
From: Julie Prestopnik
Time: Tue Jan 24 14:46:23 2017
Hi Bob.
Prior to the calculation of statistics, the climatology file must be
put on
the same grid as the forecasts and observations.
I hope that helps.
Julie
On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Point Stat Anomaly Correlation
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
>
> John or Julie, I am setting up MET to calculate 500mb HGT anomaly
> correlation. I will be using prep buffer for the observations. The
> question I have is what format does the climatology file have to be
in? I
> have month specific climo files from NCEP in binary format but not
sure
> what MET expects.
>
> Thanks
> Bob Craig
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: Point Stat Anomaly Correlation
From: John Halley Gotway
Time: Tue Jan 24 15:46:26 2017
Hi Bob,
This is John. The climatology file is any gridded data file that MET
supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After the
addition of the automated regridding feature in MET, you don't
actually
need to put it on your model domain ahead of time. Instead, you can
use
the "climo_mean -> regrid" config file option to specify how you want
to
regrid the climatology data to your model domain.
I see from your message that you have monthly binary data from NCEP.
At
the DTC, we have 2 variants of climatology data from NCEP, the
operational
monthly 2.5 degree data and an experimental daily 1.0 degree dataset.
My
guess is that you have the 2.5 degree data. We have that in both
binary
and GRIB1 format. MET can handle the GRIB version... not the binary
one.
If you'd like, I can send you that GRIB data.
But there actually is a pretty big difference between the binary and
GRIB
datasets. The GRIB files contain the climatological mean and spread
at
each grid point. Instead of mean/spread, the binary files contain 101
values for each grid point representing the distribution. I believe
NCEP
uses percentiles from the distribution in the computation of
statistics.
We have an open development task for MET version 6.0 to mimic NCEP's
use of
climatological data and will likely need to handle those binary files.
But MET version 5.2 only supports using the mean climatology value
read
from a gridded data file.
Thanks,
John
On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hi Bob.
>
> Prior to the calculation of statistics, the climatology file must be
put on
> the same grid as the forecasts and observations.
>
> I hope that helps.
>
> Julie
>
> On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Point Stat Anomaly Correlation
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> >
> > John or Julie, I am setting up MET to calculate 500mb HGT anomaly
> > correlation. I will be using prep buffer for the observations.
The
> > question I have is what format does the climatology file have to
be in?
> I
> > have month specific climo files from NCEP in binary format but not
sure
> > what MET expects.
> >
> > Thanks
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
From: robert.craig.2 at us.af.mil
Time: Wed Jan 25 07:17:45 2017
Thanks for the info. I found out that I have grib climo files from
ECMWF and I will use those. One question: I am having some
difficulty getting MET to recognize the climo grib parameter.
Below is an exert from my config file:
//
// Forecast and observation fields to be verified
//
fcst = {
message_type = [ "ADPUPA" ];
sid_exc = [];
field = [
{
name = "HGT";
level = [ "P500" ];
cat_thresh = [ ];
}
];
}
obs = fcst;
////////////////////////////////////////////////////////////////////////////////
//
// Climatology mean data
//
climo_mean = {
file_name =
["/h/data/global/WXQC/data/climo/ecmwf/z500hPa_mean.grib"];
field = [
{
GRIB1_ptv = 3;
GRIB1_rec = 730;
level = [ "P500" ];
cat_thresh = [ ];
}
];
regrid = {
to_grid = FCST;
vld_thresh = 0.5;
method = NEAREST;
width = 1;
}
time_interp_method = DW_MEAN;
match_day = TRUE;
time_step = 43200;
}
Below is the data grib header:
rec 730:42401556:date 25655123012 GP kpds5=6 kpds6=100 kpds7=500
levels=(1,244) grid=255 500 mb anl:
GP=Geopotential [m^2/s^2]
timerange 0 P1 0 P2 0 TimeU 1 nx 240 ny 121 GDS grid 0 num_in_ave
1220 missing 0
center 98 subcenter 0 process 128 Table 3 scan: WE:NS winds(N/S)
latlon: lat 90.000000 to -90.000000 by 1.500000 nxny 29040
long 0.000000 to 358.500000 by 1.500000, (240 x 121) scan 0
mode 128 bdsgrid 1
min/max data 49246.6 57740.3 num bits 16 BDS_Ref 49246.6 DecScale
0 BinScale -2
rec 731:42459720:date 25655123100 GP kpds5=6 kpds6=100 kpds7=500
levels=(1,244) grid=255 500 mb anl:
GP=Geopotential [m^2/s^2]
timerange 0 P1 0 P2 0 TimeU 1 nx 240 ny 121 GDS grid 0 num_in_ave
1220 missing 0
center 98 subcenter 0 process 128 Table 3 scan: WE:NS winds(N/S)
latlon: lat 90.000000 to -90.000000 by 1.500000 nxny 29040
long 0.000000 to 358.500000 by 1.500000, (240 x 121) scan 0
mode 128 bdsgrid 1
min/max data 49252.4 57768.1 num bits 16 BDS_Ref 49252.4 DecScale
0 BinScale -2
rec 732:42517884:date 25655123112 GP kpds5=6 kpds6=100 kpds7=500
levels=(1,244) grid=255 500 mb anl:
GP=Geopotential [m^2/s^2]
timerange 0 P1 0 P2 0 TimeU 1 nx 240 ny 121 GDS grid 0 num_in_ave
1220 missing 0
center 98 subcenter 0 process 128 Table 3 scan: WE:NS winds(N/S)
latlon: lat 90.000000 to -90.000000 by 1.500000 nxny 29040
long 0.000000 to 358.500000 by 1.500000, (240 x 121) scan 0
mode 128 bdsgrid 1
min/max data 49230.2 57738.2 num bits 16 BDS_Ref 49230.2 DecScale
0 BinScale -2
I figure I will have to tell MET which grib record to use since the
records are for each 12hr period during a year. So I tried setting
the GRIB1_rec and GRIB1_ptv values but get the following error:
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=598837336
DEBUG 1: Forecast File: /h/data/global/WXQC/data/umm/1701181224
DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2017011912
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for HGT/P500.
DEBUG 3: MetGrib2DataFile::data_plane_array() - Found exact match for
"HGT/P500" in GRIB2 record 69 field 1 of GRIB2 file
"/h/data/global/WXQC/data/umm/1701181224"
DEBUG 4:
DEBUG 4: Data plane information:
DEBUG 4: plane min: 4682
DEBUG 4: plane max: 5947
DEBUG 4: scan mode: 64
DEBUG 4: valid time: 20170119_120000
DEBUG 4: lead time: 240000
DEBUG 4: init time: 20170118_120000
DEBUG 4: bitmap flag: 255
DEBUG 4:
DEBUG 3: Use the matching forecast and observation grids.
DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 1536 Ny: 1152
lat_ll: -89.922 lon_ll: -0.117 delta_lat: 0.156 delta_lon: 0.234
DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_Gb1".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type "FileType_Gb1".
ERROR :
ERROR : VarInfoGrib::set_dict() - no parameter found with matching
GRIB1_ptv (3) GRIB1_rec (730)
ERROR :
If it is not obvious, I will place the climo file on your server.
Thanks
Bob Craig
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, January 24, 2017 4:46 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
Hi Bob,
This is John. The climatology file is any gridded data file that MET
supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After the
addition of the automated regridding feature in MET, you don't
actually need to put it on your model domain ahead of time. Instead,
you can use the "climo_mean -> regrid" config file option to specify
how you want to regrid the climatology data to your model domain.
I see from your message that you have monthly binary data from NCEP.
At the DTC, we have 2 variants of climatology data from NCEP, the
operational monthly 2.5 degree data and an experimental daily 1.0
degree dataset. My guess is that you have the 2.5 degree data. We
have that in both binary and GRIB1 format. MET can handle the GRIB
version... not the binary one.
If you'd like, I can send you that GRIB data.
But there actually is a pretty big difference between the binary and
GRIB datasets. The GRIB files contain the climatological mean and
spread at each grid point. Instead of mean/spread, the binary files
contain 101 values for each grid point representing the distribution.
I believe NCEP uses percentiles from the distribution in the
computation of statistics.
We have an open development task for MET version 6.0 to mimic NCEP's
use of climatological data and will likely need to handle those binary
files.
But MET version 5.2 only supports using the mean climatology value
read from a gridded data file.
Thanks,
John
On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hi Bob.
>
> Prior to the calculation of statistics, the climatology file must be
> put on the same grid as the forecasts and observations.
>
> I hope that helps.
>
> Julie
>
> On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Point Stat Anomaly Correlation
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> >
> > John or Julie, I am setting up MET to calculate 500mb HGT anomaly
> > correlation. I will be using prep buffer for the observations.
The
> > question I have is what format does the climatology file have to
be in?
> I
> > have month specific climo files from NCEP in binary format but not
> > sure what MET expects.
> >
> > Thanks
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
From: robert.craig.2 at us.af.mil
Time: Wed Jan 25 08:11:07 2017
John, I place the ecmwf grib file on your server.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, January 24, 2017 4:46 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
Hi Bob,
This is John. The climatology file is any gridded data file that MET
supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After the
addition of the automated regridding feature in MET, you don't
actually need to put it on your model domain ahead of time. Instead,
you can use the "climo_mean -> regrid" config file option to specify
how you want to regrid the climatology data to your model domain.
I see from your message that you have monthly binary data from NCEP.
At the DTC, we have 2 variants of climatology data from NCEP, the
operational monthly 2.5 degree data and an experimental daily 1.0
degree dataset. My guess is that you have the 2.5 degree data. We
have that in both binary and GRIB1 format. MET can handle the GRIB
version... not the binary one.
If you'd like, I can send you that GRIB data.
But there actually is a pretty big difference between the binary and
GRIB datasets. The GRIB files contain the climatological mean and
spread at each grid point. Instead of mean/spread, the binary files
contain 101 values for each grid point representing the distribution.
I believe NCEP uses percentiles from the distribution in the
computation of statistics.
We have an open development task for MET version 6.0 to mimic NCEP's
use of climatological data and will likely need to handle those binary
files.
But MET version 5.2 only supports using the mean climatology value
read from a gridded data file.
Thanks,
John
On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hi Bob.
>
> Prior to the calculation of statistics, the climatology file must be
> put on the same grid as the forecasts and observations.
>
> I hope that helps.
>
> Julie
>
> On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Point Stat Anomaly Correlation
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> >
> > John or Julie, I am setting up MET to calculate 500mb HGT anomaly
> > correlation. I will be using prep buffer for the observations.
The
> > question I have is what format does the climatology file have to
be in?
> I
> > have month specific climo files from NCEP in binary format but not
> > sure what MET expects.
> >
> > Thanks
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
From: robert.craig.2 at us.af.mil
Time: Wed Jan 25 08:27:13 2017
John, Please send my the 2.5 degree grib 1 dataset.
Thanks
Bob Craig
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, January 24, 2017 4:46 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
Hi Bob,
This is John. The climatology file is any gridded data file that MET
supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After the
addition of the automated regridding feature in MET, you don't
actually need to put it on your model domain ahead of time. Instead,
you can use the "climo_mean -> regrid" config file option to specify
how you want to regrid the climatology data to your model domain.
I see from your message that you have monthly binary data from NCEP.
At the DTC, we have 2 variants of climatology data from NCEP, the
operational monthly 2.5 degree data and an experimental daily 1.0
degree dataset. My guess is that you have the 2.5 degree data. We
have that in both binary and GRIB1 format. MET can handle the GRIB
version... not the binary one.
If you'd like, I can send you that GRIB data.
But there actually is a pretty big difference between the binary and
GRIB datasets. The GRIB files contain the climatological mean and
spread at each grid point. Instead of mean/spread, the binary files
contain 101 values for each grid point representing the distribution.
I believe NCEP uses percentiles from the distribution in the
computation of statistics.
We have an open development task for MET version 6.0 to mimic NCEP's
use of climatological data and will likely need to handle those binary
files.
But MET version 5.2 only supports using the mean climatology value
read from a gridded data file.
Thanks,
John
On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hi Bob.
>
> Prior to the calculation of statistics, the climatology file must be
> put on the same grid as the forecasts and observations.
>
> I hope that helps.
>
> Julie
>
> On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Point Stat Anomaly Correlation
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> >
> > John or Julie, I am setting up MET to calculate 500mb HGT anomaly
> > correlation. I will be using prep buffer for the observations.
The
> > question I have is what format does the climatology file have to
be in?
> I
> > have month specific climo files from NCEP in binary format but not
> > sure what MET expects.
> >
> > Thanks
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Jan 25 08:29:38 2017
Let's get the 1 degree daily too... might be useful some time. No
harm in holding on to it.
Hey John and Julie!
Matt
-----Original Message-----
From: robert.craig.2 at us.af.mil via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, January 25, 2017 9:27 AM
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
John, Please send my the 2.5 degree grib 1 dataset.
Thanks
Bob Craig
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, January 24, 2017 4:46 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
Hi Bob,
This is John. The climatology file is any gridded data file that MET
supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After the
addition of the automated regridding feature in MET, you don't
actually need to put it on your model domain ahead of time. Instead,
you can use the "climo_mean -> regrid" config file option to specify
how you want to regrid the climatology data to your model domain.
I see from your message that you have monthly binary data from NCEP.
At the DTC, we have 2 variants of climatology data from NCEP, the
operational monthly 2.5 degree data and an experimental daily 1.0
degree dataset. My guess is that you have the 2.5 degree data. We
have that in both binary and GRIB1 format. MET can handle the GRIB
version... not the binary one.
If you'd like, I can send you that GRIB data.
But there actually is a pretty big difference between the binary and
GRIB datasets. The GRIB files contain the climatological mean and
spread at each grid point. Instead of mean/spread, the binary files
contain 101 values for each grid point representing the distribution.
I believe NCEP uses percentiles from the distribution in the
computation of statistics.
We have an open development task for MET version 6.0 to mimic NCEP's
use of climatological data and will likely need to handle those binary
files.
But MET version 5.2 only supports using the mean climatology value
read from a gridded data file.
Thanks,
John
On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hi Bob.
>
> Prior to the calculation of statistics, the climatology file must be
> put on the same grid as the forecasts and observations.
>
> I hope that helps.
>
> Julie
>
> On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Point Stat Anomaly Correlation
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> >
> > John or Julie, I am setting up MET to calculate 500mb HGT anomaly
> > correlation. I will be using prep buffer for the observations.
The
> > question I have is what format does the climatology file have to
be in?
> I
> > have month specific climo files from NCEP in binary format but not
> > sure what MET expects.
> >
> > Thanks
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
------------------------------------------------
Subject: Point Stat Anomaly Correlation
From: John Halley Gotway
Time: Wed Jan 25 10:01:18 2017
Hi Bob and Matt,
Thanks for sending that sample data file. Running wgrib on it, I see
that
it contains 12-hourly climatological means for 500 mb geopotential
heights. Some points to make about this are...
(1) As of met-5.2, we added support for ECMWF GRIB tables... so this
should
be pretty easy. Just use:
name = "GP"; and level = "P500";
By way of example, try running this plot_data_plane command:
met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
'name="GP"; level="P500";'
You'll see this in the output log messages...
WARNING: MetGrib1DataFile::data_plane() -> Found 732 matches for
VarInfo
"GP/P500" in GRIB file "z500hPa_mean.grib". Using the first match
found.
All 732 records match since they all have the same variable type (GP)
and
level value (P500).
(2) I see that you were trying to tell Grid-Stat exactly which record
to
use by setting:
GRIB1_ptv = 3;
GRIB1_rec = 730;
Unfortunately, the "GRIB1_rec" option is very poorly named. And in
fact,
in met-5.2 we renamed it to GRIB1_code (but do still silently support
GRIB1_rec). This is meant to be the GRIB1 variable code (i.e. 11 is
usually temperature, 61 is usually APCP). It's the kpds5 value in the
wgrib output:
1:0:d=55010100:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:P2=0:TimeU=1:500
mb:anl:NAve=1220
So for this data, that value is 6, not 730. Looking at the README
file, I
see why you're confused! We incorrectly describe it as "the GRIB1
record
number". I'll correct that description in the next release.
OK, so now run this plot_data_plane command:
met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
'GRIB1_ptv=3; GRIB1_code=6; level="P500";' -v 5
Notice that the behavior is exactly the same as before... all 732
records
match. So setting name="GP"; is the same as setting GRIB1_ptv=3;
GRIB1_code=6;.
If you want to pick off a single record number you do that in the
level
string using R### to indicate the record number you want:
met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean_R730.ps
'name="GP"; level="R730";' -v 5
Notice that that command only selects record number 730.
(3) But all of this isn't really necessary... Grid-Stat is smart
enough to
find the matching record for you! Try setting:
//
// Climatology mean data
//
climo_mean = {
file_name =
["/h/data/global/WXQC/data/climo/ecmwf/z500hPa_mean.grib"];
field = [
{ name = "GP"; level = [ "P500" ]; }
];
regrid = {
to_grid = FCST;
vld_thresh = 0.5;
method = NEAREST;
width = 1;
}
time_interp_method = DW_MEAN;
match_day = TRUE;
time_step = 43200;
}
Grid-Stat will read through the records in "z500hPa_mean.grib". It
ignores
the year for the data but keeps track of the month, day, and hour.
Since
"match_day" is set to true it searches until it finds records whose
month
and day match the verification valid time. Then it finds records
whose
time_stamp is within 43200 seconds (i.e. 12 hours) of the forecast
valid
time. If there's not an exact match for the time stamp, it'll
interpolate
between the two closest ones.
For example, if your forecast is valid at 20160807_09, it should find
record numbers 439 and 440, valid at 00Z and 12Z on that day:
439:25475832:d=55080700:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:P2=0:TimeU=1:500
mb:anl:NAve=1220
440:25533996:d=55080712:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:P2=0:TimeU=1:500
mb:anl:NAve=1220
And it'll take their time-weighted average... 1/4 weight for 00Z and
3/4
weight for 12Z since 09Z is closer to 12Z than 00Z.
And finally, that's the climo field it'll use in the statistics.
Does that all make sense?
FYI, I posted NCEP's 1.0 and 2.5 degree climo files to our anonymous
ftp
site here:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_1.0deg.tar.gz
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_2.5deg.tar.gz
Thanks,
John
On Wed, Jan 25, 2017 at 8:29 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Let's get the 1 degree daily too... might be useful some time. No
harm in
> holding on to it.
>
> Hey John and Julie!
>
> Matt
>
> -----Original Message-----
> From: robert.craig.2 at us.af.mil via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 25, 2017 9:27 AM
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
>
> John, Please send my the 2.5 degree grib 1 dataset.
>
> Thanks
> Bob Craig
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, January 24, 2017 4:46 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
>
> Hi Bob,
>
> This is John. The climatology file is any gridded data file that
MET
> supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After
the
> addition of the automated regridding feature in MET, you don't
actually
> need to put it on your model domain ahead of time. Instead, you can
use
> the "climo_mean -> regrid" config file option to specify how you
want to
> regrid the climatology data to your model domain.
>
> I see from your message that you have monthly binary data from NCEP.
At
> the DTC, we have 2 variants of climatology data from NCEP, the
operational
> monthly 2.5 degree data and an experimental daily 1.0 degree
dataset. My
> guess is that you have the 2.5 degree data. We have that in both
binary
> and GRIB1 format. MET can handle the GRIB version... not the binary
one.
>
> If you'd like, I can send you that GRIB data.
>
> But there actually is a pretty big difference between the binary and
GRIB
> datasets. The GRIB files contain the climatological mean and spread
at
> each grid point. Instead of mean/spread, the binary files contain
101
> values for each grid point representing the distribution. I believe
NCEP
> uses percentiles from the distribution in the computation of
statistics.
> We have an open development task for MET version 6.0 to mimic NCEP's
use
> of climatological data and will likely need to handle those binary
files.
>
> But MET version 5.2 only supports using the mean climatology value
read
> from a gridded data file.
>
> Thanks,
> John
>
>
>
>
> On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> > Hi Bob.
> >
> > Prior to the calculation of statistics, the climatology file must
be
> > put on the same grid as the forecasts and observations.
> >
> > I hope that helps.
> >
> > Julie
> >
> > On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > Queue: met_help
> > > Subject: Point Stat Anomaly Correlation
> > > Owner: Nobody
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> > >
> > >
> > > John or Julie, I am setting up MET to calculate 500mb HGT
anomaly
> > > correlation. I will be using prep buffer for the observations.
The
> > > question I have is what format does the climatology file have to
be in?
> > I
> > > have month specific climo files from NCEP in binary format but
not
> > > sure what MET expects.
> > >
> > > Thanks
> > > Bob Craig
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
>
>
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Jan 25 10:29:39 2017
Hey John,
Can you put the 5.2 tarball out there too?
Is 6.0 coming soon?
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, January 25, 2017 11:01 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
Hi Bob and Matt,
Thanks for sending that sample data file. Running wgrib on it, I see
that it contains 12-hourly climatological means for 500 mb
geopotential heights. Some points to make about this are...
(1) As of met-5.2, we added support for ECMWF GRIB tables... so this
should be pretty easy. Just use:
name = "GP"; and level = "P500";
By way of example, try running this plot_data_plane command:
met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
'name="GP"; level="P500";'
You'll see this in the output log messages...
WARNING: MetGrib1DataFile::data_plane() -> Found 732 matches for
VarInfo "GP/P500" in GRIB file "z500hPa_mean.grib". Using the first
match found.
All 732 records match since they all have the same variable type (GP)
and level value (P500).
(2) I see that you were trying to tell Grid-Stat exactly which record
to use by setting:
GRIB1_ptv = 3;
GRIB1_rec = 730;
Unfortunately, the "GRIB1_rec" option is very poorly named. And in
fact, in met-5.2 we renamed it to GRIB1_code (but do still silently
support GRIB1_rec). This is meant to be the GRIB1 variable code (i.e.
11 is usually temperature, 61 is usually APCP). It's the kpds5 value
in the wgrib output:
1:0:d=55010100:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:P2=0:TimeU=1:500
mb:anl:NAve=1220
So for this data, that value is 6, not 730. Looking at the README
file, I see why you're confused! We incorrectly describe it as "the
GRIB1 record number". I'll correct that description in the next
release.
OK, so now run this plot_data_plane command:
met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
'GRIB1_ptv=3; GRIB1_code=6; level="P500";' -v 5
Notice that the behavior is exactly the same as before... all 732
records match. So setting name="GP"; is the same as setting
GRIB1_ptv=3; GRIB1_code=6;.
If you want to pick off a single record number you do that in the
level string using R### to indicate the record number you want:
met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean_R730.ps
'name="GP"; level="R730";' -v 5
Notice that that command only selects record number 730.
(3) But all of this isn't really necessary... Grid-Stat is smart
enough to find the matching record for you! Try setting:
//
// Climatology mean data
//
climo_mean = {
file_name =
["/h/data/global/WXQC/data/climo/ecmwf/z500hPa_mean.grib"];
field = [
{ name = "GP"; level = [ "P500" ]; }
];
regrid = {
to_grid = FCST;
vld_thresh = 0.5;
method = NEAREST;
width = 1;
}
time_interp_method = DW_MEAN;
match_day = TRUE;
time_step = 43200;
}
Grid-Stat will read through the records in "z500hPa_mean.grib". It
ignores the year for the data but keeps track of the month, day, and
hour. Since "match_day" is set to true it searches until it finds
records whose month and day match the verification valid time. Then
it finds records whose time_stamp is within 43200 seconds (i.e. 12
hours) of the forecast valid time. If there's not an exact match for
the time stamp, it'll interpolate between the two closest ones.
For example, if your forecast is valid at 20160807_09, it should find
record numbers 439 and 440, valid at 00Z and 12Z on that day:
439:25475832:d=55080700:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:P2=0:TimeU=1:500
mb:anl:NAve=1220
440:25533996:d=55080712:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:P2=0:TimeU=1:500
mb:anl:NAve=1220
And it'll take their time-weighted average... 1/4 weight for 00Z and
3/4 weight for 12Z since 09Z is closer to 12Z than 00Z.
And finally, that's the climo field it'll use in the statistics.
Does that all make sense?
FYI, I posted NCEP's 1.0 and 2.5 degree climo files to our anonymous
ftp site here:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_1.0deg.tar.gz
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_2.5deg.tar.gz
Thanks,
John
On Wed, Jan 25, 2017 at 8:29 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Let's get the 1 degree daily too... might be useful some time. No
> harm in holding on to it.
>
> Hey John and Julie!
>
> Matt
>
> -----Original Message-----
> From: robert.craig.2 at us.af.mil via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 25, 2017 9:27 AM
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
>
> John, Please send my the 2.5 degree grib 1 dataset.
>
> Thanks
> Bob Craig
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, January 24, 2017 4:46 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
>
> Hi Bob,
>
> This is John. The climatology file is any gridded data file that
MET
> supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After
the
> addition of the automated regridding feature in MET, you don't
> actually need to put it on your model domain ahead of time.
Instead,
> you can use the "climo_mean -> regrid" config file option to specify
> how you want to regrid the climatology data to your model domain.
>
> I see from your message that you have monthly binary data from NCEP.
> At the DTC, we have 2 variants of climatology data from NCEP, the
> operational monthly 2.5 degree data and an experimental daily 1.0
> degree dataset. My guess is that you have the 2.5 degree data. We
> have that in both binary and GRIB1 format. MET can handle the GRIB
version... not the binary one.
>
> If you'd like, I can send you that GRIB data.
>
> But there actually is a pretty big difference between the binary and
> GRIB datasets. The GRIB files contain the climatological mean and
> spread at each grid point. Instead of mean/spread, the binary files
> contain 101 values for each grid point representing the
distribution.
> I believe NCEP uses percentiles from the distribution in the
computation of statistics.
> We have an open development task for MET version 6.0 to mimic NCEP's
> use of climatological data and will likely need to handle those
binary files.
>
> But MET version 5.2 only supports using the mean climatology value
> read from a gridded data file.
>
> Thanks,
> John
>
>
>
>
> On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> > Hi Bob.
> >
> > Prior to the calculation of statistics, the climatology file must
be
> > put on the same grid as the forecasts and observations.
> >
> > I hope that helps.
> >
> > Julie
> >
> > On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > Queue: met_help
> > > Subject: Point Stat Anomaly Correlation
> > > Owner: Nobody
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> > >
> > >
> > > John or Julie, I am setting up MET to calculate 500mb HGT
anomaly
> > > correlation. I will be using prep buffer for the observations.
> > > The question I have is what format does the climatology file
have to be in?
> > I
> > > have month specific climo files from NCEP in binary format but
not
> > > sure what MET expects.
> > >
> > > Thanks
> > > Bob Craig
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
>
>
>
>
>
>
------------------------------------------------
Subject: Point Stat Anomaly Correlation
From: John Halley Gotway
Time: Wed Jan 25 11:31:42 2017
Done. The met-5.2 tarball plus the met-5.2 patch file are available
in
that same "climo" directory:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo
We're still working away on version 6.0 but don't expect to release it
until mid-March.
Adding support for the HIRA framework is still on the list, so we may
ask
you and/or Bob to try it out for us.
Thanks,
John
On Wed, Jan 25, 2017 at 10:29 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hey John,
>
> Can you put the 5.2 tarball out there too?
>
> Is 6.0 coming soon?
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 25, 2017 11:01 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
>
> Hi Bob and Matt,
>
> Thanks for sending that sample data file. Running wgrib on it, I
see that
> it contains 12-hourly climatological means for 500 mb geopotential
> heights. Some points to make about this are...
>
> (1) As of met-5.2, we added support for ECMWF GRIB tables... so this
> should be pretty easy. Just use:
> name = "GP"; and level = "P500";
>
> By way of example, try running this plot_data_plane command:
> met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
> 'name="GP"; level="P500";'
>
> You'll see this in the output log messages...
> WARNING: MetGrib1DataFile::data_plane() -> Found 732 matches for
> VarInfo "GP/P500" in GRIB file "z500hPa_mean.grib". Using the first
match
> found.
>
> All 732 records match since they all have the same variable type
(GP) and
> level value (P500).
>
> (2) I see that you were trying to tell Grid-Stat exactly which
record to
> use by setting:
> GRIB1_ptv = 3;
> GRIB1_rec = 730;
>
> Unfortunately, the "GRIB1_rec" option is very poorly named. And in
fact,
> in met-5.2 we renamed it to GRIB1_code (but do still silently
support
> GRIB1_rec). This is meant to be the GRIB1 variable code (i.e. 11 is
> usually temperature, 61 is usually APCP). It's the kpds5 value in
the
> wgrib output:
> 1:0:d=55010100:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:
> P2=0:TimeU=1:500
> mb:anl:NAve=1220
>
> So for this data, that value is 6, not 730. Looking at the README
file, I
> see why you're confused! We incorrectly describe it as "the GRIB1
record
> number". I'll correct that description in the next release.
>
> OK, so now run this plot_data_plane command:
> met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
> 'GRIB1_ptv=3; GRIB1_code=6; level="P500";' -v 5
>
> Notice that the behavior is exactly the same as before... all 732
records
> match. So setting name="GP"; is the same as setting GRIB1_ptv=3;
> GRIB1_code=6;.
>
> If you want to pick off a single record number you do that in the
level
> string using R### to indicate the record number you want:
> met-5.2/bin/plot_data_plane z500hPa_mean.grib
z500hPa_mean_R730.ps
> 'name="GP"; level="R730";' -v 5
>
> Notice that that command only selects record number 730.
>
> (3) But all of this isn't really necessary... Grid-Stat is smart
enough to
> find the matching record for you! Try setting:
>
> //
> // Climatology mean data
> //
> climo_mean = {
>
> file_name =
["/h/data/global/WXQC/data/climo/ecmwf/z500hPa_mean.grib"];
> field = [
> { name = "GP"; level = [ "P500" ]; }
> ];
>
> regrid = {
> to_grid = FCST;
> vld_thresh = 0.5;
> method = NEAREST;
> width = 1;
> }
>
> time_interp_method = DW_MEAN;
> match_day = TRUE;
> time_step = 43200;
> }
>
> Grid-Stat will read through the records in "z500hPa_mean.grib". It
> ignores the year for the data but keeps track of the month, day, and
hour.
> Since "match_day" is set to true it searches until it finds records
whose
> month and day match the verification valid time. Then it finds
records
> whose time_stamp is within 43200 seconds (i.e. 12 hours) of the
forecast
> valid time. If there's not an exact match for the time stamp, it'll
> interpolate between the two closest ones.
>
> For example, if your forecast is valid at 20160807_09, it should
find
> record numbers 439 and 440, valid at 00Z and 12Z on that day:
>
> 439:25475832:d=55080700:GP:kpds5=6:kpds6=100:kpds7=500:
> TR=0:P1=0:P2=0:TimeU=1:500
> mb:anl:NAve=1220
>
> 440:25533996:d=55080712:GP:kpds5=6:kpds6=100:kpds7=500:
> TR=0:P1=0:P2=0:TimeU=1:500
> mb:anl:NAve=1220
>
> And it'll take their time-weighted average... 1/4 weight for 00Z and
3/4
> weight for 12Z since 09Z is closer to 12Z than 00Z.
>
> And finally, that's the climo field it'll use in the statistics.
>
> Does that all make sense?
>
> FYI, I posted NCEP's 1.0 and 2.5 degree climo files to our anonymous
ftp
> site here:
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_1.0deg.tar.gz
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_2.5deg.tar.gz
>
> Thanks,
> John
>
> On Wed, Jan 25, 2017 at 8:29 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> > Let's get the 1 degree daily too... might be useful some time. No
> > harm in holding on to it.
> >
> > Hey John and Julie!
> >
> > Matt
> >
> > -----Original Message-----
> > From: robert.craig.2 at us.af.mil via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, January 25, 2017 9:27 AM
> > Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> > matthew.sittel.ctr at us.af.mil>
> > Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly
Correlation
> >
> > John, Please send my the 2.5 degree grib 1 dataset.
> >
> > Thanks
> > Bob Craig
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, January 24, 2017 4:46 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > <robert.craig.2 at us.af.mil>
> > Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> > matthew.sittel.ctr at us.af.mil>
> > Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly
Correlation
> >
> > Hi Bob,
> >
> > This is John. The climatology file is any gridded data file that
MET
> > supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF. After
the
> > addition of the automated regridding feature in MET, you don't
> > actually need to put it on your model domain ahead of time.
Instead,
> > you can use the "climo_mean -> regrid" config file option to
specify
> > how you want to regrid the climatology data to your model domain.
> >
> > I see from your message that you have monthly binary data from
NCEP.
> > At the DTC, we have 2 variants of climatology data from NCEP, the
> > operational monthly 2.5 degree data and an experimental daily 1.0
> > degree dataset. My guess is that you have the 2.5 degree data.
We
> > have that in both binary and GRIB1 format. MET can handle the
GRIB
> version... not the binary one.
> >
> > If you'd like, I can send you that GRIB data.
> >
> > But there actually is a pretty big difference between the binary
and
> > GRIB datasets. The GRIB files contain the climatological mean and
> > spread at each grid point. Instead of mean/spread, the binary
files
> > contain 101 values for each grid point representing the
distribution.
> > I believe NCEP uses percentiles from the distribution in the
computation
> of statistics.
> > We have an open development task for MET version 6.0 to mimic
NCEP's
> > use of climatological data and will likely need to handle those
binary
> files.
> >
> > But MET version 5.2 only supports using the mean climatology value
> > read from a gridded data file.
> >
> > Thanks,
> > John
> >
> >
> >
> >
> > On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> > >
> > > Hi Bob.
> > >
> > > Prior to the calculation of statistics, the climatology file
must be
> > > put on the same grid as the forecasts and observations.
> > >
> > > I hope that helps.
> > >
> > > Julie
> > >
> > > On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > > Queue: met_help
> > > > Subject: Point Stat Anomaly Correlation
> > > > Owner: Nobody
> > > > Requestors: robert.craig.2 at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> > > >
> > > >
> > > > John or Julie, I am setting up MET to calculate 500mb HGT
anomaly
> > > > correlation. I will be using prep buffer for the
observations.
> > > > The question I have is what format does the climatology file
have to
> be in?
> > > I
> > > > have month specific climo files from NCEP in binary format but
not
> > > > sure what MET expects.
> > > >
> > > > Thanks
> > > > Bob Craig
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> >
> >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
From: robert.craig.2 at us.af.mil
Time: Wed Jan 25 11:48:20 2017
Great, thanks.
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, January 25, 2017 12:32 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
Done. The met-5.2 tarball plus the met-5.2 patch file are available
in that same "climo" directory:
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo
We're still working away on version 6.0 but don't expect to release it
until mid-March.
Adding support for the HIRA framework is still on the list, so we may
ask you and/or Bob to try it out for us.
Thanks,
John
On Wed, Jan 25, 2017 at 10:29 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
>
> Hey John,
>
> Can you put the 5.2 tarball out there too?
>
> Is 6.0 coming soon?
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 25, 2017 11:01 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly Correlation
>
> Hi Bob and Matt,
>
> Thanks for sending that sample data file. Running wgrib on it, I
see
> that it contains 12-hourly climatological means for 500 mb
> geopotential heights. Some points to make about this are...
>
> (1) As of met-5.2, we added support for ECMWF GRIB tables... so this
> should be pretty easy. Just use:
> name = "GP"; and level = "P500";
>
> By way of example, try running this plot_data_plane command:
> met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
> 'name="GP"; level="P500";'
>
> You'll see this in the output log messages...
> WARNING: MetGrib1DataFile::data_plane() -> Found 732 matches for
> VarInfo "GP/P500" in GRIB file "z500hPa_mean.grib". Using the first
> match found.
>
> All 732 records match since they all have the same variable type
(GP)
> and level value (P500).
>
> (2) I see that you were trying to tell Grid-Stat exactly which
record
> to use by setting:
> GRIB1_ptv = 3;
> GRIB1_rec = 730;
>
> Unfortunately, the "GRIB1_rec" option is very poorly named. And in
> fact, in met-5.2 we renamed it to GRIB1_code (but do still silently
> support GRIB1_rec). This is meant to be the GRIB1 variable code
(i.e.
> 11 is usually temperature, 61 is usually APCP). It's the kpds5
value
> in the wgrib output:
> 1:0:d=55010100:GP:kpds5=6:kpds6=100:kpds7=500:TR=0:P1=0:
> P2=0:TimeU=1:500
> mb:anl:NAve=1220
>
> So for this data, that value is 6, not 730. Looking at the README
> file, I see why you're confused! We incorrectly describe it as "the
> GRIB1 record number". I'll correct that description in the next
release.
>
> OK, so now run this plot_data_plane command:
> met-5.2/bin/plot_data_plane z500hPa_mean.grib z500hPa_mean.ps
> 'GRIB1_ptv=3; GRIB1_code=6; level="P500";' -v 5
>
> Notice that the behavior is exactly the same as before... all 732
> records match. So setting name="GP"; is the same as setting
> GRIB1_ptv=3; GRIB1_code=6;.
>
> If you want to pick off a single record number you do that in the
> level string using R### to indicate the record number you want:
> met-5.2/bin/plot_data_plane z500hPa_mean.grib
z500hPa_mean_R730.ps
> 'name="GP"; level="R730";' -v 5
>
> Notice that that command only selects record number 730.
>
> (3) But all of this isn't really necessary... Grid-Stat is smart
> enough to find the matching record for you! Try setting:
>
> //
> // Climatology mean data
> //
> climo_mean = {
>
> file_name =
["/h/data/global/WXQC/data/climo/ecmwf/z500hPa_mean.grib"];
> field = [
> { name = "GP"; level = [ "P500" ]; }
> ];
>
> regrid = {
> to_grid = FCST;
> vld_thresh = 0.5;
> method = NEAREST;
> width = 1;
> }
>
> time_interp_method = DW_MEAN;
> match_day = TRUE;
> time_step = 43200;
> }
>
> Grid-Stat will read through the records in "z500hPa_mean.grib". It
> ignores the year for the data but keeps track of the month, day, and
hour.
> Since "match_day" is set to true it searches until it finds records
> whose month and day match the verification valid time. Then it
finds
> records whose time_stamp is within 43200 seconds (i.e. 12 hours) of
> the forecast valid time. If there's not an exact match for the time
> stamp, it'll interpolate between the two closest ones.
>
> For example, if your forecast is valid at 20160807_09, it should
find
> record numbers 439 and 440, valid at 00Z and 12Z on that day:
>
> 439:25475832:d=55080700:GP:kpds5=6:kpds6=100:kpds7=500:
> TR=0:P1=0:P2=0:TimeU=1:500
> mb:anl:NAve=1220
>
> 440:25533996:d=55080712:GP:kpds5=6:kpds6=100:kpds7=500:
> TR=0:P1=0:P2=0:TimeU=1:500
> mb:anl:NAve=1220
>
> And it'll take their time-weighted average... 1/4 weight for 00Z and
> 3/4 weight for 12Z since 09Z is closer to 12Z than 00Z.
>
> And finally, that's the climo field it'll use in the statistics.
>
> Does that all make sense?
>
> FYI, I posted NCEP's 1.0 and 2.5 degree climo files to our anonymous
> ftp site here:
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_1.0deg.tar.gz
>
>
ftp://ftp.rap.ucar.edu/incoming/irap/met_help/climo/NCEP_2.5deg.tar.gz
>
> Thanks,
> John
>
> On Wed, Jan 25, 2017 at 8:29 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> >
> > Let's get the 1 degree daily too... might be useful some time. No
> > harm in holding on to it.
> >
> > Hey John and Julie!
> >
> > Matt
> >
> > -----Original Message-----
> > From: robert.craig.2 at us.af.mil via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, January 25, 2017 9:27 AM
> > Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> > matthew.sittel.ctr at us.af.mil>
> > Subject: RE: [rt.rap.ucar.edu #79255] Point Stat Anomaly
Correlation
> >
> > John, Please send my the 2.5 degree grib 1 dataset.
> >
> > Thanks
> > Bob Craig
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, January 24, 2017 4:46 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > <robert.craig.2 at us.af.mil>
> > Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> > matthew.sittel.ctr at us.af.mil>
> > Subject: Re: [rt.rap.ucar.edu #79255] Point Stat Anomaly
Correlation
> >
> > Hi Bob,
> >
> > This is John. The climatology file is any gridded data file that
> > MET supports... i.e. GRIB1, GRIB2, and some flavors of NetCDF.
> > After the addition of the automated regridding feature in MET, you
> > don't actually need to put it on your model domain ahead of time.
> > Instead, you can use the "climo_mean -> regrid" config file option
> > to specify how you want to regrid the climatology data to your
model domain.
> >
> > I see from your message that you have monthly binary data from
NCEP.
> > At the DTC, we have 2 variants of climatology data from NCEP, the
> > operational monthly 2.5 degree data and an experimental daily 1.0
> > degree dataset. My guess is that you have the 2.5 degree data.
We
> > have that in both binary and GRIB1 format. MET can handle the
GRIB
> version... not the binary one.
> >
> > If you'd like, I can send you that GRIB data.
> >
> > But there actually is a pretty big difference between the binary
and
> > GRIB datasets. The GRIB files contain the climatological mean and
> > spread at each grid point. Instead of mean/spread, the binary
files
> > contain 101 values for each grid point representing the
distribution.
> > I believe NCEP uses percentiles from the distribution in the
> > computation
> of statistics.
> > We have an open development task for MET version 6.0 to mimic
NCEP's
> > use of climatological data and will likely need to handle those
> > binary
> files.
> >
> > But MET version 5.2 only supports using the mean climatology value
> > read from a gridded data file.
> >
> > Thanks,
> > John
> >
> >
> >
> >
> > On Tue, Jan 24, 2017 at 2:46 PM, Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> > >
> > > Hi Bob.
> > >
> > > Prior to the calculation of statistics, the climatology file
must
> > > be put on the same grid as the forecasts and observations.
> > >
> > > I hope that helps.
> > >
> > > Julie
> > >
> > > On Tue, Jan 24, 2017 at 2:22 PM, robert.craig.2 at us.af.mil via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jan 24 14:22:14 2017: Request 79255 was acted upon.
> > > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > > Queue: met_help
> > > > Subject: Point Stat Anomaly Correlation
> > > > Owner: Nobody
> > > > Requestors: robert.craig.2 at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=79255 >
> > > >
> > > >
> > > > John or Julie, I am setting up MET to calculate 500mb HGT
> > > > anomaly correlation. I will be using prep buffer for the
observations.
> > > > The question I have is what format does the climatology file
> > > > have to
> be in?
> > > I
> > > > have month specific climo files from NCEP in binary format but
> > > > not sure what MET expects.
> > > >
> > > > Thanks
> > > > Bob Craig
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> >
> >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list