[Met_help] [rt.rap.ucar.edu #83987] History for calculating anomaly correlations with GRIB surface data
John Halley Gotway via RT
met_help at ucar.edu
Thu Feb 8 11:26:09 MST 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Good Afternoon.
I am running MET on Theia. The following is an portion of my
configuration file
////////////////////////////////////////////////////////////////////////////////
// Forecast and observation fields to be verified
//
fcst = {
field = [
{
name = "UGRD";
level = ["R183"];
},
{
name = "VGRD";
level = ["R184"];
},
{
name = "TMP";
level = ["R177"];
}
];
}
///////////////////////////////////////////////////////////////
obs = {
field = [
{
name = "10U";
level = ["R1"];
},
{
name = "10V";
level = ["R2"];
},
{
name = "2T";
level = ["R3"];
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Climatology mean data
//
climo_mean = {
file_name =
["/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/climate/cmean_1d.1959051200.grb"];
field = [
{
name = "UGRD";
level = ["R18"];
},
{
name = "VGRD";
level = ["R19"];
},
{
name = "TMP";
level = ["R15"];
}
];
regrid = {
method = NEAREST;
width = 1;
vld_thresh = 0.5;
}
time_interp_method = DW_MEAN;
match_day = TRUE;
time_step = 3600;
}
////////////////////////////////////////////////////////////////////////////////
The wgrib on my climatology file is :
15:1540880:d=59030118:TMP:kpds5=11:kpds6=105:kpds7=2:TR=51:P1=6:P2=0:TimeU=1:2
m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
16:1655016:d=59030118:TMAX:kpds5=15:kpds6=105:kpds7=2:TR=51:P1=6:P2=0:TimeU=1:2
m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
17:1769152:d=59030118:TMIN:kpds5=16:kpds6=105:kpds7=2:TR=51:P1=6:P2=0:TimeU=1:2
m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
18:1883288:d=59030118:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=51:P1=6:P2=0:TimeU=1:10
m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
;---------------------------------------------------------------------------------------------------------------------------------
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "TMP/R15" in GRIB file
"/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/climate/cmean_1d.1959050200.grb".
DEBUG 3: Found 0 climatology fields.
DEBUG 3: Found 0 climatology mean field(s) for forecast TMP/R177.
It will calculate the rmse and bias but it won't calculate the anomoly
correlations. I am not sure
of why.
At your convenience could you please pass along any thoughts you may have.
I don't have
any problems doing the pressure surface anomaly correlations with the same
climate file.
All of my data files are in GRIB1 format.
Many Thanks,
Sherrie
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: calculating anomaly correlations with GRIB surface data
From: John Halley Gotway
Time: Wed Feb 07 15:07:12 2018
Hi Sherrie,
Thanks for sending some sample output. While the MET tools allow you
set
the explicit record number to be used from the GRIB file, I'd strongly
advise against doing that routinely. There's no guarantee that the
order
of the GRIB records today will be the same order tomorrow.
I'd like to run a sample case on theia using your data to see if I can
make
some useful suggestions. But I'll need to know the location of your:
(1) forecast file, (2) observation file, and (3) config file. I
already
see the path to your climo data.
Once I know where I can find this data on theia, I'll let you know
what I
find.
Thanks,
John Halley Gotway
On Wed, Feb 7, 2018 at 2:54 PM, Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Wed Feb 07 14:54:19 2018: Request 83987 was acted upon.
> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> Queue: met_help
> Subject: calculating anomaly correlations with GRIB surface
data
> Owner: Nobody
> Requestors: sherrie.fredrick at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83987 >
>
>
> Good Afternoon.
>
> I am running MET on Theia. The following is an portion of my
> configuration file
>
>
> ////////////////////////////////////////////////////////////
> ////////////////////
>
> // Forecast and observation fields to be verified
> //
> fcst = {
>
> field = [
> {
> name = "UGRD";
> level = ["R183"];
> },
> {
> name = "VGRD";
> level = ["R184"];
> },
> {
> name = "TMP";
> level = ["R177"];
> }
> ];
>
> }
> ///////////////////////////////////////////////////////////////
> obs = {
>
> field = [
> {
> name = "10U";
> level = ["R1"];
> },
> {
> name = "10V";
> level = ["R2"];
> },
> {
> name = "2T";
> level = ["R3"];
> }
> ];
>
> }
> ////////////////////////////////////////////////////////////
> ////////////////////
>
> //
> // Climatology mean data
> //
> climo_mean = {
>
> file_name =
> ["/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> climate/cmean_1d.1959051200.grb"];
> field = [
> {
> name = "UGRD";
> level = ["R18"];
> },
> {
> name = "VGRD";
> level = ["R19"];
> },
> {
> name = "TMP";
> level = ["R15"];
> }
> ];
>
> regrid = {
> method = NEAREST;
> width = 1;
> vld_thresh = 0.5;
> }
>
> time_interp_method = DW_MEAN;
> match_day = TRUE;
> time_step = 3600;
> }
>
> ////////////////////////////////////////////////////////////
> ////////////////////
> The wgrib on my climatology file is :
> 15:1540880:d=59030118:TMP:kpds5=11:kpds6=105:kpds7=2:TR=
> 51:P1=6:P2=0:TimeU=1:2
> m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> 16:1655016:d=59030118:TMAX:kpds5=15:kpds6=105:kpds7=2:TR=
> 51:P1=6:P2=0:TimeU=1:2
> m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> 17:1769152:d=59030118:TMIN:kpds5=16:kpds6=105:kpds7=2:TR=
> 51:P1=6:P2=0:TimeU=1:2
> m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> 18:1883288:d=59030118:UGRD:kpds5=33:kpds6=105:kpds7=10:
> TR=51:P1=6:P2=0:TimeU=1:10
> m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
>
> ;-----------------------------------------------------------
>
----------------------------------------------------------------------
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/R15" in GRIB file
> "/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> climate/cmean_1d.1959050200.grb".
> DEBUG 3: Found 0 climatology fields.
> DEBUG 3: Found 0 climatology mean field(s) for forecast TMP/R177.
>
>
> It will calculate the rmse and bias but it won't calculate the
anomoly
> correlations. I am not sure
> of why.
>
> At your convenience could you please pass along any thoughts you may
have.
> I don't have
> any problems doing the pressure surface anomaly correlations with
the same
> climate file.
> All of my data files are in GRIB1 format.
>
> Many Thanks,
> Sherrie
>
>
------------------------------------------------
Subject: calculating anomaly correlations with GRIB surface data
From: Sherrie Fredrick - NOAA Affiliate
Time: Wed Feb 07 15:20:26 2018
Hi John,
I made a directory on Theia called
/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/wcoss/John.
The model file is FV3_2001090500.grib.
The observation file is erai_2001090500.grib.
The climate file is cmean_1d.1959090500.grb
I hope this helps.
Thanks,
Sherrie
On Wed, Feb 7, 2018 at 3:07 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Hi Sherrie,
>
> Thanks for sending some sample output. While the MET tools allow
you set
> the explicit record number to be used from the GRIB file, I'd
strongly
> advise against doing that routinely. There's no guarantee that the
order
> of the GRIB records today will be the same order tomorrow.
>
> I'd like to run a sample case on theia using your data to see if I
can make
> some useful suggestions. But I'll need to know the location of
your:
> (1) forecast file, (2) observation file, and (3) config file. I
already
> see the path to your climo data.
>
> Once I know where I can find this data on theia, I'll let you know
what I
> find.
>
> Thanks,
> John Halley Gotway
>
>
> On Wed, Feb 7, 2018 at 2:54 PM, Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Feb 07 14:54:19 2018: Request 83987 was acted upon.
> > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > Queue: met_help
> > Subject: calculating anomaly correlations with GRIB surface
data
> > Owner: Nobody
> > Requestors: sherrie.fredrick at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83987 >
> >
> >
> > Good Afternoon.
> >
> > I am running MET on Theia. The following is an portion of my
> > configuration file
> >
> >
> > ////////////////////////////////////////////////////////////
> > ////////////////////
> >
> > // Forecast and observation fields to be verified
> > //
> > fcst = {
> >
> > field = [
> > {
> > name = "UGRD";
> > level = ["R183"];
> > },
> > {
> > name = "VGRD";
> > level = ["R184"];
> > },
> > {
> > name = "TMP";
> > level = ["R177"];
> > }
> > ];
> >
> > }
> > ///////////////////////////////////////////////////////////////
> > obs = {
> >
> > field = [
> > {
> > name = "10U";
> > level = ["R1"];
> > },
> > {
> > name = "10V";
> > level = ["R2"];
> > },
> > {
> > name = "2T";
> > level = ["R3"];
> > }
> > ];
> >
> > }
> > ////////////////////////////////////////////////////////////
> > ////////////////////
> >
> > //
> > // Climatology mean data
> > //
> > climo_mean = {
> >
> > file_name =
> > ["/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> > climate/cmean_1d.1959051200.grb"];
> > field = [
> > {
> > name = "UGRD";
> > level = ["R18"];
> > },
> > {
> > name = "VGRD";
> > level = ["R19"];
> > },
> > {
> > name = "TMP";
> > level = ["R15"];
> > }
> > ];
> >
> > regrid = {
> > method = NEAREST;
> > width = 1;
> > vld_thresh = 0.5;
> > }
> >
> > time_interp_method = DW_MEAN;
> > match_day = TRUE;
> > time_step = 3600;
> > }
> >
> > ////////////////////////////////////////////////////////////
> > ////////////////////
> > The wgrib on my climatology file is :
> > 15:1540880:d=59030118:TMP:kpds5=11:kpds6=105:kpds7=2:TR=
> > 51:P1=6:P2=0:TimeU=1:2
> > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > 16:1655016:d=59030118:TMAX:kpds5=15:kpds6=105:kpds7=2:TR=
> > 51:P1=6:P2=0:TimeU=1:2
> > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > 17:1769152:d=59030118:TMIN:kpds5=16:kpds6=105:kpds7=2:TR=
> > 51:P1=6:P2=0:TimeU=1:2
> > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > 18:1883288:d=59030118:UGRD:kpds5=33:kpds6=105:kpds7=10:
> > TR=51:P1=6:P2=0:TimeU=1:10
> > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> >
> > ;-----------------------------------------------------------
> >
----------------------------------------------------------------------
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "TMP/R15" in GRIB file
> > "/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> > climate/cmean_1d.1959050200.grb".
> > DEBUG 3: Found 0 climatology fields.
> > DEBUG 3: Found 0 climatology mean field(s) for forecast TMP/R177.
> >
> >
> > It will calculate the rmse and bias but it won't calculate the
anomoly
> > correlations. I am not sure
> > of why.
> >
> > At your convenience could you please pass along any thoughts you
may
> have.
> > I don't have
> > any problems doing the pressure surface anomaly correlations with
the
> same
> > climate file.
> > All of my data files are in GRIB1 format.
> >
> > Many Thanks,
> > Sherrie
> >
> >
>
>
------------------------------------------------
Subject: calculating anomaly correlations with GRIB surface data
From: John Halley Gotway
Time: Wed Feb 07 16:19:32 2018
Sherrie,
Thanks for setting up that sample data. Since you didn't send a
config
file, I just used the email you sent to populate one.
I see that in forecast file, you'd like to select UGRD/Z10, VGRD/Z10,
and
TMP/Z2 (meaning 10-meter winds and 2-meter temp). And those match the
record numbers 183, 184, and 177 you were using.
[ISSUE 1]
However the erai_2001090500.grib file doesn't have 10-meter winds or
2-meter temps. It has temp, U, and V from 1000mb up to 10mb, but
nothing
at the surface. So I'd suggest locating an erai file that actually
has
surface values.
***BUT*** just in order to test, I'll just use U, V, and T and 1000mb.
[ISSUE 2]
If you use wgrib to look at your climo data. For the first 15
records, the
timestamp of the data matches the timestamp of the file. But for the
last
5, the timestamp is 6 hours earlier, so on the previous day. That's
why
grid_stat isn't finding a matching climo. So I used the next day's
file
instead so that month and day numbers would match up.
Doing so, we find a matching climo field and get an anomaly
correlation
value.
I think we have some issues to iron out, but here's where I was
running...
*cd /home/John.H.Gotway/MET_Help/fredrick_data_20180207module use
/contrib/modulefilesmodule load met/6.1*
*module load wgribgrid_stat FV3_2001090500.grib erai_2001090500.grib
GridStatConfig_FV3 -outdir out -v 5 -log run_gs.log*
Thanks,
John
On Wed, Feb 7, 2018 at 3:20 PM, Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83987 >
>
> Hi John,
>
> I made a directory on Theia called
> /scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/wcoss/John.
> The model file is FV3_2001090500.grib.
> The observation file is erai_2001090500.grib.
> The climate file is cmean_1d.1959090500.grb
>
> I hope this helps.
> Thanks,
> Sherrie
>
> On Wed, Feb 7, 2018 at 3:07 PM, John Halley Gotway via RT <
> met_help at ucar.edu
> > wrote:
>
> > Hi Sherrie,
> >
> > Thanks for sending some sample output. While the MET tools allow
you set
> > the explicit record number to be used from the GRIB file, I'd
strongly
> > advise against doing that routinely. There's no guarantee that
the order
> > of the GRIB records today will be the same order tomorrow.
> >
> > I'd like to run a sample case on theia using your data to see if I
can
> make
> > some useful suggestions. But I'll need to know the location of
your:
> > (1) forecast file, (2) observation file, and (3) config file. I
already
> > see the path to your climo data.
> >
> > Once I know where I can find this data on theia, I'll let you know
what I
> > find.
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Wed, Feb 7, 2018 at 2:54 PM, Sherrie Fredrick - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Wed Feb 07 14:54:19 2018: Request 83987 was acted upon.
> > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > Queue: met_help
> > > Subject: calculating anomaly correlations with GRIB surface
data
> > > Owner: Nobody
> > > Requestors: sherrie.fredrick at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83987
> >
> > >
> > >
> > > Good Afternoon.
> > >
> > > I am running MET on Theia. The following is an portion of
my
> > > configuration file
> > >
> > >
> > > ////////////////////////////////////////////////////////////
> > > ////////////////////
> > >
> > > // Forecast and observation fields to be verified
> > > //
> > > fcst = {
> > >
> > > field = [
> > > {
> > > name = "UGRD";
> > > level = ["R183"];
> > > },
> > > {
> > > name = "VGRD";
> > > level = ["R184"];
> > > },
> > > {
> > > name = "TMP";
> > > level = ["R177"];
> > > }
> > > ];
> > >
> > > }
> > > ///////////////////////////////////////////////////////////////
> > > obs = {
> > >
> > > field = [
> > > {
> > > name = "10U";
> > > level = ["R1"];
> > > },
> > > {
> > > name = "10V";
> > > level = ["R2"];
> > > },
> > > {
> > > name = "2T";
> > > level = ["R3"];
> > > }
> > > ];
> > >
> > > }
> > > ////////////////////////////////////////////////////////////
> > > ////////////////////
> > >
> > > //
> > > // Climatology mean data
> > > //
> > > climo_mean = {
> > >
> > > file_name =
> > > ["/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> > > climate/cmean_1d.1959051200.grb"];
> > > field = [
> > > {
> > > name = "UGRD";
> > > level = ["R18"];
> > > },
> > > {
> > > name = "VGRD";
> > > level = ["R19"];
> > > },
> > > {
> > > name = "TMP";
> > > level = ["R15"];
> > > }
> > > ];
> > >
> > > regrid = {
> > > method = NEAREST;
> > > width = 1;
> > > vld_thresh = 0.5;
> > > }
> > >
> > > time_interp_method = DW_MEAN;
> > > match_day = TRUE;
> > > time_step = 3600;
> > > }
> > >
> > > ////////////////////////////////////////////////////////////
> > > ////////////////////
> > > The wgrib on my climatology file is :
> > > 15:1540880:d=59030118:TMP:kpds5=11:kpds6=105:kpds7=2:TR=
> > > 51:P1=6:P2=0:TimeU=1:2
> > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > 16:1655016:d=59030118:TMAX:kpds5=15:kpds6=105:kpds7=2:TR=
> > > 51:P1=6:P2=0:TimeU=1:2
> > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > 17:1769152:d=59030118:TMIN:kpds5=16:kpds6=105:kpds7=2:TR=
> > > 51:P1=6:P2=0:TimeU=1:2
> > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > 18:1883288:d=59030118:UGRD:kpds5=33:kpds6=105:kpds7=10:
> > > TR=51:P1=6:P2=0:TimeU=1:10
> > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > >
> > > ;-----------------------------------------------------------
> > >
----------------------------------------------------------------------
> > >
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > > matching VarInfo "TMP/R15" in GRIB file
> > > "/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> > > climate/cmean_1d.1959050200.grb".
> > > DEBUG 3: Found 0 climatology fields.
> > > DEBUG 3: Found 0 climatology mean field(s) for forecast
TMP/R177.
> > >
> > >
> > > It will calculate the rmse and bias but it won't calculate the
anomoly
> > > correlations. I am not sure
> > > of why.
> > >
> > > At your convenience could you please pass along any thoughts you
may
> > have.
> > > I don't have
> > > any problems doing the pressure surface anomaly correlations
with the
> > same
> > > climate file.
> > > All of my data files are in GRIB1 format.
> > >
> > > Many Thanks,
> > > Sherrie
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: calculating anomaly correlations with GRIB surface data
From: Sherrie Fredrick - NOAA Affiliate
Time: Thu Feb 08 08:27:00 2018
Hi John,
Thank your very much for pointing out the time discrepancy. I
never
noticed that. I need to think about
how to deal with that. It is very annoying.
Go ahead and close the help ticket. I am pretty sure that the
problem is time.
Thanks,
Sherrie
On Wed, Feb 7, 2018 at 4:19 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Sherrie,
>
> Thanks for setting up that sample data. Since you didn't send a
config
> file, I just used the email you sent to populate one.
>
> I see that in forecast file, you'd like to select UGRD/Z10,
VGRD/Z10, and
> TMP/Z2 (meaning 10-meter winds and 2-meter temp). And those match
the
> record numbers 183, 184, and 177 you were using.
>
> [ISSUE 1]
> However the erai_2001090500.grib file doesn't have 10-meter winds or
> 2-meter temps. It has temp, U, and V from 1000mb up to 10mb, but
nothing
> at the surface. So I'd suggest locating an erai file that actually
has
> surface values.
>
> ***BUT*** just in order to test, I'll just use U, V, and T and
1000mb.
>
> [ISSUE 2]
> If you use wgrib to look at your climo data. For the first 15
records, the
> timestamp of the data matches the timestamp of the file. But for
the last
> 5, the timestamp is 6 hours earlier, so on the previous day. That's
why
> grid_stat isn't finding a matching climo. So I used the next day's
file
> instead so that month and day numbers would match up.
>
> Doing so, we find a matching climo field and get an anomaly
correlation
> value.
>
> I think we have some issues to iron out, but here's where I was
running...
>
>
>
>
> *cd /home/John.H.Gotway/MET_Help/fredrick_data_20180207module use
> /contrib/modulefilesmodule load met/6.1*
>
>
>
> *module load wgribgrid_stat FV3_2001090500.grib erai_2001090500.grib
> GridStatConfig_FV3 -outdir out -v 5 -log run_gs.log*
> Thanks,
> John
>
> On Wed, Feb 7, 2018 at 3:20 PM, Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83987 >
> >
> > Hi John,
> >
> > I made a directory on Theia called
> > /scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/wcoss/John.
> > The model file is FV3_2001090500.grib.
> > The observation file is erai_2001090500.grib.
> > The climate file is cmean_1d.1959090500.grb
> >
> > I hope this helps.
> > Thanks,
> > Sherrie
> >
> > On Wed, Feb 7, 2018 at 3:07 PM, John Halley Gotway via RT <
> > met_help at ucar.edu
> > > wrote:
> >
> > > Hi Sherrie,
> > >
> > > Thanks for sending some sample output. While the MET tools
allow you
> set
> > > the explicit record number to be used from the GRIB file, I'd
strongly
> > > advise against doing that routinely. There's no guarantee that
the
> order
> > > of the GRIB records today will be the same order tomorrow.
> > >
> > > I'd like to run a sample case on theia using your data to see if
I can
> > make
> > > some useful suggestions. But I'll need to know the location of
your:
> > > (1) forecast file, (2) observation file, and (3) config file. I
> already
> > > see the path to your climo data.
> > >
> > > Once I know where I can find this data on theia, I'll let you
know
> what I
> > > find.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > >
> > > On Wed, Feb 7, 2018 at 2:54 PM, Sherrie Fredrick - NOAA
Affiliate via
> RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Wed Feb 07 14:54:19 2018: Request 83987 was acted upon.
> > > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > Queue: met_help
> > > > Subject: calculating anomaly correlations with GRIB
surface data
> > > > Owner: Nobody
> > > > Requestors: sherrie.fredrick at noaa.gov
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=83987
> > >
> > > >
> > > >
> > > > Good Afternoon.
> > > >
> > > > I am running MET on Theia. The following is an portion of
my
> > > > configuration file
> > > >
> > > >
> > > > ////////////////////////////////////////////////////////////
> > > > ////////////////////
> > > >
> > > > // Forecast and observation fields to be verified
> > > > //
> > > > fcst = {
> > > >
> > > > field = [
> > > > {
> > > > name = "UGRD";
> > > > level = ["R183"];
> > > > },
> > > > {
> > > > name = "VGRD";
> > > > level = ["R184"];
> > > > },
> > > > {
> > > > name = "TMP";
> > > > level = ["R177"];
> > > > }
> > > > ];
> > > >
> > > > }
> > > >
///////////////////////////////////////////////////////////////
> > > > obs = {
> > > >
> > > > field = [
> > > > {
> > > > name = "10U";
> > > > level = ["R1"];
> > > > },
> > > > {
> > > > name = "10V";
> > > > level = ["R2"];
> > > > },
> > > > {
> > > > name = "2T";
> > > > level = ["R3"];
> > > > }
> > > > ];
> > > >
> > > > }
> > > > ////////////////////////////////////////////////////////////
> > > > ////////////////////
> > > >
> > > > //
> > > > // Climatology mean data
> > > > //
> > > > climo_mean = {
> > > >
> > > > file_name =
> > > > ["/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> > > > climate/cmean_1d.1959051200.grb"];
> > > > field = [
> > > > {
> > > > name = "UGRD";
> > > > level = ["R18"];
> > > > },
> > > > {
> > > > name = "VGRD";
> > > > level = ["R19"];
> > > > },
> > > > {
> > > > name = "TMP";
> > > > level = ["R15"];
> > > > }
> > > > ];
> > > >
> > > > regrid = {
> > > > method = NEAREST;
> > > > width = 1;
> > > > vld_thresh = 0.5;
> > > > }
> > > >
> > > > time_interp_method = DW_MEAN;
> > > > match_day = TRUE;
> > > > time_step = 3600;
> > > > }
> > > >
> > > > ////////////////////////////////////////////////////////////
> > > > ////////////////////
> > > > The wgrib on my climatology file is :
> > > > 15:1540880:d=59030118:TMP:kpds5=11:kpds6=105:kpds7=2:TR=
> > > > 51:P1=6:P2=0:TimeU=1:2
> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > > 16:1655016:d=59030118:TMAX:kpds5=15:kpds6=105:kpds7=2:TR=
> > > > 51:P1=6:P2=0:TimeU=1:2
> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > > 17:1769152:d=59030118:TMIN:kpds5=16:kpds6=105:kpds7=2:TR=
> > > > 51:P1=6:P2=0:TimeU=1:2
> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > > 18:1883288:d=59030118:UGRD:kpds5=33:kpds6=105:kpds7=10:
> > > > TR=51:P1=6:P2=0:TimeU=1:10
> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate mean:NAve=40
> > > >
> > > > ;-----------------------------------------------------------
> > > > ------------------------------------------------------------
> ----------
> > > >
> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
> records
> > > > matching VarInfo "TMP/R15" in GRIB file
> > > > "/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
> > > > climate/cmean_1d.1959050200.grb".
> > > > DEBUG 3: Found 0 climatology fields.
> > > > DEBUG 3: Found 0 climatology mean field(s) for forecast
TMP/R177.
> > > >
> > > >
> > > > It will calculate the rmse and bias but it won't calculate the
> anomoly
> > > > correlations. I am not sure
> > > > of why.
> > > >
> > > > At your convenience could you please pass along any thoughts
you may
> > > have.
> > > > I don't have
> > > > any problems doing the pressure surface anomaly correlations
with the
> > > same
> > > > climate file.
> > > > All of my data files are in GRIB1 format.
> > > >
> > > > Many Thanks,
> > > > Sherrie
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: calculating anomaly correlations with GRIB surface data
From: Sherrie Fredrick - NOAA Affiliate
Time: Thu Feb 08 11:04:48 2018
Hi again John,
Sorry to bother you again, but I still don't have it working.
I now have the erai file in the
/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/wcoss/John.
I use the following to run MET.
/scratch4/BMC/dtc/MET/met-6.0/bin/grid_stat FV3_2001090500.grib
erai_20010905.grib surface_grid_stat_config -v 3
Thanks,
Sherrie
On Thu, Feb 8, 2018 at 8:26 AM, Sherrie Fredrick - NOAA Affiliate <
sherrie.fredrick at noaa.gov> wrote:
> Hi John,
>
> Thank your very much for pointing out the time discrepancy. I
never
> noticed that. I need to think about
> how to deal with that. It is very annoying.
>
> Go ahead and close the help ticket. I am pretty sure that the
> problem is time.
>
> Thanks,
> Sherrie
>
>
> On Wed, Feb 7, 2018 at 4:19 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Sherrie,
>>
>> Thanks for setting up that sample data. Since you didn't send a
config
>> file, I just used the email you sent to populate one.
>>
>> I see that in forecast file, you'd like to select UGRD/Z10,
VGRD/Z10, and
>> TMP/Z2 (meaning 10-meter winds and 2-meter temp). And those match
the
>> record numbers 183, 184, and 177 you were using.
>>
>> [ISSUE 1]
>> However the erai_2001090500.grib file doesn't have 10-meter winds
or
>> 2-meter temps. It has temp, U, and V from 1000mb up to 10mb, but
nothing
>> at the surface. So I'd suggest locating an erai file that actually
has
>> surface values.
>>
>> ***BUT*** just in order to test, I'll just use U, V, and T and
1000mb.
>>
>> [ISSUE 2]
>> If you use wgrib to look at your climo data. For the first 15
records,
>> the
>> timestamp of the data matches the timestamp of the file. But for
the last
>> 5, the timestamp is 6 hours earlier, so on the previous day.
That's why
>> grid_stat isn't finding a matching climo. So I used the next day's
file
>> instead so that month and day numbers would match up.
>>
>> Doing so, we find a matching climo field and get an anomaly
correlation
>> value.
>>
>> I think we have some issues to iron out, but here's where I was
running...
>>
>>
>>
>>
>> *cd /home/John.H.Gotway/MET_Help/fredrick_data_20180207module use
>> /contrib/modulefilesmodule load met/6.1*
>>
>>
>>
>> *module load wgribgrid_stat FV3_2001090500.grib
erai_2001090500.grib
>> GridStatConfig_FV3 -outdir out -v 5 -log run_gs.log*
>> Thanks,
>> John
>>
>> On Wed, Feb 7, 2018 at 3:20 PM, Sherrie Fredrick - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83987 >
>> >
>> > Hi John,
>> >
>> > I made a directory on Theia called
>> > /scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/wcoss/John.
>> > The model file is FV3_2001090500.grib.
>> > The observation file is erai_2001090500.grib.
>> > The climate file is cmean_1d.1959090500.grb
>> >
>> > I hope this helps.
>> > Thanks,
>> > Sherrie
>> >
>> > On Wed, Feb 7, 2018 at 3:07 PM, John Halley Gotway via RT <
>> > met_help at ucar.edu
>> > > wrote:
>> >
>> > > Hi Sherrie,
>> > >
>> > > Thanks for sending some sample output. While the MET tools
allow you
>> set
>> > > the explicit record number to be used from the GRIB file, I'd
strongly
>> > > advise against doing that routinely. There's no guarantee that
the
>> order
>> > > of the GRIB records today will be the same order tomorrow.
>> > >
>> > > I'd like to run a sample case on theia using your data to see
if I can
>> > make
>> > > some useful suggestions. But I'll need to know the location of
your:
>> > > (1) forecast file, (2) observation file, and (3) config file.
I
>> already
>> > > see the path to your climo data.
>> > >
>> > > Once I know where I can find this data on theia, I'll let you
know
>> what I
>> > > find.
>> > >
>> > > Thanks,
>> > > John Halley Gotway
>> > >
>> > >
>> > > On Wed, Feb 7, 2018 at 2:54 PM, Sherrie Fredrick - NOAA
Affiliate via
>> RT
>> > <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > Wed Feb 07 14:54:19 2018: Request 83987 was acted upon.
>> > > > Transaction: Ticket created by sherrie.fredrick at noaa.gov
>> > > > Queue: met_help
>> > > > Subject: calculating anomaly correlations with GRIB
surface
>> data
>> > > > Owner: Nobody
>> > > > Requestors: sherrie.fredrick at noaa.gov
>> > > > Status: new
>> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>> ket/Display.html?id=83987
>> > >
>> > > >
>> > > >
>> > > > Good Afternoon.
>> > > >
>> > > > I am running MET on Theia. The following is an portion
of my
>> > > > configuration file
>> > > >
>> > > >
>> > > > ////////////////////////////////////////////////////////////
>> > > > ////////////////////
>> > > >
>> > > > // Forecast and observation fields to be verified
>> > > > //
>> > > > fcst = {
>> > > >
>> > > > field = [
>> > > > {
>> > > > name = "UGRD";
>> > > > level = ["R183"];
>> > > > },
>> > > > {
>> > > > name = "VGRD";
>> > > > level = ["R184"];
>> > > > },
>> > > > {
>> > > > name = "TMP";
>> > > > level = ["R177"];
>> > > > }
>> > > > ];
>> > > >
>> > > > }
>> > > >
///////////////////////////////////////////////////////////////
>> > > > obs = {
>> > > >
>> > > > field = [
>> > > > {
>> > > > name = "10U";
>> > > > level = ["R1"];
>> > > > },
>> > > > {
>> > > > name = "10V";
>> > > > level = ["R2"];
>> > > > },
>> > > > {
>> > > > name = "2T";
>> > > > level = ["R3"];
>> > > > }
>> > > > ];
>> > > >
>> > > > }
>> > > > ////////////////////////////////////////////////////////////
>> > > > ////////////////////
>> > > >
>> > > > //
>> > > > // Climatology mean data
>> > > > //
>> > > > climo_mean = {
>> > > >
>> > > > file_name =
>> > > > ["/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
>> > > > climate/cmean_1d.1959051200.grb"];
>> > > > field = [
>> > > > {
>> > > > name = "UGRD";
>> > > > level = ["R18"];
>> > > > },
>> > > > {
>> > > > name = "VGRD";
>> > > > level = ["R19"];
>> > > > },
>> > > > {
>> > > > name = "TMP";
>> > > > level = ["R15"];
>> > > > }
>> > > > ];
>> > > >
>> > > > regrid = {
>> > > > method = NEAREST;
>> > > > width = 1;
>> > > > vld_thresh = 0.5;
>> > > > }
>> > > >
>> > > > time_interp_method = DW_MEAN;
>> > > > match_day = TRUE;
>> > > > time_step = 3600;
>> > > > }
>> > > >
>> > > > ////////////////////////////////////////////////////////////
>> > > > ////////////////////
>> > > > The wgrib on my climatology file is :
>> > > > 15:1540880:d=59030118:TMP:kpds5=11:kpds6=105:kpds7=2:TR=
>> > > > 51:P1=6:P2=0:TimeU=1:2
>> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate
mean:NAve=40
>> > > > 16:1655016:d=59030118:TMAX:kpds5=15:kpds6=105:kpds7=2:TR=
>> > > > 51:P1=6:P2=0:TimeU=1:2
>> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate
mean:NAve=40
>> > > > 17:1769152:d=59030118:TMIN:kpds5=16:kpds6=105:kpds7=2:TR=
>> > > > 51:P1=6:P2=0:TimeU=1:2
>> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate
mean:NAve=40
>> > > > 18:1883288:d=59030118:UGRD:kpds5=33:kpds6=105:kpds7=10:
>> > > > TR=51:P1=6:P2=0:TimeU=1:10
>> > > > m above gnd:clim? p1=6? 0hr?:ens-0::daily climate
mean:NAve=40
>> > > >
>> > > > ;-----------------------------------------------------------
>> > > > ------------------------------------------------------------
>> ----------
>> > > >
>> > > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
>> records
>> > > > matching VarInfo "TMP/R15" in GRIB file
>> > > > "/scratch4/BMC/refcst2/Sherrie.Fredrick/gfs/run/
>> > > > climate/cmean_1d.1959050200.grb".
>> > > > DEBUG 3: Found 0 climatology fields.
>> > > > DEBUG 3: Found 0 climatology mean field(s) for forecast
TMP/R177.
>> > > >
>> > > >
>> > > > It will calculate the rmse and bias but it won't calculate
the
>> anomoly
>> > > > correlations. I am not sure
>> > > > of why.
>> > > >
>> > > > At your convenience could you please pass along any thoughts
you may
>> > > have.
>> > > > I don't have
>> > > > any problems doing the pressure surface anomaly correlations
with
>> the
>> > > same
>> > > > climate file.
>> > > > All of my data files are in GRIB1 format.
>> > > >
>> > > > Many Thanks,
>> > > > Sherrie
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
>
------------------------------------------------
More information about the Met_help
mailing list