[Met_help] [rt.rap.ucar.edu #69036] History for MET 4.1: Point_stat issue
Julie Prestopnik via RT
met_help at ucar.edu
Tue Jan 6 11:09:11 MST 2015
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello again,
I'm trying to find a problem with my inability to run point_stat for a pair of NetCDF files. Here's what I am seeing from MET:
DEBUG 1: Default Config File: /home/qcteam/METv4.1/data/config/PointStatConfig_default
DEBUG 1: User Config File: /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_NcMet".
NetCDF: Attribute not found
Here's what I defined for the fields in the point_stat config file:
fcst = {
message_type = [ "ADPSFC" ];
field = [
{
name = "PM2_5_DRY(0,0,*,*)";
cat_thresh = [ >=0.0 ];
}
];
};
obs = {
field = [
{
name = "obs_arr(*,4)";
cat_thresh = [ >=0.0 ];
}
];
};
Within the forecast NetCDF file the following values are set:
dimensions:
Time = UNLIMITED ; // (1 currently)
west_east = 249 ;
south_north = 149 ;
bottom_top = 56 ;
float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
PM2_5_DRY:FieldType = 104 ;
PM2_5_DRY:MemoryOrder = "XYZ" ;
PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
PM2_5_DRY:units = "ug m^-3" ;
PM2_5_DRY:stagger = "" ;
PM2_5_DRY:coordinates = "XLONG XLAT" ;
Within the observation file:
dimensions:
obs_arr_len = 5 ;
nhdr = 1820 ;
nobs = UNLIMITED ; // (1820 currently)
float obs_arr(nobs, obs_arr_len) ;
obs_arr:long_name = "array of observation values" ;
obs_arr:_fill_value = -9999.f ;
obs_arr:columns = "hdr_id gc lvl hgt ob" ;
obs_arr:hdr_id_long_name = "index of matching header data" ;
obs_arr:gc_long_name = "grib code corresponding to the observation type" ;
obs_arr:lvl_long_name = "pressure level (hPa) or accumulation interval (sec)" ;
obs_arr:hgt_long_name = "height in meters above sea level or ground level (msl or agl)" ;
obs_arr:ob_long_name = "observation value" ;
Additional information:
"ADPSFC" is the message type within the header of the observation files, but it does not appear in the forecast file anywhere.
I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0 represents the first member of an array and not 1. The second 0 indicates the surface data should be used, the first of 56 levels. I used "*" to represent the X and Y nature of the grid.
I am using obs_arr(*,4) because there are 5 data fields, and only the 5th one (numbered 4 when starting from 0) is the actual data field of interest. The first * indicates to use all 1820 observations.
I built the observation file using 'ascii2nc'; the forecast file was built separately by a colleague.
The observation file is ~350KB so I could easily e-mail it, but the model data is ~1.3GB, so obviously I can't easily send that along for you to test with. Perhaps I could FTP it somewhere you could try to replicate the error?
Do you see anything here that I've made a mistake with? I've spent a couple hours staring at code and reading the user's manual; I can't find any guidance as to where I made the mistake. I'm hoping you can help.
Thanks,
Matt
// SIGNED //
Matthew C. Sittel
Associate Scientist
University Corporation for Atmospheric Research
16WS/WXN, Offutt AFB, NE
(402) 294-3473 DSN: 271-3473
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Mon Sep 22 09:36:33 2014
Hi Matt. I see that you are having problems running point_stat.
Thank you
for including the details of the situation. Could you please follow
these
instructions for posting the forecast file, observation file, and
config
file to our ftp site:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
I can take a closer look and try to resolve the problem.
Thanks,
Julie
On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> Queue: met_help
> Subject: MET 4.1: Point_stat issue
> Owner: Nobody
> Requestors: matthew.sittel.ctr at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
>
> Hello again,
>
> I'm trying to find a problem with my inability to run point_stat for
a
> pair of NetCDF files. Here's what I am seeing from MET:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Here's what I defined for the fields in the point_stat config file:
>
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "PM2_5_DRY(0,0,*,*)";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> obs = {
> field = [
> {
> name = "obs_arr(*,4)";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> Within the forecast NetCDF file the following values are set:
>
> dimensions:
> Time = UNLIMITED ; // (1 currently)
> west_east = 249 ;
> south_north = 149 ;
> bottom_top = 56 ;
>
> float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
> PM2_5_DRY:FieldType = 104 ;
> PM2_5_DRY:MemoryOrder = "XYZ" ;
> PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> PM2_5_DRY:units = "ug m^-3" ;
> PM2_5_DRY:stagger = "" ;
> PM2_5_DRY:coordinates = "XLONG XLAT" ;
>
> Within the observation file:
>
> dimensions:
> obs_arr_len = 5 ;
> nhdr = 1820 ;
> nobs = UNLIMITED ; // (1820 currently)
>
> float obs_arr(nobs, obs_arr_len) ;
> obs_arr:long_name = "array of observation values" ;
> obs_arr:_fill_value = -9999.f ;
> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
> obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
> obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
> obs_arr:hgt_long_name = "height in meters above sea
level
> or ground level (msl or agl)" ;
> obs_arr:ob_long_name = "observation value" ;
>
> Additional information:
>
> "ADPSFC" is the message type within the header of the observation
files,
> but it does not appear in the forecast file anywhere.
> I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents the
> first member of an array and not 1. The second 0 indicates the
surface
> data should be used, the first of 56 levels. I used "*" to
represent the X
> and Y nature of the grid.
> I am using obs_arr(*,4) because there are 5 data fields, and only
the 5th
> one (numbered 4 when starting from 0) is the actual data field of
> interest. The first * indicates to use all 1820 observations.
> I built the observation file using 'ascii2nc'; the forecast file was
built
> separately by a colleague.
>
> The observation file is ~350KB so I could easily e-mail it, but the
model
> data is ~1.3GB, so obviously I can't easily send that along for you
to test
> with. Perhaps I could FTP it somewhere you could try to replicate
the
> error?
>
> Do you see anything here that I've made a mistake with? I've spent
a
> couple hours staring at code and reading the user's manual; I can't
find
> any guidance as to where I made the mistake. I'm hoping you can
help.
>
> Thanks,
> Matt
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research
> 16WS/WXN, Offutt AFB, NE
> (402) 294-3473 DSN: 271-3473
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 10:01:33 2014
Hi Julie,
I have placed the files in the directory-let me know if you can't read
them. I'm sorry the model file size is so huge... but it's not my
model so I have no say in the matter! If I did I would have asked for
GriB data instead since I have had much more success with GriB in MET
than I have NetCDF. Something about the notation the config file
needs always trips me up. I was sure I had figured it out this time
but the code falls flat when it hits the model data that I'm not sure
what it needs.
I *think* the model file is transferring. Full file size is 1 310 160
288 bytes. I need to leave for a couple hours so I can't monitor it.
I did push the observation and config files already so hopefully you
see those-the last file will be there once the full file size is
reached. With no diagnostics returned I'm not sure how slow the
transfer rate is...
Thanks,
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 10:37 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Hi Matt. I see that you are having problems running point_stat.
Thank you for including the details of the situation. Could you
please follow these instructions for posting the forecast file,
observation file, and config file to our ftp site:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
I can take a closer look and try to resolve the problem.
Thanks,
Julie
On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> Queue: met_help
> Subject: MET 4.1: Point_stat issue
> Owner: Nobody
> Requestors: matthew.sittel.ctr at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> >
>
>
> Hello again,
>
> I'm trying to find a problem with my inability to run point_stat for
a
> pair of NetCDF files. Here's what I am seeing from MET:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Here's what I defined for the fields in the point_stat config file:
>
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "PM2_5_DRY(0,0,*,*)";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> obs = {
> field = [
> {
> name = "obs_arr(*,4)";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> Within the forecast NetCDF file the following values are set:
>
> dimensions:
> Time = UNLIMITED ; // (1 currently)
> west_east = 249 ;
> south_north = 149 ;
> bottom_top = 56 ;
>
> float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
> PM2_5_DRY:FieldType = 104 ;
> PM2_5_DRY:MemoryOrder = "XYZ" ;
> PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> PM2_5_DRY:units = "ug m^-3" ;
> PM2_5_DRY:stagger = "" ;
> PM2_5_DRY:coordinates = "XLONG XLAT" ;
>
> Within the observation file:
>
> dimensions:
> obs_arr_len = 5 ;
> nhdr = 1820 ;
> nobs = UNLIMITED ; // (1820 currently)
>
> float obs_arr(nobs, obs_arr_len) ;
> obs_arr:long_name = "array of observation values" ;
> obs_arr:_fill_value = -9999.f ;
> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
> obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
> obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
> obs_arr:hgt_long_name = "height in meters above sea
> level or ground level (msl or agl)" ;
> obs_arr:ob_long_name = "observation value" ;
>
> Additional information:
>
> "ADPSFC" is the message type within the header of the observation
> files, but it does not appear in the forecast file anywhere.
> I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents
> the first member of an array and not 1. The second 0 indicates the
> surface data should be used, the first of 56 levels. I used "*" to
> represent the X and Y nature of the grid.
> I am using obs_arr(*,4) because there are 5 data fields, and only
the
> 5th one (numbered 4 when starting from 0) is the actual data field
of
> interest. The first * indicates to use all 1820 observations.
> I built the observation file using 'ascii2nc'; the forecast file was
> built separately by a colleague.
>
> The observation file is ~350KB so I could easily e-mail it, but the
> model data is ~1.3GB, so obviously I can't easily send that along
for
> you to test with. Perhaps I could FTP it somewhere you could try to
> replicate the error?
>
> Do you see anything here that I've made a mistake with? I've spent
a
> couple hours staring at code and reading the user's manual; I can't
> find any guidance as to where I made the mistake. I'm hoping you
can help.
>
> Thanks,
> Matt
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
> NE
> (402) 294-3473 DSN: 271-3473
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 12:06:07 2014
Looks like I need to be more patient. I assumed when I got back to
work that the file transfer was hung so I aborted it. Turns out it
has transferred 430 MB of the full 1.3 GB of the model data file. So
looks like transfer will take 6-8 hours at this rate.
With any luck it will run to completion... sorry for the delay, but I
don't know any better way to get you the file short of mailing a DVD!
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 10:37 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Hi Matt. I see that you are having problems running point_stat.
Thank you for including the details of the situation. Could you
please follow these instructions for posting the forecast file,
observation file, and config file to our ftp site:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
I can take a closer look and try to resolve the problem.
Thanks,
Julie
On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> Queue: met_help
> Subject: MET 4.1: Point_stat issue
> Owner: Nobody
> Requestors: matthew.sittel.ctr at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> >
>
>
> Hello again,
>
> I'm trying to find a problem with my inability to run point_stat for
a
> pair of NetCDF files. Here's what I am seeing from MET:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Here's what I defined for the fields in the point_stat config file:
>
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "PM2_5_DRY(0,0,*,*)";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> obs = {
> field = [
> {
> name = "obs_arr(*,4)";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> Within the forecast NetCDF file the following values are set:
>
> dimensions:
> Time = UNLIMITED ; // (1 currently)
> west_east = 249 ;
> south_north = 149 ;
> bottom_top = 56 ;
>
> float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
> PM2_5_DRY:FieldType = 104 ;
> PM2_5_DRY:MemoryOrder = "XYZ" ;
> PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> PM2_5_DRY:units = "ug m^-3" ;
> PM2_5_DRY:stagger = "" ;
> PM2_5_DRY:coordinates = "XLONG XLAT" ;
>
> Within the observation file:
>
> dimensions:
> obs_arr_len = 5 ;
> nhdr = 1820 ;
> nobs = UNLIMITED ; // (1820 currently)
>
> float obs_arr(nobs, obs_arr_len) ;
> obs_arr:long_name = "array of observation values" ;
> obs_arr:_fill_value = -9999.f ;
> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
> obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
> obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
> obs_arr:hgt_long_name = "height in meters above sea
> level or ground level (msl or agl)" ;
> obs_arr:ob_long_name = "observation value" ;
>
> Additional information:
>
> "ADPSFC" is the message type within the header of the observation
> files, but it does not appear in the forecast file anywhere.
> I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents
> the first member of an array and not 1. The second 0 indicates the
> surface data should be used, the first of 56 levels. I used "*" to
> represent the X and Y nature of the grid.
> I am using obs_arr(*,4) because there are 5 data fields, and only
the
> 5th one (numbered 4 when starting from 0) is the actual data field
of
> interest. The first * indicates to use all 1820 observations.
> I built the observation file using 'ascii2nc'; the forecast file was
> built separately by a colleague.
>
> The observation file is ~350KB so I could easily e-mail it, but the
> model data is ~1.3GB, so obviously I can't easily send that along
for
> you to test with. Perhaps I could FTP it somewhere you could try to
> replicate the error?
>
> Do you see anything here that I've made a mistake with? I've spent
a
> couple hours staring at code and reading the user's manual; I can't
> find any guidance as to where I made the mistake. I'm hoping you
can help.
>
> Thanks,
> Matt
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
> NE
> (402) 294-3473 DSN: 271-3473
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Mon Sep 22 12:08:19 2014
Ah, that's too bad. I had been watching it slowly come in. I'll keep
an
eye out for it.
Julie
On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Looks like I need to be more patient. I assumed when I got back to
work
> that the file transfer was hung so I aborted it. Turns out it has
> transferred 430 MB of the full 1.3 GB of the model data file. So
looks
> like transfer will take 6-8 hours at this rate.
>
> With any luck it will run to completion... sorry for the delay, but
I
> don't know any better way to get you the file short of mailing a
DVD!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 10:37 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. I see that you are having problems running point_stat.
Thank
> you for including the details of the situation. Could you please
follow
> these instructions for posting the forecast file, observation file,
and
> config file to our ftp site:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> I can take a closer look and try to resolve the problem.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > Queue: met_help
> > Subject: MET 4.1: Point_stat issue
> > Owner: Nobody
> > Requestors: matthew.sittel.ctr at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > >
> >
> >
> > Hello again,
> >
> > I'm trying to find a problem with my inability to run point_stat
for a
> > pair of NetCDF files. Here's what I am seeing from MET:
> >
> > DEBUG 1: Default Config File:
> > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > DEBUG 1: User Config File:
> > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > NetCDF: Attribute not found
> >
> > Here's what I defined for the fields in the point_stat config
file:
> >
> > fcst = {
> > message_type = [ "ADPSFC" ];
> > field = [
> > {
> > name = "PM2_5_DRY(0,0,*,*)";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > obs = {
> > field = [
> > {
> > name = "obs_arr(*,4)";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > Within the forecast NetCDF file the following values are set:
> >
> > dimensions:
> > Time = UNLIMITED ; // (1 currently)
> > west_east = 249 ;
> > south_north = 149 ;
> > bottom_top = 56 ;
> >
> > float PM2_5_DRY(Time, bottom_top, south_north, west_east)
;
> > PM2_5_DRY:FieldType = 104 ;
> > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> > PM2_5_DRY:units = "ug m^-3" ;
> > PM2_5_DRY:stagger = "" ;
> > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> >
> > Within the observation file:
> >
> > dimensions:
> > obs_arr_len = 5 ;
> > nhdr = 1820 ;
> > nobs = UNLIMITED ; // (1820 currently)
> >
> > float obs_arr(nobs, obs_arr_len) ;
> > obs_arr:long_name = "array of observation values"
;
> > obs_arr:_fill_value = -9999.f ;
> > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > obs_arr:hdr_id_long_name = "index of matching
header
> data"
> > ;
> > obs_arr:gc_long_name = "grib code corresponding to
the
> > observation type" ;
> > obs_arr:lvl_long_name = "pressure level (hPa) or
> > accumulation interval (sec)" ;
> > obs_arr:hgt_long_name = "height in meters above
sea
> > level or ground level (msl or agl)" ;
> > obs_arr:ob_long_name = "observation value" ;
> >
> > Additional information:
> >
> > "ADPSFC" is the message type within the header of the observation
> > files, but it does not appear in the forecast file anywhere.
> > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents
> > the first member of an array and not 1. The second 0 indicates
the
> > surface data should be used, the first of 56 levels. I used "*"
to
> > represent the X and Y nature of the grid.
> > I am using obs_arr(*,4) because there are 5 data fields, and only
the
> > 5th one (numbered 4 when starting from 0) is the actual data field
of
> > interest. The first * indicates to use all 1820 observations.
> > I built the observation file using 'ascii2nc'; the forecast file
was
> > built separately by a colleague.
> >
> > The observation file is ~350KB so I could easily e-mail it, but
the
> > model data is ~1.3GB, so obviously I can't easily send that along
for
> > you to test with. Perhaps I could FTP it somewhere you could try
to
> > replicate the error?
> >
> > Do you see anything here that I've made a mistake with? I've
spent a
> > couple hours staring at code and reading the user's manual; I
can't
> > find any guidance as to where I made the mistake. I'm hoping you
can
> help.
> >
> > Thanks,
> > Matt
> >
> > // SIGNED //
> > Matthew C. Sittel
> > Associate Scientist
> > University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
> > NE
> > (402) 294-3473 DSN: 271-3473
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 12:14:36 2014
I opened a separate window and did a list on the directory but it said
nothing was in there... which left me even more confused. I have
restarted the FTP but looks like I have no way on my end to monitor
progress... so I'll leave it running all night if I must and hope it
continues running.
If you get a chance later this afternoon, please let me know how far
along the file is so I can estimate transfer speed for future
reference.
Thanks!
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 1:08 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Ah, that's too bad. I had been watching it slowly come in. I'll keep
an eye out for it.
Julie
On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Looks like I need to be more patient. I assumed when I got back to
> work that the file transfer was hung so I aborted it. Turns out it
> has transferred 430 MB of the full 1.3 GB of the model data file.
So
> looks like transfer will take 6-8 hours at this rate.
>
> With any luck it will run to completion... sorry for the delay, but
I
> don't know any better way to get you the file short of mailing a
DVD!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 10:37 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. I see that you are having problems running point_stat.
> Thank you for including the details of the situation. Could you
> please follow these instructions for posting the forecast file,
> observation file, and config file to our ftp site:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> I can take a closer look and try to resolve the problem.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > Queue: met_help
> > Subject: MET 4.1: Point_stat issue
> > Owner: Nobody
> > Requestors: matthew.sittel.ctr at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > >
> >
> >
> > Hello again,
> >
> > I'm trying to find a problem with my inability to run point_stat
for
> > a pair of NetCDF files. Here's what I am seeing from MET:
> >
> > DEBUG 1: Default Config File:
> > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > DEBUG 1: User Config File:
> > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > NetCDF: Attribute not found
> >
> > Here's what I defined for the fields in the point_stat config
file:
> >
> > fcst = {
> > message_type = [ "ADPSFC" ];
> > field = [
> > {
> > name = "PM2_5_DRY(0,0,*,*)";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > obs = {
> > field = [
> > {
> > name = "obs_arr(*,4)";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > Within the forecast NetCDF file the following values are set:
> >
> > dimensions:
> > Time = UNLIMITED ; // (1 currently)
> > west_east = 249 ;
> > south_north = 149 ;
> > bottom_top = 56 ;
> >
> > float PM2_5_DRY(Time, bottom_top, south_north, west_east)
;
> > PM2_5_DRY:FieldType = 104 ;
> > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> > PM2_5_DRY:units = "ug m^-3" ;
> > PM2_5_DRY:stagger = "" ;
> > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> >
> > Within the observation file:
> >
> > dimensions:
> > obs_arr_len = 5 ;
> > nhdr = 1820 ;
> > nobs = UNLIMITED ; // (1820 currently)
> >
> > float obs_arr(nobs, obs_arr_len) ;
> > obs_arr:long_name = "array of observation values"
;
> > obs_arr:_fill_value = -9999.f ;
> > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > obs_arr:hdr_id_long_name = "index of matching
header
> data"
> > ;
> > obs_arr:gc_long_name = "grib code corresponding to
> > the observation type" ;
> > obs_arr:lvl_long_name = "pressure level (hPa) or
> > accumulation interval (sec)" ;
> > obs_arr:hgt_long_name = "height in meters above
sea
> > level or ground level (msl or agl)" ;
> > obs_arr:ob_long_name = "observation value" ;
> >
> > Additional information:
> >
> > "ADPSFC" is the message type within the header of the observation
> > files, but it does not appear in the forecast file anywhere.
> > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > represents the first member of an array and not 1. The second 0
> > indicates the surface data should be used, the first of 56 levels.
> > I used "*" to represent the X and Y nature of the grid.
> > I am using obs_arr(*,4) because there are 5 data fields, and only
> > the 5th one (numbered 4 when starting from 0) is the actual data
> > field of interest. The first * indicates to use all 1820
observations.
> > I built the observation file using 'ascii2nc'; the forecast file
was
> > built separately by a colleague.
> >
> > The observation file is ~350KB so I could easily e-mail it, but
the
> > model data is ~1.3GB, so obviously I can't easily send that along
> > for you to test with. Perhaps I could FTP it somewhere you could
> > try to replicate the error?
> >
> > Do you see anything here that I've made a mistake with? I've
spent
> > a couple hours staring at code and reading the user's manual; I
> > can't find any guidance as to where I made the mistake. I'm
hoping
> > you can
> help.
> >
> > Thanks,
> > Matt
> >
> > // SIGNED //
> > Matthew C. Sittel
> > Associate Scientist
> > University Corporation for Atmospheric Research 16WS/WXN, Offutt
> > AFB, NE
> > (402) 294-3473 DSN: 271-3473
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Mon Sep 22 12:18:13 2014
I understand your confusion. The file was being written to
incoming/irap/met_help/ as opposed to
incoming/irap/met_help/sittel_data/
where it is being written to now. I will get back to you this
afternoon
and let you know how far along it is.
Julie
On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> I opened a separate window and did a list on the directory but it
said
> nothing was in there... which left me even more confused. I have
> restarted the FTP but looks like I have no way on my end to monitor
> progress... so I'll leave it running all night if I must and hope it
> continues running.
>
> If you get a chance later this afternoon, please let me know how far
along
> the file is so I can estimate transfer speed for future reference.
>
> Thanks!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:08 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Ah, that's too bad. I had been watching it slowly come in. I'll
keep an
> eye out for it.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Looks like I need to be more patient. I assumed when I got back
to
> > work that the file transfer was hung so I aborted it. Turns out
it
> > has transferred 430 MB of the full 1.3 GB of the model data file.
So
> > looks like transfer will take 6-8 hours at this rate.
> >
> > With any luck it will run to completion... sorry for the delay,
but I
> > don't know any better way to get you the file short of mailing a
DVD!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 10:37 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt. I see that you are having problems running point_stat.
> > Thank you for including the details of the situation. Could you
> > please follow these instructions for posting the forecast file,
> > observation file, and config file to our ftp site:
> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > I can take a closer look and try to resolve the problem.
> >
> > Thanks,
> > Julie
> >
> > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > Queue: met_help
> > > Subject: MET 4.1: Point_stat issue
> > > Owner: Nobody
> > > Requestors: matthew.sittel.ctr at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > >
> > >
> > >
> > > Hello again,
> > >
> > > I'm trying to find a problem with my inability to run point_stat
for
> > > a pair of NetCDF files. Here's what I am seeing from MET:
> > >
> > > DEBUG 1: Default Config File:
> > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > DEBUG 1: User Config File:
> > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > > Met2dDataFile object of type "FileType_NcMet".
> > > NetCDF: Attribute not found
> > >
> > > Here's what I defined for the fields in the point_stat config
file:
> > >
> > > fcst = {
> > > message_type = [ "ADPSFC" ];
> > > field = [
> > > {
> > > name = "PM2_5_DRY(0,0,*,*)";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "obs_arr(*,4)";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > Within the forecast NetCDF file the following values are set:
> > >
> > > dimensions:
> > > Time = UNLIMITED ; // (1 currently)
> > > west_east = 249 ;
> > > south_north = 149 ;
> > > bottom_top = 56 ;
> > >
> > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > PM2_5_DRY:FieldType = 104 ;
> > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > PM2_5_DRY:description = "pm2.5 aerosol dry mass"
;
> > > PM2_5_DRY:units = "ug m^-3" ;
> > > PM2_5_DRY:stagger = "" ;
> > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > >
> > > Within the observation file:
> > >
> > > dimensions:
> > > obs_arr_len = 5 ;
> > > nhdr = 1820 ;
> > > nobs = UNLIMITED ; // (1820 currently)
> > >
> > > float obs_arr(nobs, obs_arr_len) ;
> > > obs_arr:long_name = "array of observation
values" ;
> > > obs_arr:_fill_value = -9999.f ;
> > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > obs_arr:hdr_id_long_name = "index of matching
header
> > data"
> > > ;
> > > obs_arr:gc_long_name = "grib code corresponding
to
> > > the observation type" ;
> > > obs_arr:lvl_long_name = "pressure level (hPa) or
> > > accumulation interval (sec)" ;
> > > obs_arr:hgt_long_name = "height in meters above
sea
> > > level or ground level (msl or agl)" ;
> > > obs_arr:ob_long_name = "observation value" ;
> > >
> > > Additional information:
> > >
> > > "ADPSFC" is the message type within the header of the
observation
> > > files, but it does not appear in the forecast file anywhere.
> > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > represents the first member of an array and not 1. The second 0
> > > indicates the surface data should be used, the first of 56
levels.
> > > I used "*" to represent the X and Y nature of the grid.
> > > I am using obs_arr(*,4) because there are 5 data fields, and
only
> > > the 5th one (numbered 4 when starting from 0) is the actual data
> > > field of interest. The first * indicates to use all 1820
observations.
> > > I built the observation file using 'ascii2nc'; the forecast file
was
> > > built separately by a colleague.
> > >
> > > The observation file is ~350KB so I could easily e-mail it, but
the
> > > model data is ~1.3GB, so obviously I can't easily send that
along
> > > for you to test with. Perhaps I could FTP it somewhere you
could
> > > try to replicate the error?
> > >
> > > Do you see anything here that I've made a mistake with? I've
spent
> > > a couple hours staring at code and reading the user's manual; I
> > > can't find any guidance as to where I made the mistake. I'm
hoping
> > > you can
> > help.
> > >
> > > Thanks,
> > > Matt
> > >
> > > // SIGNED //
> > > Matthew C. Sittel
> > > Associate Scientist
> > > University Corporation for Atmospheric Research 16WS/WXN, Offutt
> > > AFB, NE
> > > (402) 294-3473 DSN: 271-3473
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 12:26:10 2014
Thanks-I got it to work on my end so you don't have to monitor it.
Sorry about the directory mixup-I created the directory and thought I
was putting the files there!
58 MB right now thus far I see... let's hope it works!
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 1:18 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
I understand your confusion. The file was being written to
incoming/irap/met_help/ as opposed to
incoming/irap/met_help/sittel_data/
where it is being written to now. I will get back to you this
afternoon and let you know how far along it is.
Julie
On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> I opened a separate window and did a list on the directory but it
said
> nothing was in there... which left me even more confused. I have
> restarted the FTP but looks like I have no way on my end to monitor
> progress... so I'll leave it running all night if I must and hope it
> continues running.
>
> If you get a chance later this afternoon, please let me know how far
> along the file is so I can estimate transfer speed for future
reference.
>
> Thanks!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:08 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Ah, that's too bad. I had been watching it slowly come in. I'll
keep
> an eye out for it.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Looks like I need to be more patient. I assumed when I got back
to
> > work that the file transfer was hung so I aborted it. Turns out
it
> > has transferred 430 MB of the full 1.3 GB of the model data file.
> > So looks like transfer will take 6-8 hours at this rate.
> >
> > With any luck it will run to completion... sorry for the delay,
but
> > I don't know any better way to get you the file short of mailing a
DVD!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 10:37 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt. I see that you are having problems running point_stat.
> > Thank you for including the details of the situation. Could you
> > please follow these instructions for posting the forecast file,
> > observation file, and config file to our ftp site:
> > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > I can take a closer look and try to resolve the problem.
> >
> > Thanks,
> > Julie
> >
> > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > Queue: met_help
> > > Subject: MET 4.1: Point_stat issue
> > > Owner: Nobody
> > > Requestors: matthew.sittel.ctr at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > >
> > >
> > >
> > > Hello again,
> > >
> > > I'm trying to find a problem with my inability to run point_stat
> > > for a pair of NetCDF files. Here's what I am seeing from MET:
> > >
> > > DEBUG 1: Default Config File:
> > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > DEBUG 1: User Config File:
> > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
> > > new Met2dDataFile object of type "FileType_NcMet".
> > > NetCDF: Attribute not found
> > >
> > > Here's what I defined for the fields in the point_stat config
file:
> > >
> > > fcst = {
> > > message_type = [ "ADPSFC" ];
> > > field = [
> > > {
> > > name = "PM2_5_DRY(0,0,*,*)";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "obs_arr(*,4)";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > Within the forecast NetCDF file the following values are set:
> > >
> > > dimensions:
> > > Time = UNLIMITED ; // (1 currently)
> > > west_east = 249 ;
> > > south_north = 149 ;
> > > bottom_top = 56 ;
> > >
> > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > PM2_5_DRY:FieldType = 104 ;
> > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > PM2_5_DRY:description = "pm2.5 aerosol dry mass"
;
> > > PM2_5_DRY:units = "ug m^-3" ;
> > > PM2_5_DRY:stagger = "" ;
> > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > >
> > > Within the observation file:
> > >
> > > dimensions:
> > > obs_arr_len = 5 ;
> > > nhdr = 1820 ;
> > > nobs = UNLIMITED ; // (1820 currently)
> > >
> > > float obs_arr(nobs, obs_arr_len) ;
> > > obs_arr:long_name = "array of observation
values" ;
> > > obs_arr:_fill_value = -9999.f ;
> > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > obs_arr:hdr_id_long_name = "index of matching
> > > header
> > data"
> > > ;
> > > obs_arr:gc_long_name = "grib code corresponding
to
> > > the observation type" ;
> > > obs_arr:lvl_long_name = "pressure level (hPa) or
> > > accumulation interval (sec)" ;
> > > obs_arr:hgt_long_name = "height in meters above
> > > sea level or ground level (msl or agl)" ;
> > > obs_arr:ob_long_name = "observation value" ;
> > >
> > > Additional information:
> > >
> > > "ADPSFC" is the message type within the header of the
observation
> > > files, but it does not appear in the forecast file anywhere.
> > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > represents the first member of an array and not 1. The second 0
> > > indicates the surface data should be used, the first of 56
levels.
> > > I used "*" to represent the X and Y nature of the grid.
> > > I am using obs_arr(*,4) because there are 5 data fields, and
only
> > > the 5th one (numbered 4 when starting from 0) is the actual data
> > > field of interest. The first * indicates to use all 1820
observations.
> > > I built the observation file using 'ascii2nc'; the forecast file
> > > was built separately by a colleague.
> > >
> > > The observation file is ~350KB so I could easily e-mail it, but
> > > the model data is ~1.3GB, so obviously I can't easily send that
> > > along for you to test with. Perhaps I could FTP it somewhere
you
> > > could try to replicate the error?
> > >
> > > Do you see anything here that I've made a mistake with? I've
> > > spent a couple hours staring at code and reading the user's
> > > manual; I can't find any guidance as to where I made the
mistake.
> > > I'm hoping you can
> > help.
> > >
> > > Thanks,
> > > Matt
> > >
> > > // SIGNED //
> > > Matthew C. Sittel
> > > Associate Scientist
> > > University Corporation for Atmospheric Research 16WS/WXN, Offutt
> > > AFB, NE
> > > (402) 294-3473 DSN: 271-3473
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Tue Sep 23 14:21:59 2014
Hi Matt. Thanks for getting the data to us.
The forecast file that you sent was a raw wrfout file, which MET
doesn't
support. The raw wrfout files have to be post-processed using
p_interp or
UPP. UPP tends to work better because it writes GRIB files that MET
supports well. For p_interp output, we can't read variables that are
defined on the staggered dimensions, like wind.
We used p_interp (available here:
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing)
to convert your file into a NetCDF format that MET can read. p_interp
also
interpolates to pressure levels, and many of the grid points for the
first
few pressure levels are actually below ground. So you'll notice a lot
of
missing values in the first few pressure levels of the p_interp
output. We
also made some changes to your configuration file. I have attached
the new
configuration file for you to use. Here is a brief description of
the
changes:
-Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
-Modified
name = "PM2_5_DRY(0,0,*,*)";
to
name = "PM2_5_DRY";
level = "(0,0,*,*)";
specifying the level separately
-Modified
name = "obs_arr(*,4)";
to
name = "NCIP";
level = "L0";
because the obs information needs to be specified as if it were a GRIB
file. NCIP came from grib code 198 (specified in obs_arr)
- Changed sid to be [] instead of ""
With these changes, we were able to successfully run point_stat.
However,
please note that the forecast values for PM2_5_DRY are all zero! You
could
take a look at your WRF-Chem run to figure out why. I'd suggest using
ncview to display the PM2_5_DRY variable in the NetCDF file you sent
us.
You'll see that it's zero for all model levels. So you may want to
look
into that.
I hope this helps.
Thanks,
Julie
On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Thanks-I got it to work on my end so you don't have to monitor it.
Sorry
> about the directory mixup-I created the directory and thought I was
putting
> the files there!
>
> 58 MB right now thus far I see... let's hope it works!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:18 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> I understand your confusion. The file was being written to
> incoming/irap/met_help/ as opposed to
incoming/irap/met_help/sittel_data/
> where it is being written to now. I will get back to you this
afternoon
> and let you know how far along it is.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > I opened a separate window and did a list on the directory but it
said
> > nothing was in there... which left me even more confused. I
have
> > restarted the FTP but looks like I have no way on my end to
monitor
> > progress... so I'll leave it running all night if I must and hope
it
> > continues running.
> >
> > If you get a chance later this afternoon, please let me know how
far
> > along the file is so I can estimate transfer speed for future
reference.
> >
> > Thanks!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:08 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Ah, that's too bad. I had been watching it slowly come in. I'll
keep
> > an eye out for it.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Looks like I need to be more patient. I assumed when I got back
to
> > > work that the file transfer was hung so I aborted it. Turns out
it
> > > has transferred 430 MB of the full 1.3 GB of the model data
file.
> > > So looks like transfer will take 6-8 hours at this rate.
> > >
> > > With any luck it will run to completion... sorry for the delay,
but
> > > I don't know any better way to get you the file short of mailing
a DVD!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 10:37 AM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. I see that you are having problems running point_stat.
> > > Thank you for including the details of the situation. Could you
> > > please follow these instructions for posting the forecast file,
> > > observation file, and config file to our ftp site:
> > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > I can take a closer look and try to resolve the problem.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > Queue: met_help
> > > > Subject: MET 4.1: Point_stat issue
> > > > Owner: Nobody
> > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > >
> > > >
> > > >
> > > > Hello again,
> > > >
> > > > I'm trying to find a problem with my inability to run
point_stat
> > > > for a pair of NetCDF files. Here's what I am seeing from MET:
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > NetCDF: Attribute not found
> > > >
> > > > Here's what I defined for the fields in the point_stat config
file:
> > > >
> > > > fcst = {
> > > > message_type = [ "ADPSFC" ];
> > > > field = [
> > > > {
> > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "obs_arr(*,4)";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > Within the forecast NetCDF file the following values are set:
> > > >
> > > > dimensions:
> > > > Time = UNLIMITED ; // (1 currently)
> > > > west_east = 249 ;
> > > > south_north = 149 ;
> > > > bottom_top = 56 ;
> > > >
> > > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > PM2_5_DRY:FieldType = 104 ;
> > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > PM2_5_DRY:stagger = "" ;
> > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > >
> > > > Within the observation file:
> > > >
> > > > dimensions:
> > > > obs_arr_len = 5 ;
> > > > nhdr = 1820 ;
> > > > nobs = UNLIMITED ; // (1820 currently)
> > > >
> > > > float obs_arr(nobs, obs_arr_len) ;
> > > > obs_arr:long_name = "array of observation
values" ;
> > > > obs_arr:_fill_value = -9999.f ;
> > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > obs_arr:hdr_id_long_name = "index of matching
> > > > header
> > > data"
> > > > ;
> > > > obs_arr:gc_long_name = "grib code
corresponding to
> > > > the observation type" ;
> > > > obs_arr:lvl_long_name = "pressure level (hPa)
or
> > > > accumulation interval (sec)" ;
> > > > obs_arr:hgt_long_name = "height in meters
above
> > > > sea level or ground level (msl or agl)" ;
> > > > obs_arr:ob_long_name = "observation value" ;
> > > >
> > > > Additional information:
> > > >
> > > > "ADPSFC" is the message type within the header of the
observation
> > > > files, but it does not appear in the forecast file anywhere.
> > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > represents the first member of an array and not 1. The second
0
> > > > indicates the surface data should be used, the first of 56
levels.
> > > > I used "*" to represent the X and Y nature of the grid.
> > > > I am using obs_arr(*,4) because there are 5 data fields, and
only
> > > > the 5th one (numbered 4 when starting from 0) is the actual
data
> > > > field of interest. The first * indicates to use all 1820
> observations.
> > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > was built separately by a colleague.
> > > >
> > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > along for you to test with. Perhaps I could FTP it somewhere
you
> > > > could try to replicate the error?
> > > >
> > > > Do you see anything here that I've made a mistake with? I've
> > > > spent a couple hours staring at code and reading the user's
> > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > I'm hoping you can
> > > help.
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > // SIGNED //
> > > > Matthew C. Sittel
> > > > Associate Scientist
> > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > AFB, NE
> > > > (402) 294-3473 DSN: 271-3473
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Tue Sep 23 14:40:59 2014
Hi Julie,
Thanks for all your help! I didn't know it wasn't usable in its
current form; it was given to me that way. It's not my model, though,
so I get to tell someone else to worry about it instead of changing it
myself. I'm guessing putting it on pressure levels is acceptable
since they just wanted the lowest one for me to use as verification.
I didn't look carefully at the model output, only the observation data
since I was more comfortable with it. Sounds like another issue for
the model guys to look at.
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 23, 2014 3:22 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Hi Matt. Thanks for getting the data to us.
The forecast file that you sent was a raw wrfout file, which MET
doesn't support. The raw wrfout files have to be post-processed using
p_interp or UPP. UPP tends to work better because it writes GRIB
files that MET supports well. For p_interp output, we can't read
variables that are defined on the staggered dimensions, like wind.
We used p_interp (available here:
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing)
to convert your file into a NetCDF format that MET can read. p_interp
also interpolates to pressure levels, and many of the grid points for
the first few pressure levels are actually below ground. So you'll
notice a lot of missing values in the first few pressure levels of the
p_interp output. We also made some changes to your configuration
file. I have attached the new
configuration file for you to use. Here is a brief description of
the
changes:
-Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
-Modified
name = "PM2_5_DRY(0,0,*,*)";
to
name = "PM2_5_DRY";
level = "(0,0,*,*)";
specifying the level separately
-Modified
name = "obs_arr(*,4)";
to
name = "NCIP";
level = "L0";
because the obs information needs to be specified as if it were a GRIB
file. NCIP came from grib code 198 (specified in obs_arr)
- Changed sid to be [] instead of ""
With these changes, we were able to successfully run point_stat.
However, please note that the forecast values for PM2_5_DRY are all
zero! You could take a look at your WRF-Chem run to figure out why.
I'd suggest using ncview to display the PM2_5_DRY variable in the
NetCDF file you sent us.
You'll see that it's zero for all model levels. So you may want to
look into that.
I hope this helps.
Thanks,
Julie
On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Thanks-I got it to work on my end so you don't have to monitor it.
Sorry
> about the directory mixup-I created the directory and thought I was
> putting the files there!
>
> 58 MB right now thus far I see... let's hope it works!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:18 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> I understand your confusion. The file was being written to
> incoming/irap/met_help/ as opposed to
> incoming/irap/met_help/sittel_data/
> where it is being written to now. I will get back to you this
> afternoon and let you know how far along it is.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > I opened a separate window and did a list on the directory but it
said
> > nothing was in there... which left me even more confused. I
have
> > restarted the FTP but looks like I have no way on my end to
monitor
> > progress... so I'll leave it running all night if I must and hope
it
> > continues running.
> >
> > If you get a chance later this afternoon, please let me know how
far
> > along the file is so I can estimate transfer speed for future
reference.
> >
> > Thanks!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:08 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Ah, that's too bad. I had been watching it slowly come in. I'll
> > keep an eye out for it.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Looks like I need to be more patient. I assumed when I got back
> > > to work that the file transfer was hung so I aborted it. Turns
> > > out it has transferred 430 MB of the full 1.3 GB of the model
data file.
> > > So looks like transfer will take 6-8 hours at this rate.
> > >
> > > With any luck it will run to completion... sorry for the delay,
> > > but I don't know any better way to get you the file short of
mailing a DVD!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 10:37 AM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. I see that you are having problems running point_stat.
> > > Thank you for including the details of the situation. Could you
> > > please follow these instructions for posting the forecast file,
> > > observation file, and config file to our ftp site:
> > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > I can take a closer look and try to resolve the problem.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
> > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > Queue: met_help
> > > > Subject: MET 4.1: Point_stat issue
> > > > Owner: Nobody
> > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > >
> > > >
> > > >
> > > > Hello again,
> > > >
> > > > I'm trying to find a problem with my inability to run
point_stat
> > > > for a pair of NetCDF files. Here's what I am seeing from MET:
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > NetCDF: Attribute not found
> > > >
> > > > Here's what I defined for the fields in the point_stat config
file:
> > > >
> > > > fcst = {
> > > > message_type = [ "ADPSFC" ];
> > > > field = [
> > > > {
> > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "obs_arr(*,4)";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > Within the forecast NetCDF file the following values are set:
> > > >
> > > > dimensions:
> > > > Time = UNLIMITED ; // (1 currently)
> > > > west_east = 249 ;
> > > > south_north = 149 ;
> > > > bottom_top = 56 ;
> > > >
> > > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > PM2_5_DRY:FieldType = 104 ;
> > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > PM2_5_DRY:stagger = "" ;
> > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > >
> > > > Within the observation file:
> > > >
> > > > dimensions:
> > > > obs_arr_len = 5 ;
> > > > nhdr = 1820 ;
> > > > nobs = UNLIMITED ; // (1820 currently)
> > > >
> > > > float obs_arr(nobs, obs_arr_len) ;
> > > > obs_arr:long_name = "array of observation
values" ;
> > > > obs_arr:_fill_value = -9999.f ;
> > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > obs_arr:hdr_id_long_name = "index of matching
> > > > header
> > > data"
> > > > ;
> > > > obs_arr:gc_long_name = "grib code
corresponding
> > > > to the observation type" ;
> > > > obs_arr:lvl_long_name = "pressure level (hPa)
or
> > > > accumulation interval (sec)" ;
> > > > obs_arr:hgt_long_name = "height in meters
above
> > > > sea level or ground level (msl or agl)" ;
> > > > obs_arr:ob_long_name = "observation value" ;
> > > >
> > > > Additional information:
> > > >
> > > > "ADPSFC" is the message type within the header of the
> > > > observation files, but it does not appear in the forecast file
anywhere.
> > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > represents the first member of an array and not 1. The second
0
> > > > indicates the surface data should be used, the first of 56
levels.
> > > > I used "*" to represent the X and Y nature of the grid.
> > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > only the 5th one (numbered 4 when starting from 0) is the
actual
> > > > data field of interest. The first * indicates to use all 1820
> observations.
> > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > was built separately by a colleague.
> > > >
> > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > along for you to test with. Perhaps I could FTP it somewhere
> > > > you could try to replicate the error?
> > > >
> > > > Do you see anything here that I've made a mistake with? I've
> > > > spent a couple hours staring at code and reading the user's
> > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > I'm hoping you can
> > > help.
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > // SIGNED //
> > > > Matthew C. Sittel
> > > > Associate Scientist
> > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > AFB, NE
> > > > (402) 294-3473 DSN: 271-3473
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Tue Sep 23 14:50:48 2014
Thanks, Matt. I'll resolve this ticket for now. Please let us know
if you
encounter any other issues.
Julie
On Tue, Sep 23, 2014 at 2:40 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Thanks for all your help! I didn't know it wasn't usable in its
current
> form; it was given to me that way. It's not my model, though, so I
get to
> tell someone else to worry about it instead of changing it myself.
I'm
> guessing putting it on pressure levels is acceptable since they just
wanted
> the lowest one for me to use as verification.
>
> I didn't look carefully at the model output, only the observation
data
> since I was more comfortable with it. Sounds like another issue for
the
> model guys to look at.
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 23, 2014 3:22 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. Thanks for getting the data to us.
>
> The forecast file that you sent was a raw wrfout file, which MET
doesn't
> support. The raw wrfout files have to be post-processed using
p_interp or
> UPP. UPP tends to work better because it writes GRIB files that MET
> supports well. For p_interp output, we can't read variables that
are
> defined on the staggered dimensions, like wind.
>
> We used p_interp (available here:
>
>
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing
> )
> to convert your file into a NetCDF format that MET can read.
p_interp
> also interpolates to pressure levels, and many of the grid points
for the
> first few pressure levels are actually below ground. So you'll
notice a
> lot of missing values in the first few pressure levels of the
p_interp
> output. We also made some changes to your configuration file. I
have
> attached the new
> configuration file for you to use. Here is a brief description of
the
> changes:
>
> -Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
> -Modified
> name = "PM2_5_DRY(0,0,*,*)";
> to
> name = "PM2_5_DRY";
> level = "(0,0,*,*)";
> specifying the level separately
> -Modified
> name = "obs_arr(*,4)";
> to
> name = "NCIP";
> level = "L0";
> because the obs information needs to be specified as if it were a
GRIB
> file. NCIP came from grib code 198 (specified in obs_arr)
> - Changed sid to be [] instead of ""
>
> With these changes, we were able to successfully run point_stat.
However,
> please note that the forecast values for PM2_5_DRY are all zero!
You could
> take a look at your WRF-Chem run to figure out why. I'd suggest
using
> ncview to display the PM2_5_DRY variable in the NetCDF file you sent
us.
> You'll see that it's zero for all model levels. So you may want to
look
> into that.
>
> I hope this helps.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Thanks-I got it to work on my end so you don't have to monitor it.
> Sorry
> > about the directory mixup-I created the directory and thought I
was
> > putting the files there!
> >
> > 58 MB right now thus far I see... let's hope it works!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:18 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > I understand your confusion. The file was being written to
> > incoming/irap/met_help/ as opposed to
> > incoming/irap/met_help/sittel_data/
> > where it is being written to now. I will get back to you this
> > afternoon and let you know how far along it is.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > I opened a separate window and did a list on the directory but
it said
> > > nothing was in there... which left me even more confused. I
have
> > > restarted the FTP but looks like I have no way on my end to
monitor
> > > progress... so I'll leave it running all night if I must and
hope it
> > > continues running.
> > >
> > > If you get a chance later this afternoon, please let me know how
far
> > > along the file is so I can estimate transfer speed for future
> reference.
> > >
> > > Thanks!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 1:08 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Ah, that's too bad. I had been watching it slowly come in.
I'll
> > > keep an eye out for it.
> > >
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
AFWA 16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Looks like I need to be more patient. I assumed when I got
back
> > > > to work that the file transfer was hung so I aborted it.
Turns
> > > > out it has transferred 430 MB of the full 1.3 GB of the model
data
> file.
> > > > So looks like transfer will take 6-8 hours at this rate.
> > > >
> > > > With any luck it will run to completion... sorry for the
delay,
> > > > but I don't know any better way to get you the file short of
mailing
> a DVD!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > Thank you for including the details of the situation. Could
you
> > > > please follow these instructions for posting the forecast
file,
> > > > observation file, and config file to our ftp site:
> > > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > >
> > > > I can take a closer look and try to resolve the problem.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > > Queue: met_help
> > > > > Subject: MET 4.1: Point_stat issue
> > > > > Owner: Nobody
> > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > Status: new
> > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > >
> > > > > Hello again,
> > > > >
> > > > > I'm trying to find a problem with my inability to run
point_stat
> > > > > for a pair of NetCDF files. Here's what I am seeing from
MET:
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > DEBUG 1: User Config File:
> > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > > NetCDF: Attribute not found
> > > > >
> > > > > Here's what I defined for the fields in the point_stat
config file:
> > > > >
> > > > > fcst = {
> > > > > message_type = [ "ADPSFC" ];
> > > > > field = [
> > > > > {
> > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > obs = {
> > > > > field = [
> > > > > {
> > > > > name = "obs_arr(*,4)";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > Within the forecast NetCDF file the following values are
set:
> > > > >
> > > > > dimensions:
> > > > > Time = UNLIMITED ; // (1 currently)
> > > > > west_east = 249 ;
> > > > > south_north = 149 ;
> > > > > bottom_top = 56 ;
> > > > >
> > > > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > PM2_5_DRY:stagger = "" ;
> > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > >
> > > > > Within the observation file:
> > > > >
> > > > > dimensions:
> > > > > obs_arr_len = 5 ;
> > > > > nhdr = 1820 ;
> > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > >
> > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > obs_arr:long_name = "array of observation
values" ;
> > > > > obs_arr:_fill_value = -9999.f ;
> > > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > > obs_arr:hdr_id_long_name = "index of
matching
> > > > > header
> > > > data"
> > > > > ;
> > > > > obs_arr:gc_long_name = "grib code
corresponding
> > > > > to the observation type" ;
> > > > > obs_arr:lvl_long_name = "pressure level
(hPa) or
> > > > > accumulation interval (sec)" ;
> > > > > obs_arr:hgt_long_name = "height in meters
above
> > > > > sea level or ground level (msl or agl)" ;
> > > > > obs_arr:ob_long_name = "observation value" ;
> > > > >
> > > > > Additional information:
> > > > >
> > > > > "ADPSFC" is the message type within the header of the
> > > > > observation files, but it does not appear in the forecast
file
> anywhere.
> > > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > > represents the first member of an array and not 1. The
second 0
> > > > > indicates the surface data should be used, the first of 56
levels.
> > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > > only the 5th one (numbered 4 when starting from 0) is the
actual
> > > > > data field of interest. The first * indicates to use all
1820
> > observations.
> > > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > > was built separately by a colleague.
> > > > >
> > > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > > along for you to test with. Perhaps I could FTP it
somewhere
> > > > > you could try to replicate the error?
> > > > >
> > > > > Do you see anything here that I've made a mistake with?
I've
> > > > > spent a couple hours staring at code and reading the user's
> > > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > > I'm hoping you can
> > > > help.
> > > > >
> > > > > Thanks,
> > > > > Matt
> > > > >
> > > > > // SIGNED //
> > > > > Matthew C. Sittel
> > > > > Associate Scientist
> > > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > > AFB, NE
> > > > > (402) 294-3473 DSN: 271-3473
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Sep 24 06:46:14 2014
Hi Julie,
Do you know the answer to Steve's question? He's running the model
but isn't sure about if p_interp will work.
Thanks,
Matt
-----Original Message-----
From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
Sent: Wednesday, September 24, 2014 7:40 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Subject: RE: Verification issues
Correct, the problem is currently that the p_intep function does not
work for chem that I am aware of. The data is on sigma levels, unless
Julie knows how to do it for the chem? Did she get the EXTCOF55
variable to output using the p_interp option?
Sorry if you look at the 00Z forecast we just need to verify against
the 24 hour forecast, for the MOZART I can't cycle that variable. So
for the 00Z forecast we can just verify against wrfout_d01_2014-08-
23_00:00:00 (the 24 hour forecast).
Steve
-----Original Message-----
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Sent: Wednesday, September 24, 2014 7:35 AM
To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
Subject: RE: Verification issues
MET can read NetCDF inputs but there is something about 'wrfout'
format that it won't accept... either the staggered grid as Julie
points out and/or being on sigma levels rather than pressure levels.
Julie successfully used p_interp to convert the wrfout data into
something MET could use. I don't know if we have that already
installed on dev8... it's possible that we do. If not we definitely
need to get that.
The model file in question is
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout_d01_2014-
08-22_00:00:00
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Wed Sep 24 09:44:11 2014
Hello Matt,
This is John Halley Gotway - thought I'd jump in on this one. Julie
and I
were able to run that NetCDF file you sent through the p_interp
utility and
we do see the EXTCOF55 variable in the output. However the EXTCOF55
variable contains 0's in both the input to p_interp and the output.
I've
attached the NetCDF headers for the input to/output from p_interp.
The
output has "PLEV" in the name.
I've also attached the "namelist.pinterp" configuration file we used
in the
conversion. We just used the default pressure levels - which begin at
1000
mb.
It may also be possible to post-process the WRF-Chem output with UPP.
I
haven't tried, so I really don't know.
Either option interpolates to pressure levels, but I'm wondering if
this is
sufficient for the type of analysis you want to do. Since you've
assigned
the AIRNOW observations a message type of ADPSFC, I assume they're
defined
at the surface? If so, the problem is that none of your forecast data
exists at the surface. I really have no idea what the PM2_5_DRY
variable
and the AIRNOW observations contain. I'd just suggest thinking about
whether the comparison of surface AIRNOW to 1000mb PM2_5_DRY makes
sense.
Thanks,
John
On Wed, Sep 24, 2014 at 6:46 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Do you know the answer to Steve's question? He's running the model
but
> isn't sure about if p_interp will work.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Sent: Wednesday, September 24, 2014 7:40 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: RE: Verification issues
>
> Correct, the problem is currently that the p_intep function does not
work
> for chem that I am aware of. The data is on sigma levels, unless
Julie
> knows how to do it for the chem? Did she get the EXTCOF55 variable
to
> output using the p_interp option?
>
> Sorry if you look at the 00Z forecast we just need to verify against
the
> 24 hour forecast, for the MOZART I can't cycle that variable. So for
the
> 00Z forecast we can just verify against wrfout_d01_2014-08-
23_00:00:00 (the
> 24 hour forecast).
>
> Steve
>
> -----Original Message-----
> From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Sent: Wednesday, September 24, 2014 7:35 AM
> To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Subject: RE: Verification issues
>
> MET can read NetCDF inputs but there is something about 'wrfout'
format
> that it won't accept... either the staggered grid as Julie points
out
> and/or being on sigma levels rather than pressure levels.
>
> Julie successfully used p_interp to convert the wrfout data into
something
> MET could use. I don't know if we have that already installed on
dev8...
> it's possible that we do. If not we definitely need to get that.
>
> The model file in question is
>
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout_d01_2014-
08-22_00:00:00
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Sep 24 10:24:09 2014
Hey John (and Julie),
Thanks for chiming in with your thoughts. Turns out I got the
developer to totally overhaul the model output into GriB base on
Julie's observations, so I made many of the problems go away with the
conversion to pressure levels.
They've chosen to drop PM2_5_DRY altogether and stick with only
WXTCOF55, to be verified only at 1013 mb, so it's a surface field,
sort of... admittedly I don't know too much about the exact details of
what I've been asked to verify; I like to focus on getting the
specifics first and then worry about interpretation-if I have the
wrong kpds numbers from the get-go then nothing else will make sense
anyway. Plus it forces the developer to think about what it is they
*really* want verified!
I get the impression PM2_5_DRY can't be interpolated correctly, or so
I was told. Rather than press that issue with them, I'll just keep
quiet and be thankful they made my job easier (and yours, by
association)!
Thanks again. John, have you been out here to Omaha recently? I'm
long overdue for a visit to Boulder but I got yanked from the ensemble
verification stuff to focus exclusively on WRF-Chem this contract
year. At least I can stay somewhat involved with MET this way. Trust
that this was not my choice to be 'reassigned' and it hasn't gone over
well, but you know how it goes with the money... you do what you're
paid to do and just be glad you still have a job.
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, September 24, 2014 10:44 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Hello Matt,
This is John Halley Gotway - thought I'd jump in on this one. Julie
and I were able to run that NetCDF file you sent through the p_interp
utility and we do see the EXTCOF55 variable in the output. However
the EXTCOF55 variable contains 0's in both the input to p_interp and
the output. I've attached the NetCDF headers for the input to/output
from p_interp. The output has "PLEV" in the name.
I've also attached the "namelist.pinterp" configuration file we used
in the conversion. We just used the default pressure levels - which
begin at 1000 mb.
It may also be possible to post-process the WRF-Chem output with UPP.
I haven't tried, so I really don't know.
Either option interpolates to pressure levels, but I'm wondering if
this is sufficient for the type of analysis you want to do. Since
you've assigned the AIRNOW observations a message type of ADPSFC, I
assume they're defined at the surface? If so, the problem is that
none of your forecast data exists at the surface. I really have no
idea what the PM2_5_DRY variable and the AIRNOW observations contain.
I'd just suggest thinking about whether the comparison of surface
AIRNOW to 1000mb PM2_5_DRY makes sense.
Thanks,
John
On Wed, Sep 24, 2014 at 6:46 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Do you know the answer to Steve's question? He's running the model
> but isn't sure about if p_interp will work.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Sent: Wednesday, September 24, 2014 7:40 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: RE: Verification issues
>
> Correct, the problem is currently that the p_intep function does not
> work for chem that I am aware of. The data is on sigma levels,
unless
> Julie knows how to do it for the chem? Did she get the EXTCOF55
> variable to output using the p_interp option?
>
> Sorry if you look at the 00Z forecast we just need to verify against
> the
> 24 hour forecast, for the MOZART I can't cycle that variable. So for
> the 00Z forecast we can just verify against
> wrfout_d01_2014-08-23_00:00:00 (the
> 24 hour forecast).
>
> Steve
>
> -----Original Message-----
> From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Sent: Wednesday, September 24, 2014 7:35 AM
> To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Subject: RE: Verification issues
>
> MET can read NetCDF inputs but there is something about 'wrfout'
> format that it won't accept... either the staggered grid as Julie
> points out and/or being on sigma levels rather than pressure levels.
>
> Julie successfully used p_interp to convert the wrfout data into
> something MET could use. I don't know if we have that already
installed on dev8...
> it's possible that we do. If not we definitely need to get that.
>
> The model file in question is
>
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout
> _d01_2014-08-22_00:00:00
>
>
>
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Wed Sep 24 11:29:21 2014
Matt,
Haven't been to Omaha in a while, but I did use my in-law's (from
Omaha)
timeshare in Cabo this summer - about a month before Odile hit! That
was a
lot of fun.
Perhaps it's good to have someone applying the MET tools to WRF-Chem
problems. I'll be interested to hear how it goes and see if there's
any
tweaks or additions we could make to MET that might help.
I'll go ahead and resolve this ticket, but please let us know if more
questions or issues arise.
Thanks,
John
On Wed, Sep 24, 2014 at 10:24 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hey John (and Julie),
>
> Thanks for chiming in with your thoughts. Turns out I got the
developer
> to totally overhaul the model output into GriB base on Julie's
> observations, so I made many of the problems go away with the
conversion to
> pressure levels.
>
> They've chosen to drop PM2_5_DRY altogether and stick with only
WXTCOF55,
> to be verified only at 1013 mb, so it's a surface field, sort of...
> admittedly I don't know too much about the exact details of what
I've been
> asked to verify; I like to focus on getting the specifics first and
then
> worry about interpretation-if I have the wrong kpds numbers from the
get-go
> then nothing else will make sense anyway. Plus it forces the
developer to
> think about what it is they *really* want verified!
>
> I get the impression PM2_5_DRY can't be interpolated correctly, or
so I
> was told. Rather than press that issue with them, I'll just keep
quiet and
> be thankful they made my job easier (and yours, by association)!
>
> Thanks again. John, have you been out here to Omaha recently? I'm
long
> overdue for a visit to Boulder but I got yanked from the ensemble
> verification stuff to focus exclusively on WRF-Chem this contract
year. At
> least I can stay somewhat involved with MET this way. Trust that
this was
> not my choice to be 'reassigned' and it hasn't gone over well, but
you know
> how it goes with the money... you do what you're paid to do and just
be
> glad you still have a job.
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, September 24, 2014 10:44 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hello Matt,
>
> This is John Halley Gotway - thought I'd jump in on this one. Julie
and I
> were able to run that NetCDF file you sent through the p_interp
utility and
> we do see the EXTCOF55 variable in the output. However the EXTCOF55
> variable contains 0's in both the input to p_interp and the output.
I've
> attached the NetCDF headers for the input to/output from p_interp.
The
> output has "PLEV" in the name.
>
> I've also attached the "namelist.pinterp" configuration file we used
in
> the conversion. We just used the default pressure levels - which
begin at
> 1000 mb.
>
> It may also be possible to post-process the WRF-Chem output with
UPP. I
> haven't tried, so I really don't know.
>
> Either option interpolates to pressure levels, but I'm wondering if
this
> is sufficient for the type of analysis you want to do. Since you've
> assigned the AIRNOW observations a message type of ADPSFC, I assume
they're
> defined at the surface? If so, the problem is that none of your
forecast
> data exists at the surface. I really have no idea what the
PM2_5_DRY
> variable and the AIRNOW observations contain. I'd just suggest
thinking
> about whether the comparison of surface AIRNOW to 1000mb PM2_5_DRY
makes
> sense.
>
> Thanks,
> John
>
>
> On Wed, Sep 24, 2014 at 6:46 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi Julie,
> >
> > Do you know the answer to Steve's question? He's running the
model
> > but isn't sure about if p_interp will work.
> >
> > Thanks,
> > Matt
> >
> > -----Original Message-----
> > From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> > Sent: Wednesday, September 24, 2014 7:40 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Subject: RE: Verification issues
> >
> > Correct, the problem is currently that the p_intep function does
not
> > work for chem that I am aware of. The data is on sigma levels,
unless
> > Julie knows how to do it for the chem? Did she get the EXTCOF55
> > variable to output using the p_interp option?
> >
> > Sorry if you look at the 00Z forecast we just need to verify
against
> > the
> > 24 hour forecast, for the MOZART I can't cycle that variable. So
for
> > the 00Z forecast we can just verify against
> > wrfout_d01_2014-08-23_00:00:00 (the
> > 24 hour forecast).
> >
> > Steve
> >
> > -----Original Message-----
> > From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Sent: Wednesday, September 24, 2014 7:35 AM
> > To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> > Subject: RE: Verification issues
> >
> > MET can read NetCDF inputs but there is something about 'wrfout'
> > format that it won't accept... either the staggered grid as Julie
> > points out and/or being on sigma levels rather than pressure
levels.
> >
> > Julie successfully used p_interp to convert the wrfout data into
> > something MET could use. I don't know if we have that already
installed
> on dev8...
> > it's possible that we do. If not we definitely need to get that.
> >
> > The model file in question is
> >
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout
> > _d01_2014-08-22_00:00:00
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Oct 01 13:24:22 2014
Hi Julie,
So we've made some progress on the problems. The modelers are going
to produce the data in GriB format. Here's the GriB header for the
one field in the GriB file I need to verify:
rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
AODVIS=Aerosol optical depth (0.55 um) [numeric]
timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0 num_in_ave 0
missing 0
center 57 subcenter 6 process 15 Table 129 scan: WE:NS winds(grid)
latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
long 0.000000 to -0.500000 by 1.003000, (359 x 180) scan 0
mode 136 bdsgrid 1
min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239 DecScale 6
BinScale 0
How do I set the identifiers for the forecast field within the
point_stat config file to get MET to read this? Right now I have:
fcst = {
message_type = [ "ADPSFC" ];
field = [
{
name = "AODVIS";
level = "L0";
cat_thresh = [ >=0.0 ];
}
];
};
obs = {
field = [
{
name = "NCIP";
level = "L0";
cat_thresh = [ >=0.0 ];
}
];
};
But this isn't working. The error I get is:
DEBUG 1: User Config File:
/home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_Gb1".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type "FileType_Gb1".
ERROR :
ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
abbreviation 'AODVIS' for table version 2
ERROR :
This is the one thing in MET that confuses me more than anything else-
identifying the data fields in the config file.
How do I know the variable and level names? What field(s) in the GriB
header do these correspond to in MET settings? Does it matter that
"AODVIS" isn't the standard variable for kpds5=147? The modelers
don't always follow the naming; they'll put any output field in any
kpds number they want.
Can you help me get the variables set correctly?
Thanks,
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 23, 2014 3:22 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Hi Matt. Thanks for getting the data to us.
The forecast file that you sent was a raw wrfout file, which MET
doesn't support. The raw wrfout files have to be post-processed using
p_interp or UPP. UPP tends to work better because it writes GRIB
files that MET supports well. For p_interp output, we can't read
variables that are defined on the staggered dimensions, like wind.
We used p_interp (available here:
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing)
to convert your file into a NetCDF format that MET can read. p_interp
also interpolates to pressure levels, and many of the grid points for
the first few pressure levels are actually below ground. So you'll
notice a lot of missing values in the first few pressure levels of the
p_interp output. We also made some changes to your configuration
file. I have attached the new
configuration file for you to use. Here is a brief description of
the
changes:
-Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
-Modified
name = "PM2_5_DRY(0,0,*,*)";
to
name = "PM2_5_DRY";
level = "(0,0,*,*)";
specifying the level separately
-Modified
name = "obs_arr(*,4)";
to
name = "NCIP";
level = "L0";
because the obs information needs to be specified as if it were a GRIB
file. NCIP came from grib code 198 (specified in obs_arr)
- Changed sid to be [] instead of ""
With these changes, we were able to successfully run point_stat.
However, please note that the forecast values for PM2_5_DRY are all
zero! You could take a look at your WRF-Chem run to figure out why.
I'd suggest using ncview to display the PM2_5_DRY variable in the
NetCDF file you sent us.
You'll see that it's zero for all model levels. So you may want to
look into that.
I hope this helps.
Thanks,
Julie
On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Thanks-I got it to work on my end so you don't have to monitor it.
Sorry
> about the directory mixup-I created the directory and thought I was
> putting the files there!
>
> 58 MB right now thus far I see... let's hope it works!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:18 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> I understand your confusion. The file was being written to
> incoming/irap/met_help/ as opposed to
> incoming/irap/met_help/sittel_data/
> where it is being written to now. I will get back to you this
> afternoon and let you know how far along it is.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > I opened a separate window and did a list on the directory but it
said
> > nothing was in there... which left me even more confused. I
have
> > restarted the FTP but looks like I have no way on my end to
monitor
> > progress... so I'll leave it running all night if I must and hope
it
> > continues running.
> >
> > If you get a chance later this afternoon, please let me know how
far
> > along the file is so I can estimate transfer speed for future
reference.
> >
> > Thanks!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:08 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Ah, that's too bad. I had been watching it slowly come in. I'll
> > keep an eye out for it.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Looks like I need to be more patient. I assumed when I got back
> > > to work that the file transfer was hung so I aborted it. Turns
> > > out it has transferred 430 MB of the full 1.3 GB of the model
data file.
> > > So looks like transfer will take 6-8 hours at this rate.
> > >
> > > With any luck it will run to completion... sorry for the delay,
> > > but I don't know any better way to get you the file short of
mailing a DVD!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 10:37 AM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. I see that you are having problems running point_stat.
> > > Thank you for including the details of the situation. Could you
> > > please follow these instructions for posting the forecast file,
> > > observation file, and config file to our ftp site:
> > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > I can take a closer look and try to resolve the problem.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
> > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > Queue: met_help
> > > > Subject: MET 4.1: Point_stat issue
> > > > Owner: Nobody
> > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > >
> > > >
> > > >
> > > > Hello again,
> > > >
> > > > I'm trying to find a problem with my inability to run
point_stat
> > > > for a pair of NetCDF files. Here's what I am seeing from MET:
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > NetCDF: Attribute not found
> > > >
> > > > Here's what I defined for the fields in the point_stat config
file:
> > > >
> > > > fcst = {
> > > > message_type = [ "ADPSFC" ];
> > > > field = [
> > > > {
> > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "obs_arr(*,4)";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > Within the forecast NetCDF file the following values are set:
> > > >
> > > > dimensions:
> > > > Time = UNLIMITED ; // (1 currently)
> > > > west_east = 249 ;
> > > > south_north = 149 ;
> > > > bottom_top = 56 ;
> > > >
> > > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > PM2_5_DRY:FieldType = 104 ;
> > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > PM2_5_DRY:stagger = "" ;
> > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > >
> > > > Within the observation file:
> > > >
> > > > dimensions:
> > > > obs_arr_len = 5 ;
> > > > nhdr = 1820 ;
> > > > nobs = UNLIMITED ; // (1820 currently)
> > > >
> > > > float obs_arr(nobs, obs_arr_len) ;
> > > > obs_arr:long_name = "array of observation
values" ;
> > > > obs_arr:_fill_value = -9999.f ;
> > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > obs_arr:hdr_id_long_name = "index of matching
> > > > header
> > > data"
> > > > ;
> > > > obs_arr:gc_long_name = "grib code
corresponding
> > > > to the observation type" ;
> > > > obs_arr:lvl_long_name = "pressure level (hPa)
or
> > > > accumulation interval (sec)" ;
> > > > obs_arr:hgt_long_name = "height in meters
above
> > > > sea level or ground level (msl or agl)" ;
> > > > obs_arr:ob_long_name = "observation value" ;
> > > >
> > > > Additional information:
> > > >
> > > > "ADPSFC" is the message type within the header of the
> > > > observation files, but it does not appear in the forecast file
anywhere.
> > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > represents the first member of an array and not 1. The second
0
> > > > indicates the surface data should be used, the first of 56
levels.
> > > > I used "*" to represent the X and Y nature of the grid.
> > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > only the 5th one (numbered 4 when starting from 0) is the
actual
> > > > data field of interest. The first * indicates to use all 1820
> observations.
> > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > was built separately by a colleague.
> > > >
> > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > along for you to test with. Perhaps I could FTP it somewhere
> > > > you could try to replicate the error?
> > > >
> > > > Do you see anything here that I've made a mistake with? I've
> > > > spent a couple hours staring at code and reading the user's
> > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > I'm hoping you can
> > > help.
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > // SIGNED //
> > > > Matthew C. Sittel
> > > > Associate Scientist
> > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > AFB, NE
> > > > (402) 294-3473 DSN: 271-3473
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Wed Oct 01 14:09:15 2014
Hi Matt. Good questions. Unfortunately, I am not familiar enough with
the
software to be able to answer your questions. John is out of the
office
currently, so I can't ask him for guidance right now. Hopefully, he
will
be able to help out later this week. I'm sorry I can't be of more
help to
you at this time.
Julie
On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> So we've made some progress on the problems. The modelers are going
to
> produce the data in GriB format. Here's the GriB header for the
one field
> in the GriB file I need to verify:
>
> rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200 kpds7=0
> levels=(0,0) grid=255 atmos col anl:
> AODVIS=Aerosol optical depth (0.55 um) [numeric]
> timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0 num_in_ave
0
> missing 0
> center 57 subcenter 6 process 15 Table 129 scan: WE:NS winds(grid)
> latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> long 0.000000 to -0.500000 by 1.003000, (359 x 180) scan 0
mode
> 136 bdsgrid 1
> min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239 DecScale
6
> BinScale 0
>
> How do I set the identifiers for the forecast field within the
point_stat
> config file to get MET to read this? Right now I have:
>
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "AODVIS";
> level = "L0";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> obs = {
> field = [
> {
> name = "NCIP";
> level = "L0";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> But this isn't working. The error I get is:
>
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_Gb1".
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type "FileType_Gb1".
> ERROR :
> ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
abbreviation
> 'AODVIS' for table version 2
> ERROR :
>
> This is the one thing in MET that confuses me more than anything
> else-identifying the data fields in the config file.
>
> How do I know the variable and level names? What field(s) in the
GriB
> header do these correspond to in MET settings? Does it matter that
> "AODVIS" isn't the standard variable for kpds5=147? The modelers
don't
> always follow the naming; they'll put any output field in any kpds
number
> they want.
>
> Can you help me get the variables set correctly?
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 23, 2014 3:22 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. Thanks for getting the data to us.
>
> The forecast file that you sent was a raw wrfout file, which MET
doesn't
> support. The raw wrfout files have to be post-processed using
p_interp or
> UPP. UPP tends to work better because it writes GRIB files that MET
> supports well. For p_interp output, we can't read variables that
are
> defined on the staggered dimensions, like wind.
>
> We used p_interp (available here:
>
>
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing
> )
> to convert your file into a NetCDF format that MET can read.
p_interp
> also interpolates to pressure levels, and many of the grid points
for the
> first few pressure levels are actually below ground. So you'll
notice a
> lot of missing values in the first few pressure levels of the
p_interp
> output. We also made some changes to your configuration file. I
have
> attached the new
> configuration file for you to use. Here is a brief description of
the
> changes:
>
> -Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
> -Modified
> name = "PM2_5_DRY(0,0,*,*)";
> to
> name = "PM2_5_DRY";
> level = "(0,0,*,*)";
> specifying the level separately
> -Modified
> name = "obs_arr(*,4)";
> to
> name = "NCIP";
> level = "L0";
> because the obs information needs to be specified as if it were a
GRIB
> file. NCIP came from grib code 198 (specified in obs_arr)
> - Changed sid to be [] instead of ""
>
> With these changes, we were able to successfully run point_stat.
However,
> please note that the forecast values for PM2_5_DRY are all zero!
You could
> take a look at your WRF-Chem run to figure out why. I'd suggest
using
> ncview to display the PM2_5_DRY variable in the NetCDF file you sent
us.
> You'll see that it's zero for all model levels. So you may want to
look
> into that.
>
> I hope this helps.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Thanks-I got it to work on my end so you don't have to monitor it.
> Sorry
> > about the directory mixup-I created the directory and thought I
was
> > putting the files there!
> >
> > 58 MB right now thus far I see... let's hope it works!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:18 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > I understand your confusion. The file was being written to
> > incoming/irap/met_help/ as opposed to
> > incoming/irap/met_help/sittel_data/
> > where it is being written to now. I will get back to you this
> > afternoon and let you know how far along it is.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > I opened a separate window and did a list on the directory but
it said
> > > nothing was in there... which left me even more confused. I
have
> > > restarted the FTP but looks like I have no way on my end to
monitor
> > > progress... so I'll leave it running all night if I must and
hope it
> > > continues running.
> > >
> > > If you get a chance later this afternoon, please let me know how
far
> > > along the file is so I can estimate transfer speed for future
> reference.
> > >
> > > Thanks!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 1:08 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Ah, that's too bad. I had been watching it slowly come in.
I'll
> > > keep an eye out for it.
> > >
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
AFWA 16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Looks like I need to be more patient. I assumed when I got
back
> > > > to work that the file transfer was hung so I aborted it.
Turns
> > > > out it has transferred 430 MB of the full 1.3 GB of the model
data
> file.
> > > > So looks like transfer will take 6-8 hours at this rate.
> > > >
> > > > With any luck it will run to completion... sorry for the
delay,
> > > > but I don't know any better way to get you the file short of
mailing
> a DVD!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > Thank you for including the details of the situation. Could
you
> > > > please follow these instructions for posting the forecast
file,
> > > > observation file, and config file to our ftp site:
> > > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > >
> > > > I can take a closer look and try to resolve the problem.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > > Queue: met_help
> > > > > Subject: MET 4.1: Point_stat issue
> > > > > Owner: Nobody
> > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > Status: new
> > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > >
> > > > > Hello again,
> > > > >
> > > > > I'm trying to find a problem with my inability to run
point_stat
> > > > > for a pair of NetCDF files. Here's what I am seeing from
MET:
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > DEBUG 1: User Config File:
> > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > > NetCDF: Attribute not found
> > > > >
> > > > > Here's what I defined for the fields in the point_stat
config file:
> > > > >
> > > > > fcst = {
> > > > > message_type = [ "ADPSFC" ];
> > > > > field = [
> > > > > {
> > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > obs = {
> > > > > field = [
> > > > > {
> > > > > name = "obs_arr(*,4)";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > Within the forecast NetCDF file the following values are
set:
> > > > >
> > > > > dimensions:
> > > > > Time = UNLIMITED ; // (1 currently)
> > > > > west_east = 249 ;
> > > > > south_north = 149 ;
> > > > > bottom_top = 56 ;
> > > > >
> > > > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > PM2_5_DRY:stagger = "" ;
> > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > >
> > > > > Within the observation file:
> > > > >
> > > > > dimensions:
> > > > > obs_arr_len = 5 ;
> > > > > nhdr = 1820 ;
> > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > >
> > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > obs_arr:long_name = "array of observation
values" ;
> > > > > obs_arr:_fill_value = -9999.f ;
> > > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > > obs_arr:hdr_id_long_name = "index of
matching
> > > > > header
> > > > data"
> > > > > ;
> > > > > obs_arr:gc_long_name = "grib code
corresponding
> > > > > to the observation type" ;
> > > > > obs_arr:lvl_long_name = "pressure level
(hPa) or
> > > > > accumulation interval (sec)" ;
> > > > > obs_arr:hgt_long_name = "height in meters
above
> > > > > sea level or ground level (msl or agl)" ;
> > > > > obs_arr:ob_long_name = "observation value" ;
> > > > >
> > > > > Additional information:
> > > > >
> > > > > "ADPSFC" is the message type within the header of the
> > > > > observation files, but it does not appear in the forecast
file
> anywhere.
> > > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > > represents the first member of an array and not 1. The
second 0
> > > > > indicates the surface data should be used, the first of 56
levels.
> > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > > only the 5th one (numbered 4 when starting from 0) is the
actual
> > > > > data field of interest. The first * indicates to use all
1820
> > observations.
> > > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > > was built separately by a colleague.
> > > > >
> > > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > > along for you to test with. Perhaps I could FTP it
somewhere
> > > > > you could try to replicate the error?
> > > > >
> > > > > Do you see anything here that I've made a mistake with?
I've
> > > > > spent a couple hours staring at code and reading the user's
> > > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > > I'm hoping you can
> > > > help.
> > > > >
> > > > > Thanks,
> > > > > Matt
> > > > >
> > > > > // SIGNED //
> > > > > Matthew C. Sittel
> > > > > Associate Scientist
> > > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > > AFB, NE
> > > > > (402) 294-3473 DSN: 271-3473
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Oct 01 14:15:04 2014
Hi Julie,
Thanks for your response. I hope a future release of MET will allow
us simply to state kpds numbers when stating which field to verify.
I've mentioned it a couple times to Tressa and Tara as something that
would help us out. I realize it won't happen right away but I hope it
made the list of future upgrades!
Matt
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, October 01, 2014 3:09 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Hi Matt. Good questions. Unfortunately, I am not familiar enough with
the software to be able to answer your questions. John is out of the
office currently, so I can't ask him for guidance right now.
Hopefully, he will be able to help out later this week. I'm sorry I
can't be of more help to you at this time.
Julie
On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> So we've made some progress on the problems. The modelers are going
to
> produce the data in GriB format. Here's the GriB header for the
one field
> in the GriB file I need to verify:
>
> rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200 kpds7=0
> levels=(0,0) grid=255 atmos col anl:
> AODVIS=Aerosol optical depth (0.55 um) [numeric]
> timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0 num_in_ave
0
> missing 0
> center 57 subcenter 6 process 15 Table 129 scan: WE:NS winds(grid)
> latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> long 0.000000 to -0.500000 by 1.003000, (359 x 180) scan 0
> mode
> 136 bdsgrid 1
> min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239 DecScale
6
> BinScale 0
>
> How do I set the identifiers for the forecast field within the
> point_stat config file to get MET to read this? Right now I have:
>
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "AODVIS";
> level = "L0";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> obs = {
> field = [
> {
> name = "NCIP";
> level = "L0";
> cat_thresh = [ >=0.0 ];
> }
> ];
> };
>
> But this isn't working. The error I get is:
>
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_Gb1".
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> of type "FileType_Gb1".
> ERROR :
> ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> abbreviation 'AODVIS' for table version 2 ERROR :
>
> This is the one thing in MET that confuses me more than anything
> else-identifying the data fields in the config file.
>
> How do I know the variable and level names? What field(s) in the
GriB
> header do these correspond to in MET settings? Does it matter that
> "AODVIS" isn't the standard variable for kpds5=147? The modelers
> don't always follow the naming; they'll put any output field in any
> kpds number they want.
>
> Can you help me get the variables set correctly?
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 23, 2014 3:22 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. Thanks for getting the data to us.
>
> The forecast file that you sent was a raw wrfout file, which MET
> doesn't support. The raw wrfout files have to be post-processed
using
> p_interp or UPP. UPP tends to work better because it writes GRIB
> files that MET supports well. For p_interp output, we can't read
> variables that are defined on the staggered dimensions, like wind.
>
> We used p_interp (available here:
>
>
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_proc
> essing
> )
> to convert your file into a NetCDF format that MET can read.
p_interp
> also interpolates to pressure levels, and many of the grid points
for
> the first few pressure levels are actually below ground. So you'll
> notice a lot of missing values in the first few pressure levels of
the
> p_interp output. We also made some changes to your configuration
> file. I have attached the new
> configuration file for you to use. Here is a brief description of
the
> changes:
>
> -Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
> -Modified
> name = "PM2_5_DRY(0,0,*,*)";
> to
> name = "PM2_5_DRY";
> level = "(0,0,*,*)";
> specifying the level separately
> -Modified
> name = "obs_arr(*,4)";
> to
> name = "NCIP";
> level = "L0";
> because the obs information needs to be specified as if it were a
GRIB
> file. NCIP came from grib code 198 (specified in obs_arr)
> - Changed sid to be [] instead of ""
>
> With these changes, we were able to successfully run point_stat.
> However, please note that the forecast values for PM2_5_DRY are all
> zero! You could take a look at your WRF-Chem run to figure out why.
> I'd suggest using ncview to display the PM2_5_DRY variable in the
NetCDF file you sent us.
> You'll see that it's zero for all model levels. So you may want to
> look into that.
>
> I hope this helps.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Thanks-I got it to work on my end so you don't have to monitor it.
> Sorry
> > about the directory mixup-I created the directory and thought I
was
> > putting the files there!
> >
> > 58 MB right now thus far I see... let's hope it works!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:18 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > I understand your confusion. The file was being written to
> > incoming/irap/met_help/ as opposed to
> > incoming/irap/met_help/sittel_data/
> > where it is being written to now. I will get back to you this
> > afternoon and let you know how far along it is.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > I opened a separate window and did a list on the directory but
it said
> > > nothing was in there... which left me even more confused. I
have
> > > restarted the FTP but looks like I have no way on my end to
> > > monitor progress... so I'll leave it running all night if I must
> > > and hope it continues running.
> > >
> > > If you get a chance later this afternoon, please let me know how
> > > far along the file is so I can estimate transfer speed for
future
> reference.
> > >
> > > Thanks!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 1:08 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Ah, that's too bad. I had been watching it slowly come in.
I'll
> > > keep an eye out for it.
> > >
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Looks like I need to be more patient. I assumed when I got
back
> > > > to work that the file transfer was hung so I aborted it.
Turns
> > > > out it has transferred 430 MB of the full 1.3 GB of the model
> > > > data
> file.
> > > > So looks like transfer will take 6-8 hours at this rate.
> > > >
> > > > With any luck it will run to completion... sorry for the
delay,
> > > > but I don't know any better way to get you the file short of
> > > > mailing
> a DVD!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > Thank you for including the details of the situation. Could
you
> > > > please follow these instructions for posting the forecast
file,
> > > > observation file, and config file to our ftp site:
> > > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > >
> > > > I can take a closer look and try to resolve the problem.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > > Queue: met_help
> > > > > Subject: MET 4.1: Point_stat issue
> > > > > Owner: Nobody
> > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > Status: new
> > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > >
> > > > > Hello again,
> > > > >
> > > > > I'm trying to find a problem with my inability to run
> > > > > point_stat for a pair of NetCDF files. Here's what I am
seeing from MET:
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > DEBUG 1: User Config File:
> > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
> > > > > created new Met2dDataFile object of type "FileType_NcMet".
> > > > > NetCDF: Attribute not found
> > > > >
> > > > > Here's what I defined for the fields in the point_stat
config file:
> > > > >
> > > > > fcst = {
> > > > > message_type = [ "ADPSFC" ];
> > > > > field = [
> > > > > {
> > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > obs = {
> > > > > field = [
> > > > > {
> > > > > name = "obs_arr(*,4)";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > Within the forecast NetCDF file the following values are
set:
> > > > >
> > > > > dimensions:
> > > > > Time = UNLIMITED ; // (1 currently)
> > > > > west_east = 249 ;
> > > > > south_north = 149 ;
> > > > > bottom_top = 56 ;
> > > > >
> > > > > float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > PM2_5_DRY:stagger = "" ;
> > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > >
> > > > > Within the observation file:
> > > > >
> > > > > dimensions:
> > > > > obs_arr_len = 5 ;
> > > > > nhdr = 1820 ;
> > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > >
> > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > obs_arr:long_name = "array of observation
values" ;
> > > > > obs_arr:_fill_value = -9999.f ;
> > > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > > obs_arr:hdr_id_long_name = "index of
matching
> > > > > header
> > > > data"
> > > > > ;
> > > > > obs_arr:gc_long_name = "grib code
> > > > > corresponding to the observation type" ;
> > > > > obs_arr:lvl_long_name = "pressure level
(hPa)
> > > > > or accumulation interval (sec)" ;
> > > > > obs_arr:hgt_long_name = "height in meters
> > > > > above sea level or ground level (msl or agl)" ;
> > > > > obs_arr:ob_long_name = "observation value" ;
> > > > >
> > > > > Additional information:
> > > > >
> > > > > "ADPSFC" is the message type within the header of the
> > > > > observation files, but it does not appear in the forecast
file
> anywhere.
> > > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > > represents the first member of an array and not 1. The
second
> > > > > 0 indicates the surface data should be used, the first of 56
levels.
> > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > > only the 5th one (numbered 4 when starting from 0) is the
> > > > > actual data field of interest. The first * indicates to use
> > > > > all 1820
> > observations.
> > > > > I built the observation file using 'ascii2nc'; the forecast
> > > > > file was built separately by a colleague.
> > > > >
> > > > > The observation file is ~350KB so I could easily e-mail it,
> > > > > but the model data is ~1.3GB, so obviously I can't easily
send
> > > > > that along for you to test with. Perhaps I could FTP it
> > > > > somewhere you could try to replicate the error?
> > > > >
> > > > > Do you see anything here that I've made a mistake with?
I've
> > > > > spent a couple hours staring at code and reading the user's
> > > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > > I'm hoping you can
> > > > help.
> > > > >
> > > > > Thanks,
> > > > > Matt
> > > > >
> > > > > // SIGNED //
> > > > > Matthew C. Sittel
> > > > > Associate Scientist
> > > > > University Corporation for Atmospheric Research 16WS/WXN,
> > > > > Offutt AFB, NE
> > > > > (402) 294-3473 DSN: 271-3473
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Thu Oct 02 14:10:08 2014
Matt,
We have a MET development issue defined in JIRA to track your
suggestion
about supporting KPDS selection of GRIB1 records directly and will
consider
it for the next release. If/when we have funding to work on that, we
may
contact you to be our guinea pig!
To answer your question about GRIB1 tables... MET reads that data from
a
file named "nceptab_flat.txt". Look in the directory where you've
installed MET for the file named
"share/met/table_files/nceptab_flat.txt".
If your usage of GRIB1 records differs from NCEP's usage, you can edit
this
file to specify what abbreviations you'd like to use.
I do not see AODVIS in that file. It looks like GRIB record 147 of
parameter table version 2 is set to UGWD for "Zonal flux of gravity
wave
stress":
147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
You could change that to your local usage of the GRIB record. But
first,
you should check how they've encoded the GRIB parameter table version
number for that record. Run wgrib with the "-PDS10" option and you'll
see
output looking something like this:
PDS10(1..28)=
1: 0 0 28 2 59 89 212 192 11 100 3 207 5 8 7
0
0 1 9 0
21: 0 0 0 0 21 0 0 0
The 4th number listed will tell you the parameter table version number
for
that record. In this case, the ptv is 2. Whichever it is, find and
edit
the line in that "nceptab_flat.txt" file that begins with "147 ###"
where
### is the ptv number.
Hope that helps!
Thanks,
John
On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Thanks for your response. I hope a future release of MET will
allow us
> simply to state kpds numbers when stating which field to verify.
I've
> mentioned it a couple times to Tressa and Tara as something that
would help
> us out. I realize it won't happen right away but I hope it made the
list
> of future upgrades!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, October 01, 2014 3:09 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. Good questions. Unfortunately, I am not familiar enough
with the
> software to be able to answer your questions. John is out of the
office
> currently, so I can't ask him for guidance right now. Hopefully, he
will
> be able to help out later this week. I'm sorry I can't be of more
help to
> you at this time.
>
> Julie
>
> On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi Julie,
> >
> > So we've made some progress on the problems. The modelers are
going to
> > produce the data in GriB format. Here's the GriB header for the
one
> field
> > in the GriB file I need to verify:
> >
> > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
kpds7=0
> > levels=(0,0) grid=255 atmos col anl:
> > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
num_in_ave 0
> > missing 0
> > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
winds(grid)
> > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> > long 0.000000 to -0.500000 by 1.003000, (359 x 180) scan
0
> > mode
> > 136 bdsgrid 1
> > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
DecScale 6
> > BinScale 0
> >
> > How do I set the identifiers for the forecast field within the
> > point_stat config file to get MET to read this? Right now I have:
> >
> > fcst = {
> > message_type = [ "ADPSFC" ];
> > field = [
> > {
> > name = "AODVIS";
> > level = "L0";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > obs = {
> > field = [
> > {
> > name = "NCIP";
> > level = "L0";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > But this isn't working. The error I get is:
> >
> > DEBUG 1: User Config File:
> > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_Gb1".
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
> > of type "FileType_Gb1".
> > ERROR :
> > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > abbreviation 'AODVIS' for table version 2 ERROR :
> >
> > This is the one thing in MET that confuses me more than anything
> > else-identifying the data fields in the config file.
> >
> > How do I know the variable and level names? What field(s) in the
GriB
> > header do these correspond to in MET settings? Does it matter
that
> > "AODVIS" isn't the standard variable for kpds5=147? The modelers
> > don't always follow the naming; they'll put any output field in
any
> > kpds number they want.
> >
> > Can you help me get the variables set correctly?
> >
> > Thanks,
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, September 23, 2014 3:22 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt. Thanks for getting the data to us.
> >
> > The forecast file that you sent was a raw wrfout file, which MET
> > doesn't support. The raw wrfout files have to be post-processed
using
> > p_interp or UPP. UPP tends to work better because it writes GRIB
> > files that MET supports well. For p_interp output, we can't read
> > variables that are defined on the staggered dimensions, like wind.
> >
> > We used p_interp (available here:
> >
> >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_proc
> > essing
> > )
> > to convert your file into a NetCDF format that MET can read.
p_interp
> > also interpolates to pressure levels, and many of the grid points
for
> > the first few pressure levels are actually below ground. So
you'll
> > notice a lot of missing values in the first few pressure levels of
the
> > p_interp output. We also made some changes to your configuration
> > file. I have attached the new
> > configuration file for you to use. Here is a brief description
of the
> > changes:
> >
> > -Moved message_type = [ "ADPSFC" ]; to be within the obs
dictionary
> > -Modified
> > name = "PM2_5_DRY(0,0,*,*)";
> > to
> > name = "PM2_5_DRY";
> > level = "(0,0,*,*)";
> > specifying the level separately
> > -Modified
> > name = "obs_arr(*,4)";
> > to
> > name = "NCIP";
> > level = "L0";
> > because the obs information needs to be specified as if it were a
GRIB
> > file. NCIP came from grib code 198 (specified in obs_arr)
> > - Changed sid to be [] instead of ""
> >
> > With these changes, we were able to successfully run point_stat.
> > However, please note that the forecast values for PM2_5_DRY are
all
> > zero! You could take a look at your WRF-Chem run to figure out
why.
> > I'd suggest using ncview to display the PM2_5_DRY variable in the
NetCDF
> file you sent us.
> > You'll see that it's zero for all model levels. So you may want
to
> > look into that.
> >
> > I hope this helps.
> >
> > Thanks,
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Thanks-I got it to work on my end so you don't have to monitor
it.
> > Sorry
> > > about the directory mixup-I created the directory and thought I
was
> > > putting the files there!
> > >
> > > 58 MB right now thus far I see... let's hope it works!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 1:18 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > I understand your confusion. The file was being written to
> > > incoming/irap/met_help/ as opposed to
> > > incoming/irap/met_help/sittel_data/
> > > where it is being written to now. I will get back to you this
> > > afternoon and let you know how far along it is.
> > >
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF
AFWA 16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > I opened a separate window and did a list on the directory but
it
> said
> > > > nothing was in there... which left me even more confused. I
have
> > > > restarted the FTP but looks like I have no way on my end to
> > > > monitor progress... so I'll leave it running all night if I
must
> > > > and hope it continues running.
> > > >
> > > > If you get a chance later this afternoon, please let me know
how
> > > > far along the file is so I can estimate transfer speed for
future
> > reference.
> > > >
> > > > Thanks!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Ah, that's too bad. I had been watching it slowly come in.
I'll
> > > > keep an eye out for it.
> > > >
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > > > >
> > > > > Looks like I need to be more patient. I assumed when I got
back
> > > > > to work that the file transfer was hung so I aborted it.
Turns
> > > > > out it has transferred 430 MB of the full 1.3 GB of the
model
> > > > > data
> > file.
> > > > > So looks like transfer will take 6-8 hours at this rate.
> > > > >
> > > > > With any luck it will run to completion... sorry for the
delay,
> > > > > but I don't know any better way to get you the file short of
> > > > > mailing
> > a DVD!
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > > >
> > > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > > Thank you for including the details of the situation. Could
you
> > > > > please follow these instructions for posting the forecast
file,
> > > > > observation file, and config file to our ftp site:
> > > > >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > >
> > > > > I can take a closer look and try to resolve the problem.
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > > Transaction: Ticket created by
matthew.sittel.ctr at us.af.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > Owner: Nobody
> > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Hello again,
> > > > > >
> > > > > > I'm trying to find a problem with my inability to run
> > > > > > point_stat for a pair of NetCDF files. Here's what I am
seeing
> from MET:
> > > > > >
> > > > > > DEBUG 1: Default Config File:
> > > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > DEBUG 1: User Config File:
> > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
> > > > > > created new Met2dDataFile object of type "FileType_NcMet".
> > > > > > NetCDF: Attribute not found
> > > > > >
> > > > > > Here's what I defined for the fields in the point_stat
config
> file:
> > > > > >
> > > > > > fcst = {
> > > > > > message_type = [ "ADPSFC" ];
> > > > > > field = [
> > > > > > {
> > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > cat_thresh = [ >=0.0 ];
> > > > > > }
> > > > > > ];
> > > > > > };
> > > > > >
> > > > > > obs = {
> > > > > > field = [
> > > > > > {
> > > > > > name = "obs_arr(*,4)";
> > > > > > cat_thresh = [ >=0.0 ];
> > > > > > }
> > > > > > ];
> > > > > > };
> > > > > >
> > > > > > Within the forecast NetCDF file the following values are
set:
> > > > > >
> > > > > > dimensions:
> > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > west_east = 249 ;
> > > > > > south_north = 149 ;
> > > > > > bottom_top = 56 ;
> > > > > >
> > > > > > float PM2_5_DRY(Time, bottom_top, south_north,
> west_east) ;
> > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass"
> ;
> > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > > >
> > > > > > Within the observation file:
> > > > > >
> > > > > > dimensions:
> > > > > > obs_arr_len = 5 ;
> > > > > > nhdr = 1820 ;
> > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > >
> > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > obs_arr:long_name = "array of observation
> values" ;
> > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > > > obs_arr:hdr_id_long_name = "index of
matching
> > > > > > header
> > > > > data"
> > > > > > ;
> > > > > > obs_arr:gc_long_name = "grib code
> > > > > > corresponding to the observation type" ;
> > > > > > obs_arr:lvl_long_name = "pressure level
(hPa)
> > > > > > or accumulation interval (sec)" ;
> > > > > > obs_arr:hgt_long_name = "height in meters
> > > > > > above sea level or ground level (msl or agl)" ;
> > > > > > obs_arr:ob_long_name = "observation value"
;
> > > > > >
> > > > > > Additional information:
> > > > > >
> > > > > > "ADPSFC" is the message type within the header of the
> > > > > > observation files, but it does not appear in the forecast
file
> > anywhere.
> > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming
0
> > > > > > represents the first member of an array and not 1. The
second
> > > > > > 0 indicates the surface data should be used, the first of
56
> levels.
> > > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > > I am using obs_arr(*,4) because there are 5 data fields,
and
> > > > > > only the 5th one (numbered 4 when starting from 0) is the
> > > > > > actual data field of interest. The first * indicates to
use
> > > > > > all 1820
> > > observations.
> > > > > > I built the observation file using 'ascii2nc'; the
forecast
> > > > > > file was built separately by a colleague.
> > > > > >
> > > > > > The observation file is ~350KB so I could easily e-mail
it,
> > > > > > but the model data is ~1.3GB, so obviously I can't easily
send
> > > > > > that along for you to test with. Perhaps I could FTP it
> > > > > > somewhere you could try to replicate the error?
> > > > > >
> > > > > > Do you see anything here that I've made a mistake with?
I've
> > > > > > spent a couple hours staring at code and reading the
user's
> > > > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > > > I'm hoping you can
> > > > > help.
> > > > > >
> > > > > > Thanks,
> > > > > > Matt
> > > > > >
> > > > > > // SIGNED //
> > > > > > Matthew C. Sittel
> > > > > > Associate Scientist
> > > > > > University Corporation for Atmospheric Research 16WS/WXN,
> > > > > > Offutt AFB, NE
> > > > > > (402) 294-3473 DSN: 271-3473
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Thu Oct 02 14:26:22 2014
Hi John,
So when nceptab_flat.txt is read, which field(s) must match what is
inside my GriB file to allow processing to continue? In the version I
have for MET 4.1 I have:
147 2 "U-GWD" "Zonal gravity wave stress" "N/m^2"
I didn't know there was a hyphen but maybe that's part of my problem,
since it isn't an exact match. Or do I need to change not only the
variable name, but the descriptor and units to where everything agrees
with my GriB header?
Do I need to recompile MET if I change this file, or is it only read
at runtime?
I would be happy to test the KPDS number feature... we don't use
record number since there is no guarantee it will be a constant from
file to file, so using kpds numbers allows us to match that.
Sometimes, though, those can be still ambiguous and we have to look at
P1 and P2 fields also. So there might need to be some more elegant
method to list which GriB parameters should be checked to guarantee
uniqueness in the number of records returned (i.e., no more than one
field).
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, October 02, 2014 3:10 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Matt,
We have a MET development issue defined in JIRA to track your
suggestion about supporting KPDS selection of GRIB1 records directly
and will consider it for the next release. If/when we have funding to
work on that, we may contact you to be our guinea pig!
To answer your question about GRIB1 tables... MET reads that data from
a file named "nceptab_flat.txt". Look in the directory where you've
installed MET for the file named
"share/met/table_files/nceptab_flat.txt".
If your usage of GRIB1 records differs from NCEP's usage, you can edit
this file to specify what abbreviations you'd like to use.
I do not see AODVIS in that file. It looks like GRIB record 147 of
parameter table version 2 is set to UGWD for "Zonal flux of gravity
wave
stress":
147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
You could change that to your local usage of the GRIB record. But
first, you should check how they've encoded the GRIB parameter table
version number for that record. Run wgrib with the "-PDS10" option
and you'll see output looking something like this:
PDS10(1..28)=
1: 0 0 28 2 59 89 212 192 11 100 3 207 5 8 7
0
0 1 9 0
21: 0 0 0 0 21 0 0 0
The 4th number listed will tell you the parameter table version number
for that record. In this case, the ptv is 2. Whichever it is, find
and edit the line in that "nceptab_flat.txt" file that begins with
"147 ###" where ### is the ptv number.
Hope that helps!
Thanks,
John
On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Thanks for your response. I hope a future release of MET will
allow us
> simply to state kpds numbers when stating which field to verify.
I've
> mentioned it a couple times to Tressa and Tara as something that
would
> help us out. I realize it won't happen right away but I hope it
made
> the list of future upgrades!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, October 01, 2014 3:09 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt. Good questions. Unfortunately, I am not familiar enough
with
> the software to be able to answer your questions. John is out of
the
> office currently, so I can't ask him for guidance right now.
> Hopefully, he will be able to help out later this week. I'm sorry I
> can't be of more help to you at this time.
>
> Julie
>
> On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi Julie,
> >
> > So we've made some progress on the problems. The modelers are
going to
> > produce the data in GriB format. Here's the GriB header for the
one
> field
> > in the GriB file I need to verify:
> >
> > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
kpds7=0
> > levels=(0,0) grid=255 atmos col anl:
> > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
num_in_ave
> > 0 missing 0
> > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
winds(grid)
> > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> > long 0.000000 to -0.500000 by 1.003000, (359 x 180) scan
0
> > mode
> > 136 bdsgrid 1
> > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
DecScale
> > 6 BinScale 0
> >
> > How do I set the identifiers for the forecast field within the
> > point_stat config file to get MET to read this? Right now I have:
> >
> > fcst = {
> > message_type = [ "ADPSFC" ];
> > field = [
> > {
> > name = "AODVIS";
> > level = "L0";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > obs = {
> > field = [
> > {
> > name = "NCIP";
> > level = "L0";
> > cat_thresh = [ >=0.0 ];
> > }
> > ];
> > };
> >
> > But this isn't working. The error I get is:
> >
> > DEBUG 1: User Config File:
> > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_Gb1".
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > object of type "FileType_Gb1".
> > ERROR :
> > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > abbreviation 'AODVIS' for table version 2 ERROR :
> >
> > This is the one thing in MET that confuses me more than anything
> > else-identifying the data fields in the config file.
> >
> > How do I know the variable and level names? What field(s) in the
> > GriB header do these correspond to in MET settings? Does it
matter
> > that "AODVIS" isn't the standard variable for kpds5=147? The
> > modelers don't always follow the naming; they'll put any output
> > field in any kpds number they want.
> >
> > Can you help me get the variables set correctly?
> >
> > Thanks,
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, September 23, 2014 3:22 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt. Thanks for getting the data to us.
> >
> > The forecast file that you sent was a raw wrfout file, which MET
> > doesn't support. The raw wrfout files have to be post-processed
> > using p_interp or UPP. UPP tends to work better because it writes
> > GRIB files that MET supports well. For p_interp output, we can't
> > read variables that are defined on the staggered dimensions, like
wind.
> >
> > We used p_interp (available here:
> >
> >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_pr
> > oc
> > essing
> > )
> > to convert your file into a NetCDF format that MET can read.
> > p_interp also interpolates to pressure levels, and many of the
grid
> > points for the first few pressure levels are actually below
ground.
> > So you'll notice a lot of missing values in the first few pressure
> > levels of the p_interp output. We also made some changes to your
> > configuration file. I have attached the new
> > configuration file for you to use. Here is a brief description
of the
> > changes:
> >
> > -Moved message_type = [ "ADPSFC" ]; to be within the obs
dictionary
> > -Modified
> > name = "PM2_5_DRY(0,0,*,*)"; to
> > name = "PM2_5_DRY";
> > level = "(0,0,*,*)";
> > specifying the level separately
> > -Modified
> > name = "obs_arr(*,4)";
> > to
> > name = "NCIP";
> > level = "L0";
> > because the obs information needs to be specified as if it were a
> > GRIB file. NCIP came from grib code 198 (specified in obs_arr)
> > - Changed sid to be [] instead of ""
> >
> > With these changes, we were able to successfully run point_stat.
> > However, please note that the forecast values for PM2_5_DRY are
all
> > zero! You could take a look at your WRF-Chem run to figure out
why.
> > I'd suggest using ncview to display the PM2_5_DRY variable in the
> > NetCDF
> file you sent us.
> > You'll see that it's zero for all model levels. So you may want
to
> > look into that.
> >
> > I hope this helps.
> >
> > Thanks,
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Thanks-I got it to work on my end so you don't have to monitor
it.
> > Sorry
> > > about the directory mixup-I created the directory and thought I
> > > was putting the files there!
> > >
> > > 58 MB right now thus far I see... let's hope it works!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 1:18 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > I understand your confusion. The file was being written to
> > > incoming/irap/met_help/ as opposed to
> > > incoming/irap/met_help/sittel_data/
> > > where it is being written to now. I will get back to you this
> > > afternoon and let you know how far along it is.
> > >
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > I opened a separate window and did a list on the directory but
> > > > it
> said
> > > > nothing was in there... which left me even more confused. I
have
> > > > restarted the FTP but looks like I have no way on my end to
> > > > monitor progress... so I'll leave it running all night if I
must
> > > > and hope it continues running.
> > > >
> > > > If you get a chance later this afternoon, please let me know
how
> > > > far along the file is so I can estimate transfer speed for
> > > > future
> > reference.
> > > >
> > > > Thanks!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Ah, that's too bad. I had been watching it slowly come in.
> > > > I'll keep an eye out for it.
> > > >
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
> > > > AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > > Looks like I need to be more patient. I assumed when I got
> > > > > back to work that the file transfer was hung so I aborted
it.
> > > > > Turns out it has transferred 430 MB of the full 1.3 GB of
the
> > > > > model data
> > file.
> > > > > So looks like transfer will take 6-8 hours at this rate.
> > > > >
> > > > > With any luck it will run to completion... sorry for the
> > > > > delay, but I don't know any better way to get you the file
> > > > > short of mailing
> > a DVD!
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > issue
> > > > >
> > > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > > Thank you for including the details of the situation. Could
> > > > > you please follow these instructions for posting the
forecast
> > > > > file, observation file, and config file to our ftp site:
> > > > >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > >
> > > > > I can take a closer look and try to resolve the problem.
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF
> > > > > AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > > Transaction: Ticket created by
matthew.sittel.ctr at us.af.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > Owner: Nobody
> > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Hello again,
> > > > > >
> > > > > > I'm trying to find a problem with my inability to run
> > > > > > point_stat for a pair of NetCDF files. Here's what I am
> > > > > > seeing
> from MET:
> > > > > >
> > > > > > DEBUG 1: Default Config File:
> > > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > DEBUG 1: User Config File:
> > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
> > > > > > created new Met2dDataFile object of type "FileType_NcMet".
> > > > > > NetCDF: Attribute not found
> > > > > >
> > > > > > Here's what I defined for the fields in the point_stat
> > > > > > config
> file:
> > > > > >
> > > > > > fcst = {
> > > > > > message_type = [ "ADPSFC" ];
> > > > > > field = [
> > > > > > {
> > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > cat_thresh = [ >=0.0 ];
> > > > > > }
> > > > > > ];
> > > > > > };
> > > > > >
> > > > > > obs = {
> > > > > > field = [
> > > > > > {
> > > > > > name = "obs_arr(*,4)";
> > > > > > cat_thresh = [ >=0.0 ];
> > > > > > }
> > > > > > ];
> > > > > > };
> > > > > >
> > > > > > Within the forecast NetCDF file the following values are
set:
> > > > > >
> > > > > > dimensions:
> > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > west_east = 249 ;
> > > > > > south_north = 149 ;
> > > > > > bottom_top = 56 ;
> > > > > >
> > > > > > float PM2_5_DRY(Time, bottom_top, south_north,
> west_east) ;
> > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > PM2_5_DRY:description = "pm2.5 aerosol dry
mass"
> ;
> > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > > >
> > > > > > Within the observation file:
> > > > > >
> > > > > > dimensions:
> > > > > > obs_arr_len = 5 ;
> > > > > > nhdr = 1820 ;
> > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > >
> > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > obs_arr:long_name = "array of observation
> values" ;
> > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > > > obs_arr:hdr_id_long_name = "index of
> > > > > > matching header
> > > > > data"
> > > > > > ;
> > > > > > obs_arr:gc_long_name = "grib code
> > > > > > corresponding to the observation type" ;
> > > > > > obs_arr:lvl_long_name = "pressure level
> > > > > > (hPa) or accumulation interval (sec)" ;
> > > > > > obs_arr:hgt_long_name = "height in meters
> > > > > > above sea level or ground level (msl or agl)" ;
> > > > > > obs_arr:ob_long_name = "observation value"
;
> > > > > >
> > > > > > Additional information:
> > > > > >
> > > > > > "ADPSFC" is the message type within the header of the
> > > > > > observation files, but it does not appear in the forecast
> > > > > > file
> > anywhere.
> > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming
0
> > > > > > represents the first member of an array and not 1. The
> > > > > > second
> > > > > > 0 indicates the surface data should be used, the first of
56
> levels.
> > > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > > I am using obs_arr(*,4) because there are 5 data fields,
and
> > > > > > only the 5th one (numbered 4 when starting from 0) is the
> > > > > > actual data field of interest. The first * indicates to
use
> > > > > > all 1820
> > > observations.
> > > > > > I built the observation file using 'ascii2nc'; the
forecast
> > > > > > file was built separately by a colleague.
> > > > > >
> > > > > > The observation file is ~350KB so I could easily e-mail
it,
> > > > > > but the model data is ~1.3GB, so obviously I can't easily
> > > > > > send that along for you to test with. Perhaps I could FTP
> > > > > > it somewhere you could try to replicate the error?
> > > > > >
> > > > > > Do you see anything here that I've made a mistake with?
> > > > > > I've spent a couple hours staring at code and reading the
> > > > > > user's manual; I can't find any guidance as to where I
made the mistake.
> > > > > > I'm hoping you can
> > > > > help.
> > > > > >
> > > > > > Thanks,
> > > > > > Matt
> > > > > >
> > > > > > // SIGNED //
> > > > > > Matthew C. Sittel
> > > > > > Associate Scientist
> > > > > > University Corporation for Atmospheric Research 16WS/WXN,
> > > > > > Offutt AFB, NE
> > > > > > (402) 294-3473 DSN: 271-3473
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Thu Oct 02 14:31:21 2014
Matt,
You don't need to recompile since this table is read in at runtime.
Did
you already send us a sample file with this data in it? Might be
easiest
if I could just get my hands on the file and telly you exactly what
needs
to be done.
Thanks,
John
On Thu, Oct 2, 2014 at 2:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi John,
>
> So when nceptab_flat.txt is read, which field(s) must match what is
inside
> my GriB file to allow processing to continue? In the version I have
for
> MET 4.1 I have:
>
> 147 2 "U-GWD" "Zonal gravity wave stress" "N/m^2"
>
> I didn't know there was a hyphen but maybe that's part of my
problem,
> since it isn't an exact match. Or do I need to change not only the
> variable name, but the descriptor and units to where everything
agrees with
> my GriB header?
>
> Do I need to recompile MET if I change this file, or is it only read
at
> runtime?
>
> I would be happy to test the KPDS number feature... we don't use
record
> number since there is no guarantee it will be a constant from file
to file,
> so using kpds numbers allows us to match that. Sometimes, though,
those
> can be still ambiguous and we have to look at P1 and P2 fields also.
So
> there might need to be some more elegant method to list which GriB
> parameters should be checked to guarantee uniqueness in the number
of
> records returned (i.e., no more than one field).
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, October 02, 2014 3:10 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Matt,
>
> We have a MET development issue defined in JIRA to track your
suggestion
> about supporting KPDS selection of GRIB1 records directly and will
consider
> it for the next release. If/when we have funding to work on that,
we may
> contact you to be our guinea pig!
>
> To answer your question about GRIB1 tables... MET reads that data
from a
> file named "nceptab_flat.txt". Look in the directory where you've
> installed MET for the file named
"share/met/table_files/nceptab_flat.txt".
> If your usage of GRIB1 records differs from NCEP's usage, you can
edit
> this file to specify what abbreviations you'd like to use.
>
> I do not see AODVIS in that file. It looks like GRIB record 147 of
> parameter table version 2 is set to UGWD for "Zonal flux of gravity
wave
> stress":
> 147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
>
> You could change that to your local usage of the GRIB record. But
first,
> you should check how they've encoded the GRIB parameter table
version
> number for that record. Run wgrib with the "-PDS10" option and
you'll see
> output looking something like this:
>
> PDS10(1..28)=
> 1: 0 0 28 2 59 89 212 192 11 100 3 207 5 8
7 0
> 0 1 9 0
> 21: 0 0 0 0 21 0 0 0
>
> The 4th number listed will tell you the parameter table version
number for
> that record. In this case, the ptv is 2. Whichever it is, find and
edit
> the line in that "nceptab_flat.txt" file that begins with "147 ###"
where
> ### is the ptv number.
>
> Hope that helps!
>
> Thanks,
> John
>
> On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi Julie,
> >
> > Thanks for your response. I hope a future release of MET will
allow us
> > simply to state kpds numbers when stating which field to verify.
I've
> > mentioned it a couple times to Tressa and Tara as something that
would
> > help us out. I realize it won't happen right away but I hope it
made
> > the list of future upgrades!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, October 01, 2014 3:09 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt. Good questions. Unfortunately, I am not familiar enough
with
> > the software to be able to answer your questions. John is out of
the
> > office currently, so I can't ask him for guidance right now.
> > Hopefully, he will be able to help out later this week. I'm sorry
I
> > can't be of more help to you at this time.
> >
> > Julie
> >
> > On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Hi Julie,
> > >
> > > So we've made some progress on the problems. The modelers are
going to
> > > produce the data in GriB format. Here's the GriB header for
the one
> > field
> > > in the GriB file I need to verify:
> > >
> > > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
kpds7=0
> > > levels=(0,0) grid=255 atmos col anl:
> > > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
num_in_ave
> > > 0 missing 0
> > > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
winds(grid)
> > > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> > > long 0.000000 to -0.500000 by 1.003000, (359 x 180)
scan 0
> > > mode
> > > 136 bdsgrid 1
> > > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
DecScale
> > > 6 BinScale 0
> > >
> > > How do I set the identifiers for the forecast field within the
> > > point_stat config file to get MET to read this? Right now I
have:
> > >
> > > fcst = {
> > > message_type = [ "ADPSFC" ];
> > > field = [
> > > {
> > > name = "AODVIS";
> > > level = "L0";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "NCIP";
> > > level = "L0";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > But this isn't working. The error I get is:
> > >
> > > DEBUG 1: User Config File:
> > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > > Met2dDataFile object of type "FileType_Gb1".
> > > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > > object of type "FileType_Gb1".
> > > ERROR :
> > > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > > abbreviation 'AODVIS' for table version 2 ERROR :
> > >
> > > This is the one thing in MET that confuses me more than anything
> > > else-identifying the data fields in the config file.
> > >
> > > How do I know the variable and level names? What field(s) in
the
> > > GriB header do these correspond to in MET settings? Does it
matter
> > > that "AODVIS" isn't the standard variable for kpds5=147? The
> > > modelers don't always follow the naming; they'll put any output
> > > field in any kpds number they want.
> > >
> > > Can you help me get the variables set correctly?
> > >
> > > Thanks,
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Tuesday, September 23, 2014 3:22 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. Thanks for getting the data to us.
> > >
> > > The forecast file that you sent was a raw wrfout file, which MET
> > > doesn't support. The raw wrfout files have to be post-processed
> > > using p_interp or UPP. UPP tends to work better because it
writes
> > > GRIB files that MET supports well. For p_interp output, we
can't
> > > read variables that are defined on the staggered dimensions,
like wind.
> > >
> > > We used p_interp (available here:
> > >
> > >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_pr
> > > oc
> > > essing
> > > )
> > > to convert your file into a NetCDF format that MET can read.
> > > p_interp also interpolates to pressure levels, and many of the
grid
> > > points for the first few pressure levels are actually below
ground.
> > > So you'll notice a lot of missing values in the first few
pressure
> > > levels of the p_interp output. We also made some changes to
your
> > > configuration file. I have attached the new
> > > configuration file for you to use. Here is a brief description
of the
> > > changes:
> > >
> > > -Moved message_type = [ "ADPSFC" ]; to be within the obs
dictionary
> > > -Modified
> > > name = "PM2_5_DRY(0,0,*,*)"; to
> > > name = "PM2_5_DRY";
> > > level = "(0,0,*,*)";
> > > specifying the level separately
> > > -Modified
> > > name = "obs_arr(*,4)";
> > > to
> > > name = "NCIP";
> > > level = "L0";
> > > because the obs information needs to be specified as if it were
a
> > > GRIB file. NCIP came from grib code 198 (specified in obs_arr)
> > > - Changed sid to be [] instead of ""
> > >
> > > With these changes, we were able to successfully run point_stat.
> > > However, please note that the forecast values for PM2_5_DRY are
all
> > > zero! You could take a look at your WRF-Chem run to figure out
why.
> > > I'd suggest using ncview to display the PM2_5_DRY variable in
the
> > > NetCDF
> > file you sent us.
> > > You'll see that it's zero for all model levels. So you may want
to
> > > look into that.
> > >
> > > I hope this helps.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF
AFWA 16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Thanks-I got it to work on my end so you don't have to monitor
it.
> > > Sorry
> > > > about the directory mixup-I created the directory and thought
I
> > > > was putting the files there!
> > > >
> > > > 58 MB right now thus far I see... let's hope it works!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 1:18 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > I understand your confusion. The file was being written to
> > > > incoming/irap/met_help/ as opposed to
> > > > incoming/irap/met_help/sittel_data/
> > > > where it is being written to now. I will get back to you this
> > > > afternoon and let you know how far along it is.
> > > >
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > > > >
> > > > > I opened a separate window and did a list on the directory
but
> > > > > it
> > said
> > > > > nothing was in there... which left me even more confused.
I have
> > > > > restarted the FTP but looks like I have no way on my end to
> > > > > monitor progress... so I'll leave it running all night if I
must
> > > > > and hope it continues running.
> > > > >
> > > > > If you get a chance later this afternoon, please let me know
how
> > > > > far along the file is so I can estimate transfer speed for
> > > > > future
> > > reference.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > > >
> > > > > Ah, that's too bad. I had been watching it slowly come in.
> > > > > I'll keep an eye out for it.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
> > > > > AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > > Looks like I need to be more patient. I assumed when I
got
> > > > > > back to work that the file transfer was hung so I aborted
it.
> > > > > > Turns out it has transferred 430 MB of the full 1.3 GB of
the
> > > > > > model data
> > > file.
> > > > > > So looks like transfer will take 6-8 hours at this rate.
> > > > > >
> > > > > > With any luck it will run to completion... sorry for the
> > > > > > delay, but I don't know any better way to get you the file
> > > > > > short of mailing
> > > a DVD!
> > > > > >
> > > > > > Matt
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > > issue
> > > > > >
> > > > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > > > Thank you for including the details of the situation.
Could
> > > > > > you please follow these instructions for posting the
forecast
> > > > > > file, observation file, and config file to our ftp site:
> > > > > >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > > >
> > > > > > I can take a closer look and try to resolve the problem.
> > > > > >
> > > > > > Thanks,
> > > > > > Julie
> > > > > >
> > > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR
USAF
> > > > > > AFWA
> > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > > > Transaction: Ticket created by
matthew.sittel.ctr at us.af.mil
> > > > > > > Queue: met_help
> > > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > > Owner: Nobody
> > > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hello again,
> > > > > > >
> > > > > > > I'm trying to find a problem with my inability to run
> > > > > > > point_stat for a pair of NetCDF files. Here's what I am
> > > > > > > seeing
> > from MET:
> > > > > > >
> > > > > > > DEBUG 1: Default Config File:
> > > > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > > DEBUG 1: User Config File:
> > > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
> > > > > > > created new Met2dDataFile object of type
"FileType_NcMet".
> > > > > > > NetCDF: Attribute not found
> > > > > > >
> > > > > > > Here's what I defined for the fields in the point_stat
> > > > > > > config
> > file:
> > > > > > >
> > > > > > > fcst = {
> > > > > > > message_type = [ "ADPSFC" ];
> > > > > > > field = [
> > > > > > > {
> > > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > }
> > > > > > > ];
> > > > > > > };
> > > > > > >
> > > > > > > obs = {
> > > > > > > field = [
> > > > > > > {
> > > > > > > name = "obs_arr(*,4)";
> > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > }
> > > > > > > ];
> > > > > > > };
> > > > > > >
> > > > > > > Within the forecast NetCDF file the following values are
set:
> > > > > > >
> > > > > > > dimensions:
> > > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > > west_east = 249 ;
> > > > > > > south_north = 149 ;
> > > > > > > bottom_top = 56 ;
> > > > > > >
> > > > > > > float PM2_5_DRY(Time, bottom_top, south_north,
> > west_east) ;
> > > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > > PM2_5_DRY:description = "pm2.5 aerosol
dry
> mass"
> > ;
> > > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > > > >
> > > > > > > Within the observation file:
> > > > > > >
> > > > > > > dimensions:
> > > > > > > obs_arr_len = 5 ;
> > > > > > > nhdr = 1820 ;
> > > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > > >
> > > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > > obs_arr:long_name = "array of
observation
> > values" ;
> > > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > > obs_arr:columns = "hdr_id gc lvl hgt ob"
;
> > > > > > > obs_arr:hdr_id_long_name = "index of
> > > > > > > matching header
> > > > > > data"
> > > > > > > ;
> > > > > > > obs_arr:gc_long_name = "grib code
> > > > > > > corresponding to the observation type" ;
> > > > > > > obs_arr:lvl_long_name = "pressure level
> > > > > > > (hPa) or accumulation interval (sec)" ;
> > > > > > > obs_arr:hgt_long_name = "height in
meters
> > > > > > > above sea level or ground level (msl or agl)" ;
> > > > > > > obs_arr:ob_long_name = "observation
value" ;
> > > > > > >
> > > > > > > Additional information:
> > > > > > >
> > > > > > > "ADPSFC" is the message type within the header of the
> > > > > > > observation files, but it does not appear in the
forecast
> > > > > > > file
> > > anywhere.
> > > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation
assuming 0
> > > > > > > represents the first member of an array and not 1. The
> > > > > > > second
> > > > > > > 0 indicates the surface data should be used, the first
of 56
> > levels.
> > > > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > > > I am using obs_arr(*,4) because there are 5 data fields,
and
> > > > > > > only the 5th one (numbered 4 when starting from 0) is
the
> > > > > > > actual data field of interest. The first * indicates to
use
> > > > > > > all 1820
> > > > observations.
> > > > > > > I built the observation file using 'ascii2nc'; the
forecast
> > > > > > > file was built separately by a colleague.
> > > > > > >
> > > > > > > The observation file is ~350KB so I could easily e-mail
it,
> > > > > > > but the model data is ~1.3GB, so obviously I can't
easily
> > > > > > > send that along for you to test with. Perhaps I could
FTP
> > > > > > > it somewhere you could try to replicate the error?
> > > > > > >
> > > > > > > Do you see anything here that I've made a mistake with?
> > > > > > > I've spent a couple hours staring at code and reading
the
> > > > > > > user's manual; I can't find any guidance as to where I
made
> the mistake.
> > > > > > > I'm hoping you can
> > > > > > help.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Matt
> > > > > > >
> > > > > > > // SIGNED //
> > > > > > > Matthew C. Sittel
> > > > > > > Associate Scientist
> > > > > > > University Corporation for Atmospheric Research
16WS/WXN,
> > > > > > > Offutt AFB, NE
> > > > > > > (402) 294-3473 DSN: 271-3473
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > National Center for Atmospheric Research Research
Applications
> > > > > > Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Thu Oct 02 14:39:40 2014
Well I just got point_stat to run by just claiming it was "U-GWD" and
not "AODVIS". So I think I'm good-for now at least.
Thanks John... I'll let you know if I need more help in getting this
to run.
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, October 02, 2014 3:31 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Matt,
You don't need to recompile since this table is read in at runtime.
Did you already send us a sample file with this data in it? Might be
easiest if I could just get my hands on the file and telly you exactly
what needs to be done.
Thanks,
John
On Thu, Oct 2, 2014 at 2:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi John,
>
> So when nceptab_flat.txt is read, which field(s) must match what is
> inside my GriB file to allow processing to continue? In the version
I
> have for MET 4.1 I have:
>
> 147 2 "U-GWD" "Zonal gravity wave stress" "N/m^2"
>
> I didn't know there was a hyphen but maybe that's part of my
problem,
> since it isn't an exact match. Or do I need to change not only the
> variable name, but the descriptor and units to where everything
agrees
> with my GriB header?
>
> Do I need to recompile MET if I change this file, or is it only read
> at runtime?
>
> I would be happy to test the KPDS number feature... we don't use
> record number since there is no guarantee it will be a constant from
> file to file, so using kpds numbers allows us to match that.
> Sometimes, though, those can be still ambiguous and we have to look
at
> P1 and P2 fields also. So there might need to be some more elegant
> method to list which GriB parameters should be checked to guarantee
> uniqueness in the number of records returned (i.e., no more than one
field).
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, October 02, 2014 3:10 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Matt,
>
> We have a MET development issue defined in JIRA to track your
> suggestion about supporting KPDS selection of GRIB1 records directly
> and will consider it for the next release. If/when we have funding
to
> work on that, we may contact you to be our guinea pig!
>
> To answer your question about GRIB1 tables... MET reads that data
from
> a file named "nceptab_flat.txt". Look in the directory where you've
> installed MET for the file named
"share/met/table_files/nceptab_flat.txt".
> If your usage of GRIB1 records differs from NCEP's usage, you can
edit
> this file to specify what abbreviations you'd like to use.
>
> I do not see AODVIS in that file. It looks like GRIB record 147 of
> parameter table version 2 is set to UGWD for "Zonal flux of gravity
> wave
> stress":
> 147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
>
> You could change that to your local usage of the GRIB record. But
> first, you should check how they've encoded the GRIB parameter table
> version number for that record. Run wgrib with the "-PDS10" option
> and you'll see output looking something like this:
>
> PDS10(1..28)=
> 1: 0 0 28 2 59 89 212 192 11 100 3 207 5 8
7 0
> 0 1 9 0
> 21: 0 0 0 0 21 0 0 0
>
> The 4th number listed will tell you the parameter table version
number
> for that record. In this case, the ptv is 2. Whichever it is, find
> and edit the line in that "nceptab_flat.txt" file that begins with
> "147 ###" where ### is the ptv number.
>
> Hope that helps!
>
> Thanks,
> John
>
> On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi Julie,
> >
> > Thanks for your response. I hope a future release of MET will
allow us
> > simply to state kpds numbers when stating which field to verify.
> > I've mentioned it a couple times to Tressa and Tara as something
> > that would help us out. I realize it won't happen right away but
I
> > hope it made the list of future upgrades!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, October 01, 2014 3:09 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt. Good questions. Unfortunately, I am not familiar enough
> > with the software to be able to answer your questions. John is
out
> > of the office currently, so I can't ask him for guidance right
now.
> > Hopefully, he will be able to help out later this week. I'm sorry
I
> > can't be of more help to you at this time.
> >
> > Julie
> >
> > On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Hi Julie,
> > >
> > > So we've made some progress on the problems. The modelers are
going to
> > > produce the data in GriB format. Here's the GriB header for
the one
> > field
> > > in the GriB file I need to verify:
> > >
> > > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
> > > kpds7=0
> > > levels=(0,0) grid=255 atmos col anl:
> > > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
> > > num_in_ave
> > > 0 missing 0
> > > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
winds(grid)
> > > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> > > long 0.000000 to -0.500000 by 1.003000, (359 x 180)
scan
> > > 0 mode
> > > 136 bdsgrid 1
> > > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
> > > DecScale
> > > 6 BinScale 0
> > >
> > > How do I set the identifiers for the forecast field within the
> > > point_stat config file to get MET to read this? Right now I
have:
> > >
> > > fcst = {
> > > message_type = [ "ADPSFC" ];
> > > field = [
> > > {
> > > name = "AODVIS";
> > > level = "L0";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "NCIP";
> > > level = "L0";
> > > cat_thresh = [ >=0.0 ];
> > > }
> > > ];
> > > };
> > >
> > > But this isn't working. The error I get is:
> > >
> > > DEBUG 1: User Config File:
> > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
> > > new Met2dDataFile object of type "FileType_Gb1".
> > > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > > object of type "FileType_Gb1".
> > > ERROR :
> > > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > > abbreviation 'AODVIS' for table version 2 ERROR :
> > >
> > > This is the one thing in MET that confuses me more than anything
> > > else-identifying the data fields in the config file.
> > >
> > > How do I know the variable and level names? What field(s) in
the
> > > GriB header do these correspond to in MET settings? Does it
> > > matter that "AODVIS" isn't the standard variable for kpds5=147?
> > > The modelers don't always follow the naming; they'll put any
> > > output field in any kpds number they want.
> > >
> > > Can you help me get the variables set correctly?
> > >
> > > Thanks,
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Tuesday, September 23, 2014 3:22 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. Thanks for getting the data to us.
> > >
> > > The forecast file that you sent was a raw wrfout file, which MET
> > > doesn't support. The raw wrfout files have to be post-processed
> > > using p_interp or UPP. UPP tends to work better because it
writes
> > > GRIB files that MET supports well. For p_interp output, we
can't
> > > read variables that are defined on the staggered dimensions,
like wind.
> > >
> > > We used p_interp (available here:
> > >
> > >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_
> > > pr
> > > oc
> > > essing
> > > )
> > > to convert your file into a NetCDF format that MET can read.
> > > p_interp also interpolates to pressure levels, and many of the
> > > grid points for the first few pressure levels are actually below
ground.
> > > So you'll notice a lot of missing values in the first few
pressure
> > > levels of the p_interp output. We also made some changes to
your
> > > configuration file. I have attached the new
> > > configuration file for you to use. Here is a brief description
of the
> > > changes:
> > >
> > > -Moved message_type = [ "ADPSFC" ]; to be within the obs
> > > dictionary -Modified
> > > name = "PM2_5_DRY(0,0,*,*)"; to
> > > name = "PM2_5_DRY";
> > > level = "(0,0,*,*)";
> > > specifying the level separately
> > > -Modified
> > > name = "obs_arr(*,4)";
> > > to
> > > name = "NCIP";
> > > level = "L0";
> > > because the obs information needs to be specified as if it were
a
> > > GRIB file. NCIP came from grib code 198 (specified in obs_arr)
> > > - Changed sid to be [] instead of ""
> > >
> > > With these changes, we were able to successfully run point_stat.
> > > However, please note that the forecast values for PM2_5_DRY are
> > > all zero! You could take a look at your WRF-Chem run to figure
out why.
> > > I'd suggest using ncview to display the PM2_5_DRY variable in
the
> > > NetCDF
> > file you sent us.
> > > You'll see that it's zero for all model levels. So you may want
> > > to look into that.
> > >
> > > I hope this helps.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Thanks-I got it to work on my end so you don't have to monitor
it.
> > > Sorry
> > > > about the directory mixup-I created the directory and thought
I
> > > > was putting the files there!
> > > >
> > > > 58 MB right now thus far I see... let's hope it works!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 1:18 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > I understand your confusion. The file was being written to
> > > > incoming/irap/met_help/ as opposed to
> > > > incoming/irap/met_help/sittel_data/
> > > > where it is being written to now. I will get back to you this
> > > > afternoon and let you know how far along it is.
> > > >
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF
> > > > AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > > I opened a separate window and did a list on the directory
but
> > > > > it
> > said
> > > > > nothing was in there... which left me even more confused.
I have
> > > > > restarted the FTP but looks like I have no way on my end to
> > > > > monitor progress... so I'll leave it running all night if I
> > > > > must and hope it continues running.
> > > > >
> > > > > If you get a chance later this afternoon, please let me know
> > > > > how far along the file is so I can estimate transfer speed
for
> > > > > future
> > > reference.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > issue
> > > > >
> > > > > Ah, that's too bad. I had been watching it slowly come in.
> > > > > I'll keep an eye out for it.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
> > > > > AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > > Looks like I need to be more patient. I assumed when I
got
> > > > > > back to work that the file transfer was hung so I aborted
it.
> > > > > > Turns out it has transferred 430 MB of the full 1.3 GB of
> > > > > > the model data
> > > file.
> > > > > > So looks like transfer will take 6-8 hours at this rate.
> > > > > >
> > > > > > With any luck it will run to completion... sorry for the
> > > > > > delay, but I don't know any better way to get you the file
> > > > > > short of mailing
> > > a DVD!
> > > > > >
> > > > > > Matt
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > > issue
> > > > > >
> > > > > > Hi Matt. I see that you are having problems running
point_stat.
> > > > > > Thank you for including the details of the situation.
Could
> > > > > > you please follow these instructions for posting the
> > > > > > forecast file, observation file, and config file to our
ftp site:
> > > > > >
> > > > > > http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > > >
> > > > > > I can take a closer look and try to resolve the problem.
> > > > > >
> > > > > > Thanks,
> > > > > > Julie
> > > > > >
> > > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR
USAF
> > > > > > AFWA
> > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > > > Transaction: Ticket created by
matthew.sittel.ctr at us.af.mil
> > > > > > > Queue: met_help
> > > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > > Owner: Nobody
> > > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hello again,
> > > > > > >
> > > > > > > I'm trying to find a problem with my inability to run
> > > > > > > point_stat for a pair of NetCDF files. Here's what I am
> > > > > > > seeing
> > from MET:
> > > > > > >
> > > > > > > DEBUG 1: Default Config File:
> > > > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > > DEBUG 1: User Config File:
> > > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
> > > > > > > created new Met2dDataFile object of type
"FileType_NcMet".
> > > > > > > NetCDF: Attribute not found
> > > > > > >
> > > > > > > Here's what I defined for the fields in the point_stat
> > > > > > > config
> > file:
> > > > > > >
> > > > > > > fcst = {
> > > > > > > message_type = [ "ADPSFC" ];
> > > > > > > field = [
> > > > > > > {
> > > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > }
> > > > > > > ];
> > > > > > > };
> > > > > > >
> > > > > > > obs = {
> > > > > > > field = [
> > > > > > > {
> > > > > > > name = "obs_arr(*,4)";
> > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > }
> > > > > > > ];
> > > > > > > };
> > > > > > >
> > > > > > > Within the forecast NetCDF file the following values are
set:
> > > > > > >
> > > > > > > dimensions:
> > > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > > west_east = 249 ;
> > > > > > > south_north = 149 ;
> > > > > > > bottom_top = 56 ;
> > > > > > >
> > > > > > > float PM2_5_DRY(Time, bottom_top, south_north,
> > west_east) ;
> > > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > > PM2_5_DRY:description = "pm2.5 aerosol
dry
> mass"
> > ;
> > > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > > > >
> > > > > > > Within the observation file:
> > > > > > >
> > > > > > > dimensions:
> > > > > > > obs_arr_len = 5 ;
> > > > > > > nhdr = 1820 ;
> > > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > > >
> > > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > > obs_arr:long_name = "array of
observation
> > values" ;
> > > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > > obs_arr:columns = "hdr_id gc lvl hgt ob"
;
> > > > > > > obs_arr:hdr_id_long_name = "index of
> > > > > > > matching header
> > > > > > data"
> > > > > > > ;
> > > > > > > obs_arr:gc_long_name = "grib code
> > > > > > > corresponding to the observation type" ;
> > > > > > > obs_arr:lvl_long_name = "pressure level
> > > > > > > (hPa) or accumulation interval (sec)" ;
> > > > > > > obs_arr:hgt_long_name = "height in
meters
> > > > > > > above sea level or ground level (msl or agl)" ;
> > > > > > > obs_arr:ob_long_name = "observation
value"
> > > > > > > ;
> > > > > > >
> > > > > > > Additional information:
> > > > > > >
> > > > > > > "ADPSFC" is the message type within the header of the
> > > > > > > observation files, but it does not appear in the
forecast
> > > > > > > file
> > > anywhere.
> > > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation
assuming
> > > > > > > 0 represents the first member of an array and not 1.
The
> > > > > > > second
> > > > > > > 0 indicates the surface data should be used, the first
of
> > > > > > > 56
> > levels.
> > > > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > > > I am using obs_arr(*,4) because there are 5 data fields,
> > > > > > > and only the 5th one (numbered 4 when starting from 0)
is
> > > > > > > the actual data field of interest. The first *
indicates
> > > > > > > to use all 1820
> > > > observations.
> > > > > > > I built the observation file using 'ascii2nc'; the
> > > > > > > forecast file was built separately by a colleague.
> > > > > > >
> > > > > > > The observation file is ~350KB so I could easily e-mail
> > > > > > > it, but the model data is ~1.3GB, so obviously I can't
> > > > > > > easily send that along for you to test with. Perhaps I
> > > > > > > could FTP it somewhere you could try to replicate the
error?
> > > > > > >
> > > > > > > Do you see anything here that I've made a mistake with?
> > > > > > > I've spent a couple hours staring at code and reading
the
> > > > > > > user's manual; I can't find any guidance as to where I
> > > > > > > made
> the mistake.
> > > > > > > I'm hoping you can
> > > > > > help.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Matt
> > > > > > >
> > > > > > > // SIGNED //
> > > > > > > Matthew C. Sittel
> > > > > > > Associate Scientist
> > > > > > > University Corporation for Atmospheric Research
16WS/WXN,
> > > > > > > Offutt AFB, NE
> > > > > > > (402) 294-3473 DSN: 271-3473
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > National Center for Atmospheric Research Research
> > > > > > Applications Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Thu Oct 02 14:42:46 2014
Matt,
Sure, happy to help. If you'd prefer to edit that table file to
replace
U-GWD with AODVIS, feel free. That'll just change the strings used in
the
FCST_VAR and/or OBS_VAR output columns of point-stat. However, save
off a
copy of that because the MET code will continue to include NCEP's
usage of
the GRIB tables.
Thanks,
John
On Thu, Oct 2, 2014 at 2:39 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Well I just got point_stat to run by just claiming it was "U-GWD"
and not
> "AODVIS". So I think I'm good-for now at least.
>
> Thanks John... I'll let you know if I need more help in getting this
to
> run.
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, October 02, 2014 3:31 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Matt,
>
> You don't need to recompile since this table is read in at runtime.
Did
> you already send us a sample file with this data in it? Might be
easiest
> if I could just get my hands on the file and telly you exactly what
needs
> to be done.
>
> Thanks,
> John
>
> On Thu, Oct 2, 2014 at 2:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi John,
> >
> > So when nceptab_flat.txt is read, which field(s) must match what
is
> > inside my GriB file to allow processing to continue? In the
version I
> > have for MET 4.1 I have:
> >
> > 147 2 "U-GWD" "Zonal gravity wave stress" "N/m^2"
> >
> > I didn't know there was a hyphen but maybe that's part of my
problem,
> > since it isn't an exact match. Or do I need to change not only
the
> > variable name, but the descriptor and units to where everything
agrees
> > with my GriB header?
> >
> > Do I need to recompile MET if I change this file, or is it only
read
> > at runtime?
> >
> > I would be happy to test the KPDS number feature... we don't use
> > record number since there is no guarantee it will be a constant
from
> > file to file, so using kpds numbers allows us to match that.
> > Sometimes, though, those can be still ambiguous and we have to
look at
> > P1 and P2 fields also. So there might need to be some more
elegant
> > method to list which GriB parameters should be checked to
guarantee
> > uniqueness in the number of records returned (i.e., no more than
one
> field).
> >
> > Matt
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, October 02, 2014 3:10 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Matt,
> >
> > We have a MET development issue defined in JIRA to track your
> > suggestion about supporting KPDS selection of GRIB1 records
directly
> > and will consider it for the next release. If/when we have
funding to
> > work on that, we may contact you to be our guinea pig!
> >
> > To answer your question about GRIB1 tables... MET reads that data
from
> > a file named "nceptab_flat.txt". Look in the directory where
you've
> > installed MET for the file named
> "share/met/table_files/nceptab_flat.txt".
> > If your usage of GRIB1 records differs from NCEP's usage, you can
edit
> > this file to specify what abbreviations you'd like to use.
> >
> > I do not see AODVIS in that file. It looks like GRIB record 147
of
> > parameter table version 2 is set to UGWD for "Zonal flux of
gravity
> > wave
> > stress":
> > 147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
> >
> > You could change that to your local usage of the GRIB record. But
> > first, you should check how they've encoded the GRIB parameter
table
> > version number for that record. Run wgrib with the "-PDS10"
option
> > and you'll see output looking something like this:
> >
> > PDS10(1..28)=
> > 1: 0 0 28 2 59 89 212 192 11 100 3 207 5 8
7 0
> > 0 1 9 0
> > 21: 0 0 0 0 21 0 0 0
> >
> > The 4th number listed will tell you the parameter table version
number
> > for that record. In this case, the ptv is 2. Whichever it is,
find
> > and edit the line in that "nceptab_flat.txt" file that begins with
> > "147 ###" where ### is the ptv number.
> >
> > Hope that helps!
> >
> > Thanks,
> > John
> >
> > On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Hi Julie,
> > >
> > > Thanks for your response. I hope a future release of MET will
allow
> us
> > > simply to state kpds numbers when stating which field to verify.
> > > I've mentioned it a couple times to Tressa and Tara as something
> > > that would help us out. I realize it won't happen right away
but I
> > > hope it made the list of future upgrades!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, October 01, 2014 3:09 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. Good questions. Unfortunately, I am not familiar
enough
> > > with the software to be able to answer your questions. John is
out
> > > of the office currently, so I can't ask him for guidance right
now.
> > > Hopefully, he will be able to help out later this week. I'm
sorry I
> > > can't be of more help to you at this time.
> > >
> > > Julie
> > >
> > > On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Hi Julie,
> > > >
> > > > So we've made some progress on the problems. The modelers are
going
> to
> > > > produce the data in GriB format. Here's the GriB header for
the one
> > > field
> > > > in the GriB file I need to verify:
> > > >
> > > > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
> > > > kpds7=0
> > > > levels=(0,0) grid=255 atmos col anl:
> > > > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > > > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
> > > > num_in_ave
> > > > 0 missing 0
> > > > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
winds(grid)
> > > > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> > > > long 0.000000 to -0.500000 by 1.003000, (359 x 180)
scan
> > > > 0 mode
> > > > 136 bdsgrid 1
> > > > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
> > > > DecScale
> > > > 6 BinScale 0
> > > >
> > > > How do I set the identifiers for the forecast field within the
> > > > point_stat config file to get MET to read this? Right now I
have:
> > > >
> > > > fcst = {
> > > > message_type = [ "ADPSFC" ];
> > > > field = [
> > > > {
> > > > name = "AODVIS";
> > > > level = "L0";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "NCIP";
> > > > level = "L0";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > But this isn't working. The error I get is:
> > > >
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_Gb1".
> > > > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > > > object of type "FileType_Gb1".
> > > > ERROR :
> > > > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > > > abbreviation 'AODVIS' for table version 2 ERROR :
> > > >
> > > > This is the one thing in MET that confuses me more than
anything
> > > > else-identifying the data fields in the config file.
> > > >
> > > > How do I know the variable and level names? What field(s) in
the
> > > > GriB header do these correspond to in MET settings? Does it
> > > > matter that "AODVIS" isn't the standard variable for
kpds5=147?
> > > > The modelers don't always follow the naming; they'll put any
> > > > output field in any kpds number they want.
> > > >
> > > > Can you help me get the variables set correctly?
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Tuesday, September 23, 2014 3:22 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt. Thanks for getting the data to us.
> > > >
> > > > The forecast file that you sent was a raw wrfout file, which
MET
> > > > doesn't support. The raw wrfout files have to be post-
processed
> > > > using p_interp or UPP. UPP tends to work better because it
writes
> > > > GRIB files that MET supports well. For p_interp output, we
can't
> > > > read variables that are defined on the staggered dimensions,
like
> wind.
> > > >
> > > > We used p_interp (available here:
> > > >
> > > >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_
> > > > pr
> > > > oc
> > > > essing
> > > > )
> > > > to convert your file into a NetCDF format that MET can read.
> > > > p_interp also interpolates to pressure levels, and many of the
> > > > grid points for the first few pressure levels are actually
below
> ground.
> > > > So you'll notice a lot of missing values in the first few
pressure
> > > > levels of the p_interp output. We also made some changes to
your
> > > > configuration file. I have attached the new
> > > > configuration file for you to use. Here is a brief
description of
> the
> > > > changes:
> > > >
> > > > -Moved message_type = [ "ADPSFC" ]; to be within the obs
> > > > dictionary -Modified
> > > > name = "PM2_5_DRY(0,0,*,*)"; to
> > > > name = "PM2_5_DRY";
> > > > level = "(0,0,*,*)";
> > > > specifying the level separately
> > > > -Modified
> > > > name = "obs_arr(*,4)";
> > > > to
> > > > name = "NCIP";
> > > > level = "L0";
> > > > because the obs information needs to be specified as if it
were a
> > > > GRIB file. NCIP came from grib code 198 (specified in
obs_arr)
> > > > - Changed sid to be [] instead of ""
> > > >
> > > > With these changes, we were able to successfully run
point_stat.
> > > > However, please note that the forecast values for PM2_5_DRY
are
> > > > all zero! You could take a look at your WRF-Chem run to
figure out
> why.
> > > > I'd suggest using ncview to display the PM2_5_DRY variable in
the
> > > > NetCDF
> > > file you sent us.
> > > > You'll see that it's zero for all model levels. So you may
want
> > > > to look into that.
> > > >
> > > > I hope this helps.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > > > >
> > > > > Thanks-I got it to work on my end so you don't have to
monitor it.
> > > > Sorry
> > > > > about the directory mixup-I created the directory and
thought I
> > > > > was putting the files there!
> > > > >
> > > > > 58 MB right now thus far I see... let's hope it works!
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Monday, September 22, 2014 1:18 PM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > > >
> > > > > I understand your confusion. The file was being written to
> > > > > incoming/irap/met_help/ as opposed to
> > > > > incoming/irap/met_help/sittel_data/
> > > > > where it is being written to now. I will get back to you
this
> > > > > afternoon and let you know how far along it is.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF
> > > > > AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > > I opened a separate window and did a list on the directory
but
> > > > > > it
> > > said
> > > > > > nothing was in there... which left me even more confused.
I
> have
> > > > > > restarted the FTP but looks like I have no way on my end
to
> > > > > > monitor progress... so I'll leave it running all night if
I
> > > > > > must and hope it continues running.
> > > > > >
> > > > > > If you get a chance later this afternoon, please let me
know
> > > > > > how far along the file is so I can estimate transfer speed
for
> > > > > > future
> > > > reference.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Matt
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > > issue
> > > > > >
> > > > > > Ah, that's too bad. I had been watching it slowly come
in.
> > > > > > I'll keep an eye out for it.
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR
USAF
> > > > > > AFWA
> > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > >
> > > > > > >
> > > > > > > Looks like I need to be more patient. I assumed when I
got
> > > > > > > back to work that the file transfer was hung so I
aborted it.
> > > > > > > Turns out it has transferred 430 MB of the full 1.3 GB
of
> > > > > > > the model data
> > > > file.
> > > > > > > So looks like transfer will take 6-8 hours at this rate.
> > > > > > >
> > > > > > > With any luck it will run to completion... sorry for the
> > > > > > > delay, but I don't know any better way to get you the
file
> > > > > > > short of mailing
> > > > a DVD!
> > > > > > >
> > > > > > > Matt
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1:
Point_stat
> > > > > > > issue
> > > > > > >
> > > > > > > Hi Matt. I see that you are having problems running
> point_stat.
> > > > > > > Thank you for including the details of the situation.
Could
> > > > > > > you please follow these instructions for posting the
> > > > > > > forecast file, observation file, and config file to our
ftp
> site:
> > > > > > >
> > > > > > >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > > > >
> > > > > > > I can take a closer look and try to resolve the problem.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR
USAF
> > > > > > > AFWA
> > > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted
upon.
> > > > > > > > Transaction: Ticket created by
matthew.sittel.ctr at us.af.mil
> > > > > > > > Queue: met_help
> > > > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hello again,
> > > > > > > >
> > > > > > > > I'm trying to find a problem with my inability to run
> > > > > > > > point_stat for a pair of NetCDF files. Here's what I
am
> > > > > > > > seeing
> > > from MET:
> > > > > > > >
> > > > > > > > DEBUG 1: Default Config File:
> > > > > > > >
/home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > > > DEBUG 1: User Config File:
> > > > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file()
->
> > > > > > > > created new Met2dDataFile object of type
"FileType_NcMet".
> > > > > > > > NetCDF: Attribute not found
> > > > > > > >
> > > > > > > > Here's what I defined for the fields in the point_stat
> > > > > > > > config
> > > file:
> > > > > > > >
> > > > > > > > fcst = {
> > > > > > > > message_type = [ "ADPSFC" ];
> > > > > > > > field = [
> > > > > > > > {
> > > > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > > }
> > > > > > > > ];
> > > > > > > > };
> > > > > > > >
> > > > > > > > obs = {
> > > > > > > > field = [
> > > > > > > > {
> > > > > > > > name = "obs_arr(*,4)";
> > > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > > }
> > > > > > > > ];
> > > > > > > > };
> > > > > > > >
> > > > > > > > Within the forecast NetCDF file the following values
are set:
> > > > > > > >
> > > > > > > > dimensions:
> > > > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > > > west_east = 249 ;
> > > > > > > > south_north = 149 ;
> > > > > > > > bottom_top = 56 ;
> > > > > > > >
> > > > > > > > float PM2_5_DRY(Time, bottom_top, south_north,
> > > west_east) ;
> > > > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > > > PM2_5_DRY:description = "pm2.5 aerosol
dry
> > mass"
> > > ;
> > > > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > > > > >
> > > > > > > > Within the observation file:
> > > > > > > >
> > > > > > > > dimensions:
> > > > > > > > obs_arr_len = 5 ;
> > > > > > > > nhdr = 1820 ;
> > > > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > > > >
> > > > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > > > obs_arr:long_name = "array of
observation
> > > values" ;
> > > > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > > > obs_arr:columns = "hdr_id gc lvl hgt
ob" ;
> > > > > > > > obs_arr:hdr_id_long_name = "index of
> > > > > > > > matching header
> > > > > > > data"
> > > > > > > > ;
> > > > > > > > obs_arr:gc_long_name = "grib code
> > > > > > > > corresponding to the observation type" ;
> > > > > > > > obs_arr:lvl_long_name = "pressure
level
> > > > > > > > (hPa) or accumulation interval (sec)" ;
> > > > > > > > obs_arr:hgt_long_name = "height in
meters
> > > > > > > > above sea level or ground level (msl or agl)" ;
> > > > > > > > obs_arr:ob_long_name = "observation
value"
> > > > > > > > ;
> > > > > > > >
> > > > > > > > Additional information:
> > > > > > > >
> > > > > > > > "ADPSFC" is the message type within the header of the
> > > > > > > > observation files, but it does not appear in the
forecast
> > > > > > > > file
> > > > anywhere.
> > > > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation
assuming
> > > > > > > > 0 represents the first member of an array and not 1.
The
> > > > > > > > second
> > > > > > > > 0 indicates the surface data should be used, the first
of
> > > > > > > > 56
> > > levels.
> > > > > > > > I used "*" to represent the X and Y nature of the
grid.
> > > > > > > > I am using obs_arr(*,4) because there are 5 data
fields,
> > > > > > > > and only the 5th one (numbered 4 when starting from 0)
is
> > > > > > > > the actual data field of interest. The first *
indicates
> > > > > > > > to use all 1820
> > > > > observations.
> > > > > > > > I built the observation file using 'ascii2nc'; the
> > > > > > > > forecast file was built separately by a colleague.
> > > > > > > >
> > > > > > > > The observation file is ~350KB so I could easily e-
mail
> > > > > > > > it, but the model data is ~1.3GB, so obviously I can't
> > > > > > > > easily send that along for you to test with. Perhaps
I
> > > > > > > > could FTP it somewhere you could try to replicate the
error?
> > > > > > > >
> > > > > > > > Do you see anything here that I've made a mistake
with?
> > > > > > > > I've spent a couple hours staring at code and reading
the
> > > > > > > > user's manual; I can't find any guidance as to where I
> > > > > > > > made
> > the mistake.
> > > > > > > > I'm hoping you can
> > > > > > > help.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > // SIGNED //
> > > > > > > > Matthew C. Sittel
> > > > > > > > Associate Scientist
> > > > > > > > University Corporation for Atmospheric Research
16WS/WXN,
> > > > > > > > Offutt AFB, NE
> > > > > > > > (402) 294-3473 DSN: 271-3473
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik
> > > > > > > National Center for Atmospheric Research Research
> > > > > > > Applications Laboratory
> > > > > > > Phone: 303.497.8399
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > National Center for Atmospheric Research Research
Applications
> > > > > > Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Thu Oct 02 14:46:12 2014
Yeah, the problem there is what one model uses for kpds5=147 may be
different for another model, so changing it could break something else
if it's also a GriB field we verify somewhere else. So I'll leave it
alone. But it works at least!
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, October 02, 2014 3:43 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
Matt,
Sure, happy to help. If you'd prefer to edit that table file to
replace U-GWD with AODVIS, feel free. That'll just change the strings
used in the FCST_VAR and/or OBS_VAR output columns of point-stat.
However, save off a copy of that because the MET code will continue to
include NCEP's usage of the GRIB tables.
Thanks,
John
On Thu, Oct 2, 2014 at 2:39 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Well I just got point_stat to run by just claiming it was "U-GWD"
and
> not "AODVIS". So I think I'm good-for now at least.
>
> Thanks John... I'll let you know if I need more help in getting this
> to run.
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, October 02, 2014 3:31 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Matt,
>
> You don't need to recompile since this table is read in at runtime.
> Did you already send us a sample file with this data in it? Might
be
> easiest if I could just get my hands on the file and telly you
exactly
> what needs to be done.
>
> Thanks,
> John
>
> On Thu, Oct 2, 2014 at 2:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Hi John,
> >
> > So when nceptab_flat.txt is read, which field(s) must match what
is
> > inside my GriB file to allow processing to continue? In the
version
> > I have for MET 4.1 I have:
> >
> > 147 2 "U-GWD" "Zonal gravity wave stress" "N/m^2"
> >
> > I didn't know there was a hyphen but maybe that's part of my
> > problem, since it isn't an exact match. Or do I need to change
not
> > only the variable name, but the descriptor and units to where
> > everything agrees with my GriB header?
> >
> > Do I need to recompile MET if I change this file, or is it only
read
> > at runtime?
> >
> > I would be happy to test the KPDS number feature... we don't use
> > record number since there is no guarantee it will be a constant
from
> > file to file, so using kpds numbers allows us to match that.
> > Sometimes, though, those can be still ambiguous and we have to
look
> > at
> > P1 and P2 fields also. So there might need to be some more
elegant
> > method to list which GriB parameters should be checked to
guarantee
> > uniqueness in the number of records returned (i.e., no more than
one
> field).
> >
> > Matt
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, October 02, 2014 3:10 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Matt,
> >
> > We have a MET development issue defined in JIRA to track your
> > suggestion about supporting KPDS selection of GRIB1 records
directly
> > and will consider it for the next release. If/when we have
funding
> > to work on that, we may contact you to be our guinea pig!
> >
> > To answer your question about GRIB1 tables... MET reads that data
> > from a file named "nceptab_flat.txt". Look in the directory where
> > you've installed MET for the file named
> "share/met/table_files/nceptab_flat.txt".
> > If your usage of GRIB1 records differs from NCEP's usage, you can
> > edit this file to specify what abbreviations you'd like to use.
> >
> > I do not see AODVIS in that file. It looks like GRIB record 147
of
> > parameter table version 2 is set to UGWD for "Zonal flux of
gravity
> > wave
> > stress":
> > 147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
> >
> > You could change that to your local usage of the GRIB record. But
> > first, you should check how they've encoded the GRIB parameter
table
> > version number for that record. Run wgrib with the "-PDS10"
option
> > and you'll see output looking something like this:
> >
> > PDS10(1..28)=
> > 1: 0 0 28 2 59 89 212 192 11 100 3 207 5 8
7 0
> > 0 1 9 0
> > 21: 0 0 0 0 21 0 0 0
> >
> > The 4th number listed will tell you the parameter table version
> > number for that record. In this case, the ptv is 2. Whichever it
> > is, find and edit the line in that "nceptab_flat.txt" file that
> > begins with
> > "147 ###" where ### is the ptv number.
> >
> > Hope that helps!
> >
> > Thanks,
> > John
> >
> > On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Hi Julie,
> > >
> > > Thanks for your response. I hope a future release of MET will
allow
> us
> > > simply to state kpds numbers when stating which field to verify.
> > > I've mentioned it a couple times to Tressa and Tara as something
> > > that would help us out. I realize it won't happen right away
but
> > > I hope it made the list of future upgrades!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, October 01, 2014 3:09 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt. Good questions. Unfortunately, I am not familiar
enough
> > > with the software to be able to answer your questions. John is
> > > out of the office currently, so I can't ask him for guidance
right now.
> > > Hopefully, he will be able to help out later this week. I'm
sorry
> > > I can't be of more help to you at this time.
> > >
> > > Julie
> > >
> > > On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Hi Julie,
> > > >
> > > > So we've made some progress on the problems. The modelers are
> > > > going
> to
> > > > produce the data in GriB format. Here's the GriB header for
the one
> > > field
> > > > in the GriB file I need to verify:
> > > >
> > > > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
> > > > kpds7=0
> > > > levels=(0,0) grid=255 atmos col anl:
> > > > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > > > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
> > > > num_in_ave
> > > > 0 missing 0
> > > > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
winds(grid)
> > > > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny 64620
> > > > long 0.000000 to -0.500000 by 1.003000, (359 x 180)
> > > > scan
> > > > 0 mode
> > > > 136 bdsgrid 1
> > > > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
> > > > DecScale
> > > > 6 BinScale 0
> > > >
> > > > How do I set the identifiers for the forecast field within the
> > > > point_stat config file to get MET to read this? Right now I
have:
> > > >
> > > > fcst = {
> > > > message_type = [ "ADPSFC" ];
> > > > field = [
> > > > {
> > > > name = "AODVIS";
> > > > level = "L0";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "NCIP";
> > > > level = "L0";
> > > > cat_thresh = [ >=0.0 ];
> > > > }
> > > > ];
> > > > };
> > > >
> > > > But this isn't working. The error I get is:
> > > >
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_Gb1".
> > > > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> > > > object of type "FileType_Gb1".
> > > > ERROR :
> > > > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > > > abbreviation 'AODVIS' for table version 2 ERROR :
> > > >
> > > > This is the one thing in MET that confuses me more than
anything
> > > > else-identifying the data fields in the config file.
> > > >
> > > > How do I know the variable and level names? What field(s) in
> > > > the GriB header do these correspond to in MET settings? Does
it
> > > > matter that "AODVIS" isn't the standard variable for
kpds5=147?
> > > > The modelers don't always follow the naming; they'll put any
> > > > output field in any kpds number they want.
> > > >
> > > > Can you help me get the variables set correctly?
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Tuesday, September 23, 2014 3:22 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt. Thanks for getting the data to us.
> > > >
> > > > The forecast file that you sent was a raw wrfout file, which
MET
> > > > doesn't support. The raw wrfout files have to be post-
processed
> > > > using p_interp or UPP. UPP tends to work better because it
> > > > writes GRIB files that MET supports well. For p_interp
output,
> > > > we can't read variables that are defined on the staggered
> > > > dimensions, like
> wind.
> > > >
> > > > We used p_interp (available here:
> > > >
> > > >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#pos
> > > > t_
> > > > pr
> > > > oc
> > > > essing
> > > > )
> > > > to convert your file into a NetCDF format that MET can read.
> > > > p_interp also interpolates to pressure levels, and many of the
> > > > grid points for the first few pressure levels are actually
below
> ground.
> > > > So you'll notice a lot of missing values in the first few
> > > > pressure levels of the p_interp output. We also made some
> > > > changes to your configuration file. I have attached the new
> > > > configuration file for you to use. Here is a brief
description of
> the
> > > > changes:
> > > >
> > > > -Moved message_type = [ "ADPSFC" ]; to be within the obs
> > > > dictionary -Modified
> > > > name = "PM2_5_DRY(0,0,*,*)"; to
> > > > name = "PM2_5_DRY";
> > > > level = "(0,0,*,*)";
> > > > specifying the level separately
> > > > -Modified
> > > > name = "obs_arr(*,4)";
> > > > to
> > > > name = "NCIP";
> > > > level = "L0";
> > > > because the obs information needs to be specified as if it
were
> > > > a GRIB file. NCIP came from grib code 198 (specified in
> > > > obs_arr)
> > > > - Changed sid to be [] instead of ""
> > > >
> > > > With these changes, we were able to successfully run
point_stat.
> > > > However, please note that the forecast values for PM2_5_DRY
are
> > > > all zero! You could take a look at your WRF-Chem run to
figure
> > > > out
> why.
> > > > I'd suggest using ncview to display the PM2_5_DRY variable in
> > > > the NetCDF
> > > file you sent us.
> > > > You'll see that it's zero for all model levels. So you may
want
> > > > to look into that.
> > > >
> > > > I hope this helps.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF
> > > > AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > > Thanks-I got it to work on my end so you don't have to
monitor it.
> > > > Sorry
> > > > > about the directory mixup-I created the directory and
thought
> > > > > I was putting the files there!
> > > > >
> > > > > 58 MB right now thus far I see... let's hope it works!
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Monday, September 22, 2014 1:18 PM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > issue
> > > > >
> > > > > I understand your confusion. The file was being written to
> > > > > incoming/irap/met_help/ as opposed to
> > > > > incoming/irap/met_help/sittel_data/
> > > > > where it is being written to now. I will get back to you
this
> > > > > afternoon and let you know how far along it is.
> > > > >
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF
> > > > > AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > > I opened a separate window and did a list on the directory
> > > > > > but it
> > > said
> > > > > > nothing was in there... which left me even more confused.
I
> have
> > > > > > restarted the FTP but looks like I have no way on my end
to
> > > > > > monitor progress... so I'll leave it running all night if
I
> > > > > > must and hope it continues running.
> > > > > >
> > > > > > If you get a chance later this afternoon, please let me
know
> > > > > > how far along the file is so I can estimate transfer speed
> > > > > > for future
> > > > reference.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Matt
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > > issue
> > > > > >
> > > > > > Ah, that's too bad. I had been watching it slowly come
in.
> > > > > > I'll keep an eye out for it.
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR
USAF
> > > > > > AFWA
> > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > >
> > > > > > >
> > > > > > > Looks like I need to be more patient. I assumed when I
> > > > > > > got back to work that the file transfer was hung so I
aborted it.
> > > > > > > Turns out it has transferred 430 MB of the full 1.3 GB
of
> > > > > > > the model data
> > > > file.
> > > > > > > So looks like transfer will take 6-8 hours at this rate.
> > > > > > >
> > > > > > > With any luck it will run to completion... sorry for the
> > > > > > > delay, but I don't know any better way to get you the
file
> > > > > > > short of mailing
> > > > a DVD!
> > > > > > >
> > > > > > > Matt
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1:
Point_stat
> > > > > > > issue
> > > > > > >
> > > > > > > Hi Matt. I see that you are having problems running
> point_stat.
> > > > > > > Thank you for including the details of the situation.
> > > > > > > Could you please follow these instructions for posting
the
> > > > > > > forecast file, observation file, and config file to our
> > > > > > > ftp
> site:
> > > > > > >
> > > > > > >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > > > >
> > > > > > > I can take a closer look and try to resolve the problem.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Julie
> > > > > > >
> > > > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR
> > > > > > > USAF AFWA
> > > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted
upon.
> > > > > > > > Transaction: Ticket created by
matthew.sittel.ctr at us.af.mil
> > > > > > > > Queue: met_help
> > > > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hello again,
> > > > > > > >
> > > > > > > > I'm trying to find a problem with my inability to run
> > > > > > > > point_stat for a pair of NetCDF files. Here's what I
am
> > > > > > > > seeing
> > > from MET:
> > > > > > > >
> > > > > > > > DEBUG 1: Default Config File:
> > > > > > > >
/home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > > > DEBUG 1: User Config File:
> > > > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file()
->
> > > > > > > > created new Met2dDataFile object of type
"FileType_NcMet".
> > > > > > > > NetCDF: Attribute not found
> > > > > > > >
> > > > > > > > Here's what I defined for the fields in the point_stat
> > > > > > > > config
> > > file:
> > > > > > > >
> > > > > > > > fcst = {
> > > > > > > > message_type = [ "ADPSFC" ];
> > > > > > > > field = [
> > > > > > > > {
> > > > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > > }
> > > > > > > > ];
> > > > > > > > };
> > > > > > > >
> > > > > > > > obs = {
> > > > > > > > field = [
> > > > > > > > {
> > > > > > > > name = "obs_arr(*,4)";
> > > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > > }
> > > > > > > > ];
> > > > > > > > };
> > > > > > > >
> > > > > > > > Within the forecast NetCDF file the following values
are set:
> > > > > > > >
> > > > > > > > dimensions:
> > > > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > > > west_east = 249 ;
> > > > > > > > south_north = 149 ;
> > > > > > > > bottom_top = 56 ;
> > > > > > > >
> > > > > > > > float PM2_5_DRY(Time, bottom_top, south_north,
> > > west_east) ;
> > > > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > > > PM2_5_DRY:description = "pm2.5 aerosol
> > > > > > > > dry
> > mass"
> > > ;
> > > > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > > > PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > > > > >
> > > > > > > > Within the observation file:
> > > > > > > >
> > > > > > > > dimensions:
> > > > > > > > obs_arr_len = 5 ;
> > > > > > > > nhdr = 1820 ;
> > > > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > > > >
> > > > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > > > obs_arr:long_name = "array of
> > > > > > > > observation
> > > values" ;
> > > > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > > > obs_arr:columns = "hdr_id gc lvl hgt
ob" ;
> > > > > > > > obs_arr:hdr_id_long_name = "index of
> > > > > > > > matching header
> > > > > > > data"
> > > > > > > > ;
> > > > > > > > obs_arr:gc_long_name = "grib code
> > > > > > > > corresponding to the observation type" ;
> > > > > > > > obs_arr:lvl_long_name = "pressure
level
> > > > > > > > (hPa) or accumulation interval (sec)" ;
> > > > > > > > obs_arr:hgt_long_name = "height in
> > > > > > > > meters above sea level or ground level (msl or agl)" ;
> > > > > > > > obs_arr:ob_long_name = "observation
value"
> > > > > > > > ;
> > > > > > > >
> > > > > > > > Additional information:
> > > > > > > >
> > > > > > > > "ADPSFC" is the message type within the header of the
> > > > > > > > observation files, but it does not appear in the
> > > > > > > > forecast file
> > > > anywhere.
> > > > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation
> > > > > > > > assuming
> > > > > > > > 0 represents the first member of an array and not 1.
> > > > > > > > The second
> > > > > > > > 0 indicates the surface data should be used, the first
> > > > > > > > of
> > > > > > > > 56
> > > levels.
> > > > > > > > I used "*" to represent the X and Y nature of the
grid.
> > > > > > > > I am using obs_arr(*,4) because there are 5 data
fields,
> > > > > > > > and only the 5th one (numbered 4 when starting from 0)
> > > > > > > > is the actual data field of interest. The first *
> > > > > > > > indicates to use all 1820
> > > > > observations.
> > > > > > > > I built the observation file using 'ascii2nc'; the
> > > > > > > > forecast file was built separately by a colleague.
> > > > > > > >
> > > > > > > > The observation file is ~350KB so I could easily e-
mail
> > > > > > > > it, but the model data is ~1.3GB, so obviously I can't
> > > > > > > > easily send that along for you to test with. Perhaps
I
> > > > > > > > could FTP it somewhere you could try to replicate the
error?
> > > > > > > >
> > > > > > > > Do you see anything here that I've made a mistake
with?
> > > > > > > > I've spent a couple hours staring at code and reading
> > > > > > > > the user's manual; I can't find any guidance as to
where
> > > > > > > > I made
> > the mistake.
> > > > > > > > I'm hoping you can
> > > > > > > help.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > // SIGNED //
> > > > > > > > Matthew C. Sittel
> > > > > > > > Associate Scientist
> > > > > > > > University Corporation for Atmospheric Research
> > > > > > > > 16WS/WXN, Offutt AFB, NE
> > > > > > > > (402) 294-3473 DSN: 271-3473
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik
> > > > > > > National Center for Atmospheric Research Research
> > > > > > > Applications Laboratory
> > > > > > > Phone: 303.497.8399
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > National Center for Atmospheric Research Research
> > > > > > Applications Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Fri Oct 03 10:09:54 2014
OK, I'll resolve this ticket now. Just let us know what other
questions or
issues arise.
Thanks,
John
On Thu, Oct 2, 2014 at 2:46 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Yeah, the problem there is what one model uses for kpds5=147 may be
> different for another model, so changing it could break something
else if
> it's also a GriB field we verify somewhere else. So I'll leave it
alone.
> But it works at least!
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, October 02, 2014 3:43 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Matt,
>
> Sure, happy to help. If you'd prefer to edit that table file to
replace
> U-GWD with AODVIS, feel free. That'll just change the strings used
in the
> FCST_VAR and/or OBS_VAR output columns of point-stat. However, save
off a
> copy of that because the MET code will continue to include NCEP's
usage of
> the GRIB tables.
>
> Thanks,
> John
>
> On Thu, Oct 2, 2014 at 2:39 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Well I just got point_stat to run by just claiming it was "U-GWD"
and
> > not "AODVIS". So I think I'm good-for now at least.
> >
> > Thanks John... I'll let you know if I need more help in getting
this
> > to run.
> >
> > Matt
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, October 02, 2014 3:31 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Matt,
> >
> > You don't need to recompile since this table is read in at
runtime.
> > Did you already send us a sample file with this data in it? Might
be
> > easiest if I could just get my hands on the file and telly you
exactly
> > what needs to be done.
> >
> > Thanks,
> > John
> >
> > On Thu, Oct 2, 2014 at 2:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Hi John,
> > >
> > > So when nceptab_flat.txt is read, which field(s) must match what
is
> > > inside my GriB file to allow processing to continue? In the
version
> > > I have for MET 4.1 I have:
> > >
> > > 147 2 "U-GWD" "Zonal gravity wave stress" "N/m^2"
> > >
> > > I didn't know there was a hyphen but maybe that's part of my
> > > problem, since it isn't an exact match. Or do I need to change
not
> > > only the variable name, but the descriptor and units to where
> > > everything agrees with my GriB header?
> > >
> > > Do I need to recompile MET if I change this file, or is it only
read
> > > at runtime?
> > >
> > > I would be happy to test the KPDS number feature... we don't use
> > > record number since there is no guarantee it will be a constant
from
> > > file to file, so using kpds numbers allows us to match that.
> > > Sometimes, though, those can be still ambiguous and we have to
look
> > > at
> > > P1 and P2 fields also. So there might need to be some more
elegant
> > > method to list which GriB parameters should be checked to
guarantee
> > > uniqueness in the number of records returned (i.e., no more than
one
> > field).
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Thursday, October 02, 2014 3:10 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Matt,
> > >
> > > We have a MET development issue defined in JIRA to track your
> > > suggestion about supporting KPDS selection of GRIB1 records
directly
> > > and will consider it for the next release. If/when we have
funding
> > > to work on that, we may contact you to be our guinea pig!
> > >
> > > To answer your question about GRIB1 tables... MET reads that
data
> > > from a file named "nceptab_flat.txt". Look in the directory
where
> > > you've installed MET for the file named
> > "share/met/table_files/nceptab_flat.txt".
> > > If your usage of GRIB1 records differs from NCEP's usage, you
can
> > > edit this file to specify what abbreviations you'd like to use.
> > >
> > > I do not see AODVIS in that file. It looks like GRIB record 147
of
> > > parameter table version 2 is set to UGWD for "Zonal flux of
gravity
> > > wave
> > > stress":
> > > 147 2 "UGWD" "Zonal flux of gravity wave stress" "N/m^2"
> > >
> > > You could change that to your local usage of the GRIB record.
But
> > > first, you should check how they've encoded the GRIB parameter
table
> > > version number for that record. Run wgrib with the "-PDS10"
option
> > > and you'll see output looking something like this:
> > >
> > > PDS10(1..28)=
> > > 1: 0 0 28 2 59 89 212 192 11 100 3 207 5
8 7
> 0
> > > 0 1 9 0
> > > 21: 0 0 0 0 21 0 0 0
> > >
> > > The 4th number listed will tell you the parameter table version
> > > number for that record. In this case, the ptv is 2. Whichever
it
> > > is, find and edit the line in that "nceptab_flat.txt" file that
> > > begins with
> > > "147 ###" where ### is the ptv number.
> > >
> > > Hope that helps!
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Oct 1, 2014 at 2:15 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Hi Julie,
> > > >
> > > > Thanks for your response. I hope a future release of MET
will allow
> > us
> > > > simply to state kpds numbers when stating which field to
verify.
> > > > I've mentioned it a couple times to Tressa and Tara as
something
> > > > that would help us out. I realize it won't happen right away
but
> > > > I hope it made the list of future upgrades!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Wednesday, October 01, 2014 3:09 PM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt. Good questions. Unfortunately, I am not familiar
enough
> > > > with the software to be able to answer your questions. John
is
> > > > out of the office currently, so I can't ask him for guidance
right
> now.
> > > > Hopefully, he will be able to help out later this week. I'm
sorry
> > > > I can't be of more help to you at this time.
> > > >
> > > > Julie
> > > >
> > > > On Wed, Oct 1, 2014 at 1:24 PM, SITTEL, MATTHEW C CTR USAF
AFWA 16
> > > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > > > >
> > > > > Hi Julie,
> > > > >
> > > > > So we've made some progress on the problems. The modelers
are
> > > > > going
> > to
> > > > > produce the data in GriB format. Here's the GriB header
for the
> one
> > > > field
> > > > > in the GriB file I need to verify:
> > > > >
> > > > > rec 502:59723108:date 2014091900 AODVIS kpds5=147 kpds6=200
> > > > > kpds7=0
> > > > > levels=(0,0) grid=255 atmos col anl:
> > > > > AODVIS=Aerosol optical depth (0.55 um) [numeric]
> > > > > timerange 0 P1 0 P2 0 TimeU 1 nx 359 ny 180 GDS grid 0
> > > > > num_in_ave
> > > > > 0 missing 0
> > > > > center 57 subcenter 6 process 15 Table 129 scan: WE:NS
> winds(grid)
> > > > > latlon: lat -90.000000 to 90.000000 by 1.003000 nxny
64620
> > > > > long 0.000000 to -0.500000 by 1.003000, (359 x
180)
> > > > > scan
> > > > > 0 mode
> > > > > 136 bdsgrid 1
> > > > > min/max data 0.009239 0.02811 num bits 15 BDS_Ref 9239
> > > > > DecScale
> > > > > 6 BinScale 0
> > > > >
> > > > > How do I set the identifiers for the forecast field within
the
> > > > > point_stat config file to get MET to read this? Right now I
have:
> > > > >
> > > > > fcst = {
> > > > > message_type = [ "ADPSFC" ];
> > > > > field = [
> > > > > {
> > > > > name = "AODVIS";
> > > > > level = "L0";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > obs = {
> > > > > field = [
> > > > > {
> > > > > name = "NCIP";
> > > > > level = "L0";
> > > > > cat_thresh = [ >=0.0 ];
> > > > > }
> > > > > ];
> > > > > };
> > > > >
> > > > > But this isn't working. The error I get is:
> > > > >
> > > > > DEBUG 1: User Config File:
> > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > > new Met2dDataFile object of type "FileType_Gb1".
> > > > > DEBUG 4: VarInfoFactory::new_var_info() -> created new
VarInfo
> > > > > object of type "FileType_Gb1".
> > > > > ERROR :
> > > > > ERROR : VarInfoGrib::set_dict() - unrecognized GRIB1 field
> > > > > abbreviation 'AODVIS' for table version 2 ERROR :
> > > > >
> > > > > This is the one thing in MET that confuses me more than
anything
> > > > > else-identifying the data fields in the config file.
> > > > >
> > > > > How do I know the variable and level names? What field(s)
in
> > > > > the GriB header do these correspond to in MET settings?
Does it
> > > > > matter that "AODVIS" isn't the standard variable for
kpds5=147?
> > > > > The modelers don't always follow the naming; they'll put any
> > > > > output field in any kpds number they want.
> > > > >
> > > > > Can you help me get the variables set correctly?
> > > > >
> > > > > Thanks,
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Tuesday, September 23, 2014 3:22 PM
> > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > > >
> > > > > Hi Matt. Thanks for getting the data to us.
> > > > >
> > > > > The forecast file that you sent was a raw wrfout file, which
MET
> > > > > doesn't support. The raw wrfout files have to be post-
processed
> > > > > using p_interp or UPP. UPP tends to work better because it
> > > > > writes GRIB files that MET supports well. For p_interp
output,
> > > > > we can't read variables that are defined on the staggered
> > > > > dimensions, like
> > wind.
> > > > >
> > > > > We used p_interp (available here:
> > > > >
> > > > >
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#pos
> > > > > t_
> > > > > pr
> > > > > oc
> > > > > essing
> > > > > )
> > > > > to convert your file into a NetCDF format that MET can read.
> > > > > p_interp also interpolates to pressure levels, and many of
the
> > > > > grid points for the first few pressure levels are actually
below
> > ground.
> > > > > So you'll notice a lot of missing values in the first few
> > > > > pressure levels of the p_interp output. We also made some
> > > > > changes to your configuration file. I have attached the new
> > > > > configuration file for you to use. Here is a brief
description of
> > the
> > > > > changes:
> > > > >
> > > > > -Moved message_type = [ "ADPSFC" ]; to be within the obs
> > > > > dictionary -Modified
> > > > > name = "PM2_5_DRY(0,0,*,*)"; to
> > > > > name = "PM2_5_DRY";
> > > > > level = "(0,0,*,*)";
> > > > > specifying the level separately
> > > > > -Modified
> > > > > name = "obs_arr(*,4)";
> > > > > to
> > > > > name = "NCIP";
> > > > > level = "L0";
> > > > > because the obs information needs to be specified as if it
were
> > > > > a GRIB file. NCIP came from grib code 198 (specified in
> > > > > obs_arr)
> > > > > - Changed sid to be [] instead of ""
> > > > >
> > > > > With these changes, we were able to successfully run
point_stat.
> > > > > However, please note that the forecast values for PM2_5_DRY
are
> > > > > all zero! You could take a look at your WRF-Chem run to
figure
> > > > > out
> > why.
> > > > > I'd suggest using ncview to display the PM2_5_DRY variable
in
> > > > > the NetCDF
> > > > file you sent us.
> > > > > You'll see that it's zero for all model levels. So you may
want
> > > > > to look into that.
> > > > >
> > > > > I hope this helps.
> > > > >
> > > > > Thanks,
> > > > > Julie
> > > > >
> > > > > On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF
> > > > > AFWA
> > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > >
> > > > > >
> > > > > > Thanks-I got it to work on my end so you don't have to
monitor
> it.
> > > > > Sorry
> > > > > > about the directory mixup-I created the directory and
thought
> > > > > > I was putting the files there!
> > > > > >
> > > > > > 58 MB right now thus far I see... let's hope it works!
> > > > > >
> > > > > > Matt
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Monday, September 22, 2014 1:18 PM
> > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
> > > > > > issue
> > > > > >
> > > > > > I understand your confusion. The file was being written
to
> > > > > > incoming/irap/met_help/ as opposed to
> > > > > > incoming/irap/met_help/sittel_data/
> > > > > > where it is being written to now. I will get back to you
this
> > > > > > afternoon and let you know how far along it is.
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR
USAF
> > > > > > AFWA
> > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > >
> > > > > > >
> > > > > > > I opened a separate window and did a list on the
directory
> > > > > > > but it
> > > > said
> > > > > > > nothing was in there... which left me even more
confused. I
> > have
> > > > > > > restarted the FTP but looks like I have no way on my end
to
> > > > > > > monitor progress... so I'll leave it running all night
if I
> > > > > > > must and hope it continues running.
> > > > > > >
> > > > > > > If you get a chance later this afternoon, please let me
know
> > > > > > > how far along the file is so I can estimate transfer
speed
> > > > > > > for future
> > > > > reference.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > Matt
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > > Sent: Monday, September 22, 2014 1:08 PM
> > > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1:
Point_stat
> > > > > > > issue
> > > > > > >
> > > > > > > Ah, that's too bad. I had been watching it slowly come
in.
> > > > > > > I'll keep an eye out for it.
> > > > > > >
> > > > > > > Julie
> > > > > > >
> > > > > > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR
USAF
> > > > > > > AFWA
> > > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > > >
> > > > > > > >
> > > > > > > > Looks like I need to be more patient. I assumed when
I
> > > > > > > > got back to work that the file transfer was hung so I
> aborted it.
> > > > > > > > Turns out it has transferred 430 MB of the full 1.3 GB
of
> > > > > > > > the model data
> > > > > file.
> > > > > > > > So looks like transfer will take 6-8 hours at this
rate.
> > > > > > > >
> > > > > > > > With any luck it will run to completion... sorry for
the
> > > > > > > > delay, but I don't know any better way to get you the
file
> > > > > > > > short of mailing
> > > > > a DVD!
> > > > > > > >
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Julie Prestopnik via RT
[mailto:met_help at ucar.edu]
> > > > > > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > > > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > > > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > > > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1:
Point_stat
> > > > > > > > issue
> > > > > > > >
> > > > > > > > Hi Matt. I see that you are having problems running
> > point_stat.
> > > > > > > > Thank you for including the details of the situation.
> > > > > > > > Could you please follow these instructions for posting
the
> > > > > > > > forecast file, observation file, and config file to
our
> > > > > > > > ftp
> > site:
> > > > > > > >
> > > > > > > >
http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > > > > > >
> > > > > > > > I can take a closer look and try to resolve the
problem.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Julie
> > > > > > > >
> > > > > > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR
> > > > > > > > USAF AFWA
> > > > > > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted
upon.
> > > > > > > > > Transaction: Ticket created by
> matthew.sittel.ctr at us.af.mil
> > > > > > > > > Queue: met_help
> > > > > > > > > Subject: MET 4.1: Point_stat issue
> > > > > > > > > Owner: Nobody
> > > > > > > > > Requestors: matthew.sittel.ctr at us.af.mil
> > > > > > > > > Status: new
> > > > > > > > > Ticket <URL:
> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hello again,
> > > > > > > > >
> > > > > > > > > I'm trying to find a problem with my inability to
run
> > > > > > > > > point_stat for a pair of NetCDF files. Here's what
I am
> > > > > > > > > seeing
> > > > from MET:
> > > > > > > > >
> > > > > > > > > DEBUG 1: Default Config File:
> > > > > > > > >
/home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > > > > > DEBUG 1: User Config File:
> > > > > > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > > > > > DEBUG 4:
Met2dDataFileFactory::new_met_2d_data_file() ->
> > > > > > > > > created new Met2dDataFile object of type
"FileType_NcMet".
> > > > > > > > > NetCDF: Attribute not found
> > > > > > > > >
> > > > > > > > > Here's what I defined for the fields in the
point_stat
> > > > > > > > > config
> > > > file:
> > > > > > > > >
> > > > > > > > > fcst = {
> > > > > > > > > message_type = [ "ADPSFC" ];
> > > > > > > > > field = [
> > > > > > > > > {
> > > > > > > > > name = "PM2_5_DRY(0,0,*,*)";
> > > > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > > > }
> > > > > > > > > ];
> > > > > > > > > };
> > > > > > > > >
> > > > > > > > > obs = {
> > > > > > > > > field = [
> > > > > > > > > {
> > > > > > > > > name = "obs_arr(*,4)";
> > > > > > > > > cat_thresh = [ >=0.0 ];
> > > > > > > > > }
> > > > > > > > > ];
> > > > > > > > > };
> > > > > > > > >
> > > > > > > > > Within the forecast NetCDF file the following values
are
> set:
> > > > > > > > >
> > > > > > > > > dimensions:
> > > > > > > > > Time = UNLIMITED ; // (1 currently)
> > > > > > > > > west_east = 249 ;
> > > > > > > > > south_north = 149 ;
> > > > > > > > > bottom_top = 56 ;
> > > > > > > > >
> > > > > > > > > float PM2_5_DRY(Time, bottom_top,
south_north,
> > > > west_east) ;
> > > > > > > > > PM2_5_DRY:FieldType = 104 ;
> > > > > > > > > PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > > > > > > PM2_5_DRY:description = "pm2.5
aerosol
> > > > > > > > > dry
> > > mass"
> > > > ;
> > > > > > > > > PM2_5_DRY:units = "ug m^-3" ;
> > > > > > > > > PM2_5_DRY:stagger = "" ;
> > > > > > > > > PM2_5_DRY:coordinates = "XLONG XLAT"
;
> > > > > > > > >
> > > > > > > > > Within the observation file:
> > > > > > > > >
> > > > > > > > > dimensions:
> > > > > > > > > obs_arr_len = 5 ;
> > > > > > > > > nhdr = 1820 ;
> > > > > > > > > nobs = UNLIMITED ; // (1820 currently)
> > > > > > > > >
> > > > > > > > > float obs_arr(nobs, obs_arr_len) ;
> > > > > > > > > obs_arr:long_name = "array of
> > > > > > > > > observation
> > > > values" ;
> > > > > > > > > obs_arr:_fill_value = -9999.f ;
> > > > > > > > > obs_arr:columns = "hdr_id gc lvl hgt
ob" ;
> > > > > > > > > obs_arr:hdr_id_long_name = "index of
> > > > > > > > > matching header
> > > > > > > > data"
> > > > > > > > > ;
> > > > > > > > > obs_arr:gc_long_name = "grib code
> > > > > > > > > corresponding to the observation type" ;
> > > > > > > > > obs_arr:lvl_long_name = "pressure
level
> > > > > > > > > (hPa) or accumulation interval (sec)" ;
> > > > > > > > > obs_arr:hgt_long_name = "height in
> > > > > > > > > meters above sea level or ground level (msl or agl)"
;
> > > > > > > > > obs_arr:ob_long_name = "observation
value"
> > > > > > > > > ;
> > > > > > > > >
> > > > > > > > > Additional information:
> > > > > > > > >
> > > > > > > > > "ADPSFC" is the message type within the header of
the
> > > > > > > > > observation files, but it does not appear in the
> > > > > > > > > forecast file
> > > > > anywhere.
> > > > > > > > > I am using PM2_5_DRY(0,0,*,*) as the designation
> > > > > > > > > assuming
> > > > > > > > > 0 represents the first member of an array and not 1.
> > > > > > > > > The second
> > > > > > > > > 0 indicates the surface data should be used, the
first
> > > > > > > > > of
> > > > > > > > > 56
> > > > levels.
> > > > > > > > > I used "*" to represent the X and Y nature of the
grid.
> > > > > > > > > I am using obs_arr(*,4) because there are 5 data
fields,
> > > > > > > > > and only the 5th one (numbered 4 when starting from
0)
> > > > > > > > > is the actual data field of interest. The first *
> > > > > > > > > indicates to use all 1820
> > > > > > observations.
> > > > > > > > > I built the observation file using 'ascii2nc'; the
> > > > > > > > > forecast file was built separately by a colleague.
> > > > > > > > >
> > > > > > > > > The observation file is ~350KB so I could easily e-
mail
> > > > > > > > > it, but the model data is ~1.3GB, so obviously I
can't
> > > > > > > > > easily send that along for you to test with.
Perhaps I
> > > > > > > > > could FTP it somewhere you could try to replicate
the
> error?
> > > > > > > > >
> > > > > > > > > Do you see anything here that I've made a mistake
with?
> > > > > > > > > I've spent a couple hours staring at code and
reading
> > > > > > > > > the user's manual; I can't find any guidance as to
where
> > > > > > > > > I made
> > > the mistake.
> > > > > > > > > I'm hoping you can
> > > > > > > > help.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Matt
> > > > > > > > >
> > > > > > > > > // SIGNED //
> > > > > > > > > Matthew C. Sittel
> > > > > > > > > Associate Scientist
> > > > > > > > > University Corporation for Atmospheric Research
> > > > > > > > > 16WS/WXN, Offutt AFB, NE
> > > > > > > > > (402) 294-3473 DSN: 271-3473
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Julie Prestopnik
> > > > > > > > National Center for Atmospheric Research Research
> > > > > > > > Applications Laboratory
> > > > > > > > Phone: 303.497.8399
> > > > > > > > Email: jpresto at ucar.edu
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Julie Prestopnik
> > > > > > > National Center for Atmospheric Research Research
> > > > > > > Applications Laboratory
> > > > > > > Phone: 303.497.8399
> > > > > > > Email: jpresto at ucar.edu
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > National Center for Atmospheric Research Research
Applications
> > > > > > Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > National Center for Atmospheric Research Research
Applications
> > > > > Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list