[Met_help] [rt.rap.ucar.edu #69036] History for MET 4.1: Point_stat issue

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello again,

I'm trying to find a problem with my inability to run point_stat for a pair of NetCDF files.  Here's what I am seeing from MET:

DEBUG 1: Default Config File: /home/qcteam/METv4.1/data/config/PointStatConfig_default
DEBUG 1: User Config File: /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_NcMet".
NetCDF: Attribute not found

Here's what I defined for the fields in the point_stat config file:

fcst = {
   message_type = [ "ADPSFC" ];
   field = [ 
      {
         name = "PM2_5_DRY(0,0,*,*)";
         cat_thresh = [ >=0.0 ];
      }
   ];
};

obs = {
   field = [
      {
         name = "obs_arr(*,4)";
         cat_thresh = [ >=0.0 ];
      }
   ];
};

Within the forecast NetCDF file the following values are set:

dimensions:
        Time = UNLIMITED ; // (1 currently)
        west_east = 249 ;
        south_north = 149 ;
        bottom_top = 56 ;

        float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
                PM2_5_DRY:FieldType = 104 ;
                PM2_5_DRY:MemoryOrder = "XYZ" ;
                PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
                PM2_5_DRY:units = "ug m^-3" ;
                PM2_5_DRY:stagger = "" ;
                PM2_5_DRY:coordinates = "XLONG XLAT" ;

Within the observation file:

dimensions:
        obs_arr_len = 5 ;
        nhdr = 1820 ;
        nobs = UNLIMITED ; // (1820 currently)

        float obs_arr(nobs, obs_arr_len) ;
                obs_arr:long_name = "array of observation values" ;
                obs_arr:_fill_value = -9999.f ;
                obs_arr:columns = "hdr_id gc lvl hgt ob" ;
                obs_arr:hdr_id_long_name = "index of matching header data" ;
                obs_arr:gc_long_name = "grib code corresponding to the observation type" ;
                obs_arr:lvl_long_name = "pressure level (hPa) or accumulation interval (sec)" ;
                obs_arr:hgt_long_name = "height in meters above sea level or ground level (msl or agl)" ;
                obs_arr:ob_long_name = "observation value" ;

Additional information:

"ADPSFC" is the message type within the header of the observation files, but it does not appear in the forecast file anywhere.
I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0 represents the first member of an array and not 1.  The second 0 indicates the surface data should be used, the first of 56 levels.  I used "*" to represent the X and Y nature of the grid.
I am using obs_arr(*,4) because there are 5 data fields, and only the 5th one (numbered 4 when starting from 0) is the actual data field of interest.  The first * indicates to use all 1820 observations.
I built the observation file using 'ascii2nc'; the forecast file was built separately by a colleague.

The observation file is ~350KB so I could easily e-mail it, but the model data is ~1.3GB, so obviously I can't easily send that along for you to test with.  Perhaps I could FTP it somewhere you could try to replicate the error?

Do you see anything here that I've made a mistake with?  I've spent a couple hours staring at code and reading the user's manual; I can't find any guidance as to where I made the mistake.  I'm hoping you can help.

Thanks,
Matt

// SIGNED //
Matthew C. Sittel
Associate Scientist
University Corporation for Atmospheric Research
16WS/WXN, Offutt AFB, NE
(402) 294-3473  DSN: 271-3473




----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Mon Sep 22 09:36:33 2014

Hi Matt.  I see that you are having problems running point_stat.
Thank you
for including the details of the situation.  Could you please follow
these
instructions for posting the forecast file, observation file, and
config
file to our ftp site:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp

I can take a closer look and try to resolve the problem.

Thanks,
Julie

On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:

>
> Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
>        Queue: met_help
>      Subject: MET 4.1: Point_stat issue
>        Owner: Nobody
>   Requestors: matthew.sittel.ctr at us.af.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
>
> Hello again,
>
> I'm trying to find a problem with my inability to run point_stat for
a
> pair of NetCDF files.  Here's what I am seeing from MET:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Here's what I defined for the fields in the point_stat config file:
>
> fcst = {
>    message_type = [ "ADPSFC" ];
>    field = [
>       {
>          name = "PM2_5_DRY(0,0,*,*)";
>          cat_thresh = [ >=0.0 ];
>       }
>    ];
> };
>
> obs = {
>    field = [
>       {
>          name = "obs_arr(*,4)";
>          cat_thresh = [ >=0.0 ];
>       }
>    ];
> };
>
> Within the forecast NetCDF file the following values are set:
>
> dimensions:
>         Time = UNLIMITED ; // (1 currently)
>         west_east = 249 ;
>         south_north = 149 ;
>         bottom_top = 56 ;
>
>         float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
>                 PM2_5_DRY:FieldType = 104 ;
>                 PM2_5_DRY:MemoryOrder = "XYZ" ;
>                 PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
>                 PM2_5_DRY:units = "ug m^-3" ;
>                 PM2_5_DRY:stagger = "" ;
>                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
>
> Within the observation file:
>
> dimensions:
>         obs_arr_len = 5 ;
>         nhdr = 1820 ;
>         nobs = UNLIMITED ; // (1820 currently)
>
>         float obs_arr(nobs, obs_arr_len) ;
>                 obs_arr:long_name = "array of observation values" ;
>                 obs_arr:_fill_value = -9999.f ;
>                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>                 obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
>                 obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
>                 obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
>                 obs_arr:hgt_long_name = "height in meters above sea
level
> or ground level (msl or agl)" ;
>                 obs_arr:ob_long_name = "observation value" ;
>
> Additional information:
>
> "ADPSFC" is the message type within the header of the observation
files,
> but it does not appear in the forecast file anywhere.
> I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents the
> first member of an array and not 1.  The second 0 indicates the
surface
> data should be used, the first of 56 levels.  I used "*" to
represent the X
> and Y nature of the grid.
> I am using obs_arr(*,4) because there are 5 data fields, and only
the 5th
> one (numbered 4 when starting from 0) is the actual data field of
> interest.  The first * indicates to use all 1820 observations.
> I built the observation file using 'ascii2nc'; the forecast file was
built
> separately by a colleague.
>
> The observation file is ~350KB so I could easily e-mail it, but the
model
> data is ~1.3GB, so obviously I can't easily send that along for you
to test
> with.  Perhaps I could FTP it somewhere you could try to replicate
the
> error?
>
> Do you see anything here that I've made a mistake with?  I've spent
a
> couple hours staring at code and reading the user's manual; I can't
find
> any guidance as to where I made the mistake.  I'm hoping you can
help.
>
> Thanks,
> Matt
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research
> 16WS/WXN, Offutt AFB, NE
> (402) 294-3473  DSN: 271-3473
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 10:01:33 2014

Hi Julie,

I have placed the files in the directory-let me know if you can't read
them.  I'm sorry the model file size is so huge... but it's not my
model so I have no say in the matter!  If I did I would have asked for
GriB data instead since I have had much more success with GriB in MET
than I have NetCDF.  Something about the notation the config file
needs always trips me up.  I was sure I had figured it out this time
but the code falls flat when it hits the model data that I'm not sure
what it needs.

I *think* the model file is transferring.  Full file size is 1 310 160
288 bytes.  I need to leave for a couple hours so I can't monitor it.
I did push the observation and config files already so hopefully you
see those-the last file will be there once the full file size is
reached.  With no diagnostics returned I'm not sure how slow the
transfer rate is...

Thanks,
Matt

-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 10:37 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue

Hi Matt.  I see that you are having problems running point_stat.
Thank you for including the details of the situation.  Could you
please follow these instructions for posting the forecast file,
observation file, and config file to our ftp site:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp

I can take a closer look and try to resolve the problem.

Thanks,
Julie

On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:

>
> Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
>        Queue: met_help
>      Subject: MET 4.1: Point_stat issue
>        Owner: Nobody
>   Requestors: matthew.sittel.ctr at us.af.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> >
>
>
> Hello again,
>
> I'm trying to find a problem with my inability to run point_stat for
a
> pair of NetCDF files.  Here's what I am seeing from MET:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Here's what I defined for the fields in the point_stat config file:
>
> fcst = {
>    message_type = [ "ADPSFC" ];
>    field = [
>       {
>          name = "PM2_5_DRY(0,0,*,*)";
>          cat_thresh = [ >=0.0 ];
>       }
>    ];
> };
>
> obs = {
>    field = [
>       {
>          name = "obs_arr(*,4)";
>          cat_thresh = [ >=0.0 ];
>       }
>    ];
> };
>
> Within the forecast NetCDF file the following values are set:
>
> dimensions:
>         Time = UNLIMITED ; // (1 currently)
>         west_east = 249 ;
>         south_north = 149 ;
>         bottom_top = 56 ;
>
>         float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
>                 PM2_5_DRY:FieldType = 104 ;
>                 PM2_5_DRY:MemoryOrder = "XYZ" ;
>                 PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
>                 PM2_5_DRY:units = "ug m^-3" ;
>                 PM2_5_DRY:stagger = "" ;
>                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
>
> Within the observation file:
>
> dimensions:
>         obs_arr_len = 5 ;
>         nhdr = 1820 ;
>         nobs = UNLIMITED ; // (1820 currently)
>
>         float obs_arr(nobs, obs_arr_len) ;
>                 obs_arr:long_name = "array of observation values" ;
>                 obs_arr:_fill_value = -9999.f ;
>                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>                 obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
>                 obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
>                 obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
>                 obs_arr:hgt_long_name = "height in meters above sea
> level or ground level (msl or agl)" ;
>                 obs_arr:ob_long_name = "observation value" ;
>
> Additional information:
>
> "ADPSFC" is the message type within the header of the observation
> files, but it does not appear in the forecast file anywhere.
> I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents
> the first member of an array and not 1.  The second 0 indicates the
> surface data should be used, the first of 56 levels.  I used "*" to
> represent the X and Y nature of the grid.
> I am using obs_arr(*,4) because there are 5 data fields, and only
the
> 5th one (numbered 4 when starting from 0) is the actual data field
of
> interest.  The first * indicates to use all 1820 observations.
> I built the observation file using 'ascii2nc'; the forecast file was
> built separately by a colleague.
>
> The observation file is ~350KB so I could easily e-mail it, but the
> model data is ~1.3GB, so obviously I can't easily send that along
for
> you to test with.  Perhaps I could FTP it somewhere you could try to
> replicate the error?
>
> Do you see anything here that I've made a mistake with?  I've spent
a
> couple hours staring at code and reading the user's manual; I can't
> find any guidance as to where I made the mistake.  I'm hoping you
can help.
>
> Thanks,
> Matt
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
> NE
> (402) 294-3473  DSN: 271-3473
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu



------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 12:06:07 2014

Looks like I need to be more patient.  I assumed when I got back to
work that the file transfer was hung so I aborted it.  Turns out it
has transferred 430 MB of the full 1.3 GB of the model data file.  So
looks like transfer will take 6-8 hours at this rate.

With any luck it will run to completion... sorry for the delay, but I
don't know any better way to get you the file short of mailing a DVD!

Matt

-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 10:37 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue

Hi Matt.  I see that you are having problems running point_stat.
Thank you for including the details of the situation.  Could you
please follow these instructions for posting the forecast file,
observation file, and config file to our ftp site:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp

I can take a closer look and try to resolve the problem.

Thanks,
Julie

On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:

>
> Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
>        Queue: met_help
>      Subject: MET 4.1: Point_stat issue
>        Owner: Nobody
>   Requestors: matthew.sittel.ctr at us.af.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> >
>
>
> Hello again,
>
> I'm trying to find a problem with my inability to run point_stat for
a
> pair of NetCDF files.  Here's what I am seeing from MET:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File:
> /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_NcMet".
> NetCDF: Attribute not found
>
> Here's what I defined for the fields in the point_stat config file:
>
> fcst = {
>    message_type = [ "ADPSFC" ];
>    field = [
>       {
>          name = "PM2_5_DRY(0,0,*,*)";
>          cat_thresh = [ >=0.0 ];
>       }
>    ];
> };
>
> obs = {
>    field = [
>       {
>          name = "obs_arr(*,4)";
>          cat_thresh = [ >=0.0 ];
>       }
>    ];
> };
>
> Within the forecast NetCDF file the following values are set:
>
> dimensions:
>         Time = UNLIMITED ; // (1 currently)
>         west_east = 249 ;
>         south_north = 149 ;
>         bottom_top = 56 ;
>
>         float PM2_5_DRY(Time, bottom_top, south_north, west_east) ;
>                 PM2_5_DRY:FieldType = 104 ;
>                 PM2_5_DRY:MemoryOrder = "XYZ" ;
>                 PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
>                 PM2_5_DRY:units = "ug m^-3" ;
>                 PM2_5_DRY:stagger = "" ;
>                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
>
> Within the observation file:
>
> dimensions:
>         obs_arr_len = 5 ;
>         nhdr = 1820 ;
>         nobs = UNLIMITED ; // (1820 currently)
>
>         float obs_arr(nobs, obs_arr_len) ;
>                 obs_arr:long_name = "array of observation values" ;
>                 obs_arr:_fill_value = -9999.f ;
>                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>                 obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
>                 obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
>                 obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
>                 obs_arr:hgt_long_name = "height in meters above sea
> level or ground level (msl or agl)" ;
>                 obs_arr:ob_long_name = "observation value" ;
>
> Additional information:
>
> "ADPSFC" is the message type within the header of the observation
> files, but it does not appear in the forecast file anywhere.
> I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents
> the first member of an array and not 1.  The second 0 indicates the
> surface data should be used, the first of 56 levels.  I used "*" to
> represent the X and Y nature of the grid.
> I am using obs_arr(*,4) because there are 5 data fields, and only
the
> 5th one (numbered 4 when starting from 0) is the actual data field
of
> interest.  The first * indicates to use all 1820 observations.
> I built the observation file using 'ascii2nc'; the forecast file was
> built separately by a colleague.
>
> The observation file is ~350KB so I could easily e-mail it, but the
> model data is ~1.3GB, so obviously I can't easily send that along
for
> you to test with.  Perhaps I could FTP it somewhere you could try to
> replicate the error?
>
> Do you see anything here that I've made a mistake with?  I've spent
a
> couple hours staring at code and reading the user's manual; I can't
> find any guidance as to where I made the mistake.  I'm hoping you
can help.
>
> Thanks,
> Matt
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
> NE
> (402) 294-3473  DSN: 271-3473
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu



------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Mon Sep 22 12:08:19 2014

Ah, that's too bad.  I had been watching it slowly come in.  I'll keep
an
eye out for it.

Julie

On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Looks like I need to be more patient.  I assumed when I got back to
work
> that the file transfer was hung so I aborted it.  Turns out it has
> transferred 430 MB of the full 1.3 GB of the model data file.  So
looks
> like transfer will take 6-8 hours at this rate.
>
> With any luck it will run to completion... sorry for the delay, but
I
> don't know any better way to get you the file short of mailing a
DVD!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 10:37 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt.  I see that you are having problems running point_stat.
Thank
> you for including the details of the situation.  Could you please
follow
> these instructions for posting the forecast file, observation file,
and
> config file to our ftp site:
>    http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> I can take a closer look and try to resolve the problem.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
> via RT <met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> >        Queue: met_help
> >      Subject: MET 4.1: Point_stat issue
> >        Owner: Nobody
> >   Requestors: matthew.sittel.ctr at us.af.mil
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > >
> >
> >
> > Hello again,
> >
> > I'm trying to find a problem with my inability to run point_stat
for a
> > pair of NetCDF files.  Here's what I am seeing from MET:
> >
> > DEBUG 1: Default Config File:
> > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > DEBUG 1: User Config File:
> > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > NetCDF: Attribute not found
> >
> > Here's what I defined for the fields in the point_stat config
file:
> >
> > fcst = {
> >    message_type = [ "ADPSFC" ];
> >    field = [
> >       {
> >          name = "PM2_5_DRY(0,0,*,*)";
> >          cat_thresh = [ >=0.0 ];
> >       }
> >    ];
> > };
> >
> > obs = {
> >    field = [
> >       {
> >          name = "obs_arr(*,4)";
> >          cat_thresh = [ >=0.0 ];
> >       }
> >    ];
> > };
> >
> > Within the forecast NetCDF file the following values are set:
> >
> > dimensions:
> >         Time = UNLIMITED ; // (1 currently)
> >         west_east = 249 ;
> >         south_north = 149 ;
> >         bottom_top = 56 ;
> >
> >         float PM2_5_DRY(Time, bottom_top, south_north, west_east)
;
> >                 PM2_5_DRY:FieldType = 104 ;
> >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> >                 PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> >                 PM2_5_DRY:units = "ug m^-3" ;
> >                 PM2_5_DRY:stagger = "" ;
> >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> >
> > Within the observation file:
> >
> > dimensions:
> >         obs_arr_len = 5 ;
> >         nhdr = 1820 ;
> >         nobs = UNLIMITED ; // (1820 currently)
> >
> >         float obs_arr(nobs, obs_arr_len) ;
> >                 obs_arr:long_name = "array of observation values"
;
> >                 obs_arr:_fill_value = -9999.f ;
> >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> >                 obs_arr:hdr_id_long_name = "index of matching
header
> data"
> > ;
> >                 obs_arr:gc_long_name = "grib code corresponding to
the
> > observation type" ;
> >                 obs_arr:lvl_long_name = "pressure level (hPa) or
> > accumulation interval (sec)" ;
> >                 obs_arr:hgt_long_name = "height in meters above
sea
> > level or ground level (msl or agl)" ;
> >                 obs_arr:ob_long_name = "observation value" ;
> >
> > Additional information:
> >
> > "ADPSFC" is the message type within the header of the observation
> > files, but it does not appear in the forecast file anywhere.
> > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
represents
> > the first member of an array and not 1.  The second 0 indicates
the
> > surface data should be used, the first of 56 levels.  I used "*"
to
> > represent the X and Y nature of the grid.
> > I am using obs_arr(*,4) because there are 5 data fields, and only
the
> > 5th one (numbered 4 when starting from 0) is the actual data field
of
> > interest.  The first * indicates to use all 1820 observations.
> > I built the observation file using 'ascii2nc'; the forecast file
was
> > built separately by a colleague.
> >
> > The observation file is ~350KB so I could easily e-mail it, but
the
> > model data is ~1.3GB, so obviously I can't easily send that along
for
> > you to test with.  Perhaps I could FTP it somewhere you could try
to
> > replicate the error?
> >
> > Do you see anything here that I've made a mistake with?  I've
spent a
> > couple hours staring at code and reading the user's manual; I
can't
> > find any guidance as to where I made the mistake.  I'm hoping you
can
> help.
> >
> > Thanks,
> > Matt
> >
> > // SIGNED //
> > Matthew C. Sittel
> > Associate Scientist
> > University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
> > NE
> > (402) 294-3473  DSN: 271-3473
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 12:14:36 2014

I opened a separate window and did a list on the directory but it said
nothing was in there... which left me even more confused.    I have
restarted the FTP but looks like I have no way on my end to monitor
progress... so I'll leave it running all night if I must and hope it
continues running.

If you get a chance later this afternoon, please let me know how far
along the file is so I can estimate transfer speed for future
reference.

Thanks!

Matt

-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 1:08 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue

Ah, that's too bad.  I had been watching it slowly come in.  I'll keep
an eye out for it.

Julie

On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Looks like I need to be more patient.  I assumed when I got back to
> work that the file transfer was hung so I aborted it.  Turns out it
> has transferred 430 MB of the full 1.3 GB of the model data file.
So
> looks like transfer will take 6-8 hours at this rate.
>
> With any luck it will run to completion... sorry for the delay, but
I
> don't know any better way to get you the file short of mailing a
DVD!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 10:37 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt.  I see that you are having problems running point_stat.
> Thank you for including the details of the situation.  Could you
> please follow these instructions for posting the forecast file,
> observation file, and config file to our ftp site:
>    http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> I can take a closer look and try to resolve the problem.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> >        Queue: met_help
> >      Subject: MET 4.1: Point_stat issue
> >        Owner: Nobody
> >   Requestors: matthew.sittel.ctr at us.af.mil
> >       Status: new
> >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > >
> >
> >
> > Hello again,
> >
> > I'm trying to find a problem with my inability to run point_stat
for
> > a pair of NetCDF files.  Here's what I am seeing from MET:
> >
> > DEBUG 1: Default Config File:
> > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > DEBUG 1: User Config File:
> > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > Met2dDataFile object of type "FileType_NcMet".
> > NetCDF: Attribute not found
> >
> > Here's what I defined for the fields in the point_stat config
file:
> >
> > fcst = {
> >    message_type = [ "ADPSFC" ];
> >    field = [
> >       {
> >          name = "PM2_5_DRY(0,0,*,*)";
> >          cat_thresh = [ >=0.0 ];
> >       }
> >    ];
> > };
> >
> > obs = {
> >    field = [
> >       {
> >          name = "obs_arr(*,4)";
> >          cat_thresh = [ >=0.0 ];
> >       }
> >    ];
> > };
> >
> > Within the forecast NetCDF file the following values are set:
> >
> > dimensions:
> >         Time = UNLIMITED ; // (1 currently)
> >         west_east = 249 ;
> >         south_north = 149 ;
> >         bottom_top = 56 ;
> >
> >         float PM2_5_DRY(Time, bottom_top, south_north, west_east)
;
> >                 PM2_5_DRY:FieldType = 104 ;
> >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> >                 PM2_5_DRY:description = "pm2.5 aerosol dry mass" ;
> >                 PM2_5_DRY:units = "ug m^-3" ;
> >                 PM2_5_DRY:stagger = "" ;
> >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> >
> > Within the observation file:
> >
> > dimensions:
> >         obs_arr_len = 5 ;
> >         nhdr = 1820 ;
> >         nobs = UNLIMITED ; // (1820 currently)
> >
> >         float obs_arr(nobs, obs_arr_len) ;
> >                 obs_arr:long_name = "array of observation values"
;
> >                 obs_arr:_fill_value = -9999.f ;
> >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> >                 obs_arr:hdr_id_long_name = "index of matching
header
> data"
> > ;
> >                 obs_arr:gc_long_name = "grib code corresponding to
> > the observation type" ;
> >                 obs_arr:lvl_long_name = "pressure level (hPa) or
> > accumulation interval (sec)" ;
> >                 obs_arr:hgt_long_name = "height in meters above
sea
> > level or ground level (msl or agl)" ;
> >                 obs_arr:ob_long_name = "observation value" ;
> >
> > Additional information:
> >
> > "ADPSFC" is the message type within the header of the observation
> > files, but it does not appear in the forecast file anywhere.
> > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > represents the first member of an array and not 1.  The second 0
> > indicates the surface data should be used, the first of 56 levels.
> > I used "*" to represent the X and Y nature of the grid.
> > I am using obs_arr(*,4) because there are 5 data fields, and only
> > the 5th one (numbered 4 when starting from 0) is the actual data
> > field of interest.  The first * indicates to use all 1820
observations.
> > I built the observation file using 'ascii2nc'; the forecast file
was
> > built separately by a colleague.
> >
> > The observation file is ~350KB so I could easily e-mail it, but
the
> > model data is ~1.3GB, so obviously I can't easily send that along
> > for you to test with.  Perhaps I could FTP it somewhere you could
> > try to replicate the error?
> >
> > Do you see anything here that I've made a mistake with?  I've
spent
> > a couple hours staring at code and reading the user's manual; I
> > can't find any guidance as to where I made the mistake.  I'm
hoping
> > you can
> help.
> >
> > Thanks,
> > Matt
> >
> > // SIGNED //
> > Matthew C. Sittel
> > Associate Scientist
> > University Corporation for Atmospheric Research 16WS/WXN, Offutt
> > AFB, NE
> > (402) 294-3473  DSN: 271-3473
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu



------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Mon Sep 22 12:18:13 2014

I understand your confusion.  The file was being written to
incoming/irap/met_help/ as opposed to
incoming/irap/met_help/sittel_data/
where it is being written to now.  I will get back to you this
afternoon
and let you know how far along it is.

Julie

On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> I opened a separate window and did a list on the directory but it
said
> nothing was in there... which left me even more confused.    I have
> restarted the FTP but looks like I have no way on my end to monitor
> progress... so I'll leave it running all night if I must and hope it
> continues running.
>
> If you get a chance later this afternoon, please let me know how far
along
> the file is so I can estimate transfer speed for future reference.
>
> Thanks!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:08 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Ah, that's too bad.  I had been watching it slowly come in.  I'll
keep an
> eye out for it.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Looks like I need to be more patient.  I assumed when I got back
to
> > work that the file transfer was hung so I aborted it.  Turns out
it
> > has transferred 430 MB of the full 1.3 GB of the model data file.
So
> > looks like transfer will take 6-8 hours at this rate.
> >
> > With any luck it will run to completion... sorry for the delay,
but I
> > don't know any better way to get you the file short of mailing a
DVD!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 10:37 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt.  I see that you are having problems running point_stat.
> > Thank you for including the details of the situation.  Could you
> > please follow these instructions for posting the forecast file,
> > observation file, and config file to our ftp site:
> >    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > I can take a closer look and try to resolve the problem.
> >
> > Thanks,
> > Julie
> >
> > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > >        Queue: met_help
> > >      Subject: MET 4.1: Point_stat issue
> > >        Owner: Nobody
> > >   Requestors: matthew.sittel.ctr at us.af.mil
> > >       Status: new
> > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > >
> > >
> > >
> > > Hello again,
> > >
> > > I'm trying to find a problem with my inability to run point_stat
for
> > > a pair of NetCDF files.  Here's what I am seeing from MET:
> > >
> > > DEBUG 1: Default Config File:
> > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > DEBUG 1: User Config File:
> > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> > > Met2dDataFile object of type "FileType_NcMet".
> > > NetCDF: Attribute not found
> > >
> > > Here's what I defined for the fields in the point_stat config
file:
> > >
> > > fcst = {
> > >    message_type = [ "ADPSFC" ];
> > >    field = [
> > >       {
> > >          name = "PM2_5_DRY(0,0,*,*)";
> > >          cat_thresh = [ >=0.0 ];
> > >       }
> > >    ];
> > > };
> > >
> > > obs = {
> > >    field = [
> > >       {
> > >          name = "obs_arr(*,4)";
> > >          cat_thresh = [ >=0.0 ];
> > >       }
> > >    ];
> > > };
> > >
> > > Within the forecast NetCDF file the following values are set:
> > >
> > > dimensions:
> > >         Time = UNLIMITED ; // (1 currently)
> > >         west_east = 249 ;
> > >         south_north = 149 ;
> > >         bottom_top = 56 ;
> > >
> > >         float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > >                 PM2_5_DRY:FieldType = 104 ;
> > >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> > >                 PM2_5_DRY:description = "pm2.5 aerosol dry mass"
;
> > >                 PM2_5_DRY:units = "ug m^-3" ;
> > >                 PM2_5_DRY:stagger = "" ;
> > >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > >
> > > Within the observation file:
> > >
> > > dimensions:
> > >         obs_arr_len = 5 ;
> > >         nhdr = 1820 ;
> > >         nobs = UNLIMITED ; // (1820 currently)
> > >
> > >         float obs_arr(nobs, obs_arr_len) ;
> > >                 obs_arr:long_name = "array of observation
values" ;
> > >                 obs_arr:_fill_value = -9999.f ;
> > >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > >                 obs_arr:hdr_id_long_name = "index of matching
header
> > data"
> > > ;
> > >                 obs_arr:gc_long_name = "grib code corresponding
to
> > > the observation type" ;
> > >                 obs_arr:lvl_long_name = "pressure level (hPa) or
> > > accumulation interval (sec)" ;
> > >                 obs_arr:hgt_long_name = "height in meters above
sea
> > > level or ground level (msl or agl)" ;
> > >                 obs_arr:ob_long_name = "observation value" ;
> > >
> > > Additional information:
> > >
> > > "ADPSFC" is the message type within the header of the
observation
> > > files, but it does not appear in the forecast file anywhere.
> > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > represents the first member of an array and not 1.  The second 0
> > > indicates the surface data should be used, the first of 56
levels.
> > > I used "*" to represent the X and Y nature of the grid.
> > > I am using obs_arr(*,4) because there are 5 data fields, and
only
> > > the 5th one (numbered 4 when starting from 0) is the actual data
> > > field of interest.  The first * indicates to use all 1820
observations.
> > > I built the observation file using 'ascii2nc'; the forecast file
was
> > > built separately by a colleague.
> > >
> > > The observation file is ~350KB so I could easily e-mail it, but
the
> > > model data is ~1.3GB, so obviously I can't easily send that
along
> > > for you to test with.  Perhaps I could FTP it somewhere you
could
> > > try to replicate the error?
> > >
> > > Do you see anything here that I've made a mistake with?  I've
spent
> > > a couple hours staring at code and reading the user's manual; I
> > > can't find any guidance as to where I made the mistake.  I'm
hoping
> > > you can
> > help.
> > >
> > > Thanks,
> > > Matt
> > >
> > > // SIGNED //
> > > Matthew C. Sittel
> > > Associate Scientist
> > > University Corporation for Atmospheric Research 16WS/WXN, Offutt
> > > AFB, NE
> > > (402) 294-3473  DSN: 271-3473
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Sep 22 12:26:10 2014

Thanks-I got it to work on my end so you don't have to monitor it.
Sorry about the directory mixup-I created the directory and thought I
was putting the files there!

58 MB right now thus far I see... let's hope it works!

Matt

-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Monday, September 22, 2014 1:18 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue

I understand your confusion.  The file was being written to
incoming/irap/met_help/ as opposed to
incoming/irap/met_help/sittel_data/
where it is being written to now.  I will get back to you this
afternoon and let you know how far along it is.

Julie

On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> I opened a separate window and did a list on the directory but it
said
> nothing was in there... which left me even more confused.    I have
> restarted the FTP but looks like I have no way on my end to monitor
> progress... so I'll leave it running all night if I must and hope it
> continues running.
>
> If you get a chance later this afternoon, please let me know how far
> along the file is so I can estimate transfer speed for future
reference.
>
> Thanks!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:08 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Ah, that's too bad.  I had been watching it slowly come in.  I'll
keep
> an eye out for it.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Looks like I need to be more patient.  I assumed when I got back
to
> > work that the file transfer was hung so I aborted it.  Turns out
it
> > has transferred 430 MB of the full 1.3 GB of the model data file.
> > So looks like transfer will take 6-8 hours at this rate.
> >
> > With any luck it will run to completion... sorry for the delay,
but
> > I don't know any better way to get you the file short of mailing a
DVD!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 10:37 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Hi Matt.  I see that you are having problems running point_stat.
> > Thank you for including the details of the situation.  Could you
> > please follow these instructions for posting the forecast file,
> > observation file, and config file to our ftp site:
> >    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >
> > I can take a closer look and try to resolve the problem.
> >
> > Thanks,
> > Julie
> >
> > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > >        Queue: met_help
> > >      Subject: MET 4.1: Point_stat issue
> > >        Owner: Nobody
> > >   Requestors: matthew.sittel.ctr at us.af.mil
> > >       Status: new
> > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > >
> > >
> > >
> > > Hello again,
> > >
> > > I'm trying to find a problem with my inability to run point_stat
> > > for a pair of NetCDF files.  Here's what I am seeing from MET:
> > >
> > > DEBUG 1: Default Config File:
> > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > DEBUG 1: User Config File:
> > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
> > > new Met2dDataFile object of type "FileType_NcMet".
> > > NetCDF: Attribute not found
> > >
> > > Here's what I defined for the fields in the point_stat config
file:
> > >
> > > fcst = {
> > >    message_type = [ "ADPSFC" ];
> > >    field = [
> > >       {
> > >          name = "PM2_5_DRY(0,0,*,*)";
> > >          cat_thresh = [ >=0.0 ];
> > >       }
> > >    ];
> > > };
> > >
> > > obs = {
> > >    field = [
> > >       {
> > >          name = "obs_arr(*,4)";
> > >          cat_thresh = [ >=0.0 ];
> > >       }
> > >    ];
> > > };
> > >
> > > Within the forecast NetCDF file the following values are set:
> > >
> > > dimensions:
> > >         Time = UNLIMITED ; // (1 currently)
> > >         west_east = 249 ;
> > >         south_north = 149 ;
> > >         bottom_top = 56 ;
> > >
> > >         float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > >                 PM2_5_DRY:FieldType = 104 ;
> > >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> > >                 PM2_5_DRY:description = "pm2.5 aerosol dry mass"
;
> > >                 PM2_5_DRY:units = "ug m^-3" ;
> > >                 PM2_5_DRY:stagger = "" ;
> > >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > >
> > > Within the observation file:
> > >
> > > dimensions:
> > >         obs_arr_len = 5 ;
> > >         nhdr = 1820 ;
> > >         nobs = UNLIMITED ; // (1820 currently)
> > >
> > >         float obs_arr(nobs, obs_arr_len) ;
> > >                 obs_arr:long_name = "array of observation
values" ;
> > >                 obs_arr:_fill_value = -9999.f ;
> > >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > >                 obs_arr:hdr_id_long_name = "index of matching
> > > header
> > data"
> > > ;
> > >                 obs_arr:gc_long_name = "grib code corresponding
to
> > > the observation type" ;
> > >                 obs_arr:lvl_long_name = "pressure level (hPa) or
> > > accumulation interval (sec)" ;
> > >                 obs_arr:hgt_long_name = "height in meters above
> > > sea level or ground level (msl or agl)" ;
> > >                 obs_arr:ob_long_name = "observation value" ;
> > >
> > > Additional information:
> > >
> > > "ADPSFC" is the message type within the header of the
observation
> > > files, but it does not appear in the forecast file anywhere.
> > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > represents the first member of an array and not 1.  The second 0
> > > indicates the surface data should be used, the first of 56
levels.
> > > I used "*" to represent the X and Y nature of the grid.
> > > I am using obs_arr(*,4) because there are 5 data fields, and
only
> > > the 5th one (numbered 4 when starting from 0) is the actual data
> > > field of interest.  The first * indicates to use all 1820
observations.
> > > I built the observation file using 'ascii2nc'; the forecast file
> > > was built separately by a colleague.
> > >
> > > The observation file is ~350KB so I could easily e-mail it, but
> > > the model data is ~1.3GB, so obviously I can't easily send that
> > > along for you to test with.  Perhaps I could FTP it somewhere
you
> > > could try to replicate the error?
> > >
> > > Do you see anything here that I've made a mistake with?  I've
> > > spent a couple hours staring at code and reading the user's
> > > manual; I can't find any guidance as to where I made the
mistake.
> > > I'm hoping you can
> > help.
> > >
> > > Thanks,
> > > Matt
> > >
> > > // SIGNED //
> > > Matthew C. Sittel
> > > Associate Scientist
> > > University Corporation for Atmospheric Research 16WS/WXN, Offutt
> > > AFB, NE
> > > (402) 294-3473  DSN: 271-3473
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu



------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Tue Sep 23 14:21:59 2014

Hi Matt.  Thanks for getting the data to us.

The forecast file that you sent was a raw wrfout file, which MET
doesn't
support.  The raw wrfout files have to be post-processed using
p_interp or
UPP.  UPP tends to work better because it writes GRIB files that MET
supports well.  For p_interp output, we can't read variables that are
defined on the staggered dimensions, like wind.

We used p_interp (available here:
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing)
to convert your file into a NetCDF format that MET can read.  p_interp
also
interpolates to pressure levels, and many of the grid points for the
first
few pressure levels are actually below ground.  So you'll notice a lot
of
missing values in the first few pressure levels of the p_interp
output.  We
also made some changes to your configuration file.  I have attached
the new
configuration file for you to use.   Here is a brief description of
the
changes:

-Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
-Modified
         name = "PM2_5_DRY(0,0,*,*)";
to
         name = "PM2_5_DRY";
         level = "(0,0,*,*)";
specifying the level separately
-Modified
         name = "obs_arr(*,4)";
to
         name = "NCIP";
         level = "L0";
because the obs information needs to be specified as if it were a GRIB
file.  NCIP came from grib code 198 (specified in obs_arr)
- Changed sid to be [] instead of ""

With these changes, we were able to successfully run point_stat.
However,
please note that the forecast values for PM2_5_DRY are all zero!  You
could
take a look at your WRF-Chem run to figure out why.  I'd suggest using
ncview to display the PM2_5_DRY variable in the NetCDF file you sent
us.
You'll see that it's zero for all model levels.  So you may want to
look
into that.

I hope this helps.

Thanks,
Julie

On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Thanks-I got it to work on my end so you don't have to monitor it.
Sorry
> about the directory mixup-I created the directory and thought I was
putting
> the files there!
>
> 58 MB right now thus far I see... let's hope it works!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:18 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> I understand your confusion.  The file was being written to
> incoming/irap/met_help/ as opposed to
incoming/irap/met_help/sittel_data/
> where it is being written to now.  I will get back to you this
afternoon
> and let you know how far along it is.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > I opened a separate window and did a list on the directory but it
said
> > nothing was in there... which left me even more confused.    I
have
> > restarted the FTP but looks like I have no way on my end to
monitor
> > progress... so I'll leave it running all night if I must and hope
it
> > continues running.
> >
> > If you get a chance later this afternoon, please let me know how
far
> > along the file is so I can estimate transfer speed for future
reference.
> >
> > Thanks!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:08 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Ah, that's too bad.  I had been watching it slowly come in.  I'll
keep
> > an eye out for it.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Looks like I need to be more patient.  I assumed when I got back
to
> > > work that the file transfer was hung so I aborted it.  Turns out
it
> > > has transferred 430 MB of the full 1.3 GB of the model data
file.
> > > So looks like transfer will take 6-8 hours at this rate.
> > >
> > > With any luck it will run to completion... sorry for the delay,
but
> > > I don't know any better way to get you the file short of mailing
a DVD!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 10:37 AM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt.  I see that you are having problems running point_stat.
> > > Thank you for including the details of the situation.  Could you
> > > please follow these instructions for posting the forecast file,
> > > observation file, and config file to our ftp site:
> > >    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > I can take a closer look and try to resolve the problem.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > >        Queue: met_help
> > > >      Subject: MET 4.1: Point_stat issue
> > > >        Owner: Nobody
> > > >   Requestors: matthew.sittel.ctr at us.af.mil
> > > >       Status: new
> > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > >
> > > >
> > > >
> > > > Hello again,
> > > >
> > > > I'm trying to find a problem with my inability to run
point_stat
> > > > for a pair of NetCDF files.  Here's what I am seeing from MET:
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > NetCDF: Attribute not found
> > > >
> > > > Here's what I defined for the fields in the point_stat config
file:
> > > >
> > > > fcst = {
> > > >    message_type = [ "ADPSFC" ];
> > > >    field = [
> > > >       {
> > > >          name = "PM2_5_DRY(0,0,*,*)";
> > > >          cat_thresh = [ >=0.0 ];
> > > >       }
> > > >    ];
> > > > };
> > > >
> > > > obs = {
> > > >    field = [
> > > >       {
> > > >          name = "obs_arr(*,4)";
> > > >          cat_thresh = [ >=0.0 ];
> > > >       }
> > > >    ];
> > > > };
> > > >
> > > > Within the forecast NetCDF file the following values are set:
> > > >
> > > > dimensions:
> > > >         Time = UNLIMITED ; // (1 currently)
> > > >         west_east = 249 ;
> > > >         south_north = 149 ;
> > > >         bottom_top = 56 ;
> > > >
> > > >         float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > >                 PM2_5_DRY:FieldType = 104 ;
> > > >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > >                 PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > >                 PM2_5_DRY:units = "ug m^-3" ;
> > > >                 PM2_5_DRY:stagger = "" ;
> > > >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > >
> > > > Within the observation file:
> > > >
> > > > dimensions:
> > > >         obs_arr_len = 5 ;
> > > >         nhdr = 1820 ;
> > > >         nobs = UNLIMITED ; // (1820 currently)
> > > >
> > > >         float obs_arr(nobs, obs_arr_len) ;
> > > >                 obs_arr:long_name = "array of observation
values" ;
> > > >                 obs_arr:_fill_value = -9999.f ;
> > > >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > >                 obs_arr:hdr_id_long_name = "index of matching
> > > > header
> > > data"
> > > > ;
> > > >                 obs_arr:gc_long_name = "grib code
corresponding to
> > > > the observation type" ;
> > > >                 obs_arr:lvl_long_name = "pressure level (hPa)
or
> > > > accumulation interval (sec)" ;
> > > >                 obs_arr:hgt_long_name = "height in meters
above
> > > > sea level or ground level (msl or agl)" ;
> > > >                 obs_arr:ob_long_name = "observation value" ;
> > > >
> > > > Additional information:
> > > >
> > > > "ADPSFC" is the message type within the header of the
observation
> > > > files, but it does not appear in the forecast file anywhere.
> > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > represents the first member of an array and not 1.  The second
0
> > > > indicates the surface data should be used, the first of 56
levels.
> > > > I used "*" to represent the X and Y nature of the grid.
> > > > I am using obs_arr(*,4) because there are 5 data fields, and
only
> > > > the 5th one (numbered 4 when starting from 0) is the actual
data
> > > > field of interest.  The first * indicates to use all 1820
> observations.
> > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > was built separately by a colleague.
> > > >
> > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > along for you to test with.  Perhaps I could FTP it somewhere
you
> > > > could try to replicate the error?
> > > >
> > > > Do you see anything here that I've made a mistake with?  I've
> > > > spent a couple hours staring at code and reading the user's
> > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > I'm hoping you can
> > > help.
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > // SIGNED //
> > > > Matthew C. Sittel
> > > > Associate Scientist
> > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > AFB, NE
> > > > (402) 294-3473  DSN: 271-3473
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Tue Sep 23 14:40:59 2014

Hi Julie,

Thanks for all your help!  I didn't know it wasn't usable in its
current form; it was given to me that way.  It's not my model, though,
so I get to tell someone else to worry about it instead of changing it
myself.  I'm guessing putting it on pressure levels is acceptable
since they just wanted the lowest one for me to use as verification.

I didn't look carefully at the model output, only the observation data
since I was more comfortable with it.  Sounds like another issue for
the model guys to look at.

Matt

-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 23, 2014 3:22 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue

Hi Matt.  Thanks for getting the data to us.

The forecast file that you sent was a raw wrfout file, which MET
doesn't support.  The raw wrfout files have to be post-processed using
p_interp or UPP.  UPP tends to work better because it writes GRIB
files that MET supports well.  For p_interp output, we can't read
variables that are defined on the staggered dimensions, like wind.

We used p_interp (available here:
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing)
to convert your file into a NetCDF format that MET can read.  p_interp
also interpolates to pressure levels, and many of the grid points for
the first few pressure levels are actually below ground.  So you'll
notice a lot of missing values in the first few pressure levels of the
p_interp output.  We also made some changes to your configuration
file.  I have attached the new
configuration file for you to use.   Here is a brief description of
the
changes:

-Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
-Modified
         name = "PM2_5_DRY(0,0,*,*)";
to
         name = "PM2_5_DRY";
         level = "(0,0,*,*)";
specifying the level separately
-Modified
         name = "obs_arr(*,4)";
to
         name = "NCIP";
         level = "L0";
because the obs information needs to be specified as if it were a GRIB
file.  NCIP came from grib code 198 (specified in obs_arr)
- Changed sid to be [] instead of ""

With these changes, we were able to successfully run point_stat.
However, please note that the forecast values for PM2_5_DRY are all
zero!  You could take a look at your WRF-Chem run to figure out why.
I'd suggest using ncview to display the PM2_5_DRY variable in the
NetCDF file you sent us.
You'll see that it's zero for all model levels.  So you may want to
look into that.

I hope this helps.

Thanks,
Julie

On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Thanks-I got it to work on my end so you don't have to monitor it.
Sorry
> about the directory mixup-I created the directory and thought I was
> putting the files there!
>
> 58 MB right now thus far I see... let's hope it works!
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Monday, September 22, 2014 1:18 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> I understand your confusion.  The file was being written to
> incoming/irap/met_help/ as opposed to
> incoming/irap/met_help/sittel_data/
> where it is being written to now.  I will get back to you this
> afternoon and let you know how far along it is.
>
> Julie
>
> On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > I opened a separate window and did a list on the directory but it
said
> > nothing was in there... which left me even more confused.    I
have
> > restarted the FTP but looks like I have no way on my end to
monitor
> > progress... so I'll leave it running all night if I must and hope
it
> > continues running.
> >
> > If you get a chance later this afternoon, please let me know how
far
> > along the file is so I can estimate transfer speed for future
reference.
> >
> > Thanks!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:08 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > Ah, that's too bad.  I had been watching it slowly come in.  I'll
> > keep an eye out for it.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > Looks like I need to be more patient.  I assumed when I got back
> > > to work that the file transfer was hung so I aborted it.  Turns
> > > out it has transferred 430 MB of the full 1.3 GB of the model
data file.
> > > So looks like transfer will take 6-8 hours at this rate.
> > >
> > > With any luck it will run to completion... sorry for the delay,
> > > but I don't know any better way to get you the file short of
mailing a DVD!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 10:37 AM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Hi Matt.  I see that you are having problems running point_stat.
> > > Thank you for including the details of the situation.  Could you
> > > please follow these instructions for posting the forecast file,
> > > observation file, and config file to our ftp site:
> > >    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > >
> > > I can take a closer look and try to resolve the problem.
> > >
> > > Thanks,
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF AFWA
> > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > >        Queue: met_help
> > > >      Subject: MET 4.1: Point_stat issue
> > > >        Owner: Nobody
> > > >   Requestors: matthew.sittel.ctr at us.af.mil
> > > >       Status: new
> > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > >
> > > >
> > > >
> > > > Hello again,
> > > >
> > > > I'm trying to find a problem with my inability to run
point_stat
> > > > for a pair of NetCDF files.  Here's what I am seeing from MET:
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > DEBUG 1: User Config File:
> > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > NetCDF: Attribute not found
> > > >
> > > > Here's what I defined for the fields in the point_stat config
file:
> > > >
> > > > fcst = {
> > > >    message_type = [ "ADPSFC" ];
> > > >    field = [
> > > >       {
> > > >          name = "PM2_5_DRY(0,0,*,*)";
> > > >          cat_thresh = [ >=0.0 ];
> > > >       }
> > > >    ];
> > > > };
> > > >
> > > > obs = {
> > > >    field = [
> > > >       {
> > > >          name = "obs_arr(*,4)";
> > > >          cat_thresh = [ >=0.0 ];
> > > >       }
> > > >    ];
> > > > };
> > > >
> > > > Within the forecast NetCDF file the following values are set:
> > > >
> > > > dimensions:
> > > >         Time = UNLIMITED ; // (1 currently)
> > > >         west_east = 249 ;
> > > >         south_north = 149 ;
> > > >         bottom_top = 56 ;
> > > >
> > > >         float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > >                 PM2_5_DRY:FieldType = 104 ;
> > > >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > >                 PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > >                 PM2_5_DRY:units = "ug m^-3" ;
> > > >                 PM2_5_DRY:stagger = "" ;
> > > >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > >
> > > > Within the observation file:
> > > >
> > > > dimensions:
> > > >         obs_arr_len = 5 ;
> > > >         nhdr = 1820 ;
> > > >         nobs = UNLIMITED ; // (1820 currently)
> > > >
> > > >         float obs_arr(nobs, obs_arr_len) ;
> > > >                 obs_arr:long_name = "array of observation
values" ;
> > > >                 obs_arr:_fill_value = -9999.f ;
> > > >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > >                 obs_arr:hdr_id_long_name = "index of matching
> > > > header
> > > data"
> > > > ;
> > > >                 obs_arr:gc_long_name = "grib code
corresponding
> > > > to the observation type" ;
> > > >                 obs_arr:lvl_long_name = "pressure level (hPa)
or
> > > > accumulation interval (sec)" ;
> > > >                 obs_arr:hgt_long_name = "height in meters
above
> > > > sea level or ground level (msl or agl)" ;
> > > >                 obs_arr:ob_long_name = "observation value" ;
> > > >
> > > > Additional information:
> > > >
> > > > "ADPSFC" is the message type within the header of the
> > > > observation files, but it does not appear in the forecast file
anywhere.
> > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > represents the first member of an array and not 1.  The second
0
> > > > indicates the surface data should be used, the first of 56
levels.
> > > > I used "*" to represent the X and Y nature of the grid.
> > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > only the 5th one (numbered 4 when starting from 0) is the
actual
> > > > data field of interest.  The first * indicates to use all 1820
> observations.
> > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > was built separately by a colleague.
> > > >
> > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > along for you to test with.  Perhaps I could FTP it somewhere
> > > > you could try to replicate the error?
> > > >
> > > > Do you see anything here that I've made a mistake with?  I've
> > > > spent a couple hours staring at code and reading the user's
> > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > I'm hoping you can
> > > help.
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > > > // SIGNED //
> > > > Matthew C. Sittel
> > > > Associate Scientist
> > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > AFB, NE
> > > > (402) 294-3473  DSN: 271-3473
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu



------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: Julie Prestopnik
Time: Tue Sep 23 14:50:48 2014

Thanks, Matt.  I'll resolve this ticket for now.  Please let us know
if you
encounter any other issues.

Julie

On Tue, Sep 23, 2014 at 2:40 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Thanks for all your help!  I didn't know it wasn't usable in its
current
> form; it was given to me that way.  It's not my model, though, so I
get to
> tell someone else to worry about it instead of changing it myself.
I'm
> guessing putting it on pressure levels is acceptable since they just
wanted
> the lowest one for me to use as verification.
>
> I didn't look carefully at the model output, only the observation
data
> since I was more comfortable with it.  Sounds like another issue for
the
> model guys to look at.
>
> Matt
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 23, 2014 3:22 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
>
> Hi Matt.  Thanks for getting the data to us.
>
> The forecast file that you sent was a raw wrfout file, which MET
doesn't
> support.  The raw wrfout files have to be post-processed using
p_interp or
> UPP.  UPP tends to work better because it writes GRIB files that MET
> supports well.  For p_interp output, we can't read variables that
are
> defined on the staggered dimensions, like wind.
>
> We used p_interp (available here:
>
>
http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html#post_processing
> )
> to convert your file into a NetCDF format that MET can read.
p_interp
> also interpolates to pressure levels, and many of the grid points
for the
> first few pressure levels are actually below ground.  So you'll
notice a
> lot of missing values in the first few pressure levels of the
p_interp
> output.  We also made some changes to your configuration file.  I
have
> attached the new
> configuration file for you to use.   Here is a brief description of
the
> changes:
>
> -Moved message_type = [ "ADPSFC" ]; to be within the obs dictionary
> -Modified
>          name = "PM2_5_DRY(0,0,*,*)";
> to
>          name = "PM2_5_DRY";
>          level = "(0,0,*,*)";
> specifying the level separately
> -Modified
>          name = "obs_arr(*,4)";
> to
>          name = "NCIP";
>          level = "L0";
> because the obs information needs to be specified as if it were a
GRIB
> file.  NCIP came from grib code 198 (specified in obs_arr)
> - Changed sid to be [] instead of ""
>
> With these changes, we were able to successfully run point_stat.
However,
> please note that the forecast values for PM2_5_DRY are all zero!
You could
> take a look at your WRF-Chem run to figure out why.  I'd suggest
using
> ncview to display the PM2_5_DRY variable in the NetCDF file you sent
us.
> You'll see that it's zero for all model levels.  So you may want to
look
> into that.
>
> I hope this helps.
>
> Thanks,
> Julie
>
> On Mon, Sep 22, 2014 at 12:26 PM, SITTEL, MATTHEW C CTR USAF AFWA 16
> WS/WXN via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> >
> > Thanks-I got it to work on my end so you don't have to monitor it.
>  Sorry
> > about the directory mixup-I created the directory and thought I
was
> > putting the files there!
> >
> > 58 MB right now thus far I see... let's hope it works!
> >
> > Matt
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, September 22, 2014 1:18 PM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> >
> > I understand your confusion.  The file was being written to
> > incoming/irap/met_help/ as opposed to
> > incoming/irap/met_help/sittel_data/
> > where it is being written to now.  I will get back to you this
> > afternoon and let you know how far along it is.
> >
> > Julie
> >
> > On Mon, Sep 22, 2014 at 12:14 PM, SITTEL, MATTHEW C CTR USAF AFWA
16
> > WS/WXN via RT <met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
> > >
> > > I opened a separate window and did a list on the directory but
it said
> > > nothing was in there... which left me even more confused.    I
have
> > > restarted the FTP but looks like I have no way on my end to
monitor
> > > progress... so I'll leave it running all night if I must and
hope it
> > > continues running.
> > >
> > > If you get a chance later this afternoon, please let me know how
far
> > > along the file is so I can estimate transfer speed for future
> reference.
> > >
> > > Thanks!
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, September 22, 2014 1:08 PM
> > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
> > >
> > > Ah, that's too bad.  I had been watching it slowly come in.
I'll
> > > keep an eye out for it.
> > >
> > > Julie
> > >
> > > On Mon, Sep 22, 2014 at 12:06 PM, SITTEL, MATTHEW C CTR USAF
AFWA 16
> > > WS/WXN via RT <met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
>
> > > >
> > > > Looks like I need to be more patient.  I assumed when I got
back
> > > > to work that the file transfer was hung so I aborted it.
Turns
> > > > out it has transferred 430 MB of the full 1.3 GB of the model
data
> file.
> > > > So looks like transfer will take 6-8 hours at this rate.
> > > >
> > > > With any luck it will run to completion... sorry for the
delay,
> > > > but I don't know any better way to get you the file short of
mailing
> a DVD!
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, September 22, 2014 10:37 AM
> > > > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > > > Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
> > > > Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat
issue
> > > >
> > > > Hi Matt.  I see that you are having problems running
point_stat.
> > > > Thank you for including the details of the situation.  Could
you
> > > > please follow these instructions for posting the forecast
file,
> > > > observation file, and config file to our ftp site:
> > > >    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> > > >
> > > > I can take a closer look and try to resolve the problem.
> > > >
> > > > Thanks,
> > > > Julie
> > > >
> > > > On Mon, Sep 22, 2014 at 9:07 AM, SITTEL, MATTHEW C CTR USAF
AFWA
> > > > 16 WS/WXN via RT <met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon Sep 22 09:07:19 2014: Request 69036 was acted upon.
> > > > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > > >        Queue: met_help
> > > > >      Subject: MET 4.1: Point_stat issue
> > > > >        Owner: Nobody
> > > > >   Requestors: matthew.sittel.ctr at us.af.mil
> > > > >       Status: new
> > > > >  Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036
> > > > > >
> > > > >
> > > > >
> > > > > Hello again,
> > > > >
> > > > > I'm trying to find a problem with my inability to run
point_stat
> > > > > for a pair of NetCDF files.  Here's what I am seeing from
MET:
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > > > > DEBUG 1: User Config File:
> > > > > /home/qcteam/METv4.1/data/aod/aod_point_stat_air.cfg
> > > > > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> > > > > new Met2dDataFile object of type "FileType_NcMet".
> > > > > NetCDF: Attribute not found
> > > > >
> > > > > Here's what I defined for the fields in the point_stat
config file:
> > > > >
> > > > > fcst = {
> > > > >    message_type = [ "ADPSFC" ];
> > > > >    field = [
> > > > >       {
> > > > >          name = "PM2_5_DRY(0,0,*,*)";
> > > > >          cat_thresh = [ >=0.0 ];
> > > > >       }
> > > > >    ];
> > > > > };
> > > > >
> > > > > obs = {
> > > > >    field = [
> > > > >       {
> > > > >          name = "obs_arr(*,4)";
> > > > >          cat_thresh = [ >=0.0 ];
> > > > >       }
> > > > >    ];
> > > > > };
> > > > >
> > > > > Within the forecast NetCDF file the following values are
set:
> > > > >
> > > > > dimensions:
> > > > >         Time = UNLIMITED ; // (1 currently)
> > > > >         west_east = 249 ;
> > > > >         south_north = 149 ;
> > > > >         bottom_top = 56 ;
> > > > >
> > > > >         float PM2_5_DRY(Time, bottom_top, south_north,
west_east) ;
> > > > >                 PM2_5_DRY:FieldType = 104 ;
> > > > >                 PM2_5_DRY:MemoryOrder = "XYZ" ;
> > > > >                 PM2_5_DRY:description = "pm2.5 aerosol dry
mass" ;
> > > > >                 PM2_5_DRY:units = "ug m^-3" ;
> > > > >                 PM2_5_DRY:stagger = "" ;
> > > > >                 PM2_5_DRY:coordinates = "XLONG XLAT" ;
> > > > >
> > > > > Within the observation file:
> > > > >
> > > > > dimensions:
> > > > >         obs_arr_len = 5 ;
> > > > >         nhdr = 1820 ;
> > > > >         nobs = UNLIMITED ; // (1820 currently)
> > > > >
> > > > >         float obs_arr(nobs, obs_arr_len) ;
> > > > >                 obs_arr:long_name = "array of observation
values" ;
> > > > >                 obs_arr:_fill_value = -9999.f ;
> > > > >                 obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> > > > >                 obs_arr:hdr_id_long_name = "index of
matching
> > > > > header
> > > > data"
> > > > > ;
> > > > >                 obs_arr:gc_long_name = "grib code
corresponding
> > > > > to the observation type" ;
> > > > >                 obs_arr:lvl_long_name = "pressure level
(hPa) or
> > > > > accumulation interval (sec)" ;
> > > > >                 obs_arr:hgt_long_name = "height in meters
above
> > > > > sea level or ground level (msl or agl)" ;
> > > > >                 obs_arr:ob_long_name = "observation value" ;
> > > > >
> > > > > Additional information:
> > > > >
> > > > > "ADPSFC" is the message type within the header of the
> > > > > observation files, but it does not appear in the forecast
file
> anywhere.
> > > > > I am using PM2_5_DRY(0,0,*,*) as the designation assuming 0
> > > > > represents the first member of an array and not 1.  The
second 0
> > > > > indicates the surface data should be used, the first of 56
levels.
> > > > > I used "*" to represent the X and Y nature of the grid.
> > > > > I am using obs_arr(*,4) because there are 5 data fields, and
> > > > > only the 5th one (numbered 4 when starting from 0) is the
actual
> > > > > data field of interest.  The first * indicates to use all
1820
> > observations.
> > > > > I built the observation file using 'ascii2nc'; the forecast
file
> > > > > was built separately by a colleague.
> > > > >
> > > > > The observation file is ~350KB so I could easily e-mail it,
but
> > > > > the model data is ~1.3GB, so obviously I can't easily send
that
> > > > > along for you to test with.  Perhaps I could FTP it
somewhere
> > > > > you could try to replicate the error?
> > > > >
> > > > > Do you see anything here that I've made a mistake with?
I've
> > > > > spent a couple hours staring at code and reading the user's
> > > > > manual; I can't find any guidance as to where I made the
mistake.
> > > > > I'm hoping you can
> > > > help.
> > > > >
> > > > > Thanks,
> > > > > Matt
> > > > >
> > > > > // SIGNED //
> > > > > Matthew C. Sittel
> > > > > Associate Scientist
> > > > > University Corporation for Atmospheric Research 16WS/WXN,
Offutt
> > > > > AFB, NE
> > > > > (402) 294-3473  DSN: 271-3473
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > National Center for Atmospheric Research Research Applications
> > > > Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Julie Prestopnik
> > > National Center for Atmospheric Research Research Applications
> > > Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>


--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Sep 24 06:46:14 2014

Hi Julie,

Do you know the answer to Steve's question?  He's running the model
but isn't sure about if p_interp will work.

Thanks,
Matt

-----Original Message-----
From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
Sent: Wednesday, September 24, 2014 7:40 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Subject: RE: Verification issues

Correct, the problem is currently that the p_intep function does not
work for chem that I am aware of. The data is on sigma levels, unless
Julie knows how to do it for the chem? Did she get the EXTCOF55
variable to output using the p_interp option?

Sorry if you look at the 00Z forecast we just need to verify against
the 24 hour forecast, for the MOZART I can't cycle that variable. So
for the 00Z forecast we can just verify against wrfout_d01_2014-08-
23_00:00:00 (the 24 hour forecast).

Steve

-----Original Message-----
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Sent: Wednesday, September 24, 2014 7:35 AM
To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
Subject: RE: Verification issues

MET can read NetCDF inputs but there is something about 'wrfout'
format that it won't accept... either the staggered grid as Julie
points out and/or being on sigma levels rather than pressure levels.

Julie successfully used p_interp to convert the wrfout data into
something MET could use.  I don't know if we have that already
installed on dev8... it's possible that we do.  If not we definitely
need to get that.

The model file in question is
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout_d01_2014-
08-22_00:00:00


------------------------------------------------
Subject: MET 4.1: Point_stat issue
From: John Halley Gotway
Time: Wed Sep 24 09:44:11 2014

Hello Matt,

This is John Halley Gotway - thought I'd jump in on this one.  Julie
and I
were able to run that NetCDF file you sent through the p_interp
utility and
we do see the EXTCOF55 variable in the output.  However the EXTCOF55
variable contains 0's in both the input to p_interp and the output.
I've
attached the NetCDF headers for the input to/output from p_interp.
The
output has "PLEV" in the name.

I've also attached the "namelist.pinterp" configuration file we used
in the
conversion.  We just used the default pressure levels - which begin at
1000
mb.

It may also be possible to post-process the WRF-Chem output with UPP.
I
haven't tried, so I really don't know.

Either option interpolates to pressure levels, but I'm wondering if
this is
sufficient for the type of analysis you want to do.  Since you've
assigned
the AIRNOW observations a message type of ADPSFC, I assume they're
defined
at the surface?  If so, the problem is that none of your forecast data
exists at the surface.  I really have no idea what the PM2_5_DRY
variable
and the AIRNOW observations contain.  I'd just suggest thinking about
whether the comparison of surface AIRNOW to 1000mb PM2_5_DRY makes
sense.

Thanks,
John


On Wed, Sep 24, 2014 at 6:46 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN
via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Do you know the answer to Steve's question?  He's running the model
but
> isn't sure about if p_interp will work.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Sent: Wednesday, September 24, 2014 7:40 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: RE: Verification issues
>
> Correct, the problem is currently that the p_intep function does not
work
> for chem that I am aware of. The data is on sigma levels, unless
Julie
> knows how to do it for the chem? Did she get the EXTCOF55 variable
to
> output using the p_interp option?
>
> Sorry if you look at the 00Z forecast we just need to verify against
the
> 24 hour forecast, for the MOZART I can't cycle that variable. So for
the
> 00Z forecast we can just verify against wrfout_d01_2014-08-
23_00:00:00 (the
> 24 hour forecast).
>
> Steve
>
> -----Original Message-----
> From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Sent: Wednesday, September 24, 2014 7:35 AM
> To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Subject: RE: Verification issues
>
> MET can read NetCDF inputs but there is something about 'wrfout'
format
> that it won't accept... either the staggered grid as Julie points
out
> and/or being on sigma levels rather than pressure levels.
>
> Julie successfully used p_interp to convert the wrfout data into
something
> MET could use.  I don't know if we have that already installed on
dev8...
> it's possible that we do.  If not we definitely need to get that.
>
> The model file in question is
>
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout_d01_2014-
08-22_00:00:00
>
>
>

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Wed Sep 24 10:24:09 2014

Hey John (and Julie),

Thanks for chiming in with your thoughts.  Turns out I got the
developer to totally overhaul the model output into GriB base on
Julie's observations, so I made many of the problems go away with the
conversion to pressure levels.

They've chosen to drop PM2_5_DRY altogether and stick with only
WXTCOF55, to be verified only at 1013 mb, so it's a surface field,
sort of... admittedly I don't know too much about the exact details of
what I've been asked to verify; I like to focus on getting the
specifics first and then worry about interpretation-if I have the
wrong kpds numbers from the get-go then nothing else will make sense
anyway.  Plus it forces the developer to think about what it is they
*really* want verified!

I get the impression PM2_5_DRY can't be interpolated correctly, or so
I was told.  Rather than press that issue with them, I'll just keep
quiet and be thankful they made my job easier (and yours, by
association)!

Thanks again.  John, have you been out here to Omaha recently?  I'm
long overdue for a visit to Boulder but I got yanked from the ensemble
verification stuff to focus exclusively on WRF-Chem this contract
year.  At least I can stay somewhat involved with MET this way.  Trust
that this was not my choice to be 'reassigned' and it hasn't gone over
well, but you know how it goes with the money... you do what you're
paid to do and just be glad you still have a job.

Matt

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, September 24, 2014 10:44 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Cc: CRAIG, ROBERT J GS-12 USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #69036] MET 4.1: Point_stat issue

Hello Matt,

This is John Halley Gotway - thought I'd jump in on this one.  Julie
and I were able to run that NetCDF file you sent through the p_interp
utility and we do see the EXTCOF55 variable in the output.  However
the EXTCOF55 variable contains 0's in both the input to p_interp and
the output.  I've attached the NetCDF headers for the input to/output
from p_interp.  The output has "PLEV" in the name.

I've also attached the "namelist.pinterp" configuration file we used
in the conversion.  We just used the default pressure levels - which
begin at 1000 mb.

It may also be possible to post-process the WRF-Chem output with UPP.
I haven't tried, so I really don't know.

Either option interpolates to pressure levels, but I'm wondering if
this is sufficient for the type of analysis you want to do.  Since
you've assigned the AIRNOW observations a message type of ADPSFC, I
assume they're defined at the surface?  If so, the problem is that
none of your forecast data exists at the surface.  I really have no
idea what the PM2_5_DRY variable and the AIRNOW observations contain.
I'd just suggest thinking about whether the comparison of surface
AIRNOW to 1000mb PM2_5_DRY makes sense.

Thanks,
John


On Wed, Sep 24, 2014 at 6:46 AM, SITTEL, MATTHEW C CTR USAF AFWA 16
WS/WXN via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69036 >
>
> Hi Julie,
>
> Do you know the answer to Steve's question?  He's running the model
> but isn't sure about if p_interp will work.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Sent: Wednesday, September 24, 2014 7:40 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: RE: Verification issues
>
> Correct, the problem is currently that the p_intep function does not
> work for chem that I am aware of. The data is on sigma levels,
unless
> Julie knows how to do it for the chem? Did she get the EXTCOF55
> variable to output using the p_interp option?
>
> Sorry if you look at the 00Z forecast we just need to verify against
> the
> 24 hour forecast, for the MOZART I can't cycle that variable. So for
> the 00Z forecast we can just verify against
> wrfout_d01_2014-08-23_00:00:00 (the
> 24 hour forecast).
>
> Steve
>
> -----Original Message-----
> From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Sent: Wednesday, September 24, 2014 7:35 AM
> To: BLIUJUS, STEVEN D CTR USAF AFWA 16 WS/WXE
> Subject: RE: Verification issues
>
> MET can read NetCDF inputs but there is something about 'wrfout'
> format that it won't accept... either the staggered grid as Julie
> points out and/or being on sigma levels rather than pressure levels.
>
> Julie successfully used p_interp to convert the wrfout data into
> something MET could use.  I don't know if we have that already
installed on dev8...
> it's possible that we do.  If not we definitely need to get that.
>
> The model file in question is
>
/home/bliujuss/WRFV3.6.1/WRFV3/test/em_real/MOZCART/20140822_00/wrfout
> _d01_2014-08-22_00:00:00
>
>
>



------------------------------------------------