[Met_help] [rt.rap.ucar.edu #92985] History for segfault when running point stat

Fri Jan 17 14:31:13 MST 2020

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Good Morning,

     I am trying to run point_stat.  I have the following.
     FV3file.nc4  - a file from running the FV3 model with only one field
                           snow depth.
     oct2018_met.ascii  - an ascii file with station information, lat, lon
and
                                      snowd values.
     oct2018_met.nc     - netcdf file that I got from running ascii2nc on
the
                                      oct2018_met.ascii file.

      PointStatConfig.sherrie - mypoint stat config file.  I am not sure of
what
       I should have in the config file.
       In the regrid option I have tried
       to_grid = OBS
       and to_grid =

       I am also not quite sure of what I should have in the obs dictionary.

       I have placed all of the above files on the rap ftp
site /incoming/irap/met_help/sherrie_data/

       Thanks for your help again,
       Sherrie Fredrick

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: segfault when running point stat
From: Julie Prestopnik
Time: Wed Nov 06 14:54:20 2019

Hi Sherrie.  I have assigned this ticket to John, who can better
assist
you.  He will follow up as soon as he is able to.

Thanks,
Julie

On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> Transaction: Ticket created by sherrie.fredrick at noaa.gov
>        Queue: met_help
>      Subject: segfault when running point stat
>        Owner: Nobody
>   Requestors: sherrie.fredrick at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>
>
> Good Morning,
>
>      I am trying to run point_stat.  I have the following.
>      FV3file.nc4  - a file from running the FV3 model with only one
field
>                            snow depth.
>      oct2018_met.ascii  - an ascii file with station information,
lat, lon
> and
>                                       snowd values.
>      oct2018_met.nc     - netcdf file that I got from running
ascii2nc on
> the
>                                       oct2018_met.ascii file.
>
>       PointStatConfig.sherrie - mypoint stat config file.  I am not
sure of
> what
>        I should have in the config file.
>        In the regrid option I have tried
>        to_grid = OBS
>        and to_grid =
>
>        I am also not quite sure of what I should have in the obs
> dictionary.
>
>        I have placed all of the above files on the rap ftp
> site /incoming/irap/met_help/sherrie_data/
>
>        Thanks for your help again,
>        Sherrie Fredrick
>
>

--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: segfault when running point stat
From: Sherrie Fredrick - NOAA Affiliate
Time: Wed Nov 06 15:34:53 2019

Hi Julie,

     Thanks, John has helped quite a bit with other problems.  Have a
good
evening and
stay warm.

      Sherrie

On Wed, Nov 6, 2019 at 2:54 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Hi Sherrie.  I have assigned this ticket to John, who can better
assist
> you.  He will follow up as soon as he is able to.
>
> Thanks,
> Julie
>
> On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> > Transaction: Ticket created by sherrie.fredrick at noaa.gov
> >        Queue: met_help
> >      Subject: segfault when running point stat
> >        Owner: Nobody
> >   Requestors: sherrie.fredrick at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> >
> >
> > Good Morning,
> >
> >      I am trying to run point_stat.  I have the following.
> >      FV3file.nc4  - a file from running the FV3 model with only
one field
> >                            snow depth.
> >      oct2018_met.ascii  - an ascii file with station information,
lat,
> lon
> > and
> >                                       snowd values.
> >      oct2018_met.nc     - netcdf file that I got from running
ascii2nc
> on
> > the
> >                                       oct2018_met.ascii file.
> >
> >       PointStatConfig.sherrie - mypoint stat config file.  I am
not sure
> of
> > what
> >        I should have in the config file.
> >        In the regrid option I have tried
> >        to_grid = OBS
> >        and to_grid =
> >
> >        I am also not quite sure of what I should have in the obs
> > dictionary.
> >
> >        I have placed all of the above files on the rap ftp
> > site /incoming/irap/met_help/sherrie_data/
> >
> >        Thanks for your help again,
> >        Sherrie Fredrick
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day.  Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: segfault when running point stat
From: John Halley Gotway
Time: Wed Nov 06 15:50:45 2019

Sherrie,

Whenever getting started with new datasets, I always start by running
the
plot_data_plane tool.  And I run plot_point_obs for point
observations.  To
that end, I ran the following commands which produced that attached
images.  And this data looks fine to me:

*/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
\FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
file_type=NETCDF_NCCF;'convert -rotate 90 -background white -flatten
FV3file.ps
FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
\oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
<http://oct2018_met.ps>convert -rotate 90 -background white -flatten
oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*

But when I run Point-Stat, I get a segfault just like you did!
Segfaults
are never good, so we definitely need to fix that!  But let's figure
out
why.  I found a problem in your Point-Stat config file.  You're
requesting
data for:

*   name  = "snodsfc";    level = ["(0,*,*)"];*

But there are 2 problems here.  The FV3file.nc contains a variable
named
"snowd" and not "snodsfc" and it has 2 dimensions, not 3.  So I
switched to
using:

*   name  = "snowd";    level = ["(*,*)"];*

That gets past the segfault, but we now have this error:

*WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> Not
found
time out of 1. ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo
&) ->
variable "snowd" has unknown error (999).*

So there's a problem with the timing info.  Looking in the
FV3file.nc4, I
see that you have a time dimension defined, but it is not included as
a
dimension for the data variable: *float snowd(lat, lon) ;*

MET is definitely expecting the time variable to be included.  I used
ncdump and ncgen to modify your file to add the time dimension:

*ncdump FV3file.nc4 > FV3file.nc4dump*
*# Edit FV3file.nc4dump file by adding time dim to snowd variable*

*ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
And then I modified your config file and reran Point-Stat using the
updated
file:

*   name  = "snowd";    level = ["(0,*,*)"];*

*/d1/johnhg/MET/MET_development/git/MET-master_v8.1/met/bin/point_stat
\*
*FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*

That ran but gave me 0 matched pairs because of a mismatch of "obs
type":
DEBUG 3: Observations processed   = 47880
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: obs type       = 47880

I used ncdump to figure out that your observations are named "snowd":
ncdump -v obs_var oct2018_met.nc
 obs_var =
 "snowd",
 "66" ;

So I modified your config file to request "snowd" observations be
used:

*        //name  = "SNOD";        name  = "snowd";*

And I finally got 47,415 matched pairs:
DEBUG 3: Number of matched pairs  = 47415
DEBUG 3: Observations processed   = 47880
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: obs type       = 465

But what about those other 465 ones?  Having obs_var = "66" looks
suspicious to me.  So I looked upstream as the ascii file you sent.  I
contains 47,880 lines:
cat oct2018_met.ascii | wc -l
47880

For some reason, this data files looks to be a little bit corrupted.
In
the second column, where there should be station names, I see the
"snowd"
string getting somewhat randomly embedded in there: "GME001219snowd".
And
that happens in other columns too.  In fact, you'd expect "snowd" to
show
up 47,880 times (once per line).  But it appears in this file 55,896
times.  My guess is that this is the result of some over-aggressive
use of
"sed".  So I'd suggest taking a look at that.

Hope that helps.

Thanks,
John

On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> Transaction: Ticket created by sherrie.fredrick at noaa.gov
>        Queue: met_help
>      Subject: segfault when running point stat
>        Owner: Nobody
>   Requestors: sherrie.fredrick at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>
>
> Good Morning,
>
>      I am trying to run point_stat.  I have the following.
>      FV3file.nc4  - a file from running the FV3 model with only one
field
>                            snow depth.
>      oct2018_met.ascii  - an ascii file with station information,
lat, lon
> and
>                                       snowd values.
>      oct2018_met.nc     - netcdf file that I got from running
ascii2nc on
> the
>                                       oct2018_met.ascii file.
>
>       PointStatConfig.sherrie - mypoint stat config file.  I am not
sure of
> what
>        I should have in the config file.
>        In the regrid option I have tried
>        to_grid = OBS
>        and to_grid =
>
>        I am also not quite sure of what I should have in the obs
> dictionary.
>
>        I have placed all of the above files on the rap ftp
> site /incoming/irap/met_help/sherrie_data/
>
>        Thanks for your help again,
>        Sherrie Fredrick
>
>

------------------------------------------------
Subject: segfault when running point stat
From: John Halley Gotway
Time: Wed Nov 06 16:26:20 2019

Sherrie,

FYI, I was able to find the line that produced the segfault, and I
updated
the development version of the code.  In future versions of MET,
you'll
this error message instead of the code segfaulting:

ERROR  :
ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find
requested variable name "snodsfc" in NetCDF file "FV3file.nc4".
ERROR  :

Please let us know if you ever run into another segfault.  They always
need
to be fixed and replaced with a useful error message.

Thanks,
John

On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway <johnhg at ucar.edu>
wrote:

> Sherrie,
>
> Whenever getting started with new datasets, I always start by
running the
> plot_data_plane tool.  And I run plot_point_obs for point
observations.  To
> that end, I ran the following commands which produced that attached
> images.  And this data looks fine to me:
>
>
>
>
>
>
>
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> file_type=NETCDF_NCCF;'convert -rotate 90 -background white -flatten
> FV3file.ps
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> <http://oct2018_met.ps>convert -rotate 90 -background white -flatten
> oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
>
> But when I run Point-Stat, I get a segfault just like you did!
Segfaults
> are never good, so we definitely need to fix that!  But let's figure
out
> why.  I found a problem in your Point-Stat config file.  You're
requesting
> data for:
>
> *   name  = "snodsfc";    level = ["(0,*,*)"];*
>
> But there are 2 problems here.  The FV3file.nc contains a variable
named
> "snowd" and not "snodsfc" and it has 2 dimensions, not 3.  So I
switched to
> using:
>
> *   name  = "snowd";    level = ["(*,*)"];*
>
> That gets past the segfault, but we now have this error:
>
> *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> Not
found
> time out of 1. ERROR  :
MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
> variable "snowd" has unknown error (999).*
>
> So there's a problem with the timing info.  Looking in the
FV3file.nc4, I
> see that you have a time dimension defined, but it is not included
as a
> dimension for the data variable: *float snowd(lat, lon) ;*
>
> MET is definitely expecting the time variable to be included.  I
used
> ncdump and ncgen to modify your file to add the time dimension:
>
>
> *ncdump FV3file.nc4 > FV3file.nc4dump*
> *# Edit FV3file.nc4dump file by adding time dim to snowd variable*
>
> *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> And then I modified your config file and reran Point-Stat using the
> updated file:
>
> *   name  = "snowd";    level = ["(0,*,*)"];*
>
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \*
> *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*
>
> That ran but gave me 0 matched pairs because of a mismatch of "obs
type":
> DEBUG 3: Observations processed   = 47880
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 47880
>
> I used ncdump to figure out that your observations are named
"snowd":
> ncdump -v obs_var oct2018_met.nc
>  obs_var =
>  "snowd",
>  "66" ;
>
> So I modified your config file to request "snowd" observations be
used:
>
> *        //name  = "SNOD";        name  = "snowd";*
>
> And I finally got 47,415 matched pairs:
> DEBUG 3: Number of matched pairs  = 47415
> DEBUG 3: Observations processed   = 47880
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 465
>
> But what about those other 465 ones?  Having obs_var = "66" looks
> suspicious to me.  So I looked upstream as the ascii file you sent.
I
> contains 47,880 lines:
> cat oct2018_met.ascii | wc -l
> 47880
>
> For some reason, this data files looks to be a little bit corrupted.
In
> the second column, where there should be station names, I see the
"snowd"
> string getting somewhat randomly embedded in there:
"GME001219snowd".
> And that happens in other columns too.  In fact, you'd expect
"snowd" to
> show up 47,880 times (once per line).  But it appears in this file
55,896
> times.  My guess is that this is the result of some over-aggressive
use of
> "sed".  So I'd suggest taking a look at that.
>
> Hope that helps.
>
> Thanks,
> John
>
>
> On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
>>
>> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
>> Transaction: Ticket created by sherrie.fredrick at noaa.gov
>>        Queue: met_help
>>      Subject: segfault when running point stat
>>        Owner: Nobody
>>   Requestors: sherrie.fredrick at noaa.gov
>>       Status: new
>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>>
>>
>> Good Morning,
>>
>>      I am trying to run point_stat.  I have the following.
>>      FV3file.nc4  - a file from running the FV3 model with only one
field
>>                            snow depth.
>>      oct2018_met.ascii  - an ascii file with station information,
lat, lon
>> and
>>                                       snowd values.
>>      oct2018_met.nc     - netcdf file that I got from running
ascii2nc on
>> the
>>                                       oct2018_met.ascii file.
>>
>>       PointStatConfig.sherrie - mypoint stat config file.  I am not
sure
>> of
>> what
>>        I should have in the config file.
>>        In the regrid option I have tried
>>        to_grid = OBS
>>        and to_grid =
>>
>>        I am also not quite sure of what I should have in the obs
>> dictionary.
>>
>>        I have placed all of the above files on the rap ftp
>> site /incoming/irap/met_help/sherrie_data/
>>
>>        Thanks for your help again,
>>        Sherrie Fredrick
>>
>>

------------------------------------------------
Subject: segfault when running point stat
From: Sherrie Fredrick - NOAA Affiliate
Time: Thu Nov 07 08:39:25 2019

Hi John,

     Here are the other things I have tried.
    fcst = {
   message_type = [ "ADPSFC" ];
   level = ["(0,*,*)"];
   field = [
        {
        name  = "snowd";
       }
   ];
}

fcst = {
   message_type = [ "ADPSFC" ];

   field = [
        {
        name  = "snowd";   level = "(0,*,*)";
       }
   ];
}

They all give me the same error.
VarInfo::set_level_info_grib() - failed to parse level string
'(0,*,*)'

Thanks,
Sherrie

On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Sherrie,
>
> FYI, I was able to find the line that produced the segfault, and I
updated
> the development version of the code.  In future versions of MET,
you'll
> this error message instead of the code segfaulting:
>
> ERROR  :
> ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find
> requested variable name "snodsfc" in NetCDF file "FV3file.nc4".
> ERROR  :
>
> Please let us know if you ever run into another segfault.  They
always need
> to be fixed and replaced with a useful error message.
>
> Thanks,
> John
>
> On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> > Sherrie,
> >
> > Whenever getting started with new datasets, I always start by
running the
> > plot_data_plane tool.  And I run plot_point_obs for point
observations.
> To
> > that end, I ran the following commands which produced that
attached
> > images.  And this data looks fine to me:
> >
> >
> >
> >
> >
> >
> >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > file_type=NETCDF_NCCF;'convert -rotate 90 -background white
-flatten
> > FV3file.ps
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > <http://oct2018_met.ps>convert -rotate 90 -background white
-flatten
> > oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
> >
> > But when I run Point-Stat, I get a segfault just like you did!
Segfaults
> > are never good, so we definitely need to fix that!  But let's
figure out
> > why.  I found a problem in your Point-Stat config file.  You're
> requesting
> > data for:
> >
> > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> >
> > But there are 2 problems here.  The FV3file.nc contains a variable
named
> > "snowd" and not "snodsfc" and it has 2 dimensions, not 3.  So I
switched
> to
> > using:
> >
> > *   name  = "snowd";    level = ["(*,*)"];*
> >
> > That gets past the segfault, but we now have this error:
> >
> > *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) -> Not
found
> > time out of 1. ERROR  :
MetNcCFDataFile::collect_time_offsets(VarInfo &)
> ->
> > variable "snowd" has unknown error (999).*
> >
> > So there's a problem with the timing info.  Looking in the
FV3file.nc4, I
> > see that you have a time dimension defined, but it is not included
as a
> > dimension for the data variable: *float snowd(lat, lon) ;*
> >
> > MET is definitely expecting the time variable to be included.  I
used
> > ncdump and ncgen to modify your file to add the time dimension:
> >
> >
> > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > *# Edit FV3file.nc4dump file by adding time dim to snowd variable*
> >
> > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > And then I modified your config file and reran Point-Stat using
the
> > updated file:
> >
> > *   name  = "snowd";    level = ["(0,*,*)"];*
> >
> > */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \*
> > *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> > PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*
> >
> > That ran but gave me 0 matched pairs because of a mismatch of "obs
type":
> > DEBUG 3: Observations processed   = 47880
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 47880
> >
> > I used ncdump to figure out that your observations are named
"snowd":
> > ncdump -v obs_var oct2018_met.nc
> >  obs_var =
> >  "snowd",
> >  "66" ;
> >
> > So I modified your config file to request "snowd" observations be
used:
> >
> > *        //name  = "SNOD";        name  = "snowd";*
> >
> > And I finally got 47,415 matched pairs:
> > DEBUG 3: Number of matched pairs  = 47415
> > DEBUG 3: Observations processed   = 47880
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 465
> >
> > But what about those other 465 ones?  Having obs_var = "66" looks
> > suspicious to me.  So I looked upstream as the ascii file you
sent.  I
> > contains 47,880 lines:
> > cat oct2018_met.ascii | wc -l
> > 47880
> >
> > For some reason, this data files looks to be a little bit
corrupted.  In
> > the second column, where there should be station names, I see the
"snowd"
> > string getting somewhat randomly embedded in there:
"GME001219snowd".
> > And that happens in other columns too.  In fact, you'd expect
"snowd" to
> > show up 47,880 times (once per line).  But it appears in this file
55,896
> > times.  My guess is that this is the result of some over-
aggressive use
> of
> > "sed".  So I'd suggest taking a look at that.
> >
> > Hope that helps.
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> >> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> >>        Queue: met_help
> >>      Subject: segfault when running point stat
> >>        Owner: Nobody
> >>   Requestors: sherrie.fredrick at noaa.gov
> >>       Status: new
> >>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> >>
> >>
> >> Good Morning,
> >>
> >>      I am trying to run point_stat.  I have the following.
> >>      FV3file.nc4  - a file from running the FV3 model with only
one
> field
> >>                            snow depth.
> >>      oct2018_met.ascii  - an ascii file with station information,
lat,
> lon
> >> and
> >>                                       snowd values.
> >>      oct2018_met.nc     - netcdf file that I got from running
ascii2nc
> on
> >> the
> >>                                       oct2018_met.ascii file.
> >>
> >>       PointStatConfig.sherrie - mypoint stat config file.  I am
not sure
> >> of
> >> what
> >>        I should have in the config file.
> >>        In the regrid option I have tried
> >>        to_grid = OBS
> >>        and to_grid =
> >>
> >>        I am also not quite sure of what I should have in the obs
> >> dictionary.
> >>
> >>        I have placed all of the above files on the rap ftp
> >> site /incoming/irap/met_help/sherrie_data/
> >>
> >>        Thanks for your help again,
> >>        Sherrie Fredrick
> >>
> >>
>
>

------------------------------------------------
Subject: segfault when running point stat
From: John Halley Gotway
Time: Thu Nov 07 09:50:17 2019

Sherrie,

I've attached the Point-Stat config file I used.  And here's how I
called
it on the command line:

/d1/johnhg/MET/MET_development/git/MET-master_v8.1/met/bin/point_stat
\
FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie -outdir out -v
3
-log run_ps.log

Perhaps the problem is how you set up the "obs" settings, not the
"fcst"
ones.  Here's how those are set in the attached config file:

fcst = {
  file_type = NETCDF_NCCF;
  field = [
     {
       name  = "snowd";
       level = ["(0,*,*)"];
     }
  ];
}
obs = {
  field = [
     {
       message_type = [ "ADPSFC" ];
       name  = "snowd";
       level = "L0";
     }
  ];
}

Thanks,
John

On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>
> Hi John,
>
>      Here are the other things I have tried.
>     fcst = {
>    message_type = [ "ADPSFC" ];
>    level = ["(0,*,*)"];
>    field = [
>         {
>         name  = "snowd";
>        }
>    ];
> }
>
> fcst = {
>    message_type = [ "ADPSFC" ];
>
>    field = [
>         {
>         name  = "snowd";   level = "(0,*,*)";
>        }
>    ];
> }
>
> They all give me the same error.
> VarInfo::set_level_info_grib() - failed to parse level string
'(0,*,*)'
>
> Thanks,
> Sherrie
>
>
> On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > FYI, I was able to find the line that produced the segfault, and I
> updated
> > the development version of the code.  In future versions of MET,
you'll
> > this error message instead of the code segfaulting:
> >
> > ERROR  :
> > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &) -> can't
find
> > requested variable name "snodsfc" in NetCDF file "FV3file.nc4".
> > ERROR  :
> >
> > Please let us know if you ever run into another segfault.  They
always
> need
> > to be fixed and replaced with a useful error message.
> >
> > Thanks,
> > John
> >
> > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
> >
> > > Sherrie,
> > >
> > > Whenever getting started with new datasets, I always start by
running
> the
> > > plot_data_plane tool.  And I run plot_point_obs for point
observations.
> > To
> > > that end, I ran the following commands which produced that
attached
> > > images.  And this data looks fine to me:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > > file_type=NETCDF_NCCF;'convert -rotate 90 -background white
-flatten
> > > FV3file.ps
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > > <http://oct2018_met.ps>convert -rotate 90 -background white
-flatten
> > > oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
> > >
> > > But when I run Point-Stat, I get a segfault just like you did!
> Segfaults
> > > are never good, so we definitely need to fix that!  But let's
figure
> out
> > > why.  I found a problem in your Point-Stat config file.  You're
> > requesting
> > > data for:
> > >
> > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > >
> > > But there are 2 problems here.  The FV3file.nc contains a
variable
> named
> > > "snowd" and not "snodsfc" and it has 2 dimensions, not 3.  So I
> switched
> > to
> > > using:
> > >
> > > *   name  = "snowd";    level = ["(*,*)"];*
> > >
> > > That gets past the segfault, but we now have this error:
> > >
> > > *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
Not found
> > > time out of 1. ERROR  :
MetNcCFDataFile::collect_time_offsets(VarInfo
> &)
> > ->
> > > variable "snowd" has unknown error (999).*
> > >
> > > So there's a problem with the timing info.  Looking in the
> FV3file.nc4, I
> > > see that you have a time dimension defined, but it is not
included as a
> > > dimension for the data variable: *float snowd(lat, lon) ;*
> > >
> > > MET is definitely expecting the time variable to be included.  I
used
> > > ncdump and ncgen to modify your file to add the time dimension:
> > >
> > >
> > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > *# Edit FV3file.nc4dump file by adding time dim to snowd
variable*
> > >
> > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > And then I modified your config file and reran Point-Stat using
the
> > > updated file:
> > >
> > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > >
> > > */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> \*
> > > *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> > > PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*
> > >
> > > That ran but gave me 0 matched pairs because of a mismatch of
"obs
> type":
> > > DEBUG 3: Observations processed   = 47880
> > > DEBUG 3: Rejected: SID exclusion  = 0
> > > DEBUG 3: Rejected: obs type       = 47880
> > >
> > > I used ncdump to figure out that your observations are named
"snowd":
> > > ncdump -v obs_var oct2018_met.nc
> > >  obs_var =
> > >  "snowd",
> > >  "66" ;
> > >
> > > So I modified your config file to request "snowd" observations
be used:
> > >
> > > *        //name  = "SNOD";        name  = "snowd";*
> > >
> > > And I finally got 47,415 matched pairs:
> > > DEBUG 3: Number of matched pairs  = 47415
> > > DEBUG 3: Observations processed   = 47880
> > > DEBUG 3: Rejected: SID exclusion  = 0
> > > DEBUG 3: Rejected: obs type       = 465
> > >
> > > But what about those other 465 ones?  Having obs_var = "66"
looks
> > > suspicious to me.  So I looked upstream as the ascii file you
sent.  I
> > > contains 47,880 lines:
> > > cat oct2018_met.ascii | wc -l
> > > 47880
> > >
> > > For some reason, this data files looks to be a little bit
corrupted.
> In
> > > the second column, where there should be station names, I see
the
> "snowd"
> > > string getting somewhat randomly embedded in there:
"GME001219snowd".
> > > And that happens in other columns too.  In fact, you'd expect
"snowd"
> to
> > > show up 47,880 times (once per line).  But it appears in this
file
> 55,896
> > > times.  My guess is that this is the result of some over-
aggressive use
> > of
> > > "sed".  So I'd suggest taking a look at that.
> > >
> > > Hope that helps.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA
Affiliate via
> RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> > >> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > >>        Queue: met_help
> > >>      Subject: segfault when running point stat
> > >>        Owner: Nobody
> > >>   Requestors: sherrie.fredrick at noaa.gov
> > >>       Status: new
> > >>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> >
> > >>
> > >>
> > >> Good Morning,
> > >>
> > >>      I am trying to run point_stat.  I have the following.
> > >>      FV3file.nc4  - a file from running the FV3 model with only
one
> > field
> > >>                            snow depth.
> > >>      oct2018_met.ascii  - an ascii file with station
information, lat,
> > lon
> > >> and
> > >>                                       snowd values.
> > >>      oct2018_met.nc     - netcdf file that I got from running
> ascii2nc
> > on
> > >> the
> > >>                                       oct2018_met.ascii file.
> > >>
> > >>       PointStatConfig.sherrie - mypoint stat config file.  I am
not
> sure
> > >> of
> > >> what
> > >>        I should have in the config file.
> > >>        In the regrid option I have tried
> > >>        to_grid = OBS
> > >>        and to_grid =
> > >>
> > >>        I am also not quite sure of what I should have in the
obs
> > >> dictionary.
> > >>
> > >>        I have placed all of the above files on the rap ftp
> > >> site /incoming/irap/met_help/sherrie_data/
> > >>
> > >>        Thanks for your help again,
> > >>        Sherrie Fredrick
> > >>
> > >>
> >
> >
>
>

------------------------------------------------
Subject: segfault when running point stat
From: Sherrie Fredrick - NOAA Affiliate
Time: Thu Nov 07 10:42:43 2019

Hi John,
     That worked the first time.  I spent hours trying to get
something to
work.  So thank
you very much.  I was not sure of what to put in for the obs
dictionary
because the Netcdf
file produced by the ascii2nc did not really have a variable called
snow it
just had float obs_val(nobs) ;

    I know that both of these observations that you have helped me
with
will be used a lot.
    Thanks,
    Sherrie

On Thu, Nov 7, 2019 at 9:50 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Sherrie,
>
> I've attached the Point-Stat config file I used.  And here's how I
called
> it on the command line:
>
> /d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \
> FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie -outdir out
-v 3
> -log run_ps.log
>
> Perhaps the problem is how you set up the "obs" settings, not the
"fcst"
> ones.  Here's how those are set in the attached config file:
>
> fcst = {
>   file_type = NETCDF_NCCF;
>   field = [
>      {
>        name  = "snowd";
>        level = ["(0,*,*)"];
>      }
>   ];
> }
> obs = {
>   field = [
>      {
>        message_type = [ "ADPSFC" ];
>        name  = "snowd";
>        level = "L0";
>      }
>   ];
> }
>
> Thanks,
> John
>
> On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> >
> > Hi John,
> >
> >      Here are the other things I have tried.
> >     fcst = {
> >    message_type = [ "ADPSFC" ];
> >    level = ["(0,*,*)"];
> >    field = [
> >         {
> >         name  = "snowd";
> >        }
> >    ];
> > }
> >
> > fcst = {
> >    message_type = [ "ADPSFC" ];
> >
> >    field = [
> >         {
> >         name  = "snowd";   level = "(0,*,*)";
> >        }
> >    ];
> > }
> >
> > They all give me the same error.
> > VarInfo::set_level_info_grib() - failed to parse level string
'(0,*,*)'
> >
> > Thanks,
> > Sherrie
> >
> >
> > On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Sherrie,
> > >
> > > FYI, I was able to find the line that produced the segfault, and
I
> > updated
> > > the development version of the code.  In future versions of MET,
you'll
> > > this error message instead of the code segfaulting:
> > >
> > > ERROR  :
> > > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
can't find
> > > requested variable name "snodsfc" in NetCDF file "FV3file.nc4".
> > > ERROR  :
> > >
> > > Please let us know if you ever run into another segfault.  They
always
> > need
> > > to be fixed and replaced with a useful error message.
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> > >
> > > > Sherrie,
> > > >
> > > > Whenever getting started with new datasets, I always start by
running
> > the
> > > > plot_data_plane tool.  And I run plot_point_obs for point
> observations.
> > > To
> > > > that end, I ran the following commands which produced that
attached
> > > > images.  And this data looks fine to me:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > > > file_type=NETCDF_NCCF;'convert -rotate 90 -background white
-flatten
> > > > FV3file.ps
> > > >
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > > > <http://oct2018_met.ps>convert -rotate 90 -background white
-flatten
> > > > oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
> > > >
> > > > But when I run Point-Stat, I get a segfault just like you did!
> > Segfaults
> > > > are never good, so we definitely need to fix that!  But let's
figure
> > out
> > > > why.  I found a problem in your Point-Stat config file.
You're
> > > requesting
> > > > data for:
> > > >
> > > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > > >
> > > > But there are 2 problems here.  The FV3file.nc contains a
variable
> > named
> > > > "snowd" and not "snodsfc" and it has 2 dimensions, not 3.  So
I
> > switched
> > > to
> > > > using:
> > > >
> > > > *   name  = "snowd";    level = ["(*,*)"];*
> > > >
> > > > That gets past the segfault, but we now have this error:
> > > >
> > > > *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
Not
> found
> > > > time out of 1. ERROR  :
MetNcCFDataFile::collect_time_offsets(VarInfo
> > &)
> > > ->
> > > > variable "snowd" has unknown error (999).*
> > > >
> > > > So there's a problem with the timing info.  Looking in the
> > FV3file.nc4, I
> > > > see that you have a time dimension defined, but it is not
included
> as a
> > > > dimension for the data variable: *float snowd(lat, lon) ;*
> > > >
> > > > MET is definitely expecting the time variable to be included.
I used
> > > > ncdump and ncgen to modify your file to add the time
dimension:
> > > >
> > > >
> > > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > > *# Edit FV3file.nc4dump file by adding time dim to snowd
variable*
> > > >
> > > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > > And then I modified your config file and reran Point-Stat
using the
> > > > updated file:
> > > >
> > > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > > >
> > > >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> > \*
> > > > *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> > > > PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*
> > > >
> > > > That ran but gave me 0 matched pairs because of a mismatch of
"obs
> > type":
> > > > DEBUG 3: Observations processed   = 47880
> > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > DEBUG 3: Rejected: obs type       = 47880
> > > >
> > > > I used ncdump to figure out that your observations are named
"snowd":
> > > > ncdump -v obs_var oct2018_met.nc
> > > >  obs_var =
> > > >  "snowd",
> > > >  "66" ;
> > > >
> > > > So I modified your config file to request "snowd" observations
be
> used:
> > > >
> > > > *        //name  = "SNOD";        name  = "snowd";*
> > > >
> > > > And I finally got 47,415 matched pairs:
> > > > DEBUG 3: Number of matched pairs  = 47415
> > > > DEBUG 3: Observations processed   = 47880
> > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > DEBUG 3: Rejected: obs type       = 465
> > > >
> > > > But what about those other 465 ones?  Having obs_var = "66"
looks
> > > > suspicious to me.  So I looked upstream as the ascii file you
sent.
> I
> > > > contains 47,880 lines:
> > > > cat oct2018_met.ascii | wc -l
> > > > 47880
> > > >
> > > > For some reason, this data files looks to be a little bit
corrupted.
> > In
> > > > the second column, where there should be station names, I see
the
> > "snowd"
> > > > string getting somewhat randomly embedded in there:
"GME001219snowd".
> > > > And that happens in other columns too.  In fact, you'd expect
"snowd"
> > to
> > > > show up 47,880 times (once per line).  But it appears in this
file
> > 55,896
> > > > times.  My guess is that this is the result of some over-
aggressive
> use
> > > of
> > > > "sed".  So I'd suggest taking a look at that.
> > > >
> > > > Hope that helps.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA
Affiliate via
> > RT
> > > <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> > > >> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > >>        Queue: met_help
> > > >>      Subject: segfault when running point stat
> > > >>        Owner: Nobody
> > > >>   Requestors: sherrie.fredrick at noaa.gov
> > > >>       Status: new
> > > >>  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> > >
> > > >>
> > > >>
> > > >> Good Morning,
> > > >>
> > > >>      I am trying to run point_stat.  I have the following.
> > > >>      FV3file.nc4  - a file from running the FV3 model with
only one
> > > field
> > > >>                            snow depth.
> > > >>      oct2018_met.ascii  - an ascii file with station
information,
> lat,
> > > lon
> > > >> and
> > > >>                                       snowd values.
> > > >>      oct2018_met.nc     - netcdf file that I got from running
> > ascii2nc
> > > on
> > > >> the
> > > >>                                       oct2018_met.ascii file.
> > > >>
> > > >>       PointStatConfig.sherrie - mypoint stat config file.  I
am not
> > sure
> > > >> of
> > > >> what
> > > >>        I should have in the config file.
> > > >>        In the regrid option I have tried
> > > >>        to_grid = OBS
> > > >>        and to_grid =
> > > >>
> > > >>        I am also not quite sure of what I should have in the
obs
> > > >> dictionary.
> > > >>
> > > >>        I have placed all of the above files on the rap ftp
> > > >> site /incoming/irap/met_help/sherrie_data/
> > > >>
> > > >>        Thanks for your help again,
> > > >>        Sherrie Fredrick
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: segfault when running point stat
From: John Halley Gotway
Time: Thu Nov 07 10:53:42 2019

Sherrie,

Sorry for the confusion.  There is a somewhat subtle detail here.

For gridded forecast files in NetCDF format, you set name = "NetCDF
Variable Name"; and level = "Describe indices of that variable".

For point observation files (NetCDF output of pb2nc, ascii2nc, and
madis2nc), you set name = "observation variable strings" which are
listed
in the "obs_var" variable in that file.  For example, here variable
names
present in some NetCDF output of pb2nc:

ncdump -v obs_var gdas1.20120409.t12z.prepbufr.nc
...
data:

 obs_var =
  "SPFH",
  "TMP",
  "HGT",
  "UGRD",
  "VGRD",
  "DPT",
  "WIND",
  "RH",
  "MIXR" ;
}

Make sense?

Thanks,
John

On Thu, Nov 7, 2019 at 10:43 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>
> Hi John,
>      That worked the first time.  I spent hours trying to get
something to
> work.  So thank
> you very much.  I was not sure of what to put in for the obs
dictionary
> because the Netcdf
> file produced by the ascii2nc did not really have a variable called
snow it
> just had float obs_val(nobs) ;
>
>     I know that both of these observations that you have helped me
with
> will be used a lot.
>     Thanks,
>     Sherrie
>
> On Thu, Nov 7, 2019 at 9:50 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > I've attached the Point-Stat config file I used.  And here's how I
called
> > it on the command line:
> >
> > /d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \
> > FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie -outdir out
-v 3
> > -log run_ps.log
> >
> > Perhaps the problem is how you set up the "obs" settings, not the
"fcst"
> > ones.  Here's how those are set in the attached config file:
> >
> > fcst = {
> >   file_type = NETCDF_NCCF;
> >   field = [
> >      {
> >        name  = "snowd";
> >        level = ["(0,*,*)"];
> >      }
> >   ];
> > }
> > obs = {
> >   field = [
> >      {
> >        message_type = [ "ADPSFC" ];
> >        name  = "snowd";
> >        level = "L0";
> >      }
> >   ];
> > }
> >
> > Thanks,
> > John
> >
> > On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> > >
> > > Hi John,
> > >
> > >      Here are the other things I have tried.
> > >     fcst = {
> > >    message_type = [ "ADPSFC" ];
> > >    level = ["(0,*,*)"];
> > >    field = [
> > >         {
> > >         name  = "snowd";
> > >        }
> > >    ];
> > > }
> > >
> > > fcst = {
> > >    message_type = [ "ADPSFC" ];
> > >
> > >    field = [
> > >         {
> > >         name  = "snowd";   level = "(0,*,*)";
> > >        }
> > >    ];
> > > }
> > >
> > > They all give me the same error.
> > > VarInfo::set_level_info_grib() - failed to parse level string
'(0,*,*)'
> > >
> > > Thanks,
> > > Sherrie
> > >
> > >
> > > On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Sherrie,
> > > >
> > > > FYI, I was able to find the line that produced the segfault,
and I
> > > updated
> > > > the development version of the code.  In future versions of
MET,
> you'll
> > > > this error message instead of the code segfaulting:
> > > >
> > > > ERROR  :
> > > > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
can't
> find
> > > > requested variable name "snodsfc" in NetCDF file
"FV3file.nc4".
> > > > ERROR  :
> > > >
> > > > Please let us know if you ever run into another segfault.
They
> always
> > > need
> > > > to be fixed and replaced with a useful error message.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > > >
> > > > > Sherrie,
> > > > >
> > > > > Whenever getting started with new datasets, I always start
by
> running
> > > the
> > > > > plot_data_plane tool.  And I run plot_point_obs for point
> > observations.
> > > > To
> > > > > that end, I ran the following commands which produced that
attached
> > > > > images.  And this data looks fine to me:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > > > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > > > > file_type=NETCDF_NCCF;'convert -rotate 90 -background white
> -flatten
> > > > > FV3file.ps
> > > > >
> > > >
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > > > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > > > > <http://oct2018_met.ps>convert -rotate 90 -background white
> -flatten
> > > > > oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
> > > > >
> > > > > But when I run Point-Stat, I get a segfault just like you
did!
> > > Segfaults
> > > > > are never good, so we definitely need to fix that!  But
let's
> figure
> > > out
> > > > > why.  I found a problem in your Point-Stat config file.
You're
> > > > requesting
> > > > > data for:
> > > > >
> > > > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > > > >
> > > > > But there are 2 problems here.  The FV3file.nc contains a
variable
> > > named
> > > > > "snowd" and not "snodsfc" and it has 2 dimensions, not 3.
So I
> > > switched
> > > > to
> > > > > using:
> > > > >
> > > > > *   name  = "snowd";    level = ["(*,*)"];*
> > > > >
> > > > > That gets past the segfault, but we now have this error:
> > > > >
> > > > > *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &)
-> Not
> > found
> > > > > time out of 1. ERROR  :
> MetNcCFDataFile::collect_time_offsets(VarInfo
> > > &)
> > > > ->
> > > > > variable "snowd" has unknown error (999).*
> > > > >
> > > > > So there's a problem with the timing info.  Looking in the
> > > FV3file.nc4, I
> > > > > see that you have a time dimension defined, but it is not
included
> > as a
> > > > > dimension for the data variable: *float snowd(lat, lon) ;*
> > > > >
> > > > > MET is definitely expecting the time variable to be
included.  I
> used
> > > > > ncdump and ncgen to modify your file to add the time
dimension:
> > > > >
> > > > >
> > > > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > > > *# Edit FV3file.nc4dump file by adding time dim to snowd
variable*
> > > > >
> > > > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > > > And then I modified your config file and reran Point-Stat
using the
> > > > > updated file:
> > > > >
> > > > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > > > >
> > > > >
> > */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> > > \*
> > > > > *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> > > > > PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*
> > > > >
> > > > > That ran but gave me 0 matched pairs because of a mismatch
of "obs
> > > type":
> > > > > DEBUG 3: Observations processed   = 47880
> > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > DEBUG 3: Rejected: obs type       = 47880
> > > > >
> > > > > I used ncdump to figure out that your observations are named
> "snowd":
> > > > > ncdump -v obs_var oct2018_met.nc
> > > > >  obs_var =
> > > > >  "snowd",
> > > > >  "66" ;
> > > > >
> > > > > So I modified your config file to request "snowd"
observations be
> > used:
> > > > >
> > > > > *        //name  = "SNOD";        name  = "snowd";*
> > > > >
> > > > > And I finally got 47,415 matched pairs:
> > > > > DEBUG 3: Number of matched pairs  = 47415
> > > > > DEBUG 3: Observations processed   = 47880
> > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > DEBUG 3: Rejected: obs type       = 465
> > > > >
> > > > > But what about those other 465 ones?  Having obs_var = "66"
looks
> > > > > suspicious to me.  So I looked upstream as the ascii file
you sent.
> > I
> > > > > contains 47,880 lines:
> > > > > cat oct2018_met.ascii | wc -l
> > > > > 47880
> > > > >
> > > > > For some reason, this data files looks to be a little bit
> corrupted.
> > > In
> > > > > the second column, where there should be station names, I
see the
> > > "snowd"
> > > > > string getting somewhat randomly embedded in there:
> "GME001219snowd".
> > > > > And that happens in other columns too.  In fact, you'd
expect
> "snowd"
> > > to
> > > > > show up 47,880 times (once per line).  But it appears in
this file
> > > 55,896
> > > > > times.  My guess is that this is the result of some over-
aggressive
> > use
> > > > of
> > > > > "sed".  So I'd suggest taking a look at that.
> > > > >
> > > > > Hope that helps.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT
> > > > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> > > > >> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > >>        Queue: met_help
> > > > >>      Subject: segfault when running point stat
> > > > >>        Owner: Nobody
> > > > >>   Requestors: sherrie.fredrick at noaa.gov
> > > > >>       Status: new
> > > > >>  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> > > >
> > > > >>
> > > > >>
> > > > >> Good Morning,
> > > > >>
> > > > >>      I am trying to run point_stat.  I have the following.
> > > > >>      FV3file.nc4  - a file from running the FV3 model with
only
> one
> > > > field
> > > > >>                            snow depth.
> > > > >>      oct2018_met.ascii  - an ascii file with station
information,
> > lat,
> > > > lon
> > > > >> and
> > > > >>                                       snowd values.
> > > > >>      oct2018_met.nc     - netcdf file that I got from
running
> > > ascii2nc
> > > > on
> > > > >> the
> > > > >>                                       oct2018_met.ascii
file.
> > > > >>
> > > > >>       PointStatConfig.sherrie - mypoint stat config file.
I am
> not
> > > sure
> > > > >> of
> > > > >> what
> > > > >>        I should have in the config file.
> > > > >>        In the regrid option I have tried
> > > > >>        to_grid = OBS
> > > > >>        and to_grid =
> > > > >>
> > > > >>        I am also not quite sure of what I should have in
the obs
> > > > >> dictionary.
> > > > >>
> > > > >>        I have placed all of the above files on the rap ftp
> > > > >> site /incoming/irap/met_help/sherrie_data/
> > > > >>
> > > > >>        Thanks for your help again,
> > > > >>        Sherrie Fredrick
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: segfault when running point stat
From: Sherrie Fredrick - NOAA Affiliate
Time: Thu Nov 07 11:05:39 2019

Hi John,

     Yep that makes sense.  I have one other question.  I am
interested in
the MET verification measures described
in Appendix C, section C.1.  The contingency table is exactly what I
am
looking for.  I think these values are in the
cts.txt file, but I am not sure of how to get them.  Could you tell me
what
if the cts file is the right one and what I should look for?  I have
only
used the cnt.txt file in the past.

   Many thanks,
   Sherrie

On Thu, Nov 7, 2019 at 10:53 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Sherrie,
>
> Sorry for the confusion.  There is a somewhat subtle detail here.
>
> For gridded forecast files in NetCDF format, you set name = "NetCDF
> Variable Name"; and level = "Describe indices of that variable".
>
> For point observation files (NetCDF output of pb2nc, ascii2nc, and
> madis2nc), you set name = "observation variable strings" which are
listed
> in the "obs_var" variable in that file.  For example, here variable
names
> present in some NetCDF output of pb2nc:
>
> ncdump -v obs_var gdas1.20120409.t12z.prepbufr.nc
> ...
> data:
>
>  obs_var =
>   "SPFH",
>   "TMP",
>   "HGT",
>   "UGRD",
>   "VGRD",
>   "DPT",
>   "WIND",
>   "RH",
>   "MIXR" ;
> }
>
> Make sense?
>
> Thanks,
> John
>
> On Thu, Nov 7, 2019 at 10:43 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> >
> > Hi John,
> >      That worked the first time.  I spent hours trying to get
something
> to
> > work.  So thank
> > you very much.  I was not sure of what to put in for the obs
dictionary
> > because the Netcdf
> > file produced by the ascii2nc did not really have a variable
called snow
> it
> > just had float obs_val(nobs) ;
> >
> >     I know that both of these observations that you have helped me
with
> > will be used a lot.
> >     Thanks,
> >     Sherrie
> >
> > On Thu, Nov 7, 2019 at 9:50 AM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Sherrie,
> > >
> > > I've attached the Point-Stat config file I used.  And here's how
I
> called
> > > it on the command line:
> > >
> > > /d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \
> > > FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie -outdir
out -v
> 3
> > > -log run_ps.log
> > >
> > > Perhaps the problem is how you set up the "obs" settings, not
the
> "fcst"
> > > ones.  Here's how those are set in the attached config file:
> > >
> > > fcst = {
> > >   file_type = NETCDF_NCCF;
> > >   field = [
> > >      {
> > >        name  = "snowd";
> > >        level = ["(0,*,*)"];
> > >      }
> > >   ];
> > > }
> > > obs = {
> > >   field = [
> > >      {
> > >        message_type = [ "ADPSFC" ];
> > >        name  = "snowd";
> > >        level = "L0";
> > >      }
> > >   ];
> > > }
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA Affiliate
via
> RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
>
> > > >
> > > > Hi John,
> > > >
> > > >      Here are the other things I have tried.
> > > >     fcst = {
> > > >    message_type = [ "ADPSFC" ];
> > > >    level = ["(0,*,*)"];
> > > >    field = [
> > > >         {
> > > >         name  = "snowd";
> > > >        }
> > > >    ];
> > > > }
> > > >
> > > > fcst = {
> > > >    message_type = [ "ADPSFC" ];
> > > >
> > > >    field = [
> > > >         {
> > > >         name  = "snowd";   level = "(0,*,*)";
> > > >        }
> > > >    ];
> > > > }
> > > >
> > > > They all give me the same error.
> > > > VarInfo::set_level_info_grib() - failed to parse level string
> '(0,*,*)'
> > > >
> > > > Thanks,
> > > > Sherrie
> > > >
> > > >
> > > > On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Sherrie,
> > > > >
> > > > > FYI, I was able to find the line that produced the segfault,
and I
> > > > updated
> > > > > the development version of the code.  In future versions of
MET,
> > you'll
> > > > > this error message instead of the code segfaulting:
> > > > >
> > > > > ERROR  :
> > > > > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
can't
> > find
> > > > > requested variable name "snodsfc" in NetCDF file
"FV3file.nc4".
> > > > > ERROR  :
> > > > >
> > > > > Please let us know if you ever run into another segfault.
They
> > always
> > > > need
> > > > > to be fixed and replaced with a useful error message.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway
<johnhg at ucar.edu
> >
> > > > wrote:
> > > > >
> > > > > > Sherrie,
> > > > > >
> > > > > > Whenever getting started with new datasets, I always start
by
> > running
> > > > the
> > > > > > plot_data_plane tool.  And I run plot_point_obs for point
> > > observations.
> > > > > To
> > > > > > that end, I ran the following commands which produced that
> attached
> > > > > > images.  And this data looks fine to me:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > > > > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > > > > > file_type=NETCDF_NCCF;'convert -rotate 90 -background
white
> > -flatten
> > > > > > FV3file.ps
> > > > > >
> > > > >
> > > >
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > > > > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > > > > > <http://oct2018_met.ps>convert -rotate 90 -background
white
> > -flatten
> > > > > > oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
> > > > > >
> > > > > > But when I run Point-Stat, I get a segfault just like you
did!
> > > > Segfaults
> > > > > > are never good, so we definitely need to fix that!  But
let's
> > figure
> > > > out
> > > > > > why.  I found a problem in your Point-Stat config file.
You're
> > > > > requesting
> > > > > > data for:
> > > > > >
> > > > > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > > > > >
> > > > > > But there are 2 problems here.  The FV3file.nc contains a
> variable
> > > > named
> > > > > > "snowd" and not "snodsfc" and it has 2 dimensions, not 3.
So I
> > > > switched
> > > > > to
> > > > > > using:
> > > > > >
> > > > > > *   name  = "snowd";    level = ["(*,*)"];*
> > > > > >
> > > > > > That gets past the segfault, but we now have this error:
> > > > > >
> > > > > > *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo &)
-> Not
> > > found
> > > > > > time out of 1. ERROR  :
> > MetNcCFDataFile::collect_time_offsets(VarInfo
> > > > &)
> > > > > ->
> > > > > > variable "snowd" has unknown error (999).*
> > > > > >
> > > > > > So there's a problem with the timing info.  Looking in the
> > > > FV3file.nc4, I
> > > > > > see that you have a time dimension defined, but it is not
> included
> > > as a
> > > > > > dimension for the data variable: *float snowd(lat, lon) ;*
> > > > > >
> > > > > > MET is definitely expecting the time variable to be
included.  I
> > used
> > > > > > ncdump and ncgen to modify your file to add the time
dimension:
> > > > > >
> > > > > >
> > > > > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > > > > *# Edit FV3file.nc4dump file by adding time dim to snowd
> variable*
> > > > > >
> > > > > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > > > > And then I modified your config file and reran Point-Stat
using
> the
> > > > > > updated file:
> > > > > >
> > > > > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > > > > >
> > > > > >
> > > */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> > > > \*
> > > > > > *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> > > > > > PointStatConfig.sherrie -outdir out -v 3 -log run_ps.log*
> > > > > >
> > > > > > That ran but gave me 0 matched pairs because of a mismatch
of
> "obs
> > > > type":
> > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > DEBUG 3: Rejected: obs type       = 47880
> > > > > >
> > > > > > I used ncdump to figure out that your observations are
named
> > "snowd":
> > > > > > ncdump -v obs_var oct2018_met.nc
> > > > > >  obs_var =
> > > > > >  "snowd",
> > > > > >  "66" ;
> > > > > >
> > > > > > So I modified your config file to request "snowd"
observations be
> > > used:
> > > > > >
> > > > > > *        //name  = "SNOD";        name  = "snowd";*
> > > > > >
> > > > > > And I finally got 47,415 matched pairs:
> > > > > > DEBUG 3: Number of matched pairs  = 47415
> > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > DEBUG 3: Rejected: obs type       = 465
> > > > > >
> > > > > > But what about those other 465 ones?  Having obs_var =
"66" looks
> > > > > > suspicious to me.  So I looked upstream as the ascii file
you
> sent.
> > > I
> > > > > > contains 47,880 lines:
> > > > > > cat oct2018_met.ascii | wc -l
> > > > > > 47880
> > > > > >
> > > > > > For some reason, this data files looks to be a little bit
> > corrupted.
> > > > In
> > > > > > the second column, where there should be station names, I
see the
> > > > "snowd"
> > > > > > string getting somewhat randomly embedded in there:
> > "GME001219snowd".
> > > > > > And that happens in other columns too.  In fact, you'd
expect
> > "snowd"
> > > > to
> > > > > > show up 47,880 times (once per line).  But it appears in
this
> file
> > > > 55,896
> > > > > > times.  My guess is that this is the result of some
> over-aggressive
> > > use
> > > > > of
> > > > > > "sed".  So I'd suggest taking a look at that.
> > > > > >
> > > > > > Hope that helps.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA
Affiliate
> > via
> > > > RT
> > > > > <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > >>
> > > > > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> > > > > >> Transaction: Ticket created by sherrie.fredrick at noaa.gov
> > > > > >>        Queue: met_help
> > > > > >>      Subject: segfault when running point stat
> > > > > >>        Owner: Nobody
> > > > > >>   Requestors: sherrie.fredrick at noaa.gov
> > > > > >>       Status: new
> > > > > >>  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> > > > >
> > > > > >>
> > > > > >>
> > > > > >> Good Morning,
> > > > > >>
> > > > > >>      I am trying to run point_stat.  I have the
following.
> > > > > >>      FV3file.nc4  - a file from running the FV3 model
with only
> > one
> > > > > field
> > > > > >>                            snow depth.
> > > > > >>      oct2018_met.ascii  - an ascii file with station
> information,
> > > lat,
> > > > > lon
> > > > > >> and
> > > > > >>                                       snowd values.
> > > > > >>      oct2018_met.nc     - netcdf file that I got from
running
> > > > ascii2nc
> > > > > on
> > > > > >> the
> > > > > >>                                       oct2018_met.ascii
file.
> > > > > >>
> > > > > >>       PointStatConfig.sherrie - mypoint stat config file.
I am
> > not
> > > > sure
> > > > > >> of
> > > > > >> what
> > > > > >>        I should have in the config file.
> > > > > >>        In the regrid option I have tried
> > > > > >>        to_grid = OBS
> > > > > >>        and to_grid =
> > > > > >>
> > > > > >>        I am also not quite sure of what I should have in
the obs
> > > > > >> dictionary.
> > > > > >>
> > > > > >>        I have placed all of the above files on the rap
ftp
> > > > > >> site /incoming/irap/met_help/sherrie_data/
> > > > > >>
> > > > > >>        Thanks for your help again,
> > > > > >>        Sherrie Fredrick
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: segfault when running point stat
From: John Halley Gotway
Time: Thu Nov 07 11:27:43 2019

Sherrie,

The output that point_stat writes is controlled by the "output_flag"
settings in the config file.  Looking in your config file, I see that
you
are already requesting these output line types:

output_flag = {
...
   ctc    = BOTH;
   cts    = BOTH;
...

But you're not getting any CTC or CTS output lines.  And that's
because you
haven't specified any categorical thresholds to be applied.  You'll
just
need to add those categorical threshold using the "cat_thresh" entry
in the
Point-Stat config file.  If you define "cat_thesh" at the top-level of
context (as shown below), the same categorical thresholds will be
applied
to both the forecast and observation data.  Alternatively, you could
specify cat_thresh separately inside the fcst and obs dictionaries to
define them separately.

Thanks,
John

cat_thresh = [ >800, >900, >1000 ];

fcst = {
  file_type = NETCDF_NCCF;
  field = [
     {
       name  = "snowd";
       level = ["(0,*,*)"];
     }
  ];
}
obs = {
  field = [
     {
       message_type = [ "ADPSFC" ];
       name  = "snowd";
       level = "L0";
     }
  ];
}

On Thu, Nov 7, 2019 at 11:06 AM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>
> Hi John,
>
>      Yep that makes sense.  I have one other question.  I am
interested in
> the MET verification measures described
> in Appendix C, section C.1.  The contingency table is exactly what I
am
> looking for.  I think these values are in the
> cts.txt file, but I am not sure of how to get them.  Could you tell
me what
> if the cts file is the right one and what I should look for?  I have
only
> used the cnt.txt file in the past.
>
>    Many thanks,
>    Sherrie
>
> On Thu, Nov 7, 2019 at 10:53 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > Sorry for the confusion.  There is a somewhat subtle detail here.
> >
> > For gridded forecast files in NetCDF format, you set name =
"NetCDF
> > Variable Name"; and level = "Describe indices of that variable".
> >
> > For point observation files (NetCDF output of pb2nc, ascii2nc, and
> > madis2nc), you set name = "observation variable strings" which are
listed
> > in the "obs_var" variable in that file.  For example, here
variable names
> > present in some NetCDF output of pb2nc:
> >
> > ncdump -v obs_var gdas1.20120409.t12z.prepbufr.nc
> > ...
> > data:
> >
> >  obs_var =
> >   "SPFH",
> >   "TMP",
> >   "HGT",
> >   "UGRD",
> >   "VGRD",
> >   "DPT",
> >   "WIND",
> >   "RH",
> >   "MIXR" ;
> > }
> >
> > Make sense?
> >
> > Thanks,
> > John
> >
> > On Thu, Nov 7, 2019 at 10:43 AM Sherrie Fredrick - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> > >
> > > Hi John,
> > >      That worked the first time.  I spent hours trying to get
something
> > to
> > > work.  So thank
> > > you very much.  I was not sure of what to put in for the obs
dictionary
> > > because the Netcdf
> > > file produced by the ascii2nc did not really have a variable
called
> snow
> > it
> > > just had float obs_val(nobs) ;
> > >
> > >     I know that both of these observations that you have helped
me with
> > > will be used a lot.
> > >     Thanks,
> > >     Sherrie
> > >
> > > On Thu, Nov 7, 2019 at 9:50 AM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Sherrie,
> > > >
> > > > I've attached the Point-Stat config file I used.  And here's
how I
> > called
> > > > it on the command line:
> > > >
> > > >
> /d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \
> > > > FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie -outdir
out
> -v
> > 3
> > > > -log run_ps.log
> > > >
> > > > Perhaps the problem is how you set up the "obs" settings, not
the
> > "fcst"
> > > > ones.  Here's how those are set in the attached config file:
> > > >
> > > > fcst = {
> > > >   file_type = NETCDF_NCCF;
> > > >   field = [
> > > >      {
> > > >        name  = "snowd";
> > > >        level = ["(0,*,*)"];
> > > >      }
> > > >   ];
> > > > }
> > > > obs = {
> > > >   field = [
> > > >      {
> > > >        message_type = [ "ADPSFC" ];
> > > >        name  = "snowd";
> > > >        level = "L0";
> > > >      }
> > > >   ];
> > > > }
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA
Affiliate via
> > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> > > > >
> > > > > Hi John,
> > > > >
> > > > >      Here are the other things I have tried.
> > > > >     fcst = {
> > > > >    message_type = [ "ADPSFC" ];
> > > > >    level = ["(0,*,*)"];
> > > > >    field = [
> > > > >         {
> > > > >         name  = "snowd";
> > > > >        }
> > > > >    ];
> > > > > }
> > > > >
> > > > > fcst = {
> > > > >    message_type = [ "ADPSFC" ];
> > > > >
> > > > >    field = [
> > > > >         {
> > > > >         name  = "snowd";   level = "(0,*,*)";
> > > > >        }
> > > > >    ];
> > > > > }
> > > > >
> > > > > They all give me the same error.
> > > > > VarInfo::set_level_info_grib() - failed to parse level
string
> > '(0,*,*)'
> > > > >
> > > > > Thanks,
> > > > > Sherrie
> > > > >
> > > > >
> > > > > On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Sherrie,
> > > > > >
> > > > > > FYI, I was able to find the line that produced the
segfault, and
> I
> > > > > updated
> > > > > > the development version of the code.  In future versions
of MET,
> > > you'll
> > > > > > this error message instead of the code segfaulting:
> > > > > >
> > > > > > ERROR  :
> > > > > > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo &)
->
> can't
> > > find
> > > > > > requested variable name "snodsfc" in NetCDF file
"FV3file.nc4".
> > > > > > ERROR  :
> > > > > >
> > > > > > Please let us know if you ever run into another segfault.
They
> > > always
> > > > > need
> > > > > > to be fixed and replaced with a useful error message.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway <
> johnhg at ucar.edu
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Sherrie,
> > > > > > >
> > > > > > > Whenever getting started with new datasets, I always
start by
> > > running
> > > > > the
> > > > > > > plot_data_plane tool.  And I run plot_point_obs for
point
> > > > observations.
> > > > > > To
> > > > > > > that end, I ran the following commands which produced
that
> > attached
> > > > > > > images.  And this data looks fine to me:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > > > > > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > > > > > > file_type=NETCDF_NCCF;'convert -rotate 90 -background
white
> > > -flatten
> > > > > > > FV3file.ps
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > > > > > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > > > > > > <http://oct2018_met.ps>convert -rotate 90 -background
white
> > > -flatten
> > > > > > > oct2018_met.ps <http://oct2018_met.ps> oct2018_met.png*
> > > > > > >
> > > > > > > But when I run Point-Stat, I get a segfault just like
you did!
> > > > > Segfaults
> > > > > > > are never good, so we definitely need to fix that!  But
let's
> > > figure
> > > > > out
> > > > > > > why.  I found a problem in your Point-Stat config file.
You're
> > > > > > requesting
> > > > > > > data for:
> > > > > > >
> > > > > > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > > > > > >
> > > > > > > But there are 2 problems here.  The FV3file.nc contains
a
> > variable
> > > > > named
> > > > > > > "snowd" and not "snodsfc" and it has 2 dimensions, not
3.  So I
> > > > > switched
> > > > > > to
> > > > > > > using:
> > > > > > >
> > > > > > > *   name  = "snowd";    level = ["(*,*)"];*
> > > > > > >
> > > > > > > That gets past the segfault, but we now have this error:
> > > > > > >
> > > > > > > *WARNING: MetNcCFDataFile::collect_time_offsets(VarInfo
&) ->
> Not
> > > > found
> > > > > > > time out of 1. ERROR  :
> > > MetNcCFDataFile::collect_time_offsets(VarInfo
> > > > > &)
> > > > > > ->
> > > > > > > variable "snowd" has unknown error (999).*
> > > > > > >
> > > > > > > So there's a problem with the timing info.  Looking in
the
> > > > > FV3file.nc4, I
> > > > > > > see that you have a time dimension defined, but it is
not
> > included
> > > > as a
> > > > > > > dimension for the data variable: *float snowd(lat, lon)
;*
> > > > > > >
> > > > > > > MET is definitely expecting the time variable to be
included.
> I
> > > used
> > > > > > > ncdump and ncgen to modify your file to add the time
dimension:
> > > > > > >
> > > > > > >
> > > > > > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > > > > > *# Edit FV3file.nc4dump file by adding time dim to snowd
> > variable*
> > > > > > >
> > > > > > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > > > > > And then I modified your config file and reran Point-
Stat using
> > the
> > > > > > > updated file:
> > > > > > >
> > > > > > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > > > > > >
> > > > > > >
> > > >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> > > > > \*
> > > > > > > *FV3file_MOD.nc4 oct2018_met.nc <http://oct2018_met.nc>
> > > > > > > PointStatConfig.sherrie -outdir out -v 3 -log
run_ps.log*
> > > > > > >
> > > > > > > That ran but gave me 0 matched pairs because of a
mismatch of
> > "obs
> > > > > type":
> > > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > DEBUG 3: Rejected: obs type       = 47880
> > > > > > >
> > > > > > > I used ncdump to figure out that your observations are
named
> > > "snowd":
> > > > > > > ncdump -v obs_var oct2018_met.nc
> > > > > > >  obs_var =
> > > > > > >  "snowd",
> > > > > > >  "66" ;
> > > > > > >
> > > > > > > So I modified your config file to request "snowd"
observations
> be
> > > > used:
> > > > > > >
> > > > > > > *        //name  = "SNOD";        name  = "snowd";*
> > > > > > >
> > > > > > > And I finally got 47,415 matched pairs:
> > > > > > > DEBUG 3: Number of matched pairs  = 47415
> > > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > DEBUG 3: Rejected: obs type       = 465
> > > > > > >
> > > > > > > But what about those other 465 ones?  Having obs_var =
"66"
> looks
> > > > > > > suspicious to me.  So I looked upstream as the ascii
file you
> > sent.
> > > > I
> > > > > > > contains 47,880 lines:
> > > > > > > cat oct2018_met.ascii | wc -l
> > > > > > > 47880
> > > > > > >
> > > > > > > For some reason, this data files looks to be a little
bit
> > > corrupted.
> > > > > In
> > > > > > > the second column, where there should be station names,
I see
> the
> > > > > "snowd"
> > > > > > > string getting somewhat randomly embedded in there:
> > > "GME001219snowd".
> > > > > > > And that happens in other columns too.  In fact, you'd
expect
> > > "snowd"
> > > > > to
> > > > > > > show up 47,880 times (once per line).  But it appears in
this
> > file
> > > > > 55,896
> > > > > > > times.  My guess is that this is the result of some
> > over-aggressive
> > > > use
> > > > > > of
> > > > > > > "sed".  So I'd suggest taking a look at that.
> > > > > > >
> > > > > > > Hope that helps.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick - NOAA
> Affiliate
> > > via
> > > > > RT
> > > > > > <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > >>
> > > > > > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted upon.
> > > > > > >> Transaction: Ticket created by
sherrie.fredrick at noaa.gov
> > > > > > >>        Queue: met_help
> > > > > > >>      Subject: segfault when running point stat
> > > > > > >>        Owner: Nobody
> > > > > > >>   Requestors: sherrie.fredrick at noaa.gov
> > > > > > >>       Status: new
> > > > > > >>  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> > > > > >
> > > > > > >>
> > > > > > >>
> > > > > > >> Good Morning,
> > > > > > >>
> > > > > > >>      I am trying to run point_stat.  I have the
following.
> > > > > > >>      FV3file.nc4  - a file from running the FV3 model
with
> only
> > > one
> > > > > > field
> > > > > > >>                            snow depth.
> > > > > > >>      oct2018_met.ascii  - an ascii file with station
> > information,
> > > > lat,
> > > > > > lon
> > > > > > >> and
> > > > > > >>                                       snowd values.
> > > > > > >>      oct2018_met.nc     - netcdf file that I got from
running
> > > > > ascii2nc
> > > > > > on
> > > > > > >> the
> > > > > > >>                                       oct2018_met.ascii
file.
> > > > > > >>
> > > > > > >>       PointStatConfig.sherrie - mypoint stat config
file.  I
> am
> > > not
> > > > > sure
> > > > > > >> of
> > > > > > >> what
> > > > > > >>        I should have in the config file.
> > > > > > >>        In the regrid option I have tried
> > > > > > >>        to_grid = OBS
> > > > > > >>        and to_grid =
> > > > > > >>
> > > > > > >>        I am also not quite sure of what I should have
in the
> obs
> > > > > > >> dictionary.
> > > > > > >>
> > > > > > >>        I have placed all of the above files on the rap
ftp
> > > > > > >> site /incoming/irap/met_help/sherrie_data/
> > > > > > >>
> > > > > > >>        Thanks for your help again,
> > > > > > >>        Sherrie Fredrick
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: segfault when running point stat
From: Sherrie Fredrick - NOAA Affiliate
Time: Thu Nov 07 13:58:35 2019

Hi John,

    I read the documentation in Appendix C and was able to find
everything
that I need in the cts file.
It is really cool.  It prints out exactly the values I am looking for.
YAHOO!.

   I think I am done bugging you with this.
   Many Thanks,
   Sherrie

On Thu, Nov 7, 2019 at 11:27 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Sherrie,
>
> The output that point_stat writes is controlled by the "output_flag"
> settings in the config file.  Looking in your config file, I see
that you
> are already requesting these output line types:
>
> output_flag = {
> ...
>    ctc    = BOTH;
>    cts    = BOTH;
> ...
>
> But you're not getting any CTC or CTS output lines.  And that's
because you
> haven't specified any categorical thresholds to be applied.  You'll
just
> need to add those categorical threshold using the "cat_thresh" entry
in the
> Point-Stat config file.  If you define "cat_thesh" at the top-level
of
> context (as shown below), the same categorical thresholds will be
applied
> to both the forecast and observation data.  Alternatively, you could
> specify cat_thresh separately inside the fcst and obs dictionaries
to
> define them separately.
>
> Thanks,
> John
>
> cat_thresh = [ >800, >900, >1000 ];
>
> fcst = {
>   file_type = NETCDF_NCCF;
>   field = [
>      {
>        name  = "snowd";
>        level = ["(0,*,*)"];
>      }
>   ];
> }
> obs = {
>   field = [
>      {
>        message_type = [ "ADPSFC" ];
>        name  = "snowd";
>        level = "L0";
>      }
>   ];
> }
>
>
>
>
> On Thu, Nov 7, 2019 at 11:06 AM Sherrie Fredrick - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> >
> > Hi John,
> >
> >      Yep that makes sense.  I have one other question.  I am
interested
> in
> > the MET verification measures described
> > in Appendix C, section C.1.  The contingency table is exactly what
I am
> > looking for.  I think these values are in the
> > cts.txt file, but I am not sure of how to get them.  Could you
tell me
> what
> > if the cts file is the right one and what I should look for?  I
have only
> > used the cnt.txt file in the past.
> >
> >    Many thanks,
> >    Sherrie
> >
> > On Thu, Nov 7, 2019 at 10:53 AM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Sherrie,
> > >
> > > Sorry for the confusion.  There is a somewhat subtle detail
here.
> > >
> > > For gridded forecast files in NetCDF format, you set name =
"NetCDF
> > > Variable Name"; and level = "Describe indices of that variable".
> > >
> > > For point observation files (NetCDF output of pb2nc, ascii2nc,
and
> > > madis2nc), you set name = "observation variable strings" which
are
> listed
> > > in the "obs_var" variable in that file.  For example, here
variable
> names
> > > present in some NetCDF output of pb2nc:
> > >
> > > ncdump -v obs_var gdas1.20120409.t12z.prepbufr.nc
> > > ...
> > > data:
> > >
> > >  obs_var =
> > >   "SPFH",
> > >   "TMP",
> > >   "HGT",
> > >   "UGRD",
> > >   "VGRD",
> > >   "DPT",
> > >   "WIND",
> > >   "RH",
> > >   "MIXR" ;
> > > }
> > >
> > > Make sense?
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, Nov 7, 2019 at 10:43 AM Sherrie Fredrick - NOAA
Affiliate via
> RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
>
> > > >
> > > > Hi John,
> > > >      That worked the first time.  I spent hours trying to get
> something
> > > to
> > > > work.  So thank
> > > > you very much.  I was not sure of what to put in for the obs
> dictionary
> > > > because the Netcdf
> > > > file produced by the ascii2nc did not really have a variable
called
> > snow
> > > it
> > > > just had float obs_val(nobs) ;
> > > >
> > > >     I know that both of these observations that you have
helped me
> with
> > > > will be used a lot.
> > > >     Thanks,
> > > >     Sherrie
> > > >
> > > > On Thu, Nov 7, 2019 at 9:50 AM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Sherrie,
> > > > >
> > > > > I've attached the Point-Stat config file I used.  And here's
how I
> > > called
> > > > > it on the command line:
> > > > >
> > > > >
> > /d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \
> > > > > FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie
-outdir out
> > -v
> > > 3
> > > > > -log run_ps.log
> > > > >
> > > > > Perhaps the problem is how you set up the "obs" settings,
not the
> > > "fcst"
> > > > > ones.  Here's how those are set in the attached config file:
> > > > >
> > > > > fcst = {
> > > > >   file_type = NETCDF_NCCF;
> > > > >   field = [
> > > > >      {
> > > > >        name  = "snowd";
> > > > >        level = ["(0,*,*)"];
> > > > >      }
> > > > >   ];
> > > > > }
> > > > > obs = {
> > > > >   field = [
> > > > >      {
> > > > >        message_type = [ "ADPSFC" ];
> > > > >        name  = "snowd";
> > > > >        level = "L0";
> > > > >      }
> > > > >   ];
> > > > > }
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA
Affiliate
> via
> > > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> > > > > >
> > > > > > Hi John,
> > > > > >
> > > > > >      Here are the other things I have tried.
> > > > > >     fcst = {
> > > > > >    message_type = [ "ADPSFC" ];
> > > > > >    level = ["(0,*,*)"];
> > > > > >    field = [
> > > > > >         {
> > > > > >         name  = "snowd";
> > > > > >        }
> > > > > >    ];
> > > > > > }
> > > > > >
> > > > > > fcst = {
> > > > > >    message_type = [ "ADPSFC" ];
> > > > > >
> > > > > >    field = [
> > > > > >         {
> > > > > >         name  = "snowd";   level = "(0,*,*)";
> > > > > >        }
> > > > > >    ];
> > > > > > }
> > > > > >
> > > > > > They all give me the same error.
> > > > > > VarInfo::set_level_info_grib() - failed to parse level
string
> > > '(0,*,*)'
> > > > > >
> > > > > > Thanks,
> > > > > > Sherrie
> > > > > >
> > > > > >
> > > > > > On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT <
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Sherrie,
> > > > > > >
> > > > > > > FYI, I was able to find the line that produced the
segfault,
> and
> > I
> > > > > > updated
> > > > > > > the development version of the code.  In future versions
of
> MET,
> > > > you'll
> > > > > > > this error message instead of the code segfaulting:
> > > > > > >
> > > > > > > ERROR  :
> > > > > > > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo
&) ->
> > can't
> > > > find
> > > > > > > requested variable name "snodsfc" in NetCDF file
"FV3file.nc4".
> > > > > > > ERROR  :
> > > > > > >
> > > > > > > Please let us know if you ever run into another
segfault.  They
> > > > always
> > > > > > need
> > > > > > > to be fixed and replaced with a useful error message.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway <
> > johnhg at ucar.edu
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Sherrie,
> > > > > > > >
> > > > > > > > Whenever getting started with new datasets, I always
start by
> > > > running
> > > > > > the
> > > > > > > > plot_data_plane tool.  And I run plot_point_obs for
point
> > > > > observations.
> > > > > > > To
> > > > > > > > that end, I ran the following commands which produced
that
> > > attached
> > > > > > > > images.  And this data looks fine to me:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > > > > > > \FV3file.nc4 FV3file.ps 'name="snowd"; level="(*,*)";
> > > > > > > > file_type=NETCDF_NCCF;'convert -rotate 90 -background
white
> > > > -flatten
> > > > > > > > FV3file.ps
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > > > > > > \oct2018_met.nc <http://oct2018_met.nc> oct2018_met.ps
> > > > > > > > <http://oct2018_met.ps>convert -rotate 90 -background
white
> > > > -flatten
> > > > > > > > oct2018_met.ps <http://oct2018_met.ps>
oct2018_met.png*
> > > > > > > >
> > > > > > > > But when I run Point-Stat, I get a segfault just like
you
> did!
> > > > > > Segfaults
> > > > > > > > are never good, so we definitely need to fix that!
But let's
> > > > figure
> > > > > > out
> > > > > > > > why.  I found a problem in your Point-Stat config
file.
> You're
> > > > > > > requesting
> > > > > > > > data for:
> > > > > > > >
> > > > > > > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > > > > > > >
> > > > > > > > But there are 2 problems here.  The FV3file.nc
contains a
> > > variable
> > > > > > named
> > > > > > > > "snowd" and not "snodsfc" and it has 2 dimensions, not
3.
> So I
> > > > > > switched
> > > > > > > to
> > > > > > > > using:
> > > > > > > >
> > > > > > > > *   name  = "snowd";    level = ["(*,*)"];*
> > > > > > > >
> > > > > > > > That gets past the segfault, but we now have this
error:
> > > > > > > >
> > > > > > > > *WARNING:
MetNcCFDataFile::collect_time_offsets(VarInfo &) ->
> > Not
> > > > > found
> > > > > > > > time out of 1. ERROR  :
> > > > MetNcCFDataFile::collect_time_offsets(VarInfo
> > > > > > &)
> > > > > > > ->
> > > > > > > > variable "snowd" has unknown error (999).*
> > > > > > > >
> > > > > > > > So there's a problem with the timing info.  Looking in
the
> > > > > > FV3file.nc4, I
> > > > > > > > see that you have a time dimension defined, but it is
not
> > > included
> > > > > as a
> > > > > > > > dimension for the data variable: *float snowd(lat,
lon) ;*
> > > > > > > >
> > > > > > > > MET is definitely expecting the time variable to be
included.
> > I
> > > > used
> > > > > > > > ncdump and ncgen to modify your file to add the time
> dimension:
> > > > > > > >
> > > > > > > >
> > > > > > > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > > > > > > *# Edit FV3file.nc4dump file by adding time dim to
snowd
> > > variable*
> > > > > > > >
> > > > > > > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > > > > > > And then I modified your config file and reran Point-
Stat
> using
> > > the
> > > > > > > > updated file:
> > > > > > > >
> > > > > > > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > > > > > > >
> > > > > > > >
> > > > >
> > */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> > > > > > \*
> > > > > > > > *FV3file_MOD.nc4 oct2018_met.nc
<http://oct2018_met.nc>
> > > > > > > > PointStatConfig.sherrie -outdir out -v 3 -log
run_ps.log*
> > > > > > > >
> > > > > > > > That ran but gave me 0 matched pairs because of a
mismatch of
> > > "obs
> > > > > > type":
> > > > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > DEBUG 3: Rejected: obs type       = 47880
> > > > > > > >
> > > > > > > > I used ncdump to figure out that your observations are
named
> > > > "snowd":
> > > > > > > > ncdump -v obs_var oct2018_met.nc
> > > > > > > >  obs_var =
> > > > > > > >  "snowd",
> > > > > > > >  "66" ;
> > > > > > > >
> > > > > > > > So I modified your config file to request "snowd"
> observations
> > be
> > > > > used:
> > > > > > > >
> > > > > > > > *        //name  = "SNOD";        name  = "snowd";*
> > > > > > > >
> > > > > > > > And I finally got 47,415 matched pairs:
> > > > > > > > DEBUG 3: Number of matched pairs  = 47415
> > > > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > DEBUG 3: Rejected: obs type       = 465
> > > > > > > >
> > > > > > > > But what about those other 465 ones?  Having obs_var =
"66"
> > looks
> > > > > > > > suspicious to me.  So I looked upstream as the ascii
file you
> > > sent.
> > > > > I
> > > > > > > > contains 47,880 lines:
> > > > > > > > cat oct2018_met.ascii | wc -l
> > > > > > > > 47880
> > > > > > > >
> > > > > > > > For some reason, this data files looks to be a little
bit
> > > > corrupted.
> > > > > > In
> > > > > > > > the second column, where there should be station
names, I see
> > the
> > > > > > "snowd"
> > > > > > > > string getting somewhat randomly embedded in there:
> > > > "GME001219snowd".
> > > > > > > > And that happens in other columns too.  In fact, you'd
expect
> > > > "snowd"
> > > > > > to
> > > > > > > > show up 47,880 times (once per line).  But it appears
in this
> > > file
> > > > > > 55,896
> > > > > > > > times.  My guess is that this is the result of some
> > > over-aggressive
> > > > > use
> > > > > > > of
> > > > > > > > "sed".  So I'd suggest taking a look at that.
> > > > > > > >
> > > > > > > > Hope that helps.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick -
NOAA
> > Affiliate
> > > > via
> > > > > > RT
> > > > > > > <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > >>
> > > > > > > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted
upon.
> > > > > > > >> Transaction: Ticket created by
sherrie.fredrick at noaa.gov
> > > > > > > >>        Queue: met_help
> > > > > > > >>      Subject: segfault when running point stat
> > > > > > > >>        Owner: Nobody
> > > > > > > >>   Requestors: sherrie.fredrick at noaa.gov
> > > > > > > >>       Status: new
> > > > > > > >>  Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> > > > > > >
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> Good Morning,
> > > > > > > >>
> > > > > > > >>      I am trying to run point_stat.  I have the
following.
> > > > > > > >>      FV3file.nc4  - a file from running the FV3 model
with
> > only
> > > > one
> > > > > > > field
> > > > > > > >>                            snow depth.
> > > > > > > >>      oct2018_met.ascii  - an ascii file with station
> > > information,
> > > > > lat,
> > > > > > > lon
> > > > > > > >> and
> > > > > > > >>                                       snowd values.
> > > > > > > >>      oct2018_met.nc     - netcdf file that I got from
> running
> > > > > > ascii2nc
> > > > > > > on
> > > > > > > >> the
> > > > > > > >>
oct2018_met.ascii
> file.
> > > > > > > >>
> > > > > > > >>       PointStatConfig.sherrie - mypoint stat config
file.  I
> > am
> > > > not
> > > > > > sure
> > > > > > > >> of
> > > > > > > >> what
> > > > > > > >>        I should have in the config file.
> > > > > > > >>        In the regrid option I have tried
> > > > > > > >>        to_grid = OBS
> > > > > > > >>        and to_grid =
> > > > > > > >>
> > > > > > > >>        I am also not quite sure of what I should have
in the
> > obs
> > > > > > > >> dictionary.
> > > > > > > >>
> > > > > > > >>        I have placed all of the above files on the
rap ftp
> > > > > > > >> site /incoming/irap/met_help/sherrie_data/
> > > > > > > >>
> > > > > > > >>        Thanks for your help again,
> > > > > > > >>        Sherrie Fredrick
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: segfault when running point stat
From: John Halley Gotway
Time: Thu Nov 07 14:45:11 2019

Great, glad to hear it.  Thanks for confirming.

John

On Thu, Nov 7, 2019 at 1:59 PM Sherrie Fredrick - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
>
> Hi John,
>
>     I read the documentation in Appendix C and was able to find
everything
> that I need in the cts file.
> It is really cool.  It prints out exactly the values I am looking
for.
> YAHOO!.
>
>    I think I am done bugging you with this.
>    Many Thanks,
>    Sherrie
>
> On Thu, Nov 7, 2019 at 11:27 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Sherrie,
> >
> > The output that point_stat writes is controlled by the
"output_flag"
> > settings in the config file.  Looking in your config file, I see
that you
> > are already requesting these output line types:
> >
> > output_flag = {
> > ...
> >    ctc    = BOTH;
> >    cts    = BOTH;
> > ...
> >
> > But you're not getting any CTC or CTS output lines.  And that's
because
> you
> > haven't specified any categorical thresholds to be applied.
You'll just
> > need to add those categorical threshold using the "cat_thresh"
entry in
> the
> > Point-Stat config file.  If you define "cat_thesh" at the top-
level of
> > context (as shown below), the same categorical thresholds will be
applied
> > to both the forecast and observation data.  Alternatively, you
could
> > specify cat_thresh separately inside the fcst and obs dictionaries
to
> > define them separately.
> >
> > Thanks,
> > John
> >
> > cat_thresh = [ >800, >900, >1000 ];
> >
> > fcst = {
> >   file_type = NETCDF_NCCF;
> >   field = [
> >      {
> >        name  = "snowd";
> >        level = ["(0,*,*)"];
> >      }
> >   ];
> > }
> > obs = {
> >   field = [
> >      {
> >        message_type = [ "ADPSFC" ];
> >        name  = "snowd";
> >        level = "L0";
> >      }
> >   ];
> > }
> >
> >
> >
> >
> > On Thu, Nov 7, 2019 at 11:06 AM Sherrie Fredrick - NOAA Affiliate
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> > >
> > > Hi John,
> > >
> > >      Yep that makes sense.  I have one other question.  I am
interested
> > in
> > > the MET verification measures described
> > > in Appendix C, section C.1.  The contingency table is exactly
what I am
> > > looking for.  I think these values are in the
> > > cts.txt file, but I am not sure of how to get them.  Could you
tell me
> > what
> > > if the cts file is the right one and what I should look for?  I
have
> only
> > > used the cnt.txt file in the past.
> > >
> > >    Many thanks,
> > >    Sherrie
> > >
> > > On Thu, Nov 7, 2019 at 10:53 AM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Sherrie,
> > > >
> > > > Sorry for the confusion.  There is a somewhat subtle detail
here.
> > > >
> > > > For gridded forecast files in NetCDF format, you set name =
"NetCDF
> > > > Variable Name"; and level = "Describe indices of that
variable".
> > > >
> > > > For point observation files (NetCDF output of pb2nc, ascii2nc,
and
> > > > madis2nc), you set name = "observation variable strings" which
are
> > listed
> > > > in the "obs_var" variable in that file.  For example, here
variable
> > names
> > > > present in some NetCDF output of pb2nc:
> > > >
> > > > ncdump -v obs_var gdas1.20120409.t12z.prepbufr.nc
> > > > ...
> > > > data:
> > > >
> > > >  obs_var =
> > > >   "SPFH",
> > > >   "TMP",
> > > >   "HGT",
> > > >   "UGRD",
> > > >   "VGRD",
> > > >   "DPT",
> > > >   "WIND",
> > > >   "RH",
> > > >   "MIXR" ;
> > > > }
> > > >
> > > > Make sense?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Thu, Nov 7, 2019 at 10:43 AM Sherrie Fredrick - NOAA
Affiliate via
> > RT
> > > <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985 >
> > > > >
> > > > > Hi John,
> > > > >      That worked the first time.  I spent hours trying to
get
> > something
> > > > to
> > > > > work.  So thank
> > > > > you very much.  I was not sure of what to put in for the obs
> > dictionary
> > > > > because the Netcdf
> > > > > file produced by the ascii2nc did not really have a variable
called
> > > snow
> > > > it
> > > > > just had float obs_val(nobs) ;
> > > > >
> > > > >     I know that both of these observations that you have
helped me
> > with
> > > > > will be used a lot.
> > > > >     Thanks,
> > > > >     Sherrie
> > > > >
> > > > > On Thu, Nov 7, 2019 at 9:50 AM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Sherrie,
> > > > > >
> > > > > > I've attached the Point-Stat config file I used.  And
here's how
> I
> > > > called
> > > > > > it on the command line:
> > > > > >
> > > > > >
> > > /d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat \
> > > > > > FV3file_MOD.nc4 oct2018_met.nc PointStatConfig.sherrie
-outdir
> out
> > > -v
> > > > 3
> > > > > > -log run_ps.log
> > > > > >
> > > > > > Perhaps the problem is how you set up the "obs" settings,
not the
> > > > "fcst"
> > > > > > ones.  Here's how those are set in the attached config
file:
> > > > > >
> > > > > > fcst = {
> > > > > >   file_type = NETCDF_NCCF;
> > > > > >   field = [
> > > > > >      {
> > > > > >        name  = "snowd";
> > > > > >        level = ["(0,*,*)"];
> > > > > >      }
> > > > > >   ];
> > > > > > }
> > > > > > obs = {
> > > > > >   field = [
> > > > > >      {
> > > > > >        message_type = [ "ADPSFC" ];
> > > > > >        name  = "snowd";
> > > > > >        level = "L0";
> > > > > >      }
> > > > > >   ];
> > > > > > }
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Thu, Nov 7, 2019 at 8:40 AM Sherrie Fredrick - NOAA
Affiliate
> > via
> > > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> >
> > > > > > >
> > > > > > > Hi John,
> > > > > > >
> > > > > > >      Here are the other things I have tried.
> > > > > > >     fcst = {
> > > > > > >    message_type = [ "ADPSFC" ];
> > > > > > >    level = ["(0,*,*)"];
> > > > > > >    field = [
> > > > > > >         {
> > > > > > >         name  = "snowd";
> > > > > > >        }
> > > > > > >    ];
> > > > > > > }
> > > > > > >
> > > > > > > fcst = {
> > > > > > >    message_type = [ "ADPSFC" ];
> > > > > > >
> > > > > > >    field = [
> > > > > > >         {
> > > > > > >         name  = "snowd";   level = "(0,*,*)";
> > > > > > >        }
> > > > > > >    ];
> > > > > > > }
> > > > > > >
> > > > > > > They all give me the same error.
> > > > > > > VarInfo::set_level_info_grib() - failed to parse level
string
> > > > '(0,*,*)'
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Sherrie
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Nov 6, 2019 at 4:26 PM John Halley Gotway via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Sherrie,
> > > > > > > >
> > > > > > > > FYI, I was able to find the line that produced the
segfault,
> > and
> > > I
> > > > > > > updated
> > > > > > > > the development version of the code.  In future
versions of
> > MET,
> > > > > you'll
> > > > > > > > this error message instead of the code segfaulting:
> > > > > > > >
> > > > > > > > ERROR  :
> > > > > > > > ERROR  : MetNcCFDataFile::collect_time_offsets(VarInfo
&) ->
> > > can't
> > > > > find
> > > > > > > > requested variable name "snodsfc" in NetCDF file
> "FV3file.nc4".
> > > > > > > > ERROR  :
> > > > > > > >
> > > > > > > > Please let us know if you ever run into another
segfault.
> They
> > > > > always
> > > > > > > need
> > > > > > > > to be fixed and replaced with a useful error message.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > > On Wed, Nov 6, 2019 at 3:50 PM John Halley Gotway <
> > > johnhg at ucar.edu
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Sherrie,
> > > > > > > > >
> > > > > > > > > Whenever getting started with new datasets, I always
start
> by
> > > > > running
> > > > > > > the
> > > > > > > > > plot_data_plane tool.  And I run plot_point_obs for
point
> > > > > > observations.
> > > > > > > > To
> > > > > > > > > that end, I ran the following commands which
produced that
> > > > attached
> > > > > > > > > images.  And this data looks fine to me:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_data_plane
> > > > > > > > > \FV3file.nc4 FV3file.ps 'name="snowd";
level="(*,*)";
> > > > > > > > > file_type=NETCDF_NCCF;'convert -rotate 90
-background white
> > > > > -flatten
> > > > > > > > > FV3file.ps
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> FV3file.png/d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/plot_point_obs
> > > > > > > > > \oct2018_met.nc <http://oct2018_met.nc>
oct2018_met.ps
> > > > > > > > > <http://oct2018_met.ps>convert -rotate 90
-background
> white
> > > > > -flatten
> > > > > > > > > oct2018_met.ps <http://oct2018_met.ps>
oct2018_met.png*
> > > > > > > > >
> > > > > > > > > But when I run Point-Stat, I get a segfault just
like you
> > did!
> > > > > > > Segfaults
> > > > > > > > > are never good, so we definitely need to fix that!
But
> let's
> > > > > figure
> > > > > > > out
> > > > > > > > > why.  I found a problem in your Point-Stat config
file.
> > You're
> > > > > > > > requesting
> > > > > > > > > data for:
> > > > > > > > >
> > > > > > > > > *   name  = "snodsfc";    level = ["(0,*,*)"];*
> > > > > > > > >
> > > > > > > > > But there are 2 problems here.  The FV3file.nc
contains a
> > > > variable
> > > > > > > named
> > > > > > > > > "snowd" and not "snodsfc" and it has 2 dimensions,
not 3.
> > So I
> > > > > > > switched
> > > > > > > > to
> > > > > > > > > using:
> > > > > > > > >
> > > > > > > > > *   name  = "snowd";    level = ["(*,*)"];*
> > > > > > > > >
> > > > > > > > > That gets past the segfault, but we now have this
error:
> > > > > > > > >
> > > > > > > > > *WARNING:
MetNcCFDataFile::collect_time_offsets(VarInfo &)
> ->
> > > Not
> > > > > > found
> > > > > > > > > time out of 1. ERROR  :
> > > > > MetNcCFDataFile::collect_time_offsets(VarInfo
> > > > > > > &)
> > > > > > > > ->
> > > > > > > > > variable "snowd" has unknown error (999).*
> > > > > > > > >
> > > > > > > > > So there's a problem with the timing info.  Looking
in the
> > > > > > > FV3file.nc4, I
> > > > > > > > > see that you have a time dimension defined, but it
is not
> > > > included
> > > > > > as a
> > > > > > > > > dimension for the data variable: *float snowd(lat,
lon) ;*
> > > > > > > > >
> > > > > > > > > MET is definitely expecting the time variable to be
> included.
> > > I
> > > > > used
> > > > > > > > > ncdump and ncgen to modify your file to add the time
> > dimension:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *ncdump FV3file.nc4 > FV3file.nc4dump*
> > > > > > > > > *# Edit FV3file.nc4dump file by adding time dim to
snowd
> > > > variable*
> > > > > > > > >
> > > > > > > > > *ncgen FV3file.nc4dump -o FV3file_MOD.nc4*
> > > > > > > > > And then I modified your config file and reran
Point-Stat
> > using
> > > > the
> > > > > > > > > updated file:
> > > > > > > > >
> > > > > > > > > *   name  = "snowd";    level = ["(0,*,*)"];*
> > > > > > > > >
> > > > > > > > >
> > > > > >
> > > */d1/johnhg/MET/MET_development/git/MET-
master_v8.1/met/bin/point_stat
> > > > > > > \*
> > > > > > > > > *FV3file_MOD.nc4 oct2018_met.nc
<http://oct2018_met.nc>
> > > > > > > > > PointStatConfig.sherrie -outdir out -v 3 -log
run_ps.log*
> > > > > > > > >
> > > > > > > > > That ran but gave me 0 matched pairs because of a
mismatch
> of
> > > > "obs
> > > > > > > type":
> > > > > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > > DEBUG 3: Rejected: obs type       = 47880
> > > > > > > > >
> > > > > > > > > I used ncdump to figure out that your observations
are
> named
> > > > > "snowd":
> > > > > > > > > ncdump -v obs_var oct2018_met.nc
> > > > > > > > >  obs_var =
> > > > > > > > >  "snowd",
> > > > > > > > >  "66" ;
> > > > > > > > >
> > > > > > > > > So I modified your config file to request "snowd"
> > observations
> > > be
> > > > > > used:
> > > > > > > > >
> > > > > > > > > *        //name  = "SNOD";        name  = "snowd";*
> > > > > > > > >
> > > > > > > > > And I finally got 47,415 matched pairs:
> > > > > > > > > DEBUG 3: Number of matched pairs  = 47415
> > > > > > > > > DEBUG 3: Observations processed   = 47880
> > > > > > > > > DEBUG 3: Rejected: SID exclusion  = 0
> > > > > > > > > DEBUG 3: Rejected: obs type       = 465
> > > > > > > > >
> > > > > > > > > But what about those other 465 ones?  Having obs_var
= "66"
> > > looks
> > > > > > > > > suspicious to me.  So I looked upstream as the ascii
file
> you
> > > > sent.
> > > > > > I
> > > > > > > > > contains 47,880 lines:
> > > > > > > > > cat oct2018_met.ascii | wc -l
> > > > > > > > > 47880
> > > > > > > > >
> > > > > > > > > For some reason, this data files looks to be a
little bit
> > > > > corrupted.
> > > > > > > In
> > > > > > > > > the second column, where there should be station
names, I
> see
> > > the
> > > > > > > "snowd"
> > > > > > > > > string getting somewhat randomly embedded in there:
> > > > > "GME001219snowd".
> > > > > > > > > And that happens in other columns too.  In fact,
you'd
> expect
> > > > > "snowd"
> > > > > > > to
> > > > > > > > > show up 47,880 times (once per line).  But it
appears in
> this
> > > > file
> > > > > > > 55,896
> > > > > > > > > times.  My guess is that this is the result of some
> > > > over-aggressive
> > > > > > use
> > > > > > > > of
> > > > > > > > > "sed".  So I'd suggest taking a look at that.
> > > > > > > > >
> > > > > > > > > Hope that helps.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Nov 6, 2019 at 10:38 AM Sherrie Fredrick -
NOAA
> > > Affiliate
> > > > > via
> > > > > > > RT
> > > > > > > > <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > >>
> > > > > > > > >> Wed Nov 06 10:38:51 2019: Request 92985 was acted
upon.
> > > > > > > > >> Transaction: Ticket created by
sherrie.fredrick at noaa.gov
> > > > > > > > >>        Queue: met_help
> > > > > > > > >>      Subject: segfault when running point stat
> > > > > > > > >>        Owner: Nobody
> > > > > > > > >>   Requestors: sherrie.fredrick at noaa.gov
> > > > > > > > >>       Status: new
> > > > > > > > >>  Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92985
> > > > > > > >
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> Good Morning,
> > > > > > > > >>
> > > > > > > > >>      I am trying to run point_stat.  I have the
following.
> > > > > > > > >>      FV3file.nc4  - a file from running the FV3
model with
> > > only
> > > > > one
> > > > > > > > field
> > > > > > > > >>                            snow depth.
> > > > > > > > >>      oct2018_met.ascii  - an ascii file with
station
> > > > information,
> > > > > > lat,
> > > > > > > > lon
> > > > > > > > >> and
> > > > > > > > >>                                       snowd values.
> > > > > > > > >>      oct2018_met.nc     - netcdf file that I got
from
> > running
> > > > > > > ascii2nc
> > > > > > > > on
> > > > > > > > >> the
> > > > > > > > >>
oct2018_met.ascii
> > file.
> > > > > > > > >>
> > > > > > > > >>       PointStatConfig.sherrie - mypoint stat config
> file.  I
> > > am
> > > > > not
> > > > > > > sure
> > > > > > > > >> of
> > > > > > > > >> what
> > > > > > > > >>        I should have in the config file.
> > > > > > > > >>        In the regrid option I have tried
> > > > > > > > >>        to_grid = OBS
> > > > > > > > >>        and to_grid =
> > > > > > > > >>
> > > > > > > > >>        I am also not quite sure of what I should
have in
> the
> > > obs
> > > > > > > > >> dictionary.
> > > > > > > > >>
> > > > > > > > >>        I have placed all of the above files on the
rap ftp
> > > > > > > > >> site /incoming/irap/met_help/sherrie_data/
> > > > > > > > >>
> > > > > > > > >>        Thanks for your help again,
> > > > > > > > >>        Sherrie Fredrick
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------