[Met_help] [rt.rap.ucar.edu #96501] History for core dump wtih from gen_vx_mask and grid_stat
John Halley Gotway via RT
met_help at ucar.edu
Tue Sep 1 11:22:56 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Folks - I have noticed that if I *remapbil,r360x181* a GRIB file from 0.25 deg to 1.0 deg (using cdo tool), both gen_vx_mask and grid_stat core dump. I was wondering if this has happened before. Any help would be greatly appreciated. Thanks!
Efren A. Serra (Contractor)
Physicist
DeVine Consulting, Inc.
Naval Research Laboratory
Marine Meteorology Division
7 Grace Hopper Ave., STOP 2
Monterey, CA 93943
Code 7542
Mobile: 408-425-5027
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: core dump wtih from gen_vx_mask and grid_stat
From: John Halley Gotway
Time: Mon Aug 31 13:38:57 2020
Hi Efren,
Well, core dumps are never an acceptable outcome. Whenever that
happens, it
indicates there's a problem that should be fixed. We should enhance
the MET
tools to do more error checking and exit with a useful error message
rather
than core dumping. Of course, if the core is actually coming from one
of
the dependent libraries (like NetCDF or something), then we have less
control over it.
If possible please send me the command you ran to generate the core
dump
and then post the file that caused it to our anonymous ftp site
following
these directions:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
Hopefully I'll be able to replicate this behavior and figure out
what's
going on.
Thanks,
John Halley Gotway
On Fri, Aug 28, 2020 at 5:07 PM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:
>
> Fri Aug 28 17:07:48 2020: Request 96501 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: core dump wtih from gen_vx_mask and grid_stat
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96501 >
>
>
> Folks - I have noticed that if I *remapbil,r360x181* a GRIB file
from 0.25
> deg to 1.0 deg (using cdo tool), both gen_vx_mask and grid_stat core
dump.
> I was wondering if this has happened before. Any help would be
greatly
> appreciated. Thanks!
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: core dump wtih from gen_vx_mask and grid_stat
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon Aug 31 17:40:00 2020
John - I shall do that; what's your anonymous ftp site. Here is a
little of context on the core dump. Attached are a log file and the
GridStatConfig file.
1] command line
grid_stat /omar_backup/leo_backup/serra/data_repos/ww3tcofcl-
evaluation/dynamic/2018092900/WW3NAVGEM_prb/US058GOCN-
GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
/omar_backup/leo_backup/serra/data_repos/ww3tcofcl-
evaluation/dynamic/2018100400/WW3TCOFCL_det/US058GOCN-
GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-000000sig_wav_ht
/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/WW3NAVGEM-
WW3TCOFCL_fcst_cat_thresh_10/GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120 -outdir
/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/WW3NAVGEM-
WW3TCOFCL_fcst_cat_thresh_10 -v 4
2] probability field grid_stat
/omar_backup/leo_backup/serra/data_repos/ww3tcofcl-
evaluation/dynamic/2018092900/WW3NAVGEM_prb/US058GOCN-
GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
Wgrib -V output:
Undefined parameter table (center 58-0 table 3), using NCEP-opn
rec 1:0:date 2018092900 MFLX kpds5=172 kpds6=1 kpds7=0 levels=(0,0)
grid=240 sfc 120hr fcst: bitmap: 29430 undef
MFLX=Momentum flux [N/m^2]
timerange 0 P1 120 P2 0 TimeU 1 nx 360 ny 181 GDS grid 0 num_in_ave
0 missing 0
center 58 subcenter 0 process 50 Table 3 scan: WE:SN winds(N/S)
latlon: lat -90.000000 to 90.000000 by 1.000000 nxny 65160
long 0.000000 to -1.000000 by 1.000000, (360 x 181) scan 64
mode 128 bdsgrid 1
min/max data 0 1 num bits 7 BDS_Ref 0 DecScale 2 BinScale 0
3] observation field
/omar_backup/leo_backup/serra/data_repos/ww3tcofcl-
evaluation/dynamic/2018100400/WW3TCOFCL_det/US058GOCN-
GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-000000sig_wav_ht
Wgrib -V output:
rec 1:0:date 2018100400 HTSGW kpds5=100 kpds6=1 kpds7=0 levels=(0,0)
grid=255 sfc anl: bitmap: 29874 undef
HTSGW=Sig height of wind waves and swell [m]
timerange 0 P1 0 P2 0 TimeU 1 nx 360 ny 180 GDS grid 0 num_in_ave 0
missing 0
center 58 subcenter 0 process 95 Table 3 scan: WE:SN winds(N/S)
latlon: lat -89.500000 to 89.500000 by 1.000000 nxny 64800
long 0.000000 to 359.000000 by 1.000000, (360 x 180) scan 64
mode 128 bdsgrid 1
min/max data 0 13.87 num bits 18 BDS_Ref 0 DecScale 0 BinScale
-14
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 31, 2020 12:39 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and grid_stat
Hi Efren,
Well, core dumps are never an acceptable outcome. Whenever that
happens, it indicates there's a problem that should be fixed. We
should enhance the MET tools to do more error checking and exit with a
useful error message rather than core dumping. Of course, if the core
is actually coming from one of the dependent libraries (like NetCDF or
something), then we have less control over it.
If possible please send me the command you ran to generate the core
dump and then post the file that caused it to our anonymous ftp site
following these directions:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
Hopefully I'll be able to replicate this behavior and figure out
what's going on.
Thanks,
John Halley Gotway
On Fri, Aug 28, 2020 at 5:07 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> Fri Aug 28 17:07:48 2020: Request 96501 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: core dump wtih from gen_vx_mask and grid_stat
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96501
> >
>
>
> Folks - I have noticed that if I *remapbil,r360x181* a GRIB file
from
> 0.25 deg to 1.0 deg (using cdo tool), both gen_vx_mask and grid_stat
core dump.
> I was wondering if this has happened before. Any help would be
greatly
> appreciated. Thanks!
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask and grid_stat
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon Aug 31 17:40:25 2020
I'm reading about the anonymous ftp; sorry!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 31, 2020 12:39 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and grid_stat
Hi Efren,
Well, core dumps are never an acceptable outcome. Whenever that
happens, it indicates there's a problem that should be fixed. We
should enhance the MET tools to do more error checking and exit with a
useful error message rather than core dumping. Of course, if the core
is actually coming from one of the dependent libraries (like NetCDF or
something), then we have less control over it.
If possible please send me the command you ran to generate the core
dump and then post the file that caused it to our anonymous ftp site
following these directions:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
Hopefully I'll be able to replicate this behavior and figure out
what's going on.
Thanks,
John Halley Gotway
On Fri, Aug 28, 2020 at 5:07 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> Fri Aug 28 17:07:48 2020: Request 96501 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: core dump wtih from gen_vx_mask and grid_stat
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96501
> >
>
>
> Folks - I have noticed that if I *remapbil,r360x181* a GRIB file
from
> 0.25 deg to 1.0 deg (using cdo tool), both gen_vx_mask and grid_stat
core dump.
> I was wondering if this has happened before. Any help would be
greatly
> appreciated. Thanks!
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask and grid_stat
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Sep 01 08:03:23 2020
John - Files are in my ftp directory; listing below:
ftp> ls -lrt
227 Entering Passive Mode (128,117,14,132,196,154).
150 Opening ASCII mode data connection for file list
-rw-r--r-- 1 ftp ftp 39538 Sep 1 14:02 US058GOCN-
GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
-rw-r--r-- 1 ftp ftp 86776 Sep 1 14:02 US058GOCN-
GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-000000sig_wav_ht
-rw-r--r-- 1 ftp ftp 270598 Sep 1 14:02 wp302018-
2018100400.nc
-rw-r--r-- 1 ftp ftp 3254 Sep 1 14:02
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
226 Transfer complete
ftp> pwd
257 "/incoming/irap/met_help/serra" is the current directory
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, August 31, 2020 12:39 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and grid_stat
Hi Efren,
Well, core dumps are never an acceptable outcome. Whenever that
happens, it indicates there's a problem that should be fixed. We
should enhance the MET tools to do more error checking and exit with a
useful error message rather than core dumping. Of course, if the core
is actually coming from one of the dependent libraries (like NetCDF or
something), then we have less control over it.
If possible please send me the command you ran to generate the core
dump and then post the file that caused it to our anonymous ftp site
following these directions:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
Hopefully I'll be able to replicate this behavior and figure out
what's going on.
Thanks,
John Halley Gotway
On Fri, Aug 28, 2020 at 5:07 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> Fri Aug 28 17:07:48 2020: Request 96501 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: core dump wtih from gen_vx_mask and grid_stat
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96501
> >
>
>
> Folks - I have noticed that if I *remapbil,r360x181* a GRIB file
from
> 0.25 deg to 1.0 deg (using cdo tool), both gen_vx_mask and grid_stat
core dump.
> I was wondering if this has happened before. Any help would be
greatly
> appreciated. Thanks!
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: core dump wtih from gen_vx_mask and grid_stat
From: John Halley Gotway
Time: Tue Sep 01 09:52:15 2020
Efren,
Thanks for sending your sample data and commands. I see from your
configuration file that you're using MET version 8.0 from Sept 2018.
We
released MET version 9.1 this August. I'll start by debugging with
version
8.0 (plus all posted bugfixes) to see if I can replicate the problem.
If I
can, I'll then check to see if the problem still exists in version
9.1.
I started by running the plot_data_plane tool (version 9.1) on both
input
datasets:
*plot_data_plane
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
sig_wav_ht.ps <http://sig_wav_ht.ps> 'name="HTSGW"; level="L0";'*
*plot_data_plane
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
prob_sig_wav_ht_gt12ft.ps <http://prob_sig_wav_ht_gt12ft.ps>
'name="MFLX";
level="L0";'*
That created the attached images, which look fine to me. Always a good
idea
to start with plot_data_plane when working with new data to make sure
MET
is plotting the data correctly and in the expected location.
When I run met-8.0 grid_stat with the data you sent I get an error
message
about the grids not matching:
*met-8.0_bugfix/bin/grid_stat
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
-outdir out -v 3 -log run_gs.log*
*DEBUG 1: Default Config File:
/Volumes/d1/projects/MET/MET_releases/met-
8.0_bugfix/share/met/config/GridStatConfig_default*
*DEBUG 1: User Config File:
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120*
*ERROR : *
*ERROR : parse_vx_grid() -> The forecast and observation grids do not
match:*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll: -90.000 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000 !=*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 180 lat_ll: -89.500 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000*
*ERROR : Specify regridding logic in the config file "regrid"
section.*
*ERROR : *
So I modified the config file to regrid to the "OBS" field
(regrid.to_grid
= OBS;). And then grid_stat ran to completion without an error.
However, I
do have a few suggestions:
(1) I'd recommend upgrading to met-9.1 if/when that is possible.
(2) In you config file, setting model = "WW3NAVGEM"; in the "fcst"
dictionary and model = "WW3TCOFCL"; in the "obs" dictionary will have
no
effect on the output. Instead, I'd recommend setting the "model" and
"obtype" settings at the beginning of the Grid-Stat config file to
populate
the MODEL and OBTYPE columns in the output.
(3) Listed below are your "mask" settings which define the
verification
regions over which statistics should be computed:
mask = { grid = [""]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
The grid and poly options are arrays. I suspect that you want 0 grid
entries, but you actually currently have 1... but it's set to an empty
string. Try specifying it like this instead:
mask = { grid = [ ]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
And that should only produce output for that single polyline that you
defined.
(4) Looking in wp302018-2018100400.nc, I see that the variable is
named
"box_mask":
float box_mask(lat, lon) ;
And that causes the VX_MASK column to be populated with "box_mask" in
the
output. I'd recommend choosing a more descriptive name for that
region.
When you run gen_vx_mask, you can add the "-name" command line option
to
define the name of the output variable. And then whatever name you
choose
will show up in the VX_MASK column of the Grid-Stat output.
Perhaps changing the mask.grid setting or upgrading to MET version 9.1
will
resolve the behavior you're experiencing. If not, please let me know
and I
can try to help you debug this more remotely.
Thanks,
John
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask and grid_stat
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Sep 01 10:45:41 2020
John - I requested that MET 9.1 be installed; I'm waiting for this and
I shall let you know ASAP. Thanks!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, September 1, 2020 8:52 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and grid_stat
Efren,
Thanks for sending your sample data and commands. I see from your
configuration file that you're using MET version 8.0 from Sept 2018.
We released MET version 9.1 this August. I'll start by debugging with
version
8.0 (plus all posted bugfixes) to see if I can replicate the problem.
If I can, I'll then check to see if the problem still exists in
version 9.1.
I started by running the plot_data_plane tool (version 9.1) on both
input
datasets:
*plot_data_plane
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
sig_wav_ht.ps <http://sig_wav_ht.ps> 'name="HTSGW"; level="L0";'*
*plot_data_plane
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
prob_sig_wav_ht_gt12ft.ps <http://prob_sig_wav_ht_gt12ft.ps>
'name="MFLX";
level="L0";'*
That created the attached images, which look fine to me. Always a good
idea to start with plot_data_plane when working with new data to make
sure MET is plotting the data correctly and in the expected location.
When I run met-8.0 grid_stat with the data you sent I get an error
message about the grids not matching:
*met-8.0_bugfix/bin/grid_stat
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
-outdir out -v 3 -log run_gs.log*
*DEBUG 1: Default Config File:
/Volumes/d1/projects/MET/MET_releases/met-
8.0_bugfix/share/met/config/GridStatConfig_default*
*DEBUG 1: User Config File:
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120*
*ERROR : *
*ERROR : parse_vx_grid() -> The forecast and observation grids do not
match:*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll: -90.000 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000 !=*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 180 lat_ll: -89.500 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000*
*ERROR : Specify regridding logic in the config file "regrid"
section.*
*ERROR : *
So I modified the config file to regrid to the "OBS" field
(regrid.to_grid = OBS;). And then grid_stat ran to completion without
an error. However, I do have a few suggestions:
(1) I'd recommend upgrading to met-9.1 if/when that is possible.
(2) In you config file, setting model = "WW3NAVGEM"; in the "fcst"
dictionary and model = "WW3TCOFCL"; in the "obs" dictionary will have
no effect on the output. Instead, I'd recommend setting the "model"
and "obtype" settings at the beginning of the Grid-Stat config file to
populate the MODEL and OBTYPE columns in the output.
(3) Listed below are your "mask" settings which define the
verification regions over which statistics should be computed:
mask = { grid = [""]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
The grid and poly options are arrays. I suspect that you want 0 grid
entries, but you actually currently have 1... but it's set to an empty
string. Try specifying it like this instead:
mask = { grid = [ ]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
And that should only produce output for that single polyline that you
defined.
(4) Looking in wp302018-2018100400.nc, I see that the variable is
named
"box_mask":
float box_mask(lat, lon) ;
And that causes the VX_MASK column to be populated with "box_mask" in
the output. I'd recommend choosing a more descriptive name for that
region.
When you run gen_vx_mask, you can add the "-name" command line option
to define the name of the output variable. And then whatever name you
choose will show up in the VX_MASK column of the Grid-Stat output.
Perhaps changing the mask.grid setting or upgrading to MET version 9.1
will resolve the behavior you're experiencing. If not, please let me
know and I can try to help you debug this more remotely.
Thanks,
John
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask and grid_stat
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Sep 01 10:48:45 2020
Also, the reason I had 360x180 in the probability file is that I
noticed that it fixed the core dump when I was using gen_vx_mask to
create a mask file. The 360x180 probability field I obtain via cdo
command line "cdo -f grb1 remapbil,r360x180 ..." I think upgrading to
MET v 9.1 is the ticket. Let me do that, but let's keep this issue
open. Thanks!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, September 1, 2020 8:52 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and grid_stat
Efren,
Thanks for sending your sample data and commands. I see from your
configuration file that you're using MET version 8.0 from Sept 2018.
We released MET version 9.1 this August. I'll start by debugging with
version
8.0 (plus all posted bugfixes) to see if I can replicate the problem.
If I can, I'll then check to see if the problem still exists in
version 9.1.
I started by running the plot_data_plane tool (version 9.1) on both
input
datasets:
*plot_data_plane
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
sig_wav_ht.ps <http://sig_wav_ht.ps> 'name="HTSGW"; level="L0";'*
*plot_data_plane
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
prob_sig_wav_ht_gt12ft.ps <http://prob_sig_wav_ht_gt12ft.ps>
'name="MFLX";
level="L0";'*
That created the attached images, which look fine to me. Always a good
idea to start with plot_data_plane when working with new data to make
sure MET is plotting the data correctly and in the expected location.
When I run met-8.0 grid_stat with the data you sent I get an error
message about the grids not matching:
*met-8.0_bugfix/bin/grid_stat
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
-outdir out -v 3 -log run_gs.log*
*DEBUG 1: Default Config File:
/Volumes/d1/projects/MET/MET_releases/met-
8.0_bugfix/share/met/config/GridStatConfig_default*
*DEBUG 1: User Config File:
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120*
*ERROR : *
*ERROR : parse_vx_grid() -> The forecast and observation grids do not
match:*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll: -90.000 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000 !=*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 180 lat_ll: -89.500 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000*
*ERROR : Specify regridding logic in the config file "regrid"
section.*
*ERROR : *
So I modified the config file to regrid to the "OBS" field
(regrid.to_grid = OBS;). And then grid_stat ran to completion without
an error. However, I do have a few suggestions:
(1) I'd recommend upgrading to met-9.1 if/when that is possible.
(2) In you config file, setting model = "WW3NAVGEM"; in the "fcst"
dictionary and model = "WW3TCOFCL"; in the "obs" dictionary will have
no effect on the output. Instead, I'd recommend setting the "model"
and "obtype" settings at the beginning of the Grid-Stat config file to
populate the MODEL and OBTYPE columns in the output.
(3) Listed below are your "mask" settings which define the
verification regions over which statistics should be computed:
mask = { grid = [""]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
The grid and poly options are arrays. I suspect that you want 0 grid
entries, but you actually currently have 1... but it's set to an empty
string. Try specifying it like this instead:
mask = { grid = [ ]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
And that should only produce output for that single polyline that you
defined.
(4) Looking in wp302018-2018100400.nc, I see that the variable is
named
"box_mask":
float box_mask(lat, lon) ;
And that causes the VX_MASK column to be populated with "box_mask" in
the output. I'd recommend choosing a more descriptive name for that
region.
When you run gen_vx_mask, you can add the "-name" command line option
to define the name of the output variable. And then whatever name you
choose will show up in the VX_MASK column of the Grid-Stat output.
Perhaps changing the mask.grid setting or upgrading to MET version 9.1
will resolve the behavior you're experiencing. If not, please let me
know and I can try to help you debug this more remotely.
Thanks,
John
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask and grid_stat
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Sep 01 10:56:52 2020
John - My IT guy (he's my personal IT guy) updated MET tools to v
9.0.2 and I used the lower resolution sig_wav_ht field at r360x181
(same resolution as the probability field) and it *DID NOT* core dump.
Freaking beautiful Should we keep v9.0.2 or push v9.1. Thanks for the
help mate, Buck is gonna be happy!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, September 1, 2020 8:52 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and grid_stat
Efren,
Thanks for sending your sample data and commands. I see from your
configuration file that you're using MET version 8.0 from Sept 2018.
We released MET version 9.1 this August. I'll start by debugging with
version
8.0 (plus all posted bugfixes) to see if I can replicate the problem.
If I can, I'll then check to see if the problem still exists in
version 9.1.
I started by running the plot_data_plane tool (version 9.1) on both
input
datasets:
*plot_data_plane
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
sig_wav_ht.ps <http://sig_wav_ht.ps> 'name="HTSGW"; level="L0";'*
*plot_data_plane
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
prob_sig_wav_ht_gt12ft.ps <http://prob_sig_wav_ht_gt12ft.ps>
'name="MFLX";
level="L0";'*
That created the attached images, which look fine to me. Always a good
idea to start with plot_data_plane when working with new data to make
sure MET is plotting the data correctly and in the expected location.
When I run met-8.0 grid_stat with the data you sent I get an error
message about the grids not matching:
*met-8.0_bugfix/bin/grid_stat
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
-outdir out -v 3 -log run_gs.log*
*DEBUG 1: Default Config File:
/Volumes/d1/projects/MET/MET_releases/met-
8.0_bugfix/share/met/config/GridStatConfig_default*
*DEBUG 1: User Config File:
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120*
*ERROR : *
*ERROR : parse_vx_grid() -> The forecast and observation grids do not
match:*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll: -90.000 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000 !=*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 180 lat_ll: -89.500 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000*
*ERROR : Specify regridding logic in the config file "regrid"
section.*
*ERROR : *
So I modified the config file to regrid to the "OBS" field
(regrid.to_grid = OBS;). And then grid_stat ran to completion without
an error. However, I do have a few suggestions:
(1) I'd recommend upgrading to met-9.1 if/when that is possible.
(2) In you config file, setting model = "WW3NAVGEM"; in the "fcst"
dictionary and model = "WW3TCOFCL"; in the "obs" dictionary will have
no effect on the output. Instead, I'd recommend setting the "model"
and "obtype" settings at the beginning of the Grid-Stat config file to
populate the MODEL and OBTYPE columns in the output.
(3) Listed below are your "mask" settings which define the
verification regions over which statistics should be computed:
mask = { grid = [""]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
The grid and poly options are arrays. I suspect that you want 0 grid
entries, but you actually currently have 1... but it's set to an empty
string. Try specifying it like this instead:
mask = { grid = [ ]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
And that should only produce output for that single polyline that you
defined.
(4) Looking in wp302018-2018100400.nc, I see that the variable is
named
"box_mask":
float box_mask(lat, lon) ;
And that causes the VX_MASK column to be populated with "box_mask" in
the output. I'd recommend choosing a more descriptive name for that
region.
When you run gen_vx_mask, you can add the "-name" command line option
to define the name of the output variable. And then whatever name you
choose will show up in the VX_MASK column of the Grid-Stat output.
Perhaps changing the mask.grid setting or upgrading to MET version 9.1
will resolve the behavior you're experiencing. If not, please let me
know and I can try to help you debug this more remotely.
Thanks,
John
------------------------------------------------
Subject: core dump wtih from gen_vx_mask and grid_stat
From: John Halley Gotway
Time: Tue Sep 01 11:08:17 2020
Great, glad to hear that upgrading to met-9.0.2 did the trick! There
actually was a met-9.0.3 bugfix release with release notes here (
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
version-9-0-3#notes
).
As for updating to met-9.1, it's really up to you. Here's a link to
the
met-9.1 release notes. We've switched to using GitHub pages for the
met-9.1
release:
https://dtcenter.github.io/MET/Users_Guide/release-notes.html
So you can review those changes and see if anything jumps out at you
as
being useful. If updating to met-9.1 is easy enough, I'd recommend
doing so.
Thanks,
John
On Tue, Sep 1, 2020 at 10:57 AM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96501 >
>
> John - My IT guy (he's my personal IT guy) updated MET tools to v
9.0.2
> and I used the lower resolution sig_wav_ht field at r360x181 (same
> resolution as the probability field) and it *DID NOT* core dump.
Freaking
> beautiful Should we keep v9.0.2 or push v9.1. Thanks for the help
mate,
> Buck is gonna be happy!
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, September 1, 2020 8:52 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from
gen_vx_mask and
> grid_stat
>
> Efren,
>
> Thanks for sending your sample data and commands. I see from your
> configuration file that you're using MET version 8.0 from Sept 2018.
We
> released MET version 9.1 this August. I'll start by debugging with
version
> 8.0 (plus all posted bugfixes) to see if I can replicate the
problem. If I
> can, I'll then check to see if the problem still exists in version
9.1.
>
> I started by running the plot_data_plane tool (version 9.1) on both
input
> datasets:
>
> *plot_data_plane
> US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
> sig_wav_ht.ps <http://sig_wav_ht.ps> 'name="HTSGW"; level="L0";'*
>
> *plot_data_plane
>
> US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
> prob_sig_wav_ht_gt12ft.ps <http://prob_sig_wav_ht_gt12ft.ps>
'name="MFLX";
> level="L0";'*
>
>
> That created the attached images, which look fine to me. Always a
good
> idea to start with plot_data_plane when working with new data to
make sure
> MET is plotting the data correctly and in the expected location.
>
>
> When I run met-8.0 grid_stat with the data you sent I get an error
message
> about the grids not matching:
>
>
> *met-8.0_bugfix/bin/grid_stat
>
> US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
> US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
>
> GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
> -outdir out -v 3 -log run_gs.log*
>
> *DEBUG 1: Default Config File:
>
> /Volumes/d1/projects/MET/MET_releases/met-
8.0_bugfix/share/met/config/GridStatConfig_default*
>
> *DEBUG 1: User Config File:
>
> GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120*
>
> *ERROR : *
>
> *ERROR : parse_vx_grid() -> The forecast and observation grids do
not
> match:*
>
> *ERROR : Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll: -90.000
lon_ll:
> -0.000 delta_lat: 1.000 delta_lon: 1.000 !=*
>
> *ERROR : Projection: Lat/Lon Nx: 360 Ny: 180 lat_ll: -89.500
lon_ll:
> -0.000 delta_lat: 1.000 delta_lon: 1.000*
>
> *ERROR : Specify regridding logic in the config file "regrid"
section.*
>
> *ERROR : *
>
>
> So I modified the config file to regrid to the "OBS" field
(regrid.to_grid
> = OBS;). And then grid_stat ran to completion without an error.
However, I
> do have a few suggestions:
>
>
> (1) I'd recommend upgrading to met-9.1 if/when that is possible.
>
>
> (2) In you config file, setting model = "WW3NAVGEM"; in the "fcst"
> dictionary and model = "WW3TCOFCL"; in the "obs" dictionary will
have no
> effect on the output. Instead, I'd recommend setting the "model" and
> "obtype" settings at the beginning of the Grid-Stat config file to
populate
> the MODEL and OBTYPE columns in the output.
>
>
> (3) Listed below are your "mask" settings which define the
verification
> regions over which statistics should be computed:
>
>
> mask = { grid = [""]; poly =
> ["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
> wp302018-2018100400.nc"]; }
>
>
> The grid and poly options are arrays. I suspect that you want 0 grid
> entries, but you actually currently have 1... but it's set to an
empty
> string. Try specifying it like this instead:
>
>
> mask = { grid = [ ]; poly =
> ["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
> wp302018-2018100400.nc"]; }
>
>
> And that should only produce output for that single polyline that
you
> defined.
>
>
> (4) Looking in wp302018-2018100400.nc, I see that the variable is
named
> "box_mask":
>
> float box_mask(lat, lon) ;
>
>
> And that causes the VX_MASK column to be populated with "box_mask"
in the
> output. I'd recommend choosing a more descriptive name for that
region.
> When you run gen_vx_mask, you can add the "-name" command line
option to
> define the name of the output variable. And then whatever name you
choose
> will show up in the VX_MASK column of the Grid-Stat output.
>
>
> Perhaps changing the mask.grid setting or upgrading to MET version
9.1
> will resolve the behavior you're experiencing. If not, please let me
know
> and I can try to help you debug this more remotely.
>
>
> Thanks,
> John
>
>
>
>
------------------------------------------------
Subject: core dump wtih from gen_vx_mask and grid_stat
From: Sampson, Mr. Buck
Time: Tue Sep 01 11:14:46 2020
Thanks John. We heart MET.
Buck
-----Original Message-----
From: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Sent: Tuesday, September 1, 2020 9:57 AM
To: met_help at ucar.edu
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and
grid_stat
John - My IT guy (he's my personal IT guy) updated MET tools to v
9.0.2 and I
used the lower resolution sig_wav_ht field at r360x181 (same
resolution as the
probability field) and it *DID NOT* core dump. Freaking beautiful
Should we
keep v9.0.2 or push v9.1. Thanks for the help mate, Buck is gonna be
happy!
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, September 1, 2020 8:52 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #96501] core dump wtih from gen_vx_mask
and
grid_stat
Efren,
Thanks for sending your sample data and commands. I see from your
configuration file that you're using MET version 8.0 from Sept 2018.
We
released MET version 9.1 this August. I'll start by debugging with
version
8.0 (plus all posted bugfixes) to see if I can replicate the problem.
If I
can, I'll then check to see if the problem still exists in version
9.1.
I started by running the plot_data_plane tool (version 9.1) on both
input
datasets:
*plot_data_plane
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
sig_wav_ht.ps <http://sig_wav_ht.ps> 'name="HTSGW"; level="L0";'*
*plot_data_plane
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
prob_sig_wav_ht_gt12ft.ps <http://prob_sig_wav_ht_gt12ft.ps>
'name="MFLX";
level="L0";'*
That created the attached images, which look fine to me. Always a good
idea to
start with plot_data_plane when working with new data to make sure MET
is
plotting the data correctly and in the expected location.
When I run met-8.0 grid_stat with the data you sent I get an error
message
about the grids not matching:
*met-8.0_bugfix/bin/grid_stat
US058GOCN-GR1mdl.0050_0240_12000U0RL2018092900_0001_000000-
000000prob_sig_wav_ht_gt12ft
US058GOCN-GR1mdl.0095_0200_00000F0RL2018100400_0001_000000-
000000sig_wav_ht
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120
-outdir out -v 3 -log run_gs.log*
*DEBUG 1: Default Config File:
/Volumes/d1/projects/MET/MET_releases/met-
8.0_bugfix/share/met/config/GridStatConfig_default*
*DEBUG 1: User Config File:
GridStatConfig_wp302018_gt12ft_WW3NAVGEM-
WW3TCOFCL.2018100400_2018092900-120*
*ERROR : *
*ERROR : parse_vx_grid() -> The forecast and observation grids do not
match:*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll: -90.000 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000 !=*
*ERROR : Projection: Lat/Lon Nx: 360 Ny: 180 lat_ll: -89.500 lon_ll:
-0.000 delta_lat: 1.000 delta_lon: 1.000*
*ERROR : Specify regridding logic in the config file "regrid"
section.*
*ERROR : *
So I modified the config file to regrid to the "OBS" field
(regrid.to_grid =
OBS;). And then grid_stat ran to completion without an error.
However, I do
have a few suggestions:
(1) I'd recommend upgrading to met-9.1 if/when that is possible.
(2) In you config file, setting model = "WW3NAVGEM"; in the "fcst"
dictionary and model = "WW3TCOFCL"; in the "obs" dictionary will have
no
effect on the output. Instead, I'd recommend setting the "model" and
"obtype"
settings at the beginning of the Grid-Stat config file to populate the
MODEL
and OBTYPE columns in the output.
(3) Listed below are your "mask" settings which define the
verification
regions over which statistics should be computed:
mask = { grid = [""]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
The grid and poly options are arrays. I suspect that you want 0 grid
entries,
but you actually currently have 1... but it's set to an empty string.
Try
specifying it like this instead:
mask = { grid = [ ]; poly =
["/omar_backup/leo_backup/serra/ww3tcofcl-evaluation/wp302018/
wp302018-2018100400.nc"]; }
And that should only produce output for that single polyline that you
defined.
(4) Looking in wp302018-2018100400.nc, I see that the variable is
named
"box_mask":
float box_mask(lat, lon) ;
And that causes the VX_MASK column to be populated with "box_mask" in
the
output. I'd recommend choosing a more descriptive name for that
region.
When you run gen_vx_mask, you can add the "-name" command line option
to
define the name of the output variable. And then whatever name you
choose will
show up in the VX_MASK column of the Grid-Stat output.
Perhaps changing the mask.grid setting or upgrading to MET version 9.1
will
resolve the behavior you're experiencing. If not, please let me know
and I can
try to help you debug this more remotely.
Thanks,
John
------------------------------------------------
More information about the Met_help
mailing list