[Met_help] [rt.rap.ucar.edu #98769] History for BASER question
John Halley Gotway via RT
met_help at ucar.edu
Tue Feb 23 12:52:01 MST 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Folks - I placed the following files here:
ftp> pwd
257 "/incoming/irap/met_help/serra" is the current directory
ftp> ls
227 Entering Passive Mode (128,117,14,132,197,125).
150 Opening ASCII mode data connection for file list
-rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43 GridStatConfig_sh172019_gt18ft_WW3TCOFCL-WW3TCOFCL.2019030712_2019030212-120
-rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43 US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-000000prob_sig_wav_ht_gt18ft
-rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43 US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000000sig_wav_ht
If you run the command below you are going to notice that BASER_10 is 0; why is that the case?
Command:
grid_stat US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-000000prob_sig_wav_ht_gt18ft \
US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000000sig_wav_ht \
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-WW3TCOFCL.2019030712_2019030212-120 -outdir ./ -v 4
Efren A. Serra (Contractor)
Physicist
DeVine Consulting, Inc.
Naval Research Laboratory
Marine Meteorology Division
7 Grace Hopper Ave., STOP 2
Monterey, CA 93943
Code 7542
Mobile: 408-425-5027
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: BASER question
From: David Fillmore
Time: Fri Feb 19 13:55:41 2021
We have retrieved your test files for this grid_stat issue.
John and I will have a look.
thanks,
David
On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: BASER question
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
>
> Folks - I placed the following files here:
> ftp> pwd
> 257 "/incoming/irap/met_help/serra" is the current directory
>
> ftp> ls
> 227 Entering Passive Mode (128,117,14,132,197,125).
> 150 Opening ASCII mode data connection for file list
> -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_wav_ht
>
> If you run the command below you are going to notice that BASER_10
is 0;
> why is that the case?
>
> Command:
> grid_stat
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> \
>
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_wav_ht
> \
>
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> -outdir ./ -v 4
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: BASER question
From: David Fillmore
Time: Fri Feb 19 13:59:01 2021
User provided test files on
/d1/projects/MET/MET_Help/serra_data_20210219
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Feb 19 14:09:48 2021
David - I forgot to edit the mask field on line 71 of
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120; it should be
mask = { grid = []; poly = ["sh172019-2019030712.nc"]; }
Thanks for the quick turnaround!
-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 12:56 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
We have retrieved your test files for this grid_stat issue.
John and I will have a look.
thanks,
David
On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: BASER question
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
>
>
> Folks - I placed the following files here:
> ftp> pwd
> 257 "/incoming/irap/met_help/serra" is the current directory
>
> ftp> ls
> 227 Entering Passive Mode (128,117,14,132,197,125).
> 150 Opening ASCII mode data connection for file list
> -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> av_ht
>
> If you run the command below you are going to notice that BASER_10
is
> 0; why is that the case?
>
> Command:
> grid_stat
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> sig_wav_ht_gt18ft
> \
>
>
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> av_ht
> \
>
>
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120
> -outdir ./ -v 4
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
From: David Fillmore
Time: Fri Feb 19 14:20:02 2021
Hi Efren -
Can you upload via FTP your file sh172019-2019030712.nc for the poly
mask?
I'll modify the config.
Also, is this MET version 9.1.1 or an earlier version?
thanks
David
On Fri, Feb 19, 2021 at 2:09 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120; it should be
> mask = { grid = []; poly = ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: BASER question
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
is
> > 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Feb 19 14:23:08 2021
Mask uploaded.
-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:20 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Hi Efren -
Can you upload via FTP your file sh172019-2019030712.nc for the poly
mask?
I'll modify the config.
Also, is this MET version 9.1.1 or an earlier version?
thanks
David
On Fri, Feb 19, 2021 at 2:09 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120; it should be mask = { grid = []; poly =
> ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
> <efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: BASER question
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
> > is 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
From: David Fillmore
Time: Fri Feb 19 14:23:30 2021
Hi Efren -
sorry I see I already have the poly mask.
Please disregard my last request.
thanks,
David
On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120; it should be
> mask = { grid = []; poly = ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: BASER question
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
is
> > 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
------------------------------------------------
Subject: BASER question
From: David Fillmore
Time: Fri Feb 19 14:28:39 2021
Added test.sh script to
kiowa:/d1/projects/MET/MET_Help/serra_data_20210219.
Confirmed BASER_10 is 0 with MET release 9.1.1.
------------------------------------------------
Subject: BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Feb 19 14:44:38 2021
No worries! Mr Sampson and I are really wondering why BASER_10 is zero
even though of what attached images (plot_data_plane) show.
-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:24 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Hi Efren -
sorry I see I already have the poly mask.
Please disregard my last request.
thanks,
David
On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120; it should be mask = { grid = []; poly =
> ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
> <efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: BASER question
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
> > is 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Fri Feb 19 16:26:47 2021
Dude,
You sent them the whole globe?
Buck
-----Original Message-----
From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:45 PM
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
No worries! Mr Sampson and I are really wondering why BASER_10 is zero
even
though of what attached images (plot_data_plane) show.
-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:24 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Hi Efren -
sorry I see I already have the poly mask.
Please disregard my last request.
thanks,
David
On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120; it should be mask = { grid = []; poly =
> ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
> <efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: BASER question
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
> > is 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Mon Feb 22 14:20:50 2021
Efren and Buck,
David asked me to take a look at this. I just pulled down the data and
ran
Grid-Stat using MET version 9.1. I ran using the masking region you
provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at the
BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
0.0 there.
All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a
10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded
(cat_thresh = >=18.0;) to turn it into a 0 or 1. Those matched pairs
are
used to populate the 10x2 contingency table.
Looking in the PCT (probabilistic contingency table) line type, I
selected
the following columns of output:
*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23 1*
This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of
those, 23 points fell in the probability bin from 0.9 to 1.0
(THRESH_10 to
THRESH_11). Of those 23 points, the event occurred in the observation
0
times (OY_10) and did not occur 23 times (ON_10). The BASER_10 value
from
the PJC line type is the observation event frequency within that bin,
and 0
/ 23 = 0. So there were no points where the forecast value was between
0.9
and 1.0 and the observation was >=18.0.
Next, let's see if we can independently confirm this result... because
I
assume you've already looked closely at this example. I used the R
commands
listed below to read the NetCDF output from Grid-Stat and simply plot
fcst
vs obs. As you can see in the attached image, whenever the forecast
value
is >=0.9, the obs values are below 18, as indicated by the horizontal
line.
Hope that helps clarify.
Thanks,
John
R
library(ncdf4)
nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask")
obs = ncvar_get(nc, "OBS_HTSGW_Z0_box_mask")
pdf("scatter.pdf")
plot(fcst, obs, main="fcst vs obs")
abline(h=18)
dev.off()
On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Dude,
>
> You sent them the whole globe?
>
> Buck
>
>
> -----Original Message-----
> From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:45 PM
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> even
> though of what attached images (plot_data_plane) show.
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:24 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Hi Efren -
> sorry I see I already have the poly mask.
> Please disregard my last request.
> thanks,
> David
>
>
> On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> <met_help at ucar.edu> wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > David - I forgot to edit the mask field on line 71 of
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120; it should be mask = { grid = []; poly =
> > ["sh172019-2019030712.nc"]; }
> >
> > Thanks for the quick turnaround!
> >
> > -----Original Message-----
> > From: David Fillmore via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 12:56 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531
> > <efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > We have retrieved your test files for this grid_stat issue.
> > John and I will have a look.
> > thanks,
> > David
> >
> > On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: BASER question
> > > Owner: Nobody
> > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > >
> > >
> > >
> > > Folks - I placed the following files here:
> > > ftp> pwd
> > > 257 "/incoming/irap/met_help/serra" is the current directory
> > >
> > > ftp> ls
> > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > 150 Opening ASCII mode data connection for file list
> > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > >
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > >
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > > _w
> > > av_ht
> > >
> > > If you run the command below you are going to notice that
BASER_10
> > > is 0; why is that the case?
> > >
> > > Command:
> > > grid_stat
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > > b_
> > > sig_wav_ht_gt18ft
> > > \
> > >
> > >
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > > _w
> > > av_ht
> > > \
> > >
> > >
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > > 02
> > > 12-120
> > > -outdir ./ -v 4
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Mon Feb 22 14:58:19 2021
Aha! This is great. I think maybe Efren’s area is correct and that
the 95% area predicted is just slower than actual event. Just an
unlucky miss.
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:21 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Efren and Buck,
David asked me to take a look at this. I just pulled down the data and
ran Grid-Stat using MET version 9.1. I ran using the masking region
you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
the
BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
0.0 there.
All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a
10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
matched pairs are used to populate the 10x2 contingency table.
Looking in the PCT (probabilistic contingency table) line type, I
selected the following columns of output:
*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23 1*
This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of those, 23 points fell in the probability bin from 0.9 to
1.0 (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in the observation 0 times (OY_10) and did not occur 23 times (ON_10).
The BASER_10 value from the PJC line type is the observation event
frequency within that bin, and 0 / 23 = 0. So there were no points
where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.
Next, let's see if we can independently confirm this result... because
I assume you've already looked closely at this example. I used the R
commands listed below to read the NetCDF output from Grid-Stat and
simply plot fcst vs obs. As you can see in the attached image,
whenever the forecast value is >=0.9, the obs values are below 18, as
indicated by the horizontal line.
Hope that helps clarify.
Thanks,
John
R
library(ncdf4)
nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
"OBS_HTSGW_Z0_box_mask")
pdf("scatter.pdf")
plot(fcst, obs, main="fcst vs obs")
abline(h=18)
dev.off()
On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
<mailto:met_help at ucar.edu> met_help at ucar.edu>
wrote:
>
> <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Dude,
>
> You sent them the whole globe?
>
> Buck
>
>
> -----Original Message-----
> From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:45 PM
> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> even though of what attached images (plot_data_plane) show.
>
> -----Original Message-----
> From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:24 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Hi Efren -
> sorry I see I already have the poly mask.
> Please disregard my last request.
> thanks,
> David
>
>
> On Fri, Feb 19, 2021 at 2:10 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT
> < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> >
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > David - I forgot to edit the mask field on line 71 of
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02 12-120; it should be mask = { grid = []; poly =
> > ["sh172019-2019030712.nc"]; }
> >
> > Thanks for the quick turnaround!
> >
> > -----Original Message-----
> > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 12:56 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531
> > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > We have retrieved your test files for this grid_stat issue.
> > John and I will have a look.
> > thanks,
> > David
> >
> > On Fri, Feb 19, 2021 at 11:48 AM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via
> > RT
> <
> > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > Transaction: Ticket created by
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: BASER question
> > > Owner: Nobody
> > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > >
> > >
> > >
> > > Folks - I placed the following files here:
> > > ftp> pwd
> > > 257 "/incoming/irap/met_help/serra" is the current directory
> > >
> > > ftp> ls
> > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > 150 Opening ASCII mode data connection for file list
> > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > >
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120
> > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > >
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> sig_wav_ht_gt18ft
> > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > >
> > > If you run the command below you are going to notice that
BASER_10
> > > is 0; why is that the case?
> > >
> > > Command:
> > > grid_stat
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
> > > b_
> > > sig_wav_ht_gt18ft
> > > \
> > >
> > >
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > > \
> > >
> > >
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
> > > 02
> > > 12-120
> > > -outdir ./ -v 4
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon Feb 22 15:13:53 2021
Thank you very much, John!
-----Original Message-----
From: Sampson, Mr. Buck via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:58 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
Aha! This is great. I think maybe Efren’s area is correct and that
the 95% area predicted is just slower than actual event. Just an
unlucky miss.
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:21 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Efren and Buck,
David asked me to take a look at this. I just pulled down the data and
ran Grid-Stat using MET version 9.1. I ran using the masking region
you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
the
BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
0.0 there.
All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a
10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
matched pairs are used to populate the 10x2 contingency table.
Looking in the PCT (probabilistic contingency table) line type, I
selected the following columns of output:
*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23 1*
This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of those, 23 points fell in the probability bin from 0.9 to
1.0 (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in the observation 0 times (OY_10) and did not occur 23 times (ON_10).
The BASER_10 value from the PJC line type is the observation event
frequency within that bin, and 0 / 23 = 0. So there were no points
where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.
Next, let's see if we can independently confirm this result... because
I assume you've already looked closely at this example. I used the R
commands listed below to read the NetCDF output from Grid-Stat and
simply plot fcst vs obs. As you can see in the attached image,
whenever the forecast value is >=0.9, the obs values are below 18, as
indicated by the horizontal line.
Hope that helps clarify.
Thanks,
John
R
library(ncdf4)
nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
"OBS_HTSGW_Z0_box_mask")
pdf("scatter.pdf")
plot(fcst, obs, main="fcst vs obs")
abline(h=18)
dev.off()
On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
<mailto:met_help at ucar.edu> met_help at ucar.edu>
wrote:
>
> <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Dude,
>
> You sent them the whole globe?
>
> Buck
>
>
> -----Original Message-----
> From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:45 PM
> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> even though of what attached images (plot_data_plane) show.
>
> -----Original Message-----
> From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:24 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Hi Efren -
> sorry I see I already have the poly mask.
> Please disregard my last request.
> thanks,
> David
>
>
> On Fri, Feb 19, 2021 at 2:10 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT
> < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> >
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > David - I forgot to edit the mask field on line 71 of
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02 12-120; it should be mask = { grid = []; poly =
> > ["sh172019-2019030712.nc"]; }
> >
> > Thanks for the quick turnaround!
> >
> > -----Original Message-----
> > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 12:56 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531
> > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > We have retrieved your test files for this grid_stat issue.
> > John and I will have a look.
> > thanks,
> > David
> >
> > On Fri, Feb 19, 2021 at 11:48 AM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via
> > RT
> <
> > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > Transaction: Ticket created by
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: BASER question
> > > Owner: Nobody
> > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > >
> > >
> > >
> > > Folks - I placed the following files here:
> > > ftp> pwd
> > > 257 "/incoming/irap/met_help/serra" is the current directory
> > >
> > > ftp> ls
> > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > 150 Opening ASCII mode data connection for file list
> > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > >
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120
> > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > >
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> sig_wav_ht_gt18ft
> > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > >
> > > If you run the command below you are going to notice that
BASER_10
> > > is 0; why is that the case?
> > >
> > > Command:
> > > grid_stat
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
> > > b_
> > > sig_wav_ht_gt18ft
> > > \
> > >
> > >
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > > \
> > >
> > >
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
> > > 02
> > > 12-120
> > > -outdir ./ -v 4
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon Feb 22 15:14:32 2021
Thanks for the R plot; I think this big 'little' script is great for
validation. I shall incorporate in my products.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:21 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Efren and Buck,
David asked me to take a look at this. I just pulled down the data and
ran Grid-Stat using MET version 9.1. I ran using the masking region
you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
the
BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
0.0 there.
All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a
10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
matched pairs are used to populate the 10x2 contingency table.
Looking in the PCT (probabilistic contingency table) line type, I
selected the following columns of output:
*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23 1*
This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of those, 23 points fell in the probability bin from 0.9 to
1.0 (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in the observation 0 times (OY_10) and did not occur 23 times (ON_10).
The BASER_10 value from the PJC line type is the observation event
frequency within that bin, and 0 / 23 = 0. So there were no points
where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.
Next, let's see if we can independently confirm this result... because
I assume you've already looked closely at this example. I used the R
commands listed below to read the NetCDF output from Grid-Stat and
simply plot fcst vs obs. As you can see in the attached image,
whenever the forecast value is >=0.9, the obs values are below 18, as
indicated by the horizontal line.
Hope that helps clarify.
Thanks,
John
R
library(ncdf4)
nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
"OBS_HTSGW_Z0_box_mask")
pdf("scatter.pdf")
plot(fcst, obs, main="fcst vs obs")
abline(h=18)
dev.off()
On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Dude,
>
> You sent them the whole globe?
>
> Buck
>
>
> -----Original Message-----
> From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:45 PM
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> even though of what attached images (plot_data_plane) show.
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:24 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Hi Efren -
> sorry I see I already have the poly mask.
> Please disregard my last request.
> thanks,
> David
>
>
> On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> <met_help at ucar.edu> wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > David - I forgot to edit the mask field on line 71 of
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02 12-120; it should be mask = { grid = []; poly =
> > ["sh172019-2019030712.nc"]; }
> >
> > Thanks for the quick turnaround!
> >
> > -----Original Message-----
> > From: David Fillmore via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 12:56 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531
> > <efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > We have retrieved your test files for this grid_stat issue.
> > John and I will have a look.
> > thanks,
> > David
> >
> > On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil
via
> > RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: BASER question
> > > Owner: Nobody
> > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > >
> > >
> > >
> > > Folks - I placed the following files here:
> > > ftp> pwd
> > > 257 "/incoming/irap/met_help/serra" is the current directory
> > >
> > > ftp> ls
> > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > 150 Opening ASCII mode data connection for file list
> > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > >
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120
> > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > >
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> sig_wav_ht_gt18ft
> > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > >
> > > If you run the command below you are going to notice that
BASER_10
> > > is 0; why is that the case?
> > >
> > > Command:
> > > grid_stat
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
> > > b_
> > > sig_wav_ht_gt18ft
> > > \
> > >
> > >
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > > \
> > >
> > >
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
> > > 02
> > > 12-120
> > > -outdir ./ -v 4
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Mon Feb 22 22:12:59 2021
Sounds good. You know the features in the images you sent look a lot
like
objects. And Buck, your comment about the forecast being “slow”
reminds me
of object-based verification.
Not sure if you’ve already run the MODE tool in MET on this data but
it
could be a really nice application. You certainly can run MODE to
compare
objects from 2 different fields, like probabilities >25% vs obs >18.
Hopefully it’d give you more diagnostic info about the errors, like
being
too slow (centroid offset to the west) or fast (offset to the east).
MET also includes the MODE-Time-Domain tool to define 3D space-time
objects.
Thanks
John
On Mon, Feb 22, 2021 at 3:14 PM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks for the R plot; I think this big 'little' script is great for
> validation. I shall incorporate in my products.
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 1:21 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Efren and Buck,
>
> David asked me to take a look at this. I just pulled down the data
and ran
> Grid-Stat using MET version 9.1. I ran using the masking region you
> provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at the
> BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
> 0.0 there.
>
> All of the probabilistic verification is done in MET using an Nx2
> probabilistic contingency table. Since you configured Grid-Stat with
> "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
it's a
> 10x2 contingency table. For each matched pair, we have a forecast
> probability value between 0 and 1 and observation value that's
thresholded
> (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those matched pairs
are
> used to populate the 10x2 contingency table.
>
> Looking in the PCT (probabilistic contingency table) line type, I
selected
> the following columns of output:
>
> *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23 1*
>
> This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
> Of those, 23 points fell in the probability bin from 0.9 to 1.0
(THRESH_10
> to THRESH_11). Of those 23 points, the event occurred in the
observation 0
> times (OY_10) and did not occur 23 times (ON_10). The BASER_10 value
from
> the PJC line type is the observation event frequency within that
bin, and 0
> / 23 = 0. So there were no points where the forecast value was
between 0.9
> and 1.0 and the observation was >=18.0.
>
> Next, let's see if we can independently confirm this result...
because I
> assume you've already looked closely at this example. I used the R
commands
> listed below to read the NetCDF output from Grid-Stat and simply
plot fcst
> vs obs. As you can see in the attached image, whenever the forecast
value
> is >=0.9, the obs values are below 18, as indicated by the
horizontal line.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
>
> R
> library(ncdf4)
> nc =
>
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
> fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
> "OBS_HTSGW_Z0_box_mask")
> pdf("scatter.pdf")
> plot(fcst, obs, main="fcst vs obs")
> abline(h=18)
> dev.off()
>
>
>
>
>
>
>
> On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Dude,
> >
> > You sent them the whole globe?
> >
> > Buck
> >
> >
> > -----Original Message-----
> > From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 1:45 PM
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> > even though of what attached images (plot_data_plane) show.
> >
> > -----Original Message-----
> > From: David Fillmore via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 1:24 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > Hi Efren -
> > sorry I see I already have the poly mask.
> > Please disregard my last request.
> > thanks,
> > David
> >
> >
> > On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil
via RT
> > <met_help at ucar.edu> wrote:
> > >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> > >
> > > David - I forgot to edit the mask field on line 71 of
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > > 02 12-120; it should be mask = { grid = []; poly =
> > > ["sh172019-2019030712.nc"]; }
> > >
> > > Thanks for the quick turnaround!
> > >
> > > -----Original Message-----
> > > From: David Fillmore via RT <met_help at ucar.edu>
> > > Sent: Friday, February 19, 2021 12:56 PM
> > > To: Serra, Mr. Efren, Contractor, Code 7531
> > > <efren.serra.ctr at nrlmry.navy.mil>
> > > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> > >
> > > We have retrieved your test files for this grid_stat issue.
> > > John and I will have a look.
> > > thanks,
> > > David
> > >
> > > On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil
via
> > > RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > > Queue: met_help
> > > > Subject: BASER question
> > > > Owner: Nobody
> > > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > > >
> > > >
> > > >
> > > > Folks - I placed the following files here:
> > > > ftp> pwd
> > > > 257 "/incoming/irap/met_help/serra" is the current directory
> > > >
> > > > ftp> ls
> > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > > 150 Opening ASCII mode data connection for file list
> > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > > >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120
> > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > > >
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> > sig_wav_ht_gt18ft
> > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > > ig
> > > > _w
> > > > av_ht
> > > >
> > > > If you run the command below you are going to notice that
BASER_10
> > > > is 0; why is that the case?
> > > >
> > > > Command:
> > > > grid_stat
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > > ro
> > > > b_
> > > > sig_wav_ht_gt18ft
> > > > \
> > > >
> > > >
> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > > ig
> > > > _w
> > > > av_ht
> > > > \
> > > >
> > > >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > > 03
> > > > 02
> > > > 12-120
> > > > -outdir ./ -v 4
> > > >
> > > > Efren A. Serra (Contractor)
> > > > Physicist
> > > >
> > > > DeVine Consulting, Inc.
> > > > Naval Research Laboratory
> > > > Marine Meteorology Division
> > > > 7 Grace Hopper Ave., STOP 2
> > > > Monterey, CA 93943
> > > > Code 7542
> > > > Mobile: 408-425-5027
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 08:22:49 2021
Thanks John. I have thought about the mode capabilities. Time and
money, of course. And concern about misplacement of the entire event,
which is a concern in operations. Your neighborhood capabilities
would also be more fair to any method that predicts rare events (e.g.,
90% of 18-ft waves) than an algorithm that spreads out a bunch of low
probabilities. Anyhow, we are learning something about our method
with MET that we didn't see before and learning about MET in the
process. Our post-processing is likely high biased (yet to be
proven), but is better at distinguishing events from non-events that
the ensemble without our tropical cyclone post-processing. And we
have easy ways to correct the high bias, so that's all good.
Reliability: Our question to you came up because I can't make sense of
our Reliability Diagrams. The 20x20 verification grid you should have
18-ft waves in it, like on the order of 10% (see purple contour in
second graphic). The BASER data looks to be a conditional probability
from what you sent, and I was under the impression that reliability
diagrams were strictly a count (percentage) of observed vs forecast
probabilities in our 20x20 grid. I think the reliability is probably
about right, even in the highest bin. So even though the TC forecast
was slow, there are still some verifying >18ft sig wave heights in the
20x20 grid. In fact, the percentage looks to be about right!
Buck
R,
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 9:13 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Sounds good. You know the features in the images you sent look a lot
like objects. And Buck, your comment about the forecast being “slow”
reminds me of object-based verification.
Not sure if you’ve already run the MODE tool in MET on this data but
it could be a really nice application. You certainly can run MODE to
compare objects from 2 different fields, like probabilities >25% vs
obs >18.
Hopefully it’d give you more diagnostic info about the errors, like
being too slow (centroid offset to the west) or fast (offset to the
east).
MET also includes the MODE-Time-Domain tool to define 3D space-time
objects.
Thanks
John
On Mon, Feb 22, 2021 at 3:14 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu> wrote:
>
> <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks for the R plot; I think this big 'little' script is great for
> validation. I shall incorporate in my products.
>
> -----Original Message-----
> From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> Sent: Monday, February 22, 2021 1:21 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Efren and Buck,
>
> David asked me to take a look at this. I just pulled down the data
and
> ran Grid-Stat using MET version 9.1. I ran using the masking region
> you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
> the
> BASER_10 column of the PJC output line. And you're wondering why
> BASER_10 =
> 0.0 there.
>
> All of the probabilistic verification is done in MET using an Nx2
> probabilistic contingency table. Since you configured Grid-Stat with
> "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
it's
> a
> 10x2 contingency table. For each matched pair, we have a forecast
> probability value between 0 and 1 and observation value that's
> thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
> matched pairs are used to populate the 10x2 contingency table.
>
> Looking in the PCT (probabilistic contingency table) line type, I
> selected the following columns of output:
>
> *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23 1*
>
> This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
> Of those, 23 points fell in the probability bin from 0.9 to 1.0
> (THRESH_10 to THRESH_11). Of those 23 points, the event occurred in
> the observation 0 times (OY_10) and did not occur 23 times (ON_10).
> The BASER_10 value from the PJC line type is the observation event
> frequency within that bin, and 0 / 23 = 0. So there were no points
> where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.
>
> Next, let's see if we can independently confirm this result...
because
> I assume you've already looked closely at this example. I used the R
> commands listed below to read the NetCDF output from Grid-Stat and
> simply plot fcst vs obs. As you can see in the attached image,
> whenever the forecast value is >=0.9, the obs values are below 18,
as indicated by the horizontal line.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
>
> R
> library(ncdf4)
> nc =
>
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
> "OBS_HTSGW_Z0_box_mask")
> pdf("scatter.pdf")
> plot(fcst, obs, main="fcst vs obs")
> abline(h=18)
> dev.off()
>
>
>
>
>
>
>
> On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> <mailto:met_help at ucar.edu> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Dude,
> >
> > You sent them the whole globe?
> >
> > Buck
> >
> >
> > -----Original Message-----
> > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 1:45 PM
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > No worries! Mr Sampson and I are really wondering why BASER_10 is
> > zero even though of what attached images (plot_data_plane) show.
> >
> > -----Original Message-----
> > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 1:24 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > Hi Efren -
> > sorry I see I already have the poly mask.
> > Please disregard my last request.
> > thanks,
> > David
> >
> >
> > On Fri, Feb 19, 2021 at 2:10 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via
> > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> > >
> > >
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> > >
> > > David - I forgot to edit the mask field on line 71 of
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
> > > 02 12-120; it should be mask = { grid = []; poly =
> > > ["sh172019-2019030712.nc"]; }
> > >
> > > Thanks for the quick turnaround!
> > >
> > > -----Original Message-----
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>
> > > Sent: Friday, February 19, 2021 12:56 PM
> > > To: Serra, Mr. Efren, Contractor, Code 7531
> > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> > >
> > > We have retrieved your test files for this grid_stat issue.
> > > John and I will have a look.
> > > thanks,
> > > David
> > >
> > > On Fri, Feb 19, 2021 at 11:48 AM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > via RT
> > <
> > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > > Transaction: Ticket created by
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > > Queue: met_help
> > > > Subject: BASER question
> > > > Owner: Nobody
> > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > > >
> > > >
> > > >
> > > > Folks - I placed the following files here:
> > > > ftp> pwd
> > > > 257 "/incoming/irap/met_help/serra" is the current directory
> > > >
> > > > ftp> ls
> > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > > 150 Opening ASCII mode data connection for file list
> > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> > > >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> > > >
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000
> > > > 0s
> > > > ig
> > > > _w
> > > > av_ht
> > > >
> > > > If you run the command below you are going to notice that
> > > > BASER_10 is 0; why is that the case?
> > > >
> > > > Command:
> > > > grid_stat
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
> > > > 0p
> > > > ro
> > > > b_
> > > > sig_wav_ht_gt18ft
> > > > \
> > > >
> > > >
> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000
> > > > 0s
> > > > ig
> > > > _w
> > > > av_ht
> > > > \
> > > >
> > > >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
> > > > 03
> > > > 02
> > > > 12-120
> > > > -outdir ./ -v 4
> > > >
> > > > Efren A. Serra (Contractor)
> > > > Physicist
> > > >
> > > > DeVine Consulting, Inc.
> > > > Naval Research Laboratory
> > > > Marine Meteorology Division
> > > > 7 Grace Hopper Ave., STOP 2
> > > > Monterey, CA 93943
> > > > Code 7542
> > > > Mobile: 408-425-5027
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Tue Feb 23 08:47:03 2021
Buck,
Sounds interesting. Since you mentioned the forecasts being biased,
you
might find percentile thresholds in MET to be useful. They can be used
to
de-bias the data. Search for percentile thresholds on this page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
One option I really like is setting an explicit observation threshold,
like
"cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;" for the
forecast. That tells MET to *choose* a threshold for the forecast that
will
result in a frequency bias as close to 1 as possible.
I'll go ahead and resolve this ticket.
Thanks,
John
On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John. I have thought about the mode capabilities. Time and
money,
> of course. And concern about misplacement of the entire event,
which is a
> concern in operations. Your neighborhood capabilities would also be
more
> fair to any method that predicts rare events (e.g., 90% of 18-ft
waves)
> than an algorithm that spreads out a bunch of low probabilities.
Anyhow,
> we are learning something about our method with MET that we didn't
see
> before and learning about MET in the process. Our post-processing
is
> likely high biased (yet to be proven), but is better at
distinguishing
> events from non-events that the ensemble without our tropical
cyclone
> post-processing. And we have easy ways to correct the high bias, so
that's
> all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of our
> Reliability Diagrams. The 20x20 verification grid you should have
18-ft
> waves in it, like on the order of 10% (see purple contour in second
> graphic). The BASER data looks to be a conditional probability from
what
> you sent, and I was under the impression that reliability diagrams
were
> strictly a count (percentage) of observed vs forecast probabilities
in our
> 20x20 grid. I think the reliability is probably about right, even
in the
> highest bin. So even though the TC forecast was slow, there are
still some
> verifying >18ft sig wave heights in the 20x20 grid. In fact, the
> percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
like
> objects. And Buck, your comment about the forecast being “slow”
reminds me
> of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
it
> could be a really nice application. You certainly can run MODE to
compare
> objects from 2 different fields, like probabilities >25% vs obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
being
> too slow (centroid offset to the west) or fast (offset to the east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation was
> >=18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> > <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Queue: met_help
>
> > > > > Subject: BASER question
>
> > > > > Owner: Nobody
>
> > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Status: new
>
> > > > > Ticket <URL:
>
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 08:56:14 2021
John,
Wait! Can you comment on the second part? Am I dead wrong on that?
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Buck,
Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.
I'll go ahead and resolve this ticket.
Thanks,
John
On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John. I have thought about the mode capabilities. Time and
> money, of course. And concern about misplacement of the entire
event,
> which is a concern in operations. Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities. Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process. Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing. And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams. The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic). The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid. I think the reliability is probably about right, even
in
> the highest bin. So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid. In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application. You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> > <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Queue: met_help
>
> > > > > Subject: BASER question
>
> > > > > Owner: Nobody
>
> > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Status: new
>
> > > > > Ticket <URL:
>
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 09:24:14 2021
Ah, I think I am dead wrong. Alas.
Buck
-----Original Message-----
From: Sampson, Mr. Buck
Sent: Tuesday, February 23, 2021 7:56 AM
To: 'met_help at ucar.edu' <met_help at ucar.edu>; Serra, Mr. Efren,
Contractor, Code 7531 <efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
John,
Wait! Can you comment on the second part? Am I dead wrong on that?
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Buck,
Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.
I'll go ahead and resolve this ticket.
Thanks,
John
On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John. I have thought about the mode capabilities. Time and
> money, of course. And concern about misplacement of the entire
event,
> which is a concern in operations. Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities. Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process. Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing. And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams. The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic). The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid. I think the reliability is probably about right, even
in
> the highest bin. So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid. In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application. You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> > <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Queue: met_help
>
> > > > > Subject: BASER question
>
> > > > > Owner: Nobody
>
> > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Status: new
>
> > > > > Ticket <URL:
>
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>
------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Tue Feb 23 09:26:19 2021
Buck,
Looking in the PSTD output line type, the BASER = 0.1275 output
indicates
that the event (>=18) occurs at 12.75% of the grid points. The
BASER_1,
BASER_2,... , BASER_10 values in the PJC line type give the event
frequency within that bin. I believe the reliability diagram simply
plots
the mid-point of the forecast probability bin vs the observed event
frequency within that bin.
If needed, I could load the sample data you sent into METviewer
running
through Docker on my laptop and make a reliability diagram.
John
On Tue, Feb 23, 2021 at 8:56 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> John,
>
> Wait! Can you comment on the second part? Am I dead wrong on that?
>
> Buck
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, February 23, 2021 7:47 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Buck,
>
> Sounds interesting. Since you mentioned the forecasts being biased,
you
> might find percentile thresholds in MET to be useful. They can be
used to
> de-bias the data. Search for percentile thresholds on this page:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
>
> One option I really like is setting an explicit observation
threshold,
> like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the
> forecast. That tells MET to *choose* a threshold for the forecast
that will
> result in a frequency bias as close to 1 as possible.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Thanks John. I have thought about the mode capabilities. Time
and
> > money, of course. And concern about misplacement of the entire
event,
> > which is a concern in operations. Your neighborhood capabilities
> > would also be more fair to any method that predicts rare events
(e.g.,
> > 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> > probabilities. Anyhow, we are learning something about our method
> > with MET that we didn't see before and learning about MET in the
> > process. Our post-processing is likely high biased (yet to be
> > proven), but is better at distinguishing events from non-events
that
> > the ensemble without our tropical cyclone post-processing. And we
> > have easy ways to correct the high bias, so that's all good.
> >
> >
> >
> > Reliability: Our question to you came up because I can't make
sense of
> > our Reliability Diagrams. The 20x20 verification grid you should
have
> > 18-ft waves in it, like on the order of 10% (see purple contour in
> > second graphic). The BASER data looks to be a conditional
probability
> > from what you sent, and I was under the impression that
reliability
> > diagrams were strictly a count (percentage) of observed vs
forecast
> > probabilities in our
> > 20x20 grid. I think the reliability is probably about right, even
in
> > the highest bin. So even though the TC forecast was slow, there
are
> > still some verifying >18ft sig wave heights in the 20x20 grid. In
> > fact, the percentage looks to be about right!
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > R,
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, February 22, 2021 9:13 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> >
> >
> > Sounds good. You know the features in the images you sent look a
lot
> > like objects. And Buck, your comment about the forecast being
“slow”
> > reminds me of object-based verification.
> >
> >
> >
> > Not sure if you’ve already run the MODE tool in MET on this data
but
> > it could be a really nice application. You certainly can run MODE
to
> > compare objects from 2 different fields, like probabilities >25%
vs obs
> >18.
> >
> > Hopefully it’d give you more diagnostic info about the errors,
like
> > being too slow (centroid offset to the west) or fast (offset to
the
> east).
> >
> >
> >
> > MET also includes the MODE-Time-Domain tool to define 3D space-
time
> > objects.
> >
> >
> >
> > Thanks
> >
> > John
> >
> >
> >
> > On Mon, Feb 22, 2021 at 3:14 PM
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu> wrote:
> >
> >
> >
> > >
> >
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > >
> >
> > > Thanks for the R plot; I think this big 'little' script is great
for
> >
> > > validation. I shall incorporate in my products.
> >
> > >
> >
> > > -----Original Message-----
> >
> > > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > Sent: Monday, February 22, 2021 1:21 PM
> >
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > >
> >
> > > Efren and Buck,
> >
> > >
> >
> > > David asked me to take a look at this. I just pulled down the
data
> > > and
> >
> > > ran Grid-Stat using MET version 9.1. I ran using the masking
region
> >
> > > you provided (poly = ["sh172019-2019030712.nc"];) and I'm
looking at
> >
> > > the
> >
> > > BASER_10 column of the PJC output line. And you're wondering why
> >
> > > BASER_10 =
> >
> > > 0.0 there.
> >
> > >
> >
> > > All of the probabilistic verification is done in MET using an
Nx2
> >
> > > probabilistic contingency table. Since you configured Grid-Stat
with
> >
> > > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > > it's
> >
> > > a
> >
> > > 10x2 contingency table. For each matched pair, we have a
forecast
> >
> > > probability value between 0 and 1 and observation value that's
> >
> > > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1.
Those
> >
> > > matched pairs are used to populate the 10x2 contingency table.
> >
> > >
> >
> > > Looking in the PCT (probabilistic contingency table) line type,
I
> >
> > > selected the following columns of output:
> >
> > >
> >
> > > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
> >
> > >
> >
> > > This tells us that the entire 10x2 table consists of 400 points
> (TOTAL).
> >
> > > Of those, 23 points fell in the probability bin from 0.9 to 1.0
> >
> > > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
> >
> > > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
> >
> > > The BASER_10 value from the PJC line type is the observation
event
> >
> > > frequency within that bin, and 0 / 23 = 0. So there were no
points
> >
> > > where the forecast value was between 0.9 and 1.0 and the
observation
> > >was =18.0.
> >
> > >
> >
> > > Next, let's see if we can independently confirm this result...
> > > because
> >
> > > I assume you've already looked closely at this example. I used
the R
> >
> > > commands listed below to read the NetCDF output from Grid-Stat
and
> >
> > > simply plot fcst vs obs. As you can see in the attached image,
> >
> > > whenever the forecast value is >=0.9, the obs values are below
18,
> > > as
> > indicated by the horizontal line.
> >
> > >
> >
> > > Hope that helps clarify.
> >
> > >
> >
> > > Thanks,
> >
> > > John
> >
> > >
> >
> > >
> >
> > > R
> >
> > > library(ncdf4)
> >
> > > nc =
> >
> > >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> >
> > > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs =
ncvar_get(nc,
> >
> > > "OBS_HTSGW_Z0_box_mask")
> >
> > > pdf("scatter.pdf")
> >
> > > plot(fcst, obs, main="fcst vs obs")
> >
> > > abline(h=18)
> >
> > > dev.off()
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> >
> > > <mailto:met_help at ucar.edu> met_help at ucar.edu>
> >
> > > wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > >
> >
> > > > Dude,
> >
> > > >
> >
> > > > You sent them the whole globe?
> >
> > > >
> >
> > > > Buck
> >
> > > >
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:45 PM
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > No worries! Mr Sampson and I are really wondering why BASER_10
is
> >
> > > > zero even though of what attached images (plot_data_plane)
show.
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:24 PM
> >
> > > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > Hi Efren -
> >
> > > > sorry I see I already have the poly mask.
> >
> > > > Please disregard my last request.
> >
> > > > thanks,
> >
> > > > David
> >
> > > >
> >
> > > >
> >
> > > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
via
> >
> > > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > >
> >
> > > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > > >
> >
> > > > > David - I forgot to edit the mask field on line 71 of
> >
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > > 19
> >
> > > > > 03
> >
> > > > > 02 12-120; it should be mask = { grid = []; poly =
> >
> > > > > ["sh172019-2019030712.nc"]; }
> >
> > > > >
> >
> > > > > Thanks for the quick turnaround!
> >
> > > > >
> >
> > > > > -----Original Message-----
> >
> > > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > > Sent: Friday, February 19, 2021 12:56 PM
> >
> > > > > To: Serra, Mr. Efren, Contractor, Code 7531
> >
> > > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > > Cc: Sampson, Mr. Buck <
<mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > > >
> >
> > > > > We have retrieved your test files for this grid_stat issue.
> >
> > > > > John and I will have a look.
> >
> > > > > thanks,
> >
> > > > > David
> >
> > > > >
> >
> > > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > via RT
> >
> > > > <
> >
> > > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> >
> > > > > > Transaction: Ticket created by <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > > Queue: met_help
> >
> > > > > > Subject: BASER question
> >
> > > > > > Owner: Nobody
> >
> > > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > > Status: new
> >
> > > > > > Ticket <URL:
> >
> > > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
> > > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > Folks - I placed the following files here:
> >
> > > > > > ftp> pwd
> >
> > > > > > 257 "/incoming/irap/met_help/serra" is the current
directory
> >
> > > > > >
> >
> > > > > > ftp> ls
> >
> > > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> >
> > > > > > 150 Opening ASCII mode data connection for file list
> >
> > > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> >
> > > > > >
> >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > > 03
> >
> > > > 02
> >
> > > > 12-120
> >
> > > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> >
> > > > > >
> >
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > > ro
> >
> > > > b_
> >
> > > > sig_wav_ht_gt18ft
> >
> > > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > >
> >
> > > > > > If you run the command below you are going to notice that
> >
> > > > > > BASER_10 is 0; why is that the case?
> >
> > > > > >
> >
> > > > > > Command:
> >
> > > > > > grid_stat
> >
> > > > > > US058GOCN-
GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-000
> > > > > > 00
> >
> > > > > > 0p
> >
> > > > > > ro
> >
> > > > > > b_
> >
> > > > > > sig_wav_ht_gt18ft
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > > 20
> >
> > > > > > 19
> >
> > > > > > 03
> >
> > > > > > 02
> >
> > > > > > 12-120
> >
> > > > > > -outdir ./ -v 4
> >
> > > > > >
> >
> > > > > > Efren A. Serra (Contractor)
> >
> > > > > > Physicist
> >
> > > > > >
> >
> > > > > > DeVine Consulting, Inc.
> >
> > > > > > Naval Research Laboratory
> >
> > > > > > Marine Meteorology Division
> >
> > > > > > 7 Grace Hopper Ave., STOP 2
> >
> > > > > > Monterey, CA 93943
> >
> > > > > > Code 7542
> >
> > > > > > Mobile: 408-425-5027
> >
> > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 10:05:04 2021
No, I'm good. Thanks for the offer. I think we have issues with our
numbers of cases, and that is what is throwing me off. I see "800"
cases for ".9" bin. We may be short one bin in our table,
specifically we might be missing the "1" or ".95" bin that we are
discussing. I hope ... at least that would explain what is happening
in that highest bin.
R,
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 8:26 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Buck,
Looking in the PSTD output line type, the BASER = 0.1275 output
indicates that the event (>=18) occurs at 12.75% of the grid points.
The BASER_1, BASER_2,... , BASER_10 values in the PJC line type give
the event frequency within that bin. I believe the reliability diagram
simply plots the mid-point of the forecast probability bin vs the
observed event frequency within that bin.
If needed, I could load the sample data you sent into METviewer
running through Docker on my laptop and make a reliability diagram.
John
On Tue, Feb 23, 2021 at 8:56 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> John,
>
> Wait! Can you comment on the second part? Am I dead wrong on that?
>
> Buck
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, February 23, 2021 7:47 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Buck,
>
> Sounds interesting. Since you mentioned the forecasts being biased,
> you might find percentile thresholds in MET to be useful. They can
be
> used to de-bias the data. Search for percentile thresholds on this
page:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlig
> ht=percentile%20thresholds
>
> One option I really like is setting an explicit observation
threshold,
> like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
> for the forecast. That tells MET to *choose* a threshold for the
> forecast that will result in a frequency bias as close to 1 as
possible.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Thanks John. I have thought about the mode capabilities. Time
and
> > money, of course. And concern about misplacement of the entire
> > event, which is a concern in operations. Your neighborhood
> > capabilities would also be more fair to any method that predicts
> > rare events (e.g., 90% of 18-ft waves) than an algorithm that
> > spreads out a bunch of low probabilities. Anyhow, we are learning
> > something about our method with MET that we didn't see before and
> > learning about MET in the process. Our post-processing is likely
> > high biased (yet to be proven), but is better at distinguishing
> > events from non-events that the ensemble without our tropical
> > cyclone post-processing. And we have easy ways to correct the
high bias, so that's all good.
> >
> >
> >
> > Reliability: Our question to you came up because I can't make
sense
> > of our Reliability Diagrams. The 20x20 verification grid you
should
> > have 18-ft waves in it, like on the order of 10% (see purple
contour
> > in second graphic). The BASER data looks to be a conditional
> > probability from what you sent, and I was under the impression
that
> > reliability diagrams were strictly a count (percentage) of
observed
> > vs forecast probabilities in our
> > 20x20 grid. I think the reliability is probably about right, even
> > in the highest bin. So even though the TC forecast was slow,
there
> > are still some verifying >18ft sig wave heights in the 20x20 grid.
> > In fact, the percentage looks to be about right!
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > R,
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, February 22, 2021 9:13 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> >
> >
> > Sounds good. You know the features in the images you sent look a
lot
> > like objects. And Buck, your comment about the forecast being
“slow”
> > reminds me of object-based verification.
> >
> >
> >
> > Not sure if you’ve already run the MODE tool in MET on this data
but
> >it could be a really nice application. You certainly can run MODE
to
> >compare objects from 2 different fields, like probabilities >25% vs
> >obs 18.
> >
> > Hopefully it’d give you more diagnostic info about the errors,
like
> > being too slow (centroid offset to the west) or fast (offset to
the
> east).
> >
> >
> >
> > MET also includes the MODE-Time-Domain tool to define 3D space-
time
> > objects.
> >
> >
> >
> > Thanks
> >
> > John
> >
> >
> >
> > On Mon, Feb 22, 2021 at 3:14 PM
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu> wrote:
> >
> >
> >
> > >
> >
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > >
> >
> > > Thanks for the R plot; I think this big 'little' script is great
> > > for
> >
> > > validation. I shall incorporate in my products.
> >
> > >
> >
> > > -----Original Message-----
> >
> > > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > Sent: Monday, February 22, 2021 1:21 PM
> >
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > >
> >
> > > Efren and Buck,
> >
> > >
> >
> > > David asked me to take a look at this. I just pulled down the
data
> > > and
> >
> > > ran Grid-Stat using MET version 9.1. I ran using the masking
> > > region
> >
> > > you provided (poly = ["sh172019-2019030712.nc"];) and I'm
looking
> > > at
> >
> > > the
> >
> > > BASER_10 column of the PJC output line. And you're wondering why
> >
> > > BASER_10 =
> >
> > > 0.0 there.
> >
> > >
> >
> > > All of the probabilistic verification is done in MET using an
Nx2
> >
> > > probabilistic contingency table. Since you configured Grid-Stat
> > > with
> >
> > > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > > it's
> >
> > > a
> >
> > > 10x2 contingency table. For each matched pair, we have a
forecast
> >
> > > probability value between 0 and 1 and observation value that's
> >
> > > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1.
Those
> >
> > > matched pairs are used to populate the 10x2 contingency table.
> >
> > >
> >
> > > Looking in the PCT (probabilistic contingency table) line type,
I
> >
> > > selected the following columns of output:
> >
> > >
> >
> > > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
> >
> > >
> >
> > > This tells us that the entire 10x2 table consists of 400 points
> (TOTAL).
> >
> > > Of those, 23 points fell in the probability bin from 0.9 to 1.0
> >
> > > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
> > > in
> >
> > > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
> >
> > > The BASER_10 value from the PJC line type is the observation
event
> >
> > > frequency within that bin, and 0 / 23 = 0. So there were no
points
> >
> > > where the forecast value was between 0.9 and 1.0 and the
> > >observation was =18.0.
> >
> > >
> >
> > > Next, let's see if we can independently confirm this result...
> > > because
> >
> > > I assume you've already looked closely at this example. I used
the
> > > R
> >
> > > commands listed below to read the NetCDF output from Grid-Stat
and
> >
> > > simply plot fcst vs obs. As you can see in the attached image,
> >
> > > whenever the forecast value is >=0.9, the obs values are below
18,
> > > as
> > indicated by the horizontal line.
> >
> > >
> >
> > > Hope that helps clarify.
> >
> > >
> >
> > > Thanks,
> >
> > > John
> >
> > >
> >
> > >
> >
> > > R
> >
> > > library(ncdf4)
> >
> > > nc =
> >
> > >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> >
> > > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs =
ncvar_get(nc,
> >
> > > "OBS_HTSGW_Z0_box_mask")
> >
> > > pdf("scatter.pdf")
> >
> > > plot(fcst, obs, main="fcst vs obs")
> >
> > > abline(h=18)
> >
> > > dev.off()
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> >
> > > <mailto:met_help at ucar.edu> met_help at ucar.edu>
> >
> > > wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > >
> >
> > > > Dude,
> >
> > > >
> >
> > > > You sent them the whole globe?
> >
> > > >
> >
> > > > Buck
> >
> > > >
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:45 PM
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > No worries! Mr Sampson and I are really wondering why BASER_10
> > > > is
> >
> > > > zero even though of what attached images (plot_data_plane)
show.
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:24 PM
> >
> > > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > Hi Efren -
> >
> > > > sorry I see I already have the poly mask.
> >
> > > > Please disregard my last request.
> >
> > > > thanks,
> >
> > > > David
> >
> > > >
> >
> > > >
> >
> > > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
via
> >
> > > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > >
> >
> > > > > <URL:
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > > >
> >
> > > > > David - I forgot to edit the mask field on line 71 of
> >
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
> > > > > 19
> >
> > > > > 03
> >
> > > > > 02 12-120; it should be mask = { grid = []; poly =
> >
> > > > > ["sh172019-2019030712.nc"]; }
> >
> > > > >
> >
> > > > > Thanks for the quick turnaround!
> >
> > > > >
> >
> > > > > -----Original Message-----
> >
> > > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > > Sent: Friday, February 19, 2021 12:56 PM
> >
> > > > > To: Serra, Mr. Efren, Contractor, Code 7531
> >
> > > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > > Cc: Sampson, Mr. Buck <
<mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > > >
> >
> > > > > We have retrieved your test files for this grid_stat issue.
> >
> > > > > John and I will have a look.
> >
> > > > > thanks,
> >
> > > > > David
> >
> > > > >
> >
> > > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > via RT
> >
> > > > <
> >
> > > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> >
> > > > > > Transaction: Ticket created by <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > > Queue: met_help
> >
> > > > > > Subject: BASER question
> >
> > > > > > Owner: Nobody
> >
> > > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > > Status: new
> >
> > > > > > Ticket <URL:
> >
> > > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
> > > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > Folks - I placed the following files here:
> >
> > > > > > ftp> pwd
> >
> > > > > > 257 "/incoming/irap/met_help/serra" is the current
directory
> >
> > > > > >
> >
> > > > > > ftp> ls
> >
> > > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> >
> > > > > > 150 Opening ASCII mode data connection for file list
> >
> > > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> >
> > > > > >
> >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
> > > > 03
> >
> > > > 02
> >
> > > > 12-120
> >
> > > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> >
> > > > > >
> >
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
> > > > 0p
> > > > ro
> >
> > > > b_
> >
> > > > sig_wav_ht_gt18ft
> >
> > > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > >
> >
> > > > > > If you run the command below you are going to notice that
> >
> > > > > > BASER_10 is 0; why is that the case?
> >
> > > > > >
> >
> > > > > > Command:
> >
> > > > > > grid_stat
> >
> > > > > > US058GOCN-
GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0p
> >
> > > > > > ro
> >
> > > > > > b_
> >
> > > > > > sig_wav_ht_gt18ft
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.201903071
> > > > > > 2_
> > > > > > 20
> >
> > > > > > 19
> >
> > > > > > 03
> >
> > > > > > 02
> >
> > > > > > 12-120
> >
> > > > > > -outdir ./ -v 4
> >
> > > > > >
> >
> > > > > > Efren A. Serra (Contractor)
> >
> > > > > > Physicist
> >
> > > > > >
> >
> > > > > > DeVine Consulting, Inc.
> >
> > > > > > Naval Research Laboratory
> >
> > > > > > Marine Meteorology Division
> >
> > > > > > 7 Grace Hopper Ave., STOP 2
> >
> > > > > > Monterey, CA 93943
> >
> > > > > > Code 7542
> >
> > > > > > Mobile: 408-425-5027
> >
> > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Feb 23 11:08:25 2021
This was a nice thread to follow.
-----Original Message-----
From: Sampson, Mr. Buck via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 9:05 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
No, I'm good. Thanks for the offer. I think we have issues with our
numbers of cases, and that is what is throwing me off. I see "800"
cases for ".9" bin. We may be short one bin in our table,
specifically we might be missing the "1" or ".95" bin that we are
discussing. I hope ... at least that would explain what is happening
in that highest bin.
R,
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 8:26 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Buck,
Looking in the PSTD output line type, the BASER = 0.1275 output
indicates that the event (>=18) occurs at 12.75% of the grid points.
The BASER_1, BASER_2,... , BASER_10 values in the PJC line type give
the event frequency within that bin. I believe the reliability diagram
simply plots the mid-point of the forecast probability bin vs the
observed event frequency within that bin.
If needed, I could load the sample data you sent into METviewer
running through Docker on my laptop and make a reliability diagram.
John
On Tue, Feb 23, 2021 at 8:56 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> John,
>
> Wait! Can you comment on the second part? Am I dead wrong on that?
>
> Buck
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, February 23, 2021 7:47 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Buck,
>
> Sounds interesting. Since you mentioned the forecasts being biased,
> you might find percentile thresholds in MET to be useful. They can
be
> used to de-bias the data. Search for percentile thresholds on this
page:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlig
> ht=percentile%20thresholds
>
> One option I really like is setting an explicit observation
threshold,
> like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
> for the forecast. That tells MET to *choose* a threshold for the
> forecast that will result in a frequency bias as close to 1 as
possible.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Thanks John. I have thought about the mode capabilities. Time
and
> > money, of course. And concern about misplacement of the entire
> > event, which is a concern in operations. Your neighborhood
> > capabilities would also be more fair to any method that predicts
> > rare events (e.g., 90% of 18-ft waves) than an algorithm that
> > spreads out a bunch of low probabilities. Anyhow, we are learning
> > something about our method with MET that we didn't see before and
> > learning about MET in the process. Our post-processing is likely
> > high biased (yet to be proven), but is better at distinguishing
> > events from non-events that the ensemble without our tropical
> > cyclone post-processing. And we have easy ways to correct the
high bias, so that's all good.
> >
> >
> >
> > Reliability: Our question to you came up because I can't make
sense
> > of our Reliability Diagrams. The 20x20 verification grid you
should
> > have 18-ft waves in it, like on the order of 10% (see purple
contour
> > in second graphic). The BASER data looks to be a conditional
> > probability from what you sent, and I was under the impression
that
> > reliability diagrams were strictly a count (percentage) of
observed
> > vs forecast probabilities in our
> > 20x20 grid. I think the reliability is probably about right, even
> > in the highest bin. So even though the TC forecast was slow,
there
> > are still some verifying >18ft sig wave heights in the 20x20 grid.
> > In fact, the percentage looks to be about right!
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > R,
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, February 22, 2021 9:13 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> >
> >
> > Sounds good. You know the features in the images you sent look a
lot
> > like objects. And Buck, your comment about the forecast being
“slow”
> > reminds me of object-based verification.
> >
> >
> >
> > Not sure if you’ve already run the MODE tool in MET on this data
but
> >it could be a really nice application. You certainly can run MODE
to
> >compare objects from 2 different fields, like probabilities >25% vs
> >obs 18.
> >
> > Hopefully it’d give you more diagnostic info about the errors,
like
> > being too slow (centroid offset to the west) or fast (offset to
the
> east).
> >
> >
> >
> > MET also includes the MODE-Time-Domain tool to define 3D space-
time
> > objects.
> >
> >
> >
> > Thanks
> >
> > John
> >
> >
> >
> > On Mon, Feb 22, 2021 at 3:14 PM
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu> wrote:
> >
> >
> >
> > >
> >
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > >
> >
> > > Thanks for the R plot; I think this big 'little' script is great
> > > for
> >
> > > validation. I shall incorporate in my products.
> >
> > >
> >
> > > -----Original Message-----
> >
> > > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > Sent: Monday, February 22, 2021 1:21 PM
> >
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > >
> >
> > > Efren and Buck,
> >
> > >
> >
> > > David asked me to take a look at this. I just pulled down the
data
> > > and
> >
> > > ran Grid-Stat using MET version 9.1. I ran using the masking
> > > region
> >
> > > you provided (poly = ["sh172019-2019030712.nc"];) and I'm
looking
> > > at
> >
> > > the
> >
> > > BASER_10 column of the PJC output line. And you're wondering why
> >
> > > BASER_10 =
> >
> > > 0.0 there.
> >
> > >
> >
> > > All of the probabilistic verification is done in MET using an
Nx2
> >
> > > probabilistic contingency table. Since you configured Grid-Stat
> > > with
> >
> > > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > > it's
> >
> > > a
> >
> > > 10x2 contingency table. For each matched pair, we have a
forecast
> >
> > > probability value between 0 and 1 and observation value that's
> >
> > > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1.
Those
> >
> > > matched pairs are used to populate the 10x2 contingency table.
> >
> > >
> >
> > > Looking in the PCT (probabilistic contingency table) line type,
I
> >
> > > selected the following columns of output:
> >
> > >
> >
> > > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
> >
> > >
> >
> > > This tells us that the entire 10x2 table consists of 400 points
> (TOTAL).
> >
> > > Of those, 23 points fell in the probability bin from 0.9 to 1.0
> >
> > > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
> > > in
> >
> > > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
> >
> > > The BASER_10 value from the PJC line type is the observation
event
> >
> > > frequency within that bin, and 0 / 23 = 0. So there were no
points
> >
> > > where the forecast value was between 0.9 and 1.0 and the
> > >observation was =18.0.
> >
> > >
> >
> > > Next, let's see if we can independently confirm this result...
> > > because
> >
> > > I assume you've already looked closely at this example. I used
the
> > > R
> >
> > > commands listed below to read the NetCDF output from Grid-Stat
and
> >
> > > simply plot fcst vs obs. As you can see in the attached image,
> >
> > > whenever the forecast value is >=0.9, the obs values are below
18,
> > > as
> > indicated by the horizontal line.
> >
> > >
> >
> > > Hope that helps clarify.
> >
> > >
> >
> > > Thanks,
> >
> > > John
> >
> > >
> >
> > >
> >
> > > R
> >
> > > library(ncdf4)
> >
> > > nc =
> >
> > >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> >
> > > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs =
ncvar_get(nc,
> >
> > > "OBS_HTSGW_Z0_box_mask")
> >
> > > pdf("scatter.pdf")
> >
> > > plot(fcst, obs, main="fcst vs obs")
> >
> > > abline(h=18)
> >
> > > dev.off()
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> >
> > > <mailto:met_help at ucar.edu> met_help at ucar.edu>
> >
> > > wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > >
> >
> > > > Dude,
> >
> > > >
> >
> > > > You sent them the whole globe?
> >
> > > >
> >
> > > > Buck
> >
> > > >
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:45 PM
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > No worries! Mr Sampson and I are really wondering why BASER_10
> > > > is
> >
> > > > zero even though of what attached images (plot_data_plane)
show.
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:24 PM
> >
> > > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > Hi Efren -
> >
> > > > sorry I see I already have the poly mask.
> >
> > > > Please disregard my last request.
> >
> > > > thanks,
> >
> > > > David
> >
> > > >
> >
> > > >
> >
> > > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
via
> >
> > > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > >
> >
> > > > > <URL:
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > > >
> >
> > > > > David - I forgot to edit the mask field on line 71 of
> >
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
> > > > > 19
> >
> > > > > 03
> >
> > > > > 02 12-120; it should be mask = { grid = []; poly =
> >
> > > > > ["sh172019-2019030712.nc"]; }
> >
> > > > >
> >
> > > > > Thanks for the quick turnaround!
> >
> > > > >
> >
> > > > > -----Original Message-----
> >
> > > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > > Sent: Friday, February 19, 2021 12:56 PM
> >
> > > > > To: Serra, Mr. Efren, Contractor, Code 7531
> >
> > > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > > Cc: Sampson, Mr. Buck <
<mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > > >
> >
> > > > > We have retrieved your test files for this grid_stat issue.
> >
> > > > > John and I will have a look.
> >
> > > > > thanks,
> >
> > > > > David
> >
> > > > >
> >
> > > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > via RT
> >
> > > > <
> >
> > > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> >
> > > > > > Transaction: Ticket created by <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > > Queue: met_help
> >
> > > > > > Subject: BASER question
> >
> > > > > > Owner: Nobody
> >
> > > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > > Status: new
> >
> > > > > > Ticket <URL:
> >
> > > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
> > > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > Folks - I placed the following files here:
> >
> > > > > > ftp> pwd
> >
> > > > > > 257 "/incoming/irap/met_help/serra" is the current
directory
> >
> > > > > >
> >
> > > > > > ftp> ls
> >
> > > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> >
> > > > > > 150 Opening ASCII mode data connection for file list
> >
> > > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
> >
> > > > > >
> >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
> > > > 03
> >
> > > > 02
> >
> > > > 12-120
> >
> > > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
> >
> > > > > >
> >
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
> > > > 0p
> > > > ro
> >
> > > > b_
> >
> > > > sig_wav_ht_gt18ft
> >
> > > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > >
> >
> > > > > > If you run the command below you are going to notice that
> >
> > > > > > BASER_10 is 0; why is that the case?
> >
> > > > > >
> >
> > > > > > Command:
> >
> > > > > > grid_stat
> >
> > > > > > US058GOCN-
GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0p
> >
> > > > > > ro
> >
> > > > > > b_
> >
> > > > > > sig_wav_ht_gt18ft
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.201903071
> > > > > > 2_
> > > > > > 20
> >
> > > > > > 19
> >
> > > > > > 03
> >
> > > > > > 02
> >
> > > > > > 12-120
> >
> > > > > > -outdir ./ -v 4
> >
> > > > > >
> >
> > > > > > Efren A. Serra (Contractor)
> >
> > > > > > Physicist
> >
> > > > > >
> >
> > > > > > DeVine Consulting, Inc.
> >
> > > > > > Naval Research Laboratory
> >
> > > > > > Marine Meteorology Division
> >
> > > > > > 7 Grace Hopper Ave., STOP 2
> >
> > > > > > Monterey, CA 93943
> >
> > > > > > Code 7542
> >
> > > > > > Mobile: 408-425-5027
> >
> > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Feb 23 11:11:49 2021
This one is really good.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Buck,
Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.
I'll go ahead and resolve this ticket.
Thanks,
John
On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John. I have thought about the mode capabilities. Time and
> money, of course. And concern about misplacement of the entire
event,
> which is a concern in operations. Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities. Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process. Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing. And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams. The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic). The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid. I think the reliability is probably about right, even
in
> the highest bin. So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid. In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application. You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> > <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Queue: met_help
>
> > > > > Subject: BASER question
>
> > > > > Owner: Nobody
>
> > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Status: new
>
> > > > > Ticket <URL:
>
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>
------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 11:22:56 2021
I don't want to bias correct the waves, just the winds.
-----Original Message-----
From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 10:12 AM
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
This one is really good.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
Buck,
Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.
I'll go ahead and resolve this ticket.
Thanks,
John
On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John. I have thought about the mode capabilities. Time and
> money, of course. And concern about misplacement of the entire
event,
> which is a concern in operations. Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities. Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process. Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing. And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams. The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic). The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid. I think the reliability is probably about right, even
in
> the highest bin. So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid. In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application. You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400 0.9 0 23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> > <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL: <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > > <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Queue: met_help
>
> > > > > Subject: BASER question
>
> > > > > Owner: Nobody
>
> > > > > Requestors: <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > > Status: new
>
> > > > > Ticket <URL:
>
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r-- 1 ftp ftp 3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r-- 1 ftp ftp 908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r-- 1 ftp ftp 1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list