[Met_help] [rt.rap.ucar.edu #98769] History for BASER question

John Halley Gotway via RT met_help at ucar.edu
Tue Feb 23 12:52:01 MST 2021


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Folks - I placed the following files here:
ftp> pwd
257 "/incoming/irap/met_help/serra" is the current directory

ftp> ls
227 Entering Passive Mode (128,117,14,132,197,125).
150 Opening ASCII mode data connection for file list
-rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43 GridStatConfig_sh172019_gt18ft_WW3TCOFCL-WW3TCOFCL.2019030712_2019030212-120
-rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43 US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-000000prob_sig_wav_ht_gt18ft
-rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43 US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000000sig_wav_ht

If you run the command below you are going to notice that BASER_10 is 0; why is that the case?

Command:
grid_stat US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-000000prob_sig_wav_ht_gt18ft \
                 US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000000sig_wav_ht \
                 GridStatConfig_sh172019_gt18ft_WW3TCOFCL-WW3TCOFCL.2019030712_2019030212-120 -outdir ./ -v 4

Efren A. Serra (Contractor)
Physicist

DeVine Consulting, Inc.
Naval Research Laboratory
Marine Meteorology Division
7 Grace Hopper Ave., STOP 2
Monterey, CA 93943
Code 7542
Mobile: 408-425-5027



----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: BASER question
From: David Fillmore
Time: Fri Feb 19 13:55:41 2021

We have retrieved your test files for this grid_stat issue.
John and I will have a look.
thanks,
David

On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:

>
> Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
>        Queue: met_help
>      Subject: BASER question
>        Owner: Nobody
>   Requestors: efren.serra.ctr at nrlmry.navy.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
>
> Folks - I placed the following files here:
> ftp> pwd
> 257 "/incoming/irap/met_help/serra" is the current directory
>
> ftp> ls
> 227 Entering Passive Mode (128,117,14,132,197,125).
> 150 Opening ASCII mode data connection for file list
> -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_wav_ht
>
> If you run the command below you are going to notice that BASER_10
is 0;
> why is that the case?
>
> Command:
> grid_stat
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> \
>
>  US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_wav_ht
> \
>
>  GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> -outdir ./ -v 4
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>

------------------------------------------------
Subject: BASER question
From: David Fillmore
Time: Fri Feb 19 13:59:01 2021

User provided test files on
/d1/projects/MET/MET_Help/serra_data_20210219

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Feb 19 14:09:48 2021

David - I forgot to edit the mask field on line 71 of
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120; it should be
mask = { grid = []; poly = ["sh172019-2019030712.nc"]; }

Thanks for the quick turnaround!

-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 12:56 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

We have retrieved your test files for this grid_stat issue.
John and I will have a look.
thanks,
David

On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:

>
> Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
>        Queue: met_help
>      Subject: BASER question
>        Owner: Nobody
>   Requestors: efren.serra.ctr at nrlmry.navy.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
>
>
> Folks - I placed the following files here:
> ftp> pwd
> 257 "/incoming/irap/met_help/serra" is the current directory
>
> ftp> ls
> 227 Entering Passive Mode (128,117,14,132,197,125).
> 150 Opening ASCII mode data connection for file list
> -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> av_ht
>
> If you run the command below you are going to notice that BASER_10
is
> 0; why is that the case?
>
> Command:
> grid_stat
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> sig_wav_ht_gt18ft
> \
>
>
> US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> av_ht
> \
>
>
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120
> -outdir ./ -v 4
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>



------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
From: David Fillmore
Time: Fri Feb 19 14:20:02 2021

Hi Efren -
Can you upload via FTP your file sh172019-2019030712.nc for the poly
mask?
I'll modify the config.
Also, is this MET version 9.1.1 or an earlier version?
thanks
David

On Fri, Feb 19, 2021 at 2:09 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120; it should be
> mask = { grid = []; poly = ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: BASER question
> >        Owner: Nobody
> >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
is
> > 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Feb 19 14:23:08 2021

Mask uploaded.

-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:20 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Hi Efren -
Can you upload via FTP your file sh172019-2019030712.nc for the poly
mask?
I'll modify the config.
Also, is this MET version 9.1.1 or an earlier version?
thanks
David

On Fri, Feb 19, 2021 at 2:09 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120; it should be mask = { grid = []; poly =
> ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
> <efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: BASER question
> >        Owner: Nobody
> >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
> > is 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>



------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #98769] BASER question
From: David Fillmore
Time: Fri Feb 19 14:23:30 2021

Hi Efren -
sorry I see I already have the poly mask.
Please disregard my last request.
thanks,
David


On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120; it should be
> mask = { grid = []; poly = ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: BASER question
> >        Owner: Nobody
> >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
is
> > 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig_w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>

------------------------------------------------
Subject: BASER question
From: David Fillmore
Time: Fri Feb 19 14:28:39 2021

Added test.sh script to
kiowa:/d1/projects/MET/MET_Help/serra_data_20210219.
Confirmed BASER_10 is 0 with MET release 9.1.1.

------------------------------------------------
Subject: BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Feb 19 14:44:38 2021

No worries! Mr Sampson and I are really wondering why BASER_10 is zero
even though of what attached images (plot_data_plane) show.

-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:24 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Hi Efren -
sorry I see I already have the poly mask.
Please disregard my last request.
thanks,
David


On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120; it should be mask = { grid = []; poly =
> ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
> <efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: BASER question
> >        Owner: Nobody
> >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
> > is 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>


------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Fri Feb 19 16:26:47 2021

Dude,

You sent them the whole globe?

Buck


-----Original Message-----
From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:45 PM
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question

No worries! Mr Sampson and I are really wondering why BASER_10 is zero
even
though of what attached images (plot_data_plane) show.

-----Original Message-----
From: David Fillmore via RT <met_help at ucar.edu>
Sent: Friday, February 19, 2021 1:24 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Hi Efren -
sorry I see I already have the poly mask.
Please disregard my last request.
thanks,
David


On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via RT
<met_help at ucar.edu> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> David - I forgot to edit the mask field on line 71 of
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120; it should be mask = { grid = []; poly =
> ["sh172019-2019030712.nc"]; }
>
> Thanks for the quick turnaround!
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 12:56 PM
> To: Serra, Mr. Efren, Contractor, Code 7531
> <efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> We have retrieved your test files for this grid_stat issue.
> John and I will have a look.
> thanks,
> David
>
> On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: BASER question
> >        Owner: Nobody
> >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > >
> >
> >
> > Folks - I placed the following files here:
> > ftp> pwd
> > 257 "/incoming/irap/met_help/serra" is the current directory
> >
> > ftp> ls
> > 227 Entering Passive Mode (128,117,14,132,197,125).
> > 150 Opening ASCII mode data connection for file list
> > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> >
> > If you run the command below you are going to notice that BASER_10
> > is 0; why is that the case?
> >
> > Command:
> > grid_stat
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > b_
> > sig_wav_ht_gt18ft
> > \
> >
> >
> > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > _w
> > av_ht
> > \
> >
> >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02
> > 12-120
> > -outdir ./ -v 4
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>



------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Mon Feb 22 14:20:50 2021

Efren and Buck,

David asked me to take a look at this. I just pulled down the data and
ran
Grid-Stat using MET version 9.1. I ran using the masking region you
provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at the
BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
0.0 there.

All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a
10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded
(cat_thresh = >=18.0;) to turn it into a 0 or 1. Those matched pairs
are
used to populate the 10x2 contingency table.

Looking in the PCT (probabilistic contingency table) line type, I
selected
the following columns of output:

*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23    1*

This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of
those, 23 points fell in the  probability bin from 0.9 to 1.0
(THRESH_10 to
THRESH_11). Of those 23 points, the event occurred in the observation
0
times (OY_10) and did not occur 23 times (ON_10). The BASER_10 value
from
the PJC line type is the observation event frequency within that bin,
and 0
/ 23 = 0. So there were no points where the forecast value was between
0.9
and 1.0 and the observation was >=18.0.

Next, let's see if we can independently confirm this result... because
I
assume you've already looked closely at this example. I used the R
commands
listed below to read the NetCDF output from Grid-Stat and simply plot
fcst
vs obs. As you can see in the attached image, whenever the forecast
value
is >=0.9, the obs values are below 18, as indicated by the horizontal
line.

Hope that helps clarify.

Thanks,
John


R
library(ncdf4)
nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask")
obs = ncvar_get(nc, "OBS_HTSGW_Z0_box_mask")
pdf("scatter.pdf")
plot(fcst, obs, main="fcst vs obs")
abline(h=18)
dev.off()







On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Dude,
>
> You sent them the whole globe?
>
> Buck
>
>
> -----Original Message-----
> From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:45 PM
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> even
> though of what attached images (plot_data_plane) show.
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:24 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Hi Efren -
> sorry I see I already have the poly mask.
> Please disregard my last request.
> thanks,
> David
>
>
> On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> <met_help at ucar.edu> wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > David - I forgot to edit the mask field on line 71 of
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120; it should be mask = { grid = []; poly =
> > ["sh172019-2019030712.nc"]; }
> >
> > Thanks for the quick turnaround!
> >
> > -----Original Message-----
> > From: David Fillmore via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 12:56 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531
> > <efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > We have retrieved your test files for this grid_stat issue.
> > John and I will have a look.
> > thanks,
> > David
> >
> > On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil
via RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > >        Queue: met_help
> > >      Subject: BASER question
> > >        Owner: Nobody
> > >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> > >       Status: new
> > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > >
> > >
> > >
> > > Folks - I placed the following files here:
> > > ftp> pwd
> > > 257 "/incoming/irap/met_help/serra" is the current directory
> > >
> > > ftp> ls
> > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > 150 Opening ASCII mode data connection for file list
> > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > >
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019030212-120
> > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > >
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_sig_wav_ht_gt18ft
> > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > > _w
> > > av_ht
> > >
> > > If you run the command below you are going to notice that
BASER_10
> > > is 0; why is that the case?
> > >
> > > Command:
> > > grid_stat
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
> > > b_
> > > sig_wav_ht_gt18ft
> > > \
> > >
> > >
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000sig
> > > _w
> > > av_ht
> > > \
> > >
> > >
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > > 02
> > > 12-120
> > > -outdir ./ -v 4
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
>
>
>
>

------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Mon Feb 22 14:58:19 2021

Aha!  This is great.  I think maybe Efren’s area is correct and that
the 95% area predicted is just slower than actual event.  Just an
unlucky miss.



Buck

















































-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:21 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question



Efren and Buck,



David asked me to take a look at this. I just pulled down the data and
ran Grid-Stat using MET version 9.1. I ran using the masking region
you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
the

BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =

0.0 there.



All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a

10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
matched pairs are used to populate the 10x2 contingency table.



Looking in the PCT (probabilistic contingency table) line type, I
selected the following columns of output:



*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23    1*



This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of those, 23 points fell in the  probability bin from 0.9 to
1.0 (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in the observation 0 times (OY_10) and did not occur 23 times (ON_10).
The BASER_10 value from the PJC line type is the observation event
frequency within that bin, and 0 / 23 = 0. So there were no points
where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.



Next, let's see if we can independently confirm this result... because
I assume you've already looked closely at this example. I used the R
commands listed below to read the NetCDF output from Grid-Stat and
simply plot fcst vs obs. As you can see in the attached image,
whenever the forecast value is >=0.9, the obs values are below 18, as
indicated by the horizontal line.



Hope that helps clarify.



Thanks,

John





R

library(ncdf4)

nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")

fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
"OBS_HTSGW_Z0_box_mask")

pdf("scatter.pdf")

plot(fcst, obs, main="fcst vs obs")

abline(h=18)

dev.off()















On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
<mailto:met_help at ucar.edu> met_help at ucar.edu>

wrote:



>

> <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

>

> Dude,

>

> You sent them the whole globe?

>

> Buck

>

>

> -----Original Message-----

> From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> Sent: Friday, February 19, 2021 1:45 PM

> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> Subject: RE: [rt.rap.ucar.edu #98769] BASER question

>

> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero

> even though of what attached images (plot_data_plane) show.

>

> -----Original Message-----

> From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> Sent: Friday, February 19, 2021 1:24 PM

> To: Serra, Mr. Efren, Contractor, Code 7531 <

>  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> Subject: Re: [rt.rap.ucar.edu #98769] BASER question

>

> Hi Efren -

> sorry I see I already have the poly mask.

> Please disregard my last request.

> thanks,

> David

>

>

> On Fri, Feb 19, 2021 at 2:10 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT

> < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:

> >

> >

> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

> >

> > David - I forgot to edit the mask field on line 71 of

> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903

> > 02 12-120; it should be mask = { grid = []; poly =

> > ["sh172019-2019030712.nc"]; }

> >

> > Thanks for the quick turnaround!

> >

> > -----Original Message-----

> > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> > Sent: Friday, February 19, 2021 12:56 PM

> > To: Serra, Mr. Efren, Contractor, Code 7531

> > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question

> >

> > We have retrieved your test files for this grid_stat issue.

> > John and I will have a look.

> > thanks,

> > David

> >

> > On Fri, Feb 19, 2021 at 11:48 AM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via

> > RT

> <

> >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:

> >

> > >

> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.

> > > Transaction: Ticket created by
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > >        Queue: met_help

> > >      Subject: BASER question

> > >        Owner: Nobody

> > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > >       Status: new

> > >  Ticket <URL:

> > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769

> > > >

> > >

> > >

> > > Folks - I placed the following files here:

> > > ftp> pwd

> > > 257 "/incoming/irap/met_help/serra" is the current directory

> > >

> > > ftp> ls

> > > 227 Entering Passive Mode (128,117,14,132,197,125).

> > > 150 Opening ASCII mode data connection for file list

> > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43

> > >

> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302

> 12-120

> > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43

> > >

> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_

> sig_wav_ht_gt18ft

> > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43

> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s

> > > ig

> > > _w

> > > av_ht

> > >

> > > If you run the command below you are going to notice that
BASER_10

> > > is 0; why is that the case?

> > >

> > > Command:

> > > grid_stat

> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p

> > > ro

> > > b_

> > > sig_wav_ht_gt18ft

> > > \

> > >

> > >

> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s

> > > ig

> > > _w

> > > av_ht

> > > \

> > >

> > >

> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019

> > > 03

> > > 02

> > > 12-120

> > > -outdir ./ -v 4

> > >

> > > Efren A. Serra (Contractor)

> > > Physicist

> > >

> > > DeVine Consulting, Inc.

> > > Naval Research Laboratory

> > > Marine Meteorology Division

> > > 7 Grace Hopper Ave., STOP 2

> > > Monterey, CA 93943

> > > Code 7542

> > > Mobile: 408-425-5027

> > >

> > >

> > >

> >

> >

> >

>

>

>

>




------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon Feb 22 15:13:53 2021

Thank you very much, John!

-----Original Message-----
From: Sampson, Mr. Buck via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:58 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question

Aha!  This is great.  I think maybe Efren’s area is correct and that
the 95% area predicted is just slower than actual event.  Just an
unlucky miss.



Buck

















































-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:21 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question



Efren and Buck,



David asked me to take a look at this. I just pulled down the data and
ran Grid-Stat using MET version 9.1. I ran using the masking region
you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
the

BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =

0.0 there.



All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a

10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
matched pairs are used to populate the 10x2 contingency table.



Looking in the PCT (probabilistic contingency table) line type, I
selected the following columns of output:



*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23    1*



This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of those, 23 points fell in the  probability bin from 0.9 to
1.0 (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in the observation 0 times (OY_10) and did not occur 23 times (ON_10).
The BASER_10 value from the PJC line type is the observation event
frequency within that bin, and 0 / 23 = 0. So there were no points
where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.



Next, let's see if we can independently confirm this result... because
I assume you've already looked closely at this example. I used the R
commands listed below to read the NetCDF output from Grid-Stat and
simply plot fcst vs obs. As you can see in the attached image,
whenever the forecast value is >=0.9, the obs values are below 18, as
indicated by the horizontal line.



Hope that helps clarify.



Thanks,

John





R

library(ncdf4)

nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")

fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
"OBS_HTSGW_Z0_box_mask")

pdf("scatter.pdf")

plot(fcst, obs, main="fcst vs obs")

abline(h=18)

dev.off()















On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
<mailto:met_help at ucar.edu> met_help at ucar.edu>

wrote:



>

> <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

>

> Dude,

>

> You sent them the whole globe?

>

> Buck

>

>

> -----Original Message-----

> From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> Sent: Friday, February 19, 2021 1:45 PM

> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> Subject: RE: [rt.rap.ucar.edu #98769] BASER question

>

> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero

> even though of what attached images (plot_data_plane) show.

>

> -----Original Message-----

> From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> Sent: Friday, February 19, 2021 1:24 PM

> To: Serra, Mr. Efren, Contractor, Code 7531 <

>  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> Subject: Re: [rt.rap.ucar.edu #98769] BASER question

>

> Hi Efren -

> sorry I see I already have the poly mask.

> Please disregard my last request.

> thanks,

> David

>

>

> On Fri, Feb 19, 2021 at 2:10 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT

> < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:

> >

> >

> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

> >

> > David - I forgot to edit the mask field on line 71 of

> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903

> > 02 12-120; it should be mask = { grid = []; poly =

> > ["sh172019-2019030712.nc"]; }

> >

> > Thanks for the quick turnaround!

> >

> > -----Original Message-----

> > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> > Sent: Friday, February 19, 2021 12:56 PM

> > To: Serra, Mr. Efren, Contractor, Code 7531

> > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question

> >

> > We have retrieved your test files for this grid_stat issue.

> > John and I will have a look.

> > thanks,

> > David

> >

> > On Fri, Feb 19, 2021 at 11:48 AM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via

> > RT

> <

> >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:

> >

> > >

> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.

> > > Transaction: Ticket created by
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > >        Queue: met_help

> > >      Subject: BASER question

> > >        Owner: Nobody

> > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > >       Status: new

> > >  Ticket <URL:

> > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769

> > > >

> > >

> > >

> > > Folks - I placed the following files here:

> > > ftp> pwd

> > > 257 "/incoming/irap/met_help/serra" is the current directory

> > >

> > > ftp> ls

> > > 227 Entering Passive Mode (128,117,14,132,197,125).

> > > 150 Opening ASCII mode data connection for file list

> > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43

> > >

> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302

> 12-120

> > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43

> > >

> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_

> sig_wav_ht_gt18ft

> > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43

> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s

> > > ig

> > > _w

> > > av_ht

> > >

> > > If you run the command below you are going to notice that
BASER_10

> > > is 0; why is that the case?

> > >

> > > Command:

> > > grid_stat

> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p

> > > ro

> > > b_

> > > sig_wav_ht_gt18ft

> > > \

> > >

> > >

> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s

> > > ig

> > > _w

> > > av_ht

> > > \

> > >

> > >

> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019

> > > 03

> > > 02

> > > 12-120

> > > -outdir ./ -v 4

> > >

> > > Efren A. Serra (Contractor)

> > > Physicist

> > >

> > > DeVine Consulting, Inc.

> > > Naval Research Laboratory

> > > Marine Meteorology Division

> > > 7 Grace Hopper Ave., STOP 2

> > > Monterey, CA 93943

> > > Code 7542

> > > Mobile: 408-425-5027

> > >

> > >

> > >

> >

> >

> >

>

>

>

>






------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon Feb 22 15:14:32 2021

Thanks for the R plot; I think this big 'little' script is great for
validation. I shall incorporate in my products.

-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 1:21 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Efren and Buck,

David asked me to take a look at this. I just pulled down the data and
ran Grid-Stat using MET version 9.1. I ran using the masking region
you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at
the
BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
0.0 there.

All of the probabilistic verification is done in MET using an Nx2
probabilistic contingency table. Since you configured Grid-Stat with
"fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so it's
a
10x2 contingency table. For each matched pair, we have a forecast
probability value between 0 and 1 and observation value that's
thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
matched pairs are used to populate the 10x2 contingency table.

Looking in the PCT (probabilistic contingency table) line type, I
selected the following columns of output:

*TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23    1*

This tells us that the entire 10x2 table consists of 400 points
(TOTAL). Of those, 23 points fell in the  probability bin from 0.9 to
1.0 (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in the observation 0 times (OY_10) and did not occur 23 times (ON_10).
The BASER_10 value from the PJC line type is the observation event
frequency within that bin, and 0 / 23 = 0. So there were no points
where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.

Next, let's see if we can independently confirm this result... because
I assume you've already looked closely at this example. I used the R
commands listed below to read the NetCDF output from Grid-Stat and
simply plot fcst vs obs. As you can see in the attached image,
whenever the forecast value is >=0.9, the obs values are below 18, as
indicated by the horizontal line.

Hope that helps clarify.

Thanks,
John


R
library(ncdf4)
nc =
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
"OBS_HTSGW_Z0_box_mask")
pdf("scatter.pdf")
plot(fcst, obs, main="fcst vs obs")
abline(h=18)
dev.off()







On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Dude,
>
> You sent them the whole globe?
>
> Buck
>
>
> -----Original Message-----
> From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:45 PM
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> even though of what attached images (plot_data_plane) show.
>
> -----Original Message-----
> From: David Fillmore via RT <met_help at ucar.edu>
> Sent: Friday, February 19, 2021 1:24 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Hi Efren -
> sorry I see I already have the poly mask.
> Please disregard my last request.
> thanks,
> David
>
>
> On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> <met_help at ucar.edu> wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > David - I forgot to edit the mask field on line 71 of
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > 02 12-120; it should be mask = { grid = []; poly =
> > ["sh172019-2019030712.nc"]; }
> >
> > Thanks for the quick turnaround!
> >
> > -----Original Message-----
> > From: David Fillmore via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 12:56 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531
> > <efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > We have retrieved your test files for this grid_stat issue.
> > John and I will have a look.
> > thanks,
> > David
> >
> > On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil
via
> > RT
> <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > >        Queue: met_help
> > >      Subject: BASER question
> > >        Owner: Nobody
> > >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> > >       Status: new
> > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > >
> > >
> > >
> > > Folks - I placed the following files here:
> > > ftp> pwd
> > > 257 "/incoming/irap/met_help/serra" is the current directory
> > >
> > > ftp> ls
> > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > 150 Opening ASCII mode data connection for file list
> > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > >
> GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> 12-120
> > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > >
> US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> sig_wav_ht_gt18ft
> > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > >
> > > If you run the command below you are going to notice that
BASER_10
> > > is 0; why is that the case?
> > >
> > > Command:
> > > grid_stat
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
> > > b_
> > > sig_wav_ht_gt18ft
> > > \
> > >
> > >
> > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > ig
> > > _w
> > > av_ht
> > > \
> > >
> > >
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
> > > 02
> > > 12-120
> > > -outdir ./ -v 4
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
>
>
>
>



------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Mon Feb 22 22:12:59 2021

Sounds good. You know the features in the images you sent look a lot
like
objects. And Buck, your comment about the forecast being “slow”
reminds me
of object-based verification.

Not sure if you’ve already run the MODE tool in MET on this data but
it
could be a really nice application.  You certainly can run MODE to
compare
objects from 2 different fields, like probabilities >25% vs obs >18.
Hopefully it’d give you more diagnostic info about the errors, like
being
too slow (centroid offset to the west) or fast (offset to the east).

MET also includes the MODE-Time-Domain tool to define 3D space-time
objects.

Thanks
John

On Mon, Feb 22, 2021 at 3:14 PM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks for the R plot; I think this big 'little' script is great for
> validation. I shall incorporate in my products.
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 1:21 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Efren and Buck,
>
> David asked me to take a look at this. I just pulled down the data
and ran
> Grid-Stat using MET version 9.1. I ran using the masking region you
> provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at the
> BASER_10 column of the PJC output line. And you're wondering why
BASER_10 =
> 0.0 there.
>
> All of the probabilistic verification is done in MET using an Nx2
> probabilistic contingency table. Since you configured Grid-Stat with
> "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
it's a
> 10x2 contingency table. For each matched pair, we have a forecast
> probability value between 0 and 1 and observation value that's
thresholded
> (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those matched pairs
are
> used to populate the 10x2 contingency table.
>
> Looking in the PCT (probabilistic contingency table) line type, I
selected
> the following columns of output:
>
> *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23    1*
>
> This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
> Of those, 23 points fell in the  probability bin from 0.9 to 1.0
(THRESH_10
> to THRESH_11). Of those 23 points, the event occurred in the
observation 0
> times (OY_10) and did not occur 23 times (ON_10). The BASER_10 value
from
> the PJC line type is the observation event frequency within that
bin, and 0
> / 23 = 0. So there were no points where the forecast value was
between 0.9
> and 1.0 and the observation was >=18.0.
>
> Next, let's see if we can independently confirm this result...
because I
> assume you've already looked closely at this example. I used the R
commands
> listed below to read the NetCDF output from Grid-Stat and simply
plot fcst
> vs obs. As you can see in the attached image, whenever the forecast
value
> is >=0.9, the obs values are below 18, as indicated by the
horizontal line.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
>
> R
> library(ncdf4)
> nc =
>
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc")
> fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
> "OBS_HTSGW_Z0_box_mask")
> pdf("scatter.pdf")
> plot(fcst, obs, main="fcst vs obs")
> abline(h=18)
> dev.off()
>
>
>
>
>
>
>
> On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Dude,
> >
> > You sent them the whole globe?
> >
> > Buck
> >
> >
> > -----Original Message-----
> > From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 1:45 PM
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > No worries! Mr Sampson and I are really wondering why BASER_10 is
zero
> > even though of what attached images (plot_data_plane) show.
> >
> > -----Original Message-----
> > From: David Fillmore via RT <met_help at ucar.edu>
> > Sent: Friday, February 19, 2021 1:24 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > Hi Efren -
> > sorry I see I already have the poly mask.
> > Please disregard my last request.
> > thanks,
> > David
> >
> >
> > On Fri, Feb 19, 2021 at 2:10 PM efren.serra.ctr at nrlmry.navy.mil
via RT
> > <met_help at ucar.edu> wrote:
> > >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> > >
> > > David - I forgot to edit the mask field on line 71 of
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
> > > 02 12-120; it should be mask = { grid = []; poly =
> > > ["sh172019-2019030712.nc"]; }
> > >
> > > Thanks for the quick turnaround!
> > >
> > > -----Original Message-----
> > > From: David Fillmore via RT <met_help at ucar.edu>
> > > Sent: Friday, February 19, 2021 12:56 PM
> > > To: Serra, Mr. Efren, Contractor, Code 7531
> > > <efren.serra.ctr at nrlmry.navy.mil>
> > > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> > >
> > > We have retrieved your test files for this grid_stat issue.
> > > John and I will have a look.
> > > thanks,
> > > David
> > >
> > > On Fri, Feb 19, 2021 at 11:48 AM efren.serra.ctr at nrlmry.navy.mil
via
> > > RT
> > <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> > > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > >        Queue: met_help
> > > >      Subject: BASER question
> > > >        Owner: Nobody
> > > >   Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > >       Status: new
> > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> > > > >
> > > >
> > > >
> > > > Folks - I placed the following files here:
> > > > ftp> pwd
> > > > 257 "/incoming/irap/met_help/serra" is the current directory
> > > >
> > > > ftp> ls
> > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> > > > 150 Opening ASCII mode data connection for file list
> > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> > > >
> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20190302
> > 12-120
> > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> > > >
> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000prob_
> > sig_wav_ht_gt18ft
> > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > > ig
> > > > _w
> > > > av_ht
> > > >
> > > > If you run the command below you are going to notice that
BASER_10
> > > > is 0; why is that the case?
> > > >
> > > > Command:
> > > > grid_stat
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > > ro
> > > > b_
> > > > sig_wav_ht_gt18ft
> > > > \
> > > >
> > > >
> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000000s
> > > > ig
> > > > _w
> > > > av_ht
> > > > \
> > > >
> > > >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > > 03
> > > > 02
> > > > 12-120
> > > > -outdir ./ -v 4
> > > >
> > > > Efren A. Serra (Contractor)
> > > > Physicist
> > > >
> > > > DeVine Consulting, Inc.
> > > > Naval Research Laboratory
> > > > Marine Meteorology Division
> > > > 7 Grace Hopper Ave., STOP 2
> > > > Monterey, CA 93943
> > > > Code 7542
> > > > Mobile: 408-425-5027
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
>

------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 08:22:49 2021

Thanks John.  I have thought about the mode capabilities.  Time and
money, of course.  And concern about misplacement of the entire event,
which is a concern in operations.  Your neighborhood capabilities
would also be more fair to any method that predicts rare events (e.g.,
90% of 18-ft waves) than an algorithm that spreads out a bunch of low
probabilities.  Anyhow, we are learning something about our method
with MET that we didn't see before and learning about MET in the
process.  Our post-processing is likely high biased (yet to be
proven), but is better at distinguishing events from non-events that
the ensemble without our tropical cyclone post-processing.  And we
have easy ways to correct the high bias, so that's all good.



Reliability: Our question to you came up because I can't make sense of
our Reliability Diagrams.  The 20x20 verification grid you should have
18-ft waves in it, like on the order of 10% (see purple contour in
second graphic).  The BASER data looks to be a conditional probability
from what you sent, and I was under the impression that reliability
diagrams were strictly a count (percentage) of observed vs forecast
probabilities in our 20x20 grid.  I think the reliability is probably
about right, even in the highest bin.  So even though the TC forecast
was slow, there are still some verifying >18ft sig wave heights in the
20x20 grid.  In fact, the percentage looks to be about right!



Buck













R,



Buck





-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 22, 2021 9:13 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question



Sounds good. You know the features in the images you sent look a lot
like objects. And Buck, your comment about the forecast being “slow”
reminds me of object-based verification.



Not sure if you’ve already run the MODE tool in MET on this data but
it could be a really nice application.  You certainly can run MODE to
compare objects from 2 different fields, like probabilities >25% vs
obs >18.

Hopefully it’d give you more diagnostic info about the errors, like
being too slow (centroid offset to the west) or fast (offset to the
east).



MET also includes the MODE-Time-Domain tool to define 3D space-time
objects.



Thanks

John



On Mon, Feb 22, 2021 at 3:14 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT <  <mailto:met_help at ucar.edu>
met_help at ucar.edu> wrote:



>

> <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

>

> Thanks for the R plot; I think this big 'little' script is great for

> validation. I shall incorporate in my products.

>

> -----Original Message-----

> From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> Sent: Monday, February 22, 2021 1:21 PM

> To: Serra, Mr. Efren, Contractor, Code 7531 <

>  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> Subject: Re: [rt.rap.ucar.edu #98769] BASER question

>

> Efren and Buck,

>

> David asked me to take a look at this. I just pulled down the data
and

> ran Grid-Stat using MET version 9.1. I ran using the masking region

> you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking at

> the

> BASER_10 column of the PJC output line. And you're wondering why

> BASER_10 =

> 0.0 there.

>

> All of the probabilistic verification is done in MET using an Nx2

> probabilistic contingency table. Since you configured Grid-Stat with

> "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
it's

> a

> 10x2 contingency table. For each matched pair, we have a forecast

> probability value between 0 and 1 and observation value that's

> thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those

> matched pairs are used to populate the 10x2 contingency table.

>

> Looking in the PCT (probabilistic contingency table) line type, I

> selected the following columns of output:

>

> *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23    1*

>

> This tells us that the entire 10x2 table consists of 400 points
(TOTAL).

> Of those, 23 points fell in the  probability bin from 0.9 to 1.0

> (THRESH_10 to THRESH_11). Of those 23 points, the event occurred in

> the observation 0 times (OY_10) and did not occur 23 times (ON_10).

> The BASER_10 value from the PJC line type is the observation event

> frequency within that bin, and 0 / 23 = 0. So there were no points

> where the forecast value was between 0.9 and 1.0 and the observation
was >=18.0.

>

> Next, let's see if we can independently confirm this result...
because

> I assume you've already looked closely at this example. I used the R

> commands listed below to read the NetCDF output from Grid-Stat and

> simply plot fcst vs obs. As you can see in the attached image,

> whenever the forecast value is >=0.9, the obs values are below 18,
as indicated by the horizontal line.

>

> Hope that helps clarify.

>

> Thanks,

> John

>

>

> R

> library(ncdf4)

> nc =

>
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"

> ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,

> "OBS_HTSGW_Z0_box_mask")

> pdf("scatter.pdf")

> plot(fcst, obs, main="fcst vs obs")

> abline(h=18)

> dev.off()

>

>

>

>

>

>

>

> On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <

>  <mailto:met_help at ucar.edu> met_help at ucar.edu>

> wrote:

>

> >

> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

> >

> > Dude,

> >

> > You sent them the whole globe?

> >

> > Buck

> >

> >

> > -----Original Message-----

> > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> > Sent: Friday, February 19, 2021 1:45 PM

> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> > Subject: RE: [rt.rap.ucar.edu #98769] BASER question

> >

> > No worries! Mr Sampson and I are really wondering why BASER_10 is

> > zero even though of what attached images (plot_data_plane) show.

> >

> > -----Original Message-----

> > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> > Sent: Friday, February 19, 2021 1:24 PM

> > To: Serra, Mr. Efren, Contractor, Code 7531 <

> >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question

> >

> > Hi Efren -

> > sorry I see I already have the poly mask.

> > Please disregard my last request.

> > thanks,

> > David

> >

> >

> > On Fri, Feb 19, 2021 at 2:10 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil via

> > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:

> > >

> > >

> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >

> > >

> > > David - I forgot to edit the mask field on line 71 of

> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019

> > > 03

> > > 02 12-120; it should be mask = { grid = []; poly =

> > > ["sh172019-2019030712.nc"]; }

> > >

> > > Thanks for the quick turnaround!

> > >

> > > -----Original Message-----

> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
met_help at ucar.edu>

> > > Sent: Friday, February 19, 2021 12:56 PM

> > > To: Serra, Mr. Efren, Contractor, Code 7531

> > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil>

> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
Buck.Sampson at nrlmry.navy.mil>

> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question

> > >

> > > We have retrieved your test files for this grid_stat issue.

> > > John and I will have a look.

> > > thanks,

> > > David

> > >

> > > On Fri, Feb 19, 2021 at 11:48 AM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > > via RT

> > <

> > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:

> > >

> > > >

> > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.

> > > > Transaction: Ticket created by
<mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > > >        Queue: met_help

> > > >      Subject: BASER question

> > > >        Owner: Nobody

> > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
efren.serra.ctr at nrlmry.navy.mil

> > > >       Status: new

> > > >  Ticket <URL:

> > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769

> > > > >

> > > >

> > > >

> > > > Folks - I placed the following files here:

> > > > ftp> pwd

> > > > 257 "/incoming/irap/met_help/serra" is the current directory

> > > >

> > > > ftp> ls

> > > > 227 Entering Passive Mode (128,117,14,132,197,125).

> > > > 150 Opening ASCII mode data connection for file list

> > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43

> > > >

> > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903

> > 02

> > 12-120

> > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43

> > > >

> > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro

> > b_

> > sig_wav_ht_gt18ft

> > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43

> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000

> > > > 0s

> > > > ig

> > > > _w

> > > > av_ht

> > > >

> > > > If you run the command below you are going to notice that

> > > > BASER_10 is 0; why is that the case?

> > > >

> > > > Command:

> > > > grid_stat

> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000

> > > > 0p

> > > > ro

> > > > b_

> > > > sig_wav_ht_gt18ft

> > > > \

> > > >

> > > >

> > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000

> > > > 0s

> > > > ig

> > > > _w

> > > > av_ht

> > > > \

> > > >

> > > >

> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20

> > > > 19

> > > > 03

> > > > 02

> > > > 12-120

> > > > -outdir ./ -v 4

> > > >

> > > > Efren A. Serra (Contractor)

> > > > Physicist

> > > >

> > > > DeVine Consulting, Inc.

> > > > Naval Research Laboratory

> > > > Marine Meteorology Division

> > > > 7 Grace Hopper Ave., STOP 2

> > > > Monterey, CA 93943

> > > > Code 7542

> > > > Mobile: 408-425-5027

> > > >

> > > >

> > > >

> > >

> > >

> > >

> >

> >

> >

> >

>

>

>

>




------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Tue Feb 23 08:47:03 2021

Buck,

Sounds interesting. Since you mentioned the forecasts being biased,
you
might find percentile thresholds in MET to be useful. They can be used
to
de-bias the data. Search for percentile thresholds on this page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds

One option I really like is setting an explicit observation threshold,
like
"cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;" for the
forecast. That tells MET to *choose* a threshold for the forecast that
will
result in a frequency bias as close to 1 as possible.

I'll go ahead and resolve this ticket.

Thanks,
John

On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John.  I have thought about the mode capabilities.  Time and
money,
> of course.  And concern about misplacement of the entire event,
which is a
> concern in operations.  Your neighborhood capabilities would also be
more
> fair to any method that predicts rare events (e.g., 90% of 18-ft
waves)
> than an algorithm that spreads out a bunch of low probabilities.
Anyhow,
> we are learning something about our method with MET that we didn't
see
> before and learning about MET in the process.  Our post-processing
is
> likely high biased (yet to be proven), but is better at
distinguishing
> events from non-events that the ensemble without our tropical
cyclone
> post-processing.  And we have easy ways to correct the high bias, so
that's
> all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of our
> Reliability Diagrams.  The 20x20 verification grid you should have
18-ft
> waves in it, like on the order of 10% (see purple contour in second
> graphic).  The BASER data looks to be a conditional probability from
what
> you sent, and I was under the impression that reliability diagrams
were
> strictly a count (percentage) of observed vs forecast probabilities
in our
> 20x20 grid.  I think the reliability is probably about right, even
in the
> highest bin.  So even though the TC forecast was slow, there are
still some
> verifying >18ft sig wave heights in the 20x20 grid.  In fact, the
> percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
like
> objects. And Buck, your comment about the forecast being “slow”
reminds me
> of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
it
> could be a really nice application.  You certainly can run MODE to
compare
> objects from 2 different fields, like probabilities >25% vs obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
being
> too slow (centroid offset to the west) or fast (offset to the east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
<mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT <  <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation was
> >=18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >        Queue: met_help
>
> > > > >      Subject: BASER question
>
> > > > >        Owner: Nobody
>
> > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >       Status: new
>
> > > > >  Ticket <URL:
>
> > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_201903
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000pro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
00000
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>

------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 08:56:14 2021

John,

Wait!  Can you comment on the second part?  Am I dead wrong on that?

Buck


-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Buck,

Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds

One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.

I'll go ahead and resolve this ticket.

Thanks,
John

On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John.  I have thought about the mode capabilities.  Time and
> money, of course.  And concern about misplacement of the entire
event,
> which is a concern in operations.  Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities.  Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process.  Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing.  And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams.  The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic).  The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid.  I think the reliability is probably about right, even
in
> the highest bin.  So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid.  In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application.  You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT <  <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >        Queue: met_help
>
> > > > >      Subject: BASER question
>
> > > > >        Owner: Nobody
>
> > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >       Status: new
>
> > > > >  Ticket <URL:
>
> > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>


------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 09:24:14 2021

Ah, I think I am dead wrong.  Alas.

Buck


-----Original Message-----
From: Sampson, Mr. Buck
Sent: Tuesday, February 23, 2021 7:56 AM
To: 'met_help at ucar.edu' <met_help at ucar.edu>; Serra, Mr. Efren,
Contractor, Code 7531 <efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question

John,

Wait!  Can you comment on the second part?  Am I dead wrong on that?

Buck


-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Buck,

Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds

One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.

I'll go ahead and resolve this ticket.

Thanks,
John

On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John.  I have thought about the mode capabilities.  Time and
> money, of course.  And concern about misplacement of the entire
event,
> which is a concern in operations.  Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities.  Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process.  Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing.  And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams.  The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic).  The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid.  I think the reliability is probably about right, even
in
> the highest bin.  So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid.  In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application.  You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT <  <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >        Queue: met_help
>
> > > > >      Subject: BASER question
>
> > > > >        Owner: Nobody
>
> > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >       Status: new
>
> > > > >  Ticket <URL:
>
> > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>


------------------------------------------------
Subject: BASER question
From: John Halley Gotway
Time: Tue Feb 23 09:26:19 2021

Buck,

Looking in the PSTD output line type, the BASER = 0.1275 output
indicates
that the event (>=18) occurs at 12.75% of the grid points. The
BASER_1,
BASER_2,... , BASER_10 values in the PJC line type give the event
frequency within that bin. I believe the reliability diagram simply
plots
the mid-point of the forecast probability bin vs the observed event
frequency within that bin.

If needed, I could load the sample data you sent into METviewer
running
through Docker on my laptop and make a reliability diagram.

John

On Tue, Feb 23, 2021 at 8:56 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> John,
>
> Wait!  Can you comment on the second part?  Am I dead wrong on that?
>
> Buck
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, February 23, 2021 7:47 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Buck,
>
> Sounds interesting. Since you mentioned the forecasts being biased,
you
> might find percentile thresholds in MET to be useful. They can be
used to
> de-bias the data. Search for percentile thresholds on this page:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds
>
> One option I really like is setting an explicit observation
threshold,
> like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the
> forecast. That tells MET to *choose* a threshold for the forecast
that will
> result in a frequency bias as close to 1 as possible.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Thanks John.  I have thought about the mode capabilities.  Time
and
> > money, of course.  And concern about misplacement of the entire
event,
> > which is a concern in operations.  Your neighborhood capabilities
> > would also be more fair to any method that predicts rare events
(e.g.,
> > 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> > probabilities.  Anyhow, we are learning something about our method
> > with MET that we didn't see before and learning about MET in the
> > process.  Our post-processing is likely high biased (yet to be
> > proven), but is better at distinguishing events from non-events
that
> > the ensemble without our tropical cyclone post-processing.  And we
> > have easy ways to correct the high bias, so that's all good.
> >
> >
> >
> > Reliability: Our question to you came up because I can't make
sense of
> > our Reliability Diagrams.  The 20x20 verification grid you should
have
> > 18-ft waves in it, like on the order of 10% (see purple contour in
> > second graphic).  The BASER data looks to be a conditional
probability
> > from what you sent, and I was under the impression that
reliability
> > diagrams were strictly a count (percentage) of observed vs
forecast
> > probabilities in our
> > 20x20 grid.  I think the reliability is probably about right, even
in
> > the highest bin.  So even though the TC forecast was slow, there
are
> > still some verifying >18ft sig wave heights in the 20x20 grid.  In
> > fact, the percentage looks to be about right!
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > R,
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, February 22, 2021 9:13 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> >
> >
> > Sounds good. You know the features in the images you sent look a
lot
> > like objects. And Buck, your comment about the forecast being
“slow”
> > reminds me of object-based verification.
> >
> >
> >
> > Not sure if you’ve already run the MODE tool in MET on this data
but
> > it could be a really nice application.  You certainly can run MODE
to
> > compare objects from 2 different fields, like probabilities >25%
vs obs
> >18.
> >
> > Hopefully it’d give you more diagnostic info about the errors,
like
> > being too slow (centroid offset to the west) or fast (offset to
the
> east).
> >
> >
> >
> > MET also includes the MODE-Time-Domain tool to define 3D space-
time
> > objects.
> >
> >
> >
> > Thanks
> >
> > John
> >
> >
> >
> > On Mon, Feb 22, 2021 at 3:14 PM
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu> wrote:
> >
> >
> >
> > >
> >
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > >
> >
> > > Thanks for the R plot; I think this big 'little' script is great
for
> >
> > > validation. I shall incorporate in my products.
> >
> > >
> >
> > > -----Original Message-----
> >
> > > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > Sent: Monday, February 22, 2021 1:21 PM
> >
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > >
> >
> > > Efren and Buck,
> >
> > >
> >
> > > David asked me to take a look at this. I just pulled down the
data
> > > and
> >
> > > ran Grid-Stat using MET version 9.1. I ran using the masking
region
> >
> > > you provided (poly = ["sh172019-2019030712.nc"];) and I'm
looking at
> >
> > > the
> >
> > > BASER_10 column of the PJC output line. And you're wondering why
> >
> > > BASER_10 =
> >
> > > 0.0 there.
> >
> > >
> >
> > > All of the probabilistic verification is done in MET using an
Nx2
> >
> > > probabilistic contingency table. Since you configured Grid-Stat
with
> >
> > > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > > it's
> >
> > > a
> >
> > > 10x2 contingency table. For each matched pair, we have a
forecast
> >
> > > probability value between 0 and 1 and observation value that's
> >
> > > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1.
Those
> >
> > > matched pairs are used to populate the 10x2 contingency table.
> >
> > >
> >
> > > Looking in the PCT (probabilistic contingency table) line type,
I
> >
> > > selected the following columns of output:
> >
> > >
> >
> > > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
> >
> > >
> >
> > > This tells us that the entire 10x2 table consists of 400 points
> (TOTAL).
> >
> > > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
> >
> > > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
> >
> > > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
> >
> > > The BASER_10 value from the PJC line type is the observation
event
> >
> > > frequency within that bin, and 0 / 23 = 0. So there were no
points
> >
> > > where the forecast value was between 0.9 and 1.0 and the
observation
> > >was =18.0.
> >
> > >
> >
> > > Next, let's see if we can independently confirm this result...
> > > because
> >
> > > I assume you've already looked closely at this example. I used
the R
> >
> > > commands listed below to read the NetCDF output from Grid-Stat
and
> >
> > > simply plot fcst vs obs. As you can see in the attached image,
> >
> > > whenever the forecast value is >=0.9, the obs values are below
18,
> > > as
> > indicated by the horizontal line.
> >
> > >
> >
> > > Hope that helps clarify.
> >
> > >
> >
> > > Thanks,
> >
> > > John
> >
> > >
> >
> > >
> >
> > > R
> >
> > > library(ncdf4)
> >
> > > nc =
> >
> > >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> >
> > > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs =
ncvar_get(nc,
> >
> > > "OBS_HTSGW_Z0_box_mask")
> >
> > > pdf("scatter.pdf")
> >
> > > plot(fcst, obs, main="fcst vs obs")
> >
> > > abline(h=18)
> >
> > > dev.off()
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> >
> > >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
> >
> > > wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > >
> >
> > > > Dude,
> >
> > > >
> >
> > > > You sent them the whole globe?
> >
> > > >
> >
> > > > Buck
> >
> > > >
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:45 PM
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > No worries! Mr Sampson and I are really wondering why BASER_10
is
> >
> > > > zero even though of what attached images (plot_data_plane)
show.
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:24 PM
> >
> > > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > Hi Efren -
> >
> > > > sorry I see I already have the poly mask.
> >
> > > > Please disregard my last request.
> >
> > > > thanks,
> >
> > > > David
> >
> > > >
> >
> > > >
> >
> > > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
via
> >
> > > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > >
> >
> > > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > > >
> >
> > > > > David - I forgot to edit the mask field on line 71 of
> >
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > > 19
> >
> > > > > 03
> >
> > > > > 02 12-120; it should be mask = { grid = []; poly =
> >
> > > > > ["sh172019-2019030712.nc"]; }
> >
> > > > >
> >
> > > > > Thanks for the quick turnaround!
> >
> > > > >
> >
> > > > > -----Original Message-----
> >
> > > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > > Sent: Friday, February 19, 2021 12:56 PM
> >
> > > > > To: Serra, Mr. Efren, Contractor, Code 7531
> >
> > > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > > Cc: Sampson, Mr. Buck <
<mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > > >
> >
> > > > > We have retrieved your test files for this grid_stat issue.
> >
> > > > > John and I will have a look.
> >
> > > > > thanks,
> >
> > > > > David
> >
> > > > >
> >
> > > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > via RT
> >
> > > > <
> >
> > > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> >
> > > > > > Transaction: Ticket created by  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > >        Queue: met_help
> >
> > > > > >      Subject: BASER question
> >
> > > > > >        Owner: Nobody
> >
> > > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > >       Status: new
> >
> > > > > >  Ticket <URL:
> >
> > > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
> > > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > Folks - I placed the following files here:
> >
> > > > > > ftp> pwd
> >
> > > > > > 257 "/incoming/irap/met_help/serra" is the current
directory
> >
> > > > > >
> >
> > > > > > ftp> ls
> >
> > > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> >
> > > > > > 150 Opening ASCII mode data connection for file list
> >
> > > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> >
> > > > > >
> >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > > 03
> >
> > > > 02
> >
> > > > 12-120
> >
> > > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> >
> > > > > >
> >
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > > ro
> >
> > > > b_
> >
> > > > sig_wav_ht_gt18ft
> >
> > > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > >
> >
> > > > > > If you run the command below you are going to notice that
> >
> > > > > > BASER_10 is 0; why is that the case?
> >
> > > > > >
> >
> > > > > > Command:
> >
> > > > > > grid_stat
> >
> > > > > > US058GOCN-
GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-000
> > > > > > 00
> >
> > > > > > 0p
> >
> > > > > > ro
> >
> > > > > > b_
> >
> > > > > > sig_wav_ht_gt18ft
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-000
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > > 20
> >
> > > > > > 19
> >
> > > > > > 03
> >
> > > > > > 02
> >
> > > > > > 12-120
> >
> > > > > > -outdir ./ -v 4
> >
> > > > > >
> >
> > > > > > Efren A. Serra (Contractor)
> >
> > > > > > Physicist
> >
> > > > > >
> >
> > > > > > DeVine Consulting, Inc.
> >
> > > > > > Naval Research Laboratory
> >
> > > > > > Marine Meteorology Division
> >
> > > > > > 7 Grace Hopper Ave., STOP 2
> >
> > > > > > Monterey, CA 93943
> >
> > > > > > Code 7542
> >
> > > > > > Mobile: 408-425-5027
> >
> > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 10:05:04 2021

No, I'm good.  Thanks for the offer.  I think we have issues with our
numbers of cases, and that is what is throwing me off.  I see "800"
cases for ".9" bin.  We may be short one bin in our table,
specifically we might be missing the "1" or ".95" bin that we are
discussing.  I hope ... at least that would explain what is happening
in that highest bin.

R,

Buck


-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 8:26 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Buck,

Looking in the PSTD output line type, the BASER = 0.1275 output
indicates that the event (>=18) occurs at 12.75% of the grid points.
The BASER_1, BASER_2,... , BASER_10 values in the PJC line type give
the event frequency within that bin. I believe the reliability diagram
simply plots the mid-point of the forecast probability bin vs the
observed event frequency within that bin.

If needed, I could load the sample data you sent into METviewer
running through Docker on my laptop and make a reliability diagram.

John

On Tue, Feb 23, 2021 at 8:56 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> John,
>
> Wait!  Can you comment on the second part?  Am I dead wrong on that?
>
> Buck
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, February 23, 2021 7:47 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Buck,
>
> Sounds interesting. Since you mentioned the forecasts being biased,
> you might find percentile thresholds in MET to be useful. They can
be
> used to de-bias the data. Search for percentile thresholds on this
page:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlig
> ht=percentile%20thresholds
>
> One option I really like is setting an explicit observation
threshold,
> like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
> for the forecast. That tells MET to *choose* a threshold for the
> forecast that will result in a frequency bias as close to 1 as
possible.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Thanks John.  I have thought about the mode capabilities.  Time
and
> > money, of course.  And concern about misplacement of the entire
> > event, which is a concern in operations.  Your neighborhood
> > capabilities would also be more fair to any method that predicts
> > rare events (e.g., 90% of 18-ft waves) than an algorithm that
> > spreads out a bunch of low probabilities.  Anyhow, we are learning
> > something about our method with MET that we didn't see before and
> > learning about MET in the process.  Our post-processing is likely
> > high biased (yet to be proven), but is better at distinguishing
> > events from non-events that the ensemble without our tropical
> > cyclone post-processing.  And we have easy ways to correct the
high bias, so that's all good.
> >
> >
> >
> > Reliability: Our question to you came up because I can't make
sense
> > of our Reliability Diagrams.  The 20x20 verification grid you
should
> > have 18-ft waves in it, like on the order of 10% (see purple
contour
> > in second graphic).  The BASER data looks to be a conditional
> > probability from what you sent, and I was under the impression
that
> > reliability diagrams were strictly a count (percentage) of
observed
> > vs forecast probabilities in our
> > 20x20 grid.  I think the reliability is probably about right, even
> > in the highest bin.  So even though the TC forecast was slow,
there
> > are still some verifying >18ft sig wave heights in the 20x20 grid.
> > In fact, the percentage looks to be about right!
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > R,
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, February 22, 2021 9:13 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> >
> >
> > Sounds good. You know the features in the images you sent look a
lot
> > like objects. And Buck, your comment about the forecast being
“slow”
> > reminds me of object-based verification.
> >
> >
> >
> > Not sure if you’ve already run the MODE tool in MET on this data
but
> >it could be a really nice application.  You certainly can run MODE
to
> >compare objects from 2 different fields, like probabilities >25% vs
> >obs 18.
> >
> > Hopefully it’d give you more diagnostic info about the errors,
like
> > being too slow (centroid offset to the west) or fast (offset to
the
> east).
> >
> >
> >
> > MET also includes the MODE-Time-Domain tool to define 3D space-
time
> > objects.
> >
> >
> >
> > Thanks
> >
> > John
> >
> >
> >
> > On Mon, Feb 22, 2021 at 3:14 PM
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu> wrote:
> >
> >
> >
> > >
> >
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > >
> >
> > > Thanks for the R plot; I think this big 'little' script is great
> > > for
> >
> > > validation. I shall incorporate in my products.
> >
> > >
> >
> > > -----Original Message-----
> >
> > > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > Sent: Monday, February 22, 2021 1:21 PM
> >
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > >
> >
> > > Efren and Buck,
> >
> > >
> >
> > > David asked me to take a look at this. I just pulled down the
data
> > > and
> >
> > > ran Grid-Stat using MET version 9.1. I ran using the masking
> > > region
> >
> > > you provided (poly = ["sh172019-2019030712.nc"];) and I'm
looking
> > > at
> >
> > > the
> >
> > > BASER_10 column of the PJC output line. And you're wondering why
> >
> > > BASER_10 =
> >
> > > 0.0 there.
> >
> > >
> >
> > > All of the probabilistic verification is done in MET using an
Nx2
> >
> > > probabilistic contingency table. Since you configured Grid-Stat
> > > with
> >
> > > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > > it's
> >
> > > a
> >
> > > 10x2 contingency table. For each matched pair, we have a
forecast
> >
> > > probability value between 0 and 1 and observation value that's
> >
> > > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1.
Those
> >
> > > matched pairs are used to populate the 10x2 contingency table.
> >
> > >
> >
> > > Looking in the PCT (probabilistic contingency table) line type,
I
> >
> > > selected the following columns of output:
> >
> > >
> >
> > > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
> >
> > >
> >
> > > This tells us that the entire 10x2 table consists of 400 points
> (TOTAL).
> >
> > > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
> >
> > > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
> > > in
> >
> > > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
> >
> > > The BASER_10 value from the PJC line type is the observation
event
> >
> > > frequency within that bin, and 0 / 23 = 0. So there were no
points
> >
> > > where the forecast value was between 0.9 and 1.0 and the
> > >observation was =18.0.
> >
> > >
> >
> > > Next, let's see if we can independently confirm this result...
> > > because
> >
> > > I assume you've already looked closely at this example. I used
the
> > > R
> >
> > > commands listed below to read the NetCDF output from Grid-Stat
and
> >
> > > simply plot fcst vs obs. As you can see in the attached image,
> >
> > > whenever the forecast value is >=0.9, the obs values are below
18,
> > > as
> > indicated by the horizontal line.
> >
> > >
> >
> > > Hope that helps clarify.
> >
> > >
> >
> > > Thanks,
> >
> > > John
> >
> > >
> >
> > >
> >
> > > R
> >
> > > library(ncdf4)
> >
> > > nc =
> >
> > >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> >
> > > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs =
ncvar_get(nc,
> >
> > > "OBS_HTSGW_Z0_box_mask")
> >
> > > pdf("scatter.pdf")
> >
> > > plot(fcst, obs, main="fcst vs obs")
> >
> > > abline(h=18)
> >
> > > dev.off()
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> >
> > >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
> >
> > > wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > >
> >
> > > > Dude,
> >
> > > >
> >
> > > > You sent them the whole globe?
> >
> > > >
> >
> > > > Buck
> >
> > > >
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:45 PM
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > No worries! Mr Sampson and I are really wondering why BASER_10
> > > > is
> >
> > > > zero even though of what attached images (plot_data_plane)
show.
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:24 PM
> >
> > > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > Hi Efren -
> >
> > > > sorry I see I already have the poly mask.
> >
> > > > Please disregard my last request.
> >
> > > > thanks,
> >
> > > > David
> >
> > > >
> >
> > > >
> >
> > > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
via
> >
> > > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > >
> >
> > > > > <URL:
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > > >
> >
> > > > > David - I forgot to edit the mask field on line 71 of
> >
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
> > > > > 19
> >
> > > > > 03
> >
> > > > > 02 12-120; it should be mask = { grid = []; poly =
> >
> > > > > ["sh172019-2019030712.nc"]; }
> >
> > > > >
> >
> > > > > Thanks for the quick turnaround!
> >
> > > > >
> >
> > > > > -----Original Message-----
> >
> > > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > > Sent: Friday, February 19, 2021 12:56 PM
> >
> > > > > To: Serra, Mr. Efren, Contractor, Code 7531
> >
> > > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > > Cc: Sampson, Mr. Buck <
<mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > > >
> >
> > > > > We have retrieved your test files for this grid_stat issue.
> >
> > > > > John and I will have a look.
> >
> > > > > thanks,
> >
> > > > > David
> >
> > > > >
> >
> > > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > via RT
> >
> > > > <
> >
> > > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> >
> > > > > > Transaction: Ticket created by  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > >        Queue: met_help
> >
> > > > > >      Subject: BASER question
> >
> > > > > >        Owner: Nobody
> >
> > > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > >       Status: new
> >
> > > > > >  Ticket <URL:
> >
> > > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
> > > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > Folks - I placed the following files here:
> >
> > > > > > ftp> pwd
> >
> > > > > > 257 "/incoming/irap/met_help/serra" is the current
directory
> >
> > > > > >
> >
> > > > > > ftp> ls
> >
> > > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> >
> > > > > > 150 Opening ASCII mode data connection for file list
> >
> > > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> >
> > > > > >
> >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
> > > > 03
> >
> > > > 02
> >
> > > > 12-120
> >
> > > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> >
> > > > > >
> >
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
> > > > 0p
> > > > ro
> >
> > > > b_
> >
> > > > sig_wav_ht_gt18ft
> >
> > > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > >
> >
> > > > > > If you run the command below you are going to notice that
> >
> > > > > > BASER_10 is 0; why is that the case?
> >
> > > > > >
> >
> > > > > > Command:
> >
> > > > > > grid_stat
> >
> > > > > > US058GOCN-
GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0p
> >
> > > > > > ro
> >
> > > > > > b_
> >
> > > > > > sig_wav_ht_gt18ft
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.201903071
> > > > > > 2_
> > > > > > 20
> >
> > > > > > 19
> >
> > > > > > 03
> >
> > > > > > 02
> >
> > > > > > 12-120
> >
> > > > > > -outdir ./ -v 4
> >
> > > > > >
> >
> > > > > > Efren A. Serra (Contractor)
> >
> > > > > > Physicist
> >
> > > > > >
> >
> > > > > > DeVine Consulting, Inc.
> >
> > > > > > Naval Research Laboratory
> >
> > > > > > Marine Meteorology Division
> >
> > > > > > 7 Grace Hopper Ave., STOP 2
> >
> > > > > > Monterey, CA 93943
> >
> > > > > > Code 7542
> >
> > > > > > Mobile: 408-425-5027
> >
> > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Feb 23 11:08:25 2021

This was a nice thread to follow.

-----Original Message-----
From: Sampson, Mr. Buck via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 9:05 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question

No, I'm good.  Thanks for the offer.  I think we have issues with our
numbers of cases, and that is what is throwing me off.  I see "800"
cases for ".9" bin.  We may be short one bin in our table,
specifically we might be missing the "1" or ".95" bin that we are
discussing.  I hope ... at least that would explain what is happening
in that highest bin.

R,

Buck


-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 8:26 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Buck,

Looking in the PSTD output line type, the BASER = 0.1275 output
indicates that the event (>=18) occurs at 12.75% of the grid points.
The BASER_1, BASER_2,... , BASER_10 values in the PJC line type give
the event frequency within that bin. I believe the reliability diagram
simply plots the mid-point of the forecast probability bin vs the
observed event frequency within that bin.

If needed, I could load the sample data you sent into METviewer
running through Docker on my laptop and make a reliability diagram.

John

On Tue, Feb 23, 2021 at 8:56 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> John,
>
> Wait!  Can you comment on the second part?  Am I dead wrong on that?
>
> Buck
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, February 23, 2021 7:47 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> Buck,
>
> Sounds interesting. Since you mentioned the forecasts being biased,
> you might find percentile thresholds in MET to be useful. They can
be
> used to de-bias the data. Search for percentile thresholds on this
page:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlig
> ht=percentile%20thresholds
>
> One option I really like is setting an explicit observation
threshold,
> like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
> for the forecast. That tells MET to *choose* a threshold for the
> forecast that will result in a frequency bias as close to 1 as
possible.
>
> I'll go ahead and resolve this ticket.
>
> Thanks,
> John
>
> On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > Thanks John.  I have thought about the mode capabilities.  Time
and
> > money, of course.  And concern about misplacement of the entire
> > event, which is a concern in operations.  Your neighborhood
> > capabilities would also be more fair to any method that predicts
> > rare events (e.g., 90% of 18-ft waves) than an algorithm that
> > spreads out a bunch of low probabilities.  Anyhow, we are learning
> > something about our method with MET that we didn't see before and
> > learning about MET in the process.  Our post-processing is likely
> > high biased (yet to be proven), but is better at distinguishing
> > events from non-events that the ensemble without our tropical
> > cyclone post-processing.  And we have easy ways to correct the
high bias, so that's all good.
> >
> >
> >
> > Reliability: Our question to you came up because I can't make
sense
> > of our Reliability Diagrams.  The 20x20 verification grid you
should
> > have 18-ft waves in it, like on the order of 10% (see purple
contour
> > in second graphic).  The BASER data looks to be a conditional
> > probability from what you sent, and I was under the impression
that
> > reliability diagrams were strictly a count (percentage) of
observed
> > vs forecast probabilities in our
> > 20x20 grid.  I think the reliability is probably about right, even
> > in the highest bin.  So even though the TC forecast was slow,
there
> > are still some verifying >18ft sig wave heights in the 20x20 grid.
> > In fact, the percentage looks to be about right!
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > R,
> >
> >
> >
> > Buck
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, February 22, 2021 9:13 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> >
> >
> > Sounds good. You know the features in the images you sent look a
lot
> > like objects. And Buck, your comment about the forecast being
“slow”
> > reminds me of object-based verification.
> >
> >
> >
> > Not sure if you’ve already run the MODE tool in MET on this data
but
> >it could be a really nice application.  You certainly can run MODE
to
> >compare objects from 2 different fields, like probabilities >25% vs
> >obs 18.
> >
> > Hopefully it’d give you more diagnostic info about the errors,
like
> > being too slow (centroid offset to the west) or fast (offset to
the
> east).
> >
> >
> >
> > MET also includes the MODE-Time-Domain tool to define 3D space-
time
> > objects.
> >
> >
> >
> > Thanks
> >
> > John
> >
> >
> >
> > On Mon, Feb 22, 2021 at 3:14 PM
> > <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu> wrote:
> >
> >
> >
> > >
> >
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > >
> >
> > > Thanks for the R plot; I think this big 'little' script is great
> > > for
> >
> > > validation. I shall incorporate in my products.
> >
> > >
> >
> > > -----Original Message-----
> >
> > > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > Sent: Monday, February 22, 2021 1:21 PM
> >
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > >
> >
> > > Efren and Buck,
> >
> > >
> >
> > > David asked me to take a look at this. I just pulled down the
data
> > > and
> >
> > > ran Grid-Stat using MET version 9.1. I ran using the masking
> > > region
> >
> > > you provided (poly = ["sh172019-2019030712.nc"];) and I'm
looking
> > > at
> >
> > > the
> >
> > > BASER_10 column of the PJC output line. And you're wondering why
> >
> > > BASER_10 =
> >
> > > 0.0 there.
> >
> > >
> >
> > > All of the probabilistic verification is done in MET using an
Nx2
> >
> > > probabilistic contingency table. Since you configured Grid-Stat
> > > with
> >
> > > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > > it's
> >
> > > a
> >
> > > 10x2 contingency table. For each matched pair, we have a
forecast
> >
> > > probability value between 0 and 1 and observation value that's
> >
> > > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1.
Those
> >
> > > matched pairs are used to populate the 10x2 contingency table.
> >
> > >
> >
> > > Looking in the PCT (probabilistic contingency table) line type,
I
> >
> > > selected the following columns of output:
> >
> > >
> >
> > > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
> >
> > >
> >
> > > This tells us that the entire 10x2 table consists of 400 points
> (TOTAL).
> >
> > > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
> >
> > > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
> > > in
> >
> > > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
> >
> > > The BASER_10 value from the PJC line type is the observation
event
> >
> > > frequency within that bin, and 0 / 23 = 0. So there were no
points
> >
> > > where the forecast value was between 0.9 and 1.0 and the
> > >observation was =18.0.
> >
> > >
> >
> > > Next, let's see if we can independently confirm this result...
> > > because
> >
> > > I assume you've already looked closely at this example. I used
the
> > > R
> >
> > > commands listed below to read the NetCDF output from Grid-Stat
and
> >
> > > simply plot fcst vs obs. As you can see in the attached image,
> >
> > > whenever the forecast value is >=0.9, the obs values are below
18,
> > > as
> > indicated by the horizontal line.
> >
> > >
> >
> > > Hope that helps clarify.
> >
> > >
> >
> > > Thanks,
> >
> > > John
> >
> > >
> >
> > >
> >
> > > R
> >
> > > library(ncdf4)
> >
> > > nc =
> >
> > >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
> >
> > > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs =
ncvar_get(nc,
> >
> > > "OBS_HTSGW_Z0_box_mask")
> >
> > > pdf("scatter.pdf")
> >
> > > plot(fcst, obs, main="fcst vs obs")
> >
> > > abline(h=18)
> >
> > > dev.off()
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> > > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
> >
> > >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
> >
> > > wrote:
> >
> > >
> >
> > > >
> >
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > >
> >
> > > > Dude,
> >
> > > >
> >
> > > > You sent them the whole globe?
> >
> > > >
> >
> > > > Buck
> >
> > > >
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil via RT <
<mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:45 PM
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > No worries! Mr Sampson and I are really wondering why BASER_10
> > > > is
> >
> > > > zero even though of what attached images (plot_data_plane)
show.
> >
> > > >
> >
> > > > -----Original Message-----
> >
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > Sent: Friday, February 19, 2021 1:24 PM
> >
> > > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> >
> > > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > >
> >
> > > > Hi Efren -
> >
> > > > sorry I see I already have the poly mask.
> >
> > > > Please disregard my last request.
> >
> > > > thanks,
> >
> > > > David
> >
> > > >
> >
> > > >
> >
> > > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
via
> >
> > > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > >
> >
> > > > > <URL:
> > > > > <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
> >
> > > > >
> >
> > > > > David - I forgot to edit the mask field on line 71 of
> >
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
> > > > > 19
> >
> > > > > 03
> >
> > > > > 02 12-120; it should be mask = { grid = []; poly =
> >
> > > > > ["sh172019-2019030712.nc"]; }
> >
> > > > >
> >
> > > > > Thanks for the quick turnaround!
> >
> > > > >
> >
> > > > > -----Original Message-----
> >
> > > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> > met_help at ucar.edu>
> >
> > > > > Sent: Friday, February 19, 2021 12:56 PM
> >
> > > > > To: Serra, Mr. Efren, Contractor, Code 7531
> >
> > > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil>
> >
> > > > > Cc: Sampson, Mr. Buck <
<mailto:Buck.Sampson at nrlmry.navy.mil>
> > Buck.Sampson at nrlmry.navy.mil>
> >
> > > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
> >
> > > > >
> >
> > > > > We have retrieved your test files for this grid_stat issue.
> >
> > > > > John and I will have a look.
> >
> > > > > thanks,
> >
> > > > > David
> >
> > > > >
> >
> > > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > via RT
> >
> > > > <
> >
> > > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
> >
> > > > >
> >
> > > > > >
> >
> > > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
> >
> > > > > > Transaction: Ticket created by  <mailto:
> > efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > >        Queue: met_help
> >
> > > > > >      Subject: BASER question
> >
> > > > > >        Owner: Nobody
> >
> > > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> > efren.serra.ctr at nrlmry.navy.mil
> >
> > > > > >       Status: new
> >
> > > > > >  Ticket <URL:
> >
> > > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
> >
> > > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > Folks - I placed the following files here:
> >
> > > > > > ftp> pwd
> >
> > > > > > 257 "/incoming/irap/met_help/serra" is the current
directory
> >
> > > > > >
> >
> > > > > > ftp> ls
> >
> > > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
> >
> > > > > > 150 Opening ASCII mode data connection for file list
> >
> > > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
> >
> > > > > >
> >
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
> > > > 03
> >
> > > > 02
> >
> > > > 12-120
> >
> > > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
> >
> > > > > >
> >
> > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
00000
> > > > 0p
> > > > ro
> >
> > > > b_
> >
> > > > sig_wav_ht_gt18ft
> >
> > > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > >
> >
> > > > > > If you run the command below you are going to notice that
> >
> > > > > > BASER_10 is 0; why is that the case?
> >
> > > > > >
> >
> > > > > > Command:
> >
> > > > > > grid_stat
> >
> > > > > > US058GOCN-
GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0p
> >
> > > > > > ro
> >
> > > > > > b_
> >
> > > > > > sig_wav_ht_gt18ft
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > US058GOCN-
GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-0
> > > > > > 00
> > > > > > 00
> >
> > > > > > 0s
> >
> > > > > > ig
> >
> > > > > > _w
> >
> > > > > > av_ht
> >
> > > > > > \
> >
> > > > > >
> >
> > > > > >
> >
> > > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.201903071
> > > > > > 2_
> > > > > > 20
> >
> > > > > > 19
> >
> > > > > > 03
> >
> > > > > > 02
> >
> > > > > > 12-120
> >
> > > > > > -outdir ./ -v 4
> >
> > > > > >
> >
> > > > > > Efren A. Serra (Contractor)
> >
> > > > > > Physicist
> >
> > > > > >
> >
> > > > > > DeVine Consulting, Inc.
> >
> > > > > > Naval Research Laboratory
> >
> > > > > > Marine Meteorology Division
> >
> > > > > > 7 Grace Hopper Ave., STOP 2
> >
> > > > > > Monterey, CA 93943
> >
> > > > > > Code 7542
> >
> > > > > > Mobile: 408-425-5027
> >
> > > > > >
> >
> > > > > >
> >
> > > > > >
> >
> > > > >
> >
> > > > >
> >
> > > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > > >
> >
> > >
> >
> > >
> >
> > >
> >
> > >
> >
> >
> >
> >
> >
>
>
>




------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #98769] BASER question
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue Feb 23 11:11:49 2021

This one is really good.

-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Buck,

Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds

One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.

I'll go ahead and resolve this ticket.

Thanks,
John

On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John.  I have thought about the mode capabilities.  Time and
> money, of course.  And concern about misplacement of the entire
event,
> which is a concern in operations.  Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities.  Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process.  Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing.  And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams.  The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic).  The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid.  I think the reliability is probably about right, even
in
> the highest bin.  So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid.  In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application.  You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT <  <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >        Queue: met_help
>
> > > > >      Subject: BASER question
>
> > > > >        Owner: Nobody
>
> > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >       Status: new
>
> > > > >  Ticket <URL:
>
> > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>



------------------------------------------------
Subject: BASER question
From: Sampson, Mr. Buck
Time: Tue Feb 23 11:22:56 2021

I don't want to bias correct the waves, just the winds.

-----Original Message-----
From: efren.serra.ctr at nrlmry.navy.mil via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 10:12 AM
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #98769] BASER question

This one is really good.

-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 23, 2021 7:47 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #98769] BASER question

Buck,

Sounds interesting. Since you mentioned the forecasts being biased,
you might find percentile thresholds in MET to be useful. They can be
used to de-bias the data. Search for percentile thresholds on this
page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html?highlight=percentile%20thresholds

One option I really like is setting an explicit observation threshold,
like "cat_thresh = >18;", and then setting "cat_thresh = ==FBIAS1;"
for the forecast. That tells MET to *choose* a threshold for the
forecast that will result in a frequency bias as close to 1 as
possible.

I'll go ahead and resolve this ticket.

Thanks,
John

On Tue, Feb 23, 2021 at 8:23 AM Sampson, Mr. Buck via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> Thanks John.  I have thought about the mode capabilities.  Time and
> money, of course.  And concern about misplacement of the entire
event,
> which is a concern in operations.  Your neighborhood capabilities
> would also be more fair to any method that predicts rare events
(e.g.,
> 90% of 18-ft waves) than an algorithm that spreads out a bunch of
low
> probabilities.  Anyhow, we are learning something about our method
> with MET that we didn't see before and learning about MET in the
> process.  Our post-processing is likely high biased (yet to be
> proven), but is better at distinguishing events from non-events that
> the ensemble without our tropical cyclone post-processing.  And we
> have easy ways to correct the high bias, so that's all good.
>
>
>
> Reliability: Our question to you came up because I can't make sense
of
> our Reliability Diagrams.  The 20x20 verification grid you should
have
> 18-ft waves in it, like on the order of 10% (see purple contour in
> second graphic).  The BASER data looks to be a conditional
probability
> from what you sent, and I was under the impression that reliability
> diagrams were strictly a count (percentage) of observed vs forecast
> probabilities in our
> 20x20 grid.  I think the reliability is probably about right, even
in
> the highest bin.  So even though the TC forecast was slow, there are
> still some verifying >18ft sig wave heights in the 20x20 grid.  In
> fact, the percentage looks to be about right!
>
>
>
> Buck
>
>
>
>
>
>
>
>
>
>
>
>
>
> R,
>
>
>
> Buck
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 22, 2021 9:13 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
>
>
> Sounds good. You know the features in the images you sent look a lot
> like objects. And Buck, your comment about the forecast being “slow”
> reminds me of object-based verification.
>
>
>
> Not sure if you’ve already run the MODE tool in MET on this data but
> it could be a really nice application.  You certainly can run MODE
to
> compare objects from 2 different fields, like probabilities >25% vs
obs >18.
>
> Hopefully it’d give you more diagnostic info about the errors, like
> being too slow (centroid offset to the west) or fast (offset to the
east).
>
>
>
> MET also includes the MODE-Time-Domain tool to define 3D space-time
> objects.
>
>
>
> Thanks
>
> John
>
>
>
> On Mon, Feb 22, 2021 at 3:14 PM
> <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT <  <mailto:met_help at ucar.edu>
> met_help at ucar.edu> wrote:
>
>
>
> >
>
> > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> >
>
> > Thanks for the R plot; I think this big 'little' script is great
for
>
> > validation. I shall incorporate in my products.
>
> >
>
> > -----Original Message-----
>
> > From: John Halley Gotway via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > Sent: Monday, February 22, 2021 1:21 PM
>
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> >
>
> > Efren and Buck,
>
> >
>
> > David asked me to take a look at this. I just pulled down the data
> > and
>
> > ran Grid-Stat using MET version 9.1. I ran using the masking
region
>
> > you provided (poly = ["sh172019-2019030712.nc"];) and I'm looking
at
>
> > the
>
> > BASER_10 column of the PJC output line. And you're wondering why
>
> > BASER_10 =
>
> > 0.0 there.
>
> >
>
> > All of the probabilistic verification is done in MET using an Nx2
>
> > probabilistic contingency table. Since you configured Grid-Stat
with
>
> > "fcst.cat_thresh = ==0.1;", it's using 10 probability bins... so
> > it's
>
> > a
>
> > 10x2 contingency table. For each matched pair, we have a forecast
>
> > probability value between 0 and 1 and observation value that's
>
> > thresholded (cat_thresh = >=18.0;) to turn it into a 0 or 1. Those
>
> > matched pairs are used to populate the 10x2 contingency table.
>
> >
>
> > Looking in the PCT (probabilistic contingency table) line type, I
>
> > selected the following columns of output:
>
> >
>
> > *TOTAL THRESH_10 OY_10 ON_10 THRESH_11400   0.9       0     23
1*
>
> >
>
> > This tells us that the entire 10x2 table consists of 400 points
(TOTAL).
>
> > Of those, 23 points fell in the  probability bin from 0.9 to 1.0
>
> > (THRESH_10 to THRESH_11). Of those 23 points, the event occurred
in
>
> > the observation 0 times (OY_10) and did not occur 23 times
(ON_10).
>
> > The BASER_10 value from the PJC line type is the observation event
>
> > frequency within that bin, and 0 / 23 = 0. So there were no points
>
> > where the forecast value was between 0.9 and 1.0 and the
observation
> >was =18.0.
>
> >
>
> > Next, let's see if we can independently confirm this result...
> > because
>
> > I assume you've already looked closely at this example. I used the
R
>
> > commands listed below to read the NetCDF output from Grid-Stat and
>
> > simply plot fcst vs obs. As you can see in the attached image,
>
> > whenever the forecast value is >=0.9, the obs values are below 18,
> > as
> indicated by the horizontal line.
>
> >
>
> > Hope that helps clarify.
>
> >
>
> > Thanks,
>
> > John
>
> >
>
> >
>
> > R
>
> > library(ncdf4)
>
> > nc =
>
> >
nc_open("grid_stat_sh172019_gt18ft_1200000L_20190307_120000V_pairs.nc"
>
> > ) fcst = ncvar_get(nc, "FCST_LMH_Z0_box_mask") obs = ncvar_get(nc,
>
> > "OBS_HTSGW_Z0_box_mask")
>
> > pdf("scatter.pdf")
>
> > plot(fcst, obs, main="fcst vs obs")
>
> > abline(h=18)
>
> > dev.off()
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Fri, Feb 19, 2021 at 4:27 PM Sampson, Mr. Buck via RT <
>
> >  <mailto:met_help at ucar.edu> met_help at ucar.edu>
>
> > wrote:
>
> >
>
> > >
>
> > > <URL:  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > >
>
> > > Dude,
>
> > >
>
> > > You sent them the whole globe?
>
> > >
>
> > > Buck
>
> > >
>
> > >
>
> > > -----Original Message-----
>
> > > From:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:45 PM
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: RE: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > No worries! Mr Sampson and I are really wondering why BASER_10
is
>
> > > zero even though of what attached images (plot_data_plane) show.
>
> > >
>
> > > -----Original Message-----
>
> > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > Sent: Friday, February 19, 2021 1:24 PM
>
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
>
> > >  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > >
>
> > > Hi Efren -
>
> > > sorry I see I already have the poly mask.
>
> > > Please disregard my last request.
>
> > > thanks,
>
> > > David
>
> > >
>
> > >
>
> > > On Fri, Feb 19, 2021 at 2:10 PM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil via
>
> > > RT < <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > >
>
> > > > <URL:
<https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769 >
>
> > > >
>
> > > > David - I forgot to edit the mask field on line 71 of
>
> > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_20
> > > > 19
>
> > > > 03
>
> > > > 02 12-120; it should be mask = { grid = []; poly =
>
> > > > ["sh172019-2019030712.nc"]; }
>
> > > >
>
> > > > Thanks for the quick turnaround!
>
> > > >
>
> > > > -----Original Message-----
>
> > > > From: David Fillmore via RT < <mailto:met_help at ucar.edu>
> met_help at ucar.edu>
>
> > > > Sent: Friday, February 19, 2021 12:56 PM
>
> > > > To: Serra, Mr. Efren, Contractor, Code 7531
>
> > > > < <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil>
>
> > > > Cc: Sampson, Mr. Buck < <mailto:Buck.Sampson at nrlmry.navy.mil>
> Buck.Sampson at nrlmry.navy.mil>
>
> > > > Subject: Re: [rt.rap.ucar.edu #98769] BASER question
>
> > > >
>
> > > > We have retrieved your test files for this grid_stat issue.
>
> > > > John and I will have a look.
>
> > > > thanks,
>
> > > > David
>
> > > >
>
> > > > On Fri, Feb 19, 2021 at 11:48 AM  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > via RT
>
> > > <
>
> > > >  <mailto:met_help at ucar.edu> met_help at ucar.edu> wrote:
>
> > > >
>
> > > > >
>
> > > > > Fri Feb 19 11:47:53 2021: Request 98769 was acted upon.
>
> > > > > Transaction: Ticket created by  <mailto:
> efren.serra.ctr at nrlmry.navy.mil> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >        Queue: met_help
>
> > > > >      Subject: BASER question
>
> > > > >        Owner: Nobody
>
> > > > >   Requestors:  <mailto:efren.serra.ctr at nrlmry.navy.mil>
> efren.serra.ctr at nrlmry.navy.mil
>
> > > > >       Status: new
>
> > > > >  Ticket <URL:
>
> > > > >  <https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98769
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > > Folks - I placed the following files here:
>
> > > > > ftp> pwd
>
> > > > > 257 "/incoming/irap/met_help/serra" is the current directory
>
> > > > >
>
> > > > > ftp> ls
>
> > > > > 227 Entering Passive Mode (128,117,14,132,197,125).
>
> > > > > 150 Opening ASCII mode data connection for file list
>
> > > > > -rw-r--r--   1 ftp      ftp          3357 Feb 19 18:43
>
> > > > >
>
> > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_2019
> > > 03
>
> > > 02
>
> > > 12-120
>
> > > > > -rw-r--r--   1 ftp      ftp        908544 Feb 19 18:43
>
> > > > >
>
> > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000000p
> > > ro
>
> > > b_
>
> > > sig_wav_ht_gt18ft
>
> > > > > -rw-r--r--   1 ftp      ftp       1376998 Feb 19 18:43
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > >
>
> > > > > If you run the command below you are going to notice that
>
> > > > > BASER_10 is 0; why is that the case?
>
> > > > >
>
> > > > > Command:
>
> > > > > grid_stat
>
> > > > > US058GOCN-GR1mdl.0011_0255_12000U0RL2019030212_0001_000000-
000
> > > > > 00
>
> > > > > 0p
>
> > > > > ro
>
> > > > > b_
>
> > > > > sig_wav_ht_gt18ft
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > US058GOCN-GR1mdl.0095_0200_00000F0RL2019030712_0001_000000-
000
> > > > > 00
>
> > > > > 0s
>
> > > > > ig
>
> > > > > _w
>
> > > > > av_ht
>
> > > > > \
>
> > > > >
>
> > > > >
>
> > > > > GridStatConfig_sh172019_gt18ft_WW3TCOFCL-
WW3TCOFCL.2019030712_
> > > > > 20
>
> > > > > 19
>
> > > > > 03
>
> > > > > 02
>
> > > > > 12-120
>
> > > > > -outdir ./ -v 4
>
> > > > >
>
> > > > > Efren A. Serra (Contractor)
>
> > > > > Physicist
>
> > > > >
>
> > > > > DeVine Consulting, Inc.
>
> > > > > Naval Research Laboratory
>
> > > > > Marine Meteorology Division
>
> > > > > 7 Grace Hopper Ave., STOP 2
>
> > > > > Monterey, CA 93943
>
> > > > > Code 7542
>
> > > > > Mobile: 408-425-5027
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > >
>
> > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
> >
>
> >
>
> >
>
>
>
>
>




------------------------------------------------


More information about the Met_help mailing list