[Met_help] [rt.rap.ucar.edu #95756] History for Sample_file for BSS issue in SREF verification
Tatiana Burek via RT
met_help at ucar.edu
Tue Oct 13 12:14:22 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Met_help, John,
I have created a complete testing case for looking at
the BSS issue as we discussed in METPLUS meeting today.
The sample files directory is on Mars:
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
The running script is run.sh
Thanks!
Binbin
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: John Halley Gotway
Time: Mon Jun 29 13:46:16 2020
Binbin,
Thanks for sending this example. I took a look in
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss and see that
you
run.sh is...
(1) Running ensemble_stat to generate some probability forecasts.
(2) Running grid_stat to verify those probability forecasts.
And your question is why is BSS = NA in the output of Grid-Stat?
I'm looking in:
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
And see that BSS = NA but BSS_SMPL does NOT = NA, at least for the
first 4
lines. BSS is computed relative to a climatology forecast whereas
BSS_SMPL
is computed relative to a sample climatology. The issue is pretty
clear
when looking at your Grid-Stat config file... you haven't defined a
climo
mean and standard deviation in verf_g2g_grid_stat_sref_config_ndas.
Those
sections are empty. Please try specifying the climo mean and standard
deviation... and then the BSS column should be populated.
Hope that does the trick.
Thanks,
John
On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
> Transaction: Ticket created by Binbin.Zhou at noaa.gov
> Queue: met_help
> Subject: Sample_file for BSS issue in SREF verification
> Owner: Nobody
> Requestors: Binbin.Zhou at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
>
>
> Met_help, John,
>
> I have created a complete testing case for looking at
> the BSS issue as we discussed in METPLUS meeting today.
> The sample files directory is on Mars:
> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> The running script is run.sh
>
> Thanks!
>
> Binbin
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Mon Jun 29 14:27:36 2020
John,
OK, I see the BSS_SMPLY data
Thanks!
Binbin
On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> Thanks for sending this example. I took a look in
> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss and see
that you
> run.sh is...
> (1) Running ensemble_stat to generate some probability forecasts.
> (2) Running grid_stat to verify those probability forecasts.
>
> And your question is why is BSS = NA in the output of Grid-Stat?
>
> I'm looking in:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
>
> And see that BSS = NA but BSS_SMPL does NOT = NA, at least for the
first 4
> lines. BSS is computed relative to a climatology forecast whereas
BSS_SMPL
> is computed relative to a sample climatology. The issue is pretty
clear
> when looking at your Grid-Stat config file... you haven't defined a
climo
> mean and standard deviation in verf_g2g_grid_stat_sref_config_ndas.
Those
> sections are empty. Please try specifying the climo mean and
standard
> deviation... and then the BSS column should be populated.
>
> Hope that does the trick.
>
> Thanks,
> John
>
> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> > Queue: met_help
> > Subject: Sample_file for BSS issue in SREF verification
> > Owner: Nobody
> > Requestors: Binbin.Zhou at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
> >
> >
> > Met_help, John,
> >
> > I have created a complete testing case for looking at
> > the BSS issue as we discussed in METPLUS meeting today.
> > The sample files directory is on Mars:
> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > The running script is run.sh
> >
> > Thanks!
> >
> > Binbin
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Mon Jun 29 14:35:48 2020
John,
I can see BSS_SMPL data in stat tiles, but why I still can not
generate the BSS_SMPL plot
from MetViewer?
Binbin
On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA Affiliate <
binbin.zhou at noaa.gov> wrote:
> John,
>
> OK, I see the BSS_SMPLY data
>
> Thanks!
>
> Binbin
>
> On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Binbin,
>>
>> Thanks for sending this example. I took a look in
>> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss and see
that you
>> run.sh is...
>> (1) Running ensemble_stat to generate some probability forecasts.
>> (2) Running grid_stat to verify those probability forecasts.
>>
>> And your question is why is BSS = NA in the output of Grid-Stat?
>>
>> I'm looking in:
>>
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
>>
>> And see that BSS = NA but BSS_SMPL does NOT = NA, at least for the
first 4
>> lines. BSS is computed relative to a climatology forecast whereas
BSS_SMPL
>> is computed relative to a sample climatology. The issue is pretty
clear
>> when looking at your Grid-Stat config file... you haven't defined a
climo
>> mean and standard deviation in verf_g2g_grid_stat_sref_config_ndas.
Those
>> sections are empty. Please try specifying the climo mean and
standard
>> deviation... and then the BSS column should be populated.
>>
>> Hope that does the trick.
>>
>> Thanks,
>> John
>>
>> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
>> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
>> > Queue: met_help
>> > Subject: Sample_file for BSS issue in SREF verification
>> > Owner: Nobody
>> > Requestors: Binbin.Zhou at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
>> >
>> >
>> > Met_help, John,
>> >
>> > I have created a complete testing case for looking at
>> > the BSS issue as we discussed in METPLUS meeting today.
>> > The sample files directory is on Mars:
>> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
>> > The running script is run.sh
>> >
>> > Thanks!
>> >
>> > Binbin
>> >
>> > --
>> >
>> > Binbin Zhou
>> >
>> > IMSG at NOAA/NWS/NCEP/EMC
>> >
>> > 5830 University Research Ct.
>> >
>> > College Park, MD 20740
>> >
>> > Binbin.Zhou at noaa.gov
>> >
>> > 301-683-3683
>> >
>> >
>>
>>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: John Halley Gotway
Time: Mon Jun 29 14:42:49 2020
Binbin,
Can you please send along the XML file for your attempt to plot
BSS_SMPL
with METviewer? I'll reassign this ticket to Tatiana so she can
investigate
what's going on there.
Thanks,
John
On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
>
> John,
>
> I can see BSS_SMPL data in stat tiles, but why I still can not
> generate the BSS_SMPL plot
> from MetViewer?
>
> Binbin
>
> On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA Affiliate <
> binbin.zhou at noaa.gov> wrote:
>
> > John,
> >
> > OK, I see the BSS_SMPLY data
> >
> > Thanks!
> >
> > Binbin
> >
> > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Binbin,
> >>
> >> Thanks for sending this example. I took a look in
> >> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss and see
that
> you
> >> run.sh is...
> >> (1) Running ensemble_stat to generate some probability forecasts.
> >> (2) Running grid_stat to verify those probability forecasts.
> >>
> >> And your question is why is BSS = NA in the output of Grid-Stat?
> >>
> >> I'm looking in:
> >>
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> >>
> >> And see that BSS = NA but BSS_SMPL does NOT = NA, at least for
the
> first 4
> >> lines. BSS is computed relative to a climatology forecast whereas
> BSS_SMPL
> >> is computed relative to a sample climatology. The issue is pretty
clear
> >> when looking at your Grid-Stat config file... you haven't defined
a
> climo
> >> mean and standard deviation in
verf_g2g_grid_stat_sref_config_ndas.
> Those
> >> sections are empty. Please try specifying the climo mean and
standard
> >> deviation... and then the BSS column should be populated.
> >>
> >> Hope that does the trick.
> >>
> >> Thanks,
> >> John
> >>
> >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
> >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> >> > Queue: met_help
> >> > Subject: Sample_file for BSS issue in SREF verification
> >> > Owner: Nobody
> >> > Requestors: Binbin.Zhou at noaa.gov
> >> > Status: new
> >> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> >
> >> >
> >> >
> >> > Met_help, John,
> >> >
> >> > I have created a complete testing case for looking at
> >> > the BSS issue as we discussed in METPLUS meeting today.
> >> > The sample files directory is on Mars:
> >> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> >> > The running script is run.sh
> >> >
> >> > Thanks!
> >> >
> >> > Binbin
> >> >
> >> > --
> >> >
> >> > Binbin Zhou
> >> >
> >> > IMSG at NOAA/NWS/NCEP/EMC
> >> >
> >> > 5830 University Research Ct.
> >> >
> >> > College Park, MD 20740
> >> >
> >> > Binbin.Zhou at noaa.gov
> >> >
> >> > 301-683-3683
> >> >
> >> >
> >>
> >>
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Mon Jun 29 16:27:42 2020
John,
Sure, a testing XML file is attached
Binbin
On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> Can you please send along the XML file for your attempt to plot
BSS_SMPL
> with METviewer? I'll reassign this ticket to Tatiana so she can
investigate
> what's going on there.
>
> Thanks,
> John
>
> On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
> >
> > John,
> >
> > I can see BSS_SMPL data in stat tiles, but why I still can not
> > generate the BSS_SMPL plot
> > from MetViewer?
> >
> > Binbin
> >
> > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA Affiliate <
> > binbin.zhou at noaa.gov> wrote:
> >
> > > John,
> > >
> > > OK, I see the BSS_SMPLY data
> > >
> > > Thanks!
> > >
> > > Binbin
> > >
> > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Binbin,
> > >>
> > >> Thanks for sending this example. I took a look in
> > >> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss and
see that
> > you
> > >> run.sh is...
> > >> (1) Running ensemble_stat to generate some probability
forecasts.
> > >> (2) Running grid_stat to verify those probability forecasts.
> > >>
> > >> And your question is why is BSS = NA in the output of Grid-
Stat?
> > >>
> > >> I'm looking in:
> > >>
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> > >>
> > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at least for
the
> > first 4
> > >> lines. BSS is computed relative to a climatology forecast
whereas
> > BSS_SMPL
> > >> is computed relative to a sample climatology. The issue is
pretty
> clear
> > >> when looking at your Grid-Stat config file... you haven't
defined a
> > climo
> > >> mean and standard deviation in
verf_g2g_grid_stat_sref_config_ndas.
> > Those
> > >> sections are empty. Please try specifying the climo mean and
standard
> > >> deviation... and then the BSS column should be populated.
> > >>
> > >> Hope that does the trick.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
> > >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> > >> > Queue: met_help
> > >> > Subject: Sample_file for BSS issue in SREF verification
> > >> > Owner: Nobody
> > >> > Requestors: Binbin.Zhou at noaa.gov
> > >> > Status: new
> > >> > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > >
> > >> >
> > >> >
> > >> > Met_help, John,
> > >> >
> > >> > I have created a complete testing case for looking at
> > >> > the BSS issue as we discussed in METPLUS meeting today.
> > >> > The sample files directory is on Mars:
> > >> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > >> > The running script is run.sh
> > >> >
> > >> > Thanks!
> > >> >
> > >> > Binbin
> > >> >
> > >> > --
> > >> >
> > >> > Binbin Zhou
> > >> >
> > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> >
> > >> > 5830 University Research Ct.
> > >> >
> > >> > College Park, MD 20740
> > >> >
> > >> > Binbin.Zhou at noaa.gov
> > >> >
> > >> > 301-683-3683
> > >> >
> > >> >
> > >>
> > >>
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Tatiana Burek
Time: Tue Jun 30 08:03:53 2020
Binbin,
PSTD_BSS_SMPL is a statistic that only can be calculated using
aggregation logic.
Please, select 'Aggregation statistics' -> Probability methods output
(PCT) and rerun your plot.
Tatiana
On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
> John,
>
> Sure, a testing XML file is attached
>
> Binbin
>
> On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
> <met_help at ucar.edu>
> wrote:
>
> > Binbin,
> >
> > Can you please send along the XML file for your attempt to plot
> > BSS_SMPL
> > with METviewer? I'll reassign this ticket to Tatiana so she can
> > investigate
> > what's going on there.
> >
> > Thanks,
> > John
> >
> > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
> > >
> > > John,
> > >
> > > I can see BSS_SMPL data in stat tiles, but why I still can not
> > > generate the BSS_SMPL plot
> > > from MetViewer?
> > >
> > > Binbin
> > >
> > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA Affiliate <
> > > binbin.zhou at noaa.gov> wrote:
> > >
> > > > John,
> > > >
> > > > OK, I see the BSS_SMPLY data
> > > >
> > > > Thanks!
> > > >
> > > > Binbin
> > > >
> > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Binbin,
> > > >>
> > > >> Thanks for sending this example. I took a look in
> > > >> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss and
> > > >> see that
> > > you
> > > >> run.sh is...
> > > >> (1) Running ensemble_stat to generate some probability
> > > >> forecasts.
> > > >> (2) Running grid_stat to verify those probability forecasts.
> > > >>
> > > >> And your question is why is BSS = NA in the output of Grid-
Stat?
> > > >>
> > > >> I'm looking in:
> > > >>
> > > >>
> > >
> >
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> > > >>
> > > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at least
for
> > > >> the
> > > first 4
> > > >> lines. BSS is computed relative to a climatology forecast
> > > >> whereas
> > > BSS_SMPL
> > > >> is computed relative to a sample climatology. The issue is
> > > >> pretty
> > clear
> > > >> when looking at your Grid-Stat config file... you haven't
> > > >> defined a
> > > climo
> > > >> mean and standard deviation in
> > > >> verf_g2g_grid_stat_sref_config_ndas.
> > > Those
> > > >> sections are empty. Please try specifying the climo mean and
> > > >> standard
> > > >> deviation... and then the BSS column should be populated.
> > > >>
> > > >> Hope that does the trick.
> > > >>
> > > >> Thanks,
> > > >> John
> > > >>
> > > >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via RT
<
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
> > > >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> > > >> > Queue: met_help
> > > >> > Subject: Sample_file for BSS issue in SREF
verification
> > > >> > Owner: Nobody
> > > >> > Requestors: Binbin.Zhou at noaa.gov
> > > >> > Status: new
> > > >> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > > >
> > > >> >
> > > >> >
> > > >> > Met_help, John,
> > > >> >
> > > >> > I have created a complete testing case for looking at
> > > >> > the BSS issue as we discussed in METPLUS meeting today.
> > > >> > The sample files directory is on Mars:
> > > >> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > > >> > The running script is run.sh
> > > >> >
> > > >> > Thanks!
> > > >> >
> > > >> > Binbin
> > > >> >
> > > >> > --
> > > >> >
> > > >> > Binbin Zhou
> > > >> >
> > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >> >
> > > >> > 5830 University Research Ct.
> > > >> >
> > > >> > College Park, MD 20740
> > > >> >
> > > >> > Binbin.Zhou at noaa.gov
> > > >> >
> > > >> > 301-683-3683
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > >
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
> > >
> >
> >
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Tue Jun 30 08:51:43 2020
Tatiana,
I rerun the same XML file with PCT line type. The plot can be
generated,
but the values seem no correct. All BSS values should be lower than
1.0.
BSS can be negative, but can not be over 1.0
I double checked my stat files, and can confirm that all BSS_SMPL
values in PSTD are less than 1.0. Can you double check them in the
database table?
Thanks!
Binbin
On Tue, Jun 30, 2020 at 10:04 AM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> Binbin,
> PSTD_BSS_SMPL is a statistic that only can be calculated using
aggregation
> logic.
> Please, select 'Aggregation statistics' -> Probability methods
output
> (PCT) and rerun your plot.
>
> Tatiana
>
> On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
> > John,
> >
> > Sure, a testing XML file is attached
> >
> > Binbin
> >
> > On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Binbin,
> > >
> > > Can you please send along the XML file for your attempt to plot
> > > BSS_SMPL
> > > with METviewer? I'll reassign this ticket to Tatiana so she can
> > > investigate
> > > what's going on there.
> > >
> > > Thanks,
> > > John
> > >
> > > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
>
> > > >
> > > > John,
> > > >
> > > > I can see BSS_SMPL data in stat tiles, but why I still can
not
> > > > generate the BSS_SMPL plot
> > > > from MetViewer?
> > > >
> > > > Binbin
> > > >
> > > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA Affiliate <
> > > > binbin.zhou at noaa.gov> wrote:
> > > >
> > > > > John,
> > > > >
> > > > > OK, I see the BSS_SMPLY data
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Binbin
> > > > >
> > > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Binbin,
> > > > >>
> > > > >> Thanks for sending this example. I took a look in
> > > > >> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
and
> > > > >> see that
> > > > you
> > > > >> run.sh is...
> > > > >> (1) Running ensemble_stat to generate some probability
> > > > >> forecasts.
> > > > >> (2) Running grid_stat to verify those probability
forecasts.
> > > > >>
> > > > >> And your question is why is BSS = NA in the output of Grid-
Stat?
> > > > >>
> > > > >> I'm looking in:
> > > > >>
> > > > >>
> > > >
> > >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> > > > >>
> > > > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at least
for
> > > > >> the
> > > > first 4
> > > > >> lines. BSS is computed relative to a climatology forecast
> > > > >> whereas
> > > > BSS_SMPL
> > > > >> is computed relative to a sample climatology. The issue is
> > > > >> pretty
> > > clear
> > > > >> when looking at your Grid-Stat config file... you haven't
> > > > >> defined a
> > > > climo
> > > > >> mean and standard deviation in
> > > > >> verf_g2g_grid_stat_sref_config_ndas.
> > > > Those
> > > > >> sections are empty. Please try specifying the climo mean
and
> > > > >> standard
> > > > >> deviation... and then the BSS column should be populated.
> > > > >>
> > > > >> Hope that does the trick.
> > > > >>
> > > > >> Thanks,
> > > > >> John
> > > > >>
> > > > >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via
RT <
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> >
> > > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
> > > > >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> > > > >> > Queue: met_help
> > > > >> > Subject: Sample_file for BSS issue in SREF
verification
> > > > >> > Owner: Nobody
> > > > >> > Requestors: Binbin.Zhou at noaa.gov
> > > > >> > Status: new
> > > > >> > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > > > >
> > > > >> >
> > > > >> >
> > > > >> > Met_help, John,
> > > > >> >
> > > > >> > I have created a complete testing case for looking at
> > > > >> > the BSS issue as we discussed in METPLUS meeting today.
> > > > >> > The sample files directory is on Mars:
> > > > >> > /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > > > >> > The running script is run.sh
> > > > >> >
> > > > >> > Thanks!
> > > > >> >
> > > > >> > Binbin
> > > > >> >
> > > > >> > --
> > > > >> >
> > > > >> > Binbin Zhou
> > > > >> >
> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> >
> > > > >> > 5830 University Research Ct.
> > > > >> >
> > > > >> > College Park, MD 20740
> > > > >> >
> > > > >> > Binbin.Zhou at noaa.gov
> > > > >> >
> > > > >> > 301-683-3683
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > >
> > > > > Binbin Zhou
> > > > >
> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >
> > > > > 5830 University Research Ct.
> > > > >
> > > > > College Park, MD 20740
> > > > >
> > > > > Binbin.Zhou at noaa.gov
> > > > >
> > > > > 301-683-3683
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> > > >
> > >
> > >
>
>
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Tue Jun 30 09:52:54 2020
Tatiana,
Please see case 20200630_144708 that shows BSS_SMPL
plot. All values are above 1.0. But that is not a correct BSS curve
Binbin
On Tue, Jun 30, 2020 at 10:51 AM Binbin Zhou - NOAA Affiliate <
binbin.zhou at noaa.gov> wrote:
> Tatiana,
>
> I rerun the same XML file with PCT line type. The plot can be
> generated,
> but the values seem no correct. All BSS values should be lower than
1.0.
> BSS can be negative, but can not be over 1.0
> I double checked my stat files, and can confirm that all
BSS_SMPL
> values in PSTD are less than 1.0. Can you double check them in the
> database table?
> Thanks!
>
> Binbin
>
> On Tue, Jun 30, 2020 at 10:04 AM Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
>> Binbin,
>> PSTD_BSS_SMPL is a statistic that only can be calculated using
>> aggregation logic.
>> Please, select 'Aggregation statistics' -> Probability methods
output
>> (PCT) and rerun your plot.
>>
>> Tatiana
>>
>> On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
>> > John,
>> >
>> > Sure, a testing XML file is attached
>> >
>> > Binbin
>> >
>> > On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
>> > <met_help at ucar.edu>
>> > wrote:
>> >
>> > > Binbin,
>> > >
>> > > Can you please send along the XML file for your attempt to plot
>> > > BSS_SMPL
>> > > with METviewer? I'll reassign this ticket to Tatiana so she can
>> > > investigate
>> > > what's going on there.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
>
>> > > >
>> > > > John,
>> > > >
>> > > > I can see BSS_SMPL data in stat tiles, but why I still can
not
>> > > > generate the BSS_SMPL plot
>> > > > from MetViewer?
>> > > >
>> > > > Binbin
>> > > >
>> > > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA Affiliate
<
>> > > > binbin.zhou at noaa.gov> wrote:
>> > > >
>> > > > > John,
>> > > > >
>> > > > > OK, I see the BSS_SMPLY data
>> > > > >
>> > > > > Thanks!
>> > > > >
>> > > > > Binbin
>> > > > >
>> > > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > >> Binbin,
>> > > > >>
>> > > > >> Thanks for sending this example. I took a look in
>> > > > >> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
and
>> > > > >> see that
>> > > > you
>> > > > >> run.sh is...
>> > > > >> (1) Running ensemble_stat to generate some probability
>> > > > >> forecasts.
>> > > > >> (2) Running grid_stat to verify those probability
forecasts.
>> > > > >>
>> > > > >> And your question is why is BSS = NA in the output of
Grid-Stat?
>> > > > >>
>> > > > >> I'm looking in:
>> > > > >>
>> > > > >>
>> > > >
>> > >
>>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
>> > > > >>
>> > > > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at least
for
>> > > > >> the
>> > > > first 4
>> > > > >> lines. BSS is computed relative to a climatology forecast
>> > > > >> whereas
>> > > > BSS_SMPL
>> > > > >> is computed relative to a sample climatology. The issue is
>> > > > >> pretty
>> > > clear
>> > > > >> when looking at your Grid-Stat config file... you haven't
>> > > > >> defined a
>> > > > climo
>> > > > >> mean and standard deviation in
>> > > > >> verf_g2g_grid_stat_sref_config_ndas.
>> > > > Those
>> > > > >> sections are empty. Please try specifying the climo mean
and
>> > > > >> standard
>> > > > >> deviation... and then the BSS column should be populated.
>> > > > >>
>> > > > >> Hope that does the trick.
>> > > > >>
>> > > > >> Thanks,
>> > > > >> John
>> > > > >>
>> > > > >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov via
RT <
>> > > > >> met_help at ucar.edu> wrote:
>> > > > >>
>> > > > >> >
>> > > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted upon.
>> > > > >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
>> > > > >> > Queue: met_help
>> > > > >> > Subject: Sample_file for BSS issue in SREF
verification
>> > > > >> > Owner: Nobody
>> > > > >> > Requestors: Binbin.Zhou at noaa.gov
>> > > > >> > Status: new
>> > > > >> > Ticket <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
>> > > > >
>> > > > >> >
>> > > > >> >
>> > > > >> > Met_help, John,
>> > > > >> >
>> > > > >> > I have created a complete testing case for looking at
>> > > > >> > the BSS issue as we discussed in METPLUS meeting today.
>> > > > >> > The sample files directory is on Mars:
>> > > > >> >
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
>> > > > >> > The running script is run.sh
>> > > > >> >
>> > > > >> > Thanks!
>> > > > >> >
>> > > > >> > Binbin
>> > > > >> >
>> > > > >> > --
>> > > > >> >
>> > > > >> > Binbin Zhou
>> > > > >> >
>> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >> >
>> > > > >> > 5830 University Research Ct.
>> > > > >> >
>> > > > >> > College Park, MD 20740
>> > > > >> >
>> > > > >> > Binbin.Zhou at noaa.gov
>> > > > >> >
>> > > > >> > 301-683-3683
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > > >
>> > > > > --
>> > > > >
>> > > > > Binbin Zhou
>> > > > >
>> > > > > IMSG at NOAA/NWS/NCEP/EMC
>> > > > >
>> > > > > 5830 University Research Ct.
>> > > > >
>> > > > > College Park, MD 20740
>> > > > >
>> > > > > Binbin.Zhou at noaa.gov
>> > > > >
>> > > > > 301-683-3683
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > >
>> > > > Binbin Zhou
>> > > >
>> > > > IMSG at NOAA/NWS/NCEP/EMC
>> > > >
>> > > > 5830 University Research Ct.
>> > > >
>> > > > College Park, MD 20740
>> > > >
>> > > > Binbin.Zhou at noaa.gov
>> > > >
>> > > > 301-683-3683
>> > > >
>> > > >
>> > >
>> > >
>>
>>
>>
>>
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Tatiana Burek
Time: Tue Jun 30 10:15:04 2020
It was a bug in METviewer that prevented to plot BSS_SMPL data stored
in the database.
Please rerun your job 20200630_144708 but without Aggregation stat.
Use "Summary" -> None
You should get the plot with values less then 1 (20200630_160923)
Tatiana
On Tue Jun 30 09:52:54 2020, Binbin.Zhou at noaa.gov wrote:
> Tatiana,
>
> Please see case 20200630_144708 that shows BSS_SMPL
> plot. All values are above 1.0. But that is not a correct BSS curve
>
> Binbin
>
> On Tue, Jun 30, 2020 at 10:51 AM Binbin Zhou - NOAA Affiliate <
> binbin.zhou at noaa.gov> wrote:
>
> > Tatiana,
> >
> > I rerun the same XML file with PCT line type. The plot can be
> > generated,
> > but the values seem no correct. All BSS values should be lower
than
> > 1.0.
> > BSS can be negative, but can not be over 1.0
> > I double checked my stat files, and can confirm that all BSS_SMPL
> > values in PSTD are less than 1.0. Can you double check them in
the
> > database table?
> > Thanks!
> >
> > Binbin
> >
> > On Tue, Jun 30, 2020 at 10:04 AM Tatiana Burek via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> >> Binbin,
> >> PSTD_BSS_SMPL is a statistic that only can be calculated using
> >> aggregation logic.
> >> Please, select 'Aggregation statistics' -> Probability methods
> >> output
> >> (PCT) and rerun your plot.
> >>
> >> Tatiana
> >>
> >> On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
> >> > John,
> >> >
> >> > Sure, a testing XML file is attached
> >> >
> >> > Binbin
> >> >
> >> > On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
> >> > <met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Binbin,
> >> > >
> >> > > Can you please send along the XML file for your attempt to
plot
> >> > > BSS_SMPL
> >> > > with METviewer? I'll reassign this ticket to Tatiana so she
can
> >> > > investigate
> >> > > what's going on there.
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> >> > > > >
> >> > > >
> >> > > > John,
> >> > > >
> >> > > > I can see BSS_SMPL data in stat tiles, but why I still can
> >> > > > not
> >> > > > generate the BSS_SMPL plot
> >> > > > from MetViewer?
> >> > > >
> >> > > > Binbin
> >> > > >
> >> > > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA
Affiliate <
> >> > > > binbin.zhou at noaa.gov> wrote:
> >> > > >
> >> > > > > John,
> >> > > > >
> >> > > > > OK, I see the BSS_SMPLY data
> >> > > > >
> >> > > > > Thanks!
> >> > > > >
> >> > > > > Binbin
> >> > > > >
> >> > > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via RT
<
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > >> Binbin,
> >> > > > >>
> >> > > > >> Thanks for sending this example. I took a look in
> >> > > > >>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> >> > > > >> and
> >> > > > >> see that
> >> > > > you
> >> > > > >> run.sh is...
> >> > > > >> (1) Running ensemble_stat to generate some probability
> >> > > > >> forecasts.
> >> > > > >> (2) Running grid_stat to verify those probability
> >> > > > >> forecasts.
> >> > > > >>
> >> > > > >> And your question is why is BSS = NA in the output of
Grid-
> >> > > > >> Stat?
> >> > > > >>
> >> > > > >> I'm looking in:
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > >
> >>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> >> > > > >>
> >> > > > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at
least
> >> > > > >> for
> >> > > > >> the
> >> > > > first 4
> >> > > > >> lines. BSS is computed relative to a climatology
forecast
> >> > > > >> whereas
> >> > > > BSS_SMPL
> >> > > > >> is computed relative to a sample climatology. The issue
is
> >> > > > >> pretty
> >> > > clear
> >> > > > >> when looking at your Grid-Stat config file... you
haven't
> >> > > > >> defined a
> >> > > > climo
> >> > > > >> mean and standard deviation in
> >> > > > >> verf_g2g_grid_stat_sref_config_ndas.
> >> > > > Those
> >> > > > >> sections are empty. Please try specifying the climo mean
> >> > > > >> and
> >> > > > >> standard
> >> > > > >> deviation... and then the BSS column should be
populated.
> >> > > > >>
> >> > > > >> Hope that does the trick.
> >> > > > >>
> >> > > > >> Thanks,
> >> > > > >> John
> >> > > > >>
> >> > > > >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov
via
> >> > > > >> RT <
> >> > > > >> met_help at ucar.edu> wrote:
> >> > > > >>
> >> > > > >> >
> >> > > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted
upon.
> >> > > > >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> >> > > > >> > Queue: met_help
> >> > > > >> > Subject: Sample_file for BSS issue in SREF
> >> > > > >> > verification
> >> > > > >> > Owner: Nobody
> >> > > > >> > Requestors: Binbin.Zhou at noaa.gov
> >> > > > >> > Status: new
> >> > > > >> > Ticket <URL:
> >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> >> > > > >
> >> > > > >> >
> >> > > > >> >
> >> > > > >> > Met_help, John,
> >> > > > >> >
> >> > > > >> > I have created a complete testing case for looking at
> >> > > > >> > the BSS issue as we discussed in METPLUS meeting
today.
> >> > > > >> > The sample files directory is on Mars:
> >> > > > >> >
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> >> > > > >> > The running script is run.sh
> >> > > > >> >
> >> > > > >> > Thanks!
> >> > > > >> >
> >> > > > >> > Binbin
> >> > > > >> >
> >> > > > >> > --
> >> > > > >> >
> >> > > > >> > Binbin Zhou
> >> > > > >> >
> >> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >> >
> >> > > > >> > 5830 University Research Ct.
> >> > > > >> >
> >> > > > >> > College Park, MD 20740
> >> > > > >> >
> >> > > > >> > Binbin.Zhou at noaa.gov
> >> > > > >> >
> >> > > > >> > 301-683-3683
> >> > > > >> >
> >> > > > >> >
> >> > > > >>
> >> > > > >>
> >> > > > >
> >> > > > > --
> >> > > > >
> >> > > > > Binbin Zhou
> >> > > > >
> >> > > > > IMSG at NOAA/NWS/NCEP/EMC
> >> > > > >
> >> > > > > 5830 University Research Ct.
> >> > > > >
> >> > > > > College Park, MD 20740
> >> > > > >
> >> > > > > Binbin.Zhou at noaa.gov
> >> > > > >
> >> > > > > 301-683-3683
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > >
> >> > > > Binbin Zhou
> >> > > >
> >> > > > IMSG at NOAA/NWS/NCEP/EMC
> >> > > >
> >> > > > 5830 University Research Ct.
> >> > > >
> >> > > > College Park, MD 20740
> >> > > >
> >> > > > Binbin.Zhou at noaa.gov
> >> > > >
> >> > > > 301-683-3683
> >> > > >
> >> > > >
> >> > >
> >> > >
> >>
> >>
> >>
> >>
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Tue Jun 30 10:29:47 2020
Tatiana,
Yes, now the BSS_SMPL plot is correct.
Thank you very much!
Binbin
On Tue, Jun 30, 2020 at 12:15 PM Tatiana Burek via RT
<met_help at ucar.edu>
wrote:
> It was a bug in METviewer that prevented to plot BSS_SMPL data
stored in
> the database.
> Please rerun your job 20200630_144708 but without Aggregation stat.
> Use "Summary" -> None
> You should get the plot with values less then 1 (20200630_160923)
>
> Tatiana
>
> On Tue Jun 30 09:52:54 2020, Binbin.Zhou at noaa.gov wrote:
> > Tatiana,
> >
> > Please see case 20200630_144708 that shows BSS_SMPL
> > plot. All values are above 1.0. But that is not a correct BSS
curve
> >
> > Binbin
> >
> > On Tue, Jun 30, 2020 at 10:51 AM Binbin Zhou - NOAA Affiliate <
> > binbin.zhou at noaa.gov> wrote:
> >
> > > Tatiana,
> > >
> > > I rerun the same XML file with PCT line type. The plot can be
> > > generated,
> > > but the values seem no correct. All BSS values should be lower
than
> > > 1.0.
> > > BSS can be negative, but can not be over 1.0
> > > I double checked my stat files, and can confirm that all
BSS_SMPL
> > > values in PSTD are less than 1.0. Can you double check them in
the
> > > database table?
> > > Thanks!
> > >
> > > Binbin
> > >
> > > On Tue, Jun 30, 2020 at 10:04 AM Tatiana Burek via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > >> Binbin,
> > >> PSTD_BSS_SMPL is a statistic that only can be calculated using
> > >> aggregation logic.
> > >> Please, select 'Aggregation statistics' -> Probability methods
> > >> output
> > >> (PCT) and rerun your plot.
> > >>
> > >> Tatiana
> > >>
> > >> On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
> > >> > John,
> > >> >
> > >> > Sure, a testing XML file is attached
> > >> >
> > >> > Binbin
> > >> >
> > >> > On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
> > >> > <met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > Binbin,
> > >> > >
> > >> > > Can you please send along the XML file for your attempt to
plot
> > >> > > BSS_SMPL
> > >> > > with METviewer? I'll reassign this ticket to Tatiana so she
can
> > >> > > investigate
> > >> > > what's going on there.
> > >> > >
> > >> > > Thanks,
> > >> > > John
> > >> > >
> > >> > > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via RT
<
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > >> > > > >
> > >> > > >
> > >> > > > John,
> > >> > > >
> > >> > > > I can see BSS_SMPL data in stat tiles, but why I still
can
> > >> > > > not
> > >> > > > generate the BSS_SMPL plot
> > >> > > > from MetViewer?
> > >> > > >
> > >> > > > Binbin
> > >> > > >
> > >> > > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA
Affiliate <
> > >> > > > binbin.zhou at noaa.gov> wrote:
> > >> > > >
> > >> > > > > John,
> > >> > > > >
> > >> > > > > OK, I see the BSS_SMPLY data
> > >> > > > >
> > >> > > > > Thanks!
> > >> > > > >
> > >> > > > > Binbin
> > >> > > > >
> > >> > > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway via
RT <
> > >> > > > > met_help at ucar.edu> wrote:
> > >> > > > >
> > >> > > > >> Binbin,
> > >> > > > >>
> > >> > > > >> Thanks for sending this example. I took a look in
> > >> > > > >>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > >> > > > >> and
> > >> > > > >> see that
> > >> > > > you
> > >> > > > >> run.sh is...
> > >> > > > >> (1) Running ensemble_stat to generate some probability
> > >> > > > >> forecasts.
> > >> > > > >> (2) Running grid_stat to verify those probability
> > >> > > > >> forecasts.
> > >> > > > >>
> > >> > > > >> And your question is why is BSS = NA in the output of
Grid-
> > >> > > > >> Stat?
> > >> > > > >>
> > >> > > > >> I'm looking in:
> > >> > > > >>
> > >> > > > >>
> > >> > > >
> > >> > >
> > >>
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> > >> > > > >>
> > >> > > > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at
least
> > >> > > > >> for
> > >> > > > >> the
> > >> > > > first 4
> > >> > > > >> lines. BSS is computed relative to a climatology
forecast
> > >> > > > >> whereas
> > >> > > > BSS_SMPL
> > >> > > > >> is computed relative to a sample climatology. The
issue is
> > >> > > > >> pretty
> > >> > > clear
> > >> > > > >> when looking at your Grid-Stat config file... you
haven't
> > >> > > > >> defined a
> > >> > > > climo
> > >> > > > >> mean and standard deviation in
> > >> > > > >> verf_g2g_grid_stat_sref_config_ndas.
> > >> > > > Those
> > >> > > > >> sections are empty. Please try specifying the climo
mean
> > >> > > > >> and
> > >> > > > >> standard
> > >> > > > >> deviation... and then the BSS column should be
populated.
> > >> > > > >>
> > >> > > > >> Hope that does the trick.
> > >> > > > >>
> > >> > > > >> Thanks,
> > >> > > > >> John
> > >> > > > >>
> > >> > > > >> On Mon, Jun 29, 2020 at 11:26 AM Binbin.Zhou at noaa.gov
via
> > >> > > > >> RT <
> > >> > > > >> met_help at ucar.edu> wrote:
> > >> > > > >>
> > >> > > > >> >
> > >> > > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted
upon.
> > >> > > > >> > Transaction: Ticket created by Binbin.Zhou at noaa.gov
> > >> > > > >> > Queue: met_help
> > >> > > > >> > Subject: Sample_file for BSS issue in SREF
> > >> > > > >> > verification
> > >> > > > >> > Owner: Nobody
> > >> > > > >> > Requestors: Binbin.Zhou at noaa.gov
> > >> > > > >> > Status: new
> > >> > > > >> > Ticket <URL:
> > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > >> > > > >
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > Met_help, John,
> > >> > > > >> >
> > >> > > > >> > I have created a complete testing case for looking
at
> > >> > > > >> > the BSS issue as we discussed in METPLUS meeting
today.
> > >> > > > >> > The sample files directory is on Mars:
> > >> > > > >> >
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > >> > > > >> > The running script is run.sh
> > >> > > > >> >
> > >> > > > >> > Thanks!
> > >> > > > >> >
> > >> > > > >> > Binbin
> > >> > > > >> >
> > >> > > > >> > --
> > >> > > > >> >
> > >> > > > >> > Binbin Zhou
> > >> > > > >> >
> > >> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >> >
> > >> > > > >> > 5830 University Research Ct.
> > >> > > > >> >
> > >> > > > >> > College Park, MD 20740
> > >> > > > >> >
> > >> > > > >> > Binbin.Zhou at noaa.gov
> > >> > > > >> >
> > >> > > > >> > 301-683-3683
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >>
> > >> > > > >>
> > >> > > > >
> > >> > > > > --
> > >> > > > >
> > >> > > > > Binbin Zhou
> > >> > > > >
> > >> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > > >
> > >> > > > > 5830 University Research Ct.
> > >> > > > >
> > >> > > > > College Park, MD 20740
> > >> > > > >
> > >> > > > > Binbin.Zhou at noaa.gov
> > >> > > > >
> > >> > > > > 301-683-3683
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > >
> > >> > > > Binbin Zhou
> > >> > > >
> > >> > > > IMSG at NOAA/NWS/NCEP/EMC
> > >> > > >
> > >> > > > 5830 University Research Ct.
> > >> > > >
> > >> > > > College Park, MD 20740
> > >> > > >
> > >> > > > Binbin.Zhou at noaa.gov
> > >> > > >
> > >> > > > 301-683-3683
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > >
> > > Binbin Zhou
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct.
> > >
> > > College Park, MD 20740
> > >
> > > Binbin.Zhou at noaa.gov
> > >
> > > 301-683-3683
> > >
>
>
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: John Halley Gotway
Time: Tue Jun 30 12:15:46 2020
Binbin,
FYI, we are investigating this bug in the aggregation logic in
METviewer
for the PSTD statistics.
John
On Tue, Jun 30, 2020 at 10:30 AM Binbin.Zhou at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
>
> Tatiana,
>
> Yes, now the BSS_SMPL plot is correct.
> Thank you very much!
>
> Binbin
>
> On Tue, Jun 30, 2020 at 12:15 PM Tatiana Burek via RT
<met_help at ucar.edu>
> wrote:
>
> > It was a bug in METviewer that prevented to plot BSS_SMPL data
stored in
> > the database.
> > Please rerun your job 20200630_144708 but without Aggregation
stat.
> > Use "Summary" -> None
> > You should get the plot with values less then 1 (20200630_160923)
> >
> > Tatiana
> >
> > On Tue Jun 30 09:52:54 2020, Binbin.Zhou at noaa.gov wrote:
> > > Tatiana,
> > >
> > > Please see case 20200630_144708 that shows BSS_SMPL
> > > plot. All values are above 1.0. But that is not a correct BSS
curve
> > >
> > > Binbin
> > >
> > > On Tue, Jun 30, 2020 at 10:51 AM Binbin Zhou - NOAA Affiliate <
> > > binbin.zhou at noaa.gov> wrote:
> > >
> > > > Tatiana,
> > > >
> > > > I rerun the same XML file with PCT line type. The plot can be
> > > > generated,
> > > > but the values seem no correct. All BSS values should be lower
than
> > > > 1.0.
> > > > BSS can be negative, but can not be over 1.0
> > > > I double checked my stat files, and can confirm that all
BSS_SMPL
> > > > values in PSTD are less than 1.0. Can you double check them
in the
> > > > database table?
> > > > Thanks!
> > > >
> > > > Binbin
> > > >
> > > > On Tue, Jun 30, 2020 at 10:04 AM Tatiana Burek via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > >> Binbin,
> > > >> PSTD_BSS_SMPL is a statistic that only can be calculated
using
> > > >> aggregation logic.
> > > >> Please, select 'Aggregation statistics' -> Probability
methods
> > > >> output
> > > >> (PCT) and rerun your plot.
> > > >>
> > > >> Tatiana
> > > >>
> > > >> On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
> > > >> > John,
> > > >> >
> > > >> > Sure, a testing XML file is attached
> > > >> >
> > > >> > Binbin
> > > >> >
> > > >> > On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
> > > >> > <met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> > > Binbin,
> > > >> > >
> > > >> > > Can you please send along the XML file for your attempt
to plot
> > > >> > > BSS_SMPL
> > > >> > > with METviewer? I'll reassign this ticket to Tatiana so
she can
> > > >> > > investigate
> > > >> > > what's going on there.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > John
> > > >> > >
> > > >> > > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov via
RT <
> > > >> > > met_help at ucar.edu> wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > > >> > > > >
> > > >> > > >
> > > >> > > > John,
> > > >> > > >
> > > >> > > > I can see BSS_SMPL data in stat tiles, but why I still
can
> > > >> > > > not
> > > >> > > > generate the BSS_SMPL plot
> > > >> > > > from MetViewer?
> > > >> > > >
> > > >> > > > Binbin
> > > >> > > >
> > > >> > > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA
Affiliate <
> > > >> > > > binbin.zhou at noaa.gov> wrote:
> > > >> > > >
> > > >> > > > > John,
> > > >> > > > >
> > > >> > > > > OK, I see the BSS_SMPLY data
> > > >> > > > >
> > > >> > > > > Thanks!
> > > >> > > > >
> > > >> > > > > Binbin
> > > >> > > > >
> > > >> > > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway
via RT <
> > > >> > > > > met_help at ucar.edu> wrote:
> > > >> > > > >
> > > >> > > > >> Binbin,
> > > >> > > > >>
> > > >> > > > >> Thanks for sending this example. I took a look in
> > > >> > > > >>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > > >> > > > >> and
> > > >> > > > >> see that
> > > >> > > > you
> > > >> > > > >> run.sh is...
> > > >> > > > >> (1) Running ensemble_stat to generate some
probability
> > > >> > > > >> forecasts.
> > > >> > > > >> (2) Running grid_stat to verify those probability
> > > >> > > > >> forecasts.
> > > >> > > > >>
> > > >> > > > >> And your question is why is BSS = NA in the output
of Grid-
> > > >> > > > >> Stat?
> > > >> > > > >>
> > > >> > > > >> I'm looking in:
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > >
> > > >> > >
> > > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> > > >> > > > >>
> > > >> > > > >> And see that BSS = NA but BSS_SMPL does NOT = NA, at
least
> > > >> > > > >> for
> > > >> > > > >> the
> > > >> > > > first 4
> > > >> > > > >> lines. BSS is computed relative to a climatology
forecast
> > > >> > > > >> whereas
> > > >> > > > BSS_SMPL
> > > >> > > > >> is computed relative to a sample climatology. The
issue is
> > > >> > > > >> pretty
> > > >> > > clear
> > > >> > > > >> when looking at your Grid-Stat config file... you
haven't
> > > >> > > > >> defined a
> > > >> > > > climo
> > > >> > > > >> mean and standard deviation in
> > > >> > > > >> verf_g2g_grid_stat_sref_config_ndas.
> > > >> > > > Those
> > > >> > > > >> sections are empty. Please try specifying the climo
mean
> > > >> > > > >> and
> > > >> > > > >> standard
> > > >> > > > >> deviation... and then the BSS column should be
populated.
> > > >> > > > >>
> > > >> > > > >> Hope that does the trick.
> > > >> > > > >>
> > > >> > > > >> Thanks,
> > > >> > > > >> John
> > > >> > > > >>
> > > >> > > > >> On Mon, Jun 29, 2020 at 11:26 AM
Binbin.Zhou at noaa.gov via
> > > >> > > > >> RT <
> > > >> > > > >> met_help at ucar.edu> wrote:
> > > >> > > > >>
> > > >> > > > >> >
> > > >> > > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was acted
upon.
> > > >> > > > >> > Transaction: Ticket created by
Binbin.Zhou at noaa.gov
> > > >> > > > >> > Queue: met_help
> > > >> > > > >> > Subject: Sample_file for BSS issue in SREF
> > > >> > > > >> > verification
> > > >> > > > >> > Owner: Nobody
> > > >> > > > >> > Requestors: Binbin.Zhou at noaa.gov
> > > >> > > > >> > Status: new
> > > >> > > > >> > Ticket <URL:
> > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > > >> > > > >
> > > >> > > > >> >
> > > >> > > > >> >
> > > >> > > > >> > Met_help, John,
> > > >> > > > >> >
> > > >> > > > >> > I have created a complete testing case for looking
at
> > > >> > > > >> > the BSS issue as we discussed in METPLUS meeting
today.
> > > >> > > > >> > The sample files directory is on Mars:
> > > >> > > > >> >
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > > >> > > > >> > The running script is run.sh
> > > >> > > > >> >
> > > >> > > > >> > Thanks!
> > > >> > > > >> >
> > > >> > > > >> > Binbin
> > > >> > > > >> >
> > > >> > > > >> > --
> > > >> > > > >> >
> > > >> > > > >> > Binbin Zhou
> > > >> > > > >> >
> > > >> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >> >
> > > >> > > > >> > 5830 University Research Ct.
> > > >> > > > >> >
> > > >> > > > >> > College Park, MD 20740
> > > >> > > > >> >
> > > >> > > > >> > Binbin.Zhou at noaa.gov
> > > >> > > > >> >
> > > >> > > > >> > 301-683-3683
> > > >> > > > >> >
> > > >> > > > >> >
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > >
> > > >> > > > > Binbin Zhou
> > > >> > > > >
> > > >> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > > >
> > > >> > > > > 5830 University Research Ct.
> > > >> > > > >
> > > >> > > > > College Park, MD 20740
> > > >> > > > >
> > > >> > > > > Binbin.Zhou at noaa.gov
> > > >> > > > >
> > > >> > > > > 301-683-3683
> > > >> > > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > --
> > > >> > > >
> > > >> > > > Binbin Zhou
> > > >> > > >
> > > >> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >> > > >
> > > >> > > > 5830 University Research Ct.
> > > >> > > >
> > > >> > > > College Park, MD 20740
> > > >> > > >
> > > >> > > > Binbin.Zhou at noaa.gov
> > > >> > > >
> > > >> > > > 301-683-3683
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> > > > --
> > > >
> > > > Binbin Zhou
> > > >
> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > >
> > > > 5830 University Research Ct.
> > > >
> > > > College Park, MD 20740
> > > >
> > > > Binbin.Zhou at noaa.gov
> > > >
> > > > 301-683-3683
> > > >
> >
> >
> >
> >
>
> --
>
> Binbin Zhou
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct.
>
> College Park, MD 20740
>
> Binbin.Zhou at noaa.gov
>
> 301-683-3683
>
>
------------------------------------------------
Subject: Sample_file for BSS issue in SREF verification
From: Binbin.Zhou at noaa.gov
Time: Tue Jun 30 17:18:10 2020
Great, thanks for the update.
Binbin
On Tue, Jun 30, 2020 at 2:21 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binbin,
>
> FYI, we are investigating this bug in the aggregation logic in
METviewer
> for the PSTD statistics.
>
> John
>
> On Tue, Jun 30, 2020 at 10:30 AM Binbin.Zhou at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756 >
> >
> > Tatiana,
> >
> > Yes, now the BSS_SMPL plot is correct.
> > Thank you very much!
> >
> > Binbin
> >
> > On Tue, Jun 30, 2020 at 12:15 PM Tatiana Burek via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > It was a bug in METviewer that prevented to plot BSS_SMPL data
stored
> in
> > > the database.
> > > Please rerun your job 20200630_144708 but without Aggregation
stat.
> > > Use "Summary" -> None
> > > You should get the plot with values less then 1
(20200630_160923)
> > >
> > > Tatiana
> > >
> > > On Tue Jun 30 09:52:54 2020, Binbin.Zhou at noaa.gov wrote:
> > > > Tatiana,
> > > >
> > > > Please see case 20200630_144708 that shows BSS_SMPL
> > > > plot. All values are above 1.0. But that is not a correct BSS
curve
> > > >
> > > > Binbin
> > > >
> > > > On Tue, Jun 30, 2020 at 10:51 AM Binbin Zhou - NOAA Affiliate
<
> > > > binbin.zhou at noaa.gov> wrote:
> > > >
> > > > > Tatiana,
> > > > >
> > > > > I rerun the same XML file with PCT line type. The plot can
be
> > > > > generated,
> > > > > but the values seem no correct. All BSS values should be
lower than
> > > > > 1.0.
> > > > > BSS can be negative, but can not be over 1.0
> > > > > I double checked my stat files, and can confirm that all
BSS_SMPL
> > > > > values in PSTD are less than 1.0. Can you double check them
in the
> > > > > database table?
> > > > > Thanks!
> > > > >
> > > > > Binbin
> > > > >
> > > > > On Tue, Jun 30, 2020 at 10:04 AM Tatiana Burek via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > >> Binbin,
> > > > >> PSTD_BSS_SMPL is a statistic that only can be calculated
using
> > > > >> aggregation logic.
> > > > >> Please, select 'Aggregation statistics' -> Probability
methods
> > > > >> output
> > > > >> (PCT) and rerun your plot.
> > > > >>
> > > > >> Tatiana
> > > > >>
> > > > >> On Mon Jun 29 16:27:42 2020, Binbin.Zhou at noaa.gov wrote:
> > > > >> > John,
> > > > >> >
> > > > >> > Sure, a testing XML file is attached
> > > > >> >
> > > > >> > Binbin
> > > > >> >
> > > > >> > On Mon, Jun 29, 2020 at 4:42 PM John Halley Gotway via RT
> > > > >> > <met_help at ucar.edu>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Binbin,
> > > > >> > >
> > > > >> > > Can you please send along the XML file for your attempt
to
> plot
> > > > >> > > BSS_SMPL
> > > > >> > > with METviewer? I'll reassign this ticket to Tatiana so
she
> can
> > > > >> > > investigate
> > > > >> > > what's going on there.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > John
> > > > >> > >
> > > > >> > > On Mon, Jun 29, 2020 at 2:36 PM Binbin.Zhou at noaa.gov
via RT <
> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > > > >> > > > >
> > > > >> > > >
> > > > >> > > > John,
> > > > >> > > >
> > > > >> > > > I can see BSS_SMPL data in stat tiles, but why I
still can
> > > > >> > > > not
> > > > >> > > > generate the BSS_SMPL plot
> > > > >> > > > from MetViewer?
> > > > >> > > >
> > > > >> > > > Binbin
> > > > >> > > >
> > > > >> > > > On Mon, Jun 29, 2020 at 4:27 PM Binbin Zhou - NOAA
> Affiliate <
> > > > >> > > > binbin.zhou at noaa.gov> wrote:
> > > > >> > > >
> > > > >> > > > > John,
> > > > >> > > > >
> > > > >> > > > > OK, I see the BSS_SMPLY data
> > > > >> > > > >
> > > > >> > > > > Thanks!
> > > > >> > > > >
> > > > >> > > > > Binbin
> > > > >> > > > >
> > > > >> > > > > On Mon, Jun 29, 2020 at 3:46 PM John Halley Gotway
via RT
> <
> > > > >> > > > > met_help at ucar.edu> wrote:
> > > > >> > > > >
> > > > >> > > > >> Binbin,
> > > > >> > > > >>
> > > > >> > > > >> Thanks for sending this example. I took a look in
> > > > >> > > > >>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > > > >> > > > >> and
> > > > >> > > > >> see that
> > > > >> > > > you
> > > > >> > > > >> run.sh is...
> > > > >> > > > >> (1) Running ensemble_stat to generate some
probability
> > > > >> > > > >> forecasts.
> > > > >> > > > >> (2) Running grid_stat to verify those probability
> > > > >> > > > >> forecasts.
> > > > >> > > > >>
> > > > >> > > > >> And your question is why is BSS = NA in the output
of
> Grid-
> > > > >> > > > >> Stat?
> > > > >> > > > >>
> > > > >> > > > >> I'm looking in:
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > >
> > > > >> > >
> > > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss/grid_stat_SREF_F24_NDAS_240000L_20200618_030000V_pstd.txt
> > > > >> > > > >>
> > > > >> > > > >> And see that BSS = NA but BSS_SMPL does NOT = NA,
at
> least
> > > > >> > > > >> for
> > > > >> > > > >> the
> > > > >> > > > first 4
> > > > >> > > > >> lines. BSS is computed relative to a climatology
forecast
> > > > >> > > > >> whereas
> > > > >> > > > BSS_SMPL
> > > > >> > > > >> is computed relative to a sample climatology. The
issue
> is
> > > > >> > > > >> pretty
> > > > >> > > clear
> > > > >> > > > >> when looking at your Grid-Stat config file... you
haven't
> > > > >> > > > >> defined a
> > > > >> > > > climo
> > > > >> > > > >> mean and standard deviation in
> > > > >> > > > >> verf_g2g_grid_stat_sref_config_ndas.
> > > > >> > > > Those
> > > > >> > > > >> sections are empty. Please try specifying the
climo mean
> > > > >> > > > >> and
> > > > >> > > > >> standard
> > > > >> > > > >> deviation... and then the BSS column should be
populated.
> > > > >> > > > >>
> > > > >> > > > >> Hope that does the trick.
> > > > >> > > > >>
> > > > >> > > > >> Thanks,
> > > > >> > > > >> John
> > > > >> > > > >>
> > > > >> > > > >> On Mon, Jun 29, 2020 at 11:26 AM
Binbin.Zhou at noaa.gov
> via
> > > > >> > > > >> RT <
> > > > >> > > > >> met_help at ucar.edu> wrote:
> > > > >> > > > >>
> > > > >> > > > >> >
> > > > >> > > > >> > Mon Jun 29 11:25:12 2020: Request 95756 was
acted upon.
> > > > >> > > > >> > Transaction: Ticket created by
Binbin.Zhou at noaa.gov
> > > > >> > > > >> > Queue: met_help
> > > > >> > > > >> > Subject: Sample_file for BSS issue in SREF
> > > > >> > > > >> > verification
> > > > >> > > > >> > Owner: Nobody
> > > > >> > > > >> > Requestors: Binbin.Zhou at noaa.gov
> > > > >> > > > >> > Status: new
> > > > >> > > > >> > Ticket <URL:
> > > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95756
> > > > >> > > > >
> > > > >> > > > >> >
> > > > >> > > > >> >
> > > > >> > > > >> > Met_help, John,
> > > > >> > > > >> >
> > > > >> > > > >> > I have created a complete testing case for
looking at
> > > > >> > > > >> > the BSS issue as we discussed in METPLUS meeting
today.
> > > > >> > > > >> > The sample files directory is on Mars:
> > > > >> > > > >> >
> /gpfs/dell2/emc/modeling/noscrub/Binbin.Zhou/met/sref_bss
> > > > >> > > > >> > The running script is run.sh
> > > > >> > > > >> >
> > > > >> > > > >> > Thanks!
> > > > >> > > > >> >
> > > > >> > > > >> > Binbin
> > > > >> > > > >> >
> > > > >> > > > >> > --
> > > > >> > > > >> >
> > > > >> > > > >> > Binbin Zhou
> > > > >> > > > >> >
> > > > >> > > > >> > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >> >
> > > > >> > > > >> > 5830 University Research Ct.
> > > > >> > > > >> >
> > > > >> > > > >> > College Park, MD 20740
> > > > >> > > > >> >
> > > > >> > > > >> > Binbin.Zhou at noaa.gov
> > > > >> > > > >> >
> > > > >> > > > >> > 301-683-3683
> > > > >> > > > >> >
> > > > >> > > > >> >
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > >
> > > > >> > > > > Binbin Zhou
> > > > >> > > > >
> > > > >> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > > >
> > > > >> > > > > 5830 University Research Ct.
> > > > >> > > > >
> > > > >> > > > > College Park, MD 20740
> > > > >> > > > >
> > > > >> > > > > Binbin.Zhou at noaa.gov
> > > > >> > > > >
> > > > >> > > > > 301-683-3683
> > > > >> > > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > >
> > > > >> > > > Binbin Zhou
> > > > >> > > >
> > > > >> > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >> > > >
> > > > >> > > > 5830 University Research Ct.
> > > > >> > > >
> > > > >> > > > College Park, MD 20740
> > > > >> > > >
> > > > >> > > > Binbin.Zhou at noaa.gov
> > > > >> > > >
> > > > >> > > > 301-683-3683
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > >
> > > > > Binbin Zhou
> > > > >
> > > > > IMSG at NOAA/NWS/NCEP/EMC
> > > > >
> > > > > 5830 University Research Ct.
> > > > >
> > > > > College Park, MD 20740
> > > > >
> > > > > Binbin.Zhou at noaa.gov
> > > > >
> > > > > 301-683-3683
> > > > >
> > >
> > >
> > >
> > >
> >
> > --
> >
> > Binbin Zhou
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct.
> >
> > College Park, MD 20740
> >
> > Binbin.Zhou at noaa.gov
> >
> > 301-683-3683
> >
> >
>
>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
Binbin.Zhou at noaa.gov
301-683-3683
------------------------------------------------
More information about the Met_help
mailing list