[Met_help] [rt.rap.ucar.edu #86836] History for Very slow grid_stat neighborhood (FSS) calculation?
John Halley Gotway via RT
met_help at ucar.edu
Fri Sep 7 16:37:11 MDT 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello. I've found the grid_stat calculation for neighborhood
verification (FSS) to be very slow. For example, a single grid_stat run
to verify a forecast against an analysis (ConUS, 4.763km, verified on
the analysis grid, on 6 thresholds,16 neighborhood widths) running on
Tide takes ~6900 seconds. My own FSS computation on Tide takes about
240 seconds (same grid, same number of thresholds, double the of number
of neighborhood sizes). Am I doing something really wrong? Attached
are the config file and the run script for one really simple test.
Also, I'm doing the FSS computation for the whole ConUS domain (no
sub-domains). I see that *.stat output generally have 'FULL' plus the
domains selected in 'poly' (in this case, my *.stat has 'FULL' and
'CONUS', and *.stat and METviewer menu shows stats for both FULL and
CONUS (the scores appear to be the identical when plotted). Can I turn
off computation for 'FULL' (delete 'grid = ["FULL" ] from the config??)
and just have computation for 'CONUS', in order to save time?
Thank you -
Ying
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Very slow grid_stat neighborhood (FSS) calculation?
From: John Halley Gotway
Time: Tue Sep 04 13:23:06 2018
Hi Ying,
Ben Blake wrote about this issue last week. The met-8.0_beta7 version
is
very slow when compared against met-6.0.
For comparison, his test case using met-6.0 took 1:30. With met-7.0
(and
met-8.0_beta6) it took over 10 minutes. Last week I found the issue
and
addressed it, bringing it back down to about 1:50. Still slower than
met-6.0, but not nearly as dramatically so.
We'll plan to do another beta release at the end of this week and ask
you
guys to confirm that the issue is resolved. If you'd like more detail
on
this, I can explain the change that caused the slowdown.
Thanks,
John
On Tue, Sep 4, 2018 at 11:49 AM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> Tue Sep 04 11:48:48 2018: Request 86836 was acted upon.
> Transaction: Ticket created by ying.lin at noaa.gov
> Queue: met_help
> Subject: Very slow grid_stat neighborhood (FSS) calculation?
> Owner: Nobody
> Requestors: ying.lin at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86836 >
>
>
> Hello. I've found the grid_stat calculation for neighborhood
> verification (FSS) to be very slow. For example, a single grid_stat
run
> to verify a forecast against an analysis (ConUS, 4.763km, verified
on
> the analysis grid, on 6 thresholds,16 neighborhood widths) running
on
> Tide takes ~6900 seconds. My own FSS computation on Tide takes
about
> 240 seconds (same grid, same number of thresholds, double the of
number
> of neighborhood sizes). Am I doing something really wrong?
Attached
> are the config file and the run script for one really simple test.
>
> Also, I'm doing the FSS computation for the whole ConUS domain (no
> sub-domains). I see that *.stat output generally have 'FULL' plus
the
> domains selected in 'poly' (in this case, my *.stat has 'FULL' and
> 'CONUS', and *.stat and METviewer menu shows stats for both FULL and
> CONUS (the scores appear to be the identical when plotted). Can I
turn
> off computation for 'FULL' (delete 'grid = ["FULL" ] from the
config??)
> and just have computation for 'CONUS', in order to save time?
>
> Thank you -
>
> Ying
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #86836] Very slow grid_stat neighborhood (FSS) calculation?
From: Ying Lin
Time: Wed Sep 05 10:26:46 2018
Thanks John. I tested the same set up in met 6.0 - the timing went
down to 900 seconds - similar to Ben's ratio you described. The stats
appear to be identical when plotted in METviewer.
Ying
On 09/04/2018 03:23 PM, John Halley Gotway via RT wrote:
> Hi Ying,
>
> Ben Blake wrote about this issue last week. The met-8.0_beta7
version is
> very slow when compared against met-6.0.
>
> For comparison, his test case using met-6.0 took 1:30. With met-7.0
(and
> met-8.0_beta6) it took over 10 minutes. Last week I found the issue
and
> addressed it, bringing it back down to about 1:50. Still slower
than
> met-6.0, but not nearly as dramatically so.
>
> We'll plan to do another beta release at the end of this week and
ask you
> guys to confirm that the issue is resolved. If you'd like more
detail on
> this, I can explain the change that caused the slowdown.
>
> Thanks,
> John
>
> On Tue, Sep 4, 2018 at 11:49 AM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>> Tue Sep 04 11:48:48 2018: Request 86836 was acted upon.
>> Transaction: Ticket created by ying.lin at noaa.gov
>> Queue: met_help
>> Subject: Very slow grid_stat neighborhood (FSS) calculation?
>> Owner: Nobody
>> Requestors: ying.lin at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86836 >
>>
>>
>> Hello. I've found the grid_stat calculation for neighborhood
>> verification (FSS) to be very slow. For example, a single
grid_stat run
>> to verify a forecast against an analysis (ConUS, 4.763km, verified
on
>> the analysis grid, on 6 thresholds,16 neighborhood widths) running
on
>> Tide takes ~6900 seconds. My own FSS computation on Tide takes
about
>> 240 seconds (same grid, same number of thresholds, double the of
number
>> of neighborhood sizes). Am I doing something really wrong?
Attached
>> are the config file and the run script for one really simple test.
>>
>> Also, I'm doing the FSS computation for the whole ConUS domain (no
>> sub-domains). I see that *.stat output generally have 'FULL' plus
the
>> domains selected in 'poly' (in this case, my *.stat has 'FULL' and
>> 'CONUS', and *.stat and METviewer menu shows stats for both FULL
and
>> CONUS (the scores appear to be the identical when plotted). Can I
turn
>> off computation for 'FULL' (delete 'grid = ["FULL" ] from the
config??)
>> and just have computation for 'CONUS', in order to save time?
>>
>> Thank you -
>>
>> Ying
>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Very slow grid_stat neighborhood (FSS) calculation?
From: John Halley Gotway
Time: Fri Sep 07 16:15:12 2018
Ying,
We've heard this feedback about met-7.0 being very slow in computing
neighborhood methods from a few different users via met-help. So we
decided to post a met-7.0 bugfix for this:
https://dtcenter.org/met/users/support/known_issues/METv7.0/index.php
When Julie Prestopnik is compiling the next met-8.0 beta releases,
I've
asked her to also recompile met-7.0 with this patch file.
Thanks,
John
On Wed, Sep 5, 2018 at 1:52 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86836 >
>
> Thanks John. I tested the same set up in met 6.0 - the timing went
> down to 900 seconds - similar to Ben's ratio you described. The
stats
> appear to be identical when plotted in METviewer.
>
> Ying
>
> On 09/04/2018 03:23 PM, John Halley Gotway via RT wrote:
> > Hi Ying,
> >
> > Ben Blake wrote about this issue last week. The met-8.0_beta7
version is
> > very slow when compared against met-6.0.
> >
> > For comparison, his test case using met-6.0 took 1:30. With met-
7.0 (and
> > met-8.0_beta6) it took over 10 minutes. Last week I found the
issue and
> > addressed it, bringing it back down to about 1:50. Still slower
than
> > met-6.0, but not nearly as dramatically so.
> >
> > We'll plan to do another beta release at the end of this week and
ask you
> > guys to confirm that the issue is resolved. If you'd like more
detail on
> > this, I can explain the change that caused the slowdown.
> >
> > Thanks,
> > John
> >
> > On Tue, Sep 4, 2018 at 11:49 AM Ying Lin via RT
<met_help at ucar.edu>
> wrote:
> >
> >> Tue Sep 04 11:48:48 2018: Request 86836 was acted upon.
> >> Transaction: Ticket created by ying.lin at noaa.gov
> >> Queue: met_help
> >> Subject: Very slow grid_stat neighborhood (FSS)
calculation?
> >> Owner: Nobody
> >> Requestors: ying.lin at noaa.gov
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86836
> >
> >>
> >>
> >> Hello. I've found the grid_stat calculation for neighborhood
> >> verification (FSS) to be very slow. For example, a single
grid_stat run
> >> to verify a forecast against an analysis (ConUS, 4.763km,
verified on
> >> the analysis grid, on 6 thresholds,16 neighborhood widths)
running on
> >> Tide takes ~6900 seconds. My own FSS computation on Tide takes
about
> >> 240 seconds (same grid, same number of thresholds, double the of
number
> >> of neighborhood sizes). Am I doing something really wrong?
Attached
> >> are the config file and the run script for one really simple
test.
> >>
> >> Also, I'm doing the FSS computation for the whole ConUS domain
(no
> >> sub-domains). I see that *.stat output generally have 'FULL'
plus the
> >> domains selected in 'poly' (in this case, my *.stat has 'FULL'
and
> >> 'CONUS', and *.stat and METviewer menu shows stats for both FULL
and
> >> CONUS (the scores appear to be the identical when plotted). Can
I turn
> >> off computation for 'FULL' (delete 'grid = ["FULL" ] from the
config??)
> >> and just have computation for 'CONUS', in order to save time?
> >>
> >> Thank you -
> >>
> >> Ying
> >>
> >> --
> >> Ying Lin
> >> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >> NCWCP Cubicle No. 2015
> >> Ying.Lin at noaa.gov
> >>
> >>
> >>
> >>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #86836] Very slow grid_stat neighborhood (FSS) calculation?
From: Ying Lin
Time: Fri Sep 07 16:36:31 2018
Thank you very much John.
Ying
On 09/07/2018 06:15 PM, John Halley Gotway via RT wrote:
> Ying,
>
> We've heard this feedback about met-7.0 being very slow in computing
> neighborhood methods from a few different users via met-help. So we
> decided to post a met-7.0 bugfix for this:
>
https://dtcenter.org/met/users/support/known_issues/METv7.0/index.php
>
> When Julie Prestopnik is compiling the next met-8.0 beta releases,
I've
> asked her to also recompile met-7.0 with this patch file.
>
> Thanks,
> John
>
> On Wed, Sep 5, 2018 at 1:52 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86836 >
>>
>> Thanks John. I tested the same set up in met 6.0 - the timing
went
>> down to 900 seconds - similar to Ben's ratio you described. The
stats
>> appear to be identical when plotted in METviewer.
>>
>> Ying
>>
>> On 09/04/2018 03:23 PM, John Halley Gotway via RT wrote:
>>> Hi Ying,
>>>
>>> Ben Blake wrote about this issue last week. The met-8.0_beta7
version is
>>> very slow when compared against met-6.0.
>>>
>>> For comparison, his test case using met-6.0 took 1:30. With met-
7.0 (and
>>> met-8.0_beta6) it took over 10 minutes. Last week I found the
issue and
>>> addressed it, bringing it back down to about 1:50. Still slower
than
>>> met-6.0, but not nearly as dramatically so.
>>>
>>> We'll plan to do another beta release at the end of this week and
ask you
>>> guys to confirm that the issue is resolved. If you'd like more
detail on
>>> this, I can explain the change that caused the slowdown.
>>>
>>> Thanks,
>>> John
>>>
>>> On Tue, Sep 4, 2018 at 11:49 AM Ying Lin via RT
<met_help at ucar.edu>
>> wrote:
>>>> Tue Sep 04 11:48:48 2018: Request 86836 was acted upon.
>>>> Transaction: Ticket created by ying.lin at noaa.gov
>>>> Queue: met_help
>>>> Subject: Very slow grid_stat neighborhood (FSS)
calculation?
>>>> Owner: Nobody
>>>> Requestors: ying.lin at noaa.gov
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86836
>>>>
>>>> Hello. I've found the grid_stat calculation for neighborhood
>>>> verification (FSS) to be very slow. For example, a single
grid_stat run
>>>> to verify a forecast against an analysis (ConUS, 4.763km,
verified on
>>>> the analysis grid, on 6 thresholds,16 neighborhood widths)
running on
>>>> Tide takes ~6900 seconds. My own FSS computation on Tide takes
about
>>>> 240 seconds (same grid, same number of thresholds, double the of
number
>>>> of neighborhood sizes). Am I doing something really wrong?
Attached
>>>> are the config file and the run script for one really simple
test.
>>>>
>>>> Also, I'm doing the FSS computation for the whole ConUS domain
(no
>>>> sub-domains). I see that *.stat output generally have 'FULL'
plus the
>>>> domains selected in 'poly' (in this case, my *.stat has 'FULL'
and
>>>> 'CONUS', and *.stat and METviewer menu shows stats for both FULL
and
>>>> CONUS (the scores appear to be the identical when plotted). Can
I turn
>>>> off computation for 'FULL' (delete 'grid = ["FULL" ] from the
config??)
>>>> and just have computation for 'CONUS', in order to save time?
>>>>
>>>> Thank you -
>>>>
>>>> Ying
>>>>
>>>> --
>>>> Ying Lin
>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>>> NCWCP Cubicle No. 2015
>>>> Ying.Lin at noaa.gov
>>>>
>>>>
>>>>
>>>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
More information about the Met_help
mailing list