[Met_help] [rt.rap.ucar.edu #86514] History for speed up the run of grid_stat
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 9 12:07:33 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Zhang, Mandy has shared OneDrive for Business files with you. To view them, click the links below.
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/ETAdGoB0AcRDgDAka5ZhodQBlWJj74cjo7LGedfEAbalEg>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/ETAdGoB0AcRDgDAka5ZhodQBlWJj74cjo7LGedfEAbalEg>
conus_east_of_rockies.nc<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/ETAdGoB0AcRDgDAka5ZhodQBlWJj74cjo7LGedfEAbalEg>
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EXRkjbLUAt1NpHQ6NU3lc24BdvYQqY6vK8WlsDKcs4wHfA>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EXRkjbLUAt1NpHQ6NU3lc24BdvYQqY6vK8WlsDKcs4wHfA>
Grid_Stat_mb.sh<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EXRkjbLUAt1NpHQ6NU3lc24BdvYQqY6vK8WlsDKcs4wHfA>
<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EXgIoPPN3UNKsOM5Af6n97kB7h8gV-HFwM-GqqcKiP6rhw>
[https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EXgIoPPN3UNKsOM5Af6n97kB7h8gV-HFwM-GqqcKiP6rhw>
GridStatConfig_APCP.txt<https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-1_ou_edu/EXgIoPPN3UNKsOM5Af6n97kB7h8gV-HFwM-GqqcKiP6rhw>
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/Eeej2oOYyNVOi0GGkNuJ5nsBefCtRsiccs9maOUJIMKCVw>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/Eeej2oOYyNVOi0GGkNuJ5nsBefCtRsiccs9maOUJIMKCVw>
met2018051500acc01_003600.nc<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/Eeej2oOYyNVOi0GGkNuJ5nsBefCtRsiccs9maOUJIMKCVw>
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/Ef0s7iC5sAdHvx7v3i5Nxa0BG42Q-3-ueHx0jS5VInXH7Q>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/Ef0s7iC5sAdHvx7v3i5Nxa0BG42Q-3-ueHx0jS5VInXH7Q>
obs2018051500acc01_003600.nc<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/Ef0s7iC5sAdHvx7v3i5Nxa0BG42Q-3-ueHx0jS5VInXH7Q>
Hi All,
I have a large domain CONUS area with 3*3km grid space and I mask the east of rockies area, when I run grid_stat to evaluate 60 hours (files) forecasts, it will cost almost 13 hours to completing the task. And the results includes both large area (Full grids) and mask area.
Do you know how to speed up the running? observation data, forecast data ,mask data
,configure file and run script are attached. both observation and forecast data are netcdf format.
Thank you very much!
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: speed up the run of grid_stat
From: John Halley Gotway
Time: Wed Aug 08 11:51:16 2018
Mandy,
Thanks for sending your sample data files. I ran them through grid-
stat on
my machine using met-6.0, met-7.0, and the current version under
development.
met-6.0 took about 2:58 to run.
met-7.0 took about 3:21 to run.
met-dev took about 3:08 to run.
Most of the time is spent computing the neighborhood fractional skill
scores. Applying 5 thresholds * 2 neighborhood sizes to a grid
containing
1,822,629 grid points takes some time.
Reconfiguring to write only CTC, CTS, MCTC, MCTS, and NBRCNT line
types to
the STAT file and skip the writing of NetCDF output improves those
runtimes
to 1:41, 2:00, and 1:46.
The run is actually slower in met-7.0 and met-dev... and that's due to
a
change in how the neighborhood areas are defined. met-7.0 and later
supports the use of square (legacy) or circular (new) neighborhoods.
That
flexibility leads to slightly slower runtimes. The development
version is
faster than met-7.0 likely because of storing the masking regions more
efficiently.
Thanks,
John
On Tue, Aug 7, 2018 at 4:27 PM Zhang, Mandy via RT <met_help at ucar.edu>
wrote:
>
> Tue Aug 07 16:27:15 2018: Request 86514 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: speed up the run of grid_stat
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86514 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/ETAdGoB0AcRDgDAka5ZhodQBlWJj74cjo7LGedfEAbalEg
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/ETAdGoB0AcRDgDAka5ZhodQBlWJj74cjo7LGedfEAbalEg
> >
>
> conus_east_of_rockies.nc<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/ETAdGoB0AcRDgDAka5ZhodQBlWJj74cjo7LGedfEAbalEg
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EXRkjbLUAt1NpHQ6NU3lc24BdvYQqY6vK8WlsDKcs4wHfA
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EXRkjbLUAt1NpHQ6NU3lc24BdvYQqY6vK8WlsDKcs4wHfA
> >
>
> Grid_Stat_mb.sh<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EXRkjbLUAt1NpHQ6NU3lc24BdvYQqY6vK8WlsDKcs4wHfA
> >
>
> <
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EXgIoPPN3UNKsOM5Af6n97kB7h8gV-HFwM-GqqcKiP6rhw
> >
> [https://r1.res.office365.com/owa/prem/images/dc-txt_20.png]<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EXgIoPPN3UNKsOM5Af6n97kB7h8gV-HFwM-GqqcKiP6rhw
> >
>
> GridStatConfig_APCP.txt<
> https://sooners-my.sharepoint.com/:t:/g/personal/man_zhang-
1_ou_edu/EXgIoPPN3UNKsOM5Af6n97kB7h8gV-HFwM-GqqcKiP6rhw
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/Eeej2oOYyNVOi0GGkNuJ5nsBefCtRsiccs9maOUJIMKCVw
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/Eeej2oOYyNVOi0GGkNuJ5nsBefCtRsiccs9maOUJIMKCVw
> >
>
> met2018051500acc01_003600.nc<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/Eeej2oOYyNVOi0GGkNuJ5nsBefCtRsiccs9maOUJIMKCVw
> >
>
> <
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/Ef0s7iC5sAdHvx7v3i5Nxa0BG42Q-3-ueHx0jS5VInXH7Q
> >
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/Ef0s7iC5sAdHvx7v3i5Nxa0BG42Q-3-ueHx0jS5VInXH7Q
> >
>
> obs2018051500acc01_003600.nc<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/Ef0s7iC5sAdHvx7v3i5Nxa0BG42Q-3-ueHx0jS5VInXH7Q
> >
>
>
>
>
> Hi All,
>
> I have a large domain CONUS area with 3*3km grid space and I
mask the
> east of rockies area, when I run grid_stat to evaluate 60 hours
(files)
> forecasts, it will cost almost 13 hours to completing the task. And
the
> results includes both large area (Full grids) and mask area.
>
> Do you know how to speed up the running? observation data,
forecast
> data ,mask data
>
> ,configure file and run script are attached. both observation and
forecast
> data are netcdf format.
>
> Thank you very much!
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list