[Met_help] [rt.rap.ucar.edu #93335] History for Northern Hemisphere Mask

Fri Jan 17 14:29:36 MST 2020

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

I had a question, we recently upgraded to version 8.1.2 from version 5.1

When I ran grid_stat on version 5.1 I had created a Northern Hemisphere
poly, so that I could calculate 500mb Height ACC scores from 0.5 global
model output. I define my NH.poly as follows:

NH

20 -179

20 179

80 179

80 -179

20 -179

When i ran using version 5.1, it said that I used 259,920 matched pairs for
the FULL region and 85,919 for the NH region.

But when I use version 8.1.2 using the same poly definition and a config
file I changed to mimic the 5.1 version config. I get less matched pairs
for the NH region (479 matched pairs) but the same for the FULL.

 Output below:

DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
NEAREST(1), over region FULL, using 259559 matched pairs.

DEBUG 2: Computing Categorical Statistics.

DEBUG 2: Computing Continuous Statistics.

DEBUG 2: Computing Scalar Partial Sums.

DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
NEAREST(1), over region NH, using 479 matched pairs.

DEBUG 2: Computing Categorical Statistics.

DEBUG 2: Computing Continuous Statistics.

DEBUG 2: Computing Scalar Partial Sums.

DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region FULL, using
258840 matched pairs.

DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region NH, using
479 matched pairs.

DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region FULL, using
259920 matched pairs.

DEBUG 2: Computing Neighborhood Categorical Statistics.

DEBUG 2: Computing Neighborhood Continuous Statistics.

DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region NH, using 479
matched pairs.

DEBUG 2: Computing Neighborhood Categorical Statistics.

DEBUG 2: Computing Neighborhood Continuous Statistics.

Do you have any idea why this may be happening with the new version?
Obviously I would want more matched pairs. I attached the GridStat config
file I am using. Thanks!

Allan Huffman

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Northern Hemisphere Mask
From: Julie Prestopnik
Time: Wed Dec 04 12:47:42 2019

Hi Allan.

I have assigned this ticket to John Halley Gotway who should be better
able
to answer your question.  Please allow a few days for a response.

Thanks,
Julie

On Wed, Dec 4, 2019 at 12:38 PM Allan Huffman via RT
<met_help at ucar.edu>
wrote:

>
> Wed Dec 04 12:38:45 2019: Request 93335 was acted upon.
> Transaction: Ticket created by awhuffma at gmail.com
>        Queue: met_help
>      Subject: Northern Hemisphere Mask
>        Owner: Nobody
>   Requestors: awhuffma at gmail.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93335 >
>
>
> Hi,
>
> I had a question, we recently upgraded to version 8.1.2 from version
5.1
>
> When I ran grid_stat on version 5.1 I had created a Northern
Hemisphere
> poly, so that I could calculate 500mb Height ACC scores from 0.5
global
> model output. I define my NH.poly as follows:
>
> NH
>
> 20 -179
>
> 20 179
>
> 80 179
>
> 80 -179
>
> 20 -179
>
> When i ran using version 5.1, it said that I used 259,920 matched
pairs for
> the FULL region and 85,919 for the NH region.
>
>
> But when I use version 8.1.2 using the same poly definition and a
config
> file I changed to mimic the 5.1 version config. I get less matched
pairs
> for the NH region (479 matched pairs) but the same for the FULL.
>
>
>  Output below:
>
>
> DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> NEAREST(1), over region FULL, using 259559 matched pairs.
>
> DEBUG 2: Computing Categorical Statistics.
>
> DEBUG 2: Computing Continuous Statistics.
>
> DEBUG 2: Computing Scalar Partial Sums.
>
> DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> NEAREST(1), over region NH, using 479 matched pairs.
>
> DEBUG 2: Computing Categorical Statistics.
>
> DEBUG 2: Computing Continuous Statistics.
>
> DEBUG 2: Computing Scalar Partial Sums.
>
> DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region FULL,
using
> 258840 matched pairs.
>
> DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region NH,
using
> 479 matched pairs.
>
> DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region FULL,
using
> 259920 matched pairs.
>
> DEBUG 2: Computing Neighborhood Categorical Statistics.
>
> DEBUG 2: Computing Neighborhood Continuous Statistics.
>
> DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region NH,
using 479
> matched pairs.
>
> DEBUG 2: Computing Neighborhood Categorical Statistics.
>
> DEBUG 2: Computing Neighborhood Continuous Statistics.
>
>
> Do you have any idea why this may be happening with the new version?
> Obviously I would want more matched pairs. I attached the GridStat
config
> file I am using. Thanks!
>
>
> Allan Huffman
>
>

--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: Northern Hemisphere Mask
From: John Halley Gotway
Time: Wed Dec 04 13:03:03 2019

Hi Allan,

Generating a very large masking region for global dataset can be
problematic.  Deciding which grid points fall inside/outside the
masking
region can be poorly defined.  I do see that what worked well in met-
5.1 is
not working in met-8.1.2, and I don't have a good explanation for that
now.

But really, defining the masking with a polyline in Grid-Stat is
rather
slow anyway.  Each time you run Grid-Stat, it has to go through and
decide
which points fall in/outside the mask.  I'd recommend that you run the
Gen-Vx-Mask tool instead.  Then edit your Grid-Stat configuration file
by
replacing the path to your poly file with the path to the NetCDF
output of
Gen-Vx-Mask.

Generating a NH mask is very easy with Gen-Vx-Mask.  You just run:

gen_vx_mask in_file in_file NH_mask.nc -type lat -thresh gt0

Where in_file is a data file on your target verification domain.

Please let me know how it goes.

Thanks,
John

On Wed, Dec 4, 2019 at 12:48 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93335 >
>
> Hi Allan.
>
> I have assigned this ticket to John Halley Gotway who should be
better able
> to answer your question.  Please allow a few days for a response.
>
> Thanks,
> Julie
>
> On Wed, Dec 4, 2019 at 12:38 PM Allan Huffman via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Dec 04 12:38:45 2019: Request 93335 was acted upon.
> > Transaction: Ticket created by awhuffma at gmail.com
> >        Queue: met_help
> >      Subject: Northern Hemisphere Mask
> >        Owner: Nobody
> >   Requestors: awhuffma at gmail.com
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93335 >
> >
> >
> > Hi,
> >
> > I had a question, we recently upgraded to version 8.1.2 from
version 5.1
> >
> > When I ran grid_stat on version 5.1 I had created a Northern
Hemisphere
> > poly, so that I could calculate 500mb Height ACC scores from 0.5
global
> > model output. I define my NH.poly as follows:
> >
> > NH
> >
> > 20 -179
> >
> > 20 179
> >
> > 80 179
> >
> > 80 -179
> >
> > 20 -179
> >
> > When i ran using version 5.1, it said that I used 259,920 matched
pairs
> for
> > the FULL region and 85,919 for the NH region.
> >
> >
> > But when I use version 8.1.2 using the same poly definition and a
config
> > file I changed to mimic the 5.1 version config. I get less matched
pairs
> > for the NH region (479 matched pairs) but the same for the FULL.
> >
> >
> >  Output below:
> >
> >
> > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> > NEAREST(1), over region FULL, using 259559 matched pairs.
> >
> > DEBUG 2: Computing Categorical Statistics.
> >
> > DEBUG 2: Computing Continuous Statistics.
> >
> > DEBUG 2: Computing Scalar Partial Sums.
> >
> > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> > NEAREST(1), over region NH, using 479 matched pairs.
> >
> > DEBUG 2: Computing Categorical Statistics.
> >
> > DEBUG 2: Computing Continuous Statistics.
> >
> > DEBUG 2: Computing Scalar Partial Sums.
> >
> > DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region
FULL,
> using
> > 258840 matched pairs.
> >
> > DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region NH,
using
> > 479 matched pairs.
> >
> > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> > NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region
FULL, using
> > 259920 matched pairs.
> >
> > DEBUG 2: Computing Neighborhood Categorical Statistics.
> >
> > DEBUG 2: Computing Neighborhood Continuous Statistics.
> >
> > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing method
> > NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region NH,
using
> 479
> > matched pairs.
> >
> > DEBUG 2: Computing Neighborhood Categorical Statistics.
> >
> > DEBUG 2: Computing Neighborhood Continuous Statistics.
> >
> >
> > Do you have any idea why this may be happening with the new
version?
> > Obviously I would want more matched pairs. I attached the GridStat
config
> > file I am using. Thanks!
> >
> >
> > Allan Huffman
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day.  Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: Northern Hemisphere Mask
From: Allan Huffman
Time: Wed Dec 04 13:23:01 2019

Ok Thanks John, I will try that and let you know.

Regards,

Allan

On Wed, Dec 4, 2019 at 3:03 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Hi Allan,
>
> Generating a very large masking region for global dataset can be
> problematic.  Deciding which grid points fall inside/outside the
masking
> region can be poorly defined.  I do see that what worked well in
met-5.1 is
> not working in met-8.1.2, and I don't have a good explanation for
that now.
>
> But really, defining the masking with a polyline in Grid-Stat is
rather
> slow anyway.  Each time you run Grid-Stat, it has to go through and
decide
> which points fall in/outside the mask.  I'd recommend that you run
the
> Gen-Vx-Mask tool instead.  Then edit your Grid-Stat configuration
file by
> replacing the path to your poly file with the path to the NetCDF
output of
> Gen-Vx-Mask.
>
> Generating a NH mask is very easy with Gen-Vx-Mask.  You just run:
>
> gen_vx_mask in_file in_file NH_mask.nc -type lat -thresh gt0
>
> Where in_file is a data file on your target verification domain.
>
> Please let me know how it goes.
>
> Thanks,
> John
>
>
> On Wed, Dec 4, 2019 at 12:48 PM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93335 >
> >
> > Hi Allan.
> >
> > I have assigned this ticket to John Halley Gotway who should be
better
> able
> > to answer your question.  Please allow a few days for a response.
> >
> > Thanks,
> > Julie
> >
> > On Wed, Dec 4, 2019 at 12:38 PM Allan Huffman via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Wed Dec 04 12:38:45 2019: Request 93335 was acted upon.
> > > Transaction: Ticket created by awhuffma at gmail.com
> > >        Queue: met_help
> > >      Subject: Northern Hemisphere Mask
> > >        Owner: Nobody
> > >   Requestors: awhuffma at gmail.com
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93335
> >
> > >
> > >
> > > Hi,
> > >
> > > I had a question, we recently upgraded to version 8.1.2 from
version
> 5.1
> > >
> > > When I ran grid_stat on version 5.1 I had created a Northern
Hemisphere
> > > poly, so that I could calculate 500mb Height ACC scores from 0.5
global
> > > model output. I define my NH.poly as follows:
> > >
> > > NH
> > >
> > > 20 -179
> > >
> > > 20 179
> > >
> > > 80 179
> > >
> > > 80 -179
> > >
> > > 20 -179
> > >
> > > When i ran using version 5.1, it said that I used 259,920
matched pairs
> > for
> > > the FULL region and 85,919 for the NH region.
> > >
> > >
> > > But when I use version 8.1.2 using the same poly definition and
a
> config
> > > file I changed to mimic the 5.1 version config. I get less
matched
> pairs
> > > for the NH region (479 matched pairs) but the same for the FULL.
> > >
> > >
> > >  Output below:
> > >
> > >
> > > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing
method
> > > NEAREST(1), over region FULL, using 259559 matched pairs.
> > >
> > > DEBUG 2: Computing Categorical Statistics.
> > >
> > > DEBUG 2: Computing Continuous Statistics.
> > >
> > > DEBUG 2: Computing Scalar Partial Sums.
> > >
> > > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing
method
> > > NEAREST(1), over region NH, using 479 matched pairs.
> > >
> > > DEBUG 2: Computing Categorical Statistics.
> > >
> > > DEBUG 2: Computing Continuous Statistics.
> > >
> > > DEBUG 2: Computing Scalar Partial Sums.
> > >
> > > DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region
FULL,
> > using
> > > 258840 matched pairs.
> > >
> > > DEBUG 2: Computing Gradient DX(1)/DY(1) Statistics over region
NH,
> using
> > > 479 matched pairs.
> > >
> > > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing
method
> > > NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region
FULL,
> using
> > > 259920 matched pairs.
> > >
> > > DEBUG 2: Computing Neighborhood Categorical Statistics.
> > >
> > > DEBUG 2: Computing Neighborhood Continuous Statistics.
> > >
> > > DEBUG 2: Processing HGT/P500 versus HGT/P500, for smoothing
method
> > > NBRHD_CIRCLE(1), raw thresholds of >0.0 and >0.0, over region
NH, using
> > 479
> > > matched pairs.
> > >
> > > DEBUG 2: Computing Neighborhood Categorical Statistics.
> > >
> > > DEBUG 2: Computing Neighborhood Continuous Statistics.
> > >
> > >
> > > Do you have any idea why this may be happening with the new
version?
> > > Obviously I would want more matched pairs. I attached the
GridStat
> config
> > > file I am using. Thanks!
> > >
> > >
> > > Allan Huffman
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day.  Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

------------------------------------------------