[Met_help] [rt.rap.ucar.edu #91026] History for Differing matched pair between MET 7.0 and MET 8.0/8.1

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,

I was comparing output from grid_stat from MET 8.1 to results for VSDB. I
noticed large differences for the NCEP verification regions NPO, SPO, NAO,
and SPO. Looking further into the matched pairs for these regions appear to
be way off.

DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region NPO, using 8583 pairs.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region SPO, using 8400 pairs.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region NAO, using 8409 pairs.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region SAO, using 8331 pairs.

These are tiny regions over the northern and southern Atlantic and Pacific
Oceans. The matches pairs for VSDB are NPO with 365, SPO is 179, NAO is
188, and SAO is 112.

I didn't remember this being an issue before so I went back and tested
older version of MET. I ran the same case with the same masks for 8.0 and
had the same result. However when I ran with 7.0, I saw something different.

DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region NPO, using 362 pairs.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region SPO, using 179 pairs.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region NAO, using 188 pairs.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for smoothing method NEAREST(1),
over region SAO, using 110 pairs.

These are much closer to and comparable to VSDB.

What is interesting is that the rectangular/latitude defined regions all
have the correct, and same number of matches pairs for all MET versions. It
just seems that there is a problem with regions that are more oddly shaped,
and not as "clean" per say.

I'm not sure what is going on, maybe I am missing something. I have a
directory on WCOSS (Luna) with all my log files. Any help is welcome!

Mallory


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Differing matched pair between MET 7.0 and MET 8.0/8.1
From: John Halley Gotway
Time: Fri Jul 12 15:52:04 2019

Mallory,

Sorry for the delay.  Glad you were able to make progress on this
issue.  I agree with your assessment.  The masks are defined by Grid
104... and their extent is limited by the extent of that grid.

It's always seemed awkward to me that the masking regions we use look
like stair-steps.  They're defined as a union of grid points for Grid
104, which is pretty coarse.  It would be preferable to replace that
with a set of lat/lon's which describe the regions, but using straight
lines instead of jagged stair-steps.

I know you all were talking about changes to the regions used in
verification.  I was hoping that that'd result in "smoother" areas.
Has there been any recent discussion on that?

Thanks,
John

------------------------------------------------
Subject: Differing matched pair between MET 7.0 and MET 8.0/8.1
From: Mallory Row - NOAA Affiliate
Time: Mon Jul 15 05:49:38 2019

Hi John,

There hasn't been any movement on working towards new verification
regions;
there have been bigger tasks I have been needing to work on, but
hopefully
it is something I can start looking back into sometime in the future.

I think the counting of the masked grid points in the verification
regions
is still something that needs to be address in MET. I was able to do a
workaround, but there still is the bug within MET, especially since
this
hasn't existed prior to version 8.0.

Mallory

On Fri, Jul 12, 2019 at 5:52 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Mallory,
>
> Sorry for the delay.  Glad you were able to make progress on this
issue.
> I agree with your assessment.  The masks are defined by Grid 104...
and
> their extent is limited by the extent of that grid.
>
> It's always seemed awkward to me that the masking regions we use
look like
> stair-steps.  They're defined as a union of grid points for Grid
104, which
> is pretty coarse.  It would be preferable to replace that with a set
of
> lat/lon's which describe the regions, but using straight lines
instead of
> jagged stair-steps.
>
> I know you all were talking about changes to the regions used in
> verification.  I was hoping that that'd result in "smoother" areas.
Has
> there been any recent discussion on that?
>
> Thanks,
> John
>

------------------------------------------------
Subject: Differing matched pair between MET 7.0 and MET 8.0/8.1
From: John Halley Gotway
Time: Mon Jul 15 12:03:49 2019

Mallory,

I found and fixed the problem.  Here's the GitHub issue describing the
problem and solution:
   https://github.com/NCAR/MET/issues/1163
This change will be included in met-8.1.2.

FYI, in met-7.0 we were storing mask regions using DataPlane objects,
which
use double precision values.  Working with Dana Strom (who uses tons
of
masks), we switched to using MaskPlane objects, which store data as
booleans (only 1 byte).  That made Grid-Stat much faster (when
allocating
memory), and take up much less memory.

The bug is in the code which converts a DataPlane object to a
MaskPlane
object where we failed to check for bad data values.  The fix is a one
liner.  Bad data values now result in a mask value of false, instead
of
true.

Thanks,
John

On Mon, Jul 15, 2019 at 5:50 AM Mallory Row - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91026 >
>
> Hi John,
>
> There hasn't been any movement on working towards new verification
regions;
> there have been bigger tasks I have been needing to work on, but
hopefully
> it is something I can start looking back into sometime in the
future.
>
> I think the counting of the masked grid points in the verification
regions
> is still something that needs to be address in MET. I was able to do
a
> workaround, but there still is the bug within MET, especially since
this
> hasn't existed prior to version 8.0.
>
> Mallory
>
> On Fri, Jul 12, 2019 at 5:52 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Mallory,
> >
> > Sorry for the delay.  Glad you were able to make progress on this
issue.
> > I agree with your assessment.  The masks are defined by Grid
104... and
> > their extent is limited by the extent of that grid.
> >
> > It's always seemed awkward to me that the masking regions we use
look
> like
> > stair-steps.  They're defined as a union of grid points for Grid
104,
> which
> > is pretty coarse.  It would be preferable to replace that with a
set of
> > lat/lon's which describe the regions, but using straight lines
instead of
> > jagged stair-steps.
> >
> > I know you all were talking about changes to the regions used in
> > verification.  I was hoping that that'd result in "smoother"
areas.  Has
> > there been any recent discussion on that?
> >
> > Thanks,
> > John
> >
>
>

------------------------------------------------
Subject: Differing matched pair between MET 7.0 and MET 8.0/8.1
From: Mallory Row - NOAA Affiliate
Time: Mon Jul 15 12:06:18 2019

Great, thanks for all the help and for the fix!

Mallory

On Mon, Jul 15, 2019 at 2:03 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Mallory,
>
> I found and fixed the problem.  Here's the GitHub issue describing
the
> problem and solution:
>    https://github.com/NCAR/MET/issues/1163
> This change will be included in met-8.1.2.
>
> FYI, in met-7.0 we were storing mask regions using DataPlane
objects, which
> use double precision values.  Working with Dana Strom (who uses tons
of
> masks), we switched to using MaskPlane objects, which store data as
> booleans (only 1 byte).  That made Grid-Stat much faster (when
allocating
> memory), and take up much less memory.
>
> The bug is in the code which converts a DataPlane object to a
MaskPlane
> object where we failed to check for bad data values.  The fix is a
one
> liner.  Bad data values now result in a mask value of false, instead
of
> true.
>
> Thanks,
> John
>
> On Mon, Jul 15, 2019 at 5:50 AM Mallory Row - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91026 >
> >
> > Hi John,
> >
> > There hasn't been any movement on working towards new verification
> regions;
> > there have been bigger tasks I have been needing to work on, but
> hopefully
> > it is something I can start looking back into sometime in the
future.
> >
> > I think the counting of the masked grid points in the verification
> regions
> > is still something that needs to be address in MET. I was able to
do a
> > workaround, but there still is the bug within MET, especially
since this
> > hasn't existed prior to version 8.0.
> >
> > Mallory
> >
> > On Fri, Jul 12, 2019 at 5:52 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Mallory,
> > >
> > > Sorry for the delay.  Glad you were able to make progress on
this
> issue.
> > > I agree with your assessment.  The masks are defined by Grid
104... and
> > > their extent is limited by the extent of that grid.
> > >
> > > It's always seemed awkward to me that the masking regions we use
look
> > like
> > > stair-steps.  They're defined as a union of grid points for Grid
104,
> > which
> > > is pretty coarse.  It would be preferable to replace that with a
set of
> > > lat/lon's which describe the regions, but using straight lines
instead
> of
> > > jagged stair-steps.
> > >
> > > I know you all were talking about changes to the regions used in
> > > verification.  I was hoping that that'd result in "smoother"
areas.
> Has
> > > there been any recent discussion on that?
> > >
> > > Thanks,
> > > John
> > >
> >
> >
>
>

------------------------------------------------