[Met_help] [rt.rap.ucar.edu #88158] History for grid_stat Fourier Decomposition
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 9 12:07:07 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
E-mailing about a possible issue I found in the Fourier decomposition
method using grid_stat to compute SAL1L2 for heights. Below is an output
line from MET and the same output for VSDB. Attached is an image showing
the same thing, but for a few days. Happy to provide an assistance to help
solve the issue!
METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000 20170804_000000
000000 20170804_000000 20170804_000000 HGT P250 HGT P250
fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA SAL1L2 1311
-10722.64904
-10722.63741 115033718.70405 115033873.35398 115033574.63798 2.3311
VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400 FV3GFS_MALLORYTEST G2/PNA SAL1L2
HGT_WV1/10-20 P250 = 1311. 0.475731658E+00 0.487286476E+00
0.465863958E+03 0.471534680E+03 0.470775184E+03
Mallory
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: John Halley Gotway
Time: Fri Dec 14 17:05:54 2018
Mallory,
Thanks for pointing this out. I certainly agree that those SAL1L2
numbers
look very off! When I see output values from MET that are anywhere
close
to -9999 (like -10722), the first thing I check is to see if somewhere
MET
is using the bad data value of -9999 in computing statistics. For
example,
since this is SAL1L2, it should be computing FCST - CLIMO. If the
forecast
value is bad data (-9999) and the climo value is something like 723,
then
we'd get -9999 - 723 = -10722.
Obviously this should *not* be happening. The easiest way to debug
would
be getting a very simple test case which replicates this behavior.
Can you
please send (or point me to) sample fcst, obs, and climo data for
Grid-Stat, along with the Grid-Stat config file you're using?
FYI, I'll be out of the office for the next two weeks. But if you
respond
to this email, other staff, like Julie, Minna, Howard, and Randy will
get a
copy and could try to replicate/fix this behavior.
Thanks,
John
On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
> Transaction: Ticket created by mallory.row at noaa.gov
> Queue: met_help
> Subject: grid_stat Fourier Decomposition
> Owner: Nobody
> Requestors: mallory.row at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>
>
> Hello,
>
> E-mailing about a possible issue I found in the Fourier
decomposition
> method using grid_stat to compute SAL1L2 for heights. Below is an
output
> line from MET and the same output for VSDB. Attached is an image
showing
> the same thing, but for a few days. Happy to provide an assistance
to help
> solve the issue!
>
> METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
20170804_000000
> 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
> fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA SAL1L2 1311
> -10722.64904
> -10722.63741 115033718.70405 115033873.35398 115033574.63798 2.3311
>
>
> VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400 FV3GFS_MALLORYTEST G2/PNA
SAL1L2
> HGT_WV1/10-20 P250 = 1311. 0.475731658E+00 0.487286476E+00
> 0.465863958E+03 0.471534680E+03 0.470775184E+03
>
> Mallory
>
>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: Mallory Row - NOAA Affiliate
Time: Mon Dec 17 06:29:16 2018
Hi John,
That would make sense. I set up a directory on Theia
at
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
can move copy on to WCOSS if wanted too. Thanks for all the help.
Enjoy
your two weeks off, John (hopefully you don't see this until after you
get
back!).
Mallory
On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Mallory,
>
> Thanks for pointing this out. I certainly agree that those SAL1L2
numbers
> look very off! When I see output values from MET that are anywhere
close
> to -9999 (like -10722), the first thing I check is to see if
somewhere MET
> is using the bad data value of -9999 in computing statistics. For
example,
> since this is SAL1L2, it should be computing FCST - CLIMO. If the
forecast
> value is bad data (-9999) and the climo value is something like 723,
then
> we'd get -9999 - 723 = -10722.
>
> Obviously this should *not* be happening. The easiest way to debug
would
> be getting a very simple test case which replicates this behavior.
Can you
> please send (or point me to) sample fcst, obs, and climo data for
> Grid-Stat, along with the Grid-Stat config file you're using?
>
> FYI, I'll be out of the office for the next two weeks. But if you
respond
> to this email, other staff, like Julie, Minna, Howard, and Randy
will get a
> copy and could try to replicate/fix this behavior.
>
> Thanks,
> John
>
> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
> > Transaction: Ticket created by mallory.row at noaa.gov
> > Queue: met_help
> > Subject: grid_stat Fourier Decomposition
> > Owner: Nobody
> > Requestors: mallory.row at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> >
> >
> > Hello,
> >
> > E-mailing about a possible issue I found in the Fourier
decomposition
> > method using grid_stat to compute SAL1L2 for heights. Below is an
output
> > line from MET and the same output for VSDB. Attached is an image
showing
> > the same thing, but for a few days. Happy to provide an assistance
to
> help
> > solve the issue!
> >
> > METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
> 20170804_000000
> > 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA SAL1L2
1311
> > -10722.64904
> > -10722.63741 115033718.70405 115033873.35398 115033574.63798
2.3311
> >
> >
> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400 FV3GFS_MALLORYTEST
G2/PNA
> SAL1L2
> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00 0.487286476E+00
> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
> >
> > Mallory
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: Mallory Row - NOAA Affiliate
Time: Wed Dec 19 11:38:46 2018
I did a dive more into this and decided to plot the Anomaly
Correlation,
and MET and VSDB values are pretty similar. Not sure exactly what this
all
means... maybe there is some scaling factor in VSDB I am not aware of?
Mallory
On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA Affiliate <
mallory.row at noaa.gov> wrote:
> Hi John,
>
> That would make sense. I set up a directory on Theia
> at
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
> can move copy on to WCOSS if wanted too. Thanks for all the help.
Enjoy
> your two weeks off, John (hopefully you don't see this until after
you
> get back!).
>
> Mallory
>
> On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Mallory,
>>
>> Thanks for pointing this out. I certainly agree that those SAL1L2
numbers
>> look very off! When I see output values from MET that are anywhere
close
>> to -9999 (like -10722), the first thing I check is to see if
somewhere MET
>> is using the bad data value of -9999 in computing statistics. For
>> example,
>> since this is SAL1L2, it should be computing FCST - CLIMO. If the
>> forecast
>> value is bad data (-9999) and the climo value is something like
723, then
>> we'd get -9999 - 723 = -10722.
>>
>> Obviously this should *not* be happening. The easiest way to debug
would
>> be getting a very simple test case which replicates this behavior.
Can
>> you
>> please send (or point me to) sample fcst, obs, and climo data for
>> Grid-Stat, along with the Grid-Stat config file you're using?
>>
>> FYI, I'll be out of the office for the next two weeks. But if you
respond
>> to this email, other staff, like Julie, Minna, Howard, and Randy
will get
>> a
>> copy and could try to replicate/fix this behavior.
>>
>> Thanks,
>> John
>>
>> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate via RT
<
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
>> > Transaction: Ticket created by mallory.row at noaa.gov
>> > Queue: met_help
>> > Subject: grid_stat Fourier Decomposition
>> > Owner: Nobody
>> > Requestors: mallory.row at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>> >
>> >
>> > Hello,
>> >
>> > E-mailing about a possible issue I found in the Fourier
decomposition
>> > method using grid_stat to compute SAL1L2 for heights. Below is an
output
>> > line from MET and the same output for VSDB. Attached is an image
showing
>> > the same thing, but for a few days. Happy to provide an
assistance to
>> help
>> > solve the issue!
>> >
>> > METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
>> 20170804_000000
>> > 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
>> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA SAL1L2
1311
>> > -10722.64904
>> > -10722.63741 115033718.70405 115033873.35398 115033574.63798
2.3311
>> >
>> >
>> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400 FV3GFS_MALLORYTEST
G2/PNA
>> SAL1L2
>> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00 0.487286476E+00
>> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
>> >
>> > Mallory
>> >
>> >
>>
>>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: John Halley Gotway
Time: Thu Jan 03 16:29:50 2019
Mallory,
Thanks for setting up the sample data directory on theia. I pulled
down
the files, reran it here, and was able to exactly replicate the .stat
file
you included in that directory.
I suspect the discrepancy in scores is due to a difference in how MET
and
VSDB are computing them.
MET is applying the fourier decomposition to the forecast and
observation
fields, but *NOT* the climatology fields.
However, it looks like VSDB is applying the decomposition to the
climatology data as well:
FROM:
*verf_g2g.v3.0.11/sorc/verf_g2g_grid2grid_grib2.fd/grid2grid.f*
1147 *call
FITWAV_1D(clim2d,imax(1),jmax(1),wv1(nvr),wv2(nvr))*
This difference explains the behavior we're seeing. At waves 4-9 and
10-20, the decomposed forecast and observation values are very close
to
zero. When we subtract off the climo values (which are on average
about
10599), we get anomaly values of -10599 or so. I was computing things
"manually" in R and couldn't get the numbers to match up. Turns out
it was
the Cosine Latitude weighting that was confusing me.
So it seems to me that we should enhance MET to decompose the
climatology
mean fields as well.
But this gets a little confusing. Am I correct in thinking that you
wouldn't ever compute the fourier decomposition *AND* apply binned
climatologies at the same time? It sounds like we should decompose
the
climo mean and not actually use the climo standard deviations in any
way.
Unfortunately, this change will require a patch for met-8.0. Or we
could
just fix it in met-8.1.
Thanks,
John
On Wed, Dec 19, 2018 at 11:39 AM Mallory Row - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>
> I did a dive more into this and decided to plot the Anomaly
Correlation,
> and MET and VSDB values are pretty similar. Not sure exactly what
this all
> means... maybe there is some scaling factor in VSDB I am not aware
of?
>
> Mallory
>
> On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA Affiliate <
> mallory.row at noaa.gov> wrote:
>
> > Hi John,
> >
> > That would make sense. I set up a directory on Theia
> > at
>
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
> > can move copy on to WCOSS if wanted too. Thanks for all the help.
Enjoy
> > your two weeks off, John (hopefully you don't see this until after
you
> > get back!).
> >
> > Mallory
> >
> > On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Mallory,
> >>
> >> Thanks for pointing this out. I certainly agree that those
SAL1L2
> numbers
> >> look very off! When I see output values from MET that are
anywhere
> close
> >> to -9999 (like -10722), the first thing I check is to see if
somewhere
> MET
> >> is using the bad data value of -9999 in computing statistics.
For
> >> example,
> >> since this is SAL1L2, it should be computing FCST - CLIMO. If
the
> >> forecast
> >> value is bad data (-9999) and the climo value is something like
723,
> then
> >> we'd get -9999 - 723 = -10722.
> >>
> >> Obviously this should *not* be happening. The easiest way to
debug
> would
> >> be getting a very simple test case which replicates this
behavior. Can
> >> you
> >> please send (or point me to) sample fcst, obs, and climo data for
> >> Grid-Stat, along with the Grid-Stat config file you're using?
> >>
> >> FYI, I'll be out of the office for the next two weeks. But if
you
> respond
> >> to this email, other staff, like Julie, Minna, Howard, and Randy
will
> get
> >> a
> >> copy and could try to replicate/fix this behavior.
> >>
> >> Thanks,
> >> John
> >>
> >> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate via
RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
> >> > Transaction: Ticket created by mallory.row at noaa.gov
> >> > Queue: met_help
> >> > Subject: grid_stat Fourier Decomposition
> >> > Owner: Nobody
> >> > Requestors: mallory.row at noaa.gov
> >> > Status: new
> >> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158
> >
> >> >
> >> >
> >> > Hello,
> >> >
> >> > E-mailing about a possible issue I found in the Fourier
decomposition
> >> > method using grid_stat to compute SAL1L2 for heights. Below is
an
> output
> >> > line from MET and the same output for VSDB. Attached is an
image
> showing
> >> > the same thing, but for a few days. Happy to provide an
assistance to
> >> help
> >> > solve the issue!
> >> >
> >> > METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
> >> 20170804_000000
> >> > 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
> >> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA SAL1L2
1311
> >> > -10722.64904
> >> > -10722.63741 115033718.70405 115033873.35398 115033574.63798
2.3311
> >> >
> >> >
> >> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400 FV3GFS_MALLORYTEST
G2/PNA
> >> SAL1L2
> >> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00 0.487286476E+00
> >> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
> >> >
> >> > Mallory
> >> >
> >> >
> >>
> >>
>
>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: John Halley Gotway
Time: Thu Jan 03 16:53:19 2019
Mallory,
FYI, after making that change... applying the fourier decomp to the
climo
mean field, I'm able to get the SAL1L2 values much closer to the VSDB
ones:
*HGT P250 fv3gfs_mallorytest PNA WV1_10-20 SAL1L2 1311
0.48839 0.47266 437.76605 447.04559 440.30188 2.48062*
They are not identical to what you sent, but definitely the correct
magnitude.
John
On Thu, Jan 3, 2019 at 4:29 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Mallory,
>
> Thanks for setting up the sample data directory on theia. I pulled
down
> the files, reran it here, and was able to exactly replicate the
.stat file
> you included in that directory.
>
> I suspect the discrepancy in scores is due to a difference in how
MET and
> VSDB are computing them.
>
> MET is applying the fourier decomposition to the forecast and
observation
> fields, but *NOT* the climatology fields.
>
> However, it looks like VSDB is applying the decomposition to the
> climatology data as well:
> FROM:
*verf_g2g.v3.0.11/sorc/verf_g2g_grid2grid_grib2.fd/grid2grid.f*
> 1147 *call
> FITWAV_1D(clim2d,imax(1),jmax(1),wv1(nvr),wv2(nvr))*
>
> This difference explains the behavior we're seeing. At waves 4-9
and
> 10-20, the decomposed forecast and observation values are very close
to
> zero. When we subtract off the climo values (which are on average
about
> 10599), we get anomaly values of -10599 or so. I was computing
things
> "manually" in R and couldn't get the numbers to match up. Turns out
it was
> the Cosine Latitude weighting that was confusing me.
>
> So it seems to me that we should enhance MET to decompose the
climatology
> mean fields as well.
>
> But this gets a little confusing. Am I correct in thinking that you
> wouldn't ever compute the fourier decomposition *AND* apply binned
> climatologies at the same time? It sounds like we should decompose
the
> climo mean and not actually use the climo standard deviations in any
way.
>
> Unfortunately, this change will require a patch for met-8.0. Or we
could
> just fix it in met-8.1.
>
> Thanks,
> John
>
>
>
> On Wed, Dec 19, 2018 at 11:39 AM Mallory Row - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>>
>> I did a dive more into this and decided to plot the Anomaly
Correlation,
>> and MET and VSDB values are pretty similar. Not sure exactly what
this all
>> means... maybe there is some scaling factor in VSDB I am not aware
of?
>>
>> Mallory
>>
>> On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA Affiliate <
>> mallory.row at noaa.gov> wrote:
>>
>> > Hi John,
>> >
>> > That would make sense. I set up a directory on Theia
>> > at
>>
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
>> > can move copy on to WCOSS if wanted too. Thanks for all the help.
Enjoy
>> > your two weeks off, John (hopefully you don't see this until
after you
>> > get back!).
>> >
>> > Mallory
>> >
>> > On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> >> Mallory,
>> >>
>> >> Thanks for pointing this out. I certainly agree that those
SAL1L2
>> numbers
>> >> look very off! When I see output values from MET that are
anywhere
>> close
>> >> to -9999 (like -10722), the first thing I check is to see if
somewhere
>> MET
>> >> is using the bad data value of -9999 in computing statistics.
For
>> >> example,
>> >> since this is SAL1L2, it should be computing FCST - CLIMO. If
the
>> >> forecast
>> >> value is bad data (-9999) and the climo value is something like
723,
>> then
>> >> we'd get -9999 - 723 = -10722.
>> >>
>> >> Obviously this should *not* be happening. The easiest way to
debug
>> would
>> >> be getting a very simple test case which replicates this
behavior. Can
>> >> you
>> >> please send (or point me to) sample fcst, obs, and climo data
for
>> >> Grid-Stat, along with the Grid-Stat config file you're using?
>> >>
>> >> FYI, I'll be out of the office for the next two weeks. But if
you
>> respond
>> >> to this email, other staff, like Julie, Minna, Howard, and Randy
will
>> get
>> >> a
>> >> copy and could try to replicate/fix this behavior.
>> >>
>> >> Thanks,
>> >> John
>> >>
>> >> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate via
RT <
>> >> met_help at ucar.edu> wrote:
>> >>
>> >> >
>> >> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
>> >> > Transaction: Ticket created by mallory.row at noaa.gov
>> >> > Queue: met_help
>> >> > Subject: grid_stat Fourier Decomposition
>> >> > Owner: Nobody
>> >> > Requestors: mallory.row at noaa.gov
>> >> > Status: new
>> >> > Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>> >> >
>> >> >
>> >> > Hello,
>> >> >
>> >> > E-mailing about a possible issue I found in the Fourier
decomposition
>> >> > method using grid_stat to compute SAL1L2 for heights. Below is
an
>> output
>> >> > line from MET and the same output for VSDB. Attached is an
image
>> showing
>> >> > the same thing, but for a few days. Happy to provide an
assistance to
>> >> help
>> >> > solve the issue!
>> >> >
>> >> > METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
>> >> 20170804_000000
>> >> > 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
>> >> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA
SAL1L2 1311
>> >> > -10722.64904
>> >> > -10722.63741 115033718.70405 115033873.35398 115033574.63798
2.3311
>> >> >
>> >> >
>> >> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400 FV3GFS_MALLORYTEST
G2/PNA
>> >> SAL1L2
>> >> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00 0.487286476E+00
>> >> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
>> >> >
>> >> > Mallory
>> >> >
>> >> >
>> >>
>> >>
>>
>>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: Mallory Row - NOAA Affiliate
Time: Mon Jan 28 07:35:02 2019
Hi John,
Thanks for working on this. I think I saw in another e-mail chain that
this
is included as a patch for 8.0?
Mallory
On Thu, Jan 3, 2019 at 6:53 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Mallory,
>
> FYI, after making that change... applying the fourier decomp to the
climo
> mean field, I'm able to get the SAL1L2 values much closer to the
VSDB ones:
> *HGT P250 fv3gfs_mallorytest PNA WV1_10-20 SAL1L2
1311
> 0.48839 0.47266 437.76605 447.04559 440.30188 2.48062*
>
> They are not identical to what you sent, but definitely the correct
> magnitude.
>
> John
>
>
> On Thu, Jan 3, 2019 at 4:29 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> > Mallory,
> >
> > Thanks for setting up the sample data directory on theia. I
pulled down
> > the files, reran it here, and was able to exactly replicate the
.stat
> file
> > you included in that directory.
> >
> > I suspect the discrepancy in scores is due to a difference in how
MET and
> > VSDB are computing them.
> >
> > MET is applying the fourier decomposition to the forecast and
observation
> > fields, but *NOT* the climatology fields.
> >
> > However, it looks like VSDB is applying the decomposition to the
> > climatology data as well:
> > FROM:
*verf_g2g.v3.0.11/sorc/verf_g2g_grid2grid_grib2.fd/grid2grid.f*
> > 1147 *call
> > FITWAV_1D(clim2d,imax(1),jmax(1),wv1(nvr),wv2(nvr))*
> >
> > This difference explains the behavior we're seeing. At waves 4-9
and
> > 10-20, the decomposed forecast and observation values are very
close to
> > zero. When we subtract off the climo values (which are on average
about
> > 10599), we get anomaly values of -10599 or so. I was computing
things
> > "manually" in R and couldn't get the numbers to match up. Turns
out it
> was
> > the Cosine Latitude weighting that was confusing me.
> >
> > So it seems to me that we should enhance MET to decompose the
climatology
> > mean fields as well.
> >
> > But this gets a little confusing. Am I correct in thinking that
you
> > wouldn't ever compute the fourier decomposition *AND* apply binned
> > climatologies at the same time? It sounds like we should
decompose the
> > climo mean and not actually use the climo standard deviations in
any way.
> >
> > Unfortunately, this change will require a patch for met-8.0. Or
we could
> > just fix it in met-8.1.
> >
> > Thanks,
> > John
> >
> >
> >
> > On Wed, Dec 19, 2018 at 11:39 AM Mallory Row - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> >>
> >> I did a dive more into this and decided to plot the Anomaly
Correlation,
> >> and MET and VSDB values are pretty similar. Not sure exactly what
this
> all
> >> means... maybe there is some scaling factor in VSDB I am not
aware of?
> >>
> >> Mallory
> >>
> >> On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA Affiliate <
> >> mallory.row at noaa.gov> wrote:
> >>
> >> > Hi John,
> >> >
> >> > That would make sense. I set up a directory on Theia
> >> > at
> >>
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
> >> > can move copy on to WCOSS if wanted too. Thanks for all the
help.
> Enjoy
> >> > your two weeks off, John (hopefully you don't see this until
after you
> >> > get back!).
> >> >
> >> > Mallory
> >> >
> >> > On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> >> Mallory,
> >> >>
> >> >> Thanks for pointing this out. I certainly agree that those
SAL1L2
> >> numbers
> >> >> look very off! When I see output values from MET that are
anywhere
> >> close
> >> >> to -9999 (like -10722), the first thing I check is to see if
> somewhere
> >> MET
> >> >> is using the bad data value of -9999 in computing statistics.
For
> >> >> example,
> >> >> since this is SAL1L2, it should be computing FCST - CLIMO. If
the
> >> >> forecast
> >> >> value is bad data (-9999) and the climo value is something
like 723,
> >> then
> >> >> we'd get -9999 - 723 = -10722.
> >> >>
> >> >> Obviously this should *not* be happening. The easiest way to
debug
> >> would
> >> >> be getting a very simple test case which replicates this
behavior.
> Can
> >> >> you
> >> >> please send (or point me to) sample fcst, obs, and climo data
for
> >> >> Grid-Stat, along with the Grid-Stat config file you're using?
> >> >>
> >> >> FYI, I'll be out of the office for the next two weeks. But if
you
> >> respond
> >> >> to this email, other staff, like Julie, Minna, Howard, and
Randy will
> >> get
> >> >> a
> >> >> copy and could try to replicate/fix this behavior.
> >> >>
> >> >> Thanks,
> >> >> John
> >> >>
> >> >> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate
via RT <
> >> >> met_help at ucar.edu> wrote:
> >> >>
> >> >> >
> >> >> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
> >> >> > Transaction: Ticket created by mallory.row at noaa.gov
> >> >> > Queue: met_help
> >> >> > Subject: grid_stat Fourier Decomposition
> >> >> > Owner: Nobody
> >> >> > Requestors: mallory.row at noaa.gov
> >> >> > Status: new
> >> >> > Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> >> >> >
> >> >> >
> >> >> > Hello,
> >> >> >
> >> >> > E-mailing about a possible issue I found in the Fourier
> decomposition
> >> >> > method using grid_stat to compute SAL1L2 for heights. Below
is an
> >> output
> >> >> > line from MET and the same output for VSDB. Attached is an
image
> >> showing
> >> >> > the same thing, but for a few days. Happy to provide an
assistance
> to
> >> >> help
> >> >> > solve the issue!
> >> >> >
> >> >> > METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
> >> >> 20170804_000000
> >> >> > 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
> >> >> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA
SAL1L2
> 1311
> >> >> > -10722.64904
> >> >> > -10722.63741 115033718.70405 115033873.35398
115033574.63798
> 2.3311
> >> >> >
> >> >> >
> >> >> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400
FV3GFS_MALLORYTEST
> G2/PNA
> >> >> SAL1L2
> >> >> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00
0.487286476E+00
> >> >> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
> >> >> >
> >> >> > Mallory
> >> >> >
> >> >> >
> >> >>
> >> >>
> >>
> >>
>
>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: John Halley Gotway
Time: Mon Jan 28 10:23:58 2019
Mallory,
Yes, it's posted as a patch for met-8.0 dated 01/04/2019 and described
here:
https://dtcenter.org/met/users/support/known_issues/METv8.0/index.php
Basically, grid-stat wasn't applying the fourier decomposition to the
climo
data and it's pretty clear that the VSDB code was. Julie has
recompiled
with the patch on NOAA machines but I'm not exactly sure where.
Try running a MET tool from the build you're using with the "--
version"
command line option. That should tell you the latest patch level...
and
should look something like this:
*[johnhg at dakota]% met-8.0/bin/grid_stat --versionMET Version:
V8.0Repository:
https://svn-met-dev.cgd.ucar.edu/branches/met/met-8.0_bugfix
<https://svn-met-dev.cgd.ucar.edu/branches/met/met-
8.0_bugfix>Revision:
5953Change Date: 2019-01-04 14:01:21 -0700 (Fri, 04 Jan 2019)*
On Mon, Jan 28, 2019 at 7:35 AM Mallory Row - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>
> Hi John,
>
> Thanks for working on this. I think I saw in another e-mail chain
that this
> is included as a patch for 8.0?
>
> Mallory
>
> On Thu, Jan 3, 2019 at 6:53 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Mallory,
> >
> > FYI, after making that change... applying the fourier decomp to
the climo
> > mean field, I'm able to get the SAL1L2 values much closer to the
VSDB
> ones:
> > *HGT P250 fv3gfs_mallorytest PNA WV1_10-20 SAL1L2
1311
> > 0.48839 0.47266 437.76605 447.04559 440.30188 2.48062*
> >
> > They are not identical to what you sent, but definitely the
correct
> > magnitude.
> >
> > John
> >
> >
> > On Thu, Jan 3, 2019 at 4:29 PM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
> >
> > > Mallory,
> > >
> > > Thanks for setting up the sample data directory on theia. I
pulled
> down
> > > the files, reran it here, and was able to exactly replicate the
.stat
> > file
> > > you included in that directory.
> > >
> > > I suspect the discrepancy in scores is due to a difference in
how MET
> and
> > > VSDB are computing them.
> > >
> > > MET is applying the fourier decomposition to the forecast and
> observation
> > > fields, but *NOT* the climatology fields.
> > >
> > > However, it looks like VSDB is applying the decomposition to the
> > > climatology data as well:
> > > FROM:
*verf_g2g.v3.0.11/sorc/verf_g2g_grid2grid_grib2.fd/grid2grid.f*
> > > 1147 *call
> > > FITWAV_1D(clim2d,imax(1),jmax(1),wv1(nvr),wv2(nvr))*
> > >
> > > This difference explains the behavior we're seeing. At waves 4-
9 and
> > > 10-20, the decomposed forecast and observation values are very
close to
> > > zero. When we subtract off the climo values (which are on
average
> about
> > > 10599), we get anomaly values of -10599 or so. I was computing
things
> > > "manually" in R and couldn't get the numbers to match up. Turns
out it
> > was
> > > the Cosine Latitude weighting that was confusing me.
> > >
> > > So it seems to me that we should enhance MET to decompose the
> climatology
> > > mean fields as well.
> > >
> > > But this gets a little confusing. Am I correct in thinking that
you
> > > wouldn't ever compute the fourier decomposition *AND* apply
binned
> > > climatologies at the same time? It sounds like we should
decompose the
> > > climo mean and not actually use the climo standard deviations in
any
> way.
> > >
> > > Unfortunately, this change will require a patch for met-8.0. Or
we
> could
> > > just fix it in met-8.1.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > >
> > > On Wed, Dec 19, 2018 at 11:39 AM Mallory Row - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> > >>
> > >> I did a dive more into this and decided to plot the Anomaly
> Correlation,
> > >> and MET and VSDB values are pretty similar. Not sure exactly
what this
> > all
> > >> means... maybe there is some scaling factor in VSDB I am not
aware of?
> > >>
> > >> Mallory
> > >>
> > >> On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA Affiliate <
> > >> mallory.row at noaa.gov> wrote:
> > >>
> > >> > Hi John,
> > >> >
> > >> > That would make sense. I set up a directory on Theia
> > >> > at
> > >>
>
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
> > >> > can move copy on to WCOSS if wanted too. Thanks for all the
help.
> > Enjoy
> > >> > your two weeks off, John (hopefully you don't see this until
after
> you
> > >> > get back!).
> > >> >
> > >> > Mallory
> > >> >
> > >> > On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT <
> > >> > met_help at ucar.edu> wrote:
> > >> >
> > >> >> Mallory,
> > >> >>
> > >> >> Thanks for pointing this out. I certainly agree that those
SAL1L2
> > >> numbers
> > >> >> look very off! When I see output values from MET that are
anywhere
> > >> close
> > >> >> to -9999 (like -10722), the first thing I check is to see if
> > somewhere
> > >> MET
> > >> >> is using the bad data value of -9999 in computing
statistics. For
> > >> >> example,
> > >> >> since this is SAL1L2, it should be computing FCST - CLIMO.
If the
> > >> >> forecast
> > >> >> value is bad data (-9999) and the climo value is something
like
> 723,
> > >> then
> > >> >> we'd get -9999 - 723 = -10722.
> > >> >>
> > >> >> Obviously this should *not* be happening. The easiest way
to debug
> > >> would
> > >> >> be getting a very simple test case which replicates this
behavior.
> > Can
> > >> >> you
> > >> >> please send (or point me to) sample fcst, obs, and climo
data for
> > >> >> Grid-Stat, along with the Grid-Stat config file you're
using?
> > >> >>
> > >> >> FYI, I'll be out of the office for the next two weeks. But
if you
> > >> respond
> > >> >> to this email, other staff, like Julie, Minna, Howard, and
Randy
> will
> > >> get
> > >> >> a
> > >> >> copy and could try to replicate/fix this behavior.
> > >> >>
> > >> >> Thanks,
> > >> >> John
> > >> >>
> > >> >> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA Affiliate
via
> RT <
> > >> >> met_help at ucar.edu> wrote:
> > >> >>
> > >> >> >
> > >> >> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
> > >> >> > Transaction: Ticket created by mallory.row at noaa.gov
> > >> >> > Queue: met_help
> > >> >> > Subject: grid_stat Fourier Decomposition
> > >> >> > Owner: Nobody
> > >> >> > Requestors: mallory.row at noaa.gov
> > >> >> > Status: new
> > >> >> > Ticket <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> > >> >> >
> > >> >> >
> > >> >> > Hello,
> > >> >> >
> > >> >> > E-mailing about a possible issue I found in the Fourier
> > decomposition
> > >> >> > method using grid_stat to compute SAL1L2 for heights.
Below is an
> > >> output
> > >> >> > line from MET and the same output for VSDB. Attached is an
image
> > >> showing
> > >> >> > the same thing, but for a few days. Happy to provide an
> assistance
> > to
> > >> >> help
> > >> >> > solve the issue!
> > >> >> >
> > >> >> > METplus: V6.1 fv3gfs_mallorytest NA 240000 20170804_000000
> > >> >> 20170804_000000
> > >> >> > 000000 20170804_000000 20170804_000000 HGT P250 HGT P250
> > >> >> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA
SAL1L2
> > 1311
> > >> >> > -10722.64904
> > >> >> > -10722.63741 115033718.70405 115033873.35398
115033574.63798
> > 2.3311
> > >> >> >
> > >> >> >
> > >> >> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400
FV3GFS_MALLORYTEST
> > G2/PNA
> > >> >> SAL1L2
> > >> >> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00
0.487286476E+00
> > >> >> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
> > >> >> >
> > >> >> > Mallory
> > >> >> >
> > >> >> >
> > >> >>
> > >> >>
> > >>
> > >>
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: Julie Prestopnik
Time: Mon Jan 28 10:30:05 2019
Actually, I have not done that yet, but am hoping to soon.
Julie
On Mon, Jan 28, 2019 at 10:24 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
>
> Mallory,
>
> Yes, it's posted as a patch for met-8.0 dated 01/04/2019 and
described
> here:
>
https://dtcenter.org/met/users/support/known_issues/METv8.0/index.php
>
> Basically, grid-stat wasn't applying the fourier decomposition to
the climo
> data and it's pretty clear that the VSDB code was. Julie has
recompiled
> with the patch on NOAA machines but I'm not exactly sure where.
> Try running a MET tool from the build you're using with the "--
version"
> command line option. That should tell you the latest patch level...
and
> should look something like this:
>
>
>
>
>
>
> *[johnhg at dakota]% met-8.0/bin/grid_stat --versionMET Version:
> V8.0Repository:
> https://svn-met-dev.cgd.ucar.edu/branches/met/met-8.0_bugfix
> <https://svn-met-dev.cgd.ucar.edu/branches/met/met-
8.0_bugfix>Revision:
> 5953Change Date: 2019-01-04 14:01:21 -0700 (Fri, 04 Jan 2019)*
>
>
> On Mon, Jan 28, 2019 at 7:35 AM Mallory Row - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> >
> > Hi John,
> >
> > Thanks for working on this. I think I saw in another e-mail chain
that
> this
> > is included as a patch for 8.0?
> >
> > Mallory
> >
> > On Thu, Jan 3, 2019 at 6:53 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Mallory,
> > >
> > > FYI, after making that change... applying the fourier decomp to
the
> climo
> > > mean field, I'm able to get the SAL1L2 values much closer to the
VSDB
> > ones:
> > > *HGT P250 fv3gfs_mallorytest PNA WV1_10-20 SAL1L2
1311
> > > 0.48839 0.47266 437.76605 447.04559 440.30188 2.48062*
> > >
> > > They are not identical to what you sent, but definitely the
correct
> > > magnitude.
> > >
> > > John
> > >
> > >
> > > On Thu, Jan 3, 2019 at 4:29 PM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> > >
> > > > Mallory,
> > > >
> > > > Thanks for setting up the sample data directory on theia. I
pulled
> > down
> > > > the files, reran it here, and was able to exactly replicate
the .stat
> > > file
> > > > you included in that directory.
> > > >
> > > > I suspect the discrepancy in scores is due to a difference in
how MET
> > and
> > > > VSDB are computing them.
> > > >
> > > > MET is applying the fourier decomposition to the forecast and
> > observation
> > > > fields, but *NOT* the climatology fields.
> > > >
> > > > However, it looks like VSDB is applying the decomposition to
the
> > > > climatology data as well:
> > > > FROM:
> *verf_g2g.v3.0.11/sorc/verf_g2g_grid2grid_grib2.fd/grid2grid.f*
> > > > 1147 *call
> > > > FITWAV_1D(clim2d,imax(1),jmax(1),wv1(nvr),wv2(nvr))*
> > > >
> > > > This difference explains the behavior we're seeing. At waves
4-9 and
> > > > 10-20, the decomposed forecast and observation values are very
close
> to
> > > > zero. When we subtract off the climo values (which are on
average
> > about
> > > > 10599), we get anomaly values of -10599 or so. I was
computing
> things
> > > > "manually" in R and couldn't get the numbers to match up.
Turns out
> it
> > > was
> > > > the Cosine Latitude weighting that was confusing me.
> > > >
> > > > So it seems to me that we should enhance MET to decompose the
> > climatology
> > > > mean fields as well.
> > > >
> > > > But this gets a little confusing. Am I correct in thinking
that you
> > > > wouldn't ever compute the fourier decomposition *AND* apply
binned
> > > > climatologies at the same time? It sounds like we should
decompose
> the
> > > > climo mean and not actually use the climo standard deviations
in any
> > way.
> > > >
> > > > Unfortunately, this change will require a patch for met-8.0.
Or we
> > could
> > > > just fix it in met-8.1.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > >
> > > > On Wed, Dec 19, 2018 at 11:39 AM Mallory Row - NOAA Affiliate
via RT
> <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158
>
> > > >>
> > > >> I did a dive more into this and decided to plot the Anomaly
> > Correlation,
> > > >> and MET and VSDB values are pretty similar. Not sure exactly
what
> this
> > > all
> > > >> means... maybe there is some scaling factor in VSDB I am not
aware
> of?
> > > >>
> > > >> Mallory
> > > >>
> > > >> On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA Affiliate
<
> > > >> mallory.row at noaa.gov> wrote:
> > > >>
> > > >> > Hi John,
> > > >> >
> > > >> > That would make sense. I set up a directory on Theia
> > > >> > at
> > > >>
> >
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
I
> > > >> > can move copy on to WCOSS if wanted too. Thanks for all the
help.
> > > Enjoy
> > > >> > your two weeks off, John (hopefully you don't see this
until after
> > you
> > > >> > get back!).
> > > >> >
> > > >> > Mallory
> > > >> >
> > > >> > On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT <
> > > >> > met_help at ucar.edu> wrote:
> > > >> >
> > > >> >> Mallory,
> > > >> >>
> > > >> >> Thanks for pointing this out. I certainly agree that
those
> SAL1L2
> > > >> numbers
> > > >> >> look very off! When I see output values from MET that are
> anywhere
> > > >> close
> > > >> >> to -9999 (like -10722), the first thing I check is to see
if
> > > somewhere
> > > >> MET
> > > >> >> is using the bad data value of -9999 in computing
statistics.
> For
> > > >> >> example,
> > > >> >> since this is SAL1L2, it should be computing FCST - CLIMO.
If
> the
> > > >> >> forecast
> > > >> >> value is bad data (-9999) and the climo value is something
like
> > 723,
> > > >> then
> > > >> >> we'd get -9999 - 723 = -10722.
> > > >> >>
> > > >> >> Obviously this should *not* be happening. The easiest way
to
> debug
> > > >> would
> > > >> >> be getting a very simple test case which replicates this
> behavior.
> > > Can
> > > >> >> you
> > > >> >> please send (or point me to) sample fcst, obs, and climo
data for
> > > >> >> Grid-Stat, along with the Grid-Stat config file you're
using?
> > > >> >>
> > > >> >> FYI, I'll be out of the office for the next two weeks.
But if
> you
> > > >> respond
> > > >> >> to this email, other staff, like Julie, Minna, Howard, and
Randy
> > will
> > > >> get
> > > >> >> a
> > > >> >> copy and could try to replicate/fix this behavior.
> > > >> >>
> > > >> >> Thanks,
> > > >> >> John
> > > >> >>
> > > >> >> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA
Affiliate via
> > RT <
> > > >> >> met_help at ucar.edu> wrote:
> > > >> >>
> > > >> >> >
> > > >> >> > Fri Dec 14 06:07:26 2018: Request 88158 was acted upon.
> > > >> >> > Transaction: Ticket created by mallory.row at noaa.gov
> > > >> >> > Queue: met_help
> > > >> >> > Subject: grid_stat Fourier Decomposition
> > > >> >> > Owner: Nobody
> > > >> >> > Requestors: mallory.row at noaa.gov
> > > >> >> > Status: new
> > > >> >> > Ticket <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> > > >> >> >
> > > >> >> >
> > > >> >> > Hello,
> > > >> >> >
> > > >> >> > E-mailing about a possible issue I found in the Fourier
> > > decomposition
> > > >> >> > method using grid_stat to compute SAL1L2 for heights.
Below is
> an
> > > >> output
> > > >> >> > line from MET and the same output for VSDB. Attached is
an
> image
> > > >> showing
> > > >> >> > the same thing, but for a few days. Happy to provide an
> > assistance
> > > to
> > > >> >> help
> > > >> >> > solve the issue!
> > > >> >> >
> > > >> >> > METplus: V6.1 fv3gfs_mallorytest NA 240000
20170804_000000
> > > >> >> 20170804_000000
> > > >> >> > 000000 20170804_000000 20170804_000000 HGT P250 HGT
P250
> > > >> >> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA NA
SAL1L2
> > > 1311
> > > >> >> > -10722.64904
> > > >> >> > -10722.63741 115033718.70405 115033873.35398
115033574.63798
> > > 2.3311
> > > >> >> >
> > > >> >> >
> > > >> >> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400
FV3GFS_MALLORYTEST
> > > G2/PNA
> > > >> >> SAL1L2
> > > >> >> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00
0.487286476E+00
> > > >> >> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
> > > >> >> >
> > > >> >> > Mallory
> > > >> >> >
> > > >> >> >
> > > >> >>
> > > >> >>
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: grid_stat Fourier Decomposition
From: Mallory Row - NOAA Affiliate
Time: Mon Jan 28 10:38:25 2019
Awesome! Thanks for the update and finding out what the issue was!
Feel
free to close out the ticket.
Mallory
On Mon, Jan 28, 2019 at 12:30 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Actually, I have not done that yet, but am hoping to soon.
>
> Julie
>
> On Mon, Jan 28, 2019 at 10:24 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> >
> > Mallory,
> >
> > Yes, it's posted as a patch for met-8.0 dated 01/04/2019 and
described
> > here:
> >
https://dtcenter.org/met/users/support/known_issues/METv8.0/index.php
> >
> > Basically, grid-stat wasn't applying the fourier decomposition to
the
> climo
> > data and it's pretty clear that the VSDB code was. Julie has
recompiled
> > with the patch on NOAA machines but I'm not exactly sure where.
> > Try running a MET tool from the build you're using with the "--
version"
> > command line option. That should tell you the latest patch
level... and
> > should look something like this:
> >
> >
> >
> >
> >
> >
> > *[johnhg at dakota]% met-8.0/bin/grid_stat --versionMET Version:
> > V8.0Repository:
> > https://svn-met-dev.cgd.ucar.edu/branches/met/met-8.0_bugfix
> > <https://svn-met-dev.cgd.ucar.edu/branches/met/met-
8.0_bugfix>Revision:
> > 5953Change Date: 2019-01-04 14:01:21 -0700 (Fri, 04 Jan 2019)*
> >
> >
> > On Mon, Jan 28, 2019 at 7:35 AM Mallory Row - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> > >
> > > Hi John,
> > >
> > > Thanks for working on this. I think I saw in another e-mail
chain that
> > this
> > > is included as a patch for 8.0?
> > >
> > > Mallory
> > >
> > > On Thu, Jan 3, 2019 at 6:53 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Mallory,
> > > >
> > > > FYI, after making that change... applying the fourier decomp
to the
> > climo
> > > > mean field, I'm able to get the SAL1L2 values much closer to
the VSDB
> > > ones:
> > > > *HGT P250 fv3gfs_mallorytest PNA WV1_10-20 SAL1L2
> 1311
> > > > 0.48839 0.47266 437.76605 447.04559 440.30188 2.48062*
> > > >
> > > > They are not identical to what you sent, but definitely the
correct
> > > > magnitude.
> > > >
> > > > John
> > > >
> > > >
> > > > On Thu, Jan 3, 2019 at 4:29 PM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > > >
> > > > > Mallory,
> > > > >
> > > > > Thanks for setting up the sample data directory on theia. I
pulled
> > > down
> > > > > the files, reran it here, and was able to exactly replicate
the
> .stat
> > > > file
> > > > > you included in that directory.
> > > > >
> > > > > I suspect the discrepancy in scores is due to a difference
in how
> MET
> > > and
> > > > > VSDB are computing them.
> > > > >
> > > > > MET is applying the fourier decomposition to the forecast
and
> > > observation
> > > > > fields, but *NOT* the climatology fields.
> > > > >
> > > > > However, it looks like VSDB is applying the decomposition to
the
> > > > > climatology data as well:
> > > > > FROM:
> > *verf_g2g.v3.0.11/sorc/verf_g2g_grid2grid_grib2.fd/grid2grid.f*
> > > > > 1147 *call
> > > > > FITWAV_1D(clim2d,imax(1),jmax(1),wv1(nvr),wv2(nvr))*
> > > > >
> > > > > This difference explains the behavior we're seeing. At
waves 4-9
> and
> > > > > 10-20, the decomposed forecast and observation values are
very
> close
> > to
> > > > > zero. When we subtract off the climo values (which are on
average
> > > about
> > > > > 10599), we get anomaly values of -10599 or so. I was
computing
> > things
> > > > > "manually" in R and couldn't get the numbers to match up.
Turns
> out
> > it
> > > > was
> > > > > the Cosine Latitude weighting that was confusing me.
> > > > >
> > > > > So it seems to me that we should enhance MET to decompose
the
> > > climatology
> > > > > mean fields as well.
> > > > >
> > > > > But this gets a little confusing. Am I correct in thinking
that
> you
> > > > > wouldn't ever compute the fourier decomposition *AND* apply
binned
> > > > > climatologies at the same time? It sounds like we should
decompose
> > the
> > > > > climo mean and not actually use the climo standard
deviations in
> any
> > > way.
> > > > >
> > > > > Unfortunately, this change will require a patch for met-8.0.
Or we
> > > could
> > > > > just fix it in met-8.1.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Dec 19, 2018 at 11:39 AM Mallory Row - NOAA
Affiliate via
> RT
> > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> > > > >>
> > > > >> I did a dive more into this and decided to plot the Anomaly
> > > Correlation,
> > > > >> and MET and VSDB values are pretty similar. Not sure
exactly what
> > this
> > > > all
> > > > >> means... maybe there is some scaling factor in VSDB I am
not aware
> > of?
> > > > >>
> > > > >> Mallory
> > > > >>
> > > > >> On Mon, Dec 17, 2018 at 8:29 AM Mallory Row - NOAA
Affiliate <
> > > > >> mallory.row at noaa.gov> wrote:
> > > > >>
> > > > >> > Hi John,
> > > > >> >
> > > > >> > That would make sense. I set up a directory on Theia
> > > > >> > at
> > > > >>
> > >
/scratch4/NCEPDEV/global/save/Mallory.Row/VRFY/test_grid_stat_fourier.
> I
> > > > >> > can move copy on to WCOSS if wanted too. Thanks for all
the
> help.
> > > > Enjoy
> > > > >> > your two weeks off, John (hopefully you don't see this
until
> after
> > > you
> > > > >> > get back!).
> > > > >> >
> > > > >> > Mallory
> > > > >> >
> > > > >> > On Fri, Dec 14, 2018 at 7:05 PM John Halley Gotway via RT
<
> > > > >> > met_help at ucar.edu> wrote:
> > > > >> >
> > > > >> >> Mallory,
> > > > >> >>
> > > > >> >> Thanks for pointing this out. I certainly agree that
those
> > SAL1L2
> > > > >> numbers
> > > > >> >> look very off! When I see output values from MET that
are
> > anywhere
> > > > >> close
> > > > >> >> to -9999 (like -10722), the first thing I check is to
see if
> > > > somewhere
> > > > >> MET
> > > > >> >> is using the bad data value of -9999 in computing
statistics.
> > For
> > > > >> >> example,
> > > > >> >> since this is SAL1L2, it should be computing FCST -
CLIMO. If
> > the
> > > > >> >> forecast
> > > > >> >> value is bad data (-9999) and the climo value is
something like
> > > 723,
> > > > >> then
> > > > >> >> we'd get -9999 - 723 = -10722.
> > > > >> >>
> > > > >> >> Obviously this should *not* be happening. The easiest
way to
> > debug
> > > > >> would
> > > > >> >> be getting a very simple test case which replicates this
> > behavior.
> > > > Can
> > > > >> >> you
> > > > >> >> please send (or point me to) sample fcst, obs, and climo
data
> for
> > > > >> >> Grid-Stat, along with the Grid-Stat config file you're
using?
> > > > >> >>
> > > > >> >> FYI, I'll be out of the office for the next two weeks.
But if
> > you
> > > > >> respond
> > > > >> >> to this email, other staff, like Julie, Minna, Howard,
and
> Randy
> > > will
> > > > >> get
> > > > >> >> a
> > > > >> >> copy and could try to replicate/fix this behavior.
> > > > >> >>
> > > > >> >> Thanks,
> > > > >> >> John
> > > > >> >>
> > > > >> >> On Fri, Dec 14, 2018 at 6:07 AM Mallory Row - NOAA
Affiliate
> via
> > > RT <
> > > > >> >> met_help at ucar.edu> wrote:
> > > > >> >>
> > > > >> >> >
> > > > >> >> > Fri Dec 14 06:07:26 2018: Request 88158 was acted
upon.
> > > > >> >> > Transaction: Ticket created by mallory.row at noaa.gov
> > > > >> >> > Queue: met_help
> > > > >> >> > Subject: grid_stat Fourier Decomposition
> > > > >> >> > Owner: Nobody
> > > > >> >> > Requestors: mallory.row at noaa.gov
> > > > >> >> > Status: new
> > > > >> >> > Ticket <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88158 >
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > Hello,
> > > > >> >> >
> > > > >> >> > E-mailing about a possible issue I found in the
Fourier
> > > > decomposition
> > > > >> >> > method using grid_stat to compute SAL1L2 for heights.
Below
> is
> > an
> > > > >> output
> > > > >> >> > line from MET and the same output for VSDB. Attached
is an
> > image
> > > > >> showing
> > > > >> >> > the same thing, but for a few days. Happy to provide
an
> > > assistance
> > > > to
> > > > >> >> help
> > > > >> >> > solve the issue!
> > > > >> >> >
> > > > >> >> > METplus: V6.1 fv3gfs_mallorytest NA 240000
20170804_000000
> > > > >> >> 20170804_000000
> > > > >> >> > 000000 20170804_000000 20170804_000000 HGT P250 HGT
P250
> > > > >> >> > fv3gfs_mallorytest_anl PNA WV1_10-20 NA NA NA NA
NA
> SAL1L2
> > > > 1311
> > > > >> >> > -10722.64904
> > > > >> >> > -10722.63741 115033718.70405 115033873.35398
115033574.63798
> > > > 2.3311
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > VSDB: V01 FV3GFS_MALLORYTEST 24 2017080400
FV3GFS_MALLORYTEST
> > > > G2/PNA
> > > > >> >> SAL1L2
> > > > >> >> > HGT_WV1/10-20 P250 = 1311. 0.475731658E+00
> 0.487286476E+00
> > > > >> >> > 0.465863958E+03 0.471534680E+03 0.470775184E+03
> > > > >> >> >
> > > > >> >> > Mallory
> > > > >> >> >
> > > > >> >> >
> > > > >> >>
> > > > >> >>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list