[Met_help] [rt.rap.ucar.edu #98641] History for MODE Threshold Issue
John Halley Gotway via RT
met_help at ucar.edu
Wed Feb 17 10:47:39 MST 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello MET Help:
I'm having an issue with using MODE and having some objects appear much
larger than what I expect they should be. I am running MODE (v9.0) on 24 hr
snow accumulations. The first image is the 24 hr snow accumulation for
"Model A." The second image shows the MODE objects for a threshold of 1
inch. You can see the object extends too far into Indiana, Ohio, and
eastern Michigan than I would expect based on the accumulation map in the
first image. The third attachment is my configuration file that I used. The
data is being read from grib2 files.
I've done a few tests. The first test was lowering the first threshold in
the snow accumulation graphic to 0.99" (fourth attachment). You can with
that, the MODE object appears to match the light blue 0.99" area. I then
raised the threshold in my MODE code to 1.01." You can see that the object
now matches more closely with the expected 1 inch accumulation from the
first image attachment. I was wondering if you might know what is causing
this sensitivity and if there is a configuration option I could use to
mitigate it? I've seen this behavior in other models as well so I don't
believe it is input data related but haven't completely ruled that out. If
you have any thoughts or suggestions I would really appreciate it.
Thank you,
Ben Albright
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Wed Feb 10 13:25:23 2021
Hi Ben,
I see you have some questions about sensitivity of thresholds in MODE.
Thanks for sending some graphics and your MODE config file. I do have
a
couple of thoughts.
First, do you know, are the values where there is no snowfall stored
as 0's
or missing data values?
To define objects, MODE does a 2 step process... (1) smoothing the
data
based on the conv_radius setting and (2) thresholding the smoothed
data by
applying the conv_thresh setting. If the no snowfall points are stored
as
0, then all should be well. If it's stored as missing data, then that
may
wreak havoc on the smoothing step and cause unexpected results.
Next, I'd recommend testing using "conv_radius = 0". That'll
effectively
disable the smoothing step. And the objects MODE creates should
exactly
match the contours of your plots.
In general, I'd recommend against configuring MET to read data based
on a
record number:
field = {
name = "SNOD";
level = "R1";
};
That's dangerous because it'll just use whatever data is in record
number 1
regardless of whether or not it's actually SNOD data.
If you'd like me to take a closer look, please just send me (or point
me
to) the data files used for the case you sent.
Thanks,
John Halley Gotway
On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
> Transaction: Ticket created by benjamin.albright at noaa.gov
> Queue: met_help
> Subject: MODE Threshold Issue
> Owner: Nobody
> Requestors: benjamin.albright at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>
>
> Hello MET Help:
>
> I'm having an issue with using MODE and having some objects appear
much
> larger than what I expect they should be. I am running MODE (v9.0)
on 24 hr
> snow accumulations. The first image is the 24 hr snow accumulation
for
> "Model A." The second image shows the MODE objects for a threshold
of 1
> inch. You can see the object extends too far into Indiana, Ohio, and
> eastern Michigan than I would expect based on the accumulation map
in the
> first image. The third attachment is my configuration file that I
used. The
> data is being read from grib2 files.
>
> I've done a few tests. The first test was lowering the first
threshold in
> the snow accumulation graphic to 0.99" (fourth attachment). You can
with
> that, the MODE object appears to match the light blue 0.99" area. I
then
> raised the threshold in my MODE code to 1.01." You can see that the
object
> now matches more closely with the expected 1 inch accumulation from
the
> first image attachment. I was wondering if you might know what is
causing
> this sensitivity and if there is a configuration option I could use
to
> mitigate it? I've seen this behavior in other models as well so I
don't
> believe it is input data related but haven't completely ruled that
out. If
> you have any thoughts or suggestions I would really appreciate it.
>
> Thank you,
> Ben Albright
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
>
------------------------------------------------
Subject: MODE Threshold Issue
From: Benjamin Albright - NOAA Affiliate
Time: Thu Feb 11 04:27:52 2021
Hi John,
Thanks for your reply and the suggestions. As far as missing data, I'm
not
100% sure how it is stored in this particular file we are working
with. I
asked a co-worker and he said he has not seen missing values in this
particular file before and that in some of the other files we use,
missing
values are usually relegated to the edges of the grid and stored as
10^36.
Also thanks for the warning about having MODE read data via record
number.
I typically only use files that have one record to read but that is
good to
know for the future.
As for testing I tried setting a convolution radius to 0. I've
attached
that image. You can see the model object still appears too large. I've
attached the data file I am using as well as the config file again.
Please
let me know if you have any other suggestions or see anything wrong
with
the data file or config file. I appreciate the help.
Thanks,
Ben
On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Ben,
>
> I see you have some questions about sensitivity of thresholds in
MODE.
> Thanks for sending some graphics and your MODE config file. I do
have a
> couple of thoughts.
>
> First, do you know, are the values where there is no snowfall stored
as 0's
> or missing data values?
>
> To define objects, MODE does a 2 step process... (1) smoothing the
data
> based on the conv_radius setting and (2) thresholding the smoothed
data by
> applying the conv_thresh setting. If the no snowfall points are
stored as
> 0, then all should be well. If it's stored as missing data, then
that may
> wreak havoc on the smoothing step and cause unexpected results.
>
> Next, I'd recommend testing using "conv_radius = 0". That'll
effectively
> disable the smoothing step. And the objects MODE creates should
exactly
> match the contours of your plots.
>
> In general, I'd recommend against configuring MET to read data based
on a
> record number:
> field = {
> name = "SNOD";
> level = "R1";
> };
> That's dangerous because it'll just use whatever data is in record
number 1
> regardless of whether or not it's actually SNOD data.
>
> If you'd like me to take a closer look, please just send me (or
point me
> to) the data files used for the case you sent.
>
> Thanks,
> John Halley Gotway
>
> On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
> > Transaction: Ticket created by benjamin.albright at noaa.gov
> > Queue: met_help
> > Subject: MODE Threshold Issue
> > Owner: Nobody
> > Requestors: benjamin.albright at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> >
> >
> > Hello MET Help:
> >
> > I'm having an issue with using MODE and having some objects appear
much
> > larger than what I expect they should be. I am running MODE (v9.0)
on 24
> hr
> > snow accumulations. The first image is the 24 hr snow accumulation
for
> > "Model A." The second image shows the MODE objects for a threshold
of 1
> > inch. You can see the object extends too far into Indiana, Ohio,
and
> > eastern Michigan than I would expect based on the accumulation map
in the
> > first image. The third attachment is my configuration file that I
used.
> The
> > data is being read from grib2 files.
> >
> > I've done a few tests. The first test was lowering the first
threshold in
> > the snow accumulation graphic to 0.99" (fourth attachment). You
can with
> > that, the MODE object appears to match the light blue 0.99" area.
I then
> > raised the threshold in my MODE code to 1.01." You can see that
the
> object
> > now matches more closely with the expected 1 inch accumulation
from the
> > first image attachment. I was wondering if you might know what is
causing
> > this sensitivity and if there is a configuration option I could
use to
> > mitigate it? I've seen this behavior in other models as well so I
don't
> > believe it is input data related but haven't completely ruled that
out.
> If
> > you have any thoughts or suggestions I would really appreciate it.
> >
> > Thank you,
> > Ben Albright
> >
> > --
> > Benjamin Albright, Phd.
> > Meteorological Developer
> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> > Work email: benjamin.albright at noaa.gov
> > Work phone: 301-683-1474
> >
> >
>
>
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Thu Feb 11 11:45:44 2021
Ben,
When I run the sample data you sent through MET's plot_data_plane
tool, I
get a rather telling warning message:
*plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD"; level="Z0";' -v 4*
*WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET does not
currently support Lambert Conformal grids where dx (5.07862) != dy
(5.08049) and may produce unexpected results!WARNING: *
I wonder if the unexpected extent of the MODE objects is the
"*unexpected
results*" mentioned in that warning message?
In the attached image, I've placed the image from plot_data_plane
(left)
next to the image from Unidata's IDV viewer (right). Paying particular
attention to where the data lives relative to the underlying map data.
I
was expecting to see a difference in where the raw data lives between
these
two images, but I do not. The relationship of the data to the Great
Lakes
looks very consistent. So this didn't reveal an obvious problem.
But I do see some large differences when comparing to the images you
sent.
Looking in the top-left corner, both MET and IDV include roughly 90 of
Victoria Island. However the images you sent include ALL of Victoria
Island, plus more! In the bottom right corner, MET and IDV include
only the
northern half of Cuba whereas your plot includes almost all of it.
I'd recommend checking the code you're using to create those plots. Do
you
have an independent way of plotting this data that you trust to figure
out
if there's a problem in the grid or projection being used in your
plotting?
Thanks,
John
[image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>
> Hi John,
>
> Thanks for your reply and the suggestions. As far as missing data,
I'm not
> 100% sure how it is stored in this particular file we are working
with. I
> asked a co-worker and he said he has not seen missing values in this
> particular file before and that in some of the other files we use,
missing
> values are usually relegated to the edges of the grid and stored as
10^36.
> Also thanks for the warning about having MODE read data via record
number.
> I typically only use files that have one record to read but that is
good to
> know for the future.
>
> As for testing I tried setting a convolution radius to 0. I've
attached
> that image. You can see the model object still appears too large.
I've
> attached the data file I am using as well as the config file again.
Please
> let me know if you have any other suggestions or see anything wrong
with
> the data file or config file. I appreciate the help.
>
> Thanks,
> Ben
>
> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Ben,
> >
> > I see you have some questions about sensitivity of thresholds in
MODE.
> > Thanks for sending some graphics and your MODE config file. I do
have a
> > couple of thoughts.
> >
> > First, do you know, are the values where there is no snowfall
stored as
> 0's
> > or missing data values?
> >
> > To define objects, MODE does a 2 step process... (1) smoothing the
data
> > based on the conv_radius setting and (2) thresholding the smoothed
data
> by
> > applying the conv_thresh setting. If the no snowfall points are
stored as
> > 0, then all should be well. If it's stored as missing data, then
that may
> > wreak havoc on the smoothing step and cause unexpected results.
> >
> > Next, I'd recommend testing using "conv_radius = 0". That'll
effectively
> > disable the smoothing step. And the objects MODE creates should
exactly
> > match the contours of your plots.
> >
> > In general, I'd recommend against configuring MET to read data
based on a
> > record number:
> > field = {
> > name = "SNOD";
> > level = "R1";
> > };
> > That's dangerous because it'll just use whatever data is in record
> number 1
> > regardless of whether or not it's actually SNOD data.
> >
> > If you'd like me to take a closer look, please just send me (or
point me
> > to) the data files used for the case you sent.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
> > > Transaction: Ticket created by benjamin.albright at noaa.gov
> > > Queue: met_help
> > > Subject: MODE Threshold Issue
> > > Owner: Nobody
> > > Requestors: benjamin.albright at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> >
> > >
> > >
> > > Hello MET Help:
> > >
> > > I'm having an issue with using MODE and having some objects
appear much
> > > larger than what I expect they should be. I am running MODE
(v9.0) on
> 24
> > hr
> > > snow accumulations. The first image is the 24 hr snow
accumulation for
> > > "Model A." The second image shows the MODE objects for a
threshold of 1
> > > inch. You can see the object extends too far into Indiana, Ohio,
and
> > > eastern Michigan than I would expect based on the accumulation
map in
> the
> > > first image. The third attachment is my configuration file that
I used.
> > The
> > > data is being read from grib2 files.
> > >
> > > I've done a few tests. The first test was lowering the first
threshold
> in
> > > the snow accumulation graphic to 0.99" (fourth attachment). You
can
> with
> > > that, the MODE object appears to match the light blue 0.99"
area. I
> then
> > > raised the threshold in my MODE code to 1.01." You can see that
the
> > object
> > > now matches more closely with the expected 1 inch accumulation
from the
> > > first image attachment. I was wondering if you might know what
is
> causing
> > > this sensitivity and if there is a configuration option I could
use to
> > > mitigate it? I've seen this behavior in other models as well so
I don't
> > > believe it is input data related but haven't completely ruled
that out.
> > If
> > > you have any thoughts or suggestions I would really appreciate
it.
> > >
> > > Thank you,
> > > Ben Albright
> > >
> > > --
> > > Benjamin Albright, Phd.
> > > Meteorological Developer
> > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > > Work email: benjamin.albright at noaa.gov
> > > Work phone: 301-683-1474
> > >
> > >
> >
> >
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
>
------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Thu Feb 11 12:06:23 2021
Ben,
FYI, I tried one more method to plot this data. I used the
"ncl_convert2nc"
tool to dump the values to NetCDF. And then I displayed the result
using
ncview.
ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
ncview nbmv4_wwe_2021020707F053_24hr_in.nc
I believe ncview displays the data using ONLY the lat/lon values for
each
grid point... and not any projection info. Layering on the map data
produces an image consistent with IDV and MET. I'm mostly just
checking the
extent of the map data in the corners (Victoria Island, Nova Scotia,
and
Cuba).
So that also points to a potential problem in your python plotting
code.
Thanks,
John
[image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Ben,
>
> When I run the sample data you sent through MET's plot_data_plane
tool, I
> get a rather telling warning message:
>
>
> *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
> nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD"; level="Z0";' -v 4*
>
>
>
> *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET does
not
> currently support Lambert Conformal grids where dx (5.07862) != dy
> (5.08049) and may produce unexpected results!WARNING: *
>
> I wonder if the unexpected extent of the MODE objects is the
"*unexpected
> results*" mentioned in that warning message?
>
> In the attached image, I've placed the image from plot_data_plane
(left)
> next to the image from Unidata's IDV viewer (right). Paying
particular
> attention to where the data lives relative to the underlying map
data. I
> was expecting to see a difference in where the raw data lives
between these
> two images, but I do not. The relationship of the data to the Great
Lakes
> looks very consistent. So this didn't reveal an obvious problem.
>
> But I do see some large differences when comparing to the images you
sent.
> Looking in the top-left corner, both MET and IDV include roughly 90
of
> Victoria Island. However the images you sent include ALL of Victoria
> Island, plus more! In the bottom right corner, MET and IDV include
only the
> northern half of Cuba whereas your plot includes almost all of it.
>
> I'd recommend checking the code you're using to create those plots.
Do you
> have an independent way of plotting this data that you trust to
figure out
> if there's a problem in the grid or projection being used in your
plotting?
>
> Thanks,
> John
>
>
> [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
>
> On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>>
>> Hi John,
>>
>> Thanks for your reply and the suggestions. As far as missing data,
I'm not
>> 100% sure how it is stored in this particular file we are working
with. I
>> asked a co-worker and he said he has not seen missing values in
this
>> particular file before and that in some of the other files we use,
missing
>> values are usually relegated to the edges of the grid and stored as
10^36.
>> Also thanks for the warning about having MODE read data via record
number.
>> I typically only use files that have one record to read but that is
good
>> to
>> know for the future.
>>
>> As for testing I tried setting a convolution radius to 0. I've
attached
>> that image. You can see the model object still appears too large.
I've
>> attached the data file I am using as well as the config file again.
Please
>> let me know if you have any other suggestions or see anything wrong
with
>> the data file or config file. I appreciate the help.
>>
>> Thanks,
>> Ben
>>
>> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> > Hi Ben,
>> >
>> > I see you have some questions about sensitivity of thresholds in
MODE.
>> > Thanks for sending some graphics and your MODE config file. I do
have a
>> > couple of thoughts.
>> >
>> > First, do you know, are the values where there is no snowfall
stored as
>> 0's
>> > or missing data values?
>> >
>> > To define objects, MODE does a 2 step process... (1) smoothing
the data
>> > based on the conv_radius setting and (2) thresholding the
smoothed data
>> by
>> > applying the conv_thresh setting. If the no snowfall points are
stored
>> as
>> > 0, then all should be well. If it's stored as missing data, then
that
>> may
>> > wreak havoc on the smoothing step and cause unexpected results.
>> >
>> > Next, I'd recommend testing using "conv_radius = 0". That'll
effectively
>> > disable the smoothing step. And the objects MODE creates should
exactly
>> > match the contours of your plots.
>> >
>> > In general, I'd recommend against configuring MET to read data
based on
>> a
>> > record number:
>> > field = {
>> > name = "SNOD";
>> > level = "R1";
>> > };
>> > That's dangerous because it'll just use whatever data is in
record
>> number 1
>> > regardless of whether or not it's actually SNOD data.
>> >
>> > If you'd like me to take a closer look, please just send me (or
point me
>> > to) the data files used for the case you sent.
>> >
>> > Thanks,
>> > John Halley Gotway
>> >
>> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
Affiliate via
>> RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
>> > > Transaction: Ticket created by benjamin.albright at noaa.gov
>> > > Queue: met_help
>> > > Subject: MODE Threshold Issue
>> > > Owner: Nobody
>> > > Requestors: benjamin.albright at noaa.gov
>> > > Status: new
>> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>> >
>> > >
>> > >
>> > > Hello MET Help:
>> > >
>> > > I'm having an issue with using MODE and having some objects
appear
>> much
>> > > larger than what I expect they should be. I am running MODE
(v9.0) on
>> 24
>> > hr
>> > > snow accumulations. The first image is the 24 hr snow
accumulation for
>> > > "Model A." The second image shows the MODE objects for a
threshold of
>> 1
>> > > inch. You can see the object extends too far into Indiana,
Ohio, and
>> > > eastern Michigan than I would expect based on the accumulation
map in
>> the
>> > > first image. The third attachment is my configuration file that
I
>> used.
>> > The
>> > > data is being read from grib2 files.
>> > >
>> > > I've done a few tests. The first test was lowering the first
>> threshold in
>> > > the snow accumulation graphic to 0.99" (fourth attachment). You
can
>> with
>> > > that, the MODE object appears to match the light blue 0.99"
area. I
>> then
>> > > raised the threshold in my MODE code to 1.01." You can see that
the
>> > object
>> > > now matches more closely with the expected 1 inch accumulation
from
>> the
>> > > first image attachment. I was wondering if you might know what
is
>> causing
>> > > this sensitivity and if there is a configuration option I could
use to
>> > > mitigate it? I've seen this behavior in other models as well so
I
>> don't
>> > > believe it is input data related but haven't completely ruled
that
>> out.
>> > If
>> > > you have any thoughts or suggestions I would really appreciate
it.
>> > >
>> > > Thank you,
>> > > Ben Albright
>> > >
>> > > --
>> > > Benjamin Albright, Phd.
>> > > Meteorological Developer
>> > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
>> > > Work email: benjamin.albright at noaa.gov
>> > > Work phone: 301-683-1474
>> > >
>> > >
>> >
>> >
>>
>> --
>> Benjamin Albright, Phd.
>> Meteorological Developer
>> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
>> Work email: benjamin.albright at noaa.gov
>> Work phone: 301-683-1474
>>
>>
------------------------------------------------
Subject: MODE Threshold Issue
From: Benjamin Albright - NOAA Affiliate
Time: Thu Feb 11 12:28:52 2021
Hi John,
Thanks for the responses. I just want to make sure I'm clear. Is that
warning MET displays about the projection a warning about the
projection
used in the grib2 data file? In other words, when I make the grib2
file,
the LCC grid that I am using is not compatible? Or do you suspect the
problem lies in my actual plotting code? I am using basemap to set up
an
LCC projection of the map you see in my images. You may not have an
answer
but I am just trying to narrow down if I should focus on the actual
grib2
file or my Python plotting script when trying to come up with a
solution.
Thanks,
Ben
On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ben,
>
> FYI, I tried one more method to plot this data. I used the
"ncl_convert2nc"
> tool to dump the values to NetCDF. And then I displayed the result
using
> ncview.
>
> ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
> ncview nbmv4_wwe_2021020707F053_24hr_in.nc
>
> I believe ncview displays the data using ONLY the lat/lon values for
each
> grid point... and not any projection info. Layering on the map data
> produces an image consistent with IDV and MET. I'm mostly just
checking the
> extent of the map data in the corners (Victoria Island, Nova Scotia,
and
> Cuba).
>
> So that also points to a potential problem in your python plotting
code.
>
> Thanks,
> John
>
> [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
>
>
>
> On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Ben,
> >
> > When I run the sample data you sent through MET's plot_data_plane
tool, I
> > get a rather telling warning message:
> >
> >
> > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
> > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD"; level="Z0";' -v
4*
> >
> >
> >
> > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET does
not
> > currently support Lambert Conformal grids where dx (5.07862) != dy
> > (5.08049) and may produce unexpected results!WARNING: *
> >
> > I wonder if the unexpected extent of the MODE objects is the
"*unexpected
> > results*" mentioned in that warning message?
> >
> > In the attached image, I've placed the image from plot_data_plane
(left)
> > next to the image from Unidata's IDV viewer (right). Paying
particular
> > attention to where the data lives relative to the underlying map
data. I
> > was expecting to see a difference in where the raw data lives
between
> these
> > two images, but I do not. The relationship of the data to the
Great Lakes
> > looks very consistent. So this didn't reveal an obvious problem.
> >
> > But I do see some large differences when comparing to the images
you
> sent.
> > Looking in the top-left corner, both MET and IDV include roughly
90 of
> > Victoria Island. However the images you sent include ALL of
Victoria
> > Island, plus more! In the bottom right corner, MET and IDV include
only
> the
> > northern half of Cuba whereas your plot includes almost all of it.
> >
> > I'd recommend checking the code you're using to create those
plots. Do
> you
> > have an independent way of plotting this data that you trust to
figure
> out
> > if there's a problem in the grid or projection being used in your
> plotting?
> >
> > Thanks,
> > John
> >
> >
> > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
> >
> > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> >>
> >> Hi John,
> >>
> >> Thanks for your reply and the suggestions. As far as missing
data, I'm
> not
> >> 100% sure how it is stored in this particular file we are working
with.
> I
> >> asked a co-worker and he said he has not seen missing values in
this
> >> particular file before and that in some of the other files we
use,
> missing
> >> values are usually relegated to the edges of the grid and stored
as
> 10^36.
> >> Also thanks for the warning about having MODE read data via
record
> number.
> >> I typically only use files that have one record to read but that
is good
> >> to
> >> know for the future.
> >>
> >> As for testing I tried setting a convolution radius to 0. I've
attached
> >> that image. You can see the model object still appears too large.
I've
> >> attached the data file I am using as well as the config file
again.
> Please
> >> let me know if you have any other suggestions or see anything
wrong with
> >> the data file or config file. I appreciate the help.
> >>
> >> Thanks,
> >> Ben
> >>
> >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> > Hi Ben,
> >> >
> >> > I see you have some questions about sensitivity of thresholds
in MODE.
> >> > Thanks for sending some graphics and your MODE config file. I
do have
> a
> >> > couple of thoughts.
> >> >
> >> > First, do you know, are the values where there is no snowfall
stored
> as
> >> 0's
> >> > or missing data values?
> >> >
> >> > To define objects, MODE does a 2 step process... (1) smoothing
the
> data
> >> > based on the conv_radius setting and (2) thresholding the
smoothed
> data
> >> by
> >> > applying the conv_thresh setting. If the no snowfall points are
stored
> >> as
> >> > 0, then all should be well. If it's stored as missing data,
then that
> >> may
> >> > wreak havoc on the smoothing step and cause unexpected results.
> >> >
> >> > Next, I'd recommend testing using "conv_radius = 0". That'll
> effectively
> >> > disable the smoothing step. And the objects MODE creates should
> exactly
> >> > match the contours of your plots.
> >> >
> >> > In general, I'd recommend against configuring MET to read data
based
> on
> >> a
> >> > record number:
> >> > field = {
> >> > name = "SNOD";
> >> > level = "R1";
> >> > };
> >> > That's dangerous because it'll just use whatever data is in
record
> >> number 1
> >> > regardless of whether or not it's actually SNOD data.
> >> >
> >> > If you'd like me to take a closer look, please just send me (or
point
> me
> >> > to) the data files used for the case you sent.
> >> >
> >> > Thanks,
> >> > John Halley Gotway
> >> >
> >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
Affiliate via
> >> RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > >
> >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
> >> > > Transaction: Ticket created by benjamin.albright at noaa.gov
> >> > > Queue: met_help
> >> > > Subject: MODE Threshold Issue
> >> > > Owner: Nobody
> >> > > Requestors: benjamin.albright at noaa.gov
> >> > > Status: new
> >> > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> >> >
> >> > >
> >> > >
> >> > > Hello MET Help:
> >> > >
> >> > > I'm having an issue with using MODE and having some objects
appear
> >> much
> >> > > larger than what I expect they should be. I am running MODE
(v9.0)
> on
> >> 24
> >> > hr
> >> > > snow accumulations. The first image is the 24 hr snow
accumulation
> for
> >> > > "Model A." The second image shows the MODE objects for a
threshold
> of
> >> 1
> >> > > inch. You can see the object extends too far into Indiana,
Ohio, and
> >> > > eastern Michigan than I would expect based on the
accumulation map
> in
> >> the
> >> > > first image. The third attachment is my configuration file
that I
> >> used.
> >> > The
> >> > > data is being read from grib2 files.
> >> > >
> >> > > I've done a few tests. The first test was lowering the first
> >> threshold in
> >> > > the snow accumulation graphic to 0.99" (fourth attachment).
You can
> >> with
> >> > > that, the MODE object appears to match the light blue 0.99"
area. I
> >> then
> >> > > raised the threshold in my MODE code to 1.01." You can see
that the
> >> > object
> >> > > now matches more closely with the expected 1 inch
accumulation from
> >> the
> >> > > first image attachment. I was wondering if you might know
what is
> >> causing
> >> > > this sensitivity and if there is a configuration option I
could use
> to
> >> > > mitigate it? I've seen this behavior in other models as well
so I
> >> don't
> >> > > believe it is input data related but haven't completely ruled
that
> >> out.
> >> > If
> >> > > you have any thoughts or suggestions I would really
appreciate it.
> >> > >
> >> > > Thank you,
> >> > > Ben Albright
> >> > >
> >> > > --
> >> > > Benjamin Albright, Phd.
> >> > > Meteorological Developer
> >> > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> >> > > Work email: benjamin.albright at noaa.gov
> >> > > Work phone: 301-683-1474
> >> > >
> >> > >
> >> >
> >> >
> >>
> >> --
> >> Benjamin Albright, Phd.
> >> Meteorological Developer
> >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> >> Work email: benjamin.albright at noaa.gov
> >> Work phone: 301-683-1474
> >>
> >>
>
>
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Thu Feb 11 14:06:59 2021
Ben,
I suspect the real problem is in the python code you're using to plot
the
data. I'd recommend using an alternative way of visualizing GRIB2 data
that
you trust. Compare their results. Check to see if your python code is
plotting the data in the right spot on the earth... and that the
extent of
the grid is correct.
Once you're confident in the accuracy of the python plots being
created,
recheck the extent of MODE objects to see if the issue still remains.
It's
possible there's multiple things going on here.
MET does not currently support Lambert Conformal grids where the dx
and dy
values differ.
Lambert Conformal: (1073 x 689) input WE:SN output WE:SN res 8
Lat1 20.190001 Lon1 238.449996 LoV 265.000000
LatD 25.000000 Latin1 25.000000 Latin2 25.000000
LatSP -90.000000 LonSP 0.000000
North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m mode 8
It just uses the Dx value and prints the warning about Dy being
different.
Another possible explanation is that your python script is right and
MET,
NCL, and IDV are all doing it wrong. I haven't run into many Lambert
grids
where Dx != Dy.
Regardless, perhaps I should write up a GitHub issue for us to enhance
MET
to support LC grids where Dx != Dy? Is this a common grid that you
guys
use and plan to continue using in the future?
Thanks,
John
On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA Affiliate
via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>
> Hi John,
>
> Thanks for the responses. I just want to make sure I'm clear. Is
that
> warning MET displays about the projection a warning about the
projection
> used in the grib2 data file? In other words, when I make the grib2
file,
> the LCC grid that I am using is not compatible? Or do you suspect
the
> problem lies in my actual plotting code? I am using basemap to set
up an
> LCC projection of the map you see in my images. You may not have an
answer
> but I am just trying to narrow down if I should focus on the actual
grib2
> file or my Python plotting script when trying to come up with a
solution.
>
> Thanks,
> Ben
>
> On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ben,
> >
> > FYI, I tried one more method to plot this data. I used the
> "ncl_convert2nc"
> > tool to dump the values to NetCDF. And then I displayed the result
using
> > ncview.
> >
> > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
> > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
> >
> > I believe ncview displays the data using ONLY the lat/lon values
for each
> > grid point... and not any projection info. Layering on the map
data
> > produces an image consistent with IDV and MET. I'm mostly just
checking
> the
> > extent of the map data in the corners (Victoria Island, Nova
Scotia, and
> > Cuba).
> >
> > So that also points to a potential problem in your python plotting
code.
> >
> > Thanks,
> > John
> >
> > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
> >
> >
> >
> > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway
<johnhg at ucar.edu>
> > wrote:
> >
> > > Ben,
> > >
> > > When I run the sample data you sent through MET's
plot_data_plane
> tool, I
> > > get a rather telling warning message:
> > >
> > >
> > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
> > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD"; level="Z0";'
-v 4*
> > >
> > >
> > >
> > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET
does not
> > > currently support Lambert Conformal grids where dx (5.07862) !=
dy
> > > (5.08049) and may produce unexpected results!WARNING: *
> > >
> > > I wonder if the unexpected extent of the MODE objects is the
> "*unexpected
> > > results*" mentioned in that warning message?
> > >
> > > In the attached image, I've placed the image from
plot_data_plane
> (left)
> > > next to the image from Unidata's IDV viewer (right). Paying
particular
> > > attention to where the data lives relative to the underlying map
data.
> I
> > > was expecting to see a difference in where the raw data lives
between
> > these
> > > two images, but I do not. The relationship of the data to the
Great
> Lakes
> > > looks very consistent. So this didn't reveal an obvious problem.
> > >
> > > But I do see some large differences when comparing to the images
you
> > sent.
> > > Looking in the top-left corner, both MET and IDV include roughly
90 of
> > > Victoria Island. However the images you sent include ALL of
Victoria
> > > Island, plus more! In the bottom right corner, MET and IDV
include only
> > the
> > > northern half of Cuba whereas your plot includes almost all of
it.
> > >
> > > I'd recommend checking the code you're using to create those
plots. Do
> > you
> > > have an independent way of plotting this data that you trust to
figure
> > out
> > > if there's a problem in the grid or projection being used in
your
> > plotting?
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
> > >
> > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
Affiliate via
> > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> > >>
> > >> Hi John,
> > >>
> > >> Thanks for your reply and the suggestions. As far as missing
data, I'm
> > not
> > >> 100% sure how it is stored in this particular file we are
working
> with.
> > I
> > >> asked a co-worker and he said he has not seen missing values in
this
> > >> particular file before and that in some of the other files we
use,
> > missing
> > >> values are usually relegated to the edges of the grid and
stored as
> > 10^36.
> > >> Also thanks for the warning about having MODE read data via
record
> > number.
> > >> I typically only use files that have one record to read but
that is
> good
> > >> to
> > >> know for the future.
> > >>
> > >> As for testing I tried setting a convolution radius to 0. I've
> attached
> > >> that image. You can see the model object still appears too
large. I've
> > >> attached the data file I am using as well as the config file
again.
> > Please
> > >> let me know if you have any other suggestions or see anything
wrong
> with
> > >> the data file or config file. I appreciate the help.
> > >>
> > >> Thanks,
> > >> Ben
> > >>
> > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
> > >> met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> > Hi Ben,
> > >> >
> > >> > I see you have some questions about sensitivity of thresholds
in
> MODE.
> > >> > Thanks for sending some graphics and your MODE config file. I
do
> have
> > a
> > >> > couple of thoughts.
> > >> >
> > >> > First, do you know, are the values where there is no snowfall
stored
> > as
> > >> 0's
> > >> > or missing data values?
> > >> >
> > >> > To define objects, MODE does a 2 step process... (1)
smoothing the
> > data
> > >> > based on the conv_radius setting and (2) thresholding the
smoothed
> > data
> > >> by
> > >> > applying the conv_thresh setting. If the no snowfall points
are
> stored
> > >> as
> > >> > 0, then all should be well. If it's stored as missing data,
then
> that
> > >> may
> > >> > wreak havoc on the smoothing step and cause unexpected
results.
> > >> >
> > >> > Next, I'd recommend testing using "conv_radius = 0". That'll
> > effectively
> > >> > disable the smoothing step. And the objects MODE creates
should
> > exactly
> > >> > match the contours of your plots.
> > >> >
> > >> > In general, I'd recommend against configuring MET to read
data based
> > on
> > >> a
> > >> > record number:
> > >> > field = {
> > >> > name = "SNOD";
> > >> > level = "R1";
> > >> > };
> > >> > That's dangerous because it'll just use whatever data is in
record
> > >> number 1
> > >> > regardless of whether or not it's actually SNOD data.
> > >> >
> > >> > If you'd like me to take a closer look, please just send me
(or
> point
> > me
> > >> > to) the data files used for the case you sent.
> > >> >
> > >> > Thanks,
> > >> > John Halley Gotway
> > >> >
> > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
Affiliate
> via
> > >> RT <
> > >> > met_help at ucar.edu> wrote:
> > >> >
> > >> > >
> > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
> > >> > > Transaction: Ticket created by benjamin.albright at noaa.gov
> > >> > > Queue: met_help
> > >> > > Subject: MODE Threshold Issue
> > >> > > Owner: Nobody
> > >> > > Requestors: benjamin.albright at noaa.gov
> > >> > > Status: new
> > >> > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> > >> >
> > >> > >
> > >> > >
> > >> > > Hello MET Help:
> > >> > >
> > >> > > I'm having an issue with using MODE and having some objects
appear
> > >> much
> > >> > > larger than what I expect they should be. I am running MODE
(v9.0)
> > on
> > >> 24
> > >> > hr
> > >> > > snow accumulations. The first image is the 24 hr snow
accumulation
> > for
> > >> > > "Model A." The second image shows the MODE objects for a
threshold
> > of
> > >> 1
> > >> > > inch. You can see the object extends too far into Indiana,
Ohio,
> and
> > >> > > eastern Michigan than I would expect based on the
accumulation map
> > in
> > >> the
> > >> > > first image. The third attachment is my configuration file
that I
> > >> used.
> > >> > The
> > >> > > data is being read from grib2 files.
> > >> > >
> > >> > > I've done a few tests. The first test was lowering the
first
> > >> threshold in
> > >> > > the snow accumulation graphic to 0.99" (fourth attachment).
You
> can
> > >> with
> > >> > > that, the MODE object appears to match the light blue 0.99"
area.
> I
> > >> then
> > >> > > raised the threshold in my MODE code to 1.01." You can see
that
> the
> > >> > object
> > >> > > now matches more closely with the expected 1 inch
accumulation
> from
> > >> the
> > >> > > first image attachment. I was wondering if you might know
what is
> > >> causing
> > >> > > this sensitivity and if there is a configuration option I
could
> use
> > to
> > >> > > mitigate it? I've seen this behavior in other models as
well so I
> > >> don't
> > >> > > believe it is input data related but haven't completely
ruled that
> > >> out.
> > >> > If
> > >> > > you have any thoughts or suggestions I would really
appreciate it.
> > >> > >
> > >> > > Thank you,
> > >> > > Ben Albright
> > >> > >
> > >> > > --
> > >> > > Benjamin Albright, Phd.
> > >> > > Meteorological Developer
> > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > >> > > Work email: benjamin.albright at noaa.gov
> > >> > > Work phone: 301-683-1474
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >> --
> > >> Benjamin Albright, Phd.
> > >> Meteorological Developer
> > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > >> Work email: benjamin.albright at noaa.gov
> > >> Work phone: 301-683-1474
> > >>
> > >>
> >
> >
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
>
------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Thu Feb 11 14:41:05 2021
Ben,
I wrote up this GitHub issue to support Dx != Dy:
https://github.com/dtcenter/MET/issues/1661
John
On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway <johnhg at ucar.edu>
wrote:
> Ben,
>
> I suspect the real problem is in the python code you're using to
plot the
> data. I'd recommend using an alternative way of visualizing GRIB2
data that
> you trust. Compare their results. Check to see if your python code
is
> plotting the data in the right spot on the earth... and that the
extent of
> the grid is correct.
>
> Once you're confident in the accuracy of the python plots being
created,
> recheck the extent of MODE objects to see if the issue still
remains. It's
> possible there's multiple things going on here.
>
> MET does not currently support Lambert Conformal grids where the dx
and dy
> values differ.
> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN res 8
> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> LatSP -90.000000 LonSP 0.000000
> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m mode
8
>
> It just uses the Dx value and prints the warning about Dy being
different.
>
> Another possible explanation is that your python script is right and
MET,
> NCL, and IDV are all doing it wrong. I haven't run into many Lambert
grids
> where Dx != Dy.
>
> Regardless, perhaps I should write up a GitHub issue for us to
enhance MET
> to support LC grids where Dx != Dy? Is this a common grid that you
guys
> use and plan to continue using in the future?
>
> Thanks,
> John
>
> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA Affiliate
via RT
> <met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>>
>> Hi John,
>>
>> Thanks for the responses. I just want to make sure I'm clear. Is
that
>> warning MET displays about the projection a warning about the
projection
>> used in the grib2 data file? In other words, when I make the grib2
file,
>> the LCC grid that I am using is not compatible? Or do you suspect
the
>> problem lies in my actual plotting code? I am using basemap to set
up an
>> LCC projection of the map you see in my images. You may not have an
answer
>> but I am just trying to narrow down if I should focus on the actual
grib2
>> file or my Python plotting script when trying to come up with a
solution.
>>
>> Thanks,
>> Ben
>>
>> On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> > Ben,
>> >
>> > FYI, I tried one more method to plot this data. I used the
>> "ncl_convert2nc"
>> > tool to dump the values to NetCDF. And then I displayed the
result using
>> > ncview.
>> >
>> > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
>> > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
>> >
>> > I believe ncview displays the data using ONLY the lat/lon values
for
>> each
>> > grid point... and not any projection info. Layering on the map
data
>> > produces an image consistent with IDV and MET. I'm mostly just
checking
>> the
>> > extent of the map data in the corners (Victoria Island, Nova
Scotia, and
>> > Cuba).
>> >
>> > So that also points to a potential problem in your python
plotting code.
>> >
>> > Thanks,
>> > John
>> >
>> > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
>> >
>> >
>> >
>> > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway
<johnhg at ucar.edu>
>> > wrote:
>> >
>> > > Ben,
>> > >
>> > > When I run the sample data you sent through MET's
plot_data_plane
>> tool, I
>> > > get a rather telling warning message:
>> > >
>> > >
>> > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
>> > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD"; level="Z0";'
-v 4*
>> > >
>> > >
>> > >
>> > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET
does not
>> > > currently support Lambert Conformal grids where dx (5.07862) !=
dy
>> > > (5.08049) and may produce unexpected results!WARNING: *
>> > >
>> > > I wonder if the unexpected extent of the MODE objects is the
>> "*unexpected
>> > > results*" mentioned in that warning message?
>> > >
>> > > In the attached image, I've placed the image from
plot_data_plane
>> (left)
>> > > next to the image from Unidata's IDV viewer (right). Paying
particular
>> > > attention to where the data lives relative to the underlying
map
>> data. I
>> > > was expecting to see a difference in where the raw data lives
between
>> > these
>> > > two images, but I do not. The relationship of the data to the
Great
>> Lakes
>> > > looks very consistent. So this didn't reveal an obvious
problem.
>> > >
>> > > But I do see some large differences when comparing to the
images you
>> > sent.
>> > > Looking in the top-left corner, both MET and IDV include
roughly 90 of
>> > > Victoria Island. However the images you sent include ALL of
Victoria
>> > > Island, plus more! In the bottom right corner, MET and IDV
include
>> only
>> > the
>> > > northern half of Cuba whereas your plot includes almost all of
it.
>> > >
>> > > I'd recommend checking the code you're using to create those
plots. Do
>> > you
>> > > have an independent way of plotting this data that you trust to
figure
>> > out
>> > > if there's a problem in the grid or projection being used in
your
>> > plotting?
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > >
>> > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
>> > >
>> > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
Affiliate via
>> > RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > >>
>> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>
>> > >>
>> > >> Hi John,
>> > >>
>> > >> Thanks for your reply and the suggestions. As far as missing
data,
>> I'm
>> > not
>> > >> 100% sure how it is stored in this particular file we are
working
>> with.
>> > I
>> > >> asked a co-worker and he said he has not seen missing values
in this
>> > >> particular file before and that in some of the other files we
use,
>> > missing
>> > >> values are usually relegated to the edges of the grid and
stored as
>> > 10^36.
>> > >> Also thanks for the warning about having MODE read data via
record
>> > number.
>> > >> I typically only use files that have one record to read but
that is
>> good
>> > >> to
>> > >> know for the future.
>> > >>
>> > >> As for testing I tried setting a convolution radius to 0. I've
>> attached
>> > >> that image. You can see the model object still appears too
large.
>> I've
>> > >> attached the data file I am using as well as the config file
again.
>> > Please
>> > >> let me know if you have any other suggestions or see anything
wrong
>> with
>> > >> the data file or config file. I appreciate the help.
>> > >>
>> > >> Thanks,
>> > >> Ben
>> > >>
>> > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
>> > >> met_help at ucar.edu>
>> > >> wrote:
>> > >>
>> > >> > Hi Ben,
>> > >> >
>> > >> > I see you have some questions about sensitivity of
thresholds in
>> MODE.
>> > >> > Thanks for sending some graphics and your MODE config file.
I do
>> have
>> > a
>> > >> > couple of thoughts.
>> > >> >
>> > >> > First, do you know, are the values where there is no
snowfall
>> stored
>> > as
>> > >> 0's
>> > >> > or missing data values?
>> > >> >
>> > >> > To define objects, MODE does a 2 step process... (1)
smoothing the
>> > data
>> > >> > based on the conv_radius setting and (2) thresholding the
smoothed
>> > data
>> > >> by
>> > >> > applying the conv_thresh setting. If the no snowfall points
are
>> stored
>> > >> as
>> > >> > 0, then all should be well. If it's stored as missing data,
then
>> that
>> > >> may
>> > >> > wreak havoc on the smoothing step and cause unexpected
results.
>> > >> >
>> > >> > Next, I'd recommend testing using "conv_radius = 0". That'll
>> > effectively
>> > >> > disable the smoothing step. And the objects MODE creates
should
>> > exactly
>> > >> > match the contours of your plots.
>> > >> >
>> > >> > In general, I'd recommend against configuring MET to read
data
>> based
>> > on
>> > >> a
>> > >> > record number:
>> > >> > field = {
>> > >> > name = "SNOD";
>> > >> > level = "R1";
>> > >> > };
>> > >> > That's dangerous because it'll just use whatever data is in
record
>> > >> number 1
>> > >> > regardless of whether or not it's actually SNOD data.
>> > >> >
>> > >> > If you'd like me to take a closer look, please just send me
(or
>> point
>> > me
>> > >> > to) the data files used for the case you sent.
>> > >> >
>> > >> > Thanks,
>> > >> > John Halley Gotway
>> > >> >
>> > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
Affiliate
>> via
>> > >> RT <
>> > >> > met_help at ucar.edu> wrote:
>> > >> >
>> > >> > >
>> > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
>> > >> > > Transaction: Ticket created by benjamin.albright at noaa.gov
>> > >> > > Queue: met_help
>> > >> > > Subject: MODE Threshold Issue
>> > >> > > Owner: Nobody
>> > >> > > Requestors: benjamin.albright at noaa.gov
>> > >> > > Status: new
>> > >> > > Ticket <URL:
>> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>> > >> >
>> > >> > >
>> > >> > >
>> > >> > > Hello MET Help:
>> > >> > >
>> > >> > > I'm having an issue with using MODE and having some
objects
>> appear
>> > >> much
>> > >> > > larger than what I expect they should be. I am running
MODE
>> (v9.0)
>> > on
>> > >> 24
>> > >> > hr
>> > >> > > snow accumulations. The first image is the 24 hr snow
>> accumulation
>> > for
>> > >> > > "Model A." The second image shows the MODE objects for a
>> threshold
>> > of
>> > >> 1
>> > >> > > inch. You can see the object extends too far into Indiana,
Ohio,
>> and
>> > >> > > eastern Michigan than I would expect based on the
accumulation
>> map
>> > in
>> > >> the
>> > >> > > first image. The third attachment is my configuration file
that I
>> > >> used.
>> > >> > The
>> > >> > > data is being read from grib2 files.
>> > >> > >
>> > >> > > I've done a few tests. The first test was lowering the
first
>> > >> threshold in
>> > >> > > the snow accumulation graphic to 0.99" (fourth
attachment). You
>> can
>> > >> with
>> > >> > > that, the MODE object appears to match the light blue
0.99"
>> area. I
>> > >> then
>> > >> > > raised the threshold in my MODE code to 1.01." You can see
that
>> the
>> > >> > object
>> > >> > > now matches more closely with the expected 1 inch
accumulation
>> from
>> > >> the
>> > >> > > first image attachment. I was wondering if you might know
what is
>> > >> causing
>> > >> > > this sensitivity and if there is a configuration option I
could
>> use
>> > to
>> > >> > > mitigate it? I've seen this behavior in other models as
well so I
>> > >> don't
>> > >> > > believe it is input data related but haven't completely
ruled
>> that
>> > >> out.
>> > >> > If
>> > >> > > you have any thoughts or suggestions I would really
appreciate
>> it.
>> > >> > >
>> > >> > > Thank you,
>> > >> > > Ben Albright
>> > >> > >
>> > >> > > --
>> > >> > > Benjamin Albright, Phd.
>> > >> > > Meteorological Developer
>> > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
>> Center
>> > >> > > Work email: benjamin.albright at noaa.gov
>> > >> > > Work phone: 301-683-1474
>> > >> > >
>> > >> > >
>> > >> >
>> > >> >
>> > >>
>> > >> --
>> > >> Benjamin Albright, Phd.
>> > >> Meteorological Developer
>> > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
>> > >> Work email: benjamin.albright at noaa.gov
>> > >> Work phone: 301-683-1474
>> > >>
>> > >>
>> >
>> >
>>
>> --
>> Benjamin Albright, Phd.
>> Meteorological Developer
>> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
>> Work email: benjamin.albright at noaa.gov
>> Work phone: 301-683-1474
>>
>>
------------------------------------------------
Subject: MODE Threshold Issue
From: Benjamin Albright - NOAA Affiliate
Time: Fri Feb 12 04:55:55 2021
Hi John
Thanks for the replies. I was able to go back to where I am making the
files and make a couple changes in order to get the final files to
have DX
= DY.
*New model file:* wgrib2 -V nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
1:0:vt=2021020712:surface - surface:53 hour fcst:SNOD Snow Depth [m]:
ndata=3744965:undef=0:mean=0.51457:min=0:max=36
grid_template=30:winds(grid):
Lambert Conformal: (2345 x 1597) input WE:SN output WE:SN res 8
Lat1 19.229000 Lon1 233.723396 LoV 265.000000
LatD 25.000000 Latin1 25.000000 Latin2 25.000000
LatSP -90.000000 LonSP 0.000000
North Pole (2345 x 1597) Dx 2539.703000 m Dy 2539.703000 m mode 8
*New verification file:* wgrib2 -V nohrsc_2021020712_in.grib2
1:0:vt=2021020712:surface - surface:anl:SNOD Snow Depth [m]:
ndata=2953665:undef=1654866:mean=0.514013:min=0:max=53
grid_template=30:winds(grid):
Lambert Conformal: (2145 x 1377) input WE:SN output WE:SN res 8
Lat1 20.191999 Lon1 238.445999 LoV 265.000000
LatD 25.000000 Latin1 25.000000 Latin2 25.000000
LatSP -90.000000 LonSP 0.000000
North Pole (2145 x 1377) Dx 2539.703000 m Dy 2539.703000 m mode 8
This removed the warning when running MODE about DX != DY. However,
when
running MODE on this I still am getting the same result for the one
inch
threshold where the one inch object area is much larger. I've attached
the
two new data files, the image, and config file.
I haven't had a chance yet to go back and closely check my python
plotting.
I am using basemap in python to generate the images. I'll keep
investigating but just wanted to let you know about this new test I
tried
this morning.
Thank you,
Ben
On Thu, Feb 11, 2021 at 4:07 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ben,
>
> I suspect the real problem is in the python code you're using to
plot the
> data. I'd recommend using an alternative way of visualizing GRIB2
data that
> you trust. Compare their results. Check to see if your python code
is
> plotting the data in the right spot on the earth... and that the
extent of
> the grid is correct.
>
> Once you're confident in the accuracy of the python plots being
created,
> recheck the extent of MODE objects to see if the issue still
remains. It's
> possible there's multiple things going on here.
>
> MET does not currently support Lambert Conformal grids where the dx
and dy
> values differ.
> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN res 8
> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> LatSP -90.000000 LonSP 0.000000
> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m mode
8
>
> It just uses the Dx value and prints the warning about Dy being
different.
>
> Another possible explanation is that your python script is right and
MET,
> NCL, and IDV are all doing it wrong. I haven't run into many Lambert
grids
> where Dx != Dy.
>
> Regardless, perhaps I should write up a GitHub issue for us to
enhance MET
> to support LC grids where Dx != Dy? Is this a common grid that you
guys
> use and plan to continue using in the future?
>
> Thanks,
> John
>
> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA Affiliate
via RT
> <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> >
> > Hi John,
> >
> > Thanks for the responses. I just want to make sure I'm clear. Is
that
> > warning MET displays about the projection a warning about the
projection
> > used in the grib2 data file? In other words, when I make the grib2
file,
> > the LCC grid that I am using is not compatible? Or do you suspect
the
> > problem lies in my actual plotting code? I am using basemap to set
up an
> > LCC projection of the map you see in my images. You may not have
an
> answer
> > but I am just trying to narrow down if I should focus on the
actual grib2
> > file or my Python plotting script when trying to come up with a
solution.
> >
> > Thanks,
> > Ben
> >
> > On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Ben,
> > >
> > > FYI, I tried one more method to plot this data. I used the
> > "ncl_convert2nc"
> > > tool to dump the values to NetCDF. And then I displayed the
result
> using
> > > ncview.
> > >
> > > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
> > > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
> > >
> > > I believe ncview displays the data using ONLY the lat/lon values
for
> each
> > > grid point... and not any projection info. Layering on the map
data
> > > produces an image consistent with IDV and MET. I'm mostly just
checking
> > the
> > > extent of the map data in the corners (Victoria Island, Nova
Scotia,
> and
> > > Cuba).
> > >
> > > So that also points to a potential problem in your python
plotting
> code.
> > >
> > > Thanks,
> > > John
> > >
> > > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
> > >
> > >
> > >
> > > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway
<johnhg at ucar.edu>
> > > wrote:
> > >
> > > > Ben,
> > > >
> > > > When I run the sample data you sent through MET's
plot_data_plane
> > tool, I
> > > > get a rather telling warning message:
> > > >
> > > >
> > > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
> > > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD"; level="Z0";'
-v 4*
> > > >
> > > >
> > > >
> > > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET
does
> not
> > > > currently support Lambert Conformal grids where dx (5.07862)
!= dy
> > > > (5.08049) and may produce unexpected results!WARNING: *
> > > >
> > > > I wonder if the unexpected extent of the MODE objects is the
> > "*unexpected
> > > > results*" mentioned in that warning message?
> > > >
> > > > In the attached image, I've placed the image from
plot_data_plane
> > (left)
> > > > next to the image from Unidata's IDV viewer (right). Paying
> particular
> > > > attention to where the data lives relative to the underlying
map
> data.
> > I
> > > > was expecting to see a difference in where the raw data lives
between
> > > these
> > > > two images, but I do not. The relationship of the data to the
Great
> > Lakes
> > > > looks very consistent. So this didn't reveal an obvious
problem.
> > > >
> > > > But I do see some large differences when comparing to the
images you
> > > sent.
> > > > Looking in the top-left corner, both MET and IDV include
roughly 90
> of
> > > > Victoria Island. However the images you sent include ALL of
Victoria
> > > > Island, plus more! In the bottom right corner, MET and IDV
include
> only
> > > the
> > > > northern half of Cuba whereas your plot includes almost all of
it.
> > > >
> > > > I'd recommend checking the code you're using to create those
plots.
> Do
> > > you
> > > > have an independent way of plotting this data that you trust
to
> figure
> > > out
> > > > if there's a problem in the grid or projection being used in
your
> > > plotting?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
> > > >
> > > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
Affiliate
> via
> > > RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>
> > > >>
> > > >> Hi John,
> > > >>
> > > >> Thanks for your reply and the suggestions. As far as missing
data,
> I'm
> > > not
> > > >> 100% sure how it is stored in this particular file we are
working
> > with.
> > > I
> > > >> asked a co-worker and he said he has not seen missing values
in this
> > > >> particular file before and that in some of the other files we
use,
> > > missing
> > > >> values are usually relegated to the edges of the grid and
stored as
> > > 10^36.
> > > >> Also thanks for the warning about having MODE read data via
record
> > > number.
> > > >> I typically only use files that have one record to read but
that is
> > good
> > > >> to
> > > >> know for the future.
> > > >>
> > > >> As for testing I tried setting a convolution radius to 0.
I've
> > attached
> > > >> that image. You can see the model object still appears too
large.
> I've
> > > >> attached the data file I am using as well as the config file
again.
> > > Please
> > > >> let me know if you have any other suggestions or see anything
wrong
> > with
> > > >> the data file or config file. I appreciate the help.
> > > >>
> > > >> Thanks,
> > > >> Ben
> > > >>
> > > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
> > > >> met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> > Hi Ben,
> > > >> >
> > > >> > I see you have some questions about sensitivity of
thresholds in
> > MODE.
> > > >> > Thanks for sending some graphics and your MODE config file.
I do
> > have
> > > a
> > > >> > couple of thoughts.
> > > >> >
> > > >> > First, do you know, are the values where there is no
snowfall
> stored
> > > as
> > > >> 0's
> > > >> > or missing data values?
> > > >> >
> > > >> > To define objects, MODE does a 2 step process... (1)
smoothing the
> > > data
> > > >> > based on the conv_radius setting and (2) thresholding the
smoothed
> > > data
> > > >> by
> > > >> > applying the conv_thresh setting. If the no snowfall points
are
> > stored
> > > >> as
> > > >> > 0, then all should be well. If it's stored as missing data,
then
> > that
> > > >> may
> > > >> > wreak havoc on the smoothing step and cause unexpected
results.
> > > >> >
> > > >> > Next, I'd recommend testing using "conv_radius = 0".
That'll
> > > effectively
> > > >> > disable the smoothing step. And the objects MODE creates
should
> > > exactly
> > > >> > match the contours of your plots.
> > > >> >
> > > >> > In general, I'd recommend against configuring MET to read
data
> based
> > > on
> > > >> a
> > > >> > record number:
> > > >> > field = {
> > > >> > name = "SNOD";
> > > >> > level = "R1";
> > > >> > };
> > > >> > That's dangerous because it'll just use whatever data is in
record
> > > >> number 1
> > > >> > regardless of whether or not it's actually SNOD data.
> > > >> >
> > > >> > If you'd like me to take a closer look, please just send me
(or
> > point
> > > me
> > > >> > to) the data files used for the case you sent.
> > > >> >
> > > >> > Thanks,
> > > >> > John Halley Gotway
> > > >> >
> > > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
Affiliate
> > via
> > > >> RT <
> > > >> > met_help at ucar.edu> wrote:
> > > >> >
> > > >> > >
> > > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
> > > >> > > Transaction: Ticket created by benjamin.albright at noaa.gov
> > > >> > > Queue: met_help
> > > >> > > Subject: MODE Threshold Issue
> > > >> > > Owner: Nobody
> > > >> > > Requestors: benjamin.albright at noaa.gov
> > > >> > > Status: new
> > > >> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> > > >> >
> > > >> > >
> > > >> > >
> > > >> > > Hello MET Help:
> > > >> > >
> > > >> > > I'm having an issue with using MODE and having some
objects
> appear
> > > >> much
> > > >> > > larger than what I expect they should be. I am running
MODE
> (v9.0)
> > > on
> > > >> 24
> > > >> > hr
> > > >> > > snow accumulations. The first image is the 24 hr snow
> accumulation
> > > for
> > > >> > > "Model A." The second image shows the MODE objects for a
> threshold
> > > of
> > > >> 1
> > > >> > > inch. You can see the object extends too far into
Indiana, Ohio,
> > and
> > > >> > > eastern Michigan than I would expect based on the
accumulation
> map
> > > in
> > > >> the
> > > >> > > first image. The third attachment is my configuration
file that
> I
> > > >> used.
> > > >> > The
> > > >> > > data is being read from grib2 files.
> > > >> > >
> > > >> > > I've done a few tests. The first test was lowering the
first
> > > >> threshold in
> > > >> > > the snow accumulation graphic to 0.99" (fourth
attachment). You
> > can
> > > >> with
> > > >> > > that, the MODE object appears to match the light blue
0.99"
> area.
> > I
> > > >> then
> > > >> > > raised the threshold in my MODE code to 1.01." You can
see that
> > the
> > > >> > object
> > > >> > > now matches more closely with the expected 1 inch
accumulation
> > from
> > > >> the
> > > >> > > first image attachment. I was wondering if you might know
what
> is
> > > >> causing
> > > >> > > this sensitivity and if there is a configuration option I
could
> > use
> > > to
> > > >> > > mitigate it? I've seen this behavior in other models as
well so
> I
> > > >> don't
> > > >> > > believe it is input data related but haven't completely
ruled
> that
> > > >> out.
> > > >> > If
> > > >> > > you have any thoughts or suggestions I would really
appreciate
> it.
> > > >> > >
> > > >> > > Thank you,
> > > >> > > Ben Albright
> > > >> > >
> > > >> > > --
> > > >> > > Benjamin Albright, Phd.
> > > >> > > Meteorological Developer
> > > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
> Center
> > > >> > > Work email: benjamin.albright at noaa.gov
> > > >> > > Work phone: 301-683-1474
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >> --
> > > >> Benjamin Albright, Phd.
> > > >> Meteorological Developer
> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > > >> Work email: benjamin.albright at noaa.gov
> > > >> Work phone: 301-683-1474
> > > >>
> > > >>
> > >
> > >
> >
> > --
> > Benjamin Albright, Phd.
> > Meteorological Developer
> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> > Work email: benjamin.albright at noaa.gov
> > Work phone: 301-683-1474
> >
> >
>
>
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
------------------------------------------------
Subject: MODE Threshold Issue
From: Benjamin Albright - NOAA Affiliate
Time: Fri Feb 12 06:16:59 2021
Hi John,
Another warning I receive that could be completely unrelated is the
following:
WARNING:
WARNING: Dictionary::lookup_num_array() -> numeric array lookup failed
for
name "censor_val"
WARNING:
When I upgraded my scripts to version 9.0 I replaced raw_thresh with
censor_val. In the help document, I also see there's a censor_thresh.
When
I replace censor_val with censor_thresh or add censor_thresh with
censor_val I get this warning:
ERROR :
ERROR : VarInfo::set_dict() -> The number of censor thresholds in
"censor_thresh" (1) must match the number of replacement values in
"censor_val" (0).
ERROR :
Again I'm not sure if this could be related to anything but just
wanted
to let you know.
Thanks,
Ben
On Fri, Feb 12, 2021 at 6:55 AM Benjamin Albright - NOAA Affiliate <
benjamin.albright at noaa.gov> wrote:
> Hi John
>
> Thanks for the replies. I was able to go back to where I am making
the
> files and make a couple changes in order to get the final files to
have DX
> = DY.
>
> *New model file:* wgrib2 -V
nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
> 1:0:vt=2021020712:surface - surface:53 hour fcst:SNOD Snow Depth
[m]:
> ndata=3744965:undef=0:mean=0.51457:min=0:max=36
> grid_template=30:winds(grid):
> Lambert Conformal: (2345 x 1597) input WE:SN output WE:SN res 8
> Lat1 19.229000 Lon1 233.723396 LoV 265.000000
> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> LatSP -90.000000 LonSP 0.000000
> North Pole (2345 x 1597) Dx 2539.703000 m Dy 2539.703000 m mode 8
>
> *New verification file:* wgrib2 -V nohrsc_2021020712_in.grib2
> 1:0:vt=2021020712:surface - surface:anl:SNOD Snow Depth [m]:
> ndata=2953665:undef=1654866:mean=0.514013:min=0:max=53
> grid_template=30:winds(grid):
> Lambert Conformal: (2145 x 1377) input WE:SN output WE:SN res 8
> Lat1 20.191999 Lon1 238.445999 LoV 265.000000
> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> LatSP -90.000000 LonSP 0.000000
> North Pole (2145 x 1377) Dx 2539.703000 m Dy 2539.703000 m mode 8
>
> This removed the warning when running MODE about DX != DY. However,
when
> running MODE on this I still am getting the same result for the one
inch
> threshold where the one inch object area is much larger. I've
attached the
> two new data files, the image, and config file.
>
> I haven't had a chance yet to go back and closely check my python
> plotting. I am using basemap in python to generate the images. I'll
keep
> investigating but just wanted to let you know about this new test I
tried
> this morning.
>
> Thank you,
> Ben
>
> On Thu, Feb 11, 2021 at 4:07 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ben,
>>
>> I suspect the real problem is in the python code you're using to
plot the
>> data. I'd recommend using an alternative way of visualizing GRIB2
data
>> that
>> you trust. Compare their results. Check to see if your python code
is
>> plotting the data in the right spot on the earth... and that the
extent of
>> the grid is correct.
>>
>> Once you're confident in the accuracy of the python plots being
created,
>> recheck the extent of MODE objects to see if the issue still
remains. It's
>> possible there's multiple things going on here.
>>
>> MET does not currently support Lambert Conformal grids where the dx
and dy
>> values differ.
>> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN res 8
>> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
>> LatSP -90.000000 LonSP 0.000000
>> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m mode
8
>>
>> It just uses the Dx value and prints the warning about Dy being
different.
>>
>> Another possible explanation is that your python script is right
and MET,
>> NCL, and IDV are all doing it wrong. I haven't run into many
Lambert grids
>> where Dx != Dy.
>>
>> Regardless, perhaps I should write up a GitHub issue for us to
enhance MET
>> to support LC grids where Dx != Dy? Is this a common grid that you
guys
>> use and plan to continue using in the future?
>>
>> Thanks,
>> John
>>
>> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA Affiliate
via
>> RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>> >
>> > Hi John,
>> >
>> > Thanks for the responses. I just want to make sure I'm clear. Is
that
>> > warning MET displays about the projection a warning about the
projection
>> > used in the grib2 data file? In other words, when I make the
grib2 file,
>> > the LCC grid that I am using is not compatible? Or do you suspect
the
>> > problem lies in my actual plotting code? I am using basemap to
set up an
>> > LCC projection of the map you see in my images. You may not have
an
>> answer
>> > but I am just trying to narrow down if I should focus on the
actual
>> grib2
>> > file or my Python plotting script when trying to come up with a
>> solution.
>> >
>> > Thanks,
>> > Ben
>> >
>> > On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Ben,
>> > >
>> > > FYI, I tried one more method to plot this data. I used the
>> > "ncl_convert2nc"
>> > > tool to dump the values to NetCDF. And then I displayed the
result
>> using
>> > > ncview.
>> > >
>> > > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
>> > > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
>> > >
>> > > I believe ncview displays the data using ONLY the lat/lon
values for
>> each
>> > > grid point... and not any projection info. Layering on the map
data
>> > > produces an image consistent with IDV and MET. I'm mostly just
>> checking
>> > the
>> > > extent of the map data in the corners (Victoria Island, Nova
Scotia,
>> and
>> > > Cuba).
>> > >
>> > > So that also points to a potential problem in your python
plotting
>> code.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
>> > >
>> > >
>> > >
>> > > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway
<johnhg at ucar.edu>
>> > > wrote:
>> > >
>> > > > Ben,
>> > > >
>> > > > When I run the sample data you sent through MET's
plot_data_plane
>> > tool, I
>> > > > get a rather telling warning message:
>> > > >
>> > > >
>> > > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
>> > > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD";
level="Z0";' -v
>> 4*
>> > > >
>> > > >
>> > > >
>> > > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() -> MET
does
>> not
>> > > > currently support Lambert Conformal grids where dx (5.07862)
!= dy
>> > > > (5.08049) and may produce unexpected results!WARNING: *
>> > > >
>> > > > I wonder if the unexpected extent of the MODE objects is the
>> > "*unexpected
>> > > > results*" mentioned in that warning message?
>> > > >
>> > > > In the attached image, I've placed the image from
plot_data_plane
>> > (left)
>> > > > next to the image from Unidata's IDV viewer (right). Paying
>> particular
>> > > > attention to where the data lives relative to the underlying
map
>> data.
>> > I
>> > > > was expecting to see a difference in where the raw data lives
>> between
>> > > these
>> > > > two images, but I do not. The relationship of the data to the
Great
>> > Lakes
>> > > > looks very consistent. So this didn't reveal an obvious
problem.
>> > > >
>> > > > But I do see some large differences when comparing to the
images you
>> > > sent.
>> > > > Looking in the top-left corner, both MET and IDV include
roughly 90
>> of
>> > > > Victoria Island. However the images you sent include ALL of
Victoria
>> > > > Island, plus more! In the bottom right corner, MET and IDV
include
>> only
>> > > the
>> > > > northern half of Cuba whereas your plot includes almost all
of it.
>> > > >
>> > > > I'd recommend checking the code you're using to create those
plots.
>> Do
>> > > you
>> > > > have an independent way of plotting this data that you trust
to
>> figure
>> > > out
>> > > > if there's a problem in the grid or projection being used in
your
>> > > plotting?
>> > > >
>> > > > Thanks,
>> > > > John
>> > > >
>> > > >
>> > > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
>> > > >
>> > > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
Affiliate
>> via
>> > > RT <
>> > > > met_help at ucar.edu> wrote:
>> > > >
>> > > >>
>> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>> > > >>
>> > > >> Hi John,
>> > > >>
>> > > >> Thanks for your reply and the suggestions. As far as missing
data,
>> I'm
>> > > not
>> > > >> 100% sure how it is stored in this particular file we are
working
>> > with.
>> > > I
>> > > >> asked a co-worker and he said he has not seen missing values
in
>> this
>> > > >> particular file before and that in some of the other files
we use,
>> > > missing
>> > > >> values are usually relegated to the edges of the grid and
stored as
>> > > 10^36.
>> > > >> Also thanks for the warning about having MODE read data via
record
>> > > number.
>> > > >> I typically only use files that have one record to read but
that is
>> > good
>> > > >> to
>> > > >> know for the future.
>> > > >>
>> > > >> As for testing I tried setting a convolution radius to 0.
I've
>> > attached
>> > > >> that image. You can see the model object still appears too
large.
>> I've
>> > > >> attached the data file I am using as well as the config file
again.
>> > > Please
>> > > >> let me know if you have any other suggestions or see
anything wrong
>> > with
>> > > >> the data file or config file. I appreciate the help.
>> > > >>
>> > > >> Thanks,
>> > > >> Ben
>> > > >>
>> > > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
>> > > >> met_help at ucar.edu>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi Ben,
>> > > >> >
>> > > >> > I see you have some questions about sensitivity of
thresholds in
>> > MODE.
>> > > >> > Thanks for sending some graphics and your MODE config
file. I do
>> > have
>> > > a
>> > > >> > couple of thoughts.
>> > > >> >
>> > > >> > First, do you know, are the values where there is no
snowfall
>> stored
>> > > as
>> > > >> 0's
>> > > >> > or missing data values?
>> > > >> >
>> > > >> > To define objects, MODE does a 2 step process... (1)
smoothing
>> the
>> > > data
>> > > >> > based on the conv_radius setting and (2) thresholding the
>> smoothed
>> > > data
>> > > >> by
>> > > >> > applying the conv_thresh setting. If the no snowfall
points are
>> > stored
>> > > >> as
>> > > >> > 0, then all should be well. If it's stored as missing
data, then
>> > that
>> > > >> may
>> > > >> > wreak havoc on the smoothing step and cause unexpected
results.
>> > > >> >
>> > > >> > Next, I'd recommend testing using "conv_radius = 0".
That'll
>> > > effectively
>> > > >> > disable the smoothing step. And the objects MODE creates
should
>> > > exactly
>> > > >> > match the contours of your plots.
>> > > >> >
>> > > >> > In general, I'd recommend against configuring MET to read
data
>> based
>> > > on
>> > > >> a
>> > > >> > record number:
>> > > >> > field = {
>> > > >> > name = "SNOD";
>> > > >> > level = "R1";
>> > > >> > };
>> > > >> > That's dangerous because it'll just use whatever data is
in
>> record
>> > > >> number 1
>> > > >> > regardless of whether or not it's actually SNOD data.
>> > > >> >
>> > > >> > If you'd like me to take a closer look, please just send
me (or
>> > point
>> > > me
>> > > >> > to) the data files used for the case you sent.
>> > > >> >
>> > > >> > Thanks,
>> > > >> > John Halley Gotway
>> > > >> >
>> > > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
>> Affiliate
>> > via
>> > > >> RT <
>> > > >> > met_help at ucar.edu> wrote:
>> > > >> >
>> > > >> > >
>> > > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
>> > > >> > > Transaction: Ticket created by
benjamin.albright at noaa.gov
>> > > >> > > Queue: met_help
>> > > >> > > Subject: MODE Threshold Issue
>> > > >> > > Owner: Nobody
>> > > >> > > Requestors: benjamin.albright at noaa.gov
>> > > >> > > Status: new
>> > > >> > > Ticket <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>> > > >> >
>> > > >> > >
>> > > >> > >
>> > > >> > > Hello MET Help:
>> > > >> > >
>> > > >> > > I'm having an issue with using MODE and having some
objects
>> appear
>> > > >> much
>> > > >> > > larger than what I expect they should be. I am running
MODE
>> (v9.0)
>> > > on
>> > > >> 24
>> > > >> > hr
>> > > >> > > snow accumulations. The first image is the 24 hr snow
>> accumulation
>> > > for
>> > > >> > > "Model A." The second image shows the MODE objects for a
>> threshold
>> > > of
>> > > >> 1
>> > > >> > > inch. You can see the object extends too far into
Indiana,
>> Ohio,
>> > and
>> > > >> > > eastern Michigan than I would expect based on the
accumulation
>> map
>> > > in
>> > > >> the
>> > > >> > > first image. The third attachment is my configuration
file
>> that I
>> > > >> used.
>> > > >> > The
>> > > >> > > data is being read from grib2 files.
>> > > >> > >
>> > > >> > > I've done a few tests. The first test was lowering the
first
>> > > >> threshold in
>> > > >> > > the snow accumulation graphic to 0.99" (fourth
attachment). You
>> > can
>> > > >> with
>> > > >> > > that, the MODE object appears to match the light blue
0.99"
>> area.
>> > I
>> > > >> then
>> > > >> > > raised the threshold in my MODE code to 1.01." You can
see that
>> > the
>> > > >> > object
>> > > >> > > now matches more closely with the expected 1 inch
accumulation
>> > from
>> > > >> the
>> > > >> > > first image attachment. I was wondering if you might
know what
>> is
>> > > >> causing
>> > > >> > > this sensitivity and if there is a configuration option
I could
>> > use
>> > > to
>> > > >> > > mitigate it? I've seen this behavior in other models as
well
>> so I
>> > > >> don't
>> > > >> > > believe it is input data related but haven't completely
ruled
>> that
>> > > >> out.
>> > > >> > If
>> > > >> > > you have any thoughts or suggestions I would really
appreciate
>> it.
>> > > >> > >
>> > > >> > > Thank you,
>> > > >> > > Ben Albright
>> > > >> > >
>> > > >> > > --
>> > > >> > > Benjamin Albright, Phd.
>> > > >> > > Meteorological Developer
>> > > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
>> Center
>> > > >> > > Work email: benjamin.albright at noaa.gov
>> > > >> > > Work phone: 301-683-1474
>> > > >> > >
>> > > >> > >
>> > > >> >
>> > > >> >
>> > > >>
>> > > >> --
>> > > >> Benjamin Albright, Phd.
>> > > >> Meteorological Developer
>> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
>> > > >> Work email: benjamin.albright at noaa.gov
>> > > >> Work phone: 301-683-1474
>> > > >>
>> > > >>
>> > >
>> > >
>> >
>> > --
>> > Benjamin Albright, Phd.
>> > Meteorological Developer
>> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
>> > Work email: benjamin.albright at noaa.gov
>> > Work phone: 301-683-1474
>> >
>> >
>>
>>
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
------------------------------------------------
Subject: MODE Threshold Issue
From: Benjamin Albright - NOAA Affiliate
Time: Fri Feb 12 07:26:28 2021
Hi John,
One final test I just completed. I used pcp_combine to make my file
that I
used in MODE. I got similar results. Slightly less coverage in
Michigan but
still basically about the same. I just wanted to try using the MET
tool to
make my input file rather than my other method as a test. Attached is
the
new image. Thanks again for your help.
Ben
On Fri, Feb 12, 2021 at 1:16 PM Benjamin Albright - NOAA Affiliate <
benjamin.albright at noaa.gov> wrote:
> Hi John,
>
> Another warning I receive that could be completely unrelated is the
> following:
>
> WARNING:
> WARNING: Dictionary::lookup_num_array() -> numeric array lookup
failed for
> name "censor_val"
> WARNING:
>
> When I upgraded my scripts to version 9.0 I replaced raw_thresh with
> censor_val. In the help document, I also see there's a
censor_thresh. When
> I replace censor_val with censor_thresh or add censor_thresh with
> censor_val I get this warning:
>
> ERROR :
> ERROR : VarInfo::set_dict() -> The number of censor thresholds in
> "censor_thresh" (1) must match the number of replacement values in
> "censor_val" (0).
> ERROR :
>
> Again I'm not sure if this could be related to anything but just
wanted
> to let you know.
>
> Thanks,
> Ben
>
> On Fri, Feb 12, 2021 at 6:55 AM Benjamin Albright - NOAA Affiliate <
> benjamin.albright at noaa.gov> wrote:
>
>> Hi John
>>
>> Thanks for the replies. I was able to go back to where I am making
the
>> files and make a couple changes in order to get the final files to
have DX
>> = DY.
>>
>> *New model file:* wgrib2 -V
nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
>> 1:0:vt=2021020712:surface - surface:53 hour fcst:SNOD Snow Depth
[m]:
>> ndata=3744965:undef=0:mean=0.51457:min=0:max=36
>> grid_template=30:winds(grid):
>> Lambert Conformal: (2345 x 1597) input WE:SN output WE:SN res 8
>> Lat1 19.229000 Lon1 233.723396 LoV 265.000000
>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
>> LatSP -90.000000 LonSP 0.000000
>> North Pole (2345 x 1597) Dx 2539.703000 m Dy 2539.703000 m mode 8
>>
>> *New verification file:* wgrib2 -V nohrsc_2021020712_in.grib2
>> 1:0:vt=2021020712:surface - surface:anl:SNOD Snow Depth [m]:
>> ndata=2953665:undef=1654866:mean=0.514013:min=0:max=53
>> grid_template=30:winds(grid):
>> Lambert Conformal: (2145 x 1377) input WE:SN output WE:SN res 8
>> Lat1 20.191999 Lon1 238.445999 LoV 265.000000
>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
>> LatSP -90.000000 LonSP 0.000000
>> North Pole (2145 x 1377) Dx 2539.703000 m Dy 2539.703000 m mode 8
>>
>> This removed the warning when running MODE about DX != DY. However,
when
>> running MODE on this I still am getting the same result for the one
inch
>> threshold where the one inch object area is much larger. I've
attached the
>> two new data files, the image, and config file.
>>
>> I haven't had a chance yet to go back and closely check my python
>> plotting. I am using basemap in python to generate the images. I'll
keep
>> investigating but just wanted to let you know about this new test I
tried
>> this morning.
>>
>> Thank you,
>> Ben
>>
>> On Thu, Feb 11, 2021 at 4:07 PM John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> Ben,
>>>
>>> I suspect the real problem is in the python code you're using to
plot the
>>> data. I'd recommend using an alternative way of visualizing GRIB2
data
>>> that
>>> you trust. Compare their results. Check to see if your python code
is
>>> plotting the data in the right spot on the earth... and that the
extent
>>> of
>>> the grid is correct.
>>>
>>> Once you're confident in the accuracy of the python plots being
created,
>>> recheck the extent of MODE objects to see if the issue still
remains.
>>> It's
>>> possible there's multiple things going on here.
>>>
>>> MET does not currently support Lambert Conformal grids where the
dx and
>>> dy
>>> values differ.
>>> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN res 8
>>> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
>>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
>>> LatSP -90.000000 LonSP 0.000000
>>> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m
mode 8
>>>
>>> It just uses the Dx value and prints the warning about Dy being
>>> different.
>>>
>>> Another possible explanation is that your python script is right
and MET,
>>> NCL, and IDV are all doing it wrong. I haven't run into many
Lambert
>>> grids
>>> where Dx != Dy.
>>>
>>> Regardless, perhaps I should write up a GitHub issue for us to
enhance
>>> MET
>>> to support LC grids where Dx != Dy? Is this a common grid that
you guys
>>> use and plan to continue using in the future?
>>>
>>> Thanks,
>>> John
>>>
>>> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA
Affiliate via
>>> RT <
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>>> >
>>> > Hi John,
>>> >
>>> > Thanks for the responses. I just want to make sure I'm clear. Is
that
>>> > warning MET displays about the projection a warning about the
>>> projection
>>> > used in the grib2 data file? In other words, when I make the
grib2
>>> file,
>>> > the LCC grid that I am using is not compatible? Or do you
suspect the
>>> > problem lies in my actual plotting code? I am using basemap to
set up
>>> an
>>> > LCC projection of the map you see in my images. You may not have
an
>>> answer
>>> > but I am just trying to narrow down if I should focus on the
actual
>>> grib2
>>> > file or my Python plotting script when trying to come up with a
>>> solution.
>>> >
>>> > Thanks,
>>> > Ben
>>> >
>>> > On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
>>> > met_help at ucar.edu>
>>> > wrote:
>>> >
>>> > > Ben,
>>> > >
>>> > > FYI, I tried one more method to plot this data. I used the
>>> > "ncl_convert2nc"
>>> > > tool to dump the values to NetCDF. And then I displayed the
result
>>> using
>>> > > ncview.
>>> > >
>>> > > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
>>> > > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
>>> > >
>>> > > I believe ncview displays the data using ONLY the lat/lon
values for
>>> each
>>> > > grid point... and not any projection info. Layering on the map
data
>>> > > produces an image consistent with IDV and MET. I'm mostly just
>>> checking
>>> > the
>>> > > extent of the map data in the corners (Victoria Island, Nova
Scotia,
>>> and
>>> > > Cuba).
>>> > >
>>> > > So that also points to a potential problem in your python
plotting
>>> code.
>>> > >
>>> > > Thanks,
>>> > > John
>>> > >
>>> > > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
>>> > >
>>> > >
>>> > >
>>> > > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway
<johnhg at ucar.edu
>>> >
>>> > > wrote:
>>> > >
>>> > > > Ben,
>>> > > >
>>> > > > When I run the sample data you sent through MET's
plot_data_plane
>>> > tool, I
>>> > > > get a rather telling warning message:
>>> > > >
>>> > > >
>>> > > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
>>> > > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD";
level="Z0";' -v
>>> 4*
>>> > > >
>>> > > >
>>> > > >
>>> > > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() ->
MET does
>>> not
>>> > > > currently support Lambert Conformal grids where dx (5.07862)
!= dy
>>> > > > (5.08049) and may produce unexpected results!WARNING: *
>>> > > >
>>> > > > I wonder if the unexpected extent of the MODE objects is the
>>> > "*unexpected
>>> > > > results*" mentioned in that warning message?
>>> > > >
>>> > > > In the attached image, I've placed the image from
plot_data_plane
>>> > (left)
>>> > > > next to the image from Unidata's IDV viewer (right). Paying
>>> particular
>>> > > > attention to where the data lives relative to the underlying
map
>>> data.
>>> > I
>>> > > > was expecting to see a difference in where the raw data
lives
>>> between
>>> > > these
>>> > > > two images, but I do not. The relationship of the data to
the Great
>>> > Lakes
>>> > > > looks very consistent. So this didn't reveal an obvious
problem.
>>> > > >
>>> > > > But I do see some large differences when comparing to the
images
>>> you
>>> > > sent.
>>> > > > Looking in the top-left corner, both MET and IDV include
roughly
>>> 90 of
>>> > > > Victoria Island. However the images you sent include ALL of
>>> Victoria
>>> > > > Island, plus more! In the bottom right corner, MET and IDV
include
>>> only
>>> > > the
>>> > > > northern half of Cuba whereas your plot includes almost all
of it.
>>> > > >
>>> > > > I'd recommend checking the code you're using to create those
>>> plots. Do
>>> > > you
>>> > > > have an independent way of plotting this data that you trust
to
>>> figure
>>> > > out
>>> > > > if there's a problem in the grid or projection being used in
your
>>> > > plotting?
>>> > > >
>>> > > > Thanks,
>>> > > > John
>>> > > >
>>> > > >
>>> > > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
>>> > > >
>>> > > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
Affiliate
>>> via
>>> > > RT <
>>> > > > met_help at ucar.edu> wrote:
>>> > > >
>>> > > >>
>>> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>>> > > >>
>>> > > >> Hi John,
>>> > > >>
>>> > > >> Thanks for your reply and the suggestions. As far as
missing
>>> data, I'm
>>> > > not
>>> > > >> 100% sure how it is stored in this particular file we are
working
>>> > with.
>>> > > I
>>> > > >> asked a co-worker and he said he has not seen missing
values in
>>> this
>>> > > >> particular file before and that in some of the other files
we use,
>>> > > missing
>>> > > >> values are usually relegated to the edges of the grid and
stored
>>> as
>>> > > 10^36.
>>> > > >> Also thanks for the warning about having MODE read data via
record
>>> > > number.
>>> > > >> I typically only use files that have one record to read but
that
>>> is
>>> > good
>>> > > >> to
>>> > > >> know for the future.
>>> > > >>
>>> > > >> As for testing I tried setting a convolution radius to 0.
I've
>>> > attached
>>> > > >> that image. You can see the model object still appears too
large.
>>> I've
>>> > > >> attached the data file I am using as well as the config
file
>>> again.
>>> > > Please
>>> > > >> let me know if you have any other suggestions or see
anything
>>> wrong
>>> > with
>>> > > >> the data file or config file. I appreciate the help.
>>> > > >>
>>> > > >> Thanks,
>>> > > >> Ben
>>> > > >>
>>> > > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT <
>>> > > >> met_help at ucar.edu>
>>> > > >> wrote:
>>> > > >>
>>> > > >> > Hi Ben,
>>> > > >> >
>>> > > >> > I see you have some questions about sensitivity of
thresholds in
>>> > MODE.
>>> > > >> > Thanks for sending some graphics and your MODE config
file. I do
>>> > have
>>> > > a
>>> > > >> > couple of thoughts.
>>> > > >> >
>>> > > >> > First, do you know, are the values where there is no
snowfall
>>> stored
>>> > > as
>>> > > >> 0's
>>> > > >> > or missing data values?
>>> > > >> >
>>> > > >> > To define objects, MODE does a 2 step process... (1)
smoothing
>>> the
>>> > > data
>>> > > >> > based on the conv_radius setting and (2) thresholding the
>>> smoothed
>>> > > data
>>> > > >> by
>>> > > >> > applying the conv_thresh setting. If the no snowfall
points are
>>> > stored
>>> > > >> as
>>> > > >> > 0, then all should be well. If it's stored as missing
data, then
>>> > that
>>> > > >> may
>>> > > >> > wreak havoc on the smoothing step and cause unexpected
results.
>>> > > >> >
>>> > > >> > Next, I'd recommend testing using "conv_radius = 0".
That'll
>>> > > effectively
>>> > > >> > disable the smoothing step. And the objects MODE creates
should
>>> > > exactly
>>> > > >> > match the contours of your plots.
>>> > > >> >
>>> > > >> > In general, I'd recommend against configuring MET to read
data
>>> based
>>> > > on
>>> > > >> a
>>> > > >> > record number:
>>> > > >> > field = {
>>> > > >> > name = "SNOD";
>>> > > >> > level = "R1";
>>> > > >> > };
>>> > > >> > That's dangerous because it'll just use whatever data is
in
>>> record
>>> > > >> number 1
>>> > > >> > regardless of whether or not it's actually SNOD data.
>>> > > >> >
>>> > > >> > If you'd like me to take a closer look, please just send
me (or
>>> > point
>>> > > me
>>> > > >> > to) the data files used for the case you sent.
>>> > > >> >
>>> > > >> > Thanks,
>>> > > >> > John Halley Gotway
>>> > > >> >
>>> > > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright - NOAA
>>> Affiliate
>>> > via
>>> > > >> RT <
>>> > > >> > met_help at ucar.edu> wrote:
>>> > > >> >
>>> > > >> > >
>>> > > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted upon.
>>> > > >> > > Transaction: Ticket created by
benjamin.albright at noaa.gov
>>> > > >> > > Queue: met_help
>>> > > >> > > Subject: MODE Threshold Issue
>>> > > >> > > Owner: Nobody
>>> > > >> > > Requestors: benjamin.albright at noaa.gov
>>> > > >> > > Status: new
>>> > > >> > > Ticket <URL:
>>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>>> > > >> >
>>> > > >> > >
>>> > > >> > >
>>> > > >> > > Hello MET Help:
>>> > > >> > >
>>> > > >> > > I'm having an issue with using MODE and having some
objects
>>> appear
>>> > > >> much
>>> > > >> > > larger than what I expect they should be. I am running
MODE
>>> (v9.0)
>>> > > on
>>> > > >> 24
>>> > > >> > hr
>>> > > >> > > snow accumulations. The first image is the 24 hr snow
>>> accumulation
>>> > > for
>>> > > >> > > "Model A." The second image shows the MODE objects for
a
>>> threshold
>>> > > of
>>> > > >> 1
>>> > > >> > > inch. You can see the object extends too far into
Indiana,
>>> Ohio,
>>> > and
>>> > > >> > > eastern Michigan than I would expect based on the
>>> accumulation map
>>> > > in
>>> > > >> the
>>> > > >> > > first image. The third attachment is my configuration
file
>>> that I
>>> > > >> used.
>>> > > >> > The
>>> > > >> > > data is being read from grib2 files.
>>> > > >> > >
>>> > > >> > > I've done a few tests. The first test was lowering the
first
>>> > > >> threshold in
>>> > > >> > > the snow accumulation graphic to 0.99" (fourth
attachment).
>>> You
>>> > can
>>> > > >> with
>>> > > >> > > that, the MODE object appears to match the light blue
0.99"
>>> area.
>>> > I
>>> > > >> then
>>> > > >> > > raised the threshold in my MODE code to 1.01." You can
see
>>> that
>>> > the
>>> > > >> > object
>>> > > >> > > now matches more closely with the expected 1 inch
accumulation
>>> > from
>>> > > >> the
>>> > > >> > > first image attachment. I was wondering if you might
know
>>> what is
>>> > > >> causing
>>> > > >> > > this sensitivity and if there is a configuration option
I
>>> could
>>> > use
>>> > > to
>>> > > >> > > mitigate it? I've seen this behavior in other models as
well
>>> so I
>>> > > >> don't
>>> > > >> > > believe it is input data related but haven't completely
ruled
>>> that
>>> > > >> out.
>>> > > >> > If
>>> > > >> > > you have any thoughts or suggestions I would really
>>> appreciate it.
>>> > > >> > >
>>> > > >> > > Thank you,
>>> > > >> > > Ben Albright
>>> > > >> > >
>>> > > >> > > --
>>> > > >> > > Benjamin Albright, Phd.
>>> > > >> > > Meteorological Developer
>>> > > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
>>> Center
>>> > > >> > > Work email: benjamin.albright at noaa.gov
>>> > > >> > > Work phone: 301-683-1474
>>> > > >> > >
>>> > > >> > >
>>> > > >> >
>>> > > >> >
>>> > > >>
>>> > > >> --
>>> > > >> Benjamin Albright, Phd.
>>> > > >> Meteorological Developer
>>> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
>>> > > >> Work email: benjamin.albright at noaa.gov
>>> > > >> Work phone: 301-683-1474
>>> > > >>
>>> > > >>
>>> > >
>>> > >
>>> >
>>> > --
>>> > Benjamin Albright, Phd.
>>> > Meteorological Developer
>>> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
>>> > Work email: benjamin.albright at noaa.gov
>>> > Work phone: 301-683-1474
>>> >
>>> >
>>>
>>>
>>
>> --
>> Benjamin Albright, Phd.
>> Meteorological Developer
>> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
>> Work email: benjamin.albright at noaa.gov
>> Work phone: 301-683-1474
>>
>
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Fri Feb 12 11:19:17 2021
Ben,
Thanks for sending a version where Dx=Dy and taking that out of the
equation. I think perhaps I'm raising an alarm that isn't a problem at
all.
The MET plot_data_plane tool reads gridded data and plots that data on
exactly the same grid as the input.
I'm guessing that with python plotting, that need not be the case. You
can
read data from one grid/projection and easily render it using a
different
projection. So I should stop worrying that your python grid is a
little
larger than the one produced by plot_data_plane.
So about data censoring, yes, you're right. We removed raw_thresh
since
it's function is replaced by censor_thresh and censor_val. These
options
are pretty straightforward. They are arrays that must be the same
length
(as the error message states). If specified, then for each grid point
we
check each threshold (censor_thresh) and if it meets the threshold
criteria, replace the value with the corresponding censor_val entry.
For example, let's say we define:
censor_thresh = [ lt0, gt1 ]; censor_val = [ 0, 1 ];
This reads through all grid points, replaces any values less than 0
with 0
and any values greater than 1 with 1.
To mimic "raw_thresh = 0.5", you'd set:
censor_thresh = lt05;
censor_val = 0.0;
When the arrays are length 1, the square brackets are optional.
But I'm just not seeing or appreciating the problem in the data you're
describing. I ran plot_data_plane on the GRIB file, resetting all
values
between 0 and 1 to 0. Turns out that there aren't any values between 0
and
1 anway!
*plot_data_plane nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
nbmv4_wwe_2021020507F053_24hr_in_v2.ps 'name="SNOD"; level="Z0";
censor_thresh=[gt0&<1]; censor_val=[0];' -v 3*
*DEBUG 3: Applying censor thresholds ">0&&<1" and replacing with
values
"0.00000".DEBUG 3: Censored values for 0 of 3744965 grid points.*
See that 0 of the 3744965 input values were between 0 and 1. Here's
the
resulting image:
[image: Screen Shot 2021-02-12 at 11.12.44 AM.png]
So anywhere that is gray is a value of 1 or higher. Comparing this to
the
image you sent, I just don't see/understand the problem. When you run
this
data through MODE, using "conv_thresh = 5" to smooth the raw values,
and
then threshold the result "conv_thresh = 1", the resulting object that
includes Michigan looks totally fine to me. The smoothing step causes
those
0's in NW MI to pull down surrounding values and yields a reasonable
object.
If it'd help, we could schedule a quick telecon to discuss the
details, so
I can better understand what's not looking right to you.
Thanks,
John
On Fri, Feb 12, 2021 at 7:27 AM Benjamin Albright - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>
> Hi John,
>
> One final test I just completed. I used pcp_combine to make my file
that I
> used in MODE. I got similar results. Slightly less coverage in
Michigan but
> still basically about the same. I just wanted to try using the MET
tool to
> make my input file rather than my other method as a test. Attached
is the
> new image. Thanks again for your help.
>
> Ben
>
> On Fri, Feb 12, 2021 at 1:16 PM Benjamin Albright - NOAA Affiliate <
> benjamin.albright at noaa.gov> wrote:
>
> > Hi John,
> >
> > Another warning I receive that could be completely unrelated is
the
> > following:
> >
> > WARNING:
> > WARNING: Dictionary::lookup_num_array() -> numeric array lookup
failed
> for
> > name "censor_val"
> > WARNING:
> >
> > When I upgraded my scripts to version 9.0 I replaced raw_thresh
with
> > censor_val. In the help document, I also see there's a
censor_thresh.
> When
> > I replace censor_val with censor_thresh or add censor_thresh with
> > censor_val I get this warning:
> >
> > ERROR :
> > ERROR : VarInfo::set_dict() -> The number of censor thresholds in
> > "censor_thresh" (1) must match the number of replacement values in
> > "censor_val" (0).
> > ERROR :
> >
> > Again I'm not sure if this could be related to anything but just
wanted
> > to let you know.
> >
> > Thanks,
> > Ben
> >
> > On Fri, Feb 12, 2021 at 6:55 AM Benjamin Albright - NOAA Affiliate
<
> > benjamin.albright at noaa.gov> wrote:
> >
> >> Hi John
> >>
> >> Thanks for the replies. I was able to go back to where I am
making the
> >> files and make a couple changes in order to get the final files
to have
> DX
> >> = DY.
> >>
> >> *New model file:* wgrib2 -V
nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
> >> 1:0:vt=2021020712:surface - surface:53 hour fcst:SNOD Snow Depth
[m]:
> >> ndata=3744965:undef=0:mean=0.51457:min=0:max=36
> >> grid_template=30:winds(grid):
> >> Lambert Conformal: (2345 x 1597) input WE:SN output WE:SN res 8
> >> Lat1 19.229000 Lon1 233.723396 LoV 265.000000
> >> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> >> LatSP -90.000000 LonSP 0.000000
> >> North Pole (2345 x 1597) Dx 2539.703000 m Dy 2539.703000 m mode 8
> >>
> >> *New verification file:* wgrib2 -V nohrsc_2021020712_in.grib2
> >> 1:0:vt=2021020712:surface - surface:anl:SNOD Snow Depth [m]:
> >> ndata=2953665:undef=1654866:mean=0.514013:min=0:max=53
> >> grid_template=30:winds(grid):
> >> Lambert Conformal: (2145 x 1377) input WE:SN output WE:SN res 8
> >> Lat1 20.191999 Lon1 238.445999 LoV 265.000000
> >> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> >> LatSP -90.000000 LonSP 0.000000
> >> North Pole (2145 x 1377) Dx 2539.703000 m Dy 2539.703000 m mode 8
> >>
> >> This removed the warning when running MODE about DX != DY.
However, when
> >> running MODE on this I still am getting the same result for the
one inch
> >> threshold where the one inch object area is much larger. I've
attached
> the
> >> two new data files, the image, and config file.
> >>
> >> I haven't had a chance yet to go back and closely check my python
> >> plotting. I am using basemap in python to generate the images.
I'll keep
> >> investigating but just wanted to let you know about this new test
I
> tried
> >> this morning.
> >>
> >> Thank you,
> >> Ben
> >>
> >> On Thu, Feb 11, 2021 at 4:07 PM John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >>> Ben,
> >>>
> >>> I suspect the real problem is in the python code you're using to
plot
> the
> >>> data. I'd recommend using an alternative way of visualizing
GRIB2 data
> >>> that
> >>> you trust. Compare their results. Check to see if your python
code is
> >>> plotting the data in the right spot on the earth... and that the
extent
> >>> of
> >>> the grid is correct.
> >>>
> >>> Once you're confident in the accuracy of the python plots being
> created,
> >>> recheck the extent of MODE objects to see if the issue still
remains.
> >>> It's
> >>> possible there's multiple things going on here.
> >>>
> >>> MET does not currently support Lambert Conformal grids where the
dx and
> >>> dy
> >>> values differ.
> >>> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN res
8
> >>> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
> >>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> >>> LatSP -90.000000 LonSP 0.000000
> >>> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m
mode 8
> >>>
> >>> It just uses the Dx value and prints the warning about Dy being
> >>> different.
> >>>
> >>> Another possible explanation is that your python script is right
and
> MET,
> >>> NCL, and IDV are all doing it wrong. I haven't run into many
Lambert
> >>> grids
> >>> where Dx != Dy.
> >>>
> >>> Regardless, perhaps I should write up a GitHub issue for us to
enhance
> >>> MET
> >>> to support LC grids where Dx != Dy? Is this a common grid that
you
> guys
> >>> use and plan to continue using in the future?
> >>>
> >>> Thanks,
> >>> John
> >>>
> >>> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA
Affiliate via
> >>> RT <
> >>> met_help at ucar.edu> wrote:
> >>>
> >>> >
> >>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
>
> >>> >
> >>> > Hi John,
> >>> >
> >>> > Thanks for the responses. I just want to make sure I'm clear.
Is that
> >>> > warning MET displays about the projection a warning about the
> >>> projection
> >>> > used in the grib2 data file? In other words, when I make the
grib2
> >>> file,
> >>> > the LCC grid that I am using is not compatible? Or do you
suspect the
> >>> > problem lies in my actual plotting code? I am using basemap to
set up
> >>> an
> >>> > LCC projection of the map you see in my images. You may not
have an
> >>> answer
> >>> > but I am just trying to narrow down if I should focus on the
actual
> >>> grib2
> >>> > file or my Python plotting script when trying to come up with
a
> >>> solution.
> >>> >
> >>> > Thanks,
> >>> > Ben
> >>> >
> >>> > On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
> >>> > met_help at ucar.edu>
> >>> > wrote:
> >>> >
> >>> > > Ben,
> >>> > >
> >>> > > FYI, I tried one more method to plot this data. I used the
> >>> > "ncl_convert2nc"
> >>> > > tool to dump the values to NetCDF. And then I displayed the
result
> >>> using
> >>> > > ncview.
> >>> > >
> >>> > > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
> >>> > > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
> >>> > >
> >>> > > I believe ncview displays the data using ONLY the lat/lon
values
> for
> >>> each
> >>> > > grid point... and not any projection info. Layering on the
map data
> >>> > > produces an image consistent with IDV and MET. I'm mostly
just
> >>> checking
> >>> > the
> >>> > > extent of the map data in the corners (Victoria Island, Nova
> Scotia,
> >>> and
> >>> > > Cuba).
> >>> > >
> >>> > > So that also points to a potential problem in your python
plotting
> >>> code.
> >>> > >
> >>> > > Thanks,
> >>> > > John
> >>> > >
> >>> > > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway <
> johnhg at ucar.edu
> >>> >
> >>> > > wrote:
> >>> > >
> >>> > > > Ben,
> >>> > > >
> >>> > > > When I run the sample data you sent through MET's
plot_data_plane
> >>> > tool, I
> >>> > > > get a rather telling warning message:
> >>> > > >
> >>> > > >
> >>> > > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
> >>> > > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD";
level="Z0";'
> -v
> >>> 4*
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid() ->
MET
> does
> >>> not
> >>> > > > currently support Lambert Conformal grids where dx
(5.07862) !=
> dy
> >>> > > > (5.08049) and may produce unexpected results!WARNING: *
> >>> > > >
> >>> > > > I wonder if the unexpected extent of the MODE objects is
the
> >>> > "*unexpected
> >>> > > > results*" mentioned in that warning message?
> >>> > > >
> >>> > > > In the attached image, I've placed the image from
plot_data_plane
> >>> > (left)
> >>> > > > next to the image from Unidata's IDV viewer (right).
Paying
> >>> particular
> >>> > > > attention to where the data lives relative to the
underlying map
> >>> data.
> >>> > I
> >>> > > > was expecting to see a difference in where the raw data
lives
> >>> between
> >>> > > these
> >>> > > > two images, but I do not. The relationship of the data to
the
> Great
> >>> > Lakes
> >>> > > > looks very consistent. So this didn't reveal an obvious
problem.
> >>> > > >
> >>> > > > But I do see some large differences when comparing to the
images
> >>> you
> >>> > > sent.
> >>> > > > Looking in the top-left corner, both MET and IDV include
roughly
> >>> 90 of
> >>> > > > Victoria Island. However the images you sent include ALL
of
> >>> Victoria
> >>> > > > Island, plus more! In the bottom right corner, MET and IDV
> include
> >>> only
> >>> > > the
> >>> > > > northern half of Cuba whereas your plot includes almost
all of
> it.
> >>> > > >
> >>> > > > I'd recommend checking the code you're using to create
those
> >>> plots. Do
> >>> > > you
> >>> > > > have an independent way of plotting this data that you
trust to
> >>> figure
> >>> > > out
> >>> > > > if there's a problem in the grid or projection being used
in your
> >>> > > plotting?
> >>> > > >
> >>> > > > Thanks,
> >>> > > > John
> >>> > > >
> >>> > > >
> >>> > > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
> >>> > > >
> >>> > > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
> Affiliate
> >>> via
> >>> > > RT <
> >>> > > > met_help at ucar.edu> wrote:
> >>> > > >
> >>> > > >>
> >>> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> >>> > > >>
> >>> > > >> Hi John,
> >>> > > >>
> >>> > > >> Thanks for your reply and the suggestions. As far as
missing
> >>> data, I'm
> >>> > > not
> >>> > > >> 100% sure how it is stored in this particular file we are
> working
> >>> > with.
> >>> > > I
> >>> > > >> asked a co-worker and he said he has not seen missing
values in
> >>> this
> >>> > > >> particular file before and that in some of the other
files we
> use,
> >>> > > missing
> >>> > > >> values are usually relegated to the edges of the grid and
stored
> >>> as
> >>> > > 10^36.
> >>> > > >> Also thanks for the warning about having MODE read data
via
> record
> >>> > > number.
> >>> > > >> I typically only use files that have one record to read
but that
> >>> is
> >>> > good
> >>> > > >> to
> >>> > > >> know for the future.
> >>> > > >>
> >>> > > >> As for testing I tried setting a convolution radius to 0.
I've
> >>> > attached
> >>> > > >> that image. You can see the model object still appears
too
> large.
> >>> I've
> >>> > > >> attached the data file I am using as well as the config
file
> >>> again.
> >>> > > Please
> >>> > > >> let me know if you have any other suggestions or see
anything
> >>> wrong
> >>> > with
> >>> > > >> the data file or config file. I appreciate the help.
> >>> > > >>
> >>> > > >> Thanks,
> >>> > > >> Ben
> >>> > > >>
> >>> > > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via RT
<
> >>> > > >> met_help at ucar.edu>
> >>> > > >> wrote:
> >>> > > >>
> >>> > > >> > Hi Ben,
> >>> > > >> >
> >>> > > >> > I see you have some questions about sensitivity of
thresholds
> in
> >>> > MODE.
> >>> > > >> > Thanks for sending some graphics and your MODE config
file. I
> do
> >>> > have
> >>> > > a
> >>> > > >> > couple of thoughts.
> >>> > > >> >
> >>> > > >> > First, do you know, are the values where there is no
snowfall
> >>> stored
> >>> > > as
> >>> > > >> 0's
> >>> > > >> > or missing data values?
> >>> > > >> >
> >>> > > >> > To define objects, MODE does a 2 step process... (1)
smoothing
> >>> the
> >>> > > data
> >>> > > >> > based on the conv_radius setting and (2) thresholding
the
> >>> smoothed
> >>> > > data
> >>> > > >> by
> >>> > > >> > applying the conv_thresh setting. If the no snowfall
points
> are
> >>> > stored
> >>> > > >> as
> >>> > > >> > 0, then all should be well. If it's stored as missing
data,
> then
> >>> > that
> >>> > > >> may
> >>> > > >> > wreak havoc on the smoothing step and cause unexpected
> results.
> >>> > > >> >
> >>> > > >> > Next, I'd recommend testing using "conv_radius = 0".
That'll
> >>> > > effectively
> >>> > > >> > disable the smoothing step. And the objects MODE
creates
> should
> >>> > > exactly
> >>> > > >> > match the contours of your plots.
> >>> > > >> >
> >>> > > >> > In general, I'd recommend against configuring MET to
read data
> >>> based
> >>> > > on
> >>> > > >> a
> >>> > > >> > record number:
> >>> > > >> > field = {
> >>> > > >> > name = "SNOD";
> >>> > > >> > level = "R1";
> >>> > > >> > };
> >>> > > >> > That's dangerous because it'll just use whatever data
is in
> >>> record
> >>> > > >> number 1
> >>> > > >> > regardless of whether or not it's actually SNOD data.
> >>> > > >> >
> >>> > > >> > If you'd like me to take a closer look, please just
send me
> (or
> >>> > point
> >>> > > me
> >>> > > >> > to) the data files used for the case you sent.
> >>> > > >> >
> >>> > > >> > Thanks,
> >>> > > >> > John Halley Gotway
> >>> > > >> >
> >>> > > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright -
NOAA
> >>> Affiliate
> >>> > via
> >>> > > >> RT <
> >>> > > >> > met_help at ucar.edu> wrote:
> >>> > > >> >
> >>> > > >> > >
> >>> > > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted
upon.
> >>> > > >> > > Transaction: Ticket created by
benjamin.albright at noaa.gov
> >>> > > >> > > Queue: met_help
> >>> > > >> > > Subject: MODE Threshold Issue
> >>> > > >> > > Owner: Nobody
> >>> > > >> > > Requestors: benjamin.albright at noaa.gov
> >>> > > >> > > Status: new
> >>> > > >> > > Ticket <URL:
> >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> >>> > > >> >
> >>> > > >> > >
> >>> > > >> > >
> >>> > > >> > > Hello MET Help:
> >>> > > >> > >
> >>> > > >> > > I'm having an issue with using MODE and having some
objects
> >>> appear
> >>> > > >> much
> >>> > > >> > > larger than what I expect they should be. I am
running MODE
> >>> (v9.0)
> >>> > > on
> >>> > > >> 24
> >>> > > >> > hr
> >>> > > >> > > snow accumulations. The first image is the 24 hr snow
> >>> accumulation
> >>> > > for
> >>> > > >> > > "Model A." The second image shows the MODE objects
for a
> >>> threshold
> >>> > > of
> >>> > > >> 1
> >>> > > >> > > inch. You can see the object extends too far into
Indiana,
> >>> Ohio,
> >>> > and
> >>> > > >> > > eastern Michigan than I would expect based on the
> >>> accumulation map
> >>> > > in
> >>> > > >> the
> >>> > > >> > > first image. The third attachment is my configuration
file
> >>> that I
> >>> > > >> used.
> >>> > > >> > The
> >>> > > >> > > data is being read from grib2 files.
> >>> > > >> > >
> >>> > > >> > > I've done a few tests. The first test was lowering
the first
> >>> > > >> threshold in
> >>> > > >> > > the snow accumulation graphic to 0.99" (fourth
attachment).
> >>> You
> >>> > can
> >>> > > >> with
> >>> > > >> > > that, the MODE object appears to match the light blue
0.99"
> >>> area.
> >>> > I
> >>> > > >> then
> >>> > > >> > > raised the threshold in my MODE code to 1.01." You
can see
> >>> that
> >>> > the
> >>> > > >> > object
> >>> > > >> > > now matches more closely with the expected 1 inch
> accumulation
> >>> > from
> >>> > > >> the
> >>> > > >> > > first image attachment. I was wondering if you might
know
> >>> what is
> >>> > > >> causing
> >>> > > >> > > this sensitivity and if there is a configuration
option I
> >>> could
> >>> > use
> >>> > > to
> >>> > > >> > > mitigate it? I've seen this behavior in other models
as well
> >>> so I
> >>> > > >> don't
> >>> > > >> > > believe it is input data related but haven't
completely
> ruled
> >>> that
> >>> > > >> out.
> >>> > > >> > If
> >>> > > >> > > you have any thoughts or suggestions I would really
> >>> appreciate it.
> >>> > > >> > >
> >>> > > >> > > Thank you,
> >>> > > >> > > Ben Albright
> >>> > > >> > >
> >>> > > >> > > --
> >>> > > >> > > Benjamin Albright, Phd.
> >>> > > >> > > Meteorological Developer
> >>> > > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
> >>> Center
> >>> > > >> > > Work email: benjamin.albright at noaa.gov
> >>> > > >> > > Work phone: 301-683-1474
> >>> > > >> > >
> >>> > > >> > >
> >>> > > >> >
> >>> > > >> >
> >>> > > >>
> >>> > > >> --
> >>> > > >> Benjamin Albright, Phd.
> >>> > > >> Meteorological Developer
> >>> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
> Center
> >>> > > >> Work email: benjamin.albright at noaa.gov
> >>> > > >> Work phone: 301-683-1474
> >>> > > >>
> >>> > > >>
> >>> > >
> >>> > >
> >>> >
> >>> > --
> >>> > Benjamin Albright, Phd.
> >>> > Meteorological Developer
> >>> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> >>> > Work email: benjamin.albright at noaa.gov
> >>> > Work phone: 301-683-1474
> >>> >
> >>> >
> >>>
> >>>
> >>
> >> --
> >> Benjamin Albright, Phd.
> >> Meteorological Developer
> >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> >> Work email: benjamin.albright at noaa.gov
> >> Work phone: 301-683-1474
> >>
> >
> >
> > --
> > Benjamin Albright, Phd.
> > Meteorological Developer
> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> > Work email: benjamin.albright at noaa.gov
> > Work phone: 301-683-1474
> >
>
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
>
------------------------------------------------
Subject: MODE Threshold Issue
From: Benjamin Albright - NOAA Affiliate
Time: Fri Feb 12 12:43:57 2021
Thanks again John for the help. I see what you mean when you use the
'plot_data_plane' tool and I got the same results. I guess my python
snow
accumulation code is maybe not accurate. That is what I was comparing
the
MODE image to and thinking there was a problem. For example, the first
image is a plot I generate in python using the same grib2 file. I
simply
open the file in python and read the data and put it in intervals
starting
at 1 and the area in Ohio and eastern Michigan is missing. When I
start my
plot at 0.99 I get the same image as plot_data_plane. So there must be
something happening in that code. I'll have to take a closer look at
it.
I'm not a big python expert but I'll see if there is something that
I'm
doing that is preventing those other values from plotting. Thanks
again for
your help and taking a look at this issue with me.
Ben
On Fri, Feb 12, 2021 at 6:19 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ben,
>
> Thanks for sending a version where Dx=Dy and taking that out of the
> equation. I think perhaps I'm raising an alarm that isn't a problem
at all.
> The MET plot_data_plane tool reads gridded data and plots that data
on
> exactly the same grid as the input.
>
> I'm guessing that with python plotting, that need not be the case.
You can
> read data from one grid/projection and easily render it using a
different
> projection. So I should stop worrying that your python grid is a
little
> larger than the one produced by plot_data_plane.
>
> So about data censoring, yes, you're right. We removed raw_thresh
since
> it's function is replaced by censor_thresh and censor_val. These
options
> are pretty straightforward. They are arrays that must be the same
length
> (as the error message states). If specified, then for each grid
point we
> check each threshold (censor_thresh) and if it meets the threshold
> criteria, replace the value with the corresponding censor_val entry.
>
> For example, let's say we define:
> censor_thresh = [ lt0, gt1 ]; censor_val = [ 0, 1 ];
>
> This reads through all grid points, replaces any values less than 0
with 0
> and any values greater than 1 with 1.
> To mimic "raw_thresh = 0.5", you'd set:
> censor_thresh = lt05;
> censor_val = 0.0;
>
> When the arrays are length 1, the square brackets are optional.
>
> But I'm just not seeing or appreciating the problem in the data
you're
> describing. I ran plot_data_plane on the GRIB file, resetting all
values
> between 0 and 1 to 0. Turns out that there aren't any values between
0 and
> 1 anway!
>
>
> *plot_data_plane nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
> nbmv4_wwe_2021020507F053_24hr_in_v2.ps 'name="SNOD"; level="Z0";
> censor_thresh=[gt0&<1]; censor_val=[0];' -v 3*
>
>
> *DEBUG 3: Applying censor thresholds ">0&&<1" and replacing with
values
> "0.00000".DEBUG 3: Censored values for 0 of 3744965 grid points.*
>
> See that 0 of the 3744965 input values were between 0 and 1. Here's
the
> resulting image:
>
> [image: Screen Shot 2021-02-12 at 11.12.44 AM.png]
>
> So anywhere that is gray is a value of 1 or higher. Comparing this
to the
> image you sent, I just don't see/understand the problem. When you
run this
> data through MODE, using "conv_thresh = 5" to smooth the raw values,
and
> then threshold the result "conv_thresh = 1", the resulting object
that
> includes Michigan looks totally fine to me. The smoothing step
causes those
> 0's in NW MI to pull down surrounding values and yields a reasonable
> object.
>
> If it'd help, we could schedule a quick telecon to discuss the
details, so
> I can better understand what's not looking right to you.
>
> Thanks,
> John
>
> On Fri, Feb 12, 2021 at 7:27 AM Benjamin Albright - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> >
> > Hi John,
> >
> > One final test I just completed. I used pcp_combine to make my
file that
> I
> > used in MODE. I got similar results. Slightly less coverage in
Michigan
> but
> > still basically about the same. I just wanted to try using the MET
tool
> to
> > make my input file rather than my other method as a test. Attached
is the
> > new image. Thanks again for your help.
> >
> > Ben
> >
> > On Fri, Feb 12, 2021 at 1:16 PM Benjamin Albright - NOAA Affiliate
<
> > benjamin.albright at noaa.gov> wrote:
> >
> > > Hi John,
> > >
> > > Another warning I receive that could be completely unrelated is
the
> > > following:
> > >
> > > WARNING:
> > > WARNING: Dictionary::lookup_num_array() -> numeric array lookup
failed
> > for
> > > name "censor_val"
> > > WARNING:
> > >
> > > When I upgraded my scripts to version 9.0 I replaced raw_thresh
with
> > > censor_val. In the help document, I also see there's a
censor_thresh.
> > When
> > > I replace censor_val with censor_thresh or add censor_thresh
with
> > > censor_val I get this warning:
> > >
> > > ERROR :
> > > ERROR : VarInfo::set_dict() -> The number of censor thresholds
in
> > > "censor_thresh" (1) must match the number of replacement values
in
> > > "censor_val" (0).
> > > ERROR :
> > >
> > > Again I'm not sure if this could be related to anything but just
wanted
> > > to let you know.
> > >
> > > Thanks,
> > > Ben
> > >
> > > On Fri, Feb 12, 2021 at 6:55 AM Benjamin Albright - NOAA
Affiliate <
> > > benjamin.albright at noaa.gov> wrote:
> > >
> > >> Hi John
> > >>
> > >> Thanks for the replies. I was able to go back to where I am
making the
> > >> files and make a couple changes in order to get the final files
to
> have
> > DX
> > >> = DY.
> > >>
> > >> *New model file:* wgrib2 -V
nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
> > >> 1:0:vt=2021020712:surface - surface:53 hour fcst:SNOD Snow
Depth [m]:
> > >> ndata=3744965:undef=0:mean=0.51457:min=0:max=36
> > >> grid_template=30:winds(grid):
> > >> Lambert Conformal: (2345 x 1597) input WE:SN output WE:SN res 8
> > >> Lat1 19.229000 Lon1 233.723396 LoV 265.000000
> > >> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> > >> LatSP -90.000000 LonSP 0.000000
> > >> North Pole (2345 x 1597) Dx 2539.703000 m Dy 2539.703000 m mode
8
> > >>
> > >> *New verification file:* wgrib2 -V nohrsc_2021020712_in.grib2
> > >> 1:0:vt=2021020712:surface - surface:anl:SNOD Snow Depth [m]:
> > >> ndata=2953665:undef=1654866:mean=0.514013:min=0:max=53
> > >> grid_template=30:winds(grid):
> > >> Lambert Conformal: (2145 x 1377) input WE:SN output WE:SN res 8
> > >> Lat1 20.191999 Lon1 238.445999 LoV 265.000000
> > >> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> > >> LatSP -90.000000 LonSP 0.000000
> > >> North Pole (2145 x 1377) Dx 2539.703000 m Dy 2539.703000 m mode
8
> > >>
> > >> This removed the warning when running MODE about DX != DY.
However,
> when
> > >> running MODE on this I still am getting the same result for the
one
> inch
> > >> threshold where the one inch object area is much larger. I've
attached
> > the
> > >> two new data files, the image, and config file.
> > >>
> > >> I haven't had a chance yet to go back and closely check my
python
> > >> plotting. I am using basemap in python to generate the images.
I'll
> keep
> > >> investigating but just wanted to let you know about this new
test I
> > tried
> > >> this morning.
> > >>
> > >> Thank you,
> > >> Ben
> > >>
> > >> On Thu, Feb 11, 2021 at 4:07 PM John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >>> Ben,
> > >>>
> > >>> I suspect the real problem is in the python code you're using
to plot
> > the
> > >>> data. I'd recommend using an alternative way of visualizing
GRIB2
> data
> > >>> that
> > >>> you trust. Compare their results. Check to see if your python
code is
> > >>> plotting the data in the right spot on the earth... and that
the
> extent
> > >>> of
> > >>> the grid is correct.
> > >>>
> > >>> Once you're confident in the accuracy of the python plots
being
> > created,
> > >>> recheck the extent of MODE objects to see if the issue still
remains.
> > >>> It's
> > >>> possible there's multiple things going on here.
> > >>>
> > >>> MET does not currently support Lambert Conformal grids where
the dx
> and
> > >>> dy
> > >>> values differ.
> > >>> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN
res 8
> > >>> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
> > >>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> > >>> LatSP -90.000000 LonSP 0.000000
> > >>> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000* m
mode 8
> > >>>
> > >>> It just uses the Dx value and prints the warning about Dy
being
> > >>> different.
> > >>>
> > >>> Another possible explanation is that your python script is
right and
> > MET,
> > >>> NCL, and IDV are all doing it wrong. I haven't run into many
Lambert
> > >>> grids
> > >>> where Dx != Dy.
> > >>>
> > >>> Regardless, perhaps I should write up a GitHub issue for us to
> enhance
> > >>> MET
> > >>> to support LC grids where Dx != Dy? Is this a common grid
that you
> > guys
> > >>> use and plan to continue using in the future?
> > >>>
> > >>> Thanks,
> > >>> John
> > >>>
> > >>> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA
Affiliate
> via
> > >>> RT <
> > >>> met_help at ucar.edu> wrote:
> > >>>
> > >>> >
> > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> > >>> >
> > >>> > Hi John,
> > >>> >
> > >>> > Thanks for the responses. I just want to make sure I'm
clear. Is
> that
> > >>> > warning MET displays about the projection a warning about
the
> > >>> projection
> > >>> > used in the grib2 data file? In other words, when I make the
grib2
> > >>> file,
> > >>> > the LCC grid that I am using is not compatible? Or do you
suspect
> the
> > >>> > problem lies in my actual plotting code? I am using basemap
to set
> up
> > >>> an
> > >>> > LCC projection of the map you see in my images. You may not
have an
> > >>> answer
> > >>> > but I am just trying to narrow down if I should focus on the
actual
> > >>> grib2
> > >>> > file or my Python plotting script when trying to come up
with a
> > >>> solution.
> > >>> >
> > >>> > Thanks,
> > >>> > Ben
> > >>> >
> > >>> > On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT <
> > >>> > met_help at ucar.edu>
> > >>> > wrote:
> > >>> >
> > >>> > > Ben,
> > >>> > >
> > >>> > > FYI, I tried one more method to plot this data. I used the
> > >>> > "ncl_convert2nc"
> > >>> > > tool to dump the values to NetCDF. And then I displayed
the
> result
> > >>> using
> > >>> > > ncview.
> > >>> > >
> > >>> > > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
> > >>> > > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
> > >>> > >
> > >>> > > I believe ncview displays the data using ONLY the lat/lon
values
> > for
> > >>> each
> > >>> > > grid point... and not any projection info. Layering on the
map
> data
> > >>> > > produces an image consistent with IDV and MET. I'm mostly
just
> > >>> checking
> > >>> > the
> > >>> > > extent of the map data in the corners (Victoria Island,
Nova
> > Scotia,
> > >>> and
> > >>> > > Cuba).
> > >>> > >
> > >>> > > So that also points to a potential problem in your python
> plotting
> > >>> code.
> > >>> > >
> > >>> > > Thanks,
> > >>> > > John
> > >>> > >
> > >>> > > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway <
> > johnhg at ucar.edu
> > >>> >
> > >>> > > wrote:
> > >>> > >
> > >>> > > > Ben,
> > >>> > > >
> > >>> > > > When I run the sample data you sent through MET's
> plot_data_plane
> > >>> > tool, I
> > >>> > > > get a rather telling warning message:
> > >>> > > >
> > >>> > > >
> > >>> > > > *plot_data_plane nbmv4_wwe_2021020707F053_24hr_in.grib2
> > >>> > > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD";
level="Z0";'
> > -v
> > >>> 4*
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid()
-> MET
> > does
> > >>> not
> > >>> > > > currently support Lambert Conformal grids where dx
(5.07862) !=
> > dy
> > >>> > > > (5.08049) and may produce unexpected results!WARNING: *
> > >>> > > >
> > >>> > > > I wonder if the unexpected extent of the MODE objects is
the
> > >>> > "*unexpected
> > >>> > > > results*" mentioned in that warning message?
> > >>> > > >
> > >>> > > > In the attached image, I've placed the image from
> plot_data_plane
> > >>> > (left)
> > >>> > > > next to the image from Unidata's IDV viewer (right).
Paying
> > >>> particular
> > >>> > > > attention to where the data lives relative to the
underlying
> map
> > >>> data.
> > >>> > I
> > >>> > > > was expecting to see a difference in where the raw data
lives
> > >>> between
> > >>> > > these
> > >>> > > > two images, but I do not. The relationship of the data
to the
> > Great
> > >>> > Lakes
> > >>> > > > looks very consistent. So this didn't reveal an obvious
> problem.
> > >>> > > >
> > >>> > > > But I do see some large differences when comparing to
the
> images
> > >>> you
> > >>> > > sent.
> > >>> > > > Looking in the top-left corner, both MET and IDV include
> roughly
> > >>> 90 of
> > >>> > > > Victoria Island. However the images you sent include ALL
of
> > >>> Victoria
> > >>> > > > Island, plus more! In the bottom right corner, MET and
IDV
> > include
> > >>> only
> > >>> > > the
> > >>> > > > northern half of Cuba whereas your plot includes almost
all of
> > it.
> > >>> > > >
> > >>> > > > I'd recommend checking the code you're using to create
those
> > >>> plots. Do
> > >>> > > you
> > >>> > > > have an independent way of plotting this data that you
trust to
> > >>> figure
> > >>> > > out
> > >>> > > > if there's a problem in the grid or projection being
used in
> your
> > >>> > > plotting?
> > >>> > > >
> > >>> > > > Thanks,
> > >>> > > > John
> > >>> > > >
> > >>> > > >
> > >>> > > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
> > >>> > > >
> > >>> > > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright - NOAA
> > Affiliate
> > >>> via
> > >>> > > RT <
> > >>> > > > met_help at ucar.edu> wrote:
> > >>> > > >
> > >>> > > >>
> > >>> > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> >
> > >>> > > >>
> > >>> > > >> Hi John,
> > >>> > > >>
> > >>> > > >> Thanks for your reply and the suggestions. As far as
missing
> > >>> data, I'm
> > >>> > > not
> > >>> > > >> 100% sure how it is stored in this particular file we
are
> > working
> > >>> > with.
> > >>> > > I
> > >>> > > >> asked a co-worker and he said he has not seen missing
values
> in
> > >>> this
> > >>> > > >> particular file before and that in some of the other
files we
> > use,
> > >>> > > missing
> > >>> > > >> values are usually relegated to the edges of the grid
and
> stored
> > >>> as
> > >>> > > 10^36.
> > >>> > > >> Also thanks for the warning about having MODE read data
via
> > record
> > >>> > > number.
> > >>> > > >> I typically only use files that have one record to read
but
> that
> > >>> is
> > >>> > good
> > >>> > > >> to
> > >>> > > >> know for the future.
> > >>> > > >>
> > >>> > > >> As for testing I tried setting a convolution radius to
0. I've
> > >>> > attached
> > >>> > > >> that image. You can see the model object still appears
too
> > large.
> > >>> I've
> > >>> > > >> attached the data file I am using as well as the config
file
> > >>> again.
> > >>> > > Please
> > >>> > > >> let me know if you have any other suggestions or see
anything
> > >>> wrong
> > >>> > with
> > >>> > > >> the data file or config file. I appreciate the help.
> > >>> > > >>
> > >>> > > >> Thanks,
> > >>> > > >> Ben
> > >>> > > >>
> > >>> > > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway via
RT <
> > >>> > > >> met_help at ucar.edu>
> > >>> > > >> wrote:
> > >>> > > >>
> > >>> > > >> > Hi Ben,
> > >>> > > >> >
> > >>> > > >> > I see you have some questions about sensitivity of
> thresholds
> > in
> > >>> > MODE.
> > >>> > > >> > Thanks for sending some graphics and your MODE config
file.
> I
> > do
> > >>> > have
> > >>> > > a
> > >>> > > >> > couple of thoughts.
> > >>> > > >> >
> > >>> > > >> > First, do you know, are the values where there is no
> snowfall
> > >>> stored
> > >>> > > as
> > >>> > > >> 0's
> > >>> > > >> > or missing data values?
> > >>> > > >> >
> > >>> > > >> > To define objects, MODE does a 2 step process... (1)
> smoothing
> > >>> the
> > >>> > > data
> > >>> > > >> > based on the conv_radius setting and (2) thresholding
the
> > >>> smoothed
> > >>> > > data
> > >>> > > >> by
> > >>> > > >> > applying the conv_thresh setting. If the no snowfall
points
> > are
> > >>> > stored
> > >>> > > >> as
> > >>> > > >> > 0, then all should be well. If it's stored as missing
data,
> > then
> > >>> > that
> > >>> > > >> may
> > >>> > > >> > wreak havoc on the smoothing step and cause
unexpected
> > results.
> > >>> > > >> >
> > >>> > > >> > Next, I'd recommend testing using "conv_radius = 0".
That'll
> > >>> > > effectively
> > >>> > > >> > disable the smoothing step. And the objects MODE
creates
> > should
> > >>> > > exactly
> > >>> > > >> > match the contours of your plots.
> > >>> > > >> >
> > >>> > > >> > In general, I'd recommend against configuring MET to
read
> data
> > >>> based
> > >>> > > on
> > >>> > > >> a
> > >>> > > >> > record number:
> > >>> > > >> > field = {
> > >>> > > >> > name = "SNOD";
> > >>> > > >> > level = "R1";
> > >>> > > >> > };
> > >>> > > >> > That's dangerous because it'll just use whatever data
is in
> > >>> record
> > >>> > > >> number 1
> > >>> > > >> > regardless of whether or not it's actually SNOD data.
> > >>> > > >> >
> > >>> > > >> > If you'd like me to take a closer look, please just
send me
> > (or
> > >>> > point
> > >>> > > me
> > >>> > > >> > to) the data files used for the case you sent.
> > >>> > > >> >
> > >>> > > >> > Thanks,
> > >>> > > >> > John Halley Gotway
> > >>> > > >> >
> > >>> > > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright -
NOAA
> > >>> Affiliate
> > >>> > via
> > >>> > > >> RT <
> > >>> > > >> > met_help at ucar.edu> wrote:
> > >>> > > >> >
> > >>> > > >> > >
> > >>> > > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted
upon.
> > >>> > > >> > > Transaction: Ticket created by
benjamin.albright at noaa.gov
> > >>> > > >> > > Queue: met_help
> > >>> > > >> > > Subject: MODE Threshold Issue
> > >>> > > >> > > Owner: Nobody
> > >>> > > >> > > Requestors: benjamin.albright at noaa.gov
> > >>> > > >> > > Status: new
> > >>> > > >> > > Ticket <URL:
> > >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> > >>> > > >> >
> > >>> > > >> > >
> > >>> > > >> > >
> > >>> > > >> > > Hello MET Help:
> > >>> > > >> > >
> > >>> > > >> > > I'm having an issue with using MODE and having some
> objects
> > >>> appear
> > >>> > > >> much
> > >>> > > >> > > larger than what I expect they should be. I am
running
> MODE
> > >>> (v9.0)
> > >>> > > on
> > >>> > > >> 24
> > >>> > > >> > hr
> > >>> > > >> > > snow accumulations. The first image is the 24 hr
snow
> > >>> accumulation
> > >>> > > for
> > >>> > > >> > > "Model A." The second image shows the MODE objects
for a
> > >>> threshold
> > >>> > > of
> > >>> > > >> 1
> > >>> > > >> > > inch. You can see the object extends too far into
Indiana,
> > >>> Ohio,
> > >>> > and
> > >>> > > >> > > eastern Michigan than I would expect based on the
> > >>> accumulation map
> > >>> > > in
> > >>> > > >> the
> > >>> > > >> > > first image. The third attachment is my
configuration file
> > >>> that I
> > >>> > > >> used.
> > >>> > > >> > The
> > >>> > > >> > > data is being read from grib2 files.
> > >>> > > >> > >
> > >>> > > >> > > I've done a few tests. The first test was lowering
the
> first
> > >>> > > >> threshold in
> > >>> > > >> > > the snow accumulation graphic to 0.99" (fourth
> attachment).
> > >>> You
> > >>> > can
> > >>> > > >> with
> > >>> > > >> > > that, the MODE object appears to match the light
blue
> 0.99"
> > >>> area.
> > >>> > I
> > >>> > > >> then
> > >>> > > >> > > raised the threshold in my MODE code to 1.01." You
can see
> > >>> that
> > >>> > the
> > >>> > > >> > object
> > >>> > > >> > > now matches more closely with the expected 1 inch
> > accumulation
> > >>> > from
> > >>> > > >> the
> > >>> > > >> > > first image attachment. I was wondering if you
might know
> > >>> what is
> > >>> > > >> causing
> > >>> > > >> > > this sensitivity and if there is a configuration
option I
> > >>> could
> > >>> > use
> > >>> > > to
> > >>> > > >> > > mitigate it? I've seen this behavior in other
models as
> well
> > >>> so I
> > >>> > > >> don't
> > >>> > > >> > > believe it is input data related but haven't
completely
> > ruled
> > >>> that
> > >>> > > >> out.
> > >>> > > >> > If
> > >>> > > >> > > you have any thoughts or suggestions I would really
> > >>> appreciate it.
> > >>> > > >> > >
> > >>> > > >> > > Thank you,
> > >>> > > >> > > Ben Albright
> > >>> > > >> > >
> > >>> > > >> > > --
> > >>> > > >> > > Benjamin Albright, Phd.
> > >>> > > >> > > Meteorological Developer
> > >>> > > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
> Prediction
> > >>> Center
> > >>> > > >> > > Work email: benjamin.albright at noaa.gov
> > >>> > > >> > > Work phone: 301-683-1474
> > >>> > > >> > >
> > >>> > > >> > >
> > >>> > > >> >
> > >>> > > >> >
> > >>> > > >>
> > >>> > > >> --
> > >>> > > >> Benjamin Albright, Phd.
> > >>> > > >> Meteorological Developer
> > >>> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
> > Center
> > >>> > > >> Work email: benjamin.albright at noaa.gov
> > >>> > > >> Work phone: 301-683-1474
> > >>> > > >>
> > >>> > > >>
> > >>> > >
> > >>> > >
> > >>> >
> > >>> > --
> > >>> > Benjamin Albright, Phd.
> > >>> > Meteorological Developer
> > >>> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > >>> > Work email: benjamin.albright at noaa.gov
> > >>> > Work phone: 301-683-1474
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>
> > >> --
> > >> Benjamin Albright, Phd.
> > >> Meteorological Developer
> > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > >> Work email: benjamin.albright at noaa.gov
> > >> Work phone: 301-683-1474
> > >>
> > >
> > >
> > > --
> > > Benjamin Albright, Phd.
> > > Meteorological Developer
> > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > > Work email: benjamin.albright at noaa.gov
> > > Work phone: 301-683-1474
> > >
> >
> >
> > --
> > Benjamin Albright, Phd.
> > Meteorological Developer
> > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> > Work email: benjamin.albright at noaa.gov
> > Work phone: 301-683-1474
> >
> >
>
>
--
Benjamin Albright, Phd.
Meteorological Developer
Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
Work email: benjamin.albright at noaa.gov
Work phone: 301-683-1474
------------------------------------------------
Subject: MODE Threshold Issue
From: John Halley Gotway
Time: Mon Feb 15 11:58:53 2021
Ben,
OK, sounds good. I'll go ahead and resolve this ticket. But as you
investigate, if more questions come up about MODE, please feel free to
write a new email to met-help.
Thanks,
John
On Fri, Feb 12, 2021 at 12:44 PM Benjamin Albright - NOAA Affiliate
via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
>
> Thanks again John for the help. I see what you mean when you use
the
> 'plot_data_plane' tool and I got the same results. I guess my python
snow
> accumulation code is maybe not accurate. That is what I was
comparing the
> MODE image to and thinking there was a problem. For example, the
first
> image is a plot I generate in python using the same grib2 file. I
simply
> open the file in python and read the data and put it in intervals
starting
> at 1 and the area in Ohio and eastern Michigan is missing. When I
start my
> plot at 0.99 I get the same image as plot_data_plane. So there must
be
> something happening in that code. I'll have to take a closer look at
it.
> I'm not a big python expert but I'll see if there is something that
I'm
> doing that is preventing those other values from plotting. Thanks
again for
> your help and taking a look at this issue with me.
>
> Ben
>
> On Fri, Feb 12, 2021 at 6:19 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ben,
> >
> > Thanks for sending a version where Dx=Dy and taking that out of
the
> > equation. I think perhaps I'm raising an alarm that isn't a
problem at
> all.
> > The MET plot_data_plane tool reads gridded data and plots that
data on
> > exactly the same grid as the input.
> >
> > I'm guessing that with python plotting, that need not be the case.
You
> can
> > read data from one grid/projection and easily render it using a
different
> > projection. So I should stop worrying that your python grid is a
little
> > larger than the one produced by plot_data_plane.
> >
> > So about data censoring, yes, you're right. We removed raw_thresh
since
> > it's function is replaced by censor_thresh and censor_val. These
options
> > are pretty straightforward. They are arrays that must be the same
length
> > (as the error message states). If specified, then for each grid
point we
> > check each threshold (censor_thresh) and if it meets the threshold
> > criteria, replace the value with the corresponding censor_val
entry.
> >
> > For example, let's say we define:
> > censor_thresh = [ lt0, gt1 ]; censor_val = [ 0, 1 ];
> >
> > This reads through all grid points, replaces any values less than
0 with
> 0
> > and any values greater than 1 with 1.
> > To mimic "raw_thresh = 0.5", you'd set:
> > censor_thresh = lt05;
> > censor_val = 0.0;
> >
> > When the arrays are length 1, the square brackets are optional.
> >
> > But I'm just not seeing or appreciating the problem in the data
you're
> > describing. I ran plot_data_plane on the GRIB file, resetting all
values
> > between 0 and 1 to 0. Turns out that there aren't any values
between 0
> and
> > 1 anway!
> >
> >
> > *plot_data_plane nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
> > nbmv4_wwe_2021020507F053_24hr_in_v2.ps 'name="SNOD"; level="Z0";
> > censor_thresh=[gt0&<1]; censor_val=[0];' -v 3*
> >
> >
> > *DEBUG 3: Applying censor thresholds ">0&&<1" and replacing with
values
> > "0.00000".DEBUG 3: Censored values for 0 of 3744965 grid points.*
> >
> > See that 0 of the 3744965 input values were between 0 and 1.
Here's the
> > resulting image:
> >
> > [image: Screen Shot 2021-02-12 at 11.12.44 AM.png]
> >
> > So anywhere that is gray is a value of 1 or higher. Comparing this
to the
> > image you sent, I just don't see/understand the problem. When you
run
> this
> > data through MODE, using "conv_thresh = 5" to smooth the raw
values, and
> > then threshold the result "conv_thresh = 1", the resulting object
that
> > includes Michigan looks totally fine to me. The smoothing step
causes
> those
> > 0's in NW MI to pull down surrounding values and yields a
reasonable
> > object.
> >
> > If it'd help, we could schedule a quick telecon to discuss the
details,
> so
> > I can better understand what's not looking right to you.
> >
> > Thanks,
> > John
> >
> > On Fri, Feb 12, 2021 at 7:27 AM Benjamin Albright - NOAA Affiliate
via
> RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> > >
> > > Hi John,
> > >
> > > One final test I just completed. I used pcp_combine to make my
file
> that
> > I
> > > used in MODE. I got similar results. Slightly less coverage in
Michigan
> > but
> > > still basically about the same. I just wanted to try using the
MET tool
> > to
> > > make my input file rather than my other method as a test.
Attached is
> the
> > > new image. Thanks again for your help.
> > >
> > > Ben
> > >
> > > On Fri, Feb 12, 2021 at 1:16 PM Benjamin Albright - NOAA
Affiliate <
> > > benjamin.albright at noaa.gov> wrote:
> > >
> > > > Hi John,
> > > >
> > > > Another warning I receive that could be completely unrelated
is the
> > > > following:
> > > >
> > > > WARNING:
> > > > WARNING: Dictionary::lookup_num_array() -> numeric array
lookup
> failed
> > > for
> > > > name "censor_val"
> > > > WARNING:
> > > >
> > > > When I upgraded my scripts to version 9.0 I replaced
raw_thresh with
> > > > censor_val. In the help document, I also see there's a
censor_thresh.
> > > When
> > > > I replace censor_val with censor_thresh or add censor_thresh
with
> > > > censor_val I get this warning:
> > > >
> > > > ERROR :
> > > > ERROR : VarInfo::set_dict() -> The number of censor
thresholds in
> > > > "censor_thresh" (1) must match the number of replacement
values in
> > > > "censor_val" (0).
> > > > ERROR :
> > > >
> > > > Again I'm not sure if this could be related to anything but
just
> wanted
> > > > to let you know.
> > > >
> > > > Thanks,
> > > > Ben
> > > >
> > > > On Fri, Feb 12, 2021 at 6:55 AM Benjamin Albright - NOAA
Affiliate <
> > > > benjamin.albright at noaa.gov> wrote:
> > > >
> > > >> Hi John
> > > >>
> > > >> Thanks for the replies. I was able to go back to where I am
making
> the
> > > >> files and make a couple changes in order to get the final
files to
> > have
> > > DX
> > > >> = DY.
> > > >>
> > > >> *New model file:* wgrib2 -V
> nbmv4_wwe_2021020507F053_24hr_in_v2.grib2
> > > >> 1:0:vt=2021020712:surface - surface:53 hour fcst:SNOD Snow
Depth
> [m]:
> > > >> ndata=3744965:undef=0:mean=0.51457:min=0:max=36
> > > >> grid_template=30:winds(grid):
> > > >> Lambert Conformal: (2345 x 1597) input WE:SN output WE:SN res
8
> > > >> Lat1 19.229000 Lon1 233.723396 LoV 265.000000
> > > >> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> > > >> LatSP -90.000000 LonSP 0.000000
> > > >> North Pole (2345 x 1597) Dx 2539.703000 m Dy 2539.703000 m
mode 8
> > > >>
> > > >> *New verification file:* wgrib2 -V nohrsc_2021020712_in.grib2
> > > >> 1:0:vt=2021020712:surface - surface:anl:SNOD Snow Depth [m]:
> > > >> ndata=2953665:undef=1654866:mean=0.514013:min=0:max=53
> > > >> grid_template=30:winds(grid):
> > > >> Lambert Conformal: (2145 x 1377) input WE:SN output WE:SN res
8
> > > >> Lat1 20.191999 Lon1 238.445999 LoV 265.000000
> > > >> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> > > >> LatSP -90.000000 LonSP 0.000000
> > > >> North Pole (2145 x 1377) Dx 2539.703000 m Dy 2539.703000 m
mode 8
> > > >>
> > > >> This removed the warning when running MODE about DX != DY.
However,
> > when
> > > >> running MODE on this I still am getting the same result for
the one
> > inch
> > > >> threshold where the one inch object area is much larger. I've
> attached
> > > the
> > > >> two new data files, the image, and config file.
> > > >>
> > > >> I haven't had a chance yet to go back and closely check my
python
> > > >> plotting. I am using basemap in python to generate the
images. I'll
> > keep
> > > >> investigating but just wanted to let you know about this new
test I
> > > tried
> > > >> this morning.
> > > >>
> > > >> Thank you,
> > > >> Ben
> > > >>
> > > >> On Thu, Feb 11, 2021 at 4:07 PM John Halley Gotway via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >>> Ben,
> > > >>>
> > > >>> I suspect the real problem is in the python code you're
using to
> plot
> > > the
> > > >>> data. I'd recommend using an alternative way of visualizing
GRIB2
> > data
> > > >>> that
> > > >>> you trust. Compare their results. Check to see if your
python code
> is
> > > >>> plotting the data in the right spot on the earth... and that
the
> > extent
> > > >>> of
> > > >>> the grid is correct.
> > > >>>
> > > >>> Once you're confident in the accuracy of the python plots
being
> > > created,
> > > >>> recheck the extent of MODE objects to see if the issue still
> remains.
> > > >>> It's
> > > >>> possible there's multiple things going on here.
> > > >>>
> > > >>> MET does not currently support Lambert Conformal grids where
the dx
> > and
> > > >>> dy
> > > >>> values differ.
> > > >>> Lambert Conformal: (1073 x 689) input WE:SN output WE:SN
res 8
> > > >>> Lat1 20.190001 Lon1 238.449996 LoV 265.000000
> > > >>> LatD 25.000000 Latin1 25.000000 Latin2 25.000000
> > > >>> LatSP -90.000000 LonSP 0.000000
> > > >>> North Pole (1073 x 689) *Dx 5078.616000 m Dy 5080.491000*
m
> mode 8
> > > >>>
> > > >>> It just uses the Dx value and prints the warning about Dy
being
> > > >>> different.
> > > >>>
> > > >>> Another possible explanation is that your python script is
right
> and
> > > MET,
> > > >>> NCL, and IDV are all doing it wrong. I haven't run into many
> Lambert
> > > >>> grids
> > > >>> where Dx != Dy.
> > > >>>
> > > >>> Regardless, perhaps I should write up a GitHub issue for us
to
> > enhance
> > > >>> MET
> > > >>> to support LC grids where Dx != Dy? Is this a common grid
that you
> > > guys
> > > >>> use and plan to continue using in the future?
> > > >>>
> > > >>> Thanks,
> > > >>> John
> > > >>>
> > > >>> On Thu, Feb 11, 2021 at 12:29 PM Benjamin Albright - NOAA
Affiliate
> > via
> > > >>> RT <
> > > >>> met_help at ucar.edu> wrote:
> > > >>>
> > > >>> >
> > > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641 >
> > > >>> >
> > > >>> > Hi John,
> > > >>> >
> > > >>> > Thanks for the responses. I just want to make sure I'm
clear. Is
> > that
> > > >>> > warning MET displays about the projection a warning about
the
> > > >>> projection
> > > >>> > used in the grib2 data file? In other words, when I make
the
> grib2
> > > >>> file,
> > > >>> > the LCC grid that I am using is not compatible? Or do you
suspect
> > the
> > > >>> > problem lies in my actual plotting code? I am using
basemap to
> set
> > up
> > > >>> an
> > > >>> > LCC projection of the map you see in my images. You may
not have
> an
> > > >>> answer
> > > >>> > but I am just trying to narrow down if I should focus on
the
> actual
> > > >>> grib2
> > > >>> > file or my Python plotting script when trying to come up
with a
> > > >>> solution.
> > > >>> >
> > > >>> > Thanks,
> > > >>> > Ben
> > > >>> >
> > > >>> > On Thu, Feb 11, 2021 at 2:06 PM John Halley Gotway via RT
<
> > > >>> > met_help at ucar.edu>
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Ben,
> > > >>> > >
> > > >>> > > FYI, I tried one more method to plot this data. I used
the
> > > >>> > "ncl_convert2nc"
> > > >>> > > tool to dump the values to NetCDF. And then I displayed
the
> > result
> > > >>> using
> > > >>> > > ncview.
> > > >>> > >
> > > >>> > > ncl_convert2nc nbmv4_wwe_2021020707F053_24hr_in.grib2
> > > >>> > > ncview nbmv4_wwe_2021020707F053_24hr_in.nc
> > > >>> > >
> > > >>> > > I believe ncview displays the data using ONLY the
lat/lon
> values
> > > for
> > > >>> each
> > > >>> > > grid point... and not any projection info. Layering on
the map
> > data
> > > >>> > > produces an image consistent with IDV and MET. I'm
mostly just
> > > >>> checking
> > > >>> > the
> > > >>> > > extent of the map data in the corners (Victoria Island,
Nova
> > > Scotia,
> > > >>> and
> > > >>> > > Cuba).
> > > >>> > >
> > > >>> > > So that also points to a potential problem in your
python
> > plotting
> > > >>> code.
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > > John
> > > >>> > >
> > > >>> > > [image: Screen Shot 2021-02-11 at 12.04.53 PM.png]
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > On Thu, Feb 11, 2021 at 11:45 AM John Halley Gotway <
> > > johnhg at ucar.edu
> > > >>> >
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > Ben,
> > > >>> > > >
> > > >>> > > > When I run the sample data you sent through MET's
> > plot_data_plane
> > > >>> > tool, I
> > > >>> > > > get a rather telling warning message:
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > *plot_data_plane
nbmv4_wwe_2021020707F053_24hr_in.grib2
> > > >>> > > > nbmv4_wwe_2021020707F053_24hr_in.ps 'name="SNOD";
> level="Z0";'
> > > -v
> > > >>> 4*
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > *WARNING: WARNING: MetGrib2DataFile::read_grib2_grid()
-> MET
> > > does
> > > >>> not
> > > >>> > > > currently support Lambert Conformal grids where dx
(5.07862)
> !=
> > > dy
> > > >>> > > > (5.08049) and may produce unexpected results!WARNING:
*
> > > >>> > > >
> > > >>> > > > I wonder if the unexpected extent of the MODE objects
is the
> > > >>> > "*unexpected
> > > >>> > > > results*" mentioned in that warning message?
> > > >>> > > >
> > > >>> > > > In the attached image, I've placed the image from
> > plot_data_plane
> > > >>> > (left)
> > > >>> > > > next to the image from Unidata's IDV viewer (right).
Paying
> > > >>> particular
> > > >>> > > > attention to where the data lives relative to the
underlying
> > map
> > > >>> data.
> > > >>> > I
> > > >>> > > > was expecting to see a difference in where the raw
data lives
> > > >>> between
> > > >>> > > these
> > > >>> > > > two images, but I do not. The relationship of the data
to the
> > > Great
> > > >>> > Lakes
> > > >>> > > > looks very consistent. So this didn't reveal an
obvious
> > problem.
> > > >>> > > >
> > > >>> > > > But I do see some large differences when comparing to
the
> > images
> > > >>> you
> > > >>> > > sent.
> > > >>> > > > Looking in the top-left corner, both MET and IDV
include
> > roughly
> > > >>> 90 of
> > > >>> > > > Victoria Island. However the images you sent include
ALL of
> > > >>> Victoria
> > > >>> > > > Island, plus more! In the bottom right corner, MET and
IDV
> > > include
> > > >>> only
> > > >>> > > the
> > > >>> > > > northern half of Cuba whereas your plot includes
almost all
> of
> > > it.
> > > >>> > > >
> > > >>> > > > I'd recommend checking the code you're using to create
those
> > > >>> plots. Do
> > > >>> > > you
> > > >>> > > > have an independent way of plotting this data that you
trust
> to
> > > >>> figure
> > > >>> > > out
> > > >>> > > > if there's a problem in the grid or projection being
used in
> > your
> > > >>> > > plotting?
> > > >>> > > >
> > > >>> > > > Thanks,
> > > >>> > > > John
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > [image: Screen Shot 2021-02-11 at 11.37.54 AM.png]
> > > >>> > > >
> > > >>> > > > On Thu, Feb 11, 2021 at 4:28 AM Benjamin Albright -
NOAA
> > > Affiliate
> > > >>> via
> > > >>> > > RT <
> > > >>> > > > met_help at ucar.edu> wrote:
> > > >>> > > >
> > > >>> > > >>
> > > >>> > > >> <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> > >
> > > >>> > > >>
> > > >>> > > >> Hi John,
> > > >>> > > >>
> > > >>> > > >> Thanks for your reply and the suggestions. As far as
missing
> > > >>> data, I'm
> > > >>> > > not
> > > >>> > > >> 100% sure how it is stored in this particular file we
are
> > > working
> > > >>> > with.
> > > >>> > > I
> > > >>> > > >> asked a co-worker and he said he has not seen missing
values
> > in
> > > >>> this
> > > >>> > > >> particular file before and that in some of the other
files
> we
> > > use,
> > > >>> > > missing
> > > >>> > > >> values are usually relegated to the edges of the grid
and
> > stored
> > > >>> as
> > > >>> > > 10^36.
> > > >>> > > >> Also thanks for the warning about having MODE read
data via
> > > record
> > > >>> > > number.
> > > >>> > > >> I typically only use files that have one record to
read but
> > that
> > > >>> is
> > > >>> > good
> > > >>> > > >> to
> > > >>> > > >> know for the future.
> > > >>> > > >>
> > > >>> > > >> As for testing I tried setting a convolution radius
to 0.
> I've
> > > >>> > attached
> > > >>> > > >> that image. You can see the model object still
appears too
> > > large.
> > > >>> I've
> > > >>> > > >> attached the data file I am using as well as the
config file
> > > >>> again.
> > > >>> > > Please
> > > >>> > > >> let me know if you have any other suggestions or see
> anything
> > > >>> wrong
> > > >>> > with
> > > >>> > > >> the data file or config file. I appreciate the help.
> > > >>> > > >>
> > > >>> > > >> Thanks,
> > > >>> > > >> Ben
> > > >>> > > >>
> > > >>> > > >> On Wed, Feb 10, 2021 at 8:25 PM John Halley Gotway
via RT <
> > > >>> > > >> met_help at ucar.edu>
> > > >>> > > >> wrote:
> > > >>> > > >>
> > > >>> > > >> > Hi Ben,
> > > >>> > > >> >
> > > >>> > > >> > I see you have some questions about sensitivity of
> > thresholds
> > > in
> > > >>> > MODE.
> > > >>> > > >> > Thanks for sending some graphics and your MODE
config
> file.
> > I
> > > do
> > > >>> > have
> > > >>> > > a
> > > >>> > > >> > couple of thoughts.
> > > >>> > > >> >
> > > >>> > > >> > First, do you know, are the values where there is
no
> > snowfall
> > > >>> stored
> > > >>> > > as
> > > >>> > > >> 0's
> > > >>> > > >> > or missing data values?
> > > >>> > > >> >
> > > >>> > > >> > To define objects, MODE does a 2 step process...
(1)
> > smoothing
> > > >>> the
> > > >>> > > data
> > > >>> > > >> > based on the conv_radius setting and (2)
thresholding the
> > > >>> smoothed
> > > >>> > > data
> > > >>> > > >> by
> > > >>> > > >> > applying the conv_thresh setting. If the no
snowfall
> points
> > > are
> > > >>> > stored
> > > >>> > > >> as
> > > >>> > > >> > 0, then all should be well. If it's stored as
missing
> data,
> > > then
> > > >>> > that
> > > >>> > > >> may
> > > >>> > > >> > wreak havoc on the smoothing step and cause
unexpected
> > > results.
> > > >>> > > >> >
> > > >>> > > >> > Next, I'd recommend testing using "conv_radius =
0".
> That'll
> > > >>> > > effectively
> > > >>> > > >> > disable the smoothing step. And the objects MODE
creates
> > > should
> > > >>> > > exactly
> > > >>> > > >> > match the contours of your plots.
> > > >>> > > >> >
> > > >>> > > >> > In general, I'd recommend against configuring MET
to read
> > data
> > > >>> based
> > > >>> > > on
> > > >>> > > >> a
> > > >>> > > >> > record number:
> > > >>> > > >> > field = {
> > > >>> > > >> > name = "SNOD";
> > > >>> > > >> > level = "R1";
> > > >>> > > >> > };
> > > >>> > > >> > That's dangerous because it'll just use whatever
data is
> in
> > > >>> record
> > > >>> > > >> number 1
> > > >>> > > >> > regardless of whether or not it's actually SNOD
data.
> > > >>> > > >> >
> > > >>> > > >> > If you'd like me to take a closer look, please just
send
> me
> > > (or
> > > >>> > point
> > > >>> > > me
> > > >>> > > >> > to) the data files used for the case you sent.
> > > >>> > > >> >
> > > >>> > > >> > Thanks,
> > > >>> > > >> > John Halley Gotway
> > > >>> > > >> >
> > > >>> > > >> > On Wed, Feb 10, 2021 at 1:09 PM Benjamin Albright -
NOAA
> > > >>> Affiliate
> > > >>> > via
> > > >>> > > >> RT <
> > > >>> > > >> > met_help at ucar.edu> wrote:
> > > >>> > > >> >
> > > >>> > > >> > >
> > > >>> > > >> > > Wed Feb 10 13:08:49 2021: Request 98641 was acted
upon.
> > > >>> > > >> > > Transaction: Ticket created by
> benjamin.albright at noaa.gov
> > > >>> > > >> > > Queue: met_help
> > > >>> > > >> > > Subject: MODE Threshold Issue
> > > >>> > > >> > > Owner: Nobody
> > > >>> > > >> > > Requestors: benjamin.albright at noaa.gov
> > > >>> > > >> > > Status: new
> > > >>> > > >> > > Ticket <URL:
> > > >>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98641
> > > >>> > > >> >
> > > >>> > > >> > >
> > > >>> > > >> > >
> > > >>> > > >> > > Hello MET Help:
> > > >>> > > >> > >
> > > >>> > > >> > > I'm having an issue with using MODE and having
some
> > objects
> > > >>> appear
> > > >>> > > >> much
> > > >>> > > >> > > larger than what I expect they should be. I am
running
> > MODE
> > > >>> (v9.0)
> > > >>> > > on
> > > >>> > > >> 24
> > > >>> > > >> > hr
> > > >>> > > >> > > snow accumulations. The first image is the 24 hr
snow
> > > >>> accumulation
> > > >>> > > for
> > > >>> > > >> > > "Model A." The second image shows the MODE
objects for a
> > > >>> threshold
> > > >>> > > of
> > > >>> > > >> 1
> > > >>> > > >> > > inch. You can see the object extends too far into
> Indiana,
> > > >>> Ohio,
> > > >>> > and
> > > >>> > > >> > > eastern Michigan than I would expect based on the
> > > >>> accumulation map
> > > >>> > > in
> > > >>> > > >> the
> > > >>> > > >> > > first image. The third attachment is my
configuration
> file
> > > >>> that I
> > > >>> > > >> used.
> > > >>> > > >> > The
> > > >>> > > >> > > data is being read from grib2 files.
> > > >>> > > >> > >
> > > >>> > > >> > > I've done a few tests. The first test was
lowering the
> > first
> > > >>> > > >> threshold in
> > > >>> > > >> > > the snow accumulation graphic to 0.99" (fourth
> > attachment).
> > > >>> You
> > > >>> > can
> > > >>> > > >> with
> > > >>> > > >> > > that, the MODE object appears to match the light
blue
> > 0.99"
> > > >>> area.
> > > >>> > I
> > > >>> > > >> then
> > > >>> > > >> > > raised the threshold in my MODE code to 1.01."
You can
> see
> > > >>> that
> > > >>> > the
> > > >>> > > >> > object
> > > >>> > > >> > > now matches more closely with the expected 1 inch
> > > accumulation
> > > >>> > from
> > > >>> > > >> the
> > > >>> > > >> > > first image attachment. I was wondering if you
might
> know
> > > >>> what is
> > > >>> > > >> causing
> > > >>> > > >> > > this sensitivity and if there is a configuration
option
> I
> > > >>> could
> > > >>> > use
> > > >>> > > to
> > > >>> > > >> > > mitigate it? I've seen this behavior in other
models as
> > well
> > > >>> so I
> > > >>> > > >> don't
> > > >>> > > >> > > believe it is input data related but haven't
completely
> > > ruled
> > > >>> that
> > > >>> > > >> out.
> > > >>> > > >> > If
> > > >>> > > >> > > you have any thoughts or suggestions I would
really
> > > >>> appreciate it.
> > > >>> > > >> > >
> > > >>> > > >> > > Thank you,
> > > >>> > > >> > > Ben Albright
> > > >>> > > >> > >
> > > >>> > > >> > > --
> > > >>> > > >> > > Benjamin Albright, Phd.
> > > >>> > > >> > > Meteorological Developer
> > > >>> > > >> > > Systems Research Group, Inc. at NOAA/NWS/Weather
> > Prediction
> > > >>> Center
> > > >>> > > >> > > Work email: benjamin.albright at noaa.gov
> > > >>> > > >> > > Work phone: 301-683-1474
> > > >>> > > >> > >
> > > >>> > > >> > >
> > > >>> > > >> >
> > > >>> > > >> >
> > > >>> > > >>
> > > >>> > > >> --
> > > >>> > > >> Benjamin Albright, Phd.
> > > >>> > > >> Meteorological Developer
> > > >>> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
> > > Center
> > > >>> > > >> Work email: benjamin.albright at noaa.gov
> > > >>> > > >> Work phone: 301-683-1474
> > > >>> > > >>
> > > >>> > > >>
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> > --
> > > >>> > Benjamin Albright, Phd.
> > > >>> > Meteorological Developer
> > > >>> > Systems Research Group, Inc. at NOAA/NWS/Weather
Prediction
> Center
> > > >>> > Work email: benjamin.albright at noaa.gov
> > > >>> > Work phone: 301-683-1474
> > > >>> >
> > > >>> >
> > > >>>
> > > >>>
> > > >>
> > > >> --
> > > >> Benjamin Albright, Phd.
> > > >> Meteorological Developer
> > > >> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > > >> Work email: benjamin.albright at noaa.gov
> > > >> Work phone: 301-683-1474
> > > >>
> > > >
> > > >
> > > > --
> > > > Benjamin Albright, Phd.
> > > > Meteorological Developer
> > > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > > > Work email: benjamin.albright at noaa.gov
> > > > Work phone: 301-683-1474
> > > >
> > >
> > >
> > > --
> > > Benjamin Albright, Phd.
> > > Meteorological Developer
> > > Systems Research Group, Inc. at NOAA/NWS/Weather Prediction
Center
> > > Work email: benjamin.albright at noaa.gov
> > > Work phone: 301-683-1474
> > >
> > >
> >
> >
>
> --
> Benjamin Albright, Phd.
> Meteorological Developer
> Systems Research Group, Inc. at NOAA/NWS/Weather Prediction Center
> Work email: benjamin.albright at noaa.gov
> Work phone: 301-683-1474
>
>
------------------------------------------------
More information about the Met_help
mailing list