[Met_help] [rt.rap.ucar.edu #61428] History for Odd MODE result
John Halley Gotway via RT
met_help at ucar.edu
Fri May 24 10:29:55 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Could you help diagnose what is causing the odd part of the object shown in the attached .ps file?
Part of the object follows completely around the square border of the model domain.
I cannot determine why the area around the border is being diagnosed as an object.
I've attached the original gridded observations file which was also used as the forecast file as a test.
See the attached config file we used.
Thanks.
R/
John
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
From: John Halley Gotway
Time: Thu May 16 15:25:48 2013
Hello John,
We actually received a MET-Help question last week about this same
issue. There's a small bug in MODE causing areas of missing data to
be treated as values of 0 during the convolution step. We've
fixed the issue in the development version of the code, and it will be
included in the next release. We had not planned on posting a bugfix
for this for METv4.0 since there's a workaround I'll
describe below. But if it would be helpful to you to have a bugfix, I
could work one up next week.
The workaround is to edit the MODE configuration file by setting:
raw_thresh = >=-9999;
That avoids this problem. However, METv4.1 will not have this problem
even when using the default raw_thresh of >=0.0.
Here's the other user's question about the same issue:
http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
Hope that helps.
Thanks,
John
On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>
> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: Odd MODE result
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
>
> Could you help diagnose what is causing the odd part of the object
shown in the attached .ps file?
>
> Part of the object follows completely around the square border of
the model domain.
>
> I cannot determine why the area around the border is being diagnosed
as an object.
>
> I've attached the original gridded observations file which was also
used as the forecast file as a test.
>
> See the attached config file we used.
>
> Thanks.
>
> R/
> John
>
>
------------------------------------------------
Subject: Odd MODE result
From: Raby, John W USA CIV
Time: Thu May 16 15:47:18 2013
Classification: UNCLASSIFIED
Caveats: NONE
John -
Thanks for your quick response. We will implement the change using the
workaround.
Why would there be missing data? Does this mean that our input grid
has
missing data?
Do you have a feel for the release date for METV4.1?
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, May 16, 2013 3:26 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Vaucher, Gail T CIV (US)
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
Hello John,
We actually received a MET-Help question last week about this same
issue.
There's a small bug in MODE causing areas of missing data to be
treated as
values of 0 during the convolution step. We've fixed the issue in the
development version of the code, and it will be included in the next
release.
We had not planned on posting a bugfix for this for METv4.0 since
there's a
workaround I'll describe below. But if it would be helpful to you to
have a
bugfix, I could work one up next week.
The workaround is to edit the MODE configuration file by setting:
raw_thresh = >=-9999;
That avoids this problem. However, METv4.1 will not have this problem
even
when using the default raw_thresh of >=0.0.
Here's the other user's question about the same issue:
http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
Hope that helps.
Thanks,
John
On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>
> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: Odd MODE result
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
> >
>
>
> Could you help diagnose what is causing the odd part of the object
shown in
> the attached .ps file?
>
> Part of the object follows completely around the square border of
the model
> domain.
>
> I cannot determine why the area around the border is being diagnosed
as an
> object.
>
> I've attached the original gridded observations file which was also
used as
> the forecast file as a test.
>
> See the attached config file we used.
>
> Thanks.
>
> R/
> John
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
From: John Halley Gotway
Time: Thu May 16 16:44:28 2013
John,
The MODE algorithm for identifying objects in the fields breaks down
if the objects go right up against the edge of the domain. To avoid
this, MODE sets the outer row and column to bad data prior to
identifying objects (see "zero_border_size" in the MODE config file).
It's this band of bad data values around the edge that are then being
treated as 0's instead of bad data. So the data values
near the edge of the domain are being averaged with these values of 0,
pulling down the convolved values. When you're doing less-than type
thresholding, that odd ring shows up.
Setting the raw_thresh = >=-9999; prevents the bad data getting
replaced by 0.
The METv4.1 data is getting close. I spent this week testing a beta
version of the code using 9 different platform/compiler combinations.
I've uncovered an issue on one of them that I'm trying to
track down. Of course, it only showed up on the 9th test, and not any
prior!
So basically we need to finishing up testing and documentation before
the release.
Thanks,
John
On 05/16/2013 03:47 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for your quick response. We will implement the change using
the
> workaround.
>
> Why would there be missing data? Does this mean that our input grid
has
> missing data?
>
> Do you have a feel for the release date for METV4.1?
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, May 16, 2013 3:26 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Vaucher, Gail T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
>
> Hello John,
>
> We actually received a MET-Help question last week about this same
issue.
> There's a small bug in MODE causing areas of missing data to be
treated as
> values of 0 during the convolution step. We've fixed the issue in
the
> development version of the code, and it will be included in the next
release.
> We had not planned on posting a bugfix for this for METv4.0 since
there's a
> workaround I'll describe below. But if it would be helpful to you
to have a
> bugfix, I could work one up next week.
>
> The workaround is to edit the MODE configuration file by setting:
> raw_thresh = >=-9999;
>
> That avoids this problem. However, METv4.1 will not have this
problem even
> when using the default raw_thresh of >=0.0.
>
> Here's the other user's question about the same issue:
> http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
>
> Hope that helps.
>
> Thanks,
> John
>
>
> On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>>
>> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Odd MODE result
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
>>>
>>
>>
>> Could you help diagnose what is causing the odd part of the object
shown in
>> the attached .ps file?
>>
>> Part of the object follows completely around the square border of
the model
>> domain.
>>
>> I cannot determine why the area around the border is being
diagnosed as an
>> object.
>>
>> I've attached the original gridded observations file which was also
used as
>> the forecast file as a test.
>>
>> See the attached config file we used.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: Odd MODE result
From: Raby, John W USA CIV
Time: Fri May 17 07:25:37 2013
John -
Thanks for the explanation. Very helpful. One lesson from this, which
I recall from discussions I''ve had with both yourself and others
there, is that you should be careful to run MODE so that you don't end
up with objects that touch the boundaries. This should avoid the
problem of the break-down of the algorithm per your email below.
Here at ARL, Gail Vaucher and I are using MODE to define and diagnose
objects in continuous variable fields such as 2m AGL temparature, RH
and 10m wind speed. We are working with 2.5 km res WRF output and 2.5
km res RTMA gridded observations. What we are finding, somewhat
frequently, is that the objects we define using >= and <= end up at
one geographic extreme or the other of the domain and on the edge. We
are in the beginning phase of our investigation and are in the process
of refininng the config file settings so that we can better define
objects which are in the middle of the domain and preferably of a size
which is small relative to the domain size.
Have you ever considered allowing the user to define an object using
>= AND(logical) <= to isolate objects which are not at one extreme or
the other? Seems like this would define objects which are just as
valid in continuous variable fields.
Understand about the challenges of testing before release. Better to
find the error before the release than to discover them after wards in
the hands of the users and having to expend significantly more effort
to put the genie back in the bottle and fix, then redistribute.
Good luck on the testing and release.
R/
John
________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, May 16, 2013 4:44 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Vaucher, Gail T CIV (US)
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
John,
The MODE algorithm for identifying objects in the fields breaks down
if the objects go right up against the edge of the domain. To avoid
this, MODE sets the outer row and column to bad data prior to
identifying objects (see "zero_border_size" in the MODE config file).
It's this band of bad data values around the edge that are then being
treated as 0's instead of bad data. So the data values
near the edge of the domain are being averaged with these values of 0,
pulling down the convolved values. When you're doing less-than type
thresholding, that odd ring shows up.
Setting the raw_thresh = >=-9999; prevents the bad data getting
replaced by 0.
The METv4.1 data is getting close. I spent this week testing a beta
version of the code using 9 different platform/compiler combinations.
I've uncovered an issue on one of them that I'm trying to
track down. Of course, it only showed up on the 9th test, and not any
prior!
So basically we need to finishing up testing and documentation before
the release.
Thanks,
John
On 05/16/2013 03:47 PM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for your quick response. We will implement the change using
the
> workaround.
>
> Why would there be missing data? Does this mean that our input grid
has
> missing data?
>
> Do you have a feel for the release date for METV4.1?
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, May 16, 2013 3:26 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Vaucher, Gail T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
>
> Hello John,
>
> We actually received a MET-Help question last week about this same
issue.
> There's a small bug in MODE causing areas of missing data to be
treated as
> values of 0 during the convolution step. We've fixed the issue in
the
> development version of the code, and it will be included in the next
release.
> We had not planned on posting a bugfix for this for METv4.0 since
there's a
> workaround I'll describe below. But if it would be helpful to you
to have a
> bugfix, I could work one up next week.
>
> The workaround is to edit the MODE configuration file by setting:
> raw_thresh = >=-9999;
>
> That avoids this problem. However, METv4.1 will not have this
problem even
> when using the default raw_thresh of >=0.0.
>
> Here's the other user's question about the same issue:
> http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
>
> Hope that helps.
>
> Thanks,
> John
>
>
> On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>>
>> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Odd MODE result
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
>>>
>>
>>
>> Could you help diagnose what is causing the odd part of the object
shown in
>> the attached .ps file?
>>
>> Part of the object follows completely around the square border of
the model
>> domain.
>>
>> I cannot determine why the area around the border is being
diagnosed as an
>> object.
>>
>> I've attached the original gridded observations file which was also
used as
>> the forecast file as a test.
>>
>> See the attached config file we used.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: Odd MODE result
From: Raby, John W USA CIV
Time: Wed May 22 09:09:48 2013
Classification: UNCLASSIFIED
Caveats: NONE
John -
Here at ARL, Gail Vaucher and I are using MODE to define and diagnose
objects in continuous variable fields such as 2m AGL temperature, RH
and 10m
wind speed. We are working with 2.5 km res WRF output and 2.5 km res
RTMA
gridded observations. What we are finding, somewhat frequently, is
that the
objects we define using the conv_thresh settings of >= and <= end up
at one
geographic extreme or the other of the domain and on the edge. We are
in the
beginning phase of our investigation and are in the process of
refining the
config file settings so that we can better define objects which are in
the
middle of the domain and preferably of a size which is small relative
to the
domain size.
Have you ever considered allowing the user to define an object using
>=
AND(logical) <= to isolate objects in the mid-range of the variable
and are
likely to not be located at one geographic extreme or the other? Seems
like
this would define objects which are just as valid in continuous
variable
fields.
Thanks.
R/
John
-----Original Message-----
From: Raby, John W CIV USARMY ARL (US)
Sent: Friday, May 17, 2013 10:47 AM
To: Vaucher, Gail T CIV (US)
Cc: Raby, John W CIV USARMY ARL (US)
Subject: FW: [rt.rap.ucar.edu #61428] Odd MODE result
Gail -
If you have some time, could you try using the workaround suggested by
John
Gotway below? I'm tied up with other work this morning and probably
won't be
able to meet today.
Also, you might try and find a <= conv_thresh setting which produces
an
object which doesn't touch the edge to see if this prevent the
"border"
object from occurring. Same for a >=.
That said, having the workaround is probably not the real solution
since we
shouldn't be working with objects which touch the edge anyway. At
least
that's the guidance they gave me in the past. It's probably better to
tweak
the conv_thresh until the resulting object clears the edge. There may
be
other settings in the config file which we can use to "improve" our
objects,
but this is pretty complex and I don't know to tell you where to start
any
sensitivity testing to see which changes do what? Maybe it's best to
read
the documentation on the theory behind each of those settings to
understand
what they are doing.
Thanks for all you ideas on improving the scripts and processes!
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, May 16, 2013 3:25 PM
To: Raby, John W CIV USARMY ARL (US)
Cc: Vaucher, Gail T CIV (US)
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
Hello John,
We actually received a MET-Help question last week about this same
issue.
There's a small bug in MODE causing areas of missing data to be
treated as
values of 0 during the convolution step. We've fixed the issue in the
development version of the code, and it will be included in the next
release. We had not planned on posting a bugfix for this for METv4.0
since
there's a workaround I'll describe below. But if it would be helpful
to you
to have a bugfix, I could work one up next week.
The workaround is to edit the MODE configuration file by setting:
raw_thresh = >=-9999;
That avoids this problem. However, METv4.1 will not have this problem
even
when using the default raw_thresh of >=0.0.
Here's the other user's question about the same issue:
http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
Hope that helps.
Thanks,
John
On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>
> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: Odd MODE result
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
> >
>
>
> Could you help diagnose what is causing the odd part of the object
shown
in the attached .ps file?
>
> Part of the object follows completely around the square border of
the
model domain.
>
> I cannot determine why the area around the border is being diagnosed
as an
object.
>
> I've attached the original gridded observations file which was also
used
as the forecast file as a test.
>
> See the attached config file we used.
>
> Thanks.
>
> R/
> John
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed May 22 09:34:45 2013
John,
To be honest, that suggestion has not come up yet. We use MODE to
identify and quantify "areas of interest", and typically, we're
interested in one extreme or the other, rather than the middle range.
To what field are you applying MODE that you'd like to be able to
threshold >= AND <=?
We have had one request in the past to identify objects at both ends
of the extremes by thresholding <= OR >=. They were looking at
pressure and wanted to quantify areas of very high and very low
pressure. They decided to just run MODE twice - once for high
pressure objects and once time for low pressure objects.
John
On 05/22/2013 09:09 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Here at ARL, Gail Vaucher and I are using MODE to define and
diagnose
> objects in continuous variable fields such as 2m AGL temperature, RH
and 10m
> wind speed. We are working with 2.5 km res WRF output and 2.5 km res
RTMA
> gridded observations. What we are finding, somewhat frequently, is
that the
> objects we define using the conv_thresh settings of >= and <= end up
at one
> geographic extreme or the other of the domain and on the edge. We
are in the
> beginning phase of our investigation and are in the process of
refining the
> config file settings so that we can better define objects which are
in the
> middle of the domain and preferably of a size which is small
relative to the
> domain size.
>
> Have you ever considered allowing the user to define an object using
>=
> AND(logical) <= to isolate objects in the mid-range of the variable
and are
> likely to not be located at one geographic extreme or the other?
Seems like
> this would define objects which are just as valid in continuous
variable
> fields.
>
> Thanks.
>
> R/
> John
>
> -----Original Message-----
> From: Raby, John W CIV USARMY ARL (US)
> Sent: Friday, May 17, 2013 10:47 AM
> To: Vaucher, Gail T CIV (US)
> Cc: Raby, John W CIV USARMY ARL (US)
> Subject: FW: [rt.rap.ucar.edu #61428] Odd MODE result
>
> Gail -
>
> If you have some time, could you try using the workaround suggested
by John
> Gotway below? I'm tied up with other work this morning and probably
won't be
> able to meet today.
>
> Also, you might try and find a <= conv_thresh setting which produces
an
> object which doesn't touch the edge to see if this prevent the
"border"
> object from occurring. Same for a >=.
>
> That said, having the workaround is probably not the real solution
since we
> shouldn't be working with objects which touch the edge anyway. At
least
> that's the guidance they gave me in the past. It's probably better
to tweak
> the conv_thresh until the resulting object clears the edge. There
may be
> other settings in the config file which we can use to "improve" our
objects,
> but this is pretty complex and I don't know to tell you where to
start any
> sensitivity testing to see which changes do what? Maybe it's best to
read
> the documentation on the theory behind each of those settings to
understand
> what they are doing.
>
> Thanks for all you ideas on improving the scripts and processes!
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, May 16, 2013 3:25 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Vaucher, Gail T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
>
> Hello John,
>
> We actually received a MET-Help question last week about this same
issue.
> There's a small bug in MODE causing areas of missing data to be
treated as
> values of 0 during the convolution step. We've fixed the issue in
the
> development version of the code, and it will be included in the next
> release. We had not planned on posting a bugfix for this for
METv4.0 since
> there's a workaround I'll describe below. But if it would be
helpful to you
> to have a bugfix, I could work one up next week.
>
> The workaround is to edit the MODE configuration file by setting:
> raw_thresh = >=-9999;
>
> That avoids this problem. However, METv4.1 will not have this
problem even
> when using the default raw_thresh of >=0.0.
>
> Here's the other user's question about the same issue:
> http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
>
> Hope that helps.
>
> Thanks,
> John
>
>
> On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>>
>> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Odd MODE result
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
>>>
>>
>>
>> Could you help diagnose what is causing the odd part of the object
shown
> in the attached .ps file?
>>
>> Part of the object follows completely around the square border of
the
> model domain.
>>
>> I cannot determine why the area around the border is being
diagnosed as an
> object.
>>
>> I've attached the original gridded observations file which was also
used
> as the forecast file as a test.
>>
>> See the attached config file we used.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: Odd MODE result
From: gail.t.vaucher.civ at mail.mil
Time: Wed May 22 09:43:47 2013
Classification: UNCLASSIFIED
Caveats: NONE
John(s),
Gail here. The variables will include TMP, WINDS, DPT and the U-
component of
the wind. For testing purposes, John and I are starting with TMP.
The range
has a 5-10 degree spread, since our area of interest is relatively
small.
Gail
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 22, 2013 9:35 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Vaucher, Gail T CIV (US)
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
John,
To be honest, that suggestion has not come up yet. We use MODE to
identify
and quantify "areas of interest", and typically, we're interested in
one
extreme or the other, rather than the middle range.
To what field are you applying MODE that you'd like to be able to
threshold
>= AND <=?
We have had one request in the past to identify objects at both ends
of the
extremes by thresholding <= OR >=. They were looking at pressure and
wanted
to quantify areas of very high and very low pressure. They decided to
just
run MODE twice - once for high pressure objects and once time for low
pressure
objects.
John
On 05/22/2013 09:09 AM, Raby, John W USA CIV via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Here at ARL, Gail Vaucher and I are using MODE to define and
diagnose
> objects in continuous variable fields such as 2m AGL temperature, RH
> and 10m wind speed. We are working with 2.5 km res WRF output and
2.5
> km res RTMA gridded observations. What we are finding, somewhat
> frequently, is that the objects we define using the conv_thresh
> settings of >= and <= end up at one geographic extreme or the other
of
> the domain and on the edge. We are in the beginning phase of our
> investigation and are in the process of refining the config file
> settings so that we can better define objects which are in the
middle
> of the domain and preferably of a size which is small relative to
the domain
> size.
>
> Have you ever considered allowing the user to define an object using
> >=
> AND(logical) <= to isolate objects in the mid-range of the variable
> and are likely to not be located at one geographic extreme or the
> other? Seems like this would define objects which are just as valid
in
> continuous variable fields.
>
> Thanks.
>
> R/
> John
>
> -----Original Message-----
> From: Raby, John W CIV USARMY ARL (US)
> Sent: Friday, May 17, 2013 10:47 AM
> To: Vaucher, Gail T CIV (US)
> Cc: Raby, John W CIV USARMY ARL (US)
> Subject: FW: [rt.rap.ucar.edu #61428] Odd MODE result
>
> Gail -
>
> If you have some time, could you try using the workaround suggested
by
> John Gotway below? I'm tied up with other work this morning and
> probably won't be able to meet today.
>
> Also, you might try and find a <= conv_thresh setting which produces
> an object which doesn't touch the edge to see if this prevent the
"border"
> object from occurring. Same for a >=.
>
> That said, having the workaround is probably not the real solution
> since we shouldn't be working with objects which touch the edge
> anyway. At least that's the guidance they gave me in the past. It's
> probably better to tweak the conv_thresh until the resulting object
> clears the edge. There may be other settings in the config file
which
> we can use to "improve" our objects, but this is pretty complex and
I
> don't know to tell you where to start any sensitivity testing to see
> which changes do what? Maybe it's best to read the documentation on
> the theory behind each of those settings to understand what they are
doing.
>
> Thanks for all you ideas on improving the scripts and processes!
>
> R/
> John
>
>
> Mr John W. Raby, Meteorologist
>
> U.S. Army Research Laboratory
>
> White Sands Missile Range, NM 88002
>
> (575) 678-2004 DSN 258-2004
>
> FAX (575) 678-1230 DSN 258-1230
>
> Email: john.w.raby2.civ at mail.mil
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, May 16, 2013 3:25 PM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Vaucher, Gail T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
>
> Hello John,
>
> We actually received a MET-Help question last week about this same
issue.
> There's a small bug in MODE causing areas of missing data to be
> treated as values of 0 during the convolution step. We've fixed the
> issue in the development version of the code, and it will be
included
> in the next release. We had not planned on posting a bugfix for
this
> for METv4.0 since there's a workaround I'll describe below. But if
it
> would be helpful to you to have a bugfix, I could work one up next
week.
>
> The workaround is to edit the MODE configuration file by setting:
> raw_thresh = >=-9999;
>
> That avoids this problem. However, METv4.1 will not have this
problem
> even when using the default raw_thresh of >=0.0.
>
> Here's the other user's question about the same issue:
> http://mailman.ucar.edu/pipermail/met_help/2013-May/001937.html
>
> Hope that helps.
>
> Thanks,
> John
>
>
> On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>>
>> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>> Queue: met_help
>> Subject: Odd MODE result
>> Owner: Nobody
>> Requestors: john.w.raby2.civ at mail.mil
>> Status: new
>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
>>>
>>
>>
>> Could you help diagnose what is causing the odd part of the object
>> shown
> in the attached .ps file?
>>
>> Part of the object follows completely around the square border of
the
> model domain.
>>
>> I cannot determine why the area around the border is being
diagnosed
>> as an
> object.
>>
>> I've attached the original gridded observations file which was also
>> used
> as the forecast file as a test.
>>
>> See the attached config file we used.
>>
>> Thanks.
>>
>> R/
>> John
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
Subject: Odd MODE result
From: John Halley Gotway
Time: Wed May 22 10:49:04 2013
Gail,
Out of curiosity, why are you interested in this 5-10 degree range of
temperature?
Here's something you could try now - suppose you're interested only in
the range of 293 to 296 K (I chose this range for the example I'm
sending):
raw_thresh = <=296;
conv_radius = 0;
conv_thresh = >=293;
The "raw_thresh" of <=296 will set any values greater than 296 to 0.
The "conv_radius" of 0 will skip the convolution step.
The "conv_thresh" of >=293 will define objects that are greater than
293.
This will effectively define objects between 293 and 296 degrees.
Now, why did I skip the convolution step? I've artificially
introduced a bunch of 0's. If I were to convolve them with the values
nearby, they'd have a big effect on their neighbor's values.
See my attached example. I've basically created objects with all of
the raw values that are colored yellow.
I realize this is convoluted, but it's something you could do now.
I'll discuss better options with the MET team.
Thanks,
John
On 05/22/2013 09:43 AM, gail.t.vaucher.civ at mail.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John(s),
>
> Gail here. The variables will include TMP, WINDS, DPT and the U-
component of
> the wind. For testing purposes, John and I are starting with TMP.
The range
> has a 5-10 degree spread, since our area of interest is relatively
small.
>
> Gail
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2013 9:35 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Vaucher, Gail T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
>
> John,
>
> To be honest, that suggestion has not come up yet. We use MODE to
identify
> and quantify "areas of interest", and typically, we're interested in
one
> extreme or the other, rather than the middle range.
> To what field are you applying MODE that you'd like to be able to
threshold
> >= AND <=?
>
> We have had one request in the past to identify objects at both ends
of the
> extremes by thresholding <= OR >=. They were looking at pressure
and wanted
> to quantify areas of very high and very low pressure. They decided
to just
> run MODE twice - once for high pressure objects and once time for
low pressure
> objects.
>
> John
>
> On 05/22/2013 09:09 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Here at ARL, Gail Vaucher and I are using MODE to define and
diagnose
>> objects in continuous variable fields such as 2m AGL temperature,
RH
>> and 10m wind speed. We are working with 2.5 km res WRF output and
2.5
>> km res RTMA gridded observations. What we are finding, somewhat
>> frequently, is that the objects we define using the conv_thresh
>> settings of >= and <= end up at one geographic extreme or the other
of
>> the domain and on the edge. We are in the beginning phase of our
>> investigation and are in the process of refining the config file
>> settings so that we can better define objects which are in the
middle
>> of the domain and preferably of a size which is small relative to
the domain
>> size.
>>
>> Have you ever considered allowing the user to define an object
using
>>> =
>> AND(logical) <= to isolate objects in the mid-range of the variable
>> and are likely to not be located at one geographic extreme or the
>> other? Seems like this would define objects which are just as valid
in
>> continuous variable fields.
>>
>> Thanks.
>>
>> R/
>> John
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Friday, May 17, 2013 10:47 AM
>> To: Vaucher, Gail T CIV (US)
>> Cc: Raby, John W CIV USARMY ARL (US)
>> Subject: FW: [rt.rap.ucar.edu #61428] Odd MODE result
>>
>> Gail -
>>
>> If you have some time, could you try using the workaround suggested
by
>> John Gotway below? I'm tied up with other work this morning and
>> probably won't be able to meet today.
>>
>> Also, you might try and find a <= conv_thresh setting which
produces
>> an object which doesn't touch the edge to see if this prevent the
"border"
>> object from occurring. Same for a >=.
>>
>> That said, having the workaround is probably not the real solution
>> since we shouldn't be working with objects which touch the edge
>> anyway. At least that's the guidance they gave me in the past. It's
>> probably better to tweak the conv_thresh until the resulting object
>> clears the edge. There may be other settings in the config file
which
>> we can use to "improve" our objects, but this is pretty complex and
I
>> don't know to tell you where to start any sensitivity testing to
see
>> which changes do what? Maybe it's best to read the documentation on
>> the theory behind each of those settings to understand what they
are doing.
>>
>> Thanks for all you ideas on improving the scripts and processes!
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Thursday, May 16, 2013 3:25 PM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Vaucher, Gail T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
>>
>> Hello John,
>>
>> We actually received a MET-Help question last week about this same
issue.
>> There's a small bug in MODE causing areas of missing data to be
>> treated as values of 0 during the convolution step. We've fixed
the
>> issue in the development version of the code, and it will be
included
>> in the next release. We had not planned on posting a bugfix for
this
>> for METv4.0 since there's a workaround I'll describe below. But if
it
>> would be helpful to you to have a bugfix, I could work one up next
week.
>>
>> The workaround is to edit the MODE configuration file by setting:
>> raw_thresh = >=-9999;
>>
>> That avoids this problem. However, METv4.1 will not have this
problem
>> even when using the default raw_thresh of >=0.0.
>>
>> Here's the other user's question about the same issue:
>> http://mailman.ucar.edu/pipermail/met_help/2013-
May/001937.html
>>
>> Hope that helps.
>>
>> Thanks,
>> John
>>
>>
>> On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>>>
>>> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Odd MODE result
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
>>>>
>>>
>>>
>>> Could you help diagnose what is causing the odd part of the object
>>> shown
>> in the attached .ps file?
>>>
>>> Part of the object follows completely around the square border of
the
>> model domain.
>>>
>>> I cannot determine why the area around the border is being
diagnosed
>>> as an
>> object.
>>>
>>> I've attached the original gridded observations file which was
also
>>> used
>> as the forecast file as a test.
>>>
>>> See the attached config file we used.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
------------------------------------------------
Subject: Odd MODE result
From: gail.t.vaucher.civ at mail.mil
Time: Wed May 22 11:21:27 2013
Classification: UNCLASSIFIED
Caveats: NONE
Hi John (H),
That's a very clever work around. We'll implement in this afternoon's
runs.
To your question...John (R) and I are working with a very high
resolution
(1km) area of interest, which narrows the overall range of variable
magnitudes.
BTW - the raw_thresh = >=-9999, worked like a champ! Thanks!
Gail
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 22, 2013 10:49 AM
To: Raby, John W CIV USARMY ARL (US)
Cc: Vaucher, Gail T CIV (US)
Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
Gail,
Out of curiosity, why are you interested in this 5-10 degree range of
temperature?
Here's something you could try now - suppose you're interested only in
the
range of 293 to 296 K (I chose this range for the example I'm
sending):
raw_thresh = <=296;
conv_radius = 0;
conv_thresh = >=293;
The "raw_thresh" of <=296 will set any values greater than 296 to 0.
The "conv_radius" of 0 will skip the convolution step.
The "conv_thresh" of >=293 will define objects that are greater than
293.
This will effectively define objects between 293 and 296 degrees.
Now, why
did I skip the convolution step? I've artificially introduced a bunch
of 0's.
If I were to convolve them with the values nearby, they'd have a big
effect on
their neighbor's values.
See my attached example. I've basically created objects with all of
the raw
values that are colored yellow.
I realize this is convoluted, but it's something you could do now.
I'll discuss better options with the MET team.
Thanks,
John
On 05/22/2013 09:43 AM, gail.t.vaucher.civ at mail.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John(s),
>
> Gail here. The variables will include TMP, WINDS, DPT and the
> U-component of the wind. For testing purposes, John and I are
> starting with TMP. The range has a 5-10 degree spread, since our
area of
> interest is relatively small.
>
> Gail
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2013 9:35 AM
> To: Raby, John W CIV USARMY ARL (US)
> Cc: Vaucher, Gail T CIV (US)
> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result (UNCLASSIFIED)
>
> John,
>
> To be honest, that suggestion has not come up yet. We use MODE to
> identify and quantify "areas of interest", and typically, we're
> interested in one extreme or the other, rather than the middle
range.
> To what field are you applying MODE that you'd like to be able to
> threshold
> >= AND <=?
>
> We have had one request in the past to identify objects at both ends
> of the extremes by thresholding <= OR >=. They were looking at
> pressure and wanted to quantify areas of very high and very low
> pressure. They decided to just run MODE twice - once for high
> pressure objects and once time for low pressure objects.
>
> John
>
> On 05/22/2013 09:09 AM, Raby, John W USA CIV via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428 >
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> Here at ARL, Gail Vaucher and I are using MODE to define and
diagnose
>> objects in continuous variable fields such as 2m AGL temperature,
RH
>> and 10m wind speed. We are working with 2.5 km res WRF output and
2.5
>> km res RTMA gridded observations. What we are finding, somewhat
>> frequently, is that the objects we define using the conv_thresh
>> settings of >= and <= end up at one geographic extreme or the other
>> of the domain and on the edge. We are in the beginning phase of our
>> investigation and are in the process of refining the config file
>> settings so that we can better define objects which are in the
middle
>> of the domain and preferably of a size which is small relative to
the
>> domain size.
>>
>> Have you ever considered allowing the user to define an object
using
>>> =
>> AND(logical) <= to isolate objects in the mid-range of the variable
>> and are likely to not be located at one geographic extreme or the
>> other? Seems like this would define objects which are just as valid
>> in continuous variable fields.
>>
>> Thanks.
>>
>> R/
>> John
>>
>> -----Original Message-----
>> From: Raby, John W CIV USARMY ARL (US)
>> Sent: Friday, May 17, 2013 10:47 AM
>> To: Vaucher, Gail T CIV (US)
>> Cc: Raby, John W CIV USARMY ARL (US)
>> Subject: FW: [rt.rap.ucar.edu #61428] Odd MODE result
>>
>> Gail -
>>
>> If you have some time, could you try using the workaround suggested
>> by John Gotway below? I'm tied up with other work this morning and
>> probably won't be able to meet today.
>>
>> Also, you might try and find a <= conv_thresh setting which
produces
>> an object which doesn't touch the edge to see if this prevent the
"border"
>> object from occurring. Same for a >=.
>>
>> That said, having the workaround is probably not the real solution
>> since we shouldn't be working with objects which touch the edge
>> anyway. At least that's the guidance they gave me in the past. It's
>> probably better to tweak the conv_thresh until the resulting object
>> clears the edge. There may be other settings in the config file
which
>> we can use to "improve" our objects, but this is pretty complex and
I
>> don't know to tell you where to start any sensitivity testing to
see
>> which changes do what? Maybe it's best to read the documentation on
>> the theory behind each of those settings to understand what they
are doing.
>>
>> Thanks for all you ideas on improving the scripts and processes!
>>
>> R/
>> John
>>
>>
>> Mr John W. Raby, Meteorologist
>>
>> U.S. Army Research Laboratory
>>
>> White Sands Missile Range, NM 88002
>>
>> (575) 678-2004 DSN 258-2004
>>
>> FAX (575) 678-1230 DSN 258-1230
>>
>> Email: john.w.raby2.civ at mail.mil
>>
>> ________________________________________
>> From: John Halley Gotway via RT [met_help at ucar.edu]
>> Sent: Thursday, May 16, 2013 3:25 PM
>> To: Raby, John W CIV USARMY ARL (US)
>> Cc: Vaucher, Gail T CIV (US)
>> Subject: Re: [rt.rap.ucar.edu #61428] Odd MODE result
>>
>> Hello John,
>>
>> We actually received a MET-Help question last week about this same
issue.
>> There's a small bug in MODE causing areas of missing data to be
>> treated as values of 0 during the convolution step. We've fixed
the
>> issue in the development version of the code, and it will be
included
>> in the next release. We had not planned on posting a bugfix for
this
>> for METv4.0 since there's a workaround I'll describe below. But if
>> it would be helpful to you to have a bugfix, I could work one up
next week.
>>
>> The workaround is to edit the MODE configuration file by setting:
>> raw_thresh = >=-9999;
>>
>> That avoids this problem. However, METv4.1 will not have this
>> problem even when using the default raw_thresh of >=0.0.
>>
>> Here's the other user's question about the same issue:
>> http://mailman.ucar.edu/pipermail/met_help/2013-
May/001937.html
>>
>> Hope that helps.
>>
>> Thanks,
>> John
>>
>>
>> On 05/16/2013 12:06 PM, Raby, John W USA CIV via RT wrote:
>>>
>>> Thu May 16 12:06:33 2013: Request 61428 was acted upon.
>>> Transaction: Ticket created by john.w.raby2.civ at mail.mil
>>> Queue: met_help
>>> Subject: Odd MODE result
>>> Owner: Nobody
>>> Requestors: john.w.raby2.civ at mail.mil
>>> Status: new
>>> Ticket <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61428
>>>>
>>>
>>>
>>> Could you help diagnose what is causing the odd part of the object
>>> shown
>> in the attached .ps file?
>>>
>>> Part of the object follows completely around the square border of
>>> the
>> model domain.
>>>
>>> I cannot determine why the area around the border is being
diagnosed
>>> as an
>> object.
>>>
>>> I've attached the original gridded observations file which was
also
>>> used
>> as the forecast file as a test.
>>>
>>> See the attached config file we used.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>>
>>
>>
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
>>
>
>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
>
Classification: UNCLASSIFIED
Caveats: NONE
------------------------------------------------
More information about the Met_help
mailing list