[Met_help] [rt.rap.ucar.edu #67075] History for forecast filter
John Halley Gotway via RT
met_help at ucar.edu
Tue May 13 13:00:06 MDT 2014
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi John,
I have a suggestion for the point stat tool that may already be implemented,
however, I just might not know about it.
I am currently working with cloud ceiling observations in MET. In my grib
files, if there are no modeled clouds, the cloud ceiling is technically
infinite. As clouds pass, my grib files show ceilings going from 500m to
infinity, and back again. According to my understanding, you guys deal with
these sorts of things by using categorical thresholds. You have hits, false
alarms, misses, and correct rejections. Now, I only care about the hits,
because I don't expect to get every passing cloud at the exact right time, I
just want to make sure I am getting the ceiling height correct when I get a
hit (I am dealing with lifting fog, so I get hits all the time). As far as
I know, there are no continuous stats (CNT file) computed for only the
"hits". It may be beneficial for MET uses if we could filter the fct and
obs so they are only verified when the field is < = > a specified value. So
in my case, I just want a CNT file when my forecast and observed ceiling
heights are always less than 1000m, otherwise, my stats are all over the
place. If there is a special missing value I could put into the grib file
that point stat would ignore that would work also (since I can filter the
observations). Right now I am just using the matched pairs files, running
a filter for "hits", then computing the stats I need from that.
-Travis
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #67075] forecast filter
From: John Halley Gotway
Time: Tue May 13 11:10:22 2014
Travis,
What you are referring to is known as conditional verification, only
computing continuous statistics when some condition is met. This
request has come up a few times in the past, and we do have a
development task defined for it. However, it has not been implemented
yet. For example, suppose you only care about the quality of your 2-m
temperature forecasts when they are below freezing. You'd
compute the matched pairs but throw out any where the forecast value
is greater than 0C. And then compute stats on what's left.
That's not yet implemented, but there is a way of defining a masking
region that may be useful. We call it "data masking" and you
basically pass the tool a gridded field and a threshold. Only those
points where the threshold is met will be used in the verification.
Here's how you'd use it...
cd METv4.1/scripts
Edit config/PointStatConfig by setting the following in the "mask"
section:
poly = [ "MET_BASE/data/poly/LMV.poly",
"../data/sample_fcst/2007033000/nam.t00z.awip1236.tm00.20070330.grb
{name=\"TMP\"; level=\"Z2\";} <273" ];
Then re-run the test scripts.
The first entry was already there, but the second one is new. This
specifies a gridded file name, followed by a configuration string
defining the field to be extracted, followed by a threshold. Only
those grid points where the threshold criteria is met will be
included. In your case, you'd pass it the forecast file name,
followed by a description of the field, followed by "<1000". We
provided
this functionality with things like topography in mind. You may only
want to compute stats when above/below some elevation.
This won't give you all the "hits" because it's only conditioning on
the forecast value. The other limitation is that you can't define an
intersection of masking regions. This data mask is applied
to the full domain, not just within some subregion of the domain.
Thanks for the feedback and hopefully you'll find the data masking
feature to be useful.
Thanks,
John
On 05/12/2014 07:51 PM, Travis Wilson via RT wrote:
>
> Mon May 12 19:51:55 2014: Request 67075 was acted upon.
> Transaction: Ticket created by Wilson0028 at ucla.edu
> Queue: met_help
> Subject: forecast filter
> Owner: Nobody
> Requestors: Wilson0028 at ucla.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=67075 >
>
>
> Hi John,
>
> I have a suggestion for the point stat tool that may already be
implemented,
> however, I just might not know about it.
>
> I am currently working with cloud ceiling observations in MET. In
my grib
> files, if there are no modeled clouds, the cloud ceiling is
technically
> infinite. As clouds pass, my grib files show ceilings going from
500m to
> infinity, and back again. According to my understanding, you guys
deal with
> these sorts of things by using categorical thresholds. You have
hits, false
> alarms, misses, and correct rejections. Now, I only care about the
hits,
> because I don't expect to get every passing cloud at the exact right
time, I
> just want to make sure I am getting the ceiling height correct when
I get a
> hit (I am dealing with lifting fog, so I get hits all the time). As
far as
> I know, there are no continuous stats (CNT file) computed for only
the
> "hits". It may be beneficial for MET uses if we could filter the
fct and
> obs so they are only verified when the field is < = > a specified
value. So
> in my case, I just want a CNT file when my forecast and observed
ceiling
> heights are always less than 1000m, otherwise, my stats are all over
the
> place. If there is a special missing value I could put into the
grib file
> that point stat would ignore that would work also (since I can
filter the
> observations). Right now I am just using the matched pairs files,
running
> a filter for "hits", then computing the stats I need from that.
>
> -Travis
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #67075] forecast filter
From: Travis Wilson
Time: Tue May 13 12:20:08 2014
Hi John,
Yes, this is exactly what I had in mind.
However, using this "data masking" brings up another problem of not
being able to define a subregion to verify. For now, I'll just work
with my MPR files. Though, I look forward to conditional verification
if/when it gets implemented.
-Travis
On Tue, 2014-05-13 at 11:10 -0600, John Halley Gotway via RT wrote:
> Travis,
>
> What you are referring to is known as conditional verification, only
computing continuous statistics when some condition is met. This
request has come up a few times in the past, and we do have a
> development task defined for it. However, it has not been
implemented yet. For example, suppose you only care about the quality
of your 2-m temperature forecasts when they are below freezing. You'd
> compute the matched pairs but throw out any where the forecast value
is greater than 0C. And then compute stats on what's left.
>
> That's not yet implemented, but there is a way of defining a masking
region that may be useful. We call it "data masking" and you
basically pass the tool a gridded field and a threshold. Only those
> points where the threshold is met will be used in the verification.
Here's how you'd use it...
>
> cd METv4.1/scripts
> Edit config/PointStatConfig by setting the following in the "mask"
section:
> poly = [ "MET_BASE/data/poly/LMV.poly",
"../data/sample_fcst/2007033000/nam.t00z.awip1236.tm00.20070330.grb
{name=\"TMP\"; level=\"Z2\";} <273" ];
> Then re-run the test scripts.
>
> The first entry was already there, but the second one is new. This
specifies a gridded file name, followed by a configuration string
defining the field to be extracted, followed by a threshold. Only
> those grid points where the threshold criteria is met will be
included. In your case, you'd pass it the forecast file name,
followed by a description of the field, followed by "<1000". We
provided
> this functionality with things like topography in mind. You may
only want to compute stats when above/below some elevation.
>
> This won't give you all the "hits" because it's only conditioning on
the forecast value. The other limitation is that you can't define an
intersection of masking regions. This data mask is applied
> to the full domain, not just within some subregion of the domain.
>
> Thanks for the feedback and hopefully you'll find the data masking
feature to be useful.
>
> Thanks,
> John
>
> On 05/12/2014 07:51 PM, Travis Wilson via RT wrote:
> >
> > Mon May 12 19:51:55 2014: Request 67075 was acted upon.
> > Transaction: Ticket created by Wilson0028 at ucla.edu
> > Queue: met_help
> > Subject: forecast filter
> > Owner: Nobody
> > Requestors: Wilson0028 at ucla.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=67075 >
> >
> >
> > Hi John,
> >
> > I have a suggestion for the point stat tool that may already be
implemented,
> > however, I just might not know about it.
> >
> > I am currently working with cloud ceiling observations in MET. In
my grib
> > files, if there are no modeled clouds, the cloud ceiling is
technically
> > infinite. As clouds pass, my grib files show ceilings going from
500m to
> > infinity, and back again. According to my understanding, you guys
deal with
> > these sorts of things by using categorical thresholds. You have
hits, false
> > alarms, misses, and correct rejections. Now, I only care about
the hits,
> > because I don't expect to get every passing cloud at the exact
right time, I
> > just want to make sure I am getting the ceiling height correct
when I get a
> > hit (I am dealing with lifting fog, so I get hits all the time).
As far as
> > I know, there are no continuous stats (CNT file) computed for only
the
> > "hits". It may be beneficial for MET uses if we could filter the
fct and
> > obs so they are only verified when the field is < = > a specified
value. So
> > in my case, I just want a CNT file when my forecast and observed
ceiling
> > heights are always less than 1000m, otherwise, my stats are all
over the
> > place. If there is a special missing value I could put into the
grib file
> > that point stat would ignore that would work also (since I can
filter the
> > observations). Right now I am just using the matched pairs
files, running
> > a filter for "hits", then computing the stats I need from that.
> >
> > -Travis
> >
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #67075] forecast filter
From: John Halley Gotway
Time: Tue May 13 13:00:03 2014
Travis,
Makes sense.
Thanks,
John
On 05/13/2014 12:20 PM, Travis Wilson via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=67075 >
>
> Hi John,
>
> Yes, this is exactly what I had in mind.
>
> However, using this "data masking" brings up another problem of not
> being able to define a subregion to verify. For now, I'll just work
> with my MPR files. Though, I look forward to conditional
verification
> if/when it gets implemented.
>
> -Travis
>
>
> On Tue, 2014-05-13 at 11:10 -0600, John Halley Gotway via RT wrote:
>> Travis,
>>
>> What you are referring to is known as conditional verification,
only computing continuous statistics when some condition is met. This
request has come up a few times in the past, and we do have a
>> development task defined for it. However, it has not been
implemented yet. For example, suppose you only care about the quality
of your 2-m temperature forecasts when they are below freezing. You'd
>> compute the matched pairs but throw out any where the forecast
value is greater than 0C. And then compute stats on what's left.
>>
>> That's not yet implemented, but there is a way of defining a
masking region that may be useful. We call it "data masking" and you
basically pass the tool a gridded field and a threshold. Only those
>> points where the threshold is met will be used in the verification.
Here's how you'd use it...
>>
>> cd METv4.1/scripts
>> Edit config/PointStatConfig by setting the following in the "mask"
section:
>> poly = [ "MET_BASE/data/poly/LMV.poly",
"../data/sample_fcst/2007033000/nam.t00z.awip1236.tm00.20070330.grb
{name=\"TMP\"; level=\"Z2\";} <273" ];
>> Then re-run the test scripts.
>>
>> The first entry was already there, but the second one is new. This
specifies a gridded file name, followed by a configuration string
defining the field to be extracted, followed by a threshold. Only
>> those grid points where the threshold criteria is met will be
included. In your case, you'd pass it the forecast file name,
followed by a description of the field, followed by "<1000". We
provided
>> this functionality with things like topography in mind. You may
only want to compute stats when above/below some elevation.
>>
>> This won't give you all the "hits" because it's only conditioning
on the forecast value. The other limitation is that you can't define
an intersection of masking regions. This data mask is applied
>> to the full domain, not just within some subregion of the domain.
>>
>> Thanks for the feedback and hopefully you'll find the data masking
feature to be useful.
>>
>> Thanks,
>> John
>>
>> On 05/12/2014 07:51 PM, Travis Wilson via RT wrote:
>>>
>>> Mon May 12 19:51:55 2014: Request 67075 was acted upon.
>>> Transaction: Ticket created by Wilson0028 at ucla.edu
>>> Queue: met_help
>>> Subject: forecast filter
>>> Owner: Nobody
>>> Requestors: Wilson0028 at ucla.edu
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=67075 >
>>>
>>>
>>> Hi John,
>>>
>>> I have a suggestion for the point stat tool that may already be
implemented,
>>> however, I just might not know about it.
>>>
>>> I am currently working with cloud ceiling observations in MET. In
my grib
>>> files, if there are no modeled clouds, the cloud ceiling is
technically
>>> infinite. As clouds pass, my grib files show ceilings going from
500m to
>>> infinity, and back again. According to my understanding, you guys
deal with
>>> these sorts of things by using categorical thresholds. You have
hits, false
>>> alarms, misses, and correct rejections. Now, I only care about
the hits,
>>> because I don't expect to get every passing cloud at the exact
right time, I
>>> just want to make sure I am getting the ceiling height correct
when I get a
>>> hit (I am dealing with lifting fog, so I get hits all the time).
As far as
>>> I know, there are no continuous stats (CNT file) computed for only
the
>>> "hits". It may be beneficial for MET uses if we could filter the
fct and
>>> obs so they are only verified when the field is < = > a specified
value. So
>>> in my case, I just want a CNT file when my forecast and observed
ceiling
>>> heights are always less than 1000m, otherwise, my stats are all
over the
>>> place. If there is a special missing value I could put into the
grib file
>>> that point stat would ignore that would work also (since I can
filter the
>>> observations). Right now I am just using the matched pairs
files, running
>>> a filter for "hits", then computing the stats I need from that.
>>>
>>> -Travis
>>>
>>
>
>
------------------------------------------------
More information about the Met_help
mailing list