[Met_help] [rt.rap.ucar.edu #88134] History for Question about Mode Time Domain attributes
John Halley Gotway via RT
met_help at ucar.edu
Fri Dec 14 09:17:34 MST 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I have two quick questions about the temporal attributes calculated in MTD.
1) Are the start_time_delta and end_time_delta attributes given in terms
of timesteps as opposed to hours/minutes? According to the
documentation, these attributes are "the differences in the
corresponding times associated with the two objects". I assume they are
in terms of timesteps, but I just wanted to verify.
2) Regarding the default interest functions for start_time_delta and
end_time delta, is there a reason that negative time deltas (early
forecasts) are not given any interest? For example, the interest
function for start_time_delta is:
start_time_delta = (
( 0.0, 1.0 )
( 3.0, 0.5 )
( 5.0, 0.0 )
);
For this default function, only positive start_time errors are accounted
for and a forecast that is early would be given no interest. I would
expect a symmetric function for this attribute to correctly account for
early forecasts. Meanwhile, the time_centroid_delta attribute does
account for negative time differences:
time_centroid_delta = (
( -3.0, 0.0 )
( -2.0, 0.5 )
( -1.0, 0.8 )
( 0.0, 1.0 )
( 1.0, 0.8 )
( 2.0, 0.5 )
( 3.0, 0.0 )
);
I might be misunderstanding the way the interest functions are designed.
I hope these questions make sense. Thanks again for the continuous
development of such a great verification tool!
Thanks!
Sam Degelia
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Question about Mode Time Domain attributes
From: John Halley Gotway
Time: Thu Dec 13 09:49:03 2018
Sam,
I see you have some questions about the interest functions for MODE
Time
Domain. You raise some very good points!
First, yes, the unit for time are not prescribe. They are given in
time
"steps"... i.e. the interval of time between adjacent forecasts. I
made
the following tweaks to the MET User's Guide for the next version in
the
hopes of making that more clear:
*... are the differences in the corresponding time steps associated
with
the two objects and are computed as "forecast minus obs".*
*START_TIME_DELTA = Difference in object starting time steps*
*END_TIME_DELTA = Difference in object ending time steps*
After pointing out that forecasts starting before observations yields
zero
interest, I assumed that the code was computing the absolute value of
that
time difference. But looking at the selection of code from the file
3d_att.cc, it clearly is not. And the same is true of the
end_time_delta
attribute:
1484 //
1485 // start time delta
1486 //
1487
1488 t = fcst_att.tmin() - obs_att.tmin();
1489
1490 p.set_start_time_delta(t);
1491
1492 //
1493 // end time delta
1494 //
1495
1496 t = fcst_att.tmax() - obs_att.tmax();
1497
1498 p.set_end_time_delta(t);
For that reason, I'd recommend changing the MTD config to make those
interest functions symmetric about zero:
start_time_delta = (
( -5.0, 0.0 )
( -3.0, 0.5 )
( 0.0, 1.0 )
( 3.0, 0.5 )
( 5.0, 0.0 )
);
And I'll go talk to Randy Bullock, the developer who worked on MTD to
see
if there's some detail we're missing.
Thanks for finding this issue!
John Halley Gotway
On Wed, Dec 12, 2018 at 2:58 PM Samuel Degelia via RT
<met_help at ucar.edu>
wrote:
>
> Wed Dec 12 14:58:48 2018: Request 88134 was acted upon.
> Transaction: Ticket created by sdegelia at gmail.com
> Queue: met_help
> Subject: Question about Mode Time Domain attributes
> Owner: Nobody
> Requestors: sdegelia at gmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88134 >
>
>
> Hello,
>
> I have two quick questions about the temporal attributes calculated
in MTD.
>
> 1) Are the start_time_delta and end_time_delta attributes given in
terms
> of timesteps as opposed to hours/minutes? According to the
> documentation, these attributes are "the differences in the
> corresponding times associated with the two objects". I assume they
are
> in terms of timesteps, but I just wanted to verify.
>
> 2) Regarding the default interest functions for start_time_delta and
> end_time delta, is there a reason that negative time deltas (early
> forecasts) are not given any interest? For example, the interest
> function for start_time_delta is:
>
> start_time_delta = (
>
> ( 0.0, 1.0 )
> ( 3.0, 0.5 )
> ( 5.0, 0.0 )
>
> );
>
> For this default function, only positive start_time errors are
accounted
> for and a forecast that is early would be given no interest. I would
> expect a symmetric function for this attribute to correctly account
for
> early forecasts. Meanwhile, the time_centroid_delta attribute does
> account for negative time differences:
>
> time_centroid_delta = (
>
> ( -3.0, 0.0 )
> ( -2.0, 0.5 )
> ( -1.0, 0.8 )
> ( 0.0, 1.0 )
> ( 1.0, 0.8 )
> ( 2.0, 0.5 )
> ( 3.0, 0.0 )
>
> );
>
> I might be misunderstanding the way the interest functions are
designed.
> I hope these questions make sense. Thanks again for the continuous
> development of such a great verification tool!
>
> Thanks!
> Sam Degelia
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #88134] Question about Mode Time Domain attributes
From: Samuel Degelia
Time: Thu Dec 13 15:19:28 2018
John,
Thanks for getting back to me so quickly once again! I am glad to get
clarification on these two questions, and given your response, I can
deal with both issues very easily on my end.
I discovered this issue through testing the sensitivity of my MTD
results to output frequency of my forecasts. I previously reduced the
frequency of my forecasts to work around the time smoothing in MTD.
This
change led to some issues where objects were no longer matched/merged
because they received no interest from the start_time_delta and
end_time_delta attributes. I previously was using timesteps of 30-min
which allowed for some interest up to a time difference of 3 hours.
When
I reduced my timestep to 5-min, the objects only received interest up
to
a time difference of 25 minutes.
I will probably make a change in my scripts to build the MTD config
file
in terms of a forecast frequency argument. For example, if I wanted to
assign interest for timing differences of +/- 3 hours, I would do:
start_time_delta = (
( -180/${frequency_in_mins}, 0.0 )
( -90/${frequency_in_mins}, 0.5 )
( 0.0, 1.0 )
( 90/${frequency_in_mins}, 0.5 )
( 180/${frequency_in_mins}, 0.0 )
);
This is an easy change on my end, but it could be a useful option to
add
into future versions of MTD as well. Thanks again for all of your
help!
Thanks,
Sam Degelia
On 12/13/2018 11:49 AM, John Halley Gotway via RT wrote:
> Sam,
>
> I see you have some questions about the interest functions for MODE
Time
> Domain. You raise some very good points!
>
> First, yes, the unit for time are not prescribe. They are given in
time
> "steps"... i.e. the interval of time between adjacent forecasts. I
made
> the following tweaks to the MET User's Guide for the next version in
the
> hopes of making that more clear:
>
> *... are the differences in the corresponding time steps associated
with
> the two objects and are computed as "forecast minus obs".*
> *START_TIME_DELTA = Difference in object starting time steps*
> *END_TIME_DELTA = Difference in object ending time steps*
>
> After pointing out that forecasts starting before observations
yields zero
> interest, I assumed that the code was computing the absolute value
of that
> time difference. But looking at the selection of code from the file
> 3d_att.cc, it clearly is not. And the same is true of the
end_time_delta
> attribute:
>
> 1484 //
> 1485 // start time delta
> 1486 //
> 1487
> 1488 t = fcst_att.tmin() - obs_att.tmin();
> 1489
> 1490 p.set_start_time_delta(t);
> 1491
> 1492 //
> 1493 // end time delta
> 1494 //
> 1495
> 1496 t = fcst_att.tmax() - obs_att.tmax();
> 1497
> 1498 p.set_end_time_delta(t);
>
> For that reason, I'd recommend changing the MTD config to make those
> interest functions symmetric about zero:
>
> start_time_delta = (
> ( -5.0, 0.0 )
> ( -3.0, 0.5 )
> ( 0.0, 1.0 )
> ( 3.0, 0.5 )
> ( 5.0, 0.0 )
> );
>
> And I'll go talk to Randy Bullock, the developer who worked on MTD
to see
> if there's some detail we're missing.
>
> Thanks for finding this issue!
>
> John Halley Gotway
>
> On Wed, Dec 12, 2018 at 2:58 PM Samuel Degelia via RT
<met_help at ucar.edu>
> wrote:
>
>> Wed Dec 12 14:58:48 2018: Request 88134 was acted upon.
>> Transaction: Ticket created by sdegelia at gmail.com
>> Queue: met_help
>> Subject: Question about Mode Time Domain attributes
>> Owner: Nobody
>> Requestors: sdegelia at gmail.com
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88134 >
>>
>>
>> Hello,
>>
>> I have two quick questions about the temporal attributes calculated
in MTD.
>>
>> 1) Are the start_time_delta and end_time_delta attributes given in
terms
>> of timesteps as opposed to hours/minutes? According to the
>> documentation, these attributes are "the differences in the
>> corresponding times associated with the two objects". I assume they
are
>> in terms of timesteps, but I just wanted to verify.
>>
>> 2) Regarding the default interest functions for start_time_delta
and
>> end_time delta, is there a reason that negative time deltas (early
>> forecasts) are not given any interest? For example, the interest
>> function for start_time_delta is:
>>
>> start_time_delta = (
>>
>> ( 0.0, 1.0 )
>> ( 3.0, 0.5 )
>> ( 5.0, 0.0 )
>>
>> );
>>
>> For this default function, only positive start_time errors are
accounted
>> for and a forecast that is early would be given no interest. I
would
>> expect a symmetric function for this attribute to correctly account
for
>> early forecasts. Meanwhile, the time_centroid_delta attribute does
>> account for negative time differences:
>>
>> time_centroid_delta = (
>>
>> ( -3.0, 0.0 )
>> ( -2.0, 0.5 )
>> ( -1.0, 0.8 )
>> ( 0.0, 1.0 )
>> ( 1.0, 0.8 )
>> ( 2.0, 0.5 )
>> ( 3.0, 0.0 )
>>
>> );
>>
>> I might be misunderstanding the way the interest functions are
designed.
>> I hope these questions make sense. Thanks again for the continuous
>> development of such a great verification tool!
>>
>> Thanks!
>> Sam Degelia
>>
>>
------------------------------------------------
More information about the Met_help
mailing list