[Met_help] [rt.rap.ucar.edu #76617] History for Creating mtd config file
John Halley Gotway via RT
met_help at ucar.edu
Tue Jun 7 11:10:51 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I have looked everywhere but I can't find a mtd config file anywhere. Can you please remind me how to create one with default settings?
Thanks
Marion
--
Dr Marion Mittermaier Manager: Model Diagnostics and Novel Methods
Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
Tel: +44 1392 884830 Fax: +44 1392 885631
E:mail: marion.mittermaier at metoffice.gov.uk http://www.metoffice.gov.uk/research/people/marion-mittermaier
I am also the co-chair of the WMO WWRP/WGNE Joint Working Group for Forecast Verification Research<https://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html>
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Creating mtd config file
From: John Halley Gotway
Time: Thu Jun 02 09:34:08 2016
Marion,
I'm see that you're interested in running the MODE-Time-Domain tool
and have been able to locate a sample MTD configuration file. Please
let us know what questions or issues arise in your use of MTD. We're
happy to help.
Thanks,
John Halley Gotway
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #76617] Creating mtd config file
From: marion.mittermaier at metoffice.gov.uk
Time: Fri Jun 03 01:53:18 2016
Hi John
I do have one question actually. How does the notion of a time unit
work in the config file. It is stepping in steps of 1 but is that in
physical time elapsed, i.e. hourly? If the data is not hourly then
this should be adjusted, I presume? E.g. using 12-hourly fields, the
minimum offset (other than 0) is 12 hours (which is one time unit if
one thinks of time steps).
Thanks
Marion
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: 02 June 2016 16:34
To: Mittermaier, Marion
Subject: [rt.rap.ucar.edu #76617] Creating mtd config file
Marion,
I'm see that you're interested in running the MODE-Time-Domain tool
and have been able to locate a sample MTD configuration file. Please
let us know what questions or issues arise in your use of MTD. We're
happy to help.
Thanks,
John Halley Gotway
------------------------------------------------
Subject: Creating mtd config file
From: John Halley Gotway
Time: Fri Jun 03 09:57:38 2016
Marion,
I'll try to answer this question as best as I can, but if we need more
details, I can talk to Randy next week.
I'm not exactly sure which settings in the configuration you are
referring
to. Since you've mentioned the value of "1", perhaps you're
referencing
the "weight" section of the configuration file?
weight = {
space_centroid_dist = 1.0;
time_centroid_delta = 1.0;
speed_delta = 1.0;
direction_diff = 1.0;
volume_ratio = 1.0;
axis_angle_diff = 1.0;
start_time_delta = 1.0;
end_time_delta = 1.0;
}
I can tell you that "weight" in the config file defines the relative
importance of the pair-wise differences of the attributes between
forecast
and analysis 3D space time objects. This operates exactly the same
way as
the "weight" section in the MODE config file... its just that the
pair-wise
attributes differ between MODE and MTD. The weights impact the
computation
of the total interest values computed for each pair of
forecast/analysis
objects. Therefore, it impacts the matching/merging that MTD does.
By default, all of these pair-wise attributes are given equal weight
(i.e.
1.0). But you can change that. For example, setting "volume_ratio =
2.0"
would double it's weight in the computation of total interest. In the
code, the weight for each attribute is divided by the sum of the
weights...
so it's their relative value that really matters, no the actual value.
Setting them all equally to 1.0 or 10.0 would have the same effect.
Now, you've actually asked about the time step... but I don't believe
that
really comes into play in the config file. If you pass it the same
data
that's spaced 1-hour apart or 24-hours apart, I believe you should get
the
same numbers in the output (except of course for the columns that list
the
timestamps for the files).
Does that answer your question?
Thanks,
John
On Fri, Jun 3, 2016 at 1:53 AM, marion.mittermaier at metoffice.gov.uk
via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=76617 >
>
> Hi John
>
> I do have one question actually. How does the notion of a time unit
work
> in the config file. It is stepping in steps of 1 but is that in
physical
> time elapsed, i.e. hourly? If the data is not hourly then this
should be
> adjusted, I presume? E.g. using 12-hourly fields, the minimum offset
(other
> than 0) is 12 hours (which is one time unit if one thinks of time
steps).
>
> Thanks
> Marion
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: 02 June 2016 16:34
> To: Mittermaier, Marion
> Subject: [rt.rap.ucar.edu #76617] Creating mtd config file
>
> Marion,
>
> I'm see that you're interested in running the MODE-Time-Domain tool
and
> have been able to locate a sample MTD configuration file. Please
let us
> know what questions or issues arise in your use of MTD. We're happy
to
> help.
>
> Thanks,
> John Halley Gotway
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #76617] Creating mtd config file
From: marion.mittermaier at metoffice.gov.uk
Time: Mon Jun 06 01:53:33 2016
Hi John,
I obviously didn't articulate myself very well. Let me try again...
it's the relative importance of different attributes in the interest
function I was referring to. I've added a few questions next to the
relevant bits of the config file snippet below.
Regards
Marion
======================================================================
interest_function = {
space_centroid_dist = (
( 0.0, 1.0 )
( 15.0 * grid_res, 0.5 )
( 30.0 * grid_res, 0.0 )
);
time_centroid_delta = (
( -3.0, 0.0 ) is this in time steps not time elapsed? i.e. if
data is hourly then
( -2.0, 0.5 ) time step = time elapsed but in our case data is
12-hourly, between
( -1.0, 0.8 ) one time step and the next, 12h, 24h, 36h etc.
so should this be
( 0.0, 1.0 ) reflected here? Otherwise this attribute will
never contribute to the
( 1.0, 0.8 ) interest calculation
( 2.0, 0.5 )
( 3.0, 0.0 )
);
speed_delta = (
( -10.0, 0.0 ) is this in m/s?
( -5.0, 0.5 )
( 0.0, 1.0 )
( 5.0, 0.5 )
( 10.0, 0.0 )
);
direction_diff = (
( 0.0, 1.0 )
( 90.0, 0.0 )
( 180.0, 0.0 )
);
volume_ratio = (
( 0.0, 0.0 )
( 0.5, 0.5 )
( 1.0, 1.0 )
( 1.5, 0.5 )
( 2.0, 0.0 )
);
axis_angle_diff = (
( 0.0, 1.0 )
( 30.0, 1.0 )
( 90.0, 0.0 )
);
start_time_delta = (
( 0.0, 1.0 ) same question as above about times
( 3.0, 0.5 )
( 5.0, 0.0 )
);
end_time_delta = (
( 0.0, 1.0 ) and ditto again
( 3.0, 0.5 )
( 5.0, 0.0 )
);
} // interest functions
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: 03 June 2016 16:58
To: Mittermaier, Marion
Subject: Re: [rt.rap.ucar.edu #76617] Creating mtd config file
Marion,
I'll try to answer this question as best as I can, but if we need more
details, I can talk to Randy next week.
I'm not exactly sure which settings in the configuration you are
referring to. Since you've mentioned the value of "1", perhaps you're
referencing the "weight" section of the configuration file?
weight = {
space_centroid_dist = 1.0;
time_centroid_delta = 1.0;
speed_delta = 1.0;
direction_diff = 1.0;
volume_ratio = 1.0;
axis_angle_diff = 1.0;
start_time_delta = 1.0;
end_time_delta = 1.0;
}
I can tell you that "weight" in the config file defines the relative
importance of the pair-wise differences of the attributes between
forecast and analysis 3D space time objects. This operates exactly
the same way as the "weight" section in the MODE config file... its
just that the pair-wise attributes differ between MODE and MTD. The
weights impact the computation of the total interest values computed
for each pair of forecast/analysis objects. Therefore, it impacts the
matching/merging that MTD does.
By default, all of these pair-wise attributes are given equal weight
(i.e.
1.0). But you can change that. For example, setting "volume_ratio =
2.0"
would double it's weight in the computation of total interest. In the
code, the weight for each attribute is divided by the sum of the
weights...
so it's their relative value that really matters, no the actual value.
Setting them all equally to 1.0 or 10.0 would have the same effect.
Now, you've actually asked about the time step... but I don't believe
that really comes into play in the config file. If you pass it the
same data that's spaced 1-hour apart or 24-hours apart, I believe you
should get the same numbers in the output (except of course for the
columns that list the timestamps for the files).
Does that answer your question?
Thanks,
John
On Fri, Jun 3, 2016 at 1:53 AM, marion.mittermaier at metoffice.gov.uk
via RT < met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=76617 >
>
> Hi John
>
> I do have one question actually. How does the notion of a time unit
> work in the config file. It is stepping in steps of 1 but is that in
> physical time elapsed, i.e. hourly? If the data is not hourly then
> this should be adjusted, I presume? E.g. using 12-hourly fields, the
> minimum offset (other than 0) is 12 hours (which is one time unit if
one thinks of time steps).
>
> Thanks
> Marion
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: 02 June 2016 16:34
> To: Mittermaier, Marion
> Subject: [rt.rap.ucar.edu #76617] Creating mtd config file
>
> Marion,
>
> I'm see that you're interested in running the MODE-Time-Domain tool
> and have been able to locate a sample MTD configuration file.
Please
> let us know what questions or issues arise in your use of MTD.
We're
> happy to help.
>
> Thanks,
> John Halley Gotway
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #76617] Creating mtd config file
From: marion.mittermaier at metoffice.gov.uk
Time: Mon Jun 06 01:55:21 2016
Also,
Have you got any generic bits of code kicking about for visualising
the 3D objects? Using ncview is just about ok but not brilliant... we
are looking at a Python solution but it may take us awhile.
Ta
Marion
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: 03 June 2016 16:58
To: Mittermaier, Marion
Subject: Re: [rt.rap.ucar.edu #76617] Creating mtd config file
Marion,
I'll try to answer this question as best as I can, but if we need more
details, I can talk to Randy next week.
I'm not exactly sure which settings in the configuration you are
referring to. Since you've mentioned the value of "1", perhaps you're
referencing the "weight" section of the configuration file?
weight = {
space_centroid_dist = 1.0;
time_centroid_delta = 1.0;
speed_delta = 1.0;
direction_diff = 1.0;
volume_ratio = 1.0;
axis_angle_diff = 1.0;
start_time_delta = 1.0;
end_time_delta = 1.0;
}
I can tell you that "weight" in the config file defines the relative
importance of the pair-wise differences of the attributes between
forecast and analysis 3D space time objects. This operates exactly
the same way as the "weight" section in the MODE config file... its
just that the pair-wise attributes differ between MODE and MTD. The
weights impact the computation of the total interest values computed
for each pair of forecast/analysis objects. Therefore, it impacts the
matching/merging that MTD does.
By default, all of these pair-wise attributes are given equal weight
(i.e.
1.0). But you can change that. For example, setting "volume_ratio =
2.0"
would double it's weight in the computation of total interest. In the
code, the weight for each attribute is divided by the sum of the
weights...
so it's their relative value that really matters, no the actual value.
Setting them all equally to 1.0 or 10.0 would have the same effect.
Now, you've actually asked about the time step... but I don't believe
that really comes into play in the config file. If you pass it the
same data that's spaced 1-hour apart or 24-hours apart, I believe you
should get the same numbers in the output (except of course for the
columns that list the timestamps for the files).
Does that answer your question?
Thanks,
John
On Fri, Jun 3, 2016 at 1:53 AM, marion.mittermaier at metoffice.gov.uk
via RT < met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=76617 >
>
> Hi John
>
> I do have one question actually. How does the notion of a time unit
> work in the config file. It is stepping in steps of 1 but is that in
> physical time elapsed, i.e. hourly? If the data is not hourly then
> this should be adjusted, I presume? E.g. using 12-hourly fields, the
> minimum offset (other than 0) is 12 hours (which is one time unit if
one thinks of time steps).
>
> Thanks
> Marion
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: 02 June 2016 16:34
> To: Mittermaier, Marion
> Subject: [rt.rap.ucar.edu #76617] Creating mtd config file
>
> Marion,
>
> I'm see that you're interested in running the MODE-Time-Domain tool
> and have been able to locate a sample MTD configuration file.
Please
> let us know what questions or issues arise in your use of MTD.
We're
> happy to help.
>
> Thanks,
> John Halley Gotway
>
>
>
------------------------------------------------
Subject: Creating mtd config file
From: Randy Bullock
Time: Mon Jun 06 15:05:26 2016
Hi Marion -
Grid distances are in grid units, so in a 5km grid, for example, the
distance units are integer multiples of 5km.
Likewise, time units are whatever the time separation of the MTD input
files is. So if the input files are separated by one hour of time,
the one
MTD time unit is one hour. Likewise, if the timestamps on the input
files
are separated by half an hour, then one MTD time unit is 30 minutes.
Likewise (again) speed is in distance units per time unit. So if you
have
data on a 5km grid, with the data files separated by 12 hours, then a
speed
of one corresponds to 5km per 12 hours, or 0.42 km/hour. a speed of
10
corresponds to 4.2 km/hour.
As for providing code to plot MTD output, we're kind of on the fence
about
that. I have some code for this, but we've decided not to do a
general
release of it. We could probably get you set up with it, though, if
you
wanted to. Might take a bit of work, though, since this is my code,
not
MET code. Still, it's in C++, so if you got MET to compile, you could
probably build this, too.
I've attached an image from our visualization program. It basically
makes
those rotating 3D animations that I'm sure you've seen. We make the
object
slightly transparent to make it easier to see the structure. Also, we
sometimes make plots with two different thresholds simultaneously, and
the
transparency make it possible to see the high-threshold objects inside
the
low-threshold ones.
Let me know if you want to pursue this.
Randy
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #76617] Creating mtd config file
From: marion.mittermaier at metoffice.gov.uk
Time: Tue Jun 07 04:03:27 2016
Thanks Randy.
I am familiar with the MODE TD plots from the cloud paper we worked
on. I will have to think about whether we want to take that on and
whether that is preferable to trying to create something ourselves.
Thanks for the explanation of the units. That helps!
Regards
Marion
-----Original Message-----
From: Randy Bullock via RT [mailto:met_help at ucar.edu]
Sent: 06 June 2016 22:05
To: Mittermaier, Marion
Subject: Re: [rt.rap.ucar.edu #76617] Creating mtd config file
Hi Marion -
Grid distances are in grid units, so in a 5km grid, for example, the
distance units are integer multiples of 5km.
Likewise, time units are whatever the time separation of the MTD input
files is. So if the input files are separated by one hour of time,
the one MTD time unit is one hour. Likewise, if the timestamps on the
input files are separated by half an hour, then one MTD time unit is
30 minutes.
Likewise (again) speed is in distance units per time unit. So if you
have data on a 5km grid, with the data files separated by 12 hours,
then a speed of one corresponds to 5km per 12 hours, or 0.42 km/hour.
a speed of 10 corresponds to 4.2 km/hour.
As for providing code to plot MTD output, we're kind of on the fence
about that. I have some code for this, but we've decided not to do a
general release of it. We could probably get you set up with it,
though, if you wanted to. Might take a bit of work, though, since
this is my code, not MET code. Still, it's in C++, so if you got MET
to compile, you could probably build this, too.
I've attached an image from our visualization program. It basically
makes those rotating 3D animations that I'm sure you've seen. We make
the object slightly transparent to make it easier to see the
structure. Also, we sometimes make plots with two different
thresholds simultaneously, and the transparency make it possible to
see the high-threshold objects inside the low-threshold ones.
Let me know if you want to pursue this.
Randy
------------------------------------------------
More information about the Met_help
mailing list