[Met_help] [rt.rap.ucar.edu #60725] History for Can I use MODE tool with radar observations?
John Halley Gotway via RT
met_help at ucar.edu
Thu May 23 14:37:01 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I am trying to determine if I can use the MODE tool for a research project
I am working on. I have radar data from NSSL's 3D radar mosaic grid. It is
in netcdf format and on a 3km Cartesian grid, with 500-m spacing in the
vertical. I am trying to find the "centroids" of storms in this data set,
at a particular level (say 1.5 km). MODE seemed liked a possible way to
accomplish this. I assumed that I could set the forecast and observation
data to the same file at the command line, since I am not looking to
compare to distinct sets (just get the attributes of one). However, it
seems like in the user documentation that this tool is tuned particularly
for precip. fields, and one must come at MODE using pcp_combine. I am more
than willing to restructure my data in another format. I was just
wondering if someone could comment on if it is possible for me to use radar
data with this tool (and what it might need to look like)??? I have a
lat/lon, height, etc. for each observation value.)
Thanks in advance,
Dusty Wheatley
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60725] Can I use MODE tool with radar observations?
From: John Halley Gotway
Time: Wed Mar 20 12:08:57 2013
Dusty,
Yep, it seems like MODE would be a good tool for you to use. Just to
be clear, you just want to extract the lat/lon centroids of each
object. You're not interested in comparing objects between
fields. If so, I'd suggest the following:
- Pass the same file in for the forecast and observation fields.
- Set up the forecast object definition information to define
objects the way you'd like:
- Primarily, that's the "conv_radius" and "conv_thresh" settings
for the convolution radius and threshold.
- Set the observation object definition information so that no
objects are defined.
- Just set the "raw_thresh" to a large threshold so that no raw
data values survive.
- Set the "merge_flag" to NONE in both the forecast and observation
fields.
- Then set the "match_flag" to NONE so that MODE doesn't try to
compare the fields.
This should result in only simple forecast objects being defined. And
you can extract the object lat/lon values from the ASCII output file
from MODE.
Now the only hurdle is formatting your data in such a way that MET can
read it. I'd suggest making it look like the NetCDF output of the
PCP-Combine tool.
Specifically, you'll need:
- Two dimensions named lat and lon.
- The variable name doesn't matter, but the attributes do.
- Add variable attributes. Here's an example:
float APCP_12(lat, lon) ;
APCP_12:name = "APCP_12" ;
APCP_12:long_name = "Total precipitation" ;
APCP_12:level = "A12" ;
APCP_12:units = "kg/m^2" ;
APCP_12:_FillValue = -9999.f ;
APCP_12:init_time = "20050807_000000" ;
APCP_12:init_time_ut = 1123372800 ;
APCP_12:valid_time = "20050807_120000" ;
APCP_12:valid_time_ut = 1123416000 ;
APCP_12:accum_time = "120000" ;
APCP_12:accum_time_sec = 43200 ;
Specifically, I believe we need the init_time_ut, valid_time_ut,
and accum_time_sec. The ut stands for "unix time", the number of
seconds since January 1, 1970.
- Add projection information to the global attributes. Here's an
example:
:Projection = "Lambert Conformal" ;
:scale_lat_1 = "25.000000" ;
:scale_lat_2 = "25.000000" ;
:lat_pin = "12.190000" ;
:lon_pin = "-133.459000" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "-95.000000" ;
:d_km = "40.635000" ;
:r_km = "6371.200000" ;
:nx = "185" ;
:ny = "129 grid_points" ;
The contents will change depending on your projection. So just
let me know what you're using.
- Add the "MET_version" global attribute. Here's an example:
:MET_version = "V4.0" ;
Give that a shot, and if you get stuck, just send me what you have and
I'll let you know what's missing.
Thanks,
John Halley Gotway
met_help at ucar.edu
On 03/20/2013 10:03 AM, Dustan Wheatley - NOAA Affiliate via RT wrote:
>
> Wed Mar 20 10:03:31 2013: Request 60725 was acted upon.
> Transaction: Ticket created by dustan.wheatley at noaa.gov
> Queue: met_help
> Subject: Can I use MODE tool with radar observations?
> Owner: Nobody
> Requestors: dustan.wheatley at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>
>
> Hi,
>
> I am trying to determine if I can use the MODE tool for a research
project
> I am working on. I have radar data from NSSL's 3D radar mosaic grid.
It is
> in netcdf format and on a 3km Cartesian grid, with 500-m spacing in
the
> vertical. I am trying to find the "centroids" of storms in this
data set,
> at a particular level (say 1.5 km). MODE seemed liked a possible
way to
> accomplish this. I assumed that I could set the forecast and
observation
> data to the same file at the command line, since I am not looking to
> compare to distinct sets (just get the attributes of one). However,
it
> seems like in the user documentation that this tool is tuned
particularly
> for precip. fields, and one must come at MODE using pcp_combine. I
am more
> than willing to restructure my data in another format. I was just
> wondering if someone could comment on if it is possible for me to
use radar
> data with this tool (and what it might need to look like)??? I have
a
> lat/lon, height, etc. for each observation value.)
>
> Thanks in advance,
> Dusty Wheatley
>
------------------------------------------------
Subject: Can I use MODE tool with radar observations?
From: Dustan Wheatley - NOAA Affiliate
Time: Thu Mar 21 13:29:05 2013
John,
Thanks! This was a great start. I have created a netcdf file as you
have
instructed, but have a few questions. First, here's the ncdump:
netcdf fcst {
dimensions:
lat = 151 ;
lon = 151 ;
variables:
float REFL_10CM(lat, lon) ;
REFL_10CM:name = "REFL_10CM" ;
REFL_10CM:long_name = "Radar reflectivity factor" ;
REFL_10CM:level = "A12" ;
REFL_10CM:units = "dBZ" ;
REFL_10CM:init_time = "20010520_204857" ;
REFL_10CM:init_time_ut = 990391737 ;
REFL_10CM:valid_time = "20010520_204857" ;
REFL_10CM:valid_time_ut = 990391737 ;
REFL_10CM:accum_time = "120000" ;
REFL_10CM:accum_time_sec = 43200 ;
REFL_10CM:_FillValue = -99900.f ;
// global attributes:
:Projection = "Lambert Conformal" ;
:scale_lat_1 = "35.333100" ;
:scale_lat_2 = "35.333100" ;
:lat_pin = "12.190000" ;
:lon_pin = "-133.459000" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "-97.277800" ;
:d_km = "40.635000" ;
:r_km = "6371.200000" ;
:nx = "151" ;
:ny = "151" ;
:MET_version = "V3.0.1" ;
Questions:
---> For REFL_10CM:level = "A12", does it matter what it says here.
The
radar data is just at some height (2.5 km MSL)
---> REFL_10CM:init_time = "20010520_204857", The next 4 attributes
are
correct, radar data is valid at the given time. How do I handle
accum_time
quantities (since it has no meaning in my case)?
---> REFL_10CM:init_time_ut = 990391737 ;
---> REFL_10CM:valid_time = "20010520_204857" ;
---> REFL_10CM:valid_time_ut = 990391737 ;
---> REFL_10CM:accum_time = "120000" ;
---> REFL_10CM:accum_time_sec = 43200 ;
---> REFL_10CM:_FillValue = -99900.f ;
---> I am using Lambert Conformal projections. I filled in the radar
latitude (35.3331) for scale_lat_1,2 (assuming it's like truelat1,2 in
wrf
data) and the radar longitude (-97.2778) in lon_orient. I have a
151x151
horizontal grid with 2-km spacing. Not quite certain what to do with
lat_pin, lon_pin, d_km? Assuming r_km is a constant(?).
:scale_lat_1 = "35.333100" ;
:scale_lat_2 = "35.333100" ;
:lat_pin = "12.190000" ;
:lon_pin = "-133.459000" ;
:x_pin = "0.000000" ;
:y_pin = "0.000000" ;
:lon_orient = "-97.277800" ;
:d_km = "40.635000" ;
:r_km = "6371.200000" ;
:nx = "151" ;
:ny = "151" ;
:MET_version = "V3.0.1" ;
I tried running (even knowing the projection values are somewhat
bogus),
and I get:
bigbang.protect.nssl:METtest> mode fcst.nc obs.nc config_file
Forecast File: fcst.nc
Observation File: obs.nc
Match Config File: config_file
Merge Config File: config_file
Forecast Field: REFL_10CM at A12
Observation Field: REFL_10CM at A12
Identifying objects in the forecast and observation fields...
Computing contingency table statistics...
Identified: 0 forecast objects and 0 observation objects.
Performing merging (no merging) in the forecast field.
Performing merging (no merging) in the observation field.
Remaining: 0 forecast objects and 0 observation objects.
Performing matching between the forecast and observation fields.
***WARNING*** void Engine::do_matching() -> no matching requested in
configuration file
ERROR: write_ct_stats() -> unable to open stats output file
"/home/Dustan.Wheatley/METv3.0.1/out/mode/mode_~/_000000L_20010520_204857V_120000A_cts.txt"
So, I think I am pretty close, with a few tweaks.
Thanks,
Dusty
On Wed, Mar 20, 2013 at 1:08 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Dusty,
>
> Yep, it seems like MODE would be a good tool for you to use. Just
to be
> clear, you just want to extract the lat/lon centroids of each
object.
> You're not interested in comparing objects between
> fields. If so, I'd suggest the following:
> - Pass the same file in for the forecast and observation fields.
> - Set up the forecast object definition information to define
objects
> the way you'd like:
> - Primarily, that's the "conv_radius" and "conv_thresh" settings
for
> the convolution radius and threshold.
> - Set the observation object definition information so that no
objects
> are defined.
> - Just set the "raw_thresh" to a large threshold so that no raw
data
> values survive.
> - Set the "merge_flag" to NONE in both the forecast and
observation
> fields.
> - Then set the "match_flag" to NONE so that MODE doesn't try to
compare
> the fields.
> This should result in only simple forecast objects being defined.
And you
> can extract the object lat/lon values from the ASCII output file
from MODE.
>
> Now the only hurdle is formatting your data in such a way that MET
can
> read it. I'd suggest making it look like the NetCDF output of the
> PCP-Combine tool.
>
> Specifically, you'll need:
> - Two dimensions named lat and lon.
> - The variable name doesn't matter, but the attributes do.
> - Add variable attributes. Here's an example:
> float APCP_12(lat, lon) ;
> APCP_12:name = "APCP_12" ;
> APCP_12:long_name = "Total precipitation" ;
> APCP_12:level = "A12" ;
> APCP_12:units = "kg/m^2" ;
> APCP_12:_FillValue = -9999.f ;
> APCP_12:init_time = "20050807_000000" ;
> APCP_12:init_time_ut = 1123372800 ;
> APCP_12:valid_time = "20050807_120000" ;
> APCP_12:valid_time_ut = 1123416000 ;
> APCP_12:accum_time = "120000" ;
> APCP_12:accum_time_sec = 43200 ;
> Specifically, I believe we need the init_time_ut, valid_time_ut,
and
> accum_time_sec. The ut stands for "unix time", the number of
seconds since
> January 1, 1970.
> - Add projection information to the global attributes. Here's an
example:
> :Projection = "Lambert Conformal" ;
> :scale_lat_1 = "25.000000" ;
> :scale_lat_2 = "25.000000" ;
> :lat_pin = "12.190000" ;
> :lon_pin = "-133.459000" ;
> :x_pin = "0.000000" ;
> :y_pin = "0.000000" ;
> :lon_orient = "-95.000000" ;
> :d_km = "40.635000" ;
> :r_km = "6371.200000" ;
> :nx = "185" ;
> :ny = "129 grid_points" ;
> The contents will change depending on your projection. So just
let me
> know what you're using.
> - Add the "MET_version" global attribute. Here's an example:
> :MET_version = "V4.0" ;
>
> Give that a shot, and if you get stuck, just send me what you have
and
> I'll let you know what's missing.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> On 03/20/2013 10:03 AM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
> >
> > Wed Mar 20 10:03:31 2013: Request 60725 was acted upon.
> > Transaction: Ticket created by dustan.wheatley at noaa.gov
> > Queue: met_help
> > Subject: Can I use MODE tool with radar observations?
> > Owner: Nobody
> > Requestors: dustan.wheatley at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
> >
> >
> > Hi,
> >
> > I am trying to determine if I can use the MODE tool for a research
> project
> > I am working on. I have radar data from NSSL's 3D radar mosaic
grid. It
> is
> > in netcdf format and on a 3km Cartesian grid, with 500-m spacing
in the
> > vertical. I am trying to find the "centroids" of storms in this
data
> set,
> > at a particular level (say 1.5 km). MODE seemed liked a possible
way to
> > accomplish this. I assumed that I could set the forecast and
observation
> > data to the same file at the command line, since I am not looking
to
> > compare to distinct sets (just get the attributes of one).
However, it
> > seems like in the user documentation that this tool is tuned
particularly
> > for precip. fields, and one must come at MODE using pcp_combine.
I am
> more
> > than willing to restructure my data in another format. I was just
> > wondering if someone could comment on if it is possible for me to
use
> radar
> > data with this tool (and what it might need to look like)??? I
have a
> > lat/lon, height, etc. for each observation value.)
> >
> > Thanks in advance,
> > Dusty Wheatley
> >
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60725] Can I use MODE tool with radar observations?
From: John Halley Gotway
Time: Thu Mar 21 13:55:34 2013
Hi Dusty,
My answers are inline...
On 03/21/2013 01:29 PM, Dustan Wheatley - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>
> John,
>
> Thanks! This was a great start. I have created a netcdf file as
you have
> instructed, but have a few questions. First, here's the ncdump:
>
> netcdf fcst {
> dimensions:
> lat = 151 ;
> lon = 151 ;
> variables:
> float REFL_10CM(lat, lon) ;
> REFL_10CM:name = "REFL_10CM" ;
> REFL_10CM:long_name = "Radar reflectivity factor" ;
> REFL_10CM:level = "A12" ;
> REFL_10CM:units = "dBZ" ;
> REFL_10CM:init_time = "20010520_204857" ;
> REFL_10CM:init_time_ut = 990391737 ;
> REFL_10CM:valid_time = "20010520_204857" ;
> REFL_10CM:valid_time_ut = 990391737 ;
> REFL_10CM:accum_time = "120000" ;
> REFL_10CM:accum_time_sec = 43200 ;
> REFL_10CM:_FillValue = -99900.f ;
>
> // global attributes:
> :Projection = "Lambert Conformal" ;
> :scale_lat_1 = "35.333100" ;
> :scale_lat_2 = "35.333100" ;
> :lat_pin = "12.190000" ;
> :lon_pin = "-133.459000" ;
> :x_pin = "0.000000" ;
> :y_pin = "0.000000" ;
> :lon_orient = "-97.277800" ;
> :d_km = "40.635000" ;
> :r_km = "6371.200000" ;
> :nx = "151" ;
> :ny = "151" ;
> :MET_version = "V3.0.1" ;
>
> Questions:
>
> ---> For REFL_10CM:level = "A12", does it matter what it says here.
The
> radar data is just at some height (2.5 km MSL)
No, it really doesn't matter much. If present, the level value may be
read by MET and written into a column in the output file. But
honestly I can't remember.
> ---> REFL_10CM:init_time = "20010520_204857", The next 4 attributes
are
> correct, radar data is valid at the given time. How do I handle
accum_time
> quantities (since it has no meaning in my case)?
> ---> REFL_10CM:init_time_ut = 990391737 ;
> ---> REFL_10CM:valid_time = "20010520_204857" ;
> ---> REFL_10CM:valid_time_ut = 990391737 ;
> ---> REFL_10CM:accum_time = "120000" ;
> ---> REFL_10CM:accum_time_sec = 43200 ;
> ---> REFL_10CM:_FillValue = -99900.f ;
>
Just set the accumulation interval to 0:
accum_time = "000000";
accum_time_sec = 0;
However, I think you're going to need to use a _FillValue of -9999.f
instead of -99900.f. I realize that that shouldn't matter, but I
think it will.
> ---> I am using Lambert Conformal projections. I filled in the
radar
> latitude (35.3331) for scale_lat_1,2 (assuming it's like truelat1,2
in wrf
> data) and the radar longitude (-97.2778) in lon_orient. I have a
151x151
> horizontal grid with 2-km spacing. Not quite certain what to do
with
> lat_pin, lon_pin, d_km? Assuming r_km is a constant(?).
>
> :scale_lat_1 = "35.333100" ;
> :scale_lat_2 = "35.333100" ;
> :lat_pin = "12.190000" ;
> :lon_pin = "-133.459000" ;
> :x_pin = "0.000000" ;
> :y_pin = "0.000000" ;
> :lon_orient = "-97.277800" ;
> :d_km = "40.635000" ;
> :r_km = "6371.200000" ;
> :nx = "151" ;
> :ny = "151" ;
> :MET_version = "V3.0.1" ;
>
Set x_pin = y_pin = 0.0, and then set lat_pin and lon_pin to the (lat,
lon) of the lower-left corner of your grid.
Yes, r_km is just a constant.
> I tried running (even knowing the projection values are somewhat
bogus),
> and I get:
>
> bigbang.protect.nssl:METtest> mode fcst.nc obs.nc config_file
> Forecast File: fcst.nc
> Observation File: obs.nc
> Match Config File: config_file
> Merge Config File: config_file
> Forecast Field: REFL_10CM at A12
> Observation Field: REFL_10CM at A12
> Identifying objects in the forecast and observation fields...
> Computing contingency table statistics...
> Identified: 0 forecast objects and 0 observation objects.
> Performing merging (no merging) in the forecast field.
> Performing merging (no merging) in the observation field.
> Remaining: 0 forecast objects and 0 observation objects.
> Performing matching between the forecast and observation fields.
> ***WARNING*** void Engine::do_matching() -> no matching requested in
> configuration file
>
My guess is that you're getting 0 objects because of the fill value
issue I mentioned above. Try using -9999 instead.
Give it a couple more tries, and if you can't get it, just send me
your data, and I'll take a look. I'd need your current NetCDF file
and the MODE configuration file you're using.
You can post it to our anonymous ftp site following these
instructions:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
If you do post data there, please write to let me know, and I'll go
grab it.
Also, since you're trying to use MODE, a bug was discovered just
yesterday relating to the area of the objects being reported. The
areas are artificially inflated in the output. The effect of the
bug is that roughly 1 extra row and column of data are being added to
each of the areas computed.
We're aware of the problem, and will resolve it prior to the METv4.1
release in the coming weeks.
Thanks,
John
>
> ERROR: write_ct_stats() -> unable to open stats output file
>
"/home/Dustan.Wheatley/METv3.0.1/out/mode/mode_~/_000000L_20010520_204857V_120000A_cts.txt"
>
> So, I think I am pretty close, with a few tweaks.
>
> Thanks,
> Dusty
>
>
>
>
> On Wed, Mar 20, 2013 at 1:08 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Dusty,
>>
>> Yep, it seems like MODE would be a good tool for you to use. Just
to be
>> clear, you just want to extract the lat/lon centroids of each
object.
>> You're not interested in comparing objects between
>> fields. If so, I'd suggest the following:
>> - Pass the same file in for the forecast and observation fields.
>> - Set up the forecast object definition information to define
objects
>> the way you'd like:
>> - Primarily, that's the "conv_radius" and "conv_thresh"
settings for
>> the convolution radius and threshold.
>> - Set the observation object definition information so that no
objects
>> are defined.
>> - Just set the "raw_thresh" to a large threshold so that no
raw data
>> values survive.
>> - Set the "merge_flag" to NONE in both the forecast and
observation
>> fields.
>> - Then set the "match_flag" to NONE so that MODE doesn't try to
compare
>> the fields.
>
>
>> This should result in only simple forecast objects being defined.
And you
>> can extract the object lat/lon values from the ASCII output file
from MODE.
>>
>> Now the only hurdle is formatting your data in such a way that MET
can
>> read it. I'd suggest making it look like the NetCDF output of the
>> PCP-Combine tool.
>>
>> Specifically, you'll need:
>> - Two dimensions named lat and lon.
>> - The variable name doesn't matter, but the attributes do.
>> - Add variable attributes. Here's an example:
>> float APCP_12(lat, lon) ;
>> APCP_12:name = "APCP_12" ;
>> APCP_12:long_name = "Total precipitation" ;
>> APCP_12:level = "A12" ;
>> APCP_12:units = "kg/m^2" ;
>> APCP_12:_FillValue = -9999.f ;
>> APCP_12:init_time = "20050807_000000" ;
>> APCP_12:init_time_ut = 1123372800 ;
>> APCP_12:valid_time = "20050807_120000" ;
>> APCP_12:valid_time_ut = 1123416000 ;
>> APCP_12:accum_time = "120000" ;
>> APCP_12:accum_time_sec = 43200 ;
>> Specifically, I believe we need the init_time_ut,
valid_time_ut, and
>> accum_time_sec. The ut stands for "unix time", the number of
seconds since
>> January 1, 1970.
>> - Add projection information to the global attributes. Here's an
example:
>> :Projection = "Lambert Conformal" ;
>> :scale_lat_1 = "25.000000" ;
>> :scale_lat_2 = "25.000000" ;
>> :lat_pin = "12.190000" ;
>> :lon_pin = "-133.459000" ;
>> :x_pin = "0.000000" ;
>> :y_pin = "0.000000" ;
>> :lon_orient = "-95.000000" ;
>> :d_km = "40.635000" ;
>> :r_km = "6371.200000" ;
>> :nx = "185" ;
>> :ny = "129 grid_points" ;
>> The contents will change depending on your projection. So
just let me
>> know what you're using.
>> - Add the "MET_version" global attribute. Here's an example:
>> :MET_version = "V4.0" ;
>>
>> Give that a shot, and if you get stuck, just send me what you have
and
>> I'll let you know what's missing.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>>
>> On 03/20/2013 10:03 AM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
>>>
>>> Wed Mar 20 10:03:31 2013: Request 60725 was acted upon.
>>> Transaction: Ticket created by dustan.wheatley at noaa.gov
>>> Queue: met_help
>>> Subject: Can I use MODE tool with radar observations?
>>> Owner: Nobody
>>> Requestors: dustan.wheatley at noaa.gov
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>>>
>>>
>>> Hi,
>>>
>>> I am trying to determine if I can use the MODE tool for a research
>> project
>>> I am working on. I have radar data from NSSL's 3D radar mosaic
grid. It
>> is
>>> in netcdf format and on a 3km Cartesian grid, with 500-m spacing
in the
>>> vertical. I am trying to find the "centroids" of storms in this
data
>> set,
>>> at a particular level (say 1.5 km). MODE seemed liked a possible
way to
>>> accomplish this. I assumed that I could set the forecast and
observation
>>> data to the same file at the command line, since I am not looking
to
>>> compare to distinct sets (just get the attributes of one).
However, it
>>> seems like in the user documentation that this tool is tuned
particularly
>>> for precip. fields, and one must come at MODE using pcp_combine.
I am
>> more
>>> than willing to restructure my data in another format. I was just
>>> wondering if someone could comment on if it is possible for me to
use
>> radar
>>> data with this tool (and what it might need to look like)??? I
have a
>>> lat/lon, height, etc. for each observation value.)
>>>
>>> Thanks in advance,
>>> Dusty Wheatley
>>>
>>
>>
------------------------------------------------
Subject: Can I use MODE tool with radar observations?
From: Dustan Wheatley - NOAA Affiliate
Time: Fri Mar 22 15:11:38 2013
Hi John,
I'm good. With your suggestions, I'm getting output in out/mode that
looks
very reasonable for my event. One last quick question: In the netcdf
attributes related to the grid projection, does 'r_km' refer the the
horizontal grid spacing or some effective grid spacing? Thanks again
for
the help.
-Dusty
On Thu, Mar 21, 2013 at 2:55 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hi Dusty,
>
> My answers are inline...
>
> On 03/21/2013 01:29 PM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
> >
> > John,
> >
> > Thanks! This was a great start. I have created a netcdf file as
you
> have
> > instructed, but have a few questions. First, here's the ncdump:
> >
> > netcdf fcst {
> > dimensions:
> > lat = 151 ;
> > lon = 151 ;
> > variables:
> > float REFL_10CM(lat, lon) ;
> > REFL_10CM:name = "REFL_10CM" ;
> > REFL_10CM:long_name = "Radar reflectivity factor" ;
> > REFL_10CM:level = "A12" ;
> > REFL_10CM:units = "dBZ" ;
> > REFL_10CM:init_time = "20010520_204857" ;
> > REFL_10CM:init_time_ut = 990391737 ;
> > REFL_10CM:valid_time = "20010520_204857" ;
> > REFL_10CM:valid_time_ut = 990391737 ;
> > REFL_10CM:accum_time = "120000" ;
> > REFL_10CM:accum_time_sec = 43200 ;
> > REFL_10CM:_FillValue = -99900.f ;
> >
> > // global attributes:
> > :Projection = "Lambert Conformal" ;
> > :scale_lat_1 = "35.333100" ;
> > :scale_lat_2 = "35.333100" ;
> > :lat_pin = "12.190000" ;
> > :lon_pin = "-133.459000" ;
> > :x_pin = "0.000000" ;
> > :y_pin = "0.000000" ;
> > :lon_orient = "-97.277800" ;
> > :d_km = "40.635000" ;
> > :r_km = "6371.200000" ;
> > :nx = "151" ;
> > :ny = "151" ;
> > :MET_version = "V3.0.1" ;
> >
> > Questions:
> >
> > ---> For REFL_10CM:level = "A12", does it matter what it says
here. The
> > radar data is just at some height (2.5 km MSL)
>
> No, it really doesn't matter much. If present, the level value may
be
> read by MET and written into a column in the output file. But
honestly I
> can't remember.
>
> > ---> REFL_10CM:init_time = "20010520_204857", The next 4
attributes are
> > correct, radar data is valid at the given time. How do I handle
> accum_time
> > quantities (since it has no meaning in my case)?
> > ---> REFL_10CM:init_time_ut = 990391737 ;
> > ---> REFL_10CM:valid_time = "20010520_204857" ;
> > ---> REFL_10CM:valid_time_ut = 990391737 ;
> > ---> REFL_10CM:accum_time = "120000" ;
> > ---> REFL_10CM:accum_time_sec = 43200 ;
> > ---> REFL_10CM:_FillValue = -99900.f ;
> >
>
> Just set the accumulation interval to 0:
> accum_time = "000000";
> accum_time_sec = 0;
>
> However, I think you're going to need to use a _FillValue of -9999.f
> instead of -99900.f. I realize that that shouldn't matter, but I
think it
> will.
>
> > ---> I am using Lambert Conformal projections. I filled in the
radar
> > latitude (35.3331) for scale_lat_1,2 (assuming it's like
truelat1,2 in
> wrf
> > data) and the radar longitude (-97.2778) in lon_orient. I have a
151x151
> > horizontal grid with 2-km spacing. Not quite certain what to do
with
> > lat_pin, lon_pin, d_km? Assuming r_km is a constant(?).
> >
> > :scale_lat_1 = "35.333100" ;
> > :scale_lat_2 = "35.333100" ;
> > :lat_pin = "12.190000" ;
> > :lon_pin = "-133.459000" ;
> > :x_pin = "0.000000" ;
> > :y_pin = "0.000000" ;
> > :lon_orient = "-97.277800" ;
> > :d_km = "40.635000" ;
> > :r_km = "6371.200000" ;
> > :nx = "151" ;
> > :ny = "151" ;
> > :MET_version = "V3.0.1" ;
> >
>
> Set x_pin = y_pin = 0.0, and then set lat_pin and lon_pin to the
(lat,
> lon) of the lower-left corner of your grid.
>
> Yes, r_km is just a constant.
>
> > I tried running (even knowing the projection values are somewhat
bogus),
> > and I get:
> >
> > bigbang.protect.nssl:METtest> mode fcst.nc obs.nc config_file
> > Forecast File: fcst.nc
> > Observation File: obs.nc
> > Match Config File: config_file
> > Merge Config File: config_file
> > Forecast Field: REFL_10CM at A12
> > Observation Field: REFL_10CM at A12
> > Identifying objects in the forecast and observation fields...
> > Computing contingency table statistics...
> > Identified: 0 forecast objects and 0 observation objects.
> > Performing merging (no merging) in the forecast field.
> > Performing merging (no merging) in the observation field.
> > Remaining: 0 forecast objects and 0 observation objects.
> > Performing matching between the forecast and observation fields.
> > ***WARNING*** void Engine::do_matching() -> no matching requested
in
> > configuration file
> >
>
> My guess is that you're getting 0 objects because of the fill value
issue
> I mentioned above. Try using -9999 instead.
>
> Give it a couple more tries, and if you can't get it, just send me
your
> data, and I'll take a look. I'd need your current NetCDF file and
the MODE
> configuration file you're using.
>
> You can post it to our anonymous ftp site following these
instructions:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> If you do post data there, please write to let me know, and I'll go
grab
> it.
>
> Also, since you're trying to use MODE, a bug was discovered just
yesterday
> relating to the area of the objects being reported. The areas are
> artificially inflated in the output. The effect of the
> bug is that roughly 1 extra row and column of data are being added
to each
> of the areas computed.
>
> We're aware of the problem, and will resolve it prior to the METv4.1
> release in the coming weeks.
>
> Thanks,
> John
>
>
> >
> > ERROR: write_ct_stats() -> unable to open stats output file
> >
>
"/home/Dustan.Wheatley/METv3.0.1/out/mode/mode_~/_000000L_20010520_204857V_120000A_cts.txt"
> >
> > So, I think I am pretty close, with a few tweaks.
> >
> > Thanks,
> > Dusty
> >
> >
> >
> >
> > On Wed, Mar 20, 2013 at 1:08 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Dusty,
> >>
> >> Yep, it seems like MODE would be a good tool for you to use.
Just to be
> >> clear, you just want to extract the lat/lon centroids of each
object.
> >> You're not interested in comparing objects between
> >> fields. If so, I'd suggest the following:
> >> - Pass the same file in for the forecast and observation
fields.
> >> - Set up the forecast object definition information to define
objects
> >> the way you'd like:
> >> - Primarily, that's the "conv_radius" and "conv_thresh"
settings
> for
> >> the convolution radius and threshold.
> >> - Set the observation object definition information so that no
> objects
> >> are defined.
> >> - Just set the "raw_thresh" to a large threshold so that no
raw
> data
> >> values survive.
> >> - Set the "merge_flag" to NONE in both the forecast and
observation
> >> fields.
> >> - Then set the "match_flag" to NONE so that MODE doesn't try
to
> compare
> >> the fields.
> >
> >
> >> This should result in only simple forecast objects being defined.
And
> you
> >> can extract the object lat/lon values from the ASCII output file
from
> MODE.
> >>
> >> Now the only hurdle is formatting your data in such a way that
MET can
> >> read it. I'd suggest making it look like the NetCDF output of
the
> >> PCP-Combine tool.
> >>
> >> Specifically, you'll need:
> >> - Two dimensions named lat and lon.
> >> - The variable name doesn't matter, but the attributes do.
> >> - Add variable attributes. Here's an example:
> >> float APCP_12(lat, lon) ;
> >> APCP_12:name = "APCP_12" ;
> >> APCP_12:long_name = "Total precipitation" ;
> >> APCP_12:level = "A12" ;
> >> APCP_12:units = "kg/m^2" ;
> >> APCP_12:_FillValue = -9999.f ;
> >> APCP_12:init_time = "20050807_000000" ;
> >> APCP_12:init_time_ut = 1123372800 ;
> >> APCP_12:valid_time = "20050807_120000" ;
> >> APCP_12:valid_time_ut = 1123416000 ;
> >> APCP_12:accum_time = "120000" ;
> >> APCP_12:accum_time_sec = 43200 ;
> >> Specifically, I believe we need the init_time_ut,
valid_time_ut,
> and
> >> accum_time_sec. The ut stands for "unix time", the number of
seconds
> since
> >> January 1, 1970.
> >> - Add projection information to the global attributes. Here's an
> example:
> >> :Projection = "Lambert Conformal" ;
> >> :scale_lat_1 = "25.000000" ;
> >> :scale_lat_2 = "25.000000" ;
> >> :lat_pin = "12.190000" ;
> >> :lon_pin = "-133.459000" ;
> >> :x_pin = "0.000000" ;
> >> :y_pin = "0.000000" ;
> >> :lon_orient = "-95.000000" ;
> >> :d_km = "40.635000" ;
> >> :r_km = "6371.200000" ;
> >> :nx = "185" ;
> >> :ny = "129 grid_points" ;
> >> The contents will change depending on your projection. So
just
> let me
> >> know what you're using.
> >> - Add the "MET_version" global attribute. Here's an example:
> >> :MET_version = "V4.0" ;
> >>
> >> Give that a shot, and if you get stuck, just send me what you
have and
> >> I'll let you know what's missing.
> >>
> >> Thanks,
> >> John Halley Gotway
> >> met_help at ucar.edu
> >>
> >>
> >> On 03/20/2013 10:03 AM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
> >>>
> >>> Wed Mar 20 10:03:31 2013: Request 60725 was acted upon.
> >>> Transaction: Ticket created by dustan.wheatley at noaa.gov
> >>> Queue: met_help
> >>> Subject: Can I use MODE tool with radar observations?
> >>> Owner: Nobody
> >>> Requestors: dustan.wheatley at noaa.gov
> >>> Status: new
> >>> Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
> >>>
> >>>
> >>> Hi,
> >>>
> >>> I am trying to determine if I can use the MODE tool for a
research
> >> project
> >>> I am working on. I have radar data from NSSL's 3D radar mosaic
grid.
> It
> >> is
> >>> in netcdf format and on a 3km Cartesian grid, with 500-m spacing
in the
> >>> vertical. I am trying to find the "centroids" of storms in this
data
> >> set,
> >>> at a particular level (say 1.5 km). MODE seemed liked a
possible way
> to
> >>> accomplish this. I assumed that I could set the forecast and
> observation
> >>> data to the same file at the command line, since I am not
looking to
> >>> compare to distinct sets (just get the attributes of one).
However, it
> >>> seems like in the user documentation that this tool is tuned
> particularly
> >>> for precip. fields, and one must come at MODE using pcp_combine.
I am
> >> more
> >>> than willing to restructure my data in another format. I was
just
> >>> wondering if someone could comment on if it is possible for me
to use
> >> radar
> >>> data with this tool (and what it might need to look like)??? I
have a
> >>> lat/lon, height, etc. for each observation value.)
> >>>
> >>> Thanks in advance,
> >>> Dusty Wheatley
> >>>
> >>
> >>
>
>
------------------------------------------------
Subject: Can I use MODE tool with radar observations?
From: Dustan Wheatley - NOAA Affiliate
Time: Fri Mar 22 15:16:44 2013
Correction to the email I just sent... I am interested in 'd_km.'
Does it
refer to the horizontal grid spacing or some effective grid spacing?
Thanks,
Dusty
On Fri, Mar 22, 2013 at 4:11 PM, Dustan Wheatley - NOAA Affiliate <
dustan.wheatley at noaa.gov> wrote:
> Hi John,
>
> I'm good. With your suggestions, I'm getting output in out/mode that
looks
> very reasonable for my event. One last quick question: In the
netcdf
> attributes related to the grid projection, does 'r_km' refer the the
> horizontal grid spacing or some effective grid spacing? Thanks
again for
> the help.
>
> -Dusty
>
>
>
>
> On Thu, Mar 21, 2013 at 2:55 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Hi Dusty,
>>
>> My answers are inline...
>>
>> On 03/21/2013 01:29 PM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>> >
>> > John,
>> >
>> > Thanks! This was a great start. I have created a netcdf file as
you
>> have
>> > instructed, but have a few questions. First, here's the ncdump:
>> >
>> > netcdf fcst {
>> > dimensions:
>> > lat = 151 ;
>> > lon = 151 ;
>> > variables:
>> > float REFL_10CM(lat, lon) ;
>> > REFL_10CM:name = "REFL_10CM" ;
>> > REFL_10CM:long_name = "Radar reflectivity factor" ;
>> > REFL_10CM:level = "A12" ;
>> > REFL_10CM:units = "dBZ" ;
>> > REFL_10CM:init_time = "20010520_204857" ;
>> > REFL_10CM:init_time_ut = 990391737 ;
>> > REFL_10CM:valid_time = "20010520_204857" ;
>> > REFL_10CM:valid_time_ut = 990391737 ;
>> > REFL_10CM:accum_time = "120000" ;
>> > REFL_10CM:accum_time_sec = 43200 ;
>> > REFL_10CM:_FillValue = -99900.f ;
>> >
>> > // global attributes:
>> > :Projection = "Lambert Conformal" ;
>> > :scale_lat_1 = "35.333100" ;
>> > :scale_lat_2 = "35.333100" ;
>> > :lat_pin = "12.190000" ;
>> > :lon_pin = "-133.459000" ;
>> > :x_pin = "0.000000" ;
>> > :y_pin = "0.000000" ;
>> > :lon_orient = "-97.277800" ;
>> > :d_km = "40.635000" ;
>> > :r_km = "6371.200000" ;
>> > :nx = "151" ;
>> > :ny = "151" ;
>> > :MET_version = "V3.0.1" ;
>> >
>> > Questions:
>> >
>> > ---> For REFL_10CM:level = "A12", does it matter what it says
here. The
>> > radar data is just at some height (2.5 km MSL)
>>
>> No, it really doesn't matter much. If present, the level value may
be
>> read by MET and written into a column in the output file. But
honestly I
>> can't remember.
>>
>> > ---> REFL_10CM:init_time = "20010520_204857", The next 4
attributes are
>> > correct, radar data is valid at the given time. How do I handle
>> accum_time
>> > quantities (since it has no meaning in my case)?
>> > ---> REFL_10CM:init_time_ut = 990391737 ;
>> > ---> REFL_10CM:valid_time = "20010520_204857" ;
>> > ---> REFL_10CM:valid_time_ut = 990391737 ;
>> > ---> REFL_10CM:accum_time = "120000" ;
>> > ---> REFL_10CM:accum_time_sec = 43200 ;
>> > ---> REFL_10CM:_FillValue = -99900.f ;
>> >
>>
>> Just set the accumulation interval to 0:
>> accum_time = "000000";
>> accum_time_sec = 0;
>>
>> However, I think you're going to need to use a _FillValue of
-9999.f
>> instead of -99900.f. I realize that that shouldn't matter, but I
think it
>> will.
>>
>> > ---> I am using Lambert Conformal projections. I filled in the
radar
>> > latitude (35.3331) for scale_lat_1,2 (assuming it's like
truelat1,2 in
>> wrf
>> > data) and the radar longitude (-97.2778) in lon_orient. I have a
>> 151x151
>> > horizontal grid with 2-km spacing. Not quite certain what to do
with
>> > lat_pin, lon_pin, d_km? Assuming r_km is a constant(?).
>> >
>> > :scale_lat_1 = "35.333100" ;
>> > :scale_lat_2 = "35.333100" ;
>> > :lat_pin = "12.190000" ;
>> > :lon_pin = "-133.459000" ;
>> > :x_pin = "0.000000" ;
>> > :y_pin = "0.000000" ;
>> > :lon_orient = "-97.277800" ;
>> > :d_km = "40.635000" ;
>> > :r_km = "6371.200000" ;
>> > :nx = "151" ;
>> > :ny = "151" ;
>> > :MET_version = "V3.0.1" ;
>> >
>>
>> Set x_pin = y_pin = 0.0, and then set lat_pin and lon_pin to the
(lat,
>> lon) of the lower-left corner of your grid.
>>
>> Yes, r_km is just a constant.
>>
>> > I tried running (even knowing the projection values are somewhat
bogus),
>> > and I get:
>> >
>> > bigbang.protect.nssl:METtest> mode fcst.nc obs.nc config_file
>> > Forecast File: fcst.nc
>> > Observation File: obs.nc
>> > Match Config File: config_file
>> > Merge Config File: config_file
>> > Forecast Field: REFL_10CM at A12
>> > Observation Field: REFL_10CM at A12
>> > Identifying objects in the forecast and observation fields...
>> > Computing contingency table statistics...
>> > Identified: 0 forecast objects and 0 observation objects.
>> > Performing merging (no merging) in the forecast field.
>> > Performing merging (no merging) in the observation field.
>> > Remaining: 0 forecast objects and 0 observation objects.
>> > Performing matching between the forecast and observation fields.
>> > ***WARNING*** void Engine::do_matching() -> no matching requested
in
>> > configuration file
>> >
>>
>> My guess is that you're getting 0 objects because of the fill value
issue
>> I mentioned above. Try using -9999 instead.
>>
>> Give it a couple more tries, and if you can't get it, just send me
your
>> data, and I'll take a look. I'd need your current NetCDF file and
the MODE
>> configuration file you're using.
>>
>> You can post it to our anonymous ftp site following these
instructions:
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> If you do post data there, please write to let me know, and I'll go
grab
>> it.
>>
>> Also, since you're trying to use MODE, a bug was discovered just
>> yesterday relating to the area of the objects being reported. The
areas
>> are artificially inflated in the output. The effect of the
>> bug is that roughly 1 extra row and column of data are being added
to
>> each of the areas computed.
>>
>> We're aware of the problem, and will resolve it prior to the
METv4.1
>> release in the coming weeks.
>>
>> Thanks,
>> John
>>
>>
>> >
>> > ERROR: write_ct_stats() -> unable to open stats output file
>> >
>>
"/home/Dustan.Wheatley/METv3.0.1/out/mode/mode_~/_000000L_20010520_204857V_120000A_cts.txt"
>> >
>> > So, I think I am pretty close, with a few tweaks.
>> >
>> > Thanks,
>> > Dusty
>> >
>> >
>> >
>> >
>> > On Wed, Mar 20, 2013 at 1:08 PM, John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> >> Dusty,
>> >>
>> >> Yep, it seems like MODE would be a good tool for you to use.
Just to
>> be
>> >> clear, you just want to extract the lat/lon centroids of each
object.
>> >> You're not interested in comparing objects between
>> >> fields. If so, I'd suggest the following:
>> >> - Pass the same file in for the forecast and observation
fields.
>> >> - Set up the forecast object definition information to define
>> objects
>> >> the way you'd like:
>> >> - Primarily, that's the "conv_radius" and "conv_thresh"
settings
>> for
>> >> the convolution radius and threshold.
>> >> - Set the observation object definition information so that
no
>> objects
>> >> are defined.
>> >> - Just set the "raw_thresh" to a large threshold so that no
raw
>> data
>> >> values survive.
>> >> - Set the "merge_flag" to NONE in both the forecast and
observation
>> >> fields.
>> >> - Then set the "match_flag" to NONE so that MODE doesn't try
to
>> compare
>> >> the fields.
>> >
>> >
>> >> This should result in only simple forecast objects being
defined. And
>> you
>> >> can extract the object lat/lon values from the ASCII output file
from
>> MODE.
>> >>
>> >> Now the only hurdle is formatting your data in such a way that
MET can
>> >> read it. I'd suggest making it look like the NetCDF output of
the
>> >> PCP-Combine tool.
>> >>
>> >> Specifically, you'll need:
>> >> - Two dimensions named lat and lon.
>> >> - The variable name doesn't matter, but the attributes do.
>> >> - Add variable attributes. Here's an example:
>> >> float APCP_12(lat, lon) ;
>> >> APCP_12:name = "APCP_12" ;
>> >> APCP_12:long_name = "Total precipitation" ;
>> >> APCP_12:level = "A12" ;
>> >> APCP_12:units = "kg/m^2" ;
>> >> APCP_12:_FillValue = -9999.f ;
>> >> APCP_12:init_time = "20050807_000000" ;
>> >> APCP_12:init_time_ut = 1123372800 ;
>> >> APCP_12:valid_time = "20050807_120000" ;
>> >> APCP_12:valid_time_ut = 1123416000 ;
>> >> APCP_12:accum_time = "120000" ;
>> >> APCP_12:accum_time_sec = 43200 ;
>> >> Specifically, I believe we need the init_time_ut,
valid_time_ut,
>> and
>> >> accum_time_sec. The ut stands for "unix time", the number of
seconds
>> since
>> >> January 1, 1970.
>> >> - Add projection information to the global attributes. Here's
an
>> example:
>> >> :Projection = "Lambert Conformal" ;
>> >> :scale_lat_1 = "25.000000" ;
>> >> :scale_lat_2 = "25.000000" ;
>> >> :lat_pin = "12.190000" ;
>> >> :lon_pin = "-133.459000" ;
>> >> :x_pin = "0.000000" ;
>> >> :y_pin = "0.000000" ;
>> >> :lon_orient = "-95.000000" ;
>> >> :d_km = "40.635000" ;
>> >> :r_km = "6371.200000" ;
>> >> :nx = "185" ;
>> >> :ny = "129 grid_points" ;
>> >> The contents will change depending on your projection. So
just
>> let me
>> >> know what you're using.
>> >> - Add the "MET_version" global attribute. Here's an example:
>> >> :MET_version = "V4.0" ;
>> >>
>> >> Give that a shot, and if you get stuck, just send me what you
have and
>> >> I'll let you know what's missing.
>> >>
>> >> Thanks,
>> >> John Halley Gotway
>> >> met_help at ucar.edu
>> >>
>> >>
>> >> On 03/20/2013 10:03 AM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
>> >>>
>> >>> Wed Mar 20 10:03:31 2013: Request 60725 was acted upon.
>> >>> Transaction: Ticket created by dustan.wheatley at noaa.gov
>> >>> Queue: met_help
>> >>> Subject: Can I use MODE tool with radar observations?
>> >>> Owner: Nobody
>> >>> Requestors: dustan.wheatley at noaa.gov
>> >>> Status: new
>> >>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>> >>>
>> >>>
>> >>> Hi,
>> >>>
>> >>> I am trying to determine if I can use the MODE tool for a
research
>> >> project
>> >>> I am working on. I have radar data from NSSL's 3D radar mosaic
grid.
>> It
>> >> is
>> >>> in netcdf format and on a 3km Cartesian grid, with 500-m
spacing in
>> the
>> >>> vertical. I am trying to find the "centroids" of storms in
this data
>> >> set,
>> >>> at a particular level (say 1.5 km). MODE seemed liked a
possible way
>> to
>> >>> accomplish this. I assumed that I could set the forecast and
>> observation
>> >>> data to the same file at the command line, since I am not
looking to
>> >>> compare to distinct sets (just get the attributes of one).
However,
>> it
>> >>> seems like in the user documentation that this tool is tuned
>> particularly
>> >>> for precip. fields, and one must come at MODE using
pcp_combine. I am
>> >> more
>> >>> than willing to restructure my data in another format. I was
just
>> >>> wondering if someone could comment on if it is possible for me
to use
>> >> radar
>> >>> data with this tool (and what it might need to look like)??? I
have a
>> >>> lat/lon, height, etc. for each observation value.)
>> >>>
>> >>> Thanks in advance,
>> >>> Dusty Wheatley
>> >>>
>> >>
>> >>
>>
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60725] Can I use MODE tool with radar observations?
From: John Halley Gotway
Time: Fri Mar 22 15:26:18 2013
Dusty,
The d_km entry specifies the grid spacing in kilometers. But that
grid spacing varies depending on the latitude. My co-worker who
primarily handles grids and projection information is out of the
office today. So I'm guessing here, but I believe that d_km is
measured at the "scale_lat_1" latitude.
If you have additional questions on this, I'll assign the ticket to my
coworker, and he can get back to you next week.
Hope that helps.
Thanks,
John
On 03/22/2013 03:11 PM, Dustan Wheatley - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>
> Hi John,
>
> I'm good. With your suggestions, I'm getting output in out/mode that
looks
> very reasonable for my event. One last quick question: In the
netcdf
> attributes related to the grid projection, does 'r_km' refer the the
> horizontal grid spacing or some effective grid spacing? Thanks
again for
> the help.
>
> -Dusty
>
>
>
> On Thu, Mar 21, 2013 at 2:55 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Hi Dusty,
>>
>> My answers are inline...
>>
>> On 03/21/2013 01:29 PM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>>>
>>> John,
>>>
>>> Thanks! This was a great start. I have created a netcdf file as
you
>> have
>>> instructed, but have a few questions. First, here's the ncdump:
>>>
>>> netcdf fcst {
>>> dimensions:
>>> lat = 151 ;
>>> lon = 151 ;
>>> variables:
>>> float REFL_10CM(lat, lon) ;
>>> REFL_10CM:name = "REFL_10CM" ;
>>> REFL_10CM:long_name = "Radar reflectivity factor" ;
>>> REFL_10CM:level = "A12" ;
>>> REFL_10CM:units = "dBZ" ;
>>> REFL_10CM:init_time = "20010520_204857" ;
>>> REFL_10CM:init_time_ut = 990391737 ;
>>> REFL_10CM:valid_time = "20010520_204857" ;
>>> REFL_10CM:valid_time_ut = 990391737 ;
>>> REFL_10CM:accum_time = "120000" ;
>>> REFL_10CM:accum_time_sec = 43200 ;
>>> REFL_10CM:_FillValue = -99900.f ;
>>>
>>> // global attributes:
>>> :Projection = "Lambert Conformal" ;
>>> :scale_lat_1 = "35.333100" ;
>>> :scale_lat_2 = "35.333100" ;
>>> :lat_pin = "12.190000" ;
>>> :lon_pin = "-133.459000" ;
>>> :x_pin = "0.000000" ;
>>> :y_pin = "0.000000" ;
>>> :lon_orient = "-97.277800" ;
>>> :d_km = "40.635000" ;
>>> :r_km = "6371.200000" ;
>>> :nx = "151" ;
>>> :ny = "151" ;
>>> :MET_version = "V3.0.1" ;
>>>
>>> Questions:
>>>
>>> ---> For REFL_10CM:level = "A12", does it matter what it says
here. The
>>> radar data is just at some height (2.5 km MSL)
>>
>> No, it really doesn't matter much. If present, the level value may
be
>> read by MET and written into a column in the output file. But
honestly I
>> can't remember.
>>
>>> ---> REFL_10CM:init_time = "20010520_204857", The next 4
attributes are
>>> correct, radar data is valid at the given time. How do I handle
>> accum_time
>>> quantities (since it has no meaning in my case)?
>>> ---> REFL_10CM:init_time_ut = 990391737 ;
>>> ---> REFL_10CM:valid_time = "20010520_204857" ;
>>> ---> REFL_10CM:valid_time_ut = 990391737 ;
>>> ---> REFL_10CM:accum_time = "120000" ;
>>> ---> REFL_10CM:accum_time_sec = 43200 ;
>>> ---> REFL_10CM:_FillValue = -99900.f ;
>>>
>>
>> Just set the accumulation interval to 0:
>> accum_time = "000000";
>> accum_time_sec = 0;
>>
>> However, I think you're going to need to use a _FillValue of
-9999.f
>> instead of -99900.f. I realize that that shouldn't matter, but I
think it
>> will.
>>
>>> ---> I am using Lambert Conformal projections. I filled in the
radar
>>> latitude (35.3331) for scale_lat_1,2 (assuming it's like
truelat1,2 in
>> wrf
>>> data) and the radar longitude (-97.2778) in lon_orient. I have a
151x151
>>> horizontal grid with 2-km spacing. Not quite certain what to do
with
>>> lat_pin, lon_pin, d_km? Assuming r_km is a constant(?).
>>>
>>> :scale_lat_1 = "35.333100" ;
>>> :scale_lat_2 = "35.333100" ;
>>> :lat_pin = "12.190000" ;
>>> :lon_pin = "-133.459000" ;
>>> :x_pin = "0.000000" ;
>>> :y_pin = "0.000000" ;
>>> :lon_orient = "-97.277800" ;
>>> :d_km = "40.635000" ;
>>> :r_km = "6371.200000" ;
>>> :nx = "151" ;
>>> :ny = "151" ;
>>> :MET_version = "V3.0.1" ;
>>>
>>
>> Set x_pin = y_pin = 0.0, and then set lat_pin and lon_pin to the
(lat,
>> lon) of the lower-left corner of your grid.
>>
>> Yes, r_km is just a constant.
>>
>>> I tried running (even knowing the projection values are somewhat
bogus),
>>> and I get:
>>>
>>> bigbang.protect.nssl:METtest> mode fcst.nc obs.nc config_file
>>> Forecast File: fcst.nc
>>> Observation File: obs.nc
>>> Match Config File: config_file
>>> Merge Config File: config_file
>>> Forecast Field: REFL_10CM at A12
>>> Observation Field: REFL_10CM at A12
>>> Identifying objects in the forecast and observation fields...
>>> Computing contingency table statistics...
>>> Identified: 0 forecast objects and 0 observation objects.
>>> Performing merging (no merging) in the forecast field.
>>> Performing merging (no merging) in the observation field.
>>> Remaining: 0 forecast objects and 0 observation objects.
>>> Performing matching between the forecast and observation fields.
>>> ***WARNING*** void Engine::do_matching() -> no matching requested
in
>>> configuration file
>>>
>>
>> My guess is that you're getting 0 objects because of the fill value
issue
>> I mentioned above. Try using -9999 instead.
>>
>> Give it a couple more tries, and if you can't get it, just send me
your
>> data, and I'll take a look. I'd need your current NetCDF file and
the MODE
>> configuration file you're using.
>>
>> You can post it to our anonymous ftp site following these
instructions:
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> If you do post data there, please write to let me know, and I'll go
grab
>> it.
>>
>> Also, since you're trying to use MODE, a bug was discovered just
yesterday
>> relating to the area of the objects being reported. The areas are
>> artificially inflated in the output. The effect of the
>> bug is that roughly 1 extra row and column of data are being added
to each
>> of the areas computed.
>>
>> We're aware of the problem, and will resolve it prior to the
METv4.1
>> release in the coming weeks.
>>
>> Thanks,
>> John
>>
>>
>>>
>>> ERROR: write_ct_stats() -> unable to open stats output file
>>>
>>
"/home/Dustan.Wheatley/METv3.0.1/out/mode/mode_~/_000000L_20010520_204857V_120000A_cts.txt"
>>>
>>> So, I think I am pretty close, with a few tweaks.
>>>
>>> Thanks,
>>> Dusty
>>>
>>>
>>>
>>>
>>> On Wed, Mar 20, 2013 at 1:08 PM, John Halley Gotway via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>> Dusty,
>>>>
>>>> Yep, it seems like MODE would be a good tool for you to use.
Just to be
>>>> clear, you just want to extract the lat/lon centroids of each
object.
>>>> You're not interested in comparing objects between
>>>> fields. If so, I'd suggest the following:
>>>> - Pass the same file in for the forecast and observation
fields.
>>>> - Set up the forecast object definition information to define
objects
>>>> the way you'd like:
>>>> - Primarily, that's the "conv_radius" and "conv_thresh"
settings
>> for
>>>> the convolution radius and threshold.
>>>> - Set the observation object definition information so that
no
>> objects
>>>> are defined.
>>>> - Just set the "raw_thresh" to a large threshold so that no
raw
>> data
>>>> values survive.
>>>> - Set the "merge_flag" to NONE in both the forecast and
observation
>>>> fields.
>>>> - Then set the "match_flag" to NONE so that MODE doesn't try
to
>> compare
>>>> the fields.
>>>
>>>
>>>> This should result in only simple forecast objects being defined.
And
>> you
>>>> can extract the object lat/lon values from the ASCII output file
from
>> MODE.
>>>>
>>>> Now the only hurdle is formatting your data in such a way that
MET can
>>>> read it. I'd suggest making it look like the NetCDF output of
the
>>>> PCP-Combine tool.
>>>>
>>>> Specifically, you'll need:
>>>> - Two dimensions named lat and lon.
>>>> - The variable name doesn't matter, but the attributes do.
>>>> - Add variable attributes. Here's an example:
>>>> float APCP_12(lat, lon) ;
>>>> APCP_12:name = "APCP_12" ;
>>>> APCP_12:long_name = "Total precipitation" ;
>>>> APCP_12:level = "A12" ;
>>>> APCP_12:units = "kg/m^2" ;
>>>> APCP_12:_FillValue = -9999.f ;
>>>> APCP_12:init_time = "20050807_000000" ;
>>>> APCP_12:init_time_ut = 1123372800 ;
>>>> APCP_12:valid_time = "20050807_120000" ;
>>>> APCP_12:valid_time_ut = 1123416000 ;
>>>> APCP_12:accum_time = "120000" ;
>>>> APCP_12:accum_time_sec = 43200 ;
>>>> Specifically, I believe we need the init_time_ut,
valid_time_ut,
>> and
>>>> accum_time_sec. The ut stands for "unix time", the number of
seconds
>> since
>>>> January 1, 1970.
>>>> - Add projection information to the global attributes. Here's an
>> example:
>>>> :Projection = "Lambert Conformal" ;
>>>> :scale_lat_1 = "25.000000" ;
>>>> :scale_lat_2 = "25.000000" ;
>>>> :lat_pin = "12.190000" ;
>>>> :lon_pin = "-133.459000" ;
>>>> :x_pin = "0.000000" ;
>>>> :y_pin = "0.000000" ;
>>>> :lon_orient = "-95.000000" ;
>>>> :d_km = "40.635000" ;
>>>> :r_km = "6371.200000" ;
>>>> :nx = "185" ;
>>>> :ny = "129 grid_points" ;
>>>> The contents will change depending on your projection. So
just
>> let me
>>>> know what you're using.
>>>> - Add the "MET_version" global attribute. Here's an example:
>>>> :MET_version = "V4.0" ;
>>>>
>>>> Give that a shot, and if you get stuck, just send me what you
have and
>>>> I'll let you know what's missing.
>>>>
>>>> Thanks,
>>>> John Halley Gotway
>>>> met_help at ucar.edu
>>>>
>>>>
>>>> On 03/20/2013 10:03 AM, Dustan Wheatley - NOAA Affiliate via RT
wrote:
>>>>>
>>>>> Wed Mar 20 10:03:31 2013: Request 60725 was acted upon.
>>>>> Transaction: Ticket created by dustan.wheatley at noaa.gov
>>>>> Queue: met_help
>>>>> Subject: Can I use MODE tool with radar observations?
>>>>> Owner: Nobody
>>>>> Requestors: dustan.wheatley at noaa.gov
>>>>> Status: new
>>>>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am trying to determine if I can use the MODE tool for a
research
>>>> project
>>>>> I am working on. I have radar data from NSSL's 3D radar mosaic
grid.
>> It
>>>> is
>>>>> in netcdf format and on a 3km Cartesian grid, with 500-m spacing
in the
>>>>> vertical. I am trying to find the "centroids" of storms in this
data
>>>> set,
>>>>> at a particular level (say 1.5 km). MODE seemed liked a
possible way
>> to
>>>>> accomplish this. I assumed that I could set the forecast and
>> observation
>>>>> data to the same file at the command line, since I am not
looking to
>>>>> compare to distinct sets (just get the attributes of one).
However, it
>>>>> seems like in the user documentation that this tool is tuned
>> particularly
>>>>> for precip. fields, and one must come at MODE using pcp_combine.
I am
>>>> more
>>>>> than willing to restructure my data in another format. I was
just
>>>>> wondering if someone could comment on if it is possible for me
to use
>>>> radar
>>>>> data with this tool (and what it might need to look like)??? I
have a
>>>>> lat/lon, height, etc. for each observation value.)
>>>>>
>>>>> Thanks in advance,
>>>>> Dusty Wheatley
>>>>>
>>>>
>>>>
>>
>>
------------------------------------------------
Subject: Can I use MODE tool with radar observations?
From: Dustan Wheatley - NOAA Affiliate
Time: Tue Mar 26 14:04:46 2013
Hi, one more question and I am done with ticket. Correctly
interpreting
the projection information is important to my project, so I just want
to
make certain I've got it. My interpretation of various global
attributes
in the netcdf file fed to MODE is listed below (with varying degrees
of
certainty). :
// global attributes:
:Projection = "Lambert Conformal"
:scale_lat_1 = "35.180000" ---> Are scale_lat_1,
scale_lat_2
analogous to TRUELAT1, TRUELAT2 in a wrfinput or wrfout file **???**
:scale_lat_2 = "35.180000"
:lat_pin = "29.496900" ---> Latitude at SOUTHEAST
corner of my domain
:lon_pin = "-106.938000" ---> Longitude at
SOUTHEAST
corner of my domain
:lon_orient = "-100.000000" ---> Is lon_orient
analogous to
STAND_LON in a wrfinput or wrfout file **???**
:d_km = "3.000000" ---> Horizontal grid
spacing at
scale_lat_1 **???**
:r_km = "6371.200000" ---> a constant; earth's
radius
Thanks (please consider my ticket closed after this request),
Dusty
On Tue, Mar 26, 2013 at 11:00 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> According to our records, your request has been resolved. If you
have any
> further questions or concerns, please respond to this message.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #60725] Resolved: Can I use MODE tool with radar observations?
From: John Halley Gotway
Time: Thu Mar 28 11:21:03 2013
Dusty,
Sorry for the delay in getting back to you. Here's what I did... I
took a wrfout NetCDF file, ran it through the Unified PostProcessor to
get a GRIB file, and then ran that GRIB file through the
PCP-Combine tool in MET to get a MET NetCDF file. Then I compared the
projection information from the wrfout NetCDF file to that in the MET
NetCDF file.
Here's the results...
- scale_lat_1 in MET = TRUELAT1 in wrfout.
- scale_lat_2 in MET = TRUELAT2 in wrfout.
- Setting x_pin = y_pin = 0 means the SOUTHEAST corner.
- lat_pin in MET is the latitude for the (x,y) pin.
- lon_pin in MET is the longitude for the (x,y) pin.
- lon_orient in MET = STAND_LON in wrfout.
- d_km in MET = DX and DY in wrfout
- r_km is the assumed radius of a spherical earth.
- nx in MET = west_east dimension in wrfout
- ny in MET = south_north dimension in wrfout
So yes, your interpretations all looks correct to me. You can always
look at the plot you get when running your data through MODE.
Thanks,
John
On 03/26/2013 02:04 PM, Dustan Wheatley - NOAA Affiliate via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=60725 >
>
> Hi, one more question and I am done with ticket. Correctly
interpreting
> the projection information is important to my project, so I just
want to
> make certain I've got it. My interpretation of various global
attributes
> in the netcdf file fed to MODE is listed below (with varying degrees
of
> certainty). :
>
> // global attributes:
> :Projection = "Lambert Conformal"
> :scale_lat_1 = "35.180000" ---> Are scale_lat_1,
scale_lat_2
> analogous to TRUELAT1, TRUELAT2 in a wrfinput or wrfout file **???**
> :scale_lat_2 = "35.180000"
> :lat_pin = "29.496900" ---> Latitude at
SOUTHEAST
> corner of my domain
> :lon_pin = "-106.938000" ---> Longitude at
SOUTHEAST
> corner of my domain
> :lon_orient = "-100.000000" ---> Is lon_orient
analogous to
> STAND_LON in a wrfinput or wrfout file **???**
> :d_km = "3.000000" ---> Horizontal grid
spacing at
> scale_lat_1 **???**
> :r_km = "6371.200000" ---> a constant; earth's
radius
>
> Thanks (please consider my ticket closed after this request),
> Dusty
>
> On Tue, Mar 26, 2013 at 11:00 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> According to our records, your request has been resolved. If you
have any
>> further questions or concerns, please respond to this message.
>>
------------------------------------------------
More information about the Met_help
mailing list