[Met_help] [rt.rap.ucar.edu #42534] History for poly mask for Stage IV data
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Mon Dec 6 12:07:02 MST 2010
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am looking for a way to create a poly mask for the entire Stage IV
precipitation analysis domain (it is comprised of many rectangles glued
together) to use as a mask when running MODE. There isn't a mask.poly file
in data/poly that I can see. I looked at the gen_poly_mask tool but it
didn't work the way I thought it would (it wants a lat/lon ASCII file as
input...if the utility needs an input ASCII file then why not just make that
file the *.poly file and use it as the mask in MODE instead of going to the
trouble to use the gen_poly_mask tool to do it?). I have some data to use,
and here is what I entered in the config file:
mask_poly = "winter_pcp APCP_60";
The netCDF file winter_pcp indeed exists with the variable APCP_60 which
looks correct when viewed. When I run mode with it, however, it gives me
this error: NetCDF: Attribute not found
I made sure to use mask_poly_flag = 1
Jeff Duda
--
Jeff Duda
Iowa State University
Meteorology Graduate Student
3134 Agronomy Hall
www.meteor.iastate.edu/~jdduda
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: poly mask for Stage IV data
From: John Halley Gotway
Time: Mon Dec 06 09:57:10 2010
Jeff,
Yes you're right, Gen-Poly-Mask reads in an ASCII file and a gridded
file, applies the polyline to the grid, and writes out a NetCDF file
containing 0's and 1's. And yes, when you run MODE or the
other MET tools you could supply that ASCII file directly. However,
we found that some users have VERY complex polylines. For example,
one user had a polyline consisting of over 1000 points to
described the boundary of the continental US. He found that every
time he ran Grid-Stat on a 4-km CONUS domain, it took several minutes
just to apply the polyline to his grid. That's why we created
Gen-Poly-Mask - so you could apply the polyline to your grid once and
not waste time each time you run the MET statistics tool recomputing
the masking region over and over again.
It sounds like you have a NetCDF file you're trying to use to define
your mask, and you're getting an "NetCDF: Attribute not found". I'm
guessing that that NetCDF file just doesn't contain the
attributes that MET is expecting to find to define the grid on which
the data resides.
In this case, I'd suggest you use the "data masking" option. Just try
the following steps:
(1) Go retrieve a sample Stage4 file:
wget
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/hourly/prod/nam_pcpn_anal.20101205/ST4.2010120512.06h.Z
gunzip ST4.2010120512.06h.Z
(2) Run it through copygb to interpolate it to your model domain. You
can refer to the MET Online Tutorial if you need help:
http://www.dtcenter.org/met/users/support/online_tutorial/METv3.0/copygb/index.php
For example, I interpolated it to NCEP grid 212, for use with the data
used by the MET test scripts:
copygb -xg"212" ST4.2010120512.06h ST4_G212.grib
(3) Next, edit the MODE configuration file to do "data masking":
cd METv3.0/scripts
vi config/WrfModeConfig_RH
Set:
mask_poly = "ST4_G212.grib APCP/A06 ge0.0";
mask_poly_flag = 3;
This will read the field "APCP/A06" from the file "ST4_G212.grb" and
only use the points where that field is >= 0.0.
(4) Rerun the MODE test script, and look at the resulting PostScript.
I've attached an image showing how the data masking worked.
The exact values in the Stage4 file you use don't really matter.
You're just interested in masking out the region where you have valid
data (i.e. >=0.0).
Hope this helps.
John
On 12/04/2010 01:51 PM, RAL HelpDesk {for jdduda at iastate.edu} wrote:
>
> Sat Dec 04 13:51:16 2010: Request 42534 was acted upon.
> Transaction: Ticket created by jdduda at iastate.edu
> Queue: met_help
> Subject: poly mask for Stage IV data
> Owner: Nobody
> Requestors: jdduda at iastate.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=42534 >
>
>
> Hello,
> I am looking for a way to create a poly mask for the entire Stage IV
> precipitation analysis domain (it is comprised of many rectangles
glued
> together) to use as a mask when running MODE. There isn't a
mask.poly file
> in data/poly that I can see. I looked at the gen_poly_mask tool but
it
> didn't work the way I thought it would (it wants a lat/lon ASCII
file as
> input...if the utility needs an input ASCII file then why not just
make that
> file the *.poly file and use it as the mask in MODE instead of going
to the
> trouble to use the gen_poly_mask tool to do it?). I have some data
to use,
> and here is what I entered in the config file:
>
> mask_poly = "winter_pcp APCP_60";
>
> The netCDF file winter_pcp indeed exists with the variable APCP_60
which
> looks correct when viewed. When I run mode with it, however, it
gives me
> this error: NetCDF: Attribute not found
>
> I made sure to use mask_poly_flag = 1
>
>
> Jeff Duda
>
------------------------------------------------
Subject: poly mask for Stage IV data
From: jdduda at iastate.edu
Time: Mon Dec 06 11:21:04 2010
John,
Something just isn't working right, and it's frustrating me. First of
all I
was trying to do this using V2.0. I got nowhere on that so I tried
using
V3.0. Still getting the same errors no matter what I try to do. It's
like
the netCDF file that has my observed precip in it is messed up or
something. But I know it's not because I can plot it with correct
geography
and everything. See attached image.
My forecast data set is in GRIB format. My observed data set is in
netCDF
format (output from pcp_combine -sum). The precip variable in the
netCDF
file is APCP_60. If I put just that in there it says:
ERROR: GCInfo::set_gcinfo() -> each GRIB code specified must be
followed by
an accumulation, level, or presssure level indicator "APCP_60".
If I add "/A60" to the end of "APCP_60", then it gives me the
"attribute not
found" error. It also didn't work to call it "APCP_60(*,*)"
Lastly, since my observation data set is on the same grid as the
masking
polynomial I wish to apply, I only want to mask the forecast field, so
I'm
using "mask_poly_flag = 1".
I've uploaded the neccessary files to your site.
Jeff
P.S. I reuploaded the data from the other issue awhile back and
haven't
heard from you on that.
On Mon, Dec 6, 2010 at 10:57 AM, RAL HelpDesk {for John Halley Gotway}
<
met_help at ucar.edu> wrote:
> Jeff,
>
> Yes you're right, Gen-Poly-Mask reads in an ASCII file and a gridded
file,
> applies the polyline to the grid, and writes out a NetCDF file
containing
> 0's and 1's. And yes, when you run MODE or the
> other MET tools you could supply that ASCII file directly. However,
we
> found that some users have VERY complex polylines. For example, one
user
> had a polyline consisting of over 1000 points to
> described the boundary of the continental US. He found that every
time he
> ran Grid-Stat on a 4-km CONUS domain, it took several minutes just
to apply
> the polyline to his grid. That's why we created
> Gen-Poly-Mask - so you could apply the polyline to your grid once
and not
> waste time each time you run the MET statistics tool recomputing the
masking
> region over and over again.
>
> It sounds like you have a NetCDF file you're trying to use to define
your
> mask, and you're getting an "NetCDF: Attribute not found". I'm
guessing
> that that NetCDF file just doesn't contain the
> attributes that MET is expecting to find to define the grid on which
the
> data resides.
>
> In this case, I'd suggest you use the "data masking" option. Just
try the
> following steps:
>
> (1) Go retrieve a sample Stage4 file:
> wget
>
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/hourly/prod/nam_pcpn_anal.20101205/ST4.2010120512.06h.Z
> gunzip ST4.2010120512.06h.Z
>
> (2) Run it through copygb to interpolate it to your model domain.
You can
> refer to the MET Online Tutorial if you need help:
>
>
http://www.dtcenter.org/met/users/support/online_tutorial/METv3.0/copygb/index.php
>
> For example, I interpolated it to NCEP grid 212, for use with the
data used
> by the MET test scripts:
> copygb -xg"212" ST4.2010120512.06h ST4_G212.grib
>
> (3) Next, edit the MODE configuration file to do "data masking":
> cd METv3.0/scripts
> vi config/WrfModeConfig_RH
> Set:
> mask_poly = "ST4_G212.grib APCP/A06 ge0.0";
> mask_poly_flag = 3;
>
> This will read the field "APCP/A06" from the file "ST4_G212.grb" and
only
> use the points where that field is >= 0.0.
>
> (4) Rerun the MODE test script, and look at the resulting
PostScript. I've
> attached an image showing how the data masking worked.
>
>
> The exact values in the Stage4 file you use don't really matter.
You're
> just interested in masking out the region where you have valid data
(i.e.
> >=0.0).
>
> Hope this helps.
>
> John
>
> On 12/04/2010 01:51 PM, RAL HelpDesk {for jdduda at iastate.edu} wrote:
> >
> > Sat Dec 04 13:51:16 2010: Request 42534 was acted upon.
> > Transaction: Ticket created by jdduda at iastate.edu
> > Queue: met_help
> > Subject: poly mask for Stage IV data
> > Owner: Nobody
> > Requestors: jdduda at iastate.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=42534 >
> >
> >
> > Hello,
> > I am looking for a way to create a poly mask for the entire Stage
IV
> > precipitation analysis domain (it is comprised of many rectangles
glued
> > together) to use as a mask when running MODE. There isn't a
mask.poly
> file
> > in data/poly that I can see. I looked at the gen_poly_mask tool
but it
> > didn't work the way I thought it would (it wants a lat/lon ASCII
file as
> > input...if the utility needs an input ASCII file then why not just
make
> that
> > file the *.poly file and use it as the mask in MODE instead of
going to
> the
> > trouble to use the gen_poly_mask tool to do it?). I have some
data to
> use,
> > and here is what I entered in the config file:
> >
> > mask_poly = "winter_pcp APCP_60";
> >
> > The netCDF file winter_pcp indeed exists with the variable APCP_60
which
> > looks correct when viewed. When I run mode with it, however, it
gives me
> > this error: NetCDF: Attribute not found
> >
> > I made sure to use mask_poly_flag = 1
> >
> >
> > Jeff Duda
> >
>
>
--
Jeff Duda
Iowa State University
Meteorology Graduate Student
3134 Agronomy Hall
www.meteor.iastate.edu/~jdduda
------------------------------------------------
Subject: poly mask for Stage IV data
From: John Halley Gotway
Time: Mon Dec 06 11:38:45 2010
Jeff,
Yes, sorry about not getting to that WRFPost issue. I'm swamped
working on some stuff that was supposed to be due December 1st. I
apologize for not getting to your issue yet.
You know, I ran with your data exactly as you sent it and it ran fine
without any errors. I have good news for you - you basically just
want to mask out the bad data from one field to the other. And
there's a much easier way for you to do this in MODE than defining a
mask. Instead, use the "mask_missing_flag" and set it to 3:
mask_missing_flag = 3;
That masks out the bad data back and forth between the forecast and
observation fields - so you only use grid points that contain good
data in both fields. I believe that'll have the desired effect.
Please see the attached image.
Regarding the syntax for the type of masking you were trying to do
that was so frustrating, I believe the following does the trick:
mask_poly = "winter_pcp APCP_60 >=0.0";
mask_poly_flag = 1;
But like I said, use the "mask_missing_flag" parameter instead.
Hope that helps.
John
On 12/06/2010 11:21 AM, RAL HelpDesk {for jdduda at iastate.edu} wrote:
> If I add "/A60" to the end of "APCP_60", then it gives me the
"attribute not
------------------------------------------------
Subject: poly mask for Stage IV data
From: jdduda at iastate.edu
Time: Mon Dec 06 12:01:01 2010
John,
Oh...
1) I was using the wrong executable (v2 when I meant to use that for
v3)
2) Thank you. The mask missing flag worked.
Jeff
On Mon, Dec 6, 2010 at 12:38 PM, RAL HelpDesk {for John Halley Gotway}
<
met_help at ucar.edu> wrote:
> Jeff,
>
> Yes, sorry about not getting to that WRFPost issue. I'm swamped
working on
> some stuff that was supposed to be due December 1st. I apologize
for not
> getting to your issue yet.
>
> You know, I ran with your data exactly as you sent it and it ran
fine
> without any errors. I have good news for you - you basically just
want to
> mask out the bad data from one field to the other. And
> there's a much easier way for you to do this in MODE than defining a
mask.
> Instead, use the "mask_missing_flag" and set it to 3:
> mask_missing_flag = 3;
>
> That masks out the bad data back and forth between the forecast and
> observation fields - so you only use grid points that contain good
data in
> both fields. I believe that'll have the desired effect.
> Please see the attached image.
>
> Regarding the syntax for the type of masking you were trying to do
that was
> so frustrating, I believe the following does the trick:
> mask_poly = "winter_pcp APCP_60 >=0.0";
> mask_poly_flag = 1;
>
> But like I said, use the "mask_missing_flag" parameter instead.
>
> Hope that helps.
>
> John
>
>
>
> On 12/06/2010 11:21 AM, RAL HelpDesk {for jdduda at iastate.edu} wrote:
> > If I add "/A60" to the end of "APCP_60", then it gives me the
"attribute
> not
>
>
--
Jeff Duda
Iowa State University
Meteorology Graduate Student
3134 Agronomy Hall
www.meteor.iastate.edu/~jdduda
------------------------------------------------
More information about the Met_help
mailing list