[Met_help] [rt.rap.ucar.edu #64842] History for MET Masking for Clouds
John Halley Gotway via RT
met_help at ucar.edu
Fri Jan 22 13:17:45 MST 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
We are using MODE in an attempt to define cloud objects. It's a work in progress, and definitely needs some tweaks to get it working better. One thing we need is the ability to mask data based on WWMCA pixel age. It's in the same GriB file as the actual cloud data. We need to set a threshold in which we keep all points with an age less than 120 minutes. Can this be done in MET 4.1? If so, how would you go about doing it?
Thanks,
Matt
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: Randy Bullock
Time: Fri Jan 10 10:41:08 2014
Hi Matt -
I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
We ran a test case masking out a precip field with a temperature field
in the same input file. Here's the relevant MODE config file entry
for our test case:
//
// Verification masking regions
//
mask = {
grid = "";
grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
};
In the "poly" part of the masking region spec, you give a string with
the data file,
the name and level of the field you want to mask on, and a threshold.
Copy the syntax
from the example above. You'd need to change the field name from
"TMP" to whatever
your name is for the pixel age field.
So it seems that it can be done. Go figure. Having said that,
though,
I feel compelled to point out that data field masking for MODE might
not
be a good idea unless the masking results in large connected blocks of
the grid being turned off (as may well be the case with pixel age ...
you'd know
better than I would).
Also, if you were working from the original WWMCA files instead of a
grib file,
the wwmca_regrid tool can threshold by pixel age. FYI.
Hope this helps.
Randy
On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> Queue: met_help
> Subject: MET Masking for Clouds
> Owner: Nobody
> Requestors: matthew.sittel.ctr at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
>
> We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
>
> Thanks,
> Matt
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Fri Jan 10 10:51:34 2014
Thanks Randy. I was only asked to see if it could be done.
I'm not convinced it's a good idea since I think we're going to toss
way too many points in the process and leave a fractured domain. But
I'm not in charge so... will hold 'I told you so' in reserve for later
though! Let me take a look and I'll let you know how it works for us.
I hope all is well with you. I need a trip to Boulder... been far too
long. Not going to AMS-Atlanta; I haven't gone anywhere on 'business'
since probably the last Boulder meeting! Landlocked. And going to
India for the verification workshop isn't happening either.
Matt
-----Original Message-----
From: Randy Bullock via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 10, 2014 11:41 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
Hi Matt -
I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
We ran a test case masking out a precip field with a temperature field
in the same input file. Here's the relevant MODE config file entry
for our test case:
//
// Verification masking regions
//
mask = {
grid = "";
grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
};
In the "poly" part of the masking region spec, you give a string with
the data file, the name and level of the field you want to mask on,
and a threshold. Copy the syntax from the example above. You'd need
to change the field name from "TMP" to whatever your name is for the
pixel age field.
So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
Also, if you were working from the original WWMCA files instead of a
grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
Hope this helps.
Randy
On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> Queue: met_help
> Subject: MET Masking for Clouds
> Owner: Nobody
> Requestors: matthew.sittel.ctr at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
>
> We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
>
> Thanks,
> Matt
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Fri Jan 10 11:51:16 2014
One question: when I set level="Z2", does that correspond to coding,
say, kpds7? I've never really followed how that works. Here is the
GriB field I need to code in the 'poly =' line. I'm assuming I need
'L0' rather than 'Z2' but I wanted to understand if I can override
things and explicitly say what kpds5/6/7 numbers I want.
Thanks,
Matt
rec 18:20896654:date 2013123106 LASUPD kpds5=183 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
LASUPD=Time of last update from base time [Minutes]
timerange 0 P1 0 P2 0 TimeU 1 nx 1440 ny 721 GDS grid 0 num_in_ave
0 missing 0
center 57 subcenter 2 process 32 Table 131 scan: WE:NS winds(grid)
latlon: lat 90.000000 to -90.000000 by 0.250000 nxny 1038240
long 0.000000 to 359.750000 by 0.250000, (1440 x 721) scan 0
mode 136 bdsgrid 1
min/max data 0 2.41953e+07 num bits 25 BDS_Ref 0 DecScale 0
BinScale 0
-----Original Message-----
From: Randy Bullock via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 10, 2014 11:41 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
Hi Matt -
I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
We ran a test case masking out a precip field with a temperature field
in the same input file. Here's the relevant MODE config file entry
for our test case:
//
// Verification masking regions
//
mask = {
grid = "";
grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
};
In the "poly" part of the masking region spec, you give a string with
the data file, the name and level of the field you want to mask on,
and a threshold. Copy the syntax from the example above. You'd need
to change the field name from "TMP" to whatever your name is for the
pixel age field.
So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
Also, if you were working from the original WWMCA files instead of a
grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
Hope this helps.
Randy
On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> Queue: met_help
> Subject: MET Masking for Clouds
> Owner: Nobody
> Requestors: matthew.sittel.ctr at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
>
> We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
>
> Thanks,
> Matt
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: Randy Bullock
Time: Mon Jan 13 12:57:12 2014
Hi Matt -
The level info can be tricky --- we had to come up with a
scheme that works for all the data formats that MET supports.
If you're looking for information on how to specify level information
for a grib file, try looking at lines 153 to 172 in the README file
in the data/config directory in the version 4.1 source tree.
All of the possibilities are explained there.
As for how it relates to "kpds7", now it's my turn to say that
I've never really followed that stuff. If the README file
I mentioned above doesn't answer your question, let me know
and I'll do some digging.
Randy
On Fri, Jan 10, 2014 at 11:51:17AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
> One question: when I set level="Z2", does that correspond to coding,
say, kpds7? I've never really followed how that works. Here is the
GriB field I need to code in the 'poly =' line. I'm assuming I need
'L0' rather than 'Z2' but I wanted to understand if I can override
things and explicitly say what kpds5/6/7 numbers I want.
>
> Thanks,
> Matt
>
> rec 18:20896654:date 2013123106 LASUPD kpds5=183 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
> LASUPD=Time of last update from base time [Minutes]
> timerange 0 P1 0 P2 0 TimeU 1 nx 1440 ny 721 GDS grid 0
num_in_ave 0 missing 0
> center 57 subcenter 2 process 32 Table 131 scan: WE:NS winds(grid)
> latlon: lat 90.000000 to -90.000000 by 0.250000 nxny 1038240
> long 0.000000 to 359.750000 by 0.250000, (1440 x 721) scan
0 mode 136 bdsgrid 1
> min/max data 0 2.41953e+07 num bits 25 BDS_Ref 0 DecScale 0
BinScale 0
>
> -----Original Message-----
> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 10, 2014 11:41 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>
>
> Hi Matt -
>
> I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
>
> We ran a test case masking out a precip field with a temperature
field in the same input file. Here's the relevant MODE config file
entry for our test case:
>
> //
> // Verification masking regions
> //
> mask = {
> grid = "";
> grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
> poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
> poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
> };
>
> In the "poly" part of the masking region spec, you give a string
with the data file, the name and level of the field you want to mask
on, and a threshold. Copy the syntax from the example above. You'd
need to change the field name from "TMP" to whatever your name is for
the pixel age field.
>
> So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
>
> Also, if you were working from the original WWMCA files instead of a
grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
>
>
> Hope this helps.
>
> Randy
>
>
>
> On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
> >
> > Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > Queue: met_help
> > Subject: MET Masking for Clouds
> > Owner: Nobody
> > Requestors: matthew.sittel.ctr at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
> >
> >
> > We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
> >
> > Thanks,
> > Matt
> >
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Mon Jan 13 13:25:26 2014
Thanks Randy. That's good information but I'm still unsure of how it
works. Every GriB record we pull is based on kpds5/6/7 identifiers.
We don't use record number since it may change based on the
inclusion/exclusion of data fields. I'm surprised MET has that
convention since, in an automated sense, you can't be sure the record
number won't change. Kpds numbers are constant (with respect to the
table, at least) which is why we go with those. Occasionally we even
have to look at the P1 and P2 time flags for precipitation, *or*
multiple hours will get thrown into a single GriB file and we have to
specify with.
As a user, we'd like for MET to have the ability to specify kpds5,
kpds6, pkds7, P1 and P2 flags as a means to specify exactly which
record from a GriB file is desired for verification. It ends up for
us as just being an exercise as searching the GriB file for a match on
those fields and extracting the record number matching it, and then
doing a degrib of that record number. Similar I realize to "RNNN"
but, in the interest of automation, it would be very useful for us to
have. And maybe, just maybe, it already exists and we don't know
about it?
If you don't mind digging a bit... I'd like to know!
Thanks,
Matt
-----Original Message-----
From: Randy Bullock via RT [mailto:met_help at ucar.edu]
Sent: Monday, January 13, 2014 1:57 PM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
Hi Matt -
The level info can be tricky --- we had to come up with a scheme that
works for all the data formats that MET supports.
If you're looking for information on how to specify level information
for a grib file, try looking at lines 153 to 172 in the README file in
the data/config directory in the version 4.1 source tree.
All of the possibilities are explained there.
As for how it relates to "kpds7", now it's my turn to say that I've
never really followed that stuff. If the README file I mentioned
above doesn't answer your question, let me know and I'll do some
digging.
Randy
On Fri, Jan 10, 2014 at 11:51:17AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
> One question: when I set level="Z2", does that correspond to coding,
say, kpds7? I've never really followed how that works. Here is the
GriB field I need to code in the 'poly =' line. I'm assuming I need
'L0' rather than 'Z2' but I wanted to understand if I can override
things and explicitly say what kpds5/6/7 numbers I want.
>
> Thanks,
> Matt
>
> rec 18:20896654:date 2013123106 LASUPD kpds5=183 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
> LASUPD=Time of last update from base time [Minutes]
> timerange 0 P1 0 P2 0 TimeU 1 nx 1440 ny 721 GDS grid 0
num_in_ave 0 missing 0
> center 57 subcenter 2 process 32 Table 131 scan: WE:NS winds(grid)
> latlon: lat 90.000000 to -90.000000 by 0.250000 nxny 1038240
> long 0.000000 to 359.750000 by 0.250000, (1440 x 721) scan
0 mode 136 bdsgrid 1
> min/max data 0 2.41953e+07 num bits 25 BDS_Ref 0 DecScale 0
> BinScale 0
>
> -----Original Message-----
> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 10, 2014 11:41 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>
>
> Hi Matt -
>
> I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
>
> We ran a test case masking out a precip field with a temperature
field in the same input file. Here's the relevant MODE config file
entry for our test case:
>
> //
> // Verification masking regions
> //
> mask = {
> grid = "";
> grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
> poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
> poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
> };
>
> In the "poly" part of the masking region spec, you give a string
with the data file, the name and level of the field you want to mask
on, and a threshold. Copy the syntax from the example above. You'd
need to change the field name from "TMP" to whatever your name is for
the pixel age field.
>
> So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
>
> Also, if you were working from the original WWMCA files instead of a
grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
>
>
> Hope this helps.
>
> Randy
>
>
>
> On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
> >
> > Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > Queue: met_help
> > Subject: MET Masking for Clouds
> > Owner: Nobody
> > Requestors: matthew.sittel.ctr at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
> >
> >
> > We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
> >
> > Thanks,
> > Matt
> >
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: Randy Bullock
Time: Fri Jan 17 08:22:14 2014
Matt -
Sorry for the tardy response. Could you give me some idea of how
urgently
you need the information? I'm helping some people get ready for AMS,
and there
are two tutorials I'm participating in this month, so things are a
little rushed
right now. I seem to recall you saying in one of your emails that you
didn't need
this right away --- you were just asked to inquire about the issue.
If that's true, can this wait a bit?
If it is urgent, though, just say so. It's OK.
Randy
On Mon, Jan 13, 2014 at 01:25:27PM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
> Thanks Randy. That's good information but I'm still unsure of how
it works. Every GriB record we pull is based on kpds5/6/7
identifiers. We don't use record number since it may change based on
the inclusion/exclusion of data fields. I'm surprised MET has that
convention since, in an automated sense, you can't be sure the record
number won't change. Kpds numbers are constant (with respect to the
table, at least) which is why we go with those. Occasionally we even
have to look at the P1 and P2 time flags for precipitation, *or*
multiple hours will get thrown into a single GriB file and we have to
specify with.
>
> As a user, we'd like for MET to have the ability to specify kpds5,
kpds6, pkds7, P1 and P2 flags as a means to specify exactly which
record from a GriB file is desired for verification. It ends up for
us as just being an exercise as searching the GriB file for a match on
those fields and extracting the record number matching it, and then
doing a degrib of that record number. Similar I realize to "RNNN"
but, in the interest of automation, it would be very useful for us to
have. And maybe, just maybe, it already exists and we don't know
about it?
>
> If you don't mind digging a bit... I'd like to know!
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> Sent: Monday, January 13, 2014 1:57 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>
>
> Hi Matt -
>
> The level info can be tricky --- we had to come up with a scheme
that works for all the data formats that MET supports.
>
> If you're looking for information on how to specify level
information for a grib file, try looking at lines 153 to 172 in the
README file in the data/config directory in the version 4.1 source
tree.
> All of the possibilities are explained there.
>
> As for how it relates to "kpds7", now it's my turn to say that I've
never really followed that stuff. If the README file I mentioned
above doesn't answer your question, let me know and I'll do some
digging.
>
> Randy
>
>
>
> On Fri, Jan 10, 2014 at 11:51:17AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
> >
> > One question: when I set level="Z2", does that correspond to
coding, say, kpds7? I've never really followed how that works. Here
is the GriB field I need to code in the 'poly =' line. I'm assuming I
need 'L0' rather than 'Z2' but I wanted to understand if I can
override things and explicitly say what kpds5/6/7 numbers I want.
> >
> > Thanks,
> > Matt
> >
> > rec 18:20896654:date 2013123106 LASUPD kpds5=183 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
> > LASUPD=Time of last update from base time [Minutes]
> > timerange 0 P1 0 P2 0 TimeU 1 nx 1440 ny 721 GDS grid 0
num_in_ave 0 missing 0
> > center 57 subcenter 2 process 32 Table 131 scan: WE:NS
winds(grid)
> > latlon: lat 90.000000 to -90.000000 by 0.250000 nxny 1038240
> > long 0.000000 to 359.750000 by 0.250000, (1440 x 721)
scan 0 mode 136 bdsgrid 1
> > min/max data 0 2.41953e+07 num bits 25 BDS_Ref 0 DecScale 0
> > BinScale 0
> >
> > -----Original Message-----
> > From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, January 10, 2014 11:41 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
> >
> >
> > Hi Matt -
> >
> > I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
> >
> > We ran a test case masking out a precip field with a temperature
field in the same input file. Here's the relevant MODE config file
entry for our test case:
> >
> > //
> > // Verification masking regions
> > //
> > mask = {
> > grid = "";
> > grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
> > poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
> > poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
> > };
> >
> > In the "poly" part of the masking region spec, you give a string
with the data file, the name and level of the field you want to mask
on, and a threshold. Copy the syntax from the example above. You'd
need to change the field name from "TMP" to whatever your name is for
the pixel age field.
> >
> > So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
> >
> > Also, if you were working from the original WWMCA files instead of
a grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
> >
> >
> > Hope this helps.
> >
> > Randy
> >
> >
> >
> > On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR
USAF AFWA 16 WS/WXN via RT wrote:
> > >
> > > Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > Queue: met_help
> > > Subject: MET Masking for Clouds
> > > Owner: Nobody
> > > Requestors: matthew.sittel.ctr at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
> > >
> > >
> > > We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
> > >
> > > Thanks,
> > > Matt
> > >
> >
> >
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Time: Fri Jan 17 08:59:16 2014
Hi Randy,
The kpds codes would be very useful for us since that's how we degrib
everything here. Having it in a future version would be great, but I
am wondering if it's already in there and we just don't know how to
leverage the existing codes to get at it (e.g., can I 'fake it'
through linking a GriB table I design myself with the kpds fields set
as I need them, then invoke the record number based on those manually-
set table values).
It's okay if you are busy. Management is aware of my request so
they're wanting occasional updates from me as to where things stand,
so while I don't need it right away I will need to report back to them
before my next contract meeting on Feb. 20th. One of my job tasks was
to serve as SME (subject matter expert) to MET, and act as liaison to
the verification group. Since we haven't met they're anxious about
how to cover that particular task, so any development or testing on
MET excites them as usable content.
I think we have some of Jamestown's snow here now... the winds were
very impressive yesterday throughout the eastern Dakotas, and even as
far south as here. Our building registered a 50 mph gust during a
snow squall. Nothing compared to your downsloping events but
noteworthy just the same. We dropped from 40 to 15 in about 4 hours
during the late afternoon and early evening.
Thanks,
Matt
-----Original Message-----
From: Randy Bullock via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 17, 2014 9:22 AM
To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
Matt -
Sorry for the tardy response. Could you give me some idea of how
urgently you need the information? I'm helping some people get ready
for AMS, and there are two tutorials I'm participating in this month,
so things are a little rushed right now. I seem to recall you saying
in one of your emails that you didn't need this right away --- you
were just asked to inquire about the issue.
If that's true, can this wait a bit?
If it is urgent, though, just say so. It's OK.
Randy
On Mon, Jan 13, 2014 at 01:25:27PM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
> Thanks Randy. That's good information but I'm still unsure of how
it works. Every GriB record we pull is based on kpds5/6/7
identifiers. We don't use record number since it may change based on
the inclusion/exclusion of data fields. I'm surprised MET has that
convention since, in an automated sense, you can't be sure the record
number won't change. Kpds numbers are constant (with respect to the
table, at least) which is why we go with those. Occasionally we even
have to look at the P1 and P2 time flags for precipitation, *or*
multiple hours will get thrown into a single GriB file and we have to
specify with.
>
> As a user, we'd like for MET to have the ability to specify kpds5,
kpds6, pkds7, P1 and P2 flags as a means to specify exactly which
record from a GriB file is desired for verification. It ends up for
us as just being an exercise as searching the GriB file for a match on
those fields and extracting the record number matching it, and then
doing a degrib of that record number. Similar I realize to "RNNN"
but, in the interest of automation, it would be very useful for us to
have. And maybe, just maybe, it already exists and we don't know
about it?
>
> If you don't mind digging a bit... I'd like to know!
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> Sent: Monday, January 13, 2014 1:57 PM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>
>
> Hi Matt -
>
> The level info can be tricky --- we had to come up with a scheme
that works for all the data formats that MET supports.
>
> If you're looking for information on how to specify level
information for a grib file, try looking at lines 153 to 172 in the
README file in the data/config directory in the version 4.1 source
tree.
> All of the possibilities are explained there.
>
> As for how it relates to "kpds7", now it's my turn to say that I've
never really followed that stuff. If the README file I mentioned
above doesn't answer your question, let me know and I'll do some
digging.
>
> Randy
>
>
>
> On Fri, Jan 10, 2014 at 11:51:17AM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
> >
> > One question: when I set level="Z2", does that correspond to
coding, say, kpds7? I've never really followed how that works. Here
is the GriB field I need to code in the 'poly =' line. I'm assuming I
need 'L0' rather than 'Z2' but I wanted to understand if I can
override things and explicitly say what kpds5/6/7 numbers I want.
> >
> > Thanks,
> > Matt
> >
> > rec 18:20896654:date 2013123106 LASUPD kpds5=183 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
> > LASUPD=Time of last update from base time [Minutes]
> > timerange 0 P1 0 P2 0 TimeU 1 nx 1440 ny 721 GDS grid 0
num_in_ave 0 missing 0
> > center 57 subcenter 2 process 32 Table 131 scan: WE:NS
winds(grid)
> > latlon: lat 90.000000 to -90.000000 by 0.250000 nxny 1038240
> > long 0.000000 to 359.750000 by 0.250000, (1440 x 721)
scan 0 mode 136 bdsgrid 1
> > min/max data 0 2.41953e+07 num bits 25 BDS_Ref 0 DecScale 0
> > BinScale 0
> >
> > -----Original Message-----
> > From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, January 10, 2014 11:41 AM
> > To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
> >
> >
> > Hi Matt -
> >
> > I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
> >
> > We ran a test case masking out a precip field with a temperature
field in the same input file. Here's the relevant MODE config file
entry for our test case:
> >
> > //
> > // Verification masking regions
> > //
> > mask = {
> > grid = "";
> > grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
> > poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
> > poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
> > };
> >
> > In the "poly" part of the masking region spec, you give a string
with the data file, the name and level of the field you want to mask
on, and a threshold. Copy the syntax from the example above. You'd
need to change the field name from "TMP" to whatever your name is for
the pixel age field.
> >
> > So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
> >
> > Also, if you were working from the original WWMCA files instead of
a grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
> >
> >
> > Hope this helps.
> >
> > Randy
> >
> >
> >
> > On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR
USAF AFWA 16 WS/WXN via RT wrote:
> > >
> > > Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
> > > Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
> > > Queue: met_help
> > > Subject: MET Masking for Clouds
> > > Owner: Nobody
> > > Requestors: matthew.sittel.ctr at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
> > >
> > >
> > > We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
> > >
> > > Thanks,
> > > Matt
> > >
> >
> >
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
From: John Halley Gotway
Time: Fri Jan 17 12:44:37 2014
Matt and Randy,
I've been watching this ticket and wanted to jump in here.
There are some existing config file options that may help. Instead of
using "name" and "level", these options may be used to specify which
GRIB records to use:
GRIB_lvl_typ, GRIB_lvl_val1, GRIB_lvl_val2, GRIB1_ptv, GRIB2_rec
They are not explicitly named using "kpds", but they should specify
the same sorts of values. However, as I was playing around with them
last week, they weren't working as I expected. Perhaps the
task here is to get these existing options working correctly and
document them better.
Matt, if you have a specific GRIB file and want to know how to set
"name" and "level" to extract a particular record, we'd be happy to
help. Just send us the GRIB file and a description of what you
want. And you're right, MET reads the GRIB tables from a flat file
(METv4.1/data/table_files/nceptab_flat.txt) which you can update with
your local usage. Each line in that file contains:
- GRIB code (0-255)
- Parameter table version number
- GRIB code abbreviation (i.e. TMP for temperature)
- Long name for the GRIB code
- Units
We currently have entries in there for parameter table version numbers
2, 128, 129, 130, 131, 133, 140, 141. These are taken from NCEP's
website:
http://www.nco.ncep.noaa.gov/pmb/docs/on388/table2.html
If your local GRIB tables differ, you could modify or add to that
file.
Thanks,
John
On 01/17/2014 08:59 AM, SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN via
RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>
> Hi Randy,
>
> The kpds codes would be very useful for us since that's how we
degrib everything here. Having it in a future version would be great,
but I am wondering if it's already in there and we just don't know how
to leverage the existing codes to get at it (e.g., can I 'fake it'
through linking a GriB table I design myself with the kpds fields set
as I need them, then invoke the record number based on those manually-
set table values).
>
> It's okay if you are busy. Management is aware of my request so
they're wanting occasional updates from me as to where things stand,
so while I don't need it right away I will need to report back to them
before my next contract meeting on Feb. 20th. One of my job tasks was
to serve as SME (subject matter expert) to MET, and act as liaison to
the verification group. Since we haven't met they're anxious about
how to cover that particular task, so any development or testing on
MET excites them as usable content.
>
> I think we have some of Jamestown's snow here now... the winds were
very impressive yesterday throughout the eastern Dakotas, and even as
far south as here. Our building registered a 50 mph gust during a
snow squall. Nothing compared to your downsloping events but
noteworthy just the same. We dropped from 40 to 15 in about 4 hours
during the late afternoon and early evening.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 17, 2014 9:22 AM
> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>
>
> Matt -
>
> Sorry for the tardy response. Could you give me some idea of how
urgently you need the information? I'm helping some people get ready
for AMS, and there are two tutorials I'm participating in this month,
so things are a little rushed right now. I seem to recall you saying
in one of your emails that you didn't need this right away --- you
were just asked to inquire about the issue.
> If that's true, can this wait a bit?
>
> If it is urgent, though, just say so. It's OK.
>
> Randy
>
>
>
>
> On Mon, Jan 13, 2014 at 01:25:27PM -0700, SITTEL, MATTHEW C CTR USAF
AFWA 16 WS/WXN via RT wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>>
>> Thanks Randy. That's good information but I'm still unsure of how
it works. Every GriB record we pull is based on kpds5/6/7
identifiers. We don't use record number since it may change based on
the inclusion/exclusion of data fields. I'm surprised MET has that
convention since, in an automated sense, you can't be sure the record
number won't change. Kpds numbers are constant (with respect to the
table, at least) which is why we go with those. Occasionally we even
have to look at the P1 and P2 time flags for precipitation, *or*
multiple hours will get thrown into a single GriB file and we have to
specify with.
>>
>> As a user, we'd like for MET to have the ability to specify kpds5,
kpds6, pkds7, P1 and P2 flags as a means to specify exactly which
record from a GriB file is desired for verification. It ends up for
us as just being an exercise as searching the GriB file for a match on
those fields and extracting the record number matching it, and then
doing a degrib of that record number. Similar I realize to "RNNN"
but, in the interest of automation, it would be very useful for us to
have. And maybe, just maybe, it already exists and we don't know
about it?
>>
>> If you don't mind digging a bit... I'd like to know!
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
>> Sent: Monday, January 13, 2014 1:57 PM
>> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
>> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>>
>>
>> Hi Matt -
>>
>> The level info can be tricky --- we had to come up with a scheme
that works for all the data formats that MET supports.
>>
>> If you're looking for information on how to specify level
information for a grib file, try looking at lines 153 to 172 in the
README file in the data/config directory in the version 4.1 source
tree.
>> All of the possibilities are explained there.
>>
>> As for how it relates to "kpds7", now it's my turn to say that I've
never really followed that stuff. If the README file I mentioned
above doesn't answer your question, let me know and I'll do some
digging.
>>
>> Randy
>>
>>
>>
>> On Fri, Jan 10, 2014 at 11:51:17AM -0700, SITTEL, MATTHEW C CTR
USAF AFWA 16 WS/WXN via RT wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>>>
>>> One question: when I set level="Z2", does that correspond to
coding, say, kpds7? I've never really followed how that works. Here
is the GriB field I need to code in the 'poly =' line. I'm assuming I
need 'L0' rather than 'Z2' but I wanted to understand if I can
override things and explicitly say what kpds5/6/7 numbers I want.
>>>
>>> Thanks,
>>> Matt
>>>
>>> rec 18:20896654:date 2013123106 LASUPD kpds5=183 kpds6=200 kpds7=0
levels=(0,0) grid=255 atmos col anl:
>>> LASUPD=Time of last update from base time [Minutes]
>>> timerange 0 P1 0 P2 0 TimeU 1 nx 1440 ny 721 GDS grid 0
num_in_ave 0 missing 0
>>> center 57 subcenter 2 process 32 Table 131 scan: WE:NS
winds(grid)
>>> latlon: lat 90.000000 to -90.000000 by 0.250000 nxny 1038240
>>> long 0.000000 to 359.750000 by 0.250000, (1440 x 721)
scan 0 mode 136 bdsgrid 1
>>> min/max data 0 2.41953e+07 num bits 25 BDS_Ref 0 DecScale 0
>>> BinScale 0
>>>
>>> -----Original Message-----
>>> From: Randy Bullock via RT [mailto:met_help at ucar.edu]
>>> Sent: Friday, January 10, 2014 11:41 AM
>>> To: SITTEL, MATTHEW C CTR USAF AFWA 16 WS/WXN
>>> Subject: Re: [rt.rap.ucar.edu #64842] MET Masking for Clouds
>>>
>>>
>>> Hi Matt -
>>>
>>> I was all set to email you and say this couldn't be done, but John
said "Hey, let's try it" and it seemed to work. I knew it could be
done for point_stat and grid_stat, but I didn't think it would work
for MODE.
>>>
>>> We ran a test case masking out a precip field with a temperature
field in the same input file. Here's the relevant MODE config file
entry for our test case:
>>>
>>> //
>>> // Verification masking regions
>>> //
>>> mask = {
>>> grid = "";
>>> grid_flag = NONE; // Apply to NONE, FCST, OBS, or BOTH
>>> poly =
"/d1/johnhg/MET/MET_releases/METv4.1/data/sample_fcst/2005080700/wrfprs_ruc13_24.tm00_G212
{name=\"TMP\"; level=\"Z2\";} gt310";
>>> poly_flag = BOTH; // Apply to NONE, FCST, OBS, or BOTH
>>> };
>>>
>>> In the "poly" part of the masking region spec, you give a string
with the data file, the name and level of the field you want to mask
on, and a threshold. Copy the syntax from the example above. You'd
need to change the field name from "TMP" to whatever your name is for
the pixel age field.
>>>
>>> So it seems that it can be done. Go figure. Having said that,
though, I feel compelled to point out that data field masking for MODE
might not be a good idea unless the masking results in large connected
blocks of the grid being turned off (as may well be the case with
pixel age ... you'd know better than I would).
>>>
>>> Also, if you were working from the original WWMCA files instead of
a grib file, the wwmca_regrid tool can threshold by pixel age. FYI.
>>>
>>>
>>> Hope this helps.
>>>
>>> Randy
>>>
>>>
>>>
>>> On Thu, Jan 09, 2014 at 07:44:16AM -0700, SITTEL, MATTHEW C CTR
USAF AFWA 16 WS/WXN via RT wrote:
>>>>
>>>> Thu Jan 09 07:44:16 2014: Request 64842 was acted upon.
>>>> Transaction: Ticket created by matthew.sittel.ctr at us.af.mil
>>>> Queue: met_help
>>>> Subject: MET Masking for Clouds
>>>> Owner: Nobody
>>>> Requestors: matthew.sittel.ctr at us.af.mil
>>>> Status: new
>>>> Ticket <URL:
>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=64842 >
>>>>
>>>>
>>>> We are using MODE in an attempt to define cloud objects. It's a
work in progress, and definitely needs some tweaks to get it working
better. One thing we need is the ability to mask data based on WWMCA
pixel age. It's in the same GriB file as the actual cloud data. We
need to set a threshold in which we keep all points with an age less
than 120 minutes. Can this be done in MET 4.1? If so, how would you
go about doing it?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>
>>>
>>
>>
>
>
------------------------------------------------
More information about the Met_help
mailing list