[Met_help] [rt.rap.ucar.edu #99168] History for Filtering in Stat Anal
John Halley Gotway via RT
met_help at ucar.edu
Fri Mar 26 16:38:21 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, we have a case where a bad RAOB got into our mdlob_pairs and partial sums. In an attempt to filter the bad data using the MAE field int he SL1L2 files, I am using the following stat anal job:
/h/MET/bin/stat_analysis -lookin /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat /h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galwem_HT_UPR_20201201_20210228.txt -tmp_dir /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000 -fcst_init_end 20210228_180000 -by fcst_lead -fcst_lev P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by FCST_LEV -by model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by fcst_init_hour -column_str OBS_LEV P400, -column_max MAE 17 -column_str OBS_LEV P300, -column_max MAE 12 -column_str OBS_LEV P150, -column_max MAE 14 -config /h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_aggstat -v 4.0
The desire is to filter out MAEs for the 400mb level >17, 300mb level > 12, and 150mb level > 14. Unfortunately, the resulting file has only P400, P300, and P150 in it. Am I interpeting the logic here correctly?
Thanks
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Filtering in Stat Anal
From: John Halley Gotway
Time: Mon Mar 22 20:28:01 2021
Bob,
All of the options specified for a single job are grouped together
into one
filter. Since you requested OBS_LEVEL = P150, P300, P400 in the job,
you
get the union of those in the output.
Unfortunately you’d need to do this with 3 different jobs, one for
each
level/threshold combo.
If it were me, instead of sticking together stat-analysis jobs, I’d
probably write a shell or Python script to do this filtering.
Sorry I don’t have an easier answer for you.
John
On Mon, Mar 22, 2021 at 10:33 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 22 10:33:09 2021: Request 99168 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Filtering in Stat Anal
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168 >
>
>
> John, we have a case where a bad RAOB got into our mdlob_pairs and
partial
> sums. In an attempt to filter the bad data using the MAE field int
he
> SL1L2 files, I am using the following stat anal job:
>
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat
>
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galwem_HT_UPR_20201201_20210228.txt
> -tmp_dir /h/data/global/WXQC/data/met/tmp -job aggregate_stat
-line_type
> SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000
-fcst_init_end
> 20210228_180000 -by fcst_lead -fcst_lev
> P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by FCST_LEV
-by
> model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by
fcst_init_hour
> -column_str OBS_LEV P400, -column_max MAE 17 -column_str OBS_LEV
P300,
> -column_max MAE 12 -column_str OBS_LEV P150, -column_max MAE 14
-config
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_aggstat
> -v 4.0
>
> The desire is to filter out MAEs for the 400mb level >17, 300mb
level >
> 12, and 150mb level > 14. Unfortunately, the resulting file has
only P400,
> P300, and P150 in it. Am I interpeting the logic here correctly?
>
> Thanks
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering in Stat Anal
From: robert.craig.2 at us.af.mil
Time: Tue Mar 23 07:18:07 2021
Thanks for that. One reason this can come up is that there is not a
ob exclude option in the stat_analysis job. You can specify which obs
to use but you can't exclude an ob (at least that is my understanding
for met 8.1 - maybe remedied in later versions). You can exclude in
the point stat config file but sometimes a bad level in a ob can occur
and you don't discover this until you are plotting the data.
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, March 22, 2021 9:28 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering in
Stat Anal
Bob,
All of the options specified for a single job are grouped together
into one filter. Since you requested OBS_LEVEL = P150, P300, P400 in
the job, you get the union of those in the output.
Unfortunately you’d need to do this with 3 different jobs, one for
each level/threshold combo.
If it were me, instead of sticking together stat-analysis jobs, I’d
probably write a shell or Python script to do this filtering.
Sorry I don’t have an easier answer for you.
John
On Mon, Mar 22, 2021 at 10:33 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 22 10:33:09 2021: Request 99168 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Filtering in Stat Anal
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168
> >
>
>
> John, we have a case where a bad RAOB got into our mdlob_pairs and
> partial sums. In an attempt to filter the bad data using the MAE
> field int he
> SL1L2 files, I am using the following stat anal job:
>
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat
>
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galw
> em_HT_UPR_20201201_20210228.txt -tmp_dir
> /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type
> SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000
-fcst_init_end
> 20210228_180000 -by fcst_lead -fcst_lev
> P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by FCST_LEV
> -by model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by
> fcst_init_hour -column_str OBS_LEV P400, -column_max MAE 17
> -column_str OBS_LEV P300, -column_max MAE 12 -column_str OBS_LEV
P150,
> -column_max MAE 14 -config
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_aggstat
> -v 4.0
>
> The desire is to filter out MAEs for the 400mb level >17, 300mb
level
> > 12, and 150mb level > 14. Unfortunately, the resulting file has
> only P400, P300, and P150 in it. Am I interpeting the logic here
correctly?
>
> Thanks
>
>
>
------------------------------------------------
Subject: FW: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering in Stat Anal
From: robert.craig.2 at us.af.mil
Time: Tue Mar 23 07:59:35 2021
I will go the python script route on this.
Thanks
-----Original Message-----
From: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
Sent: Tuesday, March 23, 2021 8:17 AM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering
in Stat Anal
Thanks for that. One reason this can come up is that there is not a
ob exclude option in the stat_analysis job. You can specify which obs
to use but you can't exclude an ob (at least that is my understanding
for met 8.1 - maybe remedied in later versions). You can exclude in
the point stat config file but sometimes a bad level in a ob can occur
and you don't discover this until you are plotting the data.
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, March 22, 2021 9:28 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering in
Stat Anal
Bob,
All of the options specified for a single job are grouped together
into one filter. Since you requested OBS_LEVEL = P150, P300, P400 in
the job, you get the union of those in the output.
Unfortunately you’d need to do this with 3 different jobs, one for
each level/threshold combo.
If it were me, instead of sticking together stat-analysis jobs, I’d
probably write a shell or Python script to do this filtering.
Sorry I don’t have an easier answer for you.
John
On Mon, Mar 22, 2021 at 10:33 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Mar 22 10:33:09 2021: Request 99168 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Filtering in Stat Anal
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168
> >
>
>
> John, we have a case where a bad RAOB got into our mdlob_pairs and
> partial sums. In an attempt to filter the bad data using the MAE
> field int he
> SL1L2 files, I am using the following stat anal job:
>
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat
>
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galw
> em_HT_UPR_20201201_20210228.txt -tmp_dir
> /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type
> SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000
-fcst_init_end
> 20210228_180000 -by fcst_lead -fcst_lev
> P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by FCST_LEV
> -by model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by
> fcst_init_hour -column_str OBS_LEV P400, -column_max MAE 17
> -column_str OBS_LEV P300, -column_max MAE 12 -column_str OBS_LEV
P150,
> -column_max MAE 14 -config
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_aggstat
> -v 4.0
>
> The desire is to filter out MAEs for the 400mb level >17, 300mb
level
> > 12, and 150mb level > 14. Unfortunately, the resulting file has
> only P400, P300, and P150 in it. Am I interpeting the logic here
correctly?
>
> Thanks
>
>
>
------------------------------------------------
Subject: Filtering in Stat Anal
From: John Halley Gotway
Time: Tue Mar 23 15:56:11 2021
Bob,
Aha, well that certainly would be a very easy addition to the
stat_analysis
and tc_stat tools! There currently is a filtering option named
“column_str”. Setting “-line_type MPR -column_str OBS_SID KDEN,KTCL”
would
only use MPR lines for those 2 stations.
In addition, point_stat and ensemble_stat have a “sid_exc” option to
specify which stations to skip.
We could add a stat_analysis and tc_stat option named “-column_exc” to
define which strings to EXCLUDE instead of which to include.
Do you think you’d find that enhancement to be useful? That’s such an
easy
one, I might be able to squeak it in for the upcoming 10.0.0 release.
Thanks,
John
On Tue, Mar 23, 2021 at 8:00 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168 >
>
> I will go the python script route on this.
>
> Thanks
>
> -----Original Message-----
> From: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
> Sent: Tuesday, March 23, 2021 8:17 AM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering
in
> Stat Anal
>
>
> Thanks for that. One reason this can come up is that there is not a
ob
> exclude option in the stat_analysis job. You can specify which obs
to use
> but you can't exclude an ob (at least that is my understanding for
met 8.1
> - maybe remedied in later versions). You can exclude in the point
stat
> config file but sometimes a bad level in a ob can occur and you
don't
> discover this until you are plotting the data.
>
> Bob
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, March 22, 2021 9:28 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering in
Stat
> Anal
>
> Bob,
>
> All of the options specified for a single job are grouped together
into
> one filter. Since you requested OBS_LEVEL = P150, P300, P400 in the
job,
> you get the union of those in the output.
>
> Unfortunately you’d need to do this with 3 different jobs, one for
each
> level/threshold combo.
>
> If it were me, instead of sticking together stat-analysis jobs, I’d
> probably write a shell or Python script to do this filtering.
>
> Sorry I don’t have an easier answer for you.
>
> John
>
> On Mon, Mar 22, 2021 at 10:33 AM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Mar 22 10:33:09 2021: Request 99168 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Filtering in Stat Anal
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168
> > >
> >
> >
> > John, we have a case where a bad RAOB got into our mdlob_pairs and
> > partial sums. In an attempt to filter the bad data using the MAE
> > field int he
> > SL1L2 files, I am using the following stat anal job:
> >
> > /h/MET/bin/stat_analysis -lookin
> > /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat
> >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galw
> > em_HT_UPR_20201201_20210228.txt -tmp_dir
> > /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type
> > SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000
-fcst_init_end
> > 20210228_180000 -by fcst_lead -fcst_lev
> > P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by
FCST_LEV
> > -by model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by
> > fcst_init_hour -column_str OBS_LEV P400, -column_max MAE 17
> > -column_str OBS_LEV P300, -column_max MAE 12 -column_str OBS_LEV
P150,
> > -column_max MAE 14 -config
> >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > strap_aggstat
> > -v 4.0
> >
> > The desire is to filter out MAEs for the 400mb level >17, 300mb
level
> > > 12, and 150mb level > 14. Unfortunately, the resulting file has
> > only P400, P300, and P150 in it. Am I interpeting the logic here
> correctly?
> >
> > Thanks
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: Filtering in Stat Anal
From: robert.craig.2 at us.af.mil
Time: Wed Mar 24 07:22:39 2021
Yes, There are times when a bad ob gets into the model/ob pairs data
and there needs to be a way of not using it in stat analysis. So a
-column exclude would be very useful.
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, March 23, 2021 4:56 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
Subject: Re: FW: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168]
Filtering in Stat Anal
Bob,
Aha, well that certainly would be a very easy addition to the
stat_analysis
and tc_stat tools! There currently is a filtering option named
“column_str”. Setting “-line_type MPR -column_str OBS_SID KDEN,KTCL”
would
only use MPR lines for those 2 stations.
In addition, point_stat and ensemble_stat have a “sid_exc” option to
specify which stations to skip.
We could add a stat_analysis and tc_stat option named “-column_exc” to
define which strings to EXCLUDE instead of which to include.
Do you think you’d find that enhancement to be useful? That’s such an
easy
one, I might be able to squeak it in for the upcoming 10.0.0 release.
Thanks,
John
On Tue, Mar 23, 2021 at 8:00 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168 >
>
> I will go the python script route on this.
>
> Thanks
>
> -----Original Message-----
> From: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
> Sent: Tuesday, March 23, 2021 8:17 AM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering
in
> Stat Anal
>
>
> Thanks for that. One reason this can come up is that there is not a
ob
> exclude option in the stat_analysis job. You can specify which obs
to use
> but you can't exclude an ob (at least that is my understanding for
met 8.1
> - maybe remedied in later versions). You can exclude in the point
stat
> config file but sometimes a bad level in a ob can occur and you
don't
> discover this until you are plotting the data.
>
> Bob
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, March 22, 2021 9:28 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering in
Stat
> Anal
>
> Bob,
>
> All of the options specified for a single job are grouped together
into
> one filter. Since you requested OBS_LEVEL = P150, P300, P400 in the
job,
> you get the union of those in the output.
>
> Unfortunately you’d need to do this with 3 different jobs, one for
each
> level/threshold combo.
>
> If it were me, instead of sticking together stat-analysis jobs, I’d
> probably write a shell or Python script to do this filtering.
>
> Sorry I don’t have an easier answer for you.
>
> John
>
> On Mon, Mar 22, 2021 at 10:33 AM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Mar 22 10:33:09 2021: Request 99168 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Filtering in Stat Anal
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168
> > >
> >
> >
> > John, we have a case where a bad RAOB got into our mdlob_pairs and
> > partial sums. In an attempt to filter the bad data using the MAE
> > field int he
> > SL1L2 files, I am using the following stat anal job:
> >
> > /h/MET/bin/stat_analysis -lookin
> > /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat
> >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galw
> > em_HT_UPR_20201201_20210228.txt -tmp_dir
> > /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type
> > SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000
-fcst_init_end
> > 20210228_180000 -by fcst_lead -fcst_lev
> > P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by
FCST_LEV
> > -by model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by
> > fcst_init_hour -column_str OBS_LEV P400, -column_max MAE 17
> > -column_str OBS_LEV P300, -column_max MAE 12 -column_str OBS_LEV
P150,
> > -column_max MAE 14 -config
> >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > strap_aggstat
> > -v 4.0
> >
> > The desire is to filter out MAEs for the 400mb level >17, 300mb
level
> > > 12, and 150mb level > 14. Unfortunately, the resulting file has
> > only P400, P300, and P150 in it. Am I interpeting the logic here
> correctly?
> >
> > Thanks
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: Filtering in Stat Anal
From: John Halley Gotway
Time: Fri Mar 26 16:37:32 2021
Bob,
FYI, I wrote up this GitHub issue for this feature request:
https://github.com/dtcenter/MET/issues/1733
I'll go ahead and resolve this issue.
Thanks,
John
On Wed, Mar 24, 2021 at 7:22 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168 >
>
> Yes, There are times when a bad ob gets into the model/ob pairs data
and
> there needs to be a way of not using it in stat analysis. So a
-column
> exclude would be very useful.
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, March 23, 2021 4:56 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
> Subject: Re: FW: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168]
Filtering
> in Stat Anal
>
> Bob,
>
> Aha, well that certainly would be a very easy addition to the
stat_analysis
> and tc_stat tools! There currently is a filtering option named
> “column_str”. Setting “-line_type MPR -column_str OBS_SID KDEN,KTCL”
would
> only use MPR lines for those 2 stations.
>
> In addition, point_stat and ensemble_stat have a “sid_exc” option to
> specify which stations to skip.
>
> We could add a stat_analysis and tc_stat option named “-column_exc”
to
> define which strings to EXCLUDE instead of which to include.
>
> Do you think you’d find that enhancement to be useful? That’s such
an easy
> one, I might be able to squeak it in for the upcoming 10.0.0
release.
>
> Thanks,
> John
>
> On Tue, Mar 23, 2021 at 8:00 AM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168 >
> >
> > I will go the python script route on this.
> >
> > Thanks
> >
> > -----Original Message-----
> > From: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
> > Sent: Tuesday, March 23, 2021 8:17 AM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168]
Filtering in
> > Stat Anal
> >
> >
> > Thanks for that. One reason this can come up is that there is not
a ob
> > exclude option in the stat_analysis job. You can specify which
obs to
> use
> > but you can't exclude an ob (at least that is my understanding for
met
> 8.1
> > - maybe remedied in later versions). You can exclude in the
point stat
> > config file but sometimes a bad level in a ob can occur and you
don't
> > discover this until you are plotting the data.
> >
> > Bob
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, March 22, 2021 9:28 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #99168] Filtering
in Stat
> > Anal
> >
> > Bob,
> >
> > All of the options specified for a single job are grouped together
into
> > one filter. Since you requested OBS_LEVEL = P150, P300, P400 in
the job,
> > you get the union of those in the output.
> >
> > Unfortunately you’d need to do this with 3 different jobs, one for
each
> > level/threshold combo.
> >
> > If it were me, instead of sticking together stat-analysis jobs,
I’d
> > probably write a shell or Python script to do this filtering.
> >
> > Sorry I don’t have an easier answer for you.
> >
> > John
> >
> > On Mon, Mar 22, 2021 at 10:33 AM robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Mar 22 10:33:09 2021: Request 99168 was acted upon.
> > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > Queue: met_help
> > > Subject: Filtering in Stat Anal
> > > Owner: Nobody
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99168
> > > >
> > >
> > >
> > > John, we have a case where a bad RAOB got into our mdlob_pairs
and
> > > partial sums. In an attempt to filter the bad data using the
MAE
> > > field int he
> > > SL1L2 files, I am using the following stat anal job:
> > >
> > > /h/MET/bin/stat_analysis -lookin
> > > /h/data/global/WXQC/data/met/ptstat/stats/galwem/HT -out_stat
> > >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_sl1l2_galw
> > > em_HT_UPR_20201201_20210228.txt -tmp_dir
> > > /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type
> > > SL1L2 -out_line_type CNT -fcst_init_beg 20201201_000000
-fcst_init_end
> > > 20210228_180000 -by fcst_lead -fcst_lev
> > > P1000,P925,P850,P700,P500,P400,P300,P250,P200,P150,P100 -by
FCST_LEV
> > > -by model -vx_mask FULL,uss,usn,usa,usp,usc,use -by vx_mask -by
> > > fcst_init_hour -column_str OBS_LEV P400, -column_max MAE 17
> > > -column_str OBS_LEV P300, -column_max MAE 12 -column_str OBS_LEV
P150,
> > > -column_max MAE 14 -config
> > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > > strap_aggstat
> > > -v 4.0
> > >
> > > The desire is to filter out MAEs for the 400mb level >17, 300mb
level
> > > > 12, and 150mb level > 14. Unfortunately, the resulting file
has
> > > only P400, P300, and P150 in it. Am I interpeting the logic
here
> > correctly?
> > >
> > > Thanks
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list