[Met_help] [rt.rap.ucar.edu #76773] History for MET lookin Wildcards
John Halley Gotway via RT
met_help at ucar.edu
Thu Jun 16 14:43:30 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, I asked this question before but never really got an answer. I am using a wildcard in the lookin string but it never works so I end up having to read every file in directory. Here is my command line:
/h/WXQC/met-5.1/bin/stat_analysis -lookin /h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat -out /h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z -job aggregate_stat -line_type MPR -out_line_type PSTD -fcst_lead 480000 -out_fcst_thresh ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0 -out_obs_thresh ge1 -by FCST_VAR -by FCST_THRESH -v 6
I get the following error when I run:
['/h/WXQC/met-5.1/bin/stat_analysis', '-lookin', '/h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat', '-out', '/h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z', '-job aggregate_stat', '-line_type MPR', '-out_line_type PSTD', '-fcst_lead 480000', '-out_fcst_thresh ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0', '-out_obs_thresh ge1', '-by FCST_VAR', '-by FCST_THRESH', '-v', '6']
DEBUG 1: Creating STAT-Analysis output file "/h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z"
DEBUG 4: Amending default job with command line options: "-job aggregate_stat -line_type MPR -out_line_type PSTD -fcst_lead 480000 -out_fcst_thresh ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0 -out_obs_thresh ge1 -by FCST_VAR -by FCST_THRESH"
WARNING:
WARNING: get_filenames() -> can't stat "/h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat"
WARNING:
ERROR :
ERROR : process_search_dirs() -> no STAT files found in the directories specified!
ERROR :
ERROR :
ERROR : main() -> encountered an error value of 1. Calling clean_up() and usage() before exiting.
ERROR :
So what am I doing wrong with the wildcard?
Thanks
Bob
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET lookin Wildcards
From: John Halley Gotway
Time: Thu Jun 16 09:53:30 2016
Bob,
If you were to run that command directly in a shell, it would work.
The
shell would expand those *'s and construct a list of filenames and
pass
them the STAT-Analysis. However, you're using a python wrapper
script, I
believe. That string is being passed directly to STAT-Analysis and
not
allowing the shell to perform wildcard expansion. It is the shell
that is
doing the wildcard expansion, not MET.
You'll need to do the wildcard expansion yourself in one extra step.
I'm
sure there's a way in Python to execute the equivalent of an "ls".
Prior
to calling STAT-Analysis, add a line to execute that "ls" and store
the
resulting list of filenames in a variable. And then pass that
variable
containing the list of filenames to STAT-Analysis.
That should do the trick.
John
On Thu, Jun 16, 2016 at 8:10 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Jun 16 08:10:50 2016: Request 76773 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: MET lookin Wildcards
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=76773 >
>
>
> John, I asked this question before but never really got an answer.
I am
> using a wildcard in the lookin string but it never works so I end up
having
> to read every file in directory. Here is my command line:
>
>
>
>
>
>
>
> /h/WXQC/met-5.1/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat -out
> /h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z
-job
> aggregate_stat -line_type MPR -out_line_type PSTD -fcst_lead
480000
> -out_fcst_thresh
> ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0
> -out_obs_thresh ge1 -by FCST_VAR -by FCST_THRESH -v 6
>
>
>
> I get the following error when I run:
>
>
>
> ['/h/WXQC/met-5.1/bin/stat_analysis', '-lookin',
> '/h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat', '-out',
> '/h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z',
'-job
> aggregate_stat', '-line_type MPR', '-out_line_type PSTD', '-
fcst_lead
> 480000', '-out_fcst_thresh
> ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0',
> '-out_obs_thresh ge1', '-by FCST_VAR', '-by FCST_THRESH', '-v', '6']
>
> DEBUG 1: Creating STAT-Analysis output file
> "/h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z"
>
> DEBUG 4: Amending default job with command line options: "-job
> aggregate_stat -line_type MPR -out_line_type PSTD -fcst_lead 480000
> -out_fcst_thresh
> ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0
> -out_obs_thresh ge1 -by FCST_VAR -by FCST_THRESH"
>
> WARNING:
>
> WARNING: get_filenames() -> can't stat
> "/h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat"
>
> WARNING:
>
> ERROR :
>
> ERROR : process_search_dirs() -> no STAT files found in the
directories
> specified!
>
> ERROR :
>
> ERROR :
>
> ERROR : main() -> encountered an error value of 1. Calling
clean_up()
> and usage() before exiting.
>
> ERROR :
>
>
>
> So what am I doing wrong with the wildcard?
>
>
>
> Thanks
>
> Bob
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #76773] MET lookin Wildcards
From: robert.craig.2 at us.af.mil
Time: Thu Jun 16 10:01:39 2016
Okay, that makes sense. I will try your idea. By the way, I am
getting Brier Skill Scores if I run directly against the MPR files to
PSTD, not doing the PCT step. Run times don't seem that bad and if I
can get the wildcard working better, that should help as well.
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, June 16, 2016 10:54 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [rt.rap.ucar.edu #76773] MET lookin Wildcards
Bob,
If you were to run that command directly in a shell, it would work.
The shell would expand those *'s and construct a list of filenames and
pass them the STAT-Analysis. However, you're using a python wrapper
script, I believe. That string is being passed directly to STAT-
Analysis and not allowing the shell to perform wildcard expansion. It
is the shell that is doing the wildcard expansion, not MET.
You'll need to do the wildcard expansion yourself in one extra step.
I'm sure there's a way in Python to execute the equivalent of an "ls".
Prior to calling STAT-Analysis, add a line to execute that "ls" and
store the resulting list of filenames in a variable. And then pass
that variable containing the list of filenames to STAT-Analysis.
That should do the trick.
John
On Thu, Jun 16, 2016 at 8:10 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Jun 16 08:10:50 2016: Request 76773 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: MET lookin Wildcards
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=76773
> >
>
>
> John, I asked this question before but never really got an answer.
I
> am using a wildcard in the lookin string but it never works so I end
up having
> to read every file in directory. Here is my command line:
>
>
>
>
>
>
>
> /h/WXQC/met-5.1/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat -out
> /h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z
-job
> aggregate_stat -line_type MPR -out_line_type PSTD -fcst_lead
480000
> -out_fcst_thresh
> ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0
> -out_obs_thresh ge1 -by FCST_VAR -by FCST_THRESH -v 6
>
>
>
> I get the following error when I run:
>
>
>
> ['/h/WXQC/met-5.1/bin/stat_analysis', '-lookin',
> '/h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat', '-out',
> '/h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z',
> '-job aggregate_stat', '-line_type MPR', '-out_line_type PSTD',
> '-fcst_lead 480000', '-out_fcst_thresh
> ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0',
> '-out_obs_thresh ge1', '-by FCST_VAR', '-by FCST_THRESH', '-v', '6']
>
> DEBUG 1: Creating STAT-Analysis output file
> "/h/data/global/WXQC/data/met/summary/GALWEM_APCP_48hr_9_PSTD_0Z"
>
> DEBUG 4: Amending default job with command line options: "-job
> aggregate_stat -line_type MPR -out_line_type PSTD -fcst_lead 480000
> -out_fcst_thresh
> ge0,ge0.1,ge0.2,ge0.3,ge0.4,ge0.5,ge0.6,ge0.7,ge0.8,ge0.9,ge1.0
> -out_obs_thresh ge1 -by FCST_VAR -by FCST_THRESH"
>
> WARNING:
>
> WARNING: get_filenames() -> can't stat
> "/h/data/global/WXQC/data/met/mdlob_pairs/*480000L*.stat"
>
> WARNING:
>
> ERROR :
>
> ERROR : process_search_dirs() -> no STAT files found in the
> directories specified!
>
> ERROR :
>
> ERROR :
>
> ERROR : main() -> encountered an error value of 1. Calling
> clean_up() and usage() before exiting.
>
> ERROR :
>
>
>
> So what am I doing wrong with the wildcard?
>
>
>
> Thanks
>
> Bob
>
>
------------------------------------------------
More information about the Met_help
mailing list