[Met_help] met and conditional verification across variables
John Halley Gotway
johnhg at rap.ucar.edu
Tue Apr 13 08:28:13 MDT 2010
Mike,
That makes sense to me... to modify the code to read the GRIB files as-is.
If you come up with some logic that would be generally applicable to be
included in MET, please let me know. Perhaps a "-force" command line
argument could enable the user to bypass extensive checking. But we'd
need to work out the details.
Thanks,
John
> Works with -add, but still not with -sum. The problem is with similar
> idiosyncrasies with the files as were found with some TR values of 0 I'd
> dealt with before, even though each file is a 3 hour precip accumulation.
> It'd be nice if I could circumvent some of the checks for these and just
> sum the APCP (code 61) values for all the files in the directory given, or
> using the date in the files. I'm just going in and making some
> modifications to the code to skip the checks and process that way and
> doing what I can to check that the sums make sense. Easier than modifying
> the grib files (I think). If you have another idea beyond going in and
> modifying all the grib files to work with MET, feel free to let me know!
>
> Thanks,
> Mike
>
> -----Original Message-----
> From: John Halley Gotway [mailto:johnhg at ucar.edu]
> Sent: Monday, April 12, 2010 2:59 PM
> To: Shaw, Michael J CTR USAF AFWA 16 WS/WXE
> Cc: 'met_help'
> Subject: Re: [Met_help] met and conditional verification across variables
>
> Mike,
>
> I'd suggest trying to run a different pcp_combine command to narrow in on
> where the problem actually lies. For example, try using the "-add"
> command:
> pcp_combine -add file1 3 file2 3 test_add.nc
>
> Where file1 and file2 are the paths to two files containing 3-hourly
> accumulations. If you get the same error when you run this, there's
> likely a problem in the GRIB file formatting. And if that's
> the case, feel free to post a couple of sample files to our anonymous ftp
> site. I'd be happy to run pcp-combine through the debugger and try to
> figure out where the problem is. You can post to our
> ftp site as follows:
>
> ftp ftp.rap.ucar.edu
> username=anonymous
> password="your email address"
> cd incoming/irap/met_help/shaw_data
> put "your files"
> bye
>
> Thanks,
> John
>
>
> Shaw, Michael J CTR USAF AFWA 16 WS/WXE wrote:
>> Thanks, John.
>>
>> Got the exact same error with the line you suggested below. Thanks for
>> the notice of the missing output file in what I'd put in that email.
>> I'm going to try without the pcprx part and see how that goes. The 0p25
>> was the extension of the files in the pcpdir, of course....
>>
>> Any other thoughts welcome in the mean time.
>>
>> -Mike
>>
>> -----Original Message-----
>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>> Sent: Monday, April 12, 2010 2:06 PM
>> To: Shaw, Michael J CTR USAF AFWA 16 WS/WXE
>> Cc: met_help
>> Subject: Re: [Met_help] met and conditional verification across
>> variables
>>
>> Mike,
>>
>> I think it may be a problem with how you're calling PCP-Combine. Listed
>> below is the usage statement for PCP-Combine, when summing up files:
>>
>> SUM_ARGS:
>> init_time
>> in_accum
>> valid_time
>> out_accum
>> out_file
>> [-pcpdir path]
>> [-pcprx reg_exp]
>>
>> where "init_time" indicates the initialization time of
>> the input Grib files in YYYYMMDD_HHMMSS format
>> (required).
>> "in_accum" indicates the accumulation interval
>> of the input Grib files in HH format (required).
>> "valid_time" indicates the desired valid time in
>> YYYYMMDD_HHMMSS format (required).
>> "out_accum" indicates the desired accumulation
>> interval for the output NetCDF file in HH format
>> (required).
>> "out_file" indicates the name of the output
>> NetCDF file to be written consisting of the sum
>> of the accumulation intervals (required).
>> "-pcpdir path" overrides the default
>> precipitation directory
>> (/d1/johnhg/MET/MET_releases/METv2.0_patch/data/sample_obs/ST2ml)
>> (optional).
>> "-pcprx reg_exp" overrides the default regular
>> expression for precipitation file naming
>> convention (.*) (optional).
>>
>> And here's the command line you're running:
>> pcp_combine -sum 00000000_000000 3 20100318_000000 168 -pcpdir <the
>> dir> -pcprx *0p25
>>
>> It looks like you've failed to list an output file name to be used in
>> the correct place in the command line. Try changing it something like:
>> pcp_combine -sum 00000000_000000 3 20100318_000000 168 test_output.nc
>> -pcpdir <the dir> -pcprx 0p25
>>
>> Also notice that I've removed the '*' from the -pcprx argument. Just
>> using '0p25' should do what you want. Otherwise, the shell will expand
>> that '*' on the command line and you'll get unexpected
>> behavior.
>>
>> Let me know if that helps.
>>
>> Thanks,
>> John
>>
>> Shaw, Michael J CTR USAF AFWA 16 WS/WXE wrote:
>>> Hi John.
>>>
>>> Trying to accumulate precip of a bunch of agrmet grib files. Here's
>>> what I get:
>>>
>>> pcp_combine -sum 00000000_000000 3 20100318_000000 168 -pcpdir <the
>>> dir> -pcprx *0p25
>>>
>>> [1]Reading input file:
>>>
>>> GribFile::open(const char *) -> unable to open grib file
>>>
>>> While I look around for the culprit and fix, maybe you can help me
>>> before I get there. Of course, these are the airforce grib files...
>>>
>>> Thanks,
>>> Mike
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>>> Sent: Tuesday, March 23, 2010 1:14 PM
>>> To: Shaw, Michael J CTR USAF AFWA 16 WS/WXE; met_help
>>> Subject: Re: [Met_help] met and conditional verification across
>>> variables
>>>
>>> Mike,
>>>
>>> Great, thanks. Please send me the changes when you get a chance, and
>>> I'll look into getting them incorporated into the next version of MET.
>>>
>>> John
>>>
>>> Shaw, Michael J CTR USAF AFWA 16 WS/WXE wrote:
>>>> Hi John.
>>>>
>>>> I think I was able to make the mods I needed and rebuild MET. I'm out
>>>> of town for much of the week, but I can pass those on to you early
>>>> next
>>>> week if you want.
>>>>
>>>> Thanks,
>>>> Mike
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>>> Sent: Friday, March 19, 2010 10:23 PM
>>>> To: Shaw, Michael J CTR USAF AFWA 16 WS/WXE
>>>> Cc: met_help at ucar.edu
>>>> Subject: Re: [Met_help] met and conditional verification across
>>>> variables
>>>>
>>>> Mike,
>>>>
>>>> You may be able to do what you'd like to do using what we've called
>>>> "data
>>>> masking". Listed below is an excerpt from the Point-Stat
>>>> configuration
>>>> file describing the "mask_poly" option:
>>>>
>>>> //
>>>> // Specify a comma-separated list of masking regions to be applied.
>>>> // An empty list indicates that no additional masks should be used.
>>>> // The masking regions may be defined in one of 4 ways:
>>>> //
>>>> // (1) An ASCII file containing a lat/lon polygon.
>>>> // Latitude in degrees north and longitude in degrees east.
>>>> // By default, the first and last polygon points are connected.
>>>> // e.g. "MET_BASE/data/poly/EAST.poly" which consists of n points:
>>>> // "poly_name lat1 lon1 lat2 lon2... latn lonn"
>>>> //
>>>> // (2) The NetCDF output of the gen_poly_mask tool.
>>>> //
>>>> // (3) A NetCDF data file, followed by the name of the NetCDF variable
>>>> // to be used, and optionally, a threshold to be applied to the
>>>> field.
>>>> // e.g. "sample.nc var_name gt0.00"
>>>> //
>>>> // (4) A GRIB data file, followed by a description of the field
>>>> // to be used, and optionally, a threshold to be applied to the
>>>> field.
>>>> // e.g. "sample.grb APCP/A3 gt0.00"
>>>> //
>>>> // Any NetCDF or GRIB file used must have the same grid dimensions as
>>>> the
>>>> // data being verified.
>>>> //
>>>> // MET_BASE may be used in the path for the files above.
>>>> //
>>>> // e.g. mask_poly[] = [ "MET_BASE/data/poly/EAST.poly",
>>>> // "poly_mask.ncf",
>>>> // "sample.nc APCP",
>>>> // "sample.grb HGT/Z0 gt100.0" ];
>>>> //
>>>> mask_poly[] = [ "MET_BASE/data/poly/EAST.poly" ];
>>>>
>>>> Option (4) listed above may do what you'd like. You'd need the land
>>>> use
>>>> field on the same grid as the data you're verifying. Basically, you'd
>>>> specify the GRIB file you'd like to use, the field you want (land use
>>>> in
>>>> your case), and then the threshold you'd like applied to that field -
>>>> and
>>>> you could specify that threshold as "eq0" or "eq1" - or whatever the
>>>> land
>>>> use values are. That threshold defines the data mask that's applied
>>>> to
>>>> the data. You may specify as many of those masking regions as you'd
>>>> like
>>>> - probably one per land use category you'd like to verify.
>>>>
>>>> Hope that helps.
>>>>
>>>> John Halley Gotway
>>>>
>>>>
>>>>> Hi.
>>>>>
>>>>>
>>>>>
>>>>> I'm interested in verifying conditionally;
>>>> if(landuse==suchandsuch)then
>>>>> "how does surface temperature verify across cases and against
>>>>> observations/retrievals?" kind of thing. Basically, doing what MET
>>>> does
>>>>> with thresholds, but applying the threshold to other variables
>>>> (preferably
>>>>> multiple - if(x<landuse<y)elseif(y<landuse<z) for instance) with a
>>>> grib
>>>>> file, e.g.. Not finding that in MET: Am I missing something or is
>>>> that
>>>>> something that's planned for a (near-)future release?
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Mike
>>>>>
>>>>>
>>>>>
>>>>> ______________________________________________
>>>>>
>>>>> Michael Shaw, Contractor
>>>>>
>>>>> SAIC/NASA Support Scientist
>>>>>
>>>>> michael.shaw.ctr at offutt.af.mil
>>>>>
>>>>> 16WS Environmental Characterization
>>>>>
>>>>> HQ AFWA-SAIC
>>>>>
>>>>> 101 Nelson Drive
>>>>>
>>>>> Offutt AFB, NE 68113-1023
>>>>>
>>>>> 402-232-7690 Comm * 402-272-7690 DSN * 402-294-8230 Fax
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Met_help mailing list
>>>>> Met_help at mailman.ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>
>
More information about the Met_help
mailing list