[Met_help] Using MET with MADIS data
John Halley Gotway
johnhg at ucar.edu
Fri Feb 26 10:34:36 MST 2010
John,
Yes, the file formats are different, but I have never worked directly with MADIS data myself. I imagine you'd need to reformat the MADIS observations to make them look like the NetCDF output of
PB2NC. There have been other MET users who would like to use MADIS observations, but we haven't had time to add direct support for their use in MET.
I'll forward your request to others in our group to see where we can put it in our development timeline.
Thanks,
John
John Henderson wrote:
> Hello again John,
>
> Thanks again for your previous help.
>
> Perhaps you could also save me a little time by explaining whether MADIS
> data can be easily (or not so easily) used in WRF-MET. Specifically, is
> the ncdf file format generated by pb2nc different from what might be
> expected in, say, a MADIS mesonet ncdf file?
>
> Thanks.
>
> John
>
> On 1/7/10 2:39 PM, John Halley Gotway wrote:
>> John,
>>
>> Unfortunately, no. The PB2NC tool is not set up to extract
>> precipitation observations from PREPBUFR files. It's only set up to
>> extract observations of Pressure, Specific Humidity, Temperature,
>> Height, and U/V-components of wind.
>>
>> For verification of QPF, you'll either need to use a gridded analysis,
>> like StageII/IV (http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage2
>> and http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4) or
>> reformat ASCII versions of gauge data using the ASCII2NC tool for
>> point verification.
>>
>> I've done both of these.
>>
>> We would like to expand our support for the observation types packed
>> into the PREPBUFR files, but right now I'm really not sure what is/is
>> not in there.
>>
>> Thanks,
>> John
>>
>> John Henderson wrote:
>>
>>> John,
>>>
>>> Oh, okay. Just thought I'd point it out! Thanks for explaining...
>>> Unfortunately, right at the moment I don't have any long (>99-h)
>>> forecast fields handy to investigate my concerns directly.
>>>
>>> There is one other somewhat-related item you might be able to help me
>>> out with. I'm having trouble finding precipitation in pb2nc-generated
>>> netcdf files. I've converted a number of PrepBufr obs files but cannot
>>> find a grib code that should contain precip. Do you have any experience
>>> working with precip in PrepBufr format?
>>>
>>> Thanks.
>>>
>>> John
>>>
>>> On 1/7/10 2:14 PM, John Halley Gotway wrote:
>>>
>>>> John,
>>>>
>>>> No those "HH" arguments to PCP-Combine should be fine. Perhaps
>>>> listing the HH as 2-digits is misleading, but it'll read the integer
>>>> number of hours (any number of digits) and handle them correctly.
>>>>
>>>> To make sure, I ran PCP-Combine on some that that included
>>>> accumulations intervals> 100 hours. All three commands (sum, add,
>>>> and subtract) worked fine even when the accumulation interval on the
>>>> command line was> 100.
>>>>
>>>> Please let me know if anything else comes up.
>>>>
>>>> John
>>>>
>>>> John Henderson wrote:
>>>>
>>>>
>>>>> Hello again John,
>>>>>
>>>>> Regarding the> 99-h forecast lead time problem from a couple of
>>>>> weeks
>>>>> ago... I believe that PCP_COMBINE will also need to be modified since
>>>>> there is an argument to the code that is of the format HH (for the
>>>>> 'sum'
>>>>> option: out_accum).
>>>>>
>>>>> Please let me know if this is actually the case.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> John Henderson
>>>>>
>>>>> On 12/23/09 5:46 PM, John Halley Gotway wrote:
>>>>>
>>>>>
>>>>>> John,
>>>>>>
>>>>>> I checked and found out that this really is a problem for the
>>>>>> STAT-Analysis and MODE-Analysis tools. When I ran STAT-Analysis and
>>>>>> specified a lead time of "-fcst_lead 114", it errored out with an
>>>>>> error
>>>>>> message about not being able to parse that time.
>>>>>>
>>>>>> So I modified the library code which parses these strings to allow
>>>>>> for
>>>>>> the
>>>>>> lead hours to be either 2 or 3 digits long.
>>>>>>
>>>>>> I've posted the fix to:
>>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv2.0/index.php
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks for pointing out this problem, and please let me know if you
>>>>>> uncover any more issues.
>>>>>>
>>>>>> Thanks,
>>>>>> John
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> John,
>>>>>>>
>>>>>>> Thanks for the clarification of everything.
>>>>>>>
>>>>>>> I'll let you know if I uncover any other problems with triple-digit
>>>>>>> lead
>>>>>>> times, but, otherwise, I'll await your fix.
>>>>>>>
>>>>>>> Thanks for the quick responses!
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> On 12/23/09 4:37 PM, John Halley Gotway wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> John,
>>>>>>>>
>>>>>>>> Regarding WPP, I really don't think that you'll have a problem.
>>>>>>>> I'm
>>>>>>>> currently working on a project with folks from NOAA where we're
>>>>>>>> creating
>>>>>>>> forecasts out to 114 hours and are running the most recently
>>>>>>>> released
>>>>>>>> version of WPP just fine. You would only potentially have an
>>>>>>>> issue if
>>>>>>>> you
>>>>>>>> were running your forecasts out> 255 hours.
>>>>>>>>
>>>>>>>> Regarding issues with MET, you're probably correct. I think this
>>>>>>>> would
>>>>>>>> come up in your use of the MODE-Analysis and STAT-Analysis tools.
>>>>>>>> That's
>>>>>>>> the place you'd be specifying the forecast lead time in the config
>>>>>>>> file
>>>>>>>> or
>>>>>>>> on the command line. I believe it would be a pretty easy fix
>>>>>>>> involving
>>>>>>>> changes to the "timestring_to_sec" routine in the file
>>>>>>>> METv2.0/lib/vx_cal/time_strings.cc.
>>>>>>>>
>>>>>>>> However, before making any changes, I'd like to do some testing to
>>>>>>>> make
>>>>>>>> sure it really is a problem. And once I verify that, I'll work
>>>>>>>> on a
>>>>>>>> fix.
>>>>>>>> If I isolate the problem and come up with a fix, I'd post it to
>>>>>>>> the MET
>>>>>>>> website and update the package of bug fixes.
>>>>>>>>
>>>>>>>> I'll let you know how it pans out. Please let me know if, in your
>>>>>>>> use
>>>>>>>> of
>>>>>>>> MET, you find another place where it's a problem to have forecast
>>>>>>>> hours>
>>>>>>>> 99.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> John
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hello John,
>>>>>>>>>
>>>>>>>>> Thanks for the prompt response. Perhaps I had briefly come across
>>>>>>>>> the
>>>>>>>>> WPP email and thought it was actually related to MET...
>>>>>>>>>
>>>>>>>>> I believe - according to the user manual - that MET allows the
>>>>>>>>> user to
>>>>>>>>> specify forecast lead time as HH[MMSS], which seems to me to
>>>>>>>>> prevent
>>>>>>>>> leads times> 99h. Am I mistaken?
>>>>>>>>>
>>>>>>>>> Regarding WPP...I intend to verify precip accumulation in MET
>>>>>>>>> for a
>>>>>>>>> year-long series of 5.5-day forecasts that overlap at the
>>>>>>>>> beginning of
>>>>>>>>> each forecast run to avoid spin-up problems. It's a pretty
>>>>>>>>> standard
>>>>>>>>> way
>>>>>>>>> of generating year-long WRF simulations. However, that means
>>>>>>>>> that I
>>>>>>>>> will
>>>>>>>>> be applying WPP to WRF forecasts with forecast lead times of 12
>>>>>>>>> h to
>>>>>>>>> 132
>>>>>>>>> h. I hadn't anticipated any problems with WPP, unfortunately. I
>>>>>>>>> have a
>>>>>>>>> vague memory that someone had come up with a fix for the long-lead
>>>>>>>>> time
>>>>>>>>> problem, but I'm not sure. Would you be able to investigate the
>>>>>>>>> status
>>>>>>>>> for me?
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>> On 12/23/09 2:51 PM, John Halley Gotway wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> John,
>>>>>>>>>>
>>>>>>>>>> I looked back through the old MET-Help emails and couldn't find
>>>>>>>>>> anything
>>>>>>>>>> that would explain a problem of using fcst_lead>= 100 hours. So
>>>>>>>>>> I'm
>>>>>>>>>> not
>>>>>>>>>> aware of a problem with that. I did find one email
>>>>>>>>>> related to this but I believe that was due to an incorrectly
>>>>>>>>>> packed
>>>>>>>>>> GRIB
>>>>>>>>>> file, not a problem in MET itself.
>>>>>>>>>>
>>>>>>>>>> However, there are some issues in WPP (WRF-PostProcessor) for
>>>>>>>>>> accumulated precip when the forecast lead time extends beyond 255
>>>>>>>>>> hours.
>>>>>>>>>> The time value of 255+ overflows the single byte that's
>>>>>>>>>> allocated in the GRIB record to store that time info. This is
>>>>>>>>>> mostly
>>>>>>>>>> an
>>>>>>>>>> issue for accumulated precip since you have to store 2 times
>>>>>>>>>> in the
>>>>>>>>>> GRIB
>>>>>>>>>> record - accumulation starting and ending times. I'm
>>>>>>>>>> not sure of the status of this issue, but if you run across it,
>>>>>>>>>> I can
>>>>>>>>>> look into it more.
>>>>>>>>>>
>>>>>>>>>> I'd say, proceed with trying to use MET to verify forecasts>= 100
>>>>>>>>>> hours.
>>>>>>>>>> And if you run into any problems, just let us know through
>>>>>>>>>> met_help at ucar.edu.
>>>>>>>>>>
>>>>>>>>>> If you haven't already done so, I would suggest retrieving the
>>>>>>>>>> latest
>>>>>>>>>> set of bug fixes for METv2.0 from:
>>>>>>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv2.0/index.php
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hope that helps,
>>>>>>>>>> John Halley Gotway
>>>>>>>>>> johnhg at ucar.edu
>>>>>>>>>>
>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> When first I installed WRF-MET I believe I saw a discussion
>>>>>>>>>>> somewhere
>>>>>>>>>>> about how to apply WRF-MET to forecasts that require a
>>>>>>>>>>> fcst_lead of
>>>>>>>>>>> at
>>>>>>>>>>> least 100 h, however, I can't seem to find it now. Can you
>>>>>>>>>>> point me
>>>>>>>>>>> in
>>>>>>>>>>> the right direction?
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> John Henderson
>>>>>>>>>>> AER, Inc.
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Met_help mailing list
>>>>>>>>>>> Met_help at mailman.ucar.edu
>>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
More information about the Met_help
mailing list