[Met_help] New MET code for processing ncdf MADIS files
John Henderson
jhenders at aer.com
Thu Mar 4 12:49:54 MST 2010
Hi John,
We noticed that the way we are applying the QC flags to wind speed and
direction may not have been correct. We're addressing that right now.
Otherwise, we noticed that the grib encoding flags for wind speed and
direction are not set according to the standard (NCEP's office note
388). The correct ones are wind direction 031 and wind speed 032.
As well, we're going to try to pair our wind components like you
suggested for use in VL1L2. We see in the pb-based ncdf file that the u-
and v- components are next to each other.
I'll provide an updated file soon - one that will address the first two
problems but likely not the pairing of wind components.
John
On 3/4/10 2:24 PM, John Halley Gotway wrote:
> John,
>
> Yes, good point about UGRD and VGRD. In Point-Stat, if you'd like to compute vector L1L2 partial sums (i.e. the VL1L2 line type), you'd need to have each UGRD observations followed by a VGRD
> observation at the same level and pointing to the same header record.
>
> For the time being, just edit the Point-Stat configuration file. In the fcst_field, you probably have "UGRD" followed by "VGRD" at the same level. For right now, just reverse their order putting
> VGRD first. That will effectively disable treating U and V as vectors. Here's some comments from the Point-Stat config file about this.
>
> // NOTE: To verify winds as vectors rather than scalars,
> // specify UGRD (or 33) followed by VGRD (or 34) with the
> // same level values.
>
> The better fix would be to modify madis2nc to list U observations at one level followed by V observations at the same level and pointing to the same header record. We imposed the convention to make
> Point-Stat run a little faster. We didn't want it wasting time searching through all the observations to figure out how to match up the U's and V's.
>
> Hope that helps,
> John
>
> John Henderson wrote:
>
>> John,
>>
>> Thanks for identifying the bug on line 885. That fixes the indexing
>> problem, however, now I get this error in point_stat:
>>
>> ...
>> Reading records for VGRD/Z010.
>> For VGRD/Z010 found 1 forecast levels and 0 climatology levels.
>>
>> ----------------------------------------
>>
>> Searching 98753 observations from 23535 PrepBufr messages.
>>
>> ERROR: process_obs_file() -> when computing vector winds, each UGRD
>> observation must be followed by a VGRD observation with the same header
>> and at the same level
>>
>> Does this suggest that we're not pairing up UGRD/VGRD obs the right way
>> in madis2nc?
>>
>> Regarding your seg fault, obviously we can't reproduce it. We're trying
>> to think of a potential problem, but, of course, it's not easy when we
>> can't reproduce the problem.
>>
>> John
>>
>> On 3/4/10 1:43 PM, John Halley Gotway wrote:
>>
>>> John,
>>>
>>> When I tried to run madis2nc with the sample data file you sent me, I
>>> actually got a segfault. After commenting out lines 760-761 for
>>> writing Wind Speed observations, the seg fault went away:
>>> 759 // Wind Speed
>>> 760 // write_obs_arr(f_in, nhdr, in_windSpeed_str,
>>> in_windSpeed_gc,
>>> 761 // bad_data_float, obs_arr_var, i_obs,
>>> conversion);
>>>
>>> Once I ran madis2nc successfully, I did see the error message from
>>> Point-Stat. The problem is a bug in indexing in the code I sent you.
>>> Just remove the +1 from line 885:
>>> 885 obs_arr[0] = i_hdr;
>>>
>>> The first element in the obs_arr is the index of the header
>>> information to which this observation corresponds. Point-Stat expects
>>> that that index starts at 0 and not 1, like I'd had it.
>>>
>>> Give that a shot and see if it fixes the problem.
>>>
>>> Thanks,
>>> John
>>>
>>> John Henderson wrote:
>>>
>>>
>>>> John,
>>>>
>>>> We've created a new file, madis2nc.cc. The new code can be compiled
>>>> inside the awips2nc directory using the attached modified Makefile. Of
>>>> course, we'll let you decide the directory details of how the new code
>>>> will be presented to the user community.
>>>>
>>>> Here is what the AER developer says:
>>>>
>>>> "It runs to completion for the following data types:
>>>>
>>>>
>>>>> acars
>>>>> acarsProfiles
>>>>> LDAD_coop
>>>>> LDAD_hydro
>>>>> maritime
>>>>> LDAD_mesonet
>>>>> metar
>>>>> sao
>>>>> LDAD_snow
>>>>>
>>>>>
>>>> I checked that for METARS it gives the same result as awips2nc, aside
>>>> from obs filtered out by madis2nc due to QC checks. It may need some
>>>> more tweaking for the report type (e.g., ADPSFC, MSONET, etc) for some
>>>> of the other data types."
>>>>
>>>> If the mesonet obs either are flagged by any QC check, or have not been
>>>> subjected to any QC, the new code will not write them out into the MET
>>>> ncdf output file.
>>>>
>>>> One big problem, however: The ncdf files generated by both our new code
>>>> (madis2nc) and your old code (awips2nc) do not work in point_stats.
>>>> Perhaps the point_stats code has changed since you wrote awips2nc or I
>>>> simply am calling things incorrectly.
>>>>
>>>> I look forward to hearing what the problem is!
>>>>
>>>> Thanks.
>>>>
>>>> John
>>>>
>>>> On 3/2/10 2:59 PM, John Halley Gotway wrote:
>>>>
>>>>
>>>>> John,
>>>>>
>>>>> Thanks, I was able to retrieve the NetCDF file you sent and take a
>>>>> look at it. I was reminded of the fact that I'd written some code
>>>>> last summer to convert AWIPS point observation files into the
>>>>> NetCDF format expected by Point-Stat. But I didn't design it to be a
>>>>> very robust converter - just to get the job done for a particular NWS
>>>>> user of MET. I tried running your sample file though it,
>>>>> but it produced an error.
>>>>>
>>>>> However, I took a look at the structure of your NetCDF file compared
>>>>> to what the converter is trying to read, and really it's pretty close.
>>>>>
>>>>> I went ahead and attached the code for that tool, which I initially
>>>>> called "awips2nc". If you have anyone there who codes in C++, I think
>>>>> it'd be pretty straight-forward for them to adapt the tool to
>>>>> handle the data you're using. Perhaps this could serve as the
>>>>> starting point for a tool to ingest MADIS data directly into MET.
>>>>> Unfortunately though, we don't have time in our development schedule
>>>>> to work on that right now.
>>>>>
>>>>> Just copy the attached file "awips2nc_patch.tar.gz" into your
>>>>> top-level METv2.0 directory. After you untar it, you'll see a
>>>>> "src/awips2nc" directory and a new top-level Makefile named
>>>>> "Makefile_new".
>>>>> That includes lines for building the awips2nc tool. So just
>>>>> configure that Makefile to point to your local paths, and when you
>>>>> rebuild MET, it should compile awips2nc as well. If you're successful
>>>>> in changing that code to get it to work on your data, please send us
>>>>> your changes.
>>>>>
>>>>> Thanks,
>>>>> John
>>>>>
>>>>> John Henderson wrote:
>>>>>
>>>>>
>>>>>
>>>>>> John,
>>>>>>
>>>>>> I've uploaded a sample file that you have requested. The filename
>>>>>> is not
>>>>>> very informative (20061022_1000), but it is a ncdf file straight from
>>>>>> the MADIS server.
>>>>>>
>>>>>> Please let me know if the line of thinking that prompted this email
>>>>>> might be fruitful for our immediate needs. Otherwise, we may actually
>>>>>> start to write a MADIS ncdf -> ASCII script for ingest into MET. We
>>>>>> really would like to use mesonet obs in MET and I've been told that
>>>>>> retrieving multiple ncdf format files from MADIS is much cleaner than
>>>>>> retrieving ASCII format.
>>>>>>
>>>>>> John
>>>>>>
>>>>>> On 3/1/10 3:57 PM, John Halley Gotway wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> John,
>>>>>>>
>>>>>>> Our of curiosity, I'm wondering if you could send me a sample MADIS
>>>>>>> NetCDF file that you'd like to use in MET. I'd like to see how it
>>>>>>> compares to some AWIPS point observation data we'd read in the past.
>>>>>>>
>>>>>>> You could post a sample file to our anonymous ftp site:
>>>>>>> ftp ftp.rap.ucar.edu
>>>>>>> username = anonymous
>>>>>>> password = "your email address"
>>>>>>> cd incoming/irap/met_help
>>>>>>> put "your file(s)"
>>>>>>> bye
>>>>>>>
>>>>>>> I'd appreciate it.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> John
>>>>>>>
>>>>>>> John Halley Gotway wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> John,
>>>>>>>>
>>>>>>>> I looked back through our MET-Help emails and did find another user
>>>>>>>> going through the same process as you. And he actually sent us some
>>>>>>>> the scripts he was using for converting MADIS to ASCII for
>>>>>>>> surface observations. I wanted to pass them along to you in case
>>>>>>>> they're of use. However I think he was starting with an ASCII
>>>>>>>> version of MADIS data instead of NetCDF.
>>>>>>>>
>>>>>>>> His scripts are attached.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> John
>>>>>>>>
>>>>>>>> -------- Original Message --------
>>>>>>>> Subject: MADIS_ASCII2NC scripts/files
>>>>>>>> Date: Wed, 4 Mar 2009 12:49:06 -0600
>>>>>>>> From: Case, Jonathan (MSFC-VP61)[Other]<jonathan.case-1 at nasa.gov>
>>>>>>>> To: John Halley Gotway<johnhg at rap.ucar.edu>
>>>>>>>> References:<A9BB815E34BC6947A9DF9EABB15F147101B5EB18 at NDMSEVS31A.ndc.nasa.gov>
>>>>>>>>
>>>>>>>>
>>>>>>>> <53403.128.117.65.62.1236126600.squirrel at mail.rap.ucar.edu>
>>>>>>>> <A9BB815E34BC6947A9DF9EABB15F147101B5EB5F at NDMSEVS31A.ndc.nasa.gov>
>>>>>>>> <49AEA2B6.7080403 at rap.ucar.edu>
>>>>>>>>
>>>>>>>> John,
>>>>>>>>
>>>>>>>> I've bundled up a bunch of scripts/files I used to re-format and
>>>>>>>> process
>>>>>>>> the MADIS sfc observations for ascii2nc. The file is attached to
>>>>>>>> this
>>>>>>>> email.
>>>>>>>>
>>>>>>>> Please keep in mind that these scripts were just written (except for
>>>>>>>> dayadd, which I've used in many other scripts), so it's still
>>>>>>>> "drafty".
>>>>>>>> Also, it's only set up to work on sfc observations only, not any
>>>>>>>> upper-air data yet.
>>>>>>>>
>>>>>>>> Take care,
>>>>>>>> Jonathan
>>>>>>>>
>>>>>>>> John Henderson wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> John,
>>>>>>>>>
>>>>>>>>> Yes, of course, reformatting into the catch-all ASCII format seems
>>>>>>>>> like
>>>>>>>>> the best start. Thanks for reminding me about that option. Will
>>>>>>>>> let you
>>>>>>>>> know how the MADIS->ASCII script works if I choose to jump into the
>>>>>>>>> task!
>>>>>>>>>
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>> On 2/26/10 1:38 PM, John Halley Gotway wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> John,
>>>>>>>>>>
>>>>>>>>>> I can tell you that unfortunately it won't be as simple as
>>>>>>>>>> renaming
>>>>>>>>>> some variables. You know, rather than modifying the NetCDF file
>>>>>>>>>> directly and trying to figure out the details of what the PB2NC
>>>>>>>>>> output looks like, I'd suggest using an intermediate ASCII file.
>>>>>>>>>>
>>>>>>>>>> It'd probably be easiest to parse the MADIS observations and dump
>>>>>>>>>> them
>>>>>>>>>> out in the ASCII format that the ASCII2NC tool can read. Then run
>>>>>>>>>> them through ASCII2NC and use them in Point-Stat. I know
>>>>>>>>>> that's a lot of steps, but I do think it'd be the most
>>>>>>>>>> straight-forward way to go.
>>>>>>>>>>
>>>>>>>>>> You can find details of what the supported ASCII format in the MET
>>>>>>>>>> User's Guide:
>>>>>>>>>>
>>>>>>>>>> http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v2.0_rev2.pdf
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Keep us posted on know how it goes. And if you're MADIS ->
>>>>>>>>>> ASCII
>>>>>>>>>> script works well, we'd be happy to post it on the MET website for
>>>>>>>>>> other users.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> John
>>>>>>>>>>
>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi John,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for your quick response.
>>>>>>>>>>>
>>>>>>>>>>> Would anyone be able to provide some basic details (e.g., what
>>>>>>>>>>> PB2NC
>>>>>>>>>>> fields are required) of the PB2NC ncdf files to help guide me
>>>>>>>>>>> with
>>>>>>>>>>> trying to match up the MADIS ncdf and PB2NC ncdf files? Perhaps
>>>>>>>>>>> it's a
>>>>>>>>>>> simple as a renaming of a field or two...
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> John
>>>>>>>>>>>
>>>>>>>>>>> On 2/26/10 12:34 PM, John Halley Gotway wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> John,
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, the file formats are different, but I have never worked
>>>>>>>>>>>> directly
>>>>>>>>>>>> with MADIS data myself. I imagine you'd need to reformat the
>>>>>>>>>>>> MADIS
>>>>>>>>>>>> observations to make them look like the NetCDF output of
>>>>>>>>>>>> PB2NC. There have been other MET users who would like to use
>>>>>>>>>>>> MADIS
>>>>>>>>>>>> observations, but we haven't had time to add direct support for
>>>>>>>>>>>> their
>>>>>>>>>>>> use in MET.
>>>>>>>>>>>>
>>>>>>>>>>>> I'll forward your request to others in our group to see where
>>>>>>>>>>>> we can
>>>>>>>>>>>> put it in our development timeline.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> John
>>>>>>>>>>>>
>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello again John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks again for your previous help.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Perhaps you could also save me a little time by explaining
>>>>>>>>>>>>> whether
>>>>>>>>>>>>> MADIS
>>>>>>>>>>>>> data can be easily (or not so easily) used in WRF-MET.
>>>>>>>>>>>>> Specifically, is
>>>>>>>>>>>>> the ncdf file format generated by pb2nc different from what
>>>>>>>>>>>>> might be
>>>>>>>>>>>>> expected in, say, a MADIS mesonet ncdf file?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> John
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 1/7/10 2:39 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Unfortunately, no. The PB2NC tool is not set up to extract
>>>>>>>>>>>>>> precipitation observations from PREPBUFR files. It's only set
>>>>>>>>>>>>>> up to
>>>>>>>>>>>>>> extract observations of Pressure, Specific Humidity,
>>>>>>>>>>>>>> Temperature,
>>>>>>>>>>>>>> Height, and U/V-components of wind.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For verification of QPF, you'll either need to use a gridded
>>>>>>>>>>>>>> analysis,
>>>>>>>>>>>>>> like StageII/IV
>>>>>>>>>>>>>> (http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage2
>>>>>>>>>>>>>> and http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4) or
>>>>>>>>>>>>>> reformat ASCII versions of gauge data using the ASCII2NC tool
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> point verification.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've done both of these.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We would like to expand our support for the observation types
>>>>>>>>>>>>>> packed
>>>>>>>>>>>>>> into the PREPBUFR files, but right now I'm really not sure
>>>>>>>>>>>>>> what
>>>>>>>>>>>>>> is/is
>>>>>>>>>>>>>> not in there.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Oh, okay. Just thought I'd point it out! Thanks for
>>>>>>>>>>>>>>> explaining...
>>>>>>>>>>>>>>> Unfortunately, right at the moment I don't have any long
>>>>>>>>>>>>>>> (>99-h)
>>>>>>>>>>>>>>> forecast fields handy to investigate my concerns directly.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is one other somewhat-related item you might be able to
>>>>>>>>>>>>>>> help me
>>>>>>>>>>>>>>> out with. I'm having trouble finding precipitation in
>>>>>>>>>>>>>>> pb2nc-generated
>>>>>>>>>>>>>>> netcdf files. I've converted a number of PrepBufr obs
>>>>>>>>>>>>>>> files but
>>>>>>>>>>>>>>> cannot
>>>>>>>>>>>>>>> find a grib code that should contain precip. Do you have any
>>>>>>>>>>>>>>> experience
>>>>>>>>>>>>>>> working with precip in PrepBufr format?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 1/7/10 2:14 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No those "HH" arguments to PCP-Combine should be fine.
>>>>>>>>>>>>>>>> Perhaps
>>>>>>>>>>>>>>>> listing the HH as 2-digits is misleading, but it'll read the
>>>>>>>>>>>>>>>> integer
>>>>>>>>>>>>>>>> number of hours (any number of digits) and handle them
>>>>>>>>>>>>>>>> correctly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To make sure, I ran PCP-Combine on some that that included
>>>>>>>>>>>>>>>> accumulations intervals> 100 hours. All three
>>>>>>>>>>>>>>>> commands
>>>>>>>>>>>>>>>> (sum, add,
>>>>>>>>>>>>>>>> and subtract) worked fine even when the accumulation
>>>>>>>>>>>>>>>> interval on
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> command line was> 100.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please let me know if anything else comes up.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello again John,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regarding the> 99-h forecast lead time problem
>>>>>>>>>>>>>>>>> from a
>>>>>>>>>>>>>>>>> couple of
>>>>>>>>>>>>>>>>> weeks
>>>>>>>>>>>>>>>>> ago... I believe that PCP_COMBINE will also need to be
>>>>>>>>>>>>>>>>> modified
>>>>>>>>>>>>>>>>> since
>>>>>>>>>>>>>>>>> there is an argument to the code that is of the format HH
>>>>>>>>>>>>>>>>> (for the
>>>>>>>>>>>>>>>>> 'sum'
>>>>>>>>>>>>>>>>> option: out_accum).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please let me know if this is actually the case.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> John Henderson
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 12/23/09 5:46 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I checked and found out that this really is a problem for
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> STAT-Analysis and MODE-Analysis tools. When I ran
>>>>>>>>>>>>>>>>>> STAT-Analysis
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> specified a lead time of "-fcst_lead 114", it errored out
>>>>>>>>>>>>>>>>>> with an
>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>> message about not being able to parse that time.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> So I modified the library code which parses these
>>>>>>>>>>>>>>>>>> strings to
>>>>>>>>>>>>>>>>>> allow
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> lead hours to be either 2 or 3 digits long.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I've posted the fix to:
>>>>>>>>>>>>>>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv2.0/index.php
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks for pointing out this problem, and please let me
>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>> if you
>>>>>>>>>>>>>>>>>> uncover any more issues.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for the clarification of everything.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'll let you know if I uncover any other problems with
>>>>>>>>>>>>>>>>>>> triple-digit
>>>>>>>>>>>>>>>>>>> lead
>>>>>>>>>>>>>>>>>>> times, but, otherwise, I'll await your fix.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for the quick responses!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 12/23/09 4:37 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Regarding WPP, I really don't think that you'll have a
>>>>>>>>>>>>>>>>>>>> problem.
>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>> currently working on a project with folks from NOAA
>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>> we're
>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>> forecasts out to 114 hours and are running the most
>>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>> released
>>>>>>>>>>>>>>>>>>>> version of WPP just fine. You would only potentially
>>>>>>>>>>>>>>>>>>>> have an
>>>>>>>>>>>>>>>>>>>> issue if
>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>> were running your forecasts out> 255 hours.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Regarding issues with MET, you're probably correct. I
>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>> come up in your use of the MODE-Analysis and
>>>>>>>>>>>>>>>>>>>> STAT-Analysis
>>>>>>>>>>>>>>>>>>>> tools.
>>>>>>>>>>>>>>>>>>>> That's
>>>>>>>>>>>>>>>>>>>> the place you'd be specifying the forecast lead time in
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> config
>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>> on the command line. I believe it would be a
>>>>>>>>>>>>>>>>>>>> pretty
>>>>>>>>>>>>>>>>>>>> easy fix
>>>>>>>>>>>>>>>>>>>> involving
>>>>>>>>>>>>>>>>>>>> changes to the "timestring_to_sec" routine in the file
>>>>>>>>>>>>>>>>>>>> METv2.0/lib/vx_cal/time_strings.cc.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> However, before making any changes, I'd like to do some
>>>>>>>>>>>>>>>>>>>> testing to
>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>> sure it really is a problem. And once I verify that,
>>>>>>>>>>>>>>>>>>>> I'll work
>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>> fix.
>>>>>>>>>>>>>>>>>>>> If I isolate the problem and come up with a fix, I'd
>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>> it to
>>>>>>>>>>>>>>>>>>>> the MET
>>>>>>>>>>>>>>>>>>>> website and update the package of bug fixes.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'll let you know how it pans out. Please let me know
>>>>>>>>>>>>>>>>>>>> if, in
>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> MET, you find another place where it's a problem to have
>>>>>>>>>>>>>>>>>>>> forecast
>>>>>>>>>>>>>>>>>>>> hours>
>>>>>>>>>>>>>>>>>>>> 99.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hello John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks for the prompt response. Perhaps I had briefly
>>>>>>>>>>>>>>>>>>>>> come
>>>>>>>>>>>>>>>>>>>>> across
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> WPP email and thought it was actually related to MET...
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I believe - according to the user manual - that MET
>>>>>>>>>>>>>>>>>>>>> allows the
>>>>>>>>>>>>>>>>>>>>> user to
>>>>>>>>>>>>>>>>>>>>> specify forecast lead time as HH[MMSS], which seems to
>>>>>>>>>>>>>>>>>>>>> me to
>>>>>>>>>>>>>>>>>>>>> prevent
>>>>>>>>>>>>>>>>>>>>> leads times> 99h. Am I mistaken?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Regarding WPP...I intend to verify precip accumulation
>>>>>>>>>>>>>>>>>>>>> in MET
>>>>>>>>>>>>>>>>>>>>> for a
>>>>>>>>>>>>>>>>>>>>> year-long series of 5.5-day forecasts that overlap
>>>>>>>>>>>>>>>>>>>>> at the
>>>>>>>>>>>>>>>>>>>>> beginning of
>>>>>>>>>>>>>>>>>>>>> each forecast run to avoid spin-up problems. It's a
>>>>>>>>>>>>>>>>>>>>> pretty
>>>>>>>>>>>>>>>>>>>>> standard
>>>>>>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>>>>> of generating year-long WRF simulations. However, that
>>>>>>>>>>>>>>>>>>>>> means
>>>>>>>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>> be applying WPP to WRF forecasts with forecast lead
>>>>>>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>>>>>> of 12
>>>>>>>>>>>>>>>>>>>>> h to
>>>>>>>>>>>>>>>>>>>>> 132
>>>>>>>>>>>>>>>>>>>>> h. I hadn't anticipated any problems with WPP,
>>>>>>>>>>>>>>>>>>>>> unfortunately. I
>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>> vague memory that someone had come up with a fix for
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> long-lead
>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>> problem, but I'm not sure. Would you be able to
>>>>>>>>>>>>>>>>>>>>> investigate
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> status
>>>>>>>>>>>>>>>>>>>>> for me?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 12/23/09 2:51 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I looked back through the old MET-Help emails and
>>>>>>>>>>>>>>>>>>>>>> couldn't
>>>>>>>>>>>>>>>>>>>>>> find
>>>>>>>>>>>>>>>>>>>>>> anything
>>>>>>>>>>>>>>>>>>>>>> that would explain a problem of using fcst_lead>= 100
>>>>>>>>>>>>>>>>>>>>>> hours. So
>>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>> aware of a problem with that. I did find one email
>>>>>>>>>>>>>>>>>>>>>> related to this but I believe that was due to an
>>>>>>>>>>>>>>>>>>>>>> incorrectly
>>>>>>>>>>>>>>>>>>>>>> packed
>>>>>>>>>>>>>>>>>>>>>> GRIB
>>>>>>>>>>>>>>>>>>>>>> file, not a problem in MET itself.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> However, there are some issues in WPP
>>>>>>>>>>>>>>>>>>>>>> (WRF-PostProcessor) for
>>>>>>>>>>>>>>>>>>>>>> accumulated precip when the forecast lead time extends
>>>>>>>>>>>>>>>>>>>>>> beyond 255
>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>> The time value of 255+ overflows the single
>>>>>>>>>>>>>>>>>>>>>> byte
>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>> allocated in the GRIB record to store that time info.
>>>>>>>>>>>>>>>>>>>>>> This is
>>>>>>>>>>>>>>>>>>>>>> mostly
>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> issue for accumulated precip since you have to store 2
>>>>>>>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>> GRIB
>>>>>>>>>>>>>>>>>>>>>> record - accumulation starting and ending times. I'm
>>>>>>>>>>>>>>>>>>>>>> not sure of the status of this issue, but if you run
>>>>>>>>>>>>>>>>>>>>>> across it,
>>>>>>>>>>>>>>>>>>>>>> I can
>>>>>>>>>>>>>>>>>>>>>> look into it more.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'd say, proceed with trying to use MET to verify
>>>>>>>>>>>>>>>>>>>>>> forecasts>= 100
>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>> And if you run into any problems, just let
>>>>>>>>>>>>>>>>>>>>>> us know
>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>> met_help at ucar.edu.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If you haven't already done so, I would suggest
>>>>>>>>>>>>>>>>>>>>>> retrieving
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> latest
>>>>>>>>>>>>>>>>>>>>>> set of bug fixes for METv2.0 from:
>>>>>>>>>>>>>>>>>>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv2.0/index.php
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hope that helps,
>>>>>>>>>>>>>>>>>>>>>> John Halley Gotway
>>>>>>>>>>>>>>>>>>>>>> johnhg at ucar.edu
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> When first I installed WRF-MET I believe I saw a
>>>>>>>>>>>>>>>>>>>>>>> discussion
>>>>>>>>>>>>>>>>>>>>>>> somewhere
>>>>>>>>>>>>>>>>>>>>>>> about how to apply WRF-MET to forecasts that
>>>>>>>>>>>>>>>>>>>>>>> require a
>>>>>>>>>>>>>>>>>>>>>>> fcst_lead of
>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>> least 100 h, however, I can't seem to find it now.
>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>> point me
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> the right direction?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> John Henderson
>>>>>>>>>>>>>>>>>>>>>>> AER, Inc.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>>>> Met_help mailing list
>>>>>>>>>>>>>>>>>>>>>>> Met_help at mailman.ucar.edu
>>>>>>>>>>>>>>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>
>>>>
More information about the Met_help
mailing list