[Met_help] New MET code for processing ncdf MADIS files
John Halley Gotway
johnhg at ucar.edu
Thu Mar 4 13:36:58 MST 2010
John,
Ah, yes, I see the bug in GRIB codes you describe on line 85 of madis2nc.cc:
84 const int in_windDir_gc = 31;
85 const int in_windSpeed_gc = 34;
Looks like the wind direction is correct at 31, but the wind speed is wrong and should be set as:
84 const int in_windDir_gc = 31;
85 const int in_windSpeed_gc = 32;
These are actually defined in a MET library: METv2.0/lib/vx_grib_classes/grib_strings.h
So if this tool is compiled as part of MET, we could just use the definitions in there.
Thanks,
John
John Henderson wrote:
> Hi John,
>
> We noticed that the way we are applying the QC flags to wind speed and
> direction may not have been correct. We're addressing that right now.
> Otherwise, we noticed that the grib encoding flags for wind speed and
> direction are not set according to the standard (NCEP's office note
> 388). The correct ones are wind direction 031 and wind speed 032.
>
> As well, we're going to try to pair our wind components like you
> suggested for use in VL1L2. We see in the pb-based ncdf file that the u-
> and v- components are next to each other.
>
> I'll provide an updated file soon - one that will address the first two
> problems but likely not the pairing of wind components.
>
> John
>
> On 3/4/10 2:24 PM, John Halley Gotway wrote:
>> John,
>>
>> Yes, good point about UGRD and VGRD. In Point-Stat, if you'd like to
>> compute vector L1L2 partial sums (i.e. the VL1L2 line type), you'd
>> need to have each UGRD observations followed by a VGRD
>> observation at the same level and pointing to the same header record.
>>
>> For the time being, just edit the Point-Stat configuration file. In
>> the fcst_field, you probably have "UGRD" followed by "VGRD" at the
>> same level. For right now, just reverse their order putting
>> VGRD first. That will effectively disable treating U and V as
>> vectors. Here's some comments from the Point-Stat config file about
>> this.
>>
>> // NOTE: To verify winds as vectors rather than scalars,
>> // specify UGRD (or 33) followed by VGRD (or 34) with the
>> // same level values.
>>
>> The better fix would be to modify madis2nc to list U observations at
>> one level followed by V observations at the same level and pointing to
>> the same header record. We imposed the convention to make
>> Point-Stat run a little faster. We didn't want it wasting time
>> searching through all the observations to figure out how to match up
>> the U's and V's.
>>
>> Hope that helps,
>> John
>>
>> John Henderson wrote:
>>
>>> John,
>>>
>>> Thanks for identifying the bug on line 885. That fixes the indexing
>>> problem, however, now I get this error in point_stat:
>>>
>>> ...
>>> Reading records for VGRD/Z010.
>>> For VGRD/Z010 found 1 forecast levels and 0 climatology levels.
>>>
>>> ----------------------------------------
>>>
>>> Searching 98753 observations from 23535 PrepBufr messages.
>>>
>>> ERROR: process_obs_file() -> when computing vector winds, each UGRD
>>> observation must be followed by a VGRD observation with the same header
>>> and at the same level
>>>
>>> Does this suggest that we're not pairing up UGRD/VGRD obs the right way
>>> in madis2nc?
>>>
>>> Regarding your seg fault, obviously we can't reproduce it. We're trying
>>> to think of a potential problem, but, of course, it's not easy when we
>>> can't reproduce the problem.
>>>
>>> John
>>>
>>> On 3/4/10 1:43 PM, John Halley Gotway wrote:
>>>
>>>> John,
>>>>
>>>> When I tried to run madis2nc with the sample data file you sent me, I
>>>> actually got a segfault. After commenting out lines 760-761 for
>>>> writing Wind Speed observations, the seg fault went away:
>>>> 759 // Wind Speed
>>>> 760 // write_obs_arr(f_in, nhdr, in_windSpeed_str,
>>>> in_windSpeed_gc,
>>>> 761 // bad_data_float, obs_arr_var, i_obs,
>>>> conversion);
>>>>
>>>> Once I ran madis2nc successfully, I did see the error message from
>>>> Point-Stat. The problem is a bug in indexing in the code I sent you.
>>>> Just remove the +1 from line 885:
>>>> 885 obs_arr[0] = i_hdr;
>>>>
>>>> The first element in the obs_arr is the index of the header
>>>> information to which this observation corresponds. Point-Stat expects
>>>> that that index starts at 0 and not 1, like I'd had it.
>>>>
>>>> Give that a shot and see if it fixes the problem.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> John Henderson wrote:
>>>>
>>>>
>>>>> John,
>>>>>
>>>>> We've created a new file, madis2nc.cc. The new code can be compiled
>>>>> inside the awips2nc directory using the attached modified Makefile. Of
>>>>> course, we'll let you decide the directory details of how the new code
>>>>> will be presented to the user community.
>>>>>
>>>>> Here is what the AER developer says:
>>>>>
>>>>> "It runs to completion for the following data types:
>>>>>
>>>>>
>>>>>> acars
>>>>>> acarsProfiles
>>>>>> LDAD_coop
>>>>>> LDAD_hydro
>>>>>> maritime
>>>>>> LDAD_mesonet
>>>>>> metar
>>>>>> sao
>>>>>> LDAD_snow
>>>>>>
>>>>>>
>>>>> I checked that for METARS it gives the same result as awips2nc, aside
>>>>> from obs filtered out by madis2nc due to QC checks. It may need some
>>>>> more tweaking for the report type (e.g., ADPSFC, MSONET, etc) for some
>>>>> of the other data types."
>>>>>
>>>>> If the mesonet obs either are flagged by any QC check, or have not
>>>>> been
>>>>> subjected to any QC, the new code will not write them out into the MET
>>>>> ncdf output file.
>>>>>
>>>>> One big problem, however: The ncdf files generated by both our new
>>>>> code
>>>>> (madis2nc) and your old code (awips2nc) do not work in point_stats.
>>>>> Perhaps the point_stats code has changed since you wrote awips2nc or I
>>>>> simply am calling things incorrectly.
>>>>>
>>>>> I look forward to hearing what the problem is!
>>>>>
>>>>> Thanks.
>>>>>
>>>>> John
>>>>>
>>>>> On 3/2/10 2:59 PM, John Halley Gotway wrote:
>>>>>
>>>>>
>>>>>> John,
>>>>>>
>>>>>> Thanks, I was able to retrieve the NetCDF file you sent and take a
>>>>>> look at it. I was reminded of the fact that I'd written some code
>>>>>> last summer to convert AWIPS point observation files into the
>>>>>> NetCDF format expected by Point-Stat. But I didn't design it to be a
>>>>>> very robust converter - just to get the job done for a particular NWS
>>>>>> user of MET. I tried running your sample file though it,
>>>>>> but it produced an error.
>>>>>>
>>>>>> However, I took a look at the structure of your NetCDF file compared
>>>>>> to what the converter is trying to read, and really it's pretty
>>>>>> close.
>>>>>>
>>>>>> I went ahead and attached the code for that tool, which I initially
>>>>>> called "awips2nc". If you have anyone there who codes in C++, I
>>>>>> think
>>>>>> it'd be pretty straight-forward for them to adapt the tool to
>>>>>> handle the data you're using. Perhaps this could serve as the
>>>>>> starting point for a tool to ingest MADIS data directly into MET.
>>>>>> Unfortunately though, we don't have time in our development schedule
>>>>>> to work on that right now.
>>>>>>
>>>>>> Just copy the attached file "awips2nc_patch.tar.gz" into your
>>>>>> top-level METv2.0 directory. After you untar it, you'll see a
>>>>>> "src/awips2nc" directory and a new top-level Makefile named
>>>>>> "Makefile_new".
>>>>>> That includes lines for building the awips2nc tool. So just
>>>>>> configure that Makefile to point to your local paths, and when you
>>>>>> rebuild MET, it should compile awips2nc as well. If you're
>>>>>> successful
>>>>>> in changing that code to get it to work on your data, please send us
>>>>>> your changes.
>>>>>>
>>>>>> Thanks,
>>>>>> John
>>>>>>
>>>>>> John Henderson wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> John,
>>>>>>>
>>>>>>> I've uploaded a sample file that you have requested. The filename
>>>>>>> is not
>>>>>>> very informative (20061022_1000), but it is a ncdf file straight
>>>>>>> from
>>>>>>> the MADIS server.
>>>>>>>
>>>>>>> Please let me know if the line of thinking that prompted this email
>>>>>>> might be fruitful for our immediate needs. Otherwise, we may
>>>>>>> actually
>>>>>>> start to write a MADIS ncdf -> ASCII script for ingest into
>>>>>>> MET. We
>>>>>>> really would like to use mesonet obs in MET and I've been told that
>>>>>>> retrieving multiple ncdf format files from MADIS is much cleaner
>>>>>>> than
>>>>>>> retrieving ASCII format.
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> On 3/1/10 3:57 PM, John Halley Gotway wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> John,
>>>>>>>>
>>>>>>>> Our of curiosity, I'm wondering if you could send me a sample MADIS
>>>>>>>> NetCDF file that you'd like to use in MET. I'd like to see how it
>>>>>>>> compares to some AWIPS point observation data we'd read in the
>>>>>>>> past.
>>>>>>>>
>>>>>>>> You could post a sample file to our anonymous ftp site:
>>>>>>>> ftp ftp.rap.ucar.edu
>>>>>>>> username = anonymous
>>>>>>>> password = "your email address"
>>>>>>>> cd incoming/irap/met_help
>>>>>>>> put "your file(s)"
>>>>>>>> bye
>>>>>>>>
>>>>>>>> I'd appreciate it.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> John
>>>>>>>>
>>>>>>>> John Halley Gotway wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> John,
>>>>>>>>>
>>>>>>>>> I looked back through our MET-Help emails and did find another
>>>>>>>>> user
>>>>>>>>> going through the same process as you. And he actually sent us
>>>>>>>>> some
>>>>>>>>> the scripts he was using for converting MADIS to ASCII for
>>>>>>>>> surface observations. I wanted to pass them along to you in case
>>>>>>>>> they're of use. However I think he was starting with an ASCII
>>>>>>>>> version of MADIS data instead of NetCDF.
>>>>>>>>>
>>>>>>>>> His scripts are attached.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>> -------- Original Message --------
>>>>>>>>> Subject: MADIS_ASCII2NC scripts/files
>>>>>>>>> Date: Wed, 4 Mar 2009 12:49:06 -0600
>>>>>>>>> From: Case, Jonathan (MSFC-VP61)[Other]<jonathan.case-1 at nasa.gov>
>>>>>>>>> To: John Halley Gotway<johnhg at rap.ucar.edu>
>>>>>>>>> References:<A9BB815E34BC6947A9DF9EABB15F147101B5EB18 at NDMSEVS31A.ndc.nasa.gov>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <53403.128.117.65.62.1236126600.squirrel at mail.rap.ucar.edu>
>>>>>>>>> <A9BB815E34BC6947A9DF9EABB15F147101B5EB5F at NDMSEVS31A.ndc.nasa.gov>
>>>>>>>>> <49AEA2B6.7080403 at rap.ucar.edu>
>>>>>>>>>
>>>>>>>>> John,
>>>>>>>>>
>>>>>>>>> I've bundled up a bunch of scripts/files I used to re-format and
>>>>>>>>> process
>>>>>>>>> the MADIS sfc observations for ascii2nc. The file is attached to
>>>>>>>>> this
>>>>>>>>> email.
>>>>>>>>>
>>>>>>>>> Please keep in mind that these scripts were just written
>>>>>>>>> (except for
>>>>>>>>> dayadd, which I've used in many other scripts), so it's still
>>>>>>>>> "drafty".
>>>>>>>>> Also, it's only set up to work on sfc observations only, not any
>>>>>>>>> upper-air data yet.
>>>>>>>>>
>>>>>>>>> Take care,
>>>>>>>>> Jonathan
>>>>>>>>>
>>>>>>>>> John Henderson wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> John,
>>>>>>>>>>
>>>>>>>>>> Yes, of course, reformatting into the catch-all ASCII format
>>>>>>>>>> seems
>>>>>>>>>> like
>>>>>>>>>> the best start. Thanks for reminding me about that option. Will
>>>>>>>>>> let you
>>>>>>>>>> know how the MADIS->ASCII script works if I choose to jump
>>>>>>>>>> into the
>>>>>>>>>> task!
>>>>>>>>>>
>>>>>>>>>> John
>>>>>>>>>>
>>>>>>>>>> On 2/26/10 1:38 PM, John Halley Gotway wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> John,
>>>>>>>>>>>
>>>>>>>>>>> I can tell you that unfortunately it won't be as simple as
>>>>>>>>>>> renaming
>>>>>>>>>>> some variables. You know, rather than modifying the NetCDF file
>>>>>>>>>>> directly and trying to figure out the details of what the PB2NC
>>>>>>>>>>> output looks like, I'd suggest using an intermediate ASCII file.
>>>>>>>>>>>
>>>>>>>>>>> It'd probably be easiest to parse the MADIS observations and
>>>>>>>>>>> dump
>>>>>>>>>>> them
>>>>>>>>>>> out in the ASCII format that the ASCII2NC tool can read.
>>>>>>>>>>> Then run
>>>>>>>>>>> them through ASCII2NC and use them in Point-Stat. I know
>>>>>>>>>>> that's a lot of steps, but I do think it'd be the most
>>>>>>>>>>> straight-forward way to go.
>>>>>>>>>>>
>>>>>>>>>>> You can find details of what the supported ASCII format in
>>>>>>>>>>> the MET
>>>>>>>>>>> User's Guide:
>>>>>>>>>>>
>>>>>>>>>>> http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v2.0_rev2.pdf
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Keep us posted on know how it goes. And if you're MADIS ->
>>>>>>>>>>> ASCII
>>>>>>>>>>> script works well, we'd be happy to post it on the MET
>>>>>>>>>>> website for
>>>>>>>>>>> other users.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> John
>>>>>>>>>>>
>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi John,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for your quick response.
>>>>>>>>>>>>
>>>>>>>>>>>> Would anyone be able to provide some basic details (e.g., what
>>>>>>>>>>>> PB2NC
>>>>>>>>>>>> fields are required) of the PB2NC ncdf files to help guide me
>>>>>>>>>>>> with
>>>>>>>>>>>> trying to match up the MADIS ncdf and PB2NC ncdf files? Perhaps
>>>>>>>>>>>> it's a
>>>>>>>>>>>> simple as a renaming of a field or two...
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>
>>>>>>>>>>>> John
>>>>>>>>>>>>
>>>>>>>>>>>> On 2/26/10 12:34 PM, John Halley Gotway wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> John,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, the file formats are different, but I have never worked
>>>>>>>>>>>>> directly
>>>>>>>>>>>>> with MADIS data myself. I imagine you'd need to reformat the
>>>>>>>>>>>>> MADIS
>>>>>>>>>>>>> observations to make them look like the NetCDF output of
>>>>>>>>>>>>> PB2NC. There have been other MET users who would like to use
>>>>>>>>>>>>> MADIS
>>>>>>>>>>>>> observations, but we haven't had time to add direct support
>>>>>>>>>>>>> for
>>>>>>>>>>>>> their
>>>>>>>>>>>>> use in MET.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll forward your request to others in our group to see where
>>>>>>>>>>>>> we can
>>>>>>>>>>>>> put it in our development timeline.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> John
>>>>>>>>>>>>>
>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello again John,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks again for your previous help.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Perhaps you could also save me a little time by explaining
>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>> MADIS
>>>>>>>>>>>>>> data can be easily (or not so easily) used in WRF-MET.
>>>>>>>>>>>>>> Specifically, is
>>>>>>>>>>>>>> the ncdf file format generated by pb2nc different from what
>>>>>>>>>>>>>> might be
>>>>>>>>>>>>>> expected in, say, a MADIS mesonet ncdf file?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 1/7/10 2:39 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Unfortunately, no. The PB2NC tool is not set up to extract
>>>>>>>>>>>>>>> precipitation observations from PREPBUFR files. It's
>>>>>>>>>>>>>>> only set
>>>>>>>>>>>>>>> up to
>>>>>>>>>>>>>>> extract observations of Pressure, Specific Humidity,
>>>>>>>>>>>>>>> Temperature,
>>>>>>>>>>>>>>> Height, and U/V-components of wind.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> For verification of QPF, you'll either need to use a gridded
>>>>>>>>>>>>>>> analysis,
>>>>>>>>>>>>>>> like StageII/IV
>>>>>>>>>>>>>>> (http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage2
>>>>>>>>>>>>>>> and http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4) or
>>>>>>>>>>>>>>> reformat ASCII versions of gauge data using the ASCII2NC
>>>>>>>>>>>>>>> tool
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> point verification.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've done both of these.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We would like to expand our support for the observation
>>>>>>>>>>>>>>> types
>>>>>>>>>>>>>>> packed
>>>>>>>>>>>>>>> into the PREPBUFR files, but right now I'm really not sure
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>> is/is
>>>>>>>>>>>>>>> not in there.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Oh, okay. Just thought I'd point it out! Thanks for
>>>>>>>>>>>>>>>> explaining...
>>>>>>>>>>>>>>>> Unfortunately, right at the moment I don't have any long
>>>>>>>>>>>>>>>> (>99-h)
>>>>>>>>>>>>>>>> forecast fields handy to investigate my concerns directly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is one other somewhat-related item you might be
>>>>>>>>>>>>>>>> able to
>>>>>>>>>>>>>>>> help me
>>>>>>>>>>>>>>>> out with. I'm having trouble finding precipitation in
>>>>>>>>>>>>>>>> pb2nc-generated
>>>>>>>>>>>>>>>> netcdf files. I've converted a number of PrepBufr obs
>>>>>>>>>>>>>>>> files but
>>>>>>>>>>>>>>>> cannot
>>>>>>>>>>>>>>>> find a grib code that should contain precip. Do you have
>>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>> experience
>>>>>>>>>>>>>>>> working with precip in PrepBufr format?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 1/7/10 2:14 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> No those "HH" arguments to PCP-Combine should be fine.
>>>>>>>>>>>>>>>>> Perhaps
>>>>>>>>>>>>>>>>> listing the HH as 2-digits is misleading, but it'll
>>>>>>>>>>>>>>>>> read the
>>>>>>>>>>>>>>>>> integer
>>>>>>>>>>>>>>>>> number of hours (any number of digits) and handle them
>>>>>>>>>>>>>>>>> correctly.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To make sure, I ran PCP-Combine on some that that included
>>>>>>>>>>>>>>>>> accumulations intervals> 100 hours. All three
>>>>>>>>>>>>>>>>> commands
>>>>>>>>>>>>>>>>> (sum, add,
>>>>>>>>>>>>>>>>> and subtract) worked fine even when the accumulation
>>>>>>>>>>>>>>>>> interval on
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> command line was> 100.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please let me know if anything else comes up.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello again John,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regarding the> 99-h forecast lead time problem
>>>>>>>>>>>>>>>>>> from a
>>>>>>>>>>>>>>>>>> couple of
>>>>>>>>>>>>>>>>>> weeks
>>>>>>>>>>>>>>>>>> ago... I believe that PCP_COMBINE will also need to be
>>>>>>>>>>>>>>>>>> modified
>>>>>>>>>>>>>>>>>> since
>>>>>>>>>>>>>>>>>> there is an argument to the code that is of the format HH
>>>>>>>>>>>>>>>>>> (for the
>>>>>>>>>>>>>>>>>> 'sum'
>>>>>>>>>>>>>>>>>> option: out_accum).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please let me know if this is actually the case.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> John Henderson
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 12/23/09 5:46 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I checked and found out that this really is a problem
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> STAT-Analysis and MODE-Analysis tools. When I ran
>>>>>>>>>>>>>>>>>>> STAT-Analysis
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> specified a lead time of "-fcst_lead 114", it errored
>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>> with an
>>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>> message about not being able to parse that time.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So I modified the library code which parses these
>>>>>>>>>>>>>>>>>>> strings to
>>>>>>>>>>>>>>>>>>> allow
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> lead hours to be either 2 or 3 digits long.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I've posted the fix to:
>>>>>>>>>>>>>>>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv2.0/index.php
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks for pointing out this problem, and please let me
>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>> if you
>>>>>>>>>>>>>>>>>>> uncover any more issues.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks for the clarification of everything.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'll let you know if I uncover any other problems with
>>>>>>>>>>>>>>>>>>>> triple-digit
>>>>>>>>>>>>>>>>>>>> lead
>>>>>>>>>>>>>>>>>>>> times, but, otherwise, I'll await your fix.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks for the quick responses!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 12/23/09 4:37 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Regarding WPP, I really don't think that you'll have a
>>>>>>>>>>>>>>>>>>>>> problem.
>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>> currently working on a project with folks from NOAA
>>>>>>>>>>>>>>>>>>>>> where
>>>>>>>>>>>>>>>>>>>>> we're
>>>>>>>>>>>>>>>>>>>>> creating
>>>>>>>>>>>>>>>>>>>>> forecasts out to 114 hours and are running the most
>>>>>>>>>>>>>>>>>>>>> recently
>>>>>>>>>>>>>>>>>>>>> released
>>>>>>>>>>>>>>>>>>>>> version of WPP just fine. You would only potentially
>>>>>>>>>>>>>>>>>>>>> have an
>>>>>>>>>>>>>>>>>>>>> issue if
>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>> were running your forecasts out> 255 hours.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Regarding issues with MET, you're probably correct. I
>>>>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>>>>> come up in your use of the MODE-Analysis and
>>>>>>>>>>>>>>>>>>>>> STAT-Analysis
>>>>>>>>>>>>>>>>>>>>> tools.
>>>>>>>>>>>>>>>>>>>>> That's
>>>>>>>>>>>>>>>>>>>>> the place you'd be specifying the forecast lead
>>>>>>>>>>>>>>>>>>>>> time in
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> config
>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>> on the command line. I believe it would
>>>>>>>>>>>>>>>>>>>>> be a
>>>>>>>>>>>>>>>>>>>>> pretty
>>>>>>>>>>>>>>>>>>>>> easy fix
>>>>>>>>>>>>>>>>>>>>> involving
>>>>>>>>>>>>>>>>>>>>> changes to the "timestring_to_sec" routine in the file
>>>>>>>>>>>>>>>>>>>>> METv2.0/lib/vx_cal/time_strings.cc.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> However, before making any changes, I'd like to do
>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>> testing to
>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>> sure it really is a problem. And once I verify that,
>>>>>>>>>>>>>>>>>>>>> I'll work
>>>>>>>>>>>>>>>>>>>>> on a
>>>>>>>>>>>>>>>>>>>>> fix.
>>>>>>>>>>>>>>>>>>>>> If I isolate the problem and come up with a fix, I'd
>>>>>>>>>>>>>>>>>>>>> post
>>>>>>>>>>>>>>>>>>>>> it to
>>>>>>>>>>>>>>>>>>>>> the MET
>>>>>>>>>>>>>>>>>>>>> website and update the package of bug fixes.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'll let you know how it pans out. Please let me know
>>>>>>>>>>>>>>>>>>>>> if, in
>>>>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>> MET, you find another place where it's a problem to
>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> forecast
>>>>>>>>>>>>>>>>>>>>> hours>
>>>>>>>>>>>>>>>>>>>>> 99.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hello John,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks for the prompt response. Perhaps I had briefly
>>>>>>>>>>>>>>>>>>>>>> come
>>>>>>>>>>>>>>>>>>>>>> across
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> WPP email and thought it was actually related to
>>>>>>>>>>>>>>>>>>>>>> MET...
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I believe - according to the user manual - that MET
>>>>>>>>>>>>>>>>>>>>>> allows the
>>>>>>>>>>>>>>>>>>>>>> user to
>>>>>>>>>>>>>>>>>>>>>> specify forecast lead time as HH[MMSS], which
>>>>>>>>>>>>>>>>>>>>>> seems to
>>>>>>>>>>>>>>>>>>>>>> me to
>>>>>>>>>>>>>>>>>>>>>> prevent
>>>>>>>>>>>>>>>>>>>>>> leads times> 99h. Am I mistaken?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Regarding WPP...I intend to verify precip
>>>>>>>>>>>>>>>>>>>>>> accumulation
>>>>>>>>>>>>>>>>>>>>>> in MET
>>>>>>>>>>>>>>>>>>>>>> for a
>>>>>>>>>>>>>>>>>>>>>> year-long series of 5.5-day forecasts that overlap
>>>>>>>>>>>>>>>>>>>>>> at the
>>>>>>>>>>>>>>>>>>>>>> beginning of
>>>>>>>>>>>>>>>>>>>>>> each forecast run to avoid spin-up problems. It's a
>>>>>>>>>>>>>>>>>>>>>> pretty
>>>>>>>>>>>>>>>>>>>>>> standard
>>>>>>>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>>>>>> of generating year-long WRF simulations. However,
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> means
>>>>>>>>>>>>>>>>>>>>>> that I
>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>> be applying WPP to WRF forecasts with forecast lead
>>>>>>>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>>>>>>> of 12
>>>>>>>>>>>>>>>>>>>>>> h to
>>>>>>>>>>>>>>>>>>>>>> 132
>>>>>>>>>>>>>>>>>>>>>> h. I hadn't anticipated any problems with WPP,
>>>>>>>>>>>>>>>>>>>>>> unfortunately. I
>>>>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>> vague memory that someone had come up with a fix for
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> long-lead
>>>>>>>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>>>>>> problem, but I'm not sure. Would you be able to
>>>>>>>>>>>>>>>>>>>>>> investigate
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> status
>>>>>>>>>>>>>>>>>>>>>> for me?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> John
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On 12/23/09 2:51 PM, John Halley Gotway wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> John,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I looked back through the old MET-Help emails and
>>>>>>>>>>>>>>>>>>>>>>> couldn't
>>>>>>>>>>>>>>>>>>>>>>> find
>>>>>>>>>>>>>>>>>>>>>>> anything
>>>>>>>>>>>>>>>>>>>>>>> that would explain a problem of using fcst_lead>=
>>>>>>>>>>>>>>>>>>>>>>> 100
>>>>>>>>>>>>>>>>>>>>>>> hours. So
>>>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> aware of a problem with that. I did find one email
>>>>>>>>>>>>>>>>>>>>>>> related to this but I believe that was due to an
>>>>>>>>>>>>>>>>>>>>>>> incorrectly
>>>>>>>>>>>>>>>>>>>>>>> packed
>>>>>>>>>>>>>>>>>>>>>>> GRIB
>>>>>>>>>>>>>>>>>>>>>>> file, not a problem in MET itself.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> However, there are some issues in WPP
>>>>>>>>>>>>>>>>>>>>>>> (WRF-PostProcessor) for
>>>>>>>>>>>>>>>>>>>>>>> accumulated precip when the forecast lead time
>>>>>>>>>>>>>>>>>>>>>>> extends
>>>>>>>>>>>>>>>>>>>>>>> beyond 255
>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>> The time value of 255+ overflows the
>>>>>>>>>>>>>>>>>>>>>>> single
>>>>>>>>>>>>>>>>>>>>>>> byte
>>>>>>>>>>>>>>>>>>>>>>> that's
>>>>>>>>>>>>>>>>>>>>>>> allocated in the GRIB record to store that time
>>>>>>>>>>>>>>>>>>>>>>> info.
>>>>>>>>>>>>>>>>>>>>>>> This is
>>>>>>>>>>>>>>>>>>>>>>> mostly
>>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>>> issue for accumulated precip since you have to
>>>>>>>>>>>>>>>>>>>>>>> store 2
>>>>>>>>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>>>>>>>> in the
>>>>>>>>>>>>>>>>>>>>>>> GRIB
>>>>>>>>>>>>>>>>>>>>>>> record - accumulation starting and ending times.
>>>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>> not sure of the status of this issue, but if you run
>>>>>>>>>>>>>>>>>>>>>>> across it,
>>>>>>>>>>>>>>>>>>>>>>> I can
>>>>>>>>>>>>>>>>>>>>>>> look into it more.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'd say, proceed with trying to use MET to verify
>>>>>>>>>>>>>>>>>>>>>>> forecasts>= 100
>>>>>>>>>>>>>>>>>>>>>>> hours.
>>>>>>>>>>>>>>>>>>>>>>> And if you run into any problems, just
>>>>>>>>>>>>>>>>>>>>>>> let
>>>>>>>>>>>>>>>>>>>>>>> us know
>>>>>>>>>>>>>>>>>>>>>>> through
>>>>>>>>>>>>>>>>>>>>>>> met_help at ucar.edu.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> If you haven't already done so, I would suggest
>>>>>>>>>>>>>>>>>>>>>>> retrieving
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> latest
>>>>>>>>>>>>>>>>>>>>>>> set of bug fixes for METv2.0 from:
>>>>>>>>>>>>>>>>>>>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv2.0/index.php
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hope that helps,
>>>>>>>>>>>>>>>>>>>>>>> John Halley Gotway
>>>>>>>>>>>>>>>>>>>>>>> johnhg at ucar.edu
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> John Henderson wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> When first I installed WRF-MET I believe I saw a
>>>>>>>>>>>>>>>>>>>>>>>> discussion
>>>>>>>>>>>>>>>>>>>>>>>> somewhere
>>>>>>>>>>>>>>>>>>>>>>>> about how to apply WRF-MET to forecasts that
>>>>>>>>>>>>>>>>>>>>>>>> require a
>>>>>>>>>>>>>>>>>>>>>>>> fcst_lead of
>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>> least 100 h, however, I can't seem to find it now.
>>>>>>>>>>>>>>>>>>>>>>>> Can
>>>>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>>>> point me
>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> the right direction?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> John Henderson
>>>>>>>>>>>>>>>>>>>>>>>> AER, Inc.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>>>>> Met_help mailing list
>>>>>>>>>>>>>>>>>>>>>>>> Met_help at mailman.ucar.edu
>>>>>>>>>>>>>>>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>
>>>>>
More information about the Met_help
mailing list