[Met_help] point stat & ascii2nc
John Halley Gotway
johnhg at rap.ucar.edu
Tue Sep 16 16:10:53 MDT 2008
Leela,
I tweaked the ASCII2NC code to keep track of the header values and only write out a new header record when the values actually change. I've attached the modified version of the file
METv1.1/src/ascii2nc/ascii2nc.cc. It should fix the incompatibility problem you found - without having to comment out that line in Point-Stat.
If you have time to test it out, I'd really appreciate hearing how it goes for you. It worked on my test case, but it's always good to have a few people test changes out.
Thanks!
John
John Halley Gotway wrote:
> Leela,
>
> OK I see what's going on, and it is a bug. The ASCII2NC tool is creating observations that are a little inconsistent with the format that Point-Stat is expecting. For a quick fix, open up the file
> METv1.1/src/point_stat/point_stat.cc, modify the code around line number 1500 as follows, and recompile:
>
> Replace this:
> if(obs_arr[0] != prev_obs_arr[0] ||
> obs_arr[2] != prev_obs_arr[2] ||
> obs_arr[3] != prev_obs_arr[3]) {
>
> With this:
> if(// obs_arr[0] != prev_obs_arr[0] || <- Quick fix to handle UGRD and VGRD obs from ASCII2NC
> obs_arr[2] != prev_obs_arr[2] ||
> obs_arr[3] != prev_obs_arr[3]) {
>
> So we're just commenting out that first check. And here's why... The first entry in "obs_arr" contains the index for the header data for this observation. Currently, we're checking to see that both
> the UGRD and VGRD observations point to the same header data to ensure that they're from the same observing location. However, ASCII2NC just writes out new header data for each observation - so those
> header id's don't match. For a real fix, I could either modify ASCII2NC to be a little smarter or modify Point-Stat to check the actual header values themselves rather than just their indices.
>
> I think that modifying Point-Stat would make it run slower, so I'm leaning toward making ASCII2NC a little smarter. But I should think about it more. In the meantime, the quick fix should work fine.
> Thanks for finding this!
>
> Regarding deriving wind speed, MET should be doing that now! The GRIB abbreviation for wind speed is simply "WIND". When you request that "WIND" be verified, MET will look for that field in the GRIB
> file. If it's not there, it'll look for UGRD and VGRD and combine those two fields to generate the wind speed field. Now, if you're using Point-Stat to compare the model data to point observations
> from ASCII2NC, your ASCII2NC output will need to have observations of wind speed already computed for the comparison. In PB2NC, you're able to derive quantities like wind speed from the raw
> observations, but we wrote ASCII2NC to be a more simple reformatting tool. We figured that users would want to derive observation values themselves using their preferred method.
>
> So as long as your ASCII observations contain wind speed, MET will derive wind speed from the UGRD and VGRD fields and verify it.
>
> Wind direction is another matter - we're not currently verifying it in MET. We're still working to define the method for how it should be verified.
>
> Hope that helps!
> John
>
> Watson.Leela wrote:
>> John,
>>
>> I have another question about the format of the ASCII text files. I now
>> would like to verify U and V winds and have reformatted my observations
>> as such:
>>
>> ADPSFC 44 20080130_000000 34.8998 -117.8901 705 33 101300 714 8.80
>> ADPSFC 44 20080130_000000 34.8998 -117.8901 705 34 101300 714 -9.57
>> ADPSFC 150 20080130_000000 34.9780 -117.8703 705 33 101300 714 -15.99
>> ADPSFC 150 20080130_000000 34.9780 -117.8703 705 34 101300 714 9.48
>> etc...
>>
>> I am getting the following error message when I try to run ascii2nc:
>>
>> ERROR: process_obs_file() -> when computing vector winds, each UGRD
>> observation must be followed by a VGRD observation with the same header
>> and at the same level
>>
>> What exactly does this mean (how must the ASCII file be formatted for
>> vector winds)?
>>
>> On another note, is there any way MET can derive wind speed from U and V
>> winds in the WRF grib file?
>>
>> Leela
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>> Sent: Tuesday, September 09, 2008 5:24 PM
>> To: Watson.Leela; met_help
>> Subject: Re: [Met_help] point stat & ascii2nc
>>
>> Leela,
>>
>> Glad it worked. We'll make a note to include it in the next version of
>> the user's guide.
>>
>> Feel free to write with any more issues or questions.
>>
>> Thanks,
>>
>> John
>>
>> Watson.Leela wrote:
>>> Thanks, John - it worked.
>>>
>>> I think that performing the matching by message type is sufficient
>>> enough. The only thing that I would request is that the user's guide
>>> mention exactly what you stated in your email below - how the matching
>>> is performed and what the message type should be for comparing surface
>>> obs to forecasts at the surface.
>>>
>>> Leela
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>> Sent: Tuesday, September 09, 2008 4:41 PM
>>> To: Watson.Leela
>>> Cc: met_help
>>> Subject: Re: [Met_help] point stat & ascii2nc
>>>
>>> Leela,
>>>
>>> Actually, after looking at your ASCII observations, I have a guess as
>> to
>>> what's going on. So you probably don't need to send the GRIB file
>>> anyway.
>>>
>>> Even though this may not be strictly correct, in your ASCII
>>> observations, please try replacing "MSONET" with "ADPSFC" and
>> rerunning
>>> ASCII2NC and Point-Stat. I'm guessing that will fix the problem.
>>>
>>> Here's what I believe is going on... In Point-Stat, observations are
>>> typically matched to forecast fields based on pressure level. So for
>>> example, if you're verifying temperature between 800mb and
>>> 500mb, it'll look at the pressure level of each observation and match
>>> them to the forecast if they fall in that range. However, that
>> doesn't
>>> work for matching observations to forecasts at the
>>> surface. So in MET, we adopted the method that NCEP's chosen which is
>>> to simply match up based on message type. When verifying fields at
>> the
>>> surface (2m temp or 10m winds), Point-Stat only matches
>>> "surface" observations of type ADPSFC or SFCSHP. Your MSONET
>>> observations aren't being matched because they're not being included
>> as
>>> a surface observation. So all observations of type ADPSFC or
>>> SFCSHP are assumed to be at the surface and are matched to forecasts
>> at
>>> the surface.
>>>
>>> As a user, I'm wondering, what's your opinion? How would you like to
>>> see the matching performed for surface observations? Is doing it by
>>> message type sufficient, or would you like to see us do
>>> something more sophisticated like comparing the elevation of the
>>> observation to the elevation at that lat/lon point?
>>>
>>> Thanks,
>>> John
>>>
>>>
>>> Watson.Leela wrote:
>>>> John,
>>>>
>>>> I cannot send the ARW GRIB file - I keep getting an email back saying
>>>> 'Message Refused: Exceeded Message Size Threshold'. Is there some
>>> other
>>>> way that I can get the GRIB file to you? The file size is 16MB. I can
>>>> zip it, but it is still large - 13MB.
>>>>
>>>> Leela
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>>> Sent: Tuesday, September 09, 2008 2:41 PM
>>>> To: Watson.Leela
>>>> Cc: met_help at ucar.edu
>>>> Subject: Re: [Met_help] point stat & ascii2nc
>>>>
>>>> Leela,
>>>>
>>>> It would probably be easiest if you could send me the sample data
>> that
>>>> you're using. I'll try running it here to see if I can figure out
>>>> what's going on.
>>>>
>>>> Please send me the following:
>>>> (1) The ASCII input file to the ASCII2NC tool.
>>>> (2) Your NetCDF output file from ASCII2NC.
>>>> (3) The forecast file containing the 2m temp field from ARW.
>>>> (4) The configuration file that you're using when running the
>>> Point-Stat
>>>> tool.
>>>>
>>>> Thanks,
>>>> John Halley-Gotway
>>>> johnhg at ucar.edu
>>>>
>>>> Watson.Leela wrote:
>>>>> I am currently trying to use MET to verify 2m temperatures from ARW
>>>>> forecasts with local observation data available in ASCII format. I
>>>>> reformatted my observations to fit the format identified in the MET
>>>>> documentation for the ASCII2NC tool. My observations cover a domain
>>> of
>>>>> approximately 20km x 20km and there are only 12 of them. The grid
>>>>> spacing of the ARW output is 1km. When I run the point_stat tool I
>>>>> receive the following message indicating that there are no matching
>>>>> pairs:
>>>>>
>>>>>
>>>>>
>>>>> Grib Record Index = 11
>>>>>
>>>>> Initialization time = 1201683600
>>>>>
>>>>> Valid time = 1201683600
>>>>>
>>>>> Accumulation time = 0
>>>>>
>>>>> Grib Record (min, max) = (229.37726, 291.74725)
>>>>>
>>>>>
>>>>>
>>>>> ----------------------------------------
>>>>>
>>>>> Processing 12 observations from 12 PrepBufr messages...
>>>>>
>>>>> For Grib Field TMP at Z2 for observation type MSONET, over region
>>>> FULL,
>>>>> for interpolation method DW_MEAN(3), found 0 pairs.
>>>>>
>>>>>
>>>>>
>>>>> I also noticed that when I ran the point_stat test script using the
>>>>> sample forecast and ascii data, it also produced similar errors for
>>>> the
>>>>> 10m UGRD and VGRD values:
>>>>>
>>>>>
>>>>>
>>>>> For Grib Field UGRD at Z10 for observation type ADPUPA, over region
>>>>> DTC165, for interpolation method UW_MEAN(1), found 0 pairs.
>>>>>
>>>>> Etc...
>>>>>
>>>>>
>>>>>
>>>>> I am confused as to how there can be no matching pairs when the
>>>>> observation point MUST fall between 4 grid points...?? Or maybe I am
>>>>> misunderstanding what the output message is saying. Any help would
>> be
>>>>> appreciated!
>>>>>
>>>>>
>>>>>
>>>>> Leela Watson
>>>>>
>>>>>
>>>>>
>>>>> *********************************************************
>>>>>
>>>>> Leela R. Watson, Meteorologist
>>>>>
>>>>> ENSCO, Inc. / Applied Meteorology Unit
>>>>>
>>>>> 1980 N. Atlantic Ave., Suite 230
>>>>>
>>>>> Cocoa Beach, FL 32931
>>>>>
>>>>> Voice: (321) 853-8264
>>>>>
>>>>> Fax: (321) 853-8415
>>>>>
>>>>> Email: watson.leela at ensco.com
>>>>>
>>>>>
>>>>>
>>>>>
>>> ......................................................................
>>>>> The information contained in this email message is intended only for
>>>> the use of the individuals to whom it is
>>>>> addressed and may contain information that is privileged and
>>>> sensitive. If the reader of this message is not
>>>>> the intended recipient, you are hereby notified that any
>>>> dissemination, distribution or copying of this
>>>>> communication is strictly prohibited. If you have received this
>>>> communication in error, please notify the
>>>>> sender immediately by email at the above referenced address. Thank
>>>> you.
>> ------------------------------------------------------------------------
>>>>> _______________________________________________
>>>>> Met_help mailing list
>>>>> Met_help at mailman.ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
> _______________________________________________
> Met_help mailing list
> Met_help at mailman.ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/met_help
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ascii2nc.cc
Type: text/x-c++src
Size: 17347 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20080916/9c6bf44c/ascii2nc-0001.bin
More information about the Met_help
mailing list