[Met_help] point stat & ascii2nc
John Halley Gotway
johnhg at rap.ucar.edu
Wed Sep 17 10:18:10 MDT 2008
Leela,
Great. Thanks for checking that. FYI, I've added this fix to the "known issues" page on the MET website: http://www.dtcenter.org/met/users/support/known_issues/METv1.1/index.php
It's the last item listed on that page. Does that description of the problem and solution look sufficient to you?
Thanks,
John
Watson.Leela wrote:
> John,
>
> I replaced the old ascii2nc.cc with the new one, recompiled the
> software, and tried to run the program again and it worked. Thanks for
> the help!
>
> Leela
>
> -----Original Message-----
> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
> Sent: Tuesday, September 16, 2008 6:11 PM
> To: Watson.Leela; met_help
> Subject: Re: [Met_help] point stat & ascii2nc
>
> Leela,
>
> I tweaked the ASCII2NC code to keep track of the header values and only
> write out a new header record when the values actually change. I've
> attached the modified version of the file
> METv1.1/src/ascii2nc/ascii2nc.cc. It should fix the incompatibility
> problem you found - without having to comment out that line in
> Point-Stat.
>
> If you have time to test it out, I'd really appreciate hearing how it
> goes for you. It worked on my test case, but it's always good to have a
> few people test changes out.
>
> Thanks!
> John
>
> John Halley Gotway wrote:
>> Leela,
>>
>> OK I see what's going on, and it is a bug. The ASCII2NC tool is
>> creating observations that are a little inconsistent with the format
> that Point-Stat is expecting. For a quick fix, open up the file
> METv1.1/src/point_stat/point_stat.cc, modify the code around line number
> 1500 as follows, and recompile:
>> Replace this:
>> if(obs_arr[0] != prev_obs_arr[0] ||
>> obs_arr[2] != prev_obs_arr[2] ||
>> obs_arr[3] != prev_obs_arr[3]) {
>>
>> With this:
>> if(// obs_arr[0] != prev_obs_arr[0] || <- Quick fix to handle
> UGRD and VGRD obs from ASCII2NC
>> obs_arr[2] != prev_obs_arr[2] ||
>> obs_arr[3] != prev_obs_arr[3]) {
>>
>> So we're just commenting out that first check. And here's why... The
>
>> first entry in "obs_arr" contains the index for the header data for
>> this observation. Currently, we're checking to see that both the UGRD
> and VGRD observations point to the same header data to ensure that
> they're from the same observing location. However, ASCII2NC just writes
> out new header data for each observation - so those header id's don't
> match. For a real fix, I could either modify ASCII2NC to be a little
> smarter or modify Point-Stat to check the actual header values
> themselves rather than just their indices.
>> I think that modifying Point-Stat would make it run slower, so I'm
> leaning toward making ASCII2NC a little smarter. But I should think
> about it more. In the meantime, the quick fix should work fine.
>> Thanks for finding this!
>>
>> Regarding deriving wind speed, MET should be doing that now! The GRIB
>
>> abbreviation for wind speed is simply "WIND". When you request that
>> "WIND" be verified, MET will look for that field in the GRIB file. If
>
>> it's not there, it'll look for UGRD and VGRD and combine those two
> fields to generate the wind speed field. Now, if you're using
> Point-Stat to compare the model data to point observations from
> ASCII2NC, your ASCII2NC output will need to have observations of wind
> speed already computed for the comparison. In PB2NC, you're able to
> derive quantities like wind speed from the raw observations, but we
> wrote ASCII2NC to be a more simple reformatting tool. We figured that
> users would want to derive observation values themselves using their
> preferred method.
>> So as long as your ASCII observations contain wind speed, MET will
> derive wind speed from the UGRD and VGRD fields and verify it.
>> Wind direction is another matter - we're not currently verifying it in
> MET. We're still working to define the method for how it should be
> verified.
>> Hope that helps!
>> John
>>
>> Watson.Leela wrote:
>>> John,
>>>
>>> I have another question about the format of the ASCII text files. I
>>> now would like to verify U and V winds and have reformatted my
>>> observations as such:
>>>
>>> ADPSFC 44 20080130_000000 34.8998 -117.8901 705 33 101300 714
> 8.80
>>> ADPSFC 44 20080130_000000 34.8998 -117.8901 705 34 101300 714
> -9.57
>>> ADPSFC 150 20080130_000000 34.9780 -117.8703 705 33 101300 714
> -15.99
>>> ADPSFC 150 20080130_000000 34.9780 -117.8703 705 34 101300 714
> 9.48
>>> etc...
>>>
>>> I am getting the following error message when I try to run ascii2nc:
>>>
>>> ERROR: process_obs_file() -> when computing vector winds, each UGRD
>>> observation must be followed by a VGRD observation with the same
>>> header and at the same level
>>>
>>> What exactly does this mean (how must the ASCII file be formatted for
>
>>> vector winds)?
>>>
>>> On another note, is there any way MET can derive wind speed from U
>>> and V winds in the WRF grib file?
>>>
>>> Leela
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>> Sent: Tuesday, September 09, 2008 5:24 PM
>>> To: Watson.Leela; met_help
>>> Subject: Re: [Met_help] point stat & ascii2nc
>>>
>>> Leela,
>>>
>>> Glad it worked. We'll make a note to include it in the next version
>>> of the user's guide.
>>>
>>> Feel free to write with any more issues or questions.
>>>
>>> Thanks,
>>>
>>> John
>>>
>>> Watson.Leela wrote:
>>>> Thanks, John - it worked.
>>>>
>>>> I think that performing the matching by message type is sufficient
>>>> enough. The only thing that I would request is that the user's guide
>
>>>> mention exactly what you stated in your email below - how the
>>>> matching is performed and what the message type should be for
>>>> comparing surface obs to forecasts at the surface.
>>>>
>>>> Leela
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>>> Sent: Tuesday, September 09, 2008 4:41 PM
>>>> To: Watson.Leela
>>>> Cc: met_help
>>>> Subject: Re: [Met_help] point stat & ascii2nc
>>>>
>>>> Leela,
>>>>
>>>> Actually, after looking at your ASCII observations, I have a guess
>>>> as
>>> to
>>>> what's going on. So you probably don't need to send the GRIB file
>>>> anyway.
>>>>
>>>> Even though this may not be strictly correct, in your ASCII
>>>> observations, please try replacing "MSONET" with "ADPSFC" and
>>> rerunning
>>>> ASCII2NC and Point-Stat. I'm guessing that will fix the problem.
>>>>
>>>> Here's what I believe is going on... In Point-Stat, observations
>>>> are typically matched to forecast fields based on pressure level.
>>>> So for example, if you're verifying temperature between 800mb and
>>>> 500mb, it'll look at the pressure level of each observation and
>>>> match them to the forecast if they fall in that range. However,
>>>> that
>>> doesn't
>>>> work for matching observations to forecasts at the surface. So in
>>>> MET, we adopted the method that NCEP's chosen which is to simply
>>>> match up based on message type. When verifying fields at
>>> the
>>>> surface (2m temp or 10m winds), Point-Stat only matches "surface"
>>>> observations of type ADPSFC or SFCSHP. Your MSONET observations
>>>> aren't being matched because they're not being included
>>> as
>>>> a surface observation. So all observations of type ADPSFC or SFCSHP
>
>>>> are assumed to be at the surface and are matched to forecasts
>>> at
>>>> the surface.
>>>>
>>>> As a user, I'm wondering, what's your opinion? How would you like
>>>> to see the matching performed for surface observations? Is doing it
>
>>>> by message type sufficient, or would you like to see us do something
>
>>>> more sophisticated like comparing the elevation of the observation
>>>> to the elevation at that lat/lon point?
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>>
>>>> Watson.Leela wrote:
>>>>> John,
>>>>>
>>>>> I cannot send the ARW GRIB file - I keep getting an email back
>>>>> saying 'Message Refused: Exceeded Message Size Threshold'. Is there
>
>>>>> some
>>>> other
>>>>> way that I can get the GRIB file to you? The file size is 16MB. I
>>>>> can zip it, but it is still large - 13MB.
>>>>>
>>>>> Leela
>>>>>
>>>>> -----Original Message-----
>>>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>>>> Sent: Tuesday, September 09, 2008 2:41 PM
>>>>> To: Watson.Leela
>>>>> Cc: met_help at ucar.edu
>>>>> Subject: Re: [Met_help] point stat & ascii2nc
>>>>>
>>>>> Leela,
>>>>>
>>>>> It would probably be easiest if you could send me the sample data
>>> that
>>>>> you're using. I'll try running it here to see if I can figure out
>>>>> what's going on.
>>>>>
>>>>> Please send me the following:
>>>>> (1) The ASCII input file to the ASCII2NC tool.
>>>>> (2) Your NetCDF output file from ASCII2NC.
>>>>> (3) The forecast file containing the 2m temp field from ARW.
>>>>> (4) The configuration file that you're using when running the
>>>> Point-Stat
>>>>> tool.
>>>>>
>>>>> Thanks,
>>>>> John Halley-Gotway
>>>>> johnhg at ucar.edu
>>>>>
>>>>> Watson.Leela wrote:
>>>>>> I am currently trying to use MET to verify 2m temperatures from
>>>>>> ARW forecasts with local observation data available in ASCII
>>>>>> format. I reformatted my observations to fit the format identified
>
>>>>>> in the MET documentation for the ASCII2NC tool. My observations
>>>>>> cover a domain
>>>> of
>>>>>> approximately 20km x 20km and there are only 12 of them. The grid
>>>>>> spacing of the ARW output is 1km. When I run the point_stat tool I
>
>>>>>> receive the following message indicating that there are no
>>>>>> matching
>>>>>> pairs:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Grib Record Index = 11
>>>>>>
>>>>>> Initialization time = 1201683600
>>>>>>
>>>>>> Valid time = 1201683600
>>>>>>
>>>>>> Accumulation time = 0
>>>>>>
>>>>>> Grib Record (min, max) = (229.37726, 291.74725)
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----------------------------------------
>>>>>>
>>>>>> Processing 12 observations from 12 PrepBufr messages...
>>>>>>
>>>>>> For Grib Field TMP at Z2 for observation type MSONET, over region
>>>>> FULL,
>>>>>> for interpolation method DW_MEAN(3), found 0 pairs.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I also noticed that when I ran the point_stat test script using
>>>>>> the sample forecast and ascii data, it also produced similar
>>>>>> errors for
>>>>> the
>>>>>> 10m UGRD and VGRD values:
>>>>>>
>>>>>>
>>>>>>
>>>>>> For Grib Field UGRD at Z10 for observation type ADPUPA, over
>>>>>> region DTC165, for interpolation method UW_MEAN(1), found 0 pairs.
>>>>>>
>>>>>> Etc...
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am confused as to how there can be no matching pairs when the
>>>>>> observation point MUST fall between 4 grid points...?? Or maybe I
>>>>>> am misunderstanding what the output message is saying. Any help
>>>>>> would
>>> be
>>>>>> appreciated!
>>>>>>
>>>>>>
>>>>>>
>>>>>> Leela Watson
>>>>>>
>>>>>>
>>>>>>
>>>>>> *********************************************************
>>>>>>
>>>>>> Leela R. Watson, Meteorologist
>>>>>>
>>>>>> ENSCO, Inc. / Applied Meteorology Unit
>>>>>>
>>>>>> 1980 N. Atlantic Ave., Suite 230
>>>>>>
>>>>>> Cocoa Beach, FL 32931
>>>>>>
>>>>>> Voice: (321) 853-8264
>>>>>>
>>>>>> Fax: (321) 853-8415
>>>>>>
>>>>>> Email: watson.leela at ensco.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
> ......................................................................
>>>>>> The information contained in this email message is intended only
>>>>>> for
>>>>> the use of the individuals to whom it is
>>>>>> addressed and may contain information that is privileged and
>>>>> sensitive. If the reader of this message is not
>>>>>> the intended recipient, you are hereby notified that any
>>>>> dissemination, distribution or copying of this
>>>>>> communication is strictly prohibited. If you have received this
>>>>> communication in error, please notify the
>>>>>> sender immediately by email at the above referenced address. Thank
>>>>> you.
>>> ---------------------------------------------------------------------
>>> ---
>>>>>> _______________________________________________
>>>>>> Met_help mailing list
>>>>>> Met_help at mailman.ucar.edu
>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>> _______________________________________________
>> Met_help mailing list
>> Met_help at mailman.ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/met_help
>
More information about the Met_help
mailing list