[Met_help] Point-Stat Tool --- Point Observation Component Issue

John Halley Gotway johnhg at rap.ucar.edu
Tue Jul 21 08:32:18 MDT 2009


Waylon,

Thanks for sending me all that data.  It made debugging the problem very easy.  Sorry about this, but I had my counts off by 1.  I was indexing the header id's from 1 to N, when I should have been
doing it from 0 to N-1.  This amounts to a 1 line change in the "awips2nc.cc" file.  I've attached the updated version.  Please replace METv2.0/src/awips2nc/awips2nc.cc with the attached version,
recompile, and try running AWIPS2NC and Point-Stat again.

Thanks,
John

Waylon.Collins wrote:
> John,
> 
> After compiling using the PGI Makefile, and after installing
> METv2.0_patches_20090701.tar from the MET website, I ran awips2nc
> without error. However, when I ran the Point Stat Tool as a test, I
> encountered the attached error. I uploaded the associated files to the
> /incoming/irap/met_help/collins_data subdirectory you prepared for me
> earlier on ftp.rap.ucar.edu server. The file 20090417_0600 and
> 20090417_0600.out are the input and output to the awips2nc program,
> respectively. the *arw_wrfout* GRIB1 file is the forecast file. The
> PointStatConfig_ARW file is the configuration file for Point Stat.
> 
> Waylon
> 
> John Halley Gotway wrote:
>> Waylon,
>>
>> The updated PGI Makefile is attached.  It includes additions in 4
>> spots to include the building of the AWIPS2NC tool.
>>
>> Hope it all goes well.
>>
>> John
>>
>> Waylon.Collins wrote:
>>  
>>> Hello John,
>>>
>>> The Fortran Compiler that I use is the from the Portland Group (PGI),
>>> although I do have gfortran installed on my computer (that runs the
>>> Redhat Enterprise Linux v4 O/S.) I created both PGI (16 June 2009) and
>>> gfortran (tonight) versions of both netCDF and libbufr.a and tried to
>>> install MET again tonight using your new Makefile_gnu file and the
>>> gfortran compiled versions of netCDF and BUFRLIB. I am having
>>> difficulty, although it appears to be minor. Nevertheless, could you
>>> prepare a Makefile_pgi file for the building of MET including awips2nc?
>>> Thanks again for your help.
>>>
>>> Waylon
>>>
>>> John Halley Gotway wrote:
>>>    
>>>> Waylon,
>>>>
>>>> I went ahead and just took a shot at it.  After talking to a
>>>> meteorologist here, I assumed that both report types, METAR and SPECI,
>>>> indicate surface observations.  I've attached some code for you to
>>>> try out.
>>>>
>>>> Please copy the attached file "METv2.0_awips2nc.20090714.tar.gz", into
>>>> the top-level METv2.0 directory.  Then uncompress and untar it:
>>>> gunzip METv2.0_awips2nc.20090714.tar.gz
>>>> tar -xvf METv2.0_awips2nc.20090714.tar
>>>>
>>>> This will create a new directory "METv2.0/src/awips2nc".  Also, it
>>>> will overwrite the file "Makefile_gnu" with an updated version with
>>>> commands for building the awips2nc tool.  I wasn't sure which
>>>> compilers you used to build MET... if it was something other than the
>>>> GNU compilers, just look at the Makefile changes I made, and make
>>>> similar ones for the version of the Makefile you're using.
>>>>
>>>> ... or let me know which compilers, and I can make the Makefile
>>>> changes for you.
>>>>
>>>> Then just rebuild MET by typing "make" in the METv2.0 directory.  That
>>>> should create the "METv2.0/bin/awips2nc" executable which you can use
>>>> to reformat your AWIPS point observation file.  Type
>>>> "METv2.0/bin/awips2nc" to see the usage statement.  It's pretty
>>>> straight-forward.
>>>>
>>>> Try it out and let me know how it goes.  Right now it's assuming all
>>>> of the point observations to be at the surface.  Just let me know if
>>>> we need to modify the logic to handle a wider variety of data
>>>> types you may encounter.
>>>>
>>>> Thanks,
>>>> John Halley Gotway
>>>> johnhg at ucar.edu
>>>>
>>>> John Halley Gotway wrote:
>>>>  
>>>>      
>>>>> Waylon,
>>>>>
>>>>> Thanks for checking in.  I am working on it, but have been slowed by
>>>>> needing to get ready for a MET tutorial we're giving next week.  I'll
>>>>> get to work on it though and will get in touch with you when
>>>>> I have something for you.  I hope to make significant progress this
>>>>> week.
>>>>>
>>>>> Thanks,
>>>>> John
>>>>>
>>>>> Waylon.Collins at noaa.gov wrote:
>>>>>           
>>>>>> Hello John,
>>>>>>
>>>>>> Just wondering if you made any progress developing software to
>>>>>> convert AWIPS METAR netCDF files to Point Stat tool compatible
>>>>>> netCDF files?
>>>>>>
>>>>>> Waylon
>>>>>> Waylon Collins (Senior Forecaster)
>>>>>> NOAA/National Weather Service
>>>>>> 300 Pinson Drive
>>>>>> Corpus Christi, TX 78406
>>>>>> (United States)
>>>>>> Telephone: (361) 289-0959   ----- Original Message -----
>>>>>> From: John Halley Gotway <johnhg at rap.ucar.edu>
>>>>>> Date: Thursday, June 25, 2009 1:43 pm
>>>>>> Subject: Re: [Met_help] Point-Stat Tool --- Point Observation
>>>>>> Component Issue
>>>>>> To: "Waylon.Collins" <Waylon.Collins at noaa.gov>, met_help
>>>>>> <met_help at ucar.edu>
>>>>>>
>>>>>>
>>>>>>               
>>>>>>> Waylon,
>>>>>>>
>>>>>>> Sounds good.  I'm busy this week with the WRF-User's Workshop.  But
>>>>>>> I'll take a look at these files next week.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> John
>>>>>>>
>>>>>>> Waylon.Collins wrote:
>>>>>>>                   
>>>>>>>> Hello John,
>>>>>>>>
>>>>>>>> Yes, lets pursue the strategy of developing a reformatting tool
>>>>>>>> that
>>>>>>>> would read an AWIPS netCDF METAR file and write a Point-Stat tool
>>>>>>>> compatible netCDF file. I uploaded three (3) METAR files to the
>>>>>>>> collins_data subdirectory on your FTP site. Go ahead and ignore
>>>>>>>> the                         
>>>>>>> QC
>>>>>>>                   
>>>>>>>> information and simply use the actual data values. The twelve (12)
>>>>>>>> observation types you mentioned below are sufficient. Thanks for
>>>>>>>>                         
>>>>>>> your help.
>>>>>>>                   
>>>>>>>> Waylon
>>>>>>>>
>>>>>>>> John Halley Gotway wrote:
>>>>>>>>                       
>>>>>>>>> Waylon,
>>>>>>>>>
>>>>>>>>> We do not currently have tools that will do either of the
>>>>>>>>> things you
>>>>>>>>> requested.  However, I took at look at the NetCDF header you
>>>>>>>>> sent,                             
>>>>>>> and
>>>>>>>                   
>>>>>>>>> the format looks pretty straight-forward.  I'd be happy to
>>>>>>>>> work with you on a reformatting tool that would read a NetCDF
>>>>>>>>> AWIPS
>>>>>>>>> file and write a NetCDF file for input into the Point-Stat
>>>>>>>>> tool.  I
>>>>>>>>> don't think it'd be too difficult.  But I'd need you to send me
>>>>>>>>> a few sample files for testing and answer some questions about the
>>>>>>>>> format.
>>>>>>>>>
>>>>>>>>> If you'd like to proceed with this, please send a few sample files
>>>>>>>>> either via email or by posting them on our anonymous FTP site:
>>>>>>>>> ftp ftp.rap.ucar.edu
>>>>>>>>> username = anonymous
>>>>>>>>> password = "your email address"
>>>>>>>>> cd incoming/irap/met_help/collins_data
>>>>>>>>> put "your sample files"
>>>>>>>>> bye
>>>>>>>>>
>>>>>>>>> Looking at the NetCDF header you sent, it looks like there's a
>>>>>>>>> lot                             
>>>>>>> of
>>>>>>>                   
>>>>>>>>> quality control information in the file.  The goal of this
>>>>>>>>> reformatting tool would be to extract the actual observation
>>>>>>>>> values
>>>>>>>>> you'd like to use in the verification.  Would it be reasonable
>>>>>>>>> initially to ignore the QC information and just use the actual
>>>>>>>>> data
>>>>>>>>> values?
>>>>>>>>>
>>>>>>>>> Here are the observation types I see that we would make sense to
>>>>>>>>> extract to use in verification.  Each is followed by the GRIB
>>>>>>>>> code                             
>>>>>>> to
>>>>>>>                   
>>>>>>>>> which it would correspond:
>>>>>>>>> (1)  Sea Level Pressure (GRIB Code 2)
>>>>>>>>> (2)  Temperature (GRIB Code 11)
>>>>>>>>> (3)  Maximum Temperature (GRIB Code 15)
>>>>>>>>> (4)  Minimum Temperature (GRIB Code 16)
>>>>>>>>> (5)  Dewpoint (GRIB Code 17)
>>>>>>>>> (6)  Wind Direction (GRIB Code 31)
>>>>>>>>> (7)  Wind Speed (GRIB Code 32)
>>>>>>>>> (8)  Wind Gust (GRIB Code 180)
>>>>>>>>> (9)  1-hourly precip (GRIB Code 61)
>>>>>>>>> (10) 3-hourly precip (GRIB Code 61)
>>>>>>>>> (11) 6-hourly precip (GRIB Code 61)
>>>>>>>>> (12) 24-hourly precip (GRIB Code 61)
>>>>>>>>>
>>>>>>>>> Does that set of observation types look reasonable to you?  Or
>>>>>>>>> am I
>>>>>>>>> missing something?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> John Halley Gotway
>>>>>>>>> johnhg at ucar.edu
>>>>>>>>>
>>>>>>>>> Waylon.Collins wrote:
>>>>>>>>>  
>>>>>>>>>                           
>>>>>>>>>> Dear Sir or Madam,
>>>>>>>>>>
>>>>>>>>>> My goal is to generate verification statistics using the
>>>>>>>>>>                                 
>>>>>>> Point-Stat Tool
>>>>>>>                   
>>>>>>>>>> by comparing point observations with WRF-ARW gridded forecast
>>>>>>>>>> output.
>>>>>>>>>> With respect to the Point-Stat Tool, I understand that the netCDF
>>>>>>>>>> obs_file should be created from pb2nc or ascii2nc. I do not have
>>>>>>>>>> access
>>>>>>>>>> to data in prepbufr format, thus ascii2nc would be used.
>>>>>>>>>> However,                                 
>>>>>>> I will
>>>>>>>                   
>>>>>>>>>> be preparing verification statistics based on 3 months of data,
>>>>>>>>>> every
>>>>>>>>>> 3-months, time resolution of 3 hours, for 5-10 observation
>>>>>>>>>> points.
>>>>>>>>>> Manually creating files to match the 10 column ASCII Point
>>>>>>>>>> Observation
>>>>>>>>>> Format as input into ascii2nc is not practical. I need an
>>>>>>>>>> efficient
>>>>>>>>>> method to create the obs_file as input into point_stat. I have
>>>>>>>>>>                                 
>>>>>>> access to
>>>>>>>                   
>>>>>>>>>> the observations in netCDF format used by AWIPS. Attached is the
>>>>>>>>>> output
>>>>>>>>>> from the linux command "ncdump -h" applied to one of these
>>>>>>>>>> files.                                 
>>>>>>> Do you
>>>>>>>                   
>>>>>>>>>> have software that will either (1) convert my AWIPS netCDF files
>>>>>>>>>> into
>>>>>>>>>> the netCDF flavor required by point_stat, or (2) input my AWIPS
>>>>>>>>>> netCDF
>>>>>>>>>> files and output an ASCII file containing the data in the 10
>>>>>>>>>> column
>>>>>>>>>> ASCII Point Observation Format required by ascii2nc?
>>>>>>>>>>
>>>>>>>>>> Waylon
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Met_help mailing list
>>>>>>>>>> Met_help at mailman.ucar.edu
>>>>>>>>>>
>>>>>>>>>>                                     
>>>>> _______________________________________________
>>>>> Met_help mailing list
>>>>> Met_help at mailman.ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>             
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: awips2nc.cc
Type: text/x-c++src
Size: 22045 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20090721/ba48e697/attachment-0001.bin 


More information about the Met_help mailing list