[Met_help] help in obtaining archived PREPBUFR observations

[John Halley Gotway]

Jonathan,

Yes, I would suggest obtaining the unblocked (*.prepbufr.unblock.nr) files and then blocking (cwordsh block) them on your machine.

If you were to obtain the already blocked files (*.prepbufr.nr), you would need to unblock (cworsh unblk) and then re-block them (cwordsh block).  So getting the unblocked files saves you one step.

I wanted to mention that we've removed the need for all this blocking and unblocking business for METv2.0.  Now we do the blocking internally in the code.  There's a compile time option you need to
select to enable this to work - you have to choose 4-byte or 8-byte blocking.  But once you get MET compiled correctly for your machine/compiler, you won't need to bother with this cwordsh step anymore.

John

Case, Jonathan (MSFC)[] wrote:
> Dear John,
> 
> I had one more point of clarification for the archived PREPBUFR files on
> NOMADS.  
> It looks like there are 2 types of files: *.prepbufr.nr and
> *.prepbufr.unblock.nr
> 
> Should I obtain the unblocked data so I can Fortran block the data in
> case there are platform incompatibility issues?  I suspect there may be
> because when I used your sample data, I had to unblock, and then
> re-fortran block it on our system.
> 
> Thanks for the clarification,
> Jonathan
> 
>> -----Original Message-----
>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>> Sent: Monday, January 26, 2009 4:14 PM
>> To: Case, Jonathan (MSFC)[]
>> Cc: met_help at mailman.ucar.edu
>> Subject: Re: [Met_help] help in obtaining archived PREPBUFR
>> observations
>>
>> Jonathan,
>>
>> I have good news and bad news for you.
>>
>> First, the data you're actually looking for are the files that contain
>> "prepbufr" in them - not simply "bufr".  And the GDAS prepbufr data is
>> stored in 6 hours chunks - 00Z, 06Z, 12Z, and 18Z.  I
>> believe each file contains +/- 3 hours of data around the time.  So
> the
>> 06Z file contains observations between 03Z and 09Z.
>>
>> Here's one of the files you're looking for:
>>
> http://nomads.ncdc.noaa.gov/data/gdas/200803/20080330/gdas1.t06z.prepbu
>> fr.nr
>>
>> Each one of these prepbufr files contains all of the observation types
>> put together.  But unfortunately, you may find that this prepbufr
>> archive does not go back in time as far as you need.  I think
>> it's only available here back to 12/14/2006.  Does that work for you?
>>
>> Retrieving the data is actually pretty easy.  You can use the "wget"
>> unix command to grab a whole bunch of files from the web.  So you'd
>> just need to write a script to construct the full path for the
>> files you'd like to retrieve, save them to a file, and pass it to
> wget.
>> For example, suppose a file named "my_prepbufr_files.txt" contains the
>> following 4 lines:
>>
>>
> http://nomads.ncdc.noaa.gov/data/gdas/200612/20061231/gdas1.t00z.prepbu
>> fr.nr
>>
> http://nomads.ncdc.noaa.gov/data/gdas/200612/20061231/gdas1.t06z.prepbu
>> fr.nr
>>
> http://nomads.ncdc.noaa.gov/data/gdas/200612/20061231/gdas1.t12z.prepbu
>> fr.nr
>>
> http://nomads.ncdc.noaa.gov/data/gdas/200612/20061231/gdas1.t18z.prepbu
>> fr.nr
>>
>> Run the following command to grab all those files:
>> wget -i my_prepbufr_files.txt
>>
>> It should be pretty straight-forward to generate a list of files you'd
>> like.  And then you can just run the wget command overnight.
>>
>> Since the observations are in 6 hour chunks, you'll need to run them
>> through the PB2NC tool to generate the 1 hour files you'd like.
>>
>> In METv1.1 (the current released version), you'd need to run PB2NC 6
>> times to generate the 6 1-hour files.  In METv2.0 (to be released in
>> Feb/March), you'd need to run it through PB2NC only once, and
>> then use command line arguments to Point-Stat to control the time
> range
>> of observations to be used for each Point-Stat run.
>>
>> If you'd like, in a couple of weeks, you'd be welcome to run the beta
>> version of METv2.0.  Having more people test it out is always a good
>> thing prior to a release.
>>
>> Hope this helps.  Let me know if you still have questions.
>>
>> John Halley Gotway
>> johnhg at ucar.edu
>>
>> Case, Jonathan (MSFC)[] wrote:
>>> Dear MET help,
>>>
>>>
>>>
>>> I'd like to get started in running the standard verification
>> statistics
>>> programs such as point-stat.
>>>
>>> On your web site, you point to
> http://nomads.ncdc.noaa.gov/data/gdas/
>> as
>>> a source of archived PREPBUFR observations that are used in the
> GDAS.
>>> However, there are numerous files with the string "bufr", the data
>> only
>>> go back to 2006, and it would be cumbersome to download each file
>> needed
>>> for verification.
>>>
>>>
>>>
>>> Therefore, I'd like to ask what is the best way to obtain hourly or
>>> sub-hourly PREPBUFR surface observations that can be used in point-
>> stat
>>> to compute typical surface verification, and what datasets should I
>> be
>>> looking for?
>>>
>>>
>>>
>>> I appreciate your assistance!
>>> Jonathan
>>>
>>>
>>>
>>> ***********************************************************
>>> Jonathan Case, ENSCO, Inc.
>>> Aerospace Sciences & Engineering Division
>>> Short-term Prediction Research and Transition Center
>>> 320 Sparkman Drive, Room 3062
>>> Huntsville, AL 35805-1912
>>> Voice: (256) 961-7504   Fax: (256) 961-7788
>>> Emails: Jonathan.Case-1 at nasa.gov
>>>
>>>              case.jonathan at ensco.com
>>>
>>> ***********************************************************
>>>
>>>
>>>
>>>
>>>
>>>
>>>
> ---------------------------------------------------------------------
>> ---
>>> _______________________________________________
>>> Met_help mailing list
>>> Met_help at mailman.ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/met_help