[Met_help] ASCII Data for MET

[John Halley Gotway]

Thomas,

Yeah, that abort message from BUFRLIB is what I expected.  BUFRLIB just isn't 64-bit compatible.  You're right that blocking/unblocking is an issue... but that still won't fix the problem.  To my
knowledge, BUFRLIB just won't work with a 64-bit compiler.

In METv2.0, we've moved the blocking step into the PB2NC tool.  And the way the blocking is performed is controlled by a compile time flag.  In the top-level MET Makefile, the "ARCH_FLAGS" line has
the "-DBLOCK4" flag commented out for GNU compilers.  You could try uncommenting that and recompiling MET, being sure to do a "make clean" first.  And then you could try rerunning that "test_pb2nc.sh"
script.  But I'm not hopeful.  I'm guessing you'll still get that abort message from BUFRLIB.

Unfortunately, we haven't made any progress on support for the ECMWF BUFR data to date.  However, I'll forward your message to the group that decides our priorities for upcoming releases to let them
know you were asking about it.

Lastly, I agree with you.  If you're dealing with ~2000 observations/day, I think it'd be better to sort them by day.  Otherwise, if they were all in one file for a month, Point-Stat would have to
sift through 60,000 observations each time you run it... which could slow it down.

Good luck,
John

Thomas Schwitalla wrote:
> John,
> 
> John Halley Gotway schrieb:
>> Thomas,
>>
>> I'm glad you were able to compile MET on a 64-bit machine.  Since it
>> sounds like you're using the ASCII2NC tool and not the PB2NC tool, you
>> shouldn't have any problems.  It's the PB2NC which
>> interfaces with BUFRLIB where the problem exists for 64-bit machines. 
>> Could you try running the test script "METv2.0/scripts/test_pb2nc.sh"
>> and let me know if you see an error message in the output?
>>   
> I get:
> 
> **************BUFR ARCHIVE LIBRARY ABORT*****************
> 
> BUFRLIB: OPENBF - FIRST 4 BYTES READ FROM RECORD IN INPUT FILE CONNECTED
> TO UNIT  11 NOT 'BUFR', DOES NOT CONTAIN BUFR DATA
> 
> **************BUFR ARCHIVE LIBRARY ABORT*****************
> 
> 
> This is obviously a problem of NCEP BUFR in combination with gfortran,
> because I get similar error messages when using PREPBUFR data from NCEP
> directly in the WRF-Var system... (currently it is not possible).
> Maybe it helps to unblock the unblocked data again and then reblock them?
> 
> Are there any news regarding the possible use of ECMWF BUFR data as
> desired at the Winter MET tutorial?
> 
>> Yes, you will still need to run your WRF output through WPP prior to
>> running it through MET.  The purpose of this is twofold: (1) to
>> convert the native model sigma levels to pressure levels and (2) to
>> interpolate the model's staggered grid to a more regular grid.  There
>> is another WRF tool that performs these functions (only for ARW)
>> called "pinterp".  We'll be working with the ARW folks to add
>> support in MET for the NetCDF output of "pinterp" for the next release
>> of MET.
>>   
> This sounds good :-)
>> For your second question, yes, you could write the whole month's
>> observations to a single NetCDF file if you like.  But each time you
>> run Point-Stat, you should use the "-valid_beg" and "-valid_end"
>> command line arguments (or set "beg_ds" and "end_ds" in the Point-Stat
>> config file) to define the time window for the observations to be used.
>>
>> Also, it depends on the number of observations we're talking about. 
>> If you were using thousands of observations for each day, I'd suggest
>> breaking them out into a single file per day to make
>> Point-Stat run faster.  But I'm guessing you're not using all that
>> many observations and won't have problems with how fast Point-Stat runs.
>>   
> Because it is a huge amount of observations (e.g. ~2000 temperature
> measurements per hour over central Europe), I think it is better to
> split the data into daily files. The run-time of the tools is not that
> time critical (even if it runs 1 day).
> 
> Thomas
>> Thanks,
>> John
>>
>> Thomas Schwitalla wrote:
>>  
>>> Hi John,
>>>
>>> now I want to start using MET for verification of my WRF runs. :-)
>>> As far as I know, MET currently only deals with GRIB files created from
>>> WPP!?
>>> Is this still valid?
>>>
>>> The second question is about the ASCII-Date used for the ascii2nc tool:
>>>
>>> Is it possible, to write all observations (for one month e.g.) into a
>>> single file or  should the timestamp of the observation files correspond
>>> to the timestep of one WRF output file (my purpose is to do a one month
>>> long statistic or timeseries of surface variables) ?
>>>
>>> Best regards
>>> Thomas
>>>
>>> P.S.: I had no problems compiling  MET and running the test dataset on a
>>> 64bit Linux machine using gfortran 8-)
>>>
>>>     
> 
>