[Met_help] Re: MET 1.0 tests not working

[John Halley Gotway]

Cory,

Thanks for all the information you sent.

Regarding the runtime problems in grid_stat relating to sprintf...  It's amazing what turns up when you try to run the same piece of code on different architectures.  We hadn't had any problems with
that anywhere else we ran MET.  But I'm glad you were able to fix the sprintf issue by making a local copy of the string before passing it to sprintf.  To be safe, I'll go through all of the MET code
and make similar changes when I'm using sprintf in that way.

The error you're seeing from PB2NC (actually it's from the BUFRLIB code itself) is expected.  Currently, BUFRLIB won't run correctly on a 64-bit machine.  It errors out when attempting to read
PrepBufr files on a 64-bit machine.  I've alerted the group at NCEP who wrote BUFRLIB about this issue, and they're working on making it compatible with 64-bit machines.  It isn't available yet, but
I'll send an email to all the registered MET users when it does become available.  Although we developed and tested MET primarily on 32-bit machines, I did briefly test it on a 64-bit machine.  The
BUFRLIB issue was the only problem I turned up.

If you do have access to a 32-bit machine, you could try running MET there.

Regarding the pb2nc output... I've attached a copy of the file sample_pb.nc.  You're welcome to reformat your point observations to mimic this format if you like, and then run your obs through
point_stat.  I know of at least one other user who has done so.  I want so say up front though that this format is less than ideal.  It's structured based on the way data is stored in PrepBufr files.
 And there's a lot of fields in there that you can just fill with a fill data value of -9999 since MET doesn't actually use them.  We've retained that info since it's present in the PrepBufr data and
NCEP has some use for it.

Basically, the NetCDF format contains both header observation location/timing information and then the actual observation values corresponding to those headers.

The observation location/timing information (indexed by "nmsg" = 9716 in sample_pb.nc) is stored in the arrays: "hdr_typ", "hdr_sid", "hdr_vld", and "hdr_arr".  They contain the message type, station
id, valid time, lat/lon, elevation, and some extra stuff.

For each one of those 9716 header messages, there are corresponding observation values (indexed by "nobs" = 89759 in sample_pb.nc).  The observation values are all stored in the "obs_arr" variable.
Now the first entry in "obs_arr" is the "hdr_id".  That points to the index of the header information (i.e. 1 to 9716) which corresponds to this observation.  For example, if there are 9 observation
values corresponding to first set of header message info (i.e. the 1st of the 9716 message), then hdr_id = 1 for the first 9 entries of the "obs_arr".

I know that's a little convoluted, but that's how we decided to store the information to try to keep the netCDF files as small as possible.

Now, let me attempt to indicate which fields in this netCDF file actually need to contain valid data to be used by "point_stat":
obs_arr:hdr_id = header index -> YES
obs_arr:level = vertical level -> NO
obs_arr:p_level = pressure level -> YES
obs_arr:gc = grib code corresponding to the observation -> YES
obs_arr:ob = observation value (in the units specified by the grib
code
documentation) -> YES
obs_arr:qm = quality mark -> NO
obs_arr:pc = program code -> NO
obs_arr:rc = reason code -> NO
obs_arr:fc = forecast value -> NO
obs_arr:an = analyzed value -> NO
obs_arr:cat = data level category -> NO
hdr_typ = header type sting -> YES
hdr_sid = header station id -> YES
hdr_vld = header valid time (YYYY-MM-DD_HH:MM:SS UTC) -> YES
hdr_arr:lon = longitude of obs -> YES
hdr_arr:lat = latitude of obs -> YES
hdr_arr:dhr = obs time - cycle time -> NO
hdr_arr:elv = elevation of obs location -> NO
hdr_arr:typ = report type -> NO
hdr_arr:t29 = input report type -> NO
hdr_arr:itp = instrument type -> NO

Hopefully, I got all of these correct.  We're planning in a future release to do a better job of supporting alternative point observation formats.  We're trying to select a simple ASCII format for
point observations.  If you have any suggestions on what type of ASCII point observation you'd like to see, we love to hear them!

Thanks and let us know if you have any other questions.

John Halley-Gotway

Cory Davis wrote:
> Sorry, further still, my machine is x86_64 running RHEL4, and I compiled
> everything with the Gnu compilers.
> Cheers,
> Cory.
On Fri, 2008-01-11 at 01:25 +0000, Cory Davis wrote:
>> Hi there,
>>
>> We are having trouble running the test cases that come with MET v1.0.
>> There were problems with test_grid_stat.sh and test_pb2nc.sh. We have
>> managed to resolve the grid_stat issue by modifying grid_stat.cc.
>>
>> The problem was occuring in the functions write_fcst_netcdf and
>> write_obs_netcdf when creating the netcdf variable names fcst_var_name,
>> diff_var_name, and obs_var_name.  I am no c expert, but the sprintf
>> calls did not seem to like the get_grib_code_abbr call as an argument,
>> and the resulting segment of the *_var_name variable was a garbage
>> string.  I have solved this problem by creating a new string
>> code_abbr_str, which is assigned thus (only once):
>>
>> strcpy(code_abbr_str,get_grib_code_abbr(gc_info[lev].code,
>> conf.ncep_defaults().ival(), status));
>>
>> and then passed to sprintf:
>> sprintf(fcst_var_name, "FCST_%s_%s_%s_%s_%i",code_abbr_str,
>> gc_info[lev].lvl_str, mask_names[i], mthd_str[mthd], wdth*wdth);
>>
>> I have attached my modified grid_stat.cc
>>
>> ---------------
>>
>> The problem with test_pb2nc.sh is the following
>>
>> [cory at hydra scripts]$ test_pb2nc.sh  *** Running PB2NC on a
>> fortran-blocked PrepBufr file ***
>> PrepBufr2NC config file: config/PB2NCConfig_G212
>> Opening PrepBufr
>> file: ../data/sample_obs/prepbufr/ndas.t00z.prepbufr.tm12.20070401.nr.blk
>>
>>  **************BUFR ARCHIVE LIBRARY ABORT*****************
>>  BUFRLIB: OPENBF - FIRST 4 BYTES READ FROM RECORD IN INPUT FILE
>> CONNECTED TO UNIT  11 NOT 'BUFR', DOES NOT CONTAIN BUFR DATA
>>  **************BUFR ARCHIVE LIBRARY ABORT*****************
>>
>>
>> I have tried unblocking this with cwordsh to no avail.  It is possible
>> that we will want to avoid pb2nc for our application, because our
>> observations are already in netcdf - we may decide to just write a
>> program converting our netcdf obs files to the MET internal netcdf
>> format.
>>
>> Cheers,
>> Cory
>>
>>
>>
>>
_______________________________________________
> Met_help mailing list
> Met_help at mailman.ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/met_help
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sample_pb.nc
Type: application/x-netcdf
Size: 4660876 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20080111/111223a5/sample_pb-0001.nc