[Met_help] Request for Assistance-UPDATE (UNCLASSIFIED)
Raby, John (Civ, ARL/CISD)
john.raby at us.army.mil
Tue Apr 27 15:15:17 MDT 2010
Classification: UNCLASSIFIED
Caveats: NONE
John -
We are confused as our sysadmin, Bob Flanigan, was in the process of
compiling and found that it appears that the patch was already compiled on
21 APR 2009. He is going ahead with the recompile to make sure it's
implemented.
Yesterday, we found a mirror image of the same problem. This problem was a
situation where the required data was fully populated for every hour, but
the optional data was mostly empty (1 out of 25 hourly files). Point-Stat
only produced results for the 1 hour where there was optional and required
data.
It appears that Point-Stat will only produce results where both datasets
have data for a given hour.
Please call us at (575) 678-2004.
Thanks.
R/
John
-----Original Message-----
From: John Halley Gotway [mailto:johnhg at ucar.edu]
Sent: Monday, April 26, 2010 10:40 AM
To: Raby, John (Civ, ARL/CISD)
Cc: Brown, Bob (Civ, ARL/CISD); met_help at ucar.edu
Subject: Re: [Met_help] Request for Assistance-UPDATE (UNCLASSIFIED)
John,
Yes, it should print a warning message to the screen when an observation
file is missing, and then process the observations in any additional
observation files. I definitely think logging the output of the MET tools
is a good idea. Please let me know if you experience any more issues or
need more clarification.
Thanks,
John
Raby, John (Civ, ARL/CISD) wrote:
> Classification: UNCLASSIFIED
> Caveats: NONE
>
> John -
>
> Thanks for the call last Friday and for your quick analysis and solution.
>
> With the fix, you say it should print a warning then just continue
> processing instead of exiting. Does "continue processing" mean it will
> go ahead and process any additional observation files for that same
> hour passed via the optional argument?
>
> Since we were calling Point-Stat 25 times in succession, we were
> unable to note the writing of errors in the info which was scrolling
> on the screen. We have re-written our script to log the output and
> error statements so we can go back and read it.
>
> We will let you know about the results after recompiling with your fix.
>
> R/
> John
>
>
> -----Original Message-----
> From: John Halley Gotway [mailto:johnhg at ucar.edu]
> Sent: Friday, April 23, 2010 3:22 PM
> To: Raby, John (Civ, ARL/CISD)
> Cc: Brown, Bob (Civ, ARL/CISD); met_help at ucar.edu
> Subject: Re: [Met_help] Request for Assistance-UPDATE (UNCLASSIFIED)
>
> John,
>
> OK, I've made a minor change in the Point-Stat source code. The
> updated version is attached.
>
> Here's the original piece of code:
> cerr << "\n\nERROR: process_obs_file() -> "
> << "can't open observation netCDF file: "
> << obs_file[i_nc] << "\n\n" << flush;
> exit(1);
>
> And here's the modified piece of code:
> cout << "\n\nWARNING: process_obs_file() -> "
> << "can't open observation netCDF file: "
> << obs_file[i_nc] << "\n\n" << flush;
> return;
>
> When one of the observation files passed to it doesn't exist, instead
> of exiting with an error message, it'll just print a warning message
> and process the remaining observation files. I suspect that if you
> look closely at the output for those hours where the PREPBUFR file
> does not exist, you'll find an error message alerting you to that fact.
>
> Please try using this version of the code, recompile MET, and rerun.
> And please let me know if this solves your problem.
>
> Thanks,
> John
>
> Raby, John (Civ, ARL/CISD) wrote:
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>> John -
>>
>> We are unable to put together the data you requested now.
>>
>> Ignoring the source of the observational (PrepBUFR or MADIS) the main
>> issue is that if the required data argument contains no data for a
>> given hour the software ignored the optional data for that same hour.
>> Conversely, if there was required data for a given hour, then the
>> optional data for that hour was ingested and used by Point-Stat. In
>> short, the software refuses to add optional data to missing required
>> data
> field.
>> Why should the lack of data in the required data for a given hour
>> preclude the ingest of the optional data for that same hour?
>>
>> Due to the complexity of all of this, it would probably help to
>> discuss it verbally. Please call me at this number (575) 678-2004.
>>
>> Thanks.
>>
>> John
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>> Sent: Friday, April 23, 2010 10:35 AM
>> To: Raby, John (Civ, ARL/CISD)
>> Cc: Brown, Bob (Civ, ARL/CISD); met_help at ucar.edu
>> Subject: Re: [Met_help] Request for Assistance-UPDATE (UNCLASSIFIED)
>>
>> John,
>>
>> The order in which you pass point observation files to Point-Stat
>> should have no impact on how the matched pairs are accumulated. For
>> each observation file passed to Point-Stat (the mandatory one or
>> additional optional ones), we just call the "process_obs_file"
>> routine once. If the behavior of Point-Stat changes depending on the
>> order of the point observation files, there may be a bug.
>>
>> I'm not entirely clear on the exact details of your situation. Would
>> you be able to provide me with an example of running Point-Stat two
>> different ways, where changing the order of the point observation
>> files has an impact on the results? The simpler you're able to make
>> the example, the easier it'll be to debug the issue.
>>
>> For this, I'd need:
>> - 1 forecast file
>> - 2 point observation files
>> - 1 Point-Stat configuration file
>> - 1 example of how you call Point-Stat on the command line
>>
>> And you could post them to our ftp following the directions I sent
>> previously.
>>
>> Thanks,
>> John
>>
>> Raby, John (Civ, ARL/CISD) wrote:
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>> John -
>>>
>>> UPDATE:
>>>
>>> We solved the problem by processing the PrepBUFR data to include the
>>> metar data (ADPSFC) in addition to the ADPUPA" and "ANYAIR" data
>>> mentioned in the email below. This ensured that there were all 25
>>> files of data present data provided in the required argument.
>>>
>>> We changed the MADIS data collection to remove the metar data and
>>> only receive the mesonet data and added this using the optional data
>>> argument (-ncfile).
>>>
>>> Now the Point-Stat results are produced for all 25 hours (06Z-06Z).
>>>
>>> In summary, if the required PrepBUFR data argument contains no data
>>> for a given hour the software ignored the optional MADIS data.
>>>
>>> Is this supposed to happen? Why should the lack of data in the
>>> PrepBUFR data (required argument) preclude the ingest all 25 hours
>>> of the
>> optional data?
>>> Please call me if we can clarify more for you or let me know if you
>>> still want the data ftp'd to you.
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>> Mr John W. Raby, Meteorologist
>>> U.S. Army Research Laboratory
>>> White Sands Missile Range, NM 88011
>>> (575) 678-2004 DSN 258-2004
>>> FAX (575) 678-1230 DSN 258-1230
>>> Email: john.raby at us.army.mil
>>>
>>> Please begin using the email address within this signature block for
>>> all future correspondence. The current @arl.army.mil address will
>>> be decommissioned in upcoming months.
>>>
>>> "When you can measure what you are speaking about and express it in
>>> numbers, you know something about it, but when you cannot measure
>>> it, when you cannot express it in number, your knowledge is of a
>>> meagre and unsatisfactory kind". - Lord Kelvin
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
>>> Sent: Thursday, April 22, 2010 8:58 AM
>>> To: Raby, John (Civ, ARL/CISD)
>>> Cc: met_help at ucar.edu
>>> Subject: Re: Request for Assistance
>>>
>>> John,
>>>
>>> Sure, we'd be happy to help you figure this out. Would you be able
>>> to send me some sample ASCII MADIS data, a sample forecast file, and
>>> the Point-Stat configuration file you're using?
>>>
>>> I'll try running ASCII2NC and Point-Stat here and figure out what's
>>> going on.
>>>
>>> You could post this sample data to our ftp site as follows:
>>> ftp ftp.rap.ucar.edu
>>> username=anonymous
>>> password="your email address"
>>> cd incoming/irap/met_help/raby_data
>>> put "your data files"
>>> bye
>>>
>>> Please let me know when the data is available, and I'll go take a look.
>>>
>>> Thanks,
>>> John
>>>
>>>
>>> We are attempting to use both PrepBUFR and MADIS observational data
>>> in MET Point_Stat to evaluate WRF performance over Utah for 2
>>> domains, one nested inside the other.
>>>
>>> We process the PrepBUFR data for a 24 hour period (06Z to 06Z) We
>>> explicitly retain only the message strings "ADPUPA" and "ANYAIR"
>>> using the PB2NC configuration file settings in order to collect
>>> upper air data (raobs, aircraft, acars, etc). After running PB2NC,
>>> we typically find only 7 hourly files (12Z, 15Z, 17Z, 19Z, 20Z, 00Z
>>> and
>>> 01Z) out of
>>> 25 possible which contain data (presumably because the upper air
>>> data is not present for every hour). These 7 include the 00Z and
>>> 12Z raob data for 2 upper air stations inside the larger of our
>>> domains (KSLC and
>> KLKN).
>>> We process the MADIS data to retain only surface mesonet and metar
>>> data in
>>> 25 hourly text files (Point observation format), then run ASCII2NC
>>> to convert the files to netcdf.
>>>
>>> We ran Point-Stat with the PrepBUFR upper air data files specified
>>> in the required argument for the "obs_file" with the MADIS (surface
>>> data) specified as additional NETCDF point observation files using
>>> the
> "-ncfile"
>> argument.
>>> The Point-Stat output contains results for only the hours 12Z, 15Z,
>>> 17Z, 19Z, 20Z, 00Z and 01Z.
>>>
>>> We then tried running Point-Stat with the MADIS data specified in
>>> the required argument and the PrepBUFR data specified as the
>>> optional (additional observation file) argument, but this gave identical
results.
>>>
>>> We expect results for all 25 hourly periods since we know that there
>>> are MADIS surface observations for every hour.
>>>
>>> Can you assist us in resolving this problem?
>>>
>>> Thanks.
>>>
>>> R/
>>> John
>>>
>>> Mr John W. Raby, Meteorologist
>>> U.S. Army Research Laboratory
>>> White Sands Missile Range, NM 88011
>>> (575) 678-2004 DSN 258-2004
>>> FAX (575) 678-1230 DSN 258-1230
>>> Email: john.raby at us.army.mil
>>>
>>>
>>>
>>>
>>> Classification: UNCLASSIFIED
>>> Caveats: NONE
>>>
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> -
>>> --
>>>
>>> _______________________________________________
>>> Met_help mailing list
>>> Met_help at mailman.ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>> Classification: UNCLASSIFIED
>> Caveats: NONE
>>
>>
> Classification: UNCLASSIFIED
> Caveats: NONE
>
>
Classification: UNCLASSIFIED
Caveats: NONE
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5292 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20100427/5415e2cd/attachment-0001.bin
More information about the Met_help
mailing list