[Met_help] Precip observations in MET
John Halley Gotway
johnhg at rap.ucar.edu
Tue Jan 29 11:39:39 MST 2008
Victor,
Thanks for posting that data. It helped me debug the problem.
I see two issues here.
The first issue is the segmentation fault you're getting when you run Point-Stat on your 30-hour precip field. Fortunately, I have an easy fix for you which should solve this. We just need to modify
one line in one of the library files. We haven't tested Point-Stat on precip data before, so we didn't catch this bug. So thanks for finding it!
Please replace the file: METv1.0/src/lib/vx_met_util/met_stats.cc with the version of met_stats.cc attached to this message.
Then rebuild MET by doing a "make clean" followed by a "make". Point-Stat should now run on your data without the error.
Please let me know if the fix works on your system. Once I verify that it works, I'll post the fix to the MET website.
Also, I've attached a version of your observations that I modified to test the fix. I just changed your 24-hour accumulation observations to 30-hour accumulations. When I run Point-Stat using the
30-hour accumulation forecast and the 30-hour point observations, it finds 2498 data pairs.
Second, you're wondering how you'd go about subtracting a 6-hour accumulation from a 30-hour accumulation to get a 24-hour accumulation. Right now, the answer is that you can't do that with MET.
But, in a future release, you'll be able to. The Pcp-Combine tool within MET can be used to sum up accumulation intervals but not to do differences. However, we do realize the need to be able to do
differences for the exact reason you've pointed out. We will implement that soon... I'll talk to the MET development team about getting it in the next release. So you'll ultimately be able to use
Pcp-Combine to do differences.
Please let me know if the fix worked for you.
Thanks,
John
Victor Homar wrote:
> Hi John,
>
> Thanks for your patience, we did not realized that detail regarding GRIB/Netcdf
> formats and vertical levels.
> Now, we converted the forecast file to GRIB but are still not successfully
> running MET.
> Let us explain our trials:
>
> We try the command:
> ../bin/point_stat ~/WRF-ARW/wrfpostprocV2/cas_John/postprd/wrfprs_d01.30
> ~/verificacio/Observations_INM_rain_24h_mm.nc \
> config/PointStatConfig -outdir ../out/prova_point_stat
>
> (with a vx_grib_code[] = [ "APCP/A030" ]; in config/PointStatConfig file)
>
> but get a:
> Forecast File:
> /megadisk/people/lorena/WRF-ARW/wrfpostprocV2/cas_John/postprd/wrfprs_d01.30
> Observation File:
> /megadisk/people/lorena/verificacio/Observations_INM_rain_24h_mm.nc
> Configuration File: config/PointStatConfig
> Climatology File: none
> Segmentation fault (core dumped)
>
> If we change vx_grib_code[] = [ "APCP/A024" ]; the output is a bit better but
> not much:
> Forecast File:
> /megadisk/people/lorena/WRF-ARW/wrfpostprocV2/cas_John/postprd/wrfprs_d01.30
> Observation File:
> /megadisk/people/lorena/verificacio/Observations_INM_rain_24h_mm.nc
> Configuration File: config/PointStatConfig
> Climatology File: none
> ***WARNING***: process_grib_codes() -> no records matching grib code 61 with
> level indicator of A024 not found in grib file:
> /megadisk/people/lorena/WRF-ARW/wrfpostprocV2/cas_John/postprd/wrfprs_d01.30
> Output VSDB file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V.vsdb
> Output FHO file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_fho.txt
> Output CTC file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_ctc.txt
> Output CTP file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_ctp.txt
> Output CFP file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_cfp.txt
> Output COP file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_cop.txt
> Output CTS file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_cts.txt
> Output CNT file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_cnt.txt
> Output SL1L2 file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_sl1l2.txt
> Output SAL1L2 file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_sal1l2.txt
> Output VL1L2 file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_vl1l2.txt
> Output VAL1L2 file:
> ../out/prova_point_stat/point_stat_300000L_20011111_060000V_val1l2.txt
>
>
> Now the program point_stat runs well but it complains about not finding 24h
> accumulated rain in the forecast file. This is actually correct, but we don't
> know how to get a GRIB forecast file with 24h accumulated rainfall (IC+06h to
> IC+30h) from our WRF output. Is there any simple method to do that?
>
> We tried to get the desired field with the nco package directly over WRF netCDF
> output but since WRF doesn't render total accumulated rain (convective and
> gridscale rain), we got stuck there and don't know how to proceed from here.
>
> Our forecast period is 30 h because we allow the first 6h of forecast for the
> finer model scales to fill up and the simulation to spin-up adequately. The
> verification period encompasses the last 24h of that period.
>
> Do you have any suggestions on how to create a GRIB file with 24h accumulated
> rain from IC+06h to IC+30h so we can feed it to point_stat?
>
> Many thanks,
> Lorena and Víctor.
> PS: We uploaded to our ftp server the two "wrfpostprocessed" GRIB files from our
> WRF output file, valid at IC+06h (
> ftp://eady.uib.es/pub/victor/MET/WRFPRS_d01.06 ) and IC+30h (
> ftp://eady.uib.es/pub/victor/MET/WRFPRS_d01.30 ) just in case these could help
> clarify the problem.
>
> John Halley Gotway wrote:
>> Victor,
>>
>> I took the ASCII point observations you provided and converted them to NetCDF
>> with the tool I'm working on. I've attached them as an example.
>>
>> However, the forecast file you provided is in NetCDF format. MET expects
>> gridded data to be provided in Grib format. That's why we suggest the use the
>> WRF-PostProcessor to destagger the WRF output files, interpolate to pressure
>> levels, and write out Grid data.
>>
>> Do you have your gridded forecasts available in Grib1 format?
>>
>> Thanks,
>> John
>>
>> Victor Homar wrote:
>>> Hi John,
>>>
>>> Thanks for your responses.
>>>
>>> Your plan to provide us with a tar ball with the whole thing to test it
>>> sounds good.
>>>
>>> Answering your questions:
>>> 1) The format makes good sense. The Message_Type field is a little artificial
>>> for the particular set of stations we are dealing with now. These are not
>>> synoptic stations but a particular network of our Met Service (They call them
>>> "climatological network"). I guess the ADPSFC code will work for all further
>>> steps along the verification process.
>>> Also, and thinking about general users, the format makes good sense because
>>> it has been straight forward to adapt our data to it.
>>>
>>> 2) No suggestions, at first thought it looks like a reasonable and versatile
>>> format to import generic observations.
>>>
>>> 3 and 4) I uploaded both an observations file and a WRF forecast file to:
>>>
>>> ftp://eady.uib.es/pub/victor/MET/
>>>
>>> The WRF forecast file is a thinned file using the utility "ncks" to include
>>> only the variables Times,RAINC,RAINNC,XLONG and XLAT. Let me know if you need
>>> the big (3.5Gb) complete WRFout file.
>>>
>>> Thanks for all your help and have a nice weekend,
>>>
>>> Víctor.
>>>
>>> John Halley Gotway wrote:
>>>> Victor,
>>>>
>>>> To answer your questions...
>>>> This piece of code, like most of MET, would be written in C++.
>>>> To test it, I'd like to try it out on some data internally and some of your
>>>> data as well.
>>>>
>>>> I was working on this tool today, and I have some ideas. As for the format
>>>> of the ASCII observation file, I'm thinking that each line could consist of
>>>> the following 9 pieces of info:
>>>> Message_Type Station_ID Valid_Time(YYYYMMDD_HHMMSS) Lat(Deg North) Lon(Deg
>>>> East) Elevation(m) Accum(h)_or_Pressure(hPa) Grib_Code Observation_Value
>>>>
>>>> So for example, here are 4 lines of ASCII observation values (I made them up):
>>>> ADPSFC SID1 20050810_120100 40.00 -105.00 -9999 12 61 2.01
>>>> ADPSFC SID2 20050810_120500 41.00 -105.00 -9999 12 61 0.85
>>>> ADPSFC SID3 20050810_121000 42.00 -105.00 -9999 12 61 0.90
>>>> ADPSFC SID3 20050810_121000 42.00 -105.00 -9999 500 11 300
>>>> They consist of 3 observation of 12-hourly accumulated precip (grib code =
>>>> 61), and 1 observation of temperature (grib code = 11) at 500 hPa. I've
>>>> attached the netCDF (sample_ascii2nc.nc) file that would be created from
>>>> these 4 observations and could be used as input to Point-Stat.
>>>>
>>>> The elevation value is not currently used in Point-Stat, but we may want to
>>>> use it in the future. So I've included it as one of the things to enter,
>>>> but you can just put fill data in there right now.
>>>>
>>>> However, I've realized that Point-Stat needs to be tweaked slightly to
>>>> perform point verification of accumulated precipitation. It wasn't matching
>>>> the observations up correctly. So when I provide you with this tool to test
>>>> on your end, I'll probably just send an updated tarball containing all of
>>>> the necessary changes.
>>>>
>>>> My questions for you are:
>>>> Does the ASCII observation format make sense, and would it be easy to use?
>>>> Do you have any suggestions for changes or additions to the ASCII
>>>> observation format?
>>>> Would you be able to provide me with some of your rain-gauge observations
>>>> formatted in this way?
>>>> And would you be able to provide me with a sample forecast file to match up
>>>> to those observations?
>>>>
>>>> One last thing to mention. I realize that the intermediate NetCDF file
>>>> format contains a lot of unnecessary fields in it. We plan to clean up that
>>>> file format in the near future and organize it a little better.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>> Victor Homar wrote:
>>>>> John,
>>>>>
>>>>> We'll be glad to collaborate in creating such a tool which would make our
>>>>> (non BUFR users) life much easier with MET.
>>>>> So far, we tried to create a NetCDF file for Point-Stat but could not
>>>>> figure out what are the minimum set of parameters per observation that the
>>>>> NetCDF file should include.
>>>>>
>>>>> We are currently concerned with observations of 24h accumulated
>>>>> precipitation and looking at page 3-10 of the MET User's Guide, it is not
>>>>> intuitive to guess valid values for parameters such as "Vertical level",
>>>>> "Pressure level"or "Reason code". Attached is a simple file (we could
>>>>> provide more information regarding raingauge station characteristics)
>>>>> showing an example of the type of ASCII files we deal with. Obviously,
>>>>> variations from this format could easily be done.
>>>>>
>>>>> What your plans to test the your code would be? Is the code written in
>>>>> Fortran or C, or you use a higher level language?
>>>>>
>>>>> Best,
>>>>> Lorena and Víctor
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> John Halley Gotway wrote:
>>>>>> Lorena and Victor,
>>>>>>
>>>>>> This question has come up a lot. We'd like to figure out an easy way for
>>>>>> users to incorporate ASCII observations into MET. I'm wondering if you'd
>>>>>> be willing to work with me on developing a
>>>>>> conversion tool that takes ASCII observations in a format similar to the
>>>>>> one you've described and reformat them into the NetCDF format that
>>>>>> Point-Stat expects?
>>>>>>
>>>>>> So I'm wondering, if I was to provide you with some code for doing this,
>>>>>> would you be willing to test it out on your system and let me know how
>>>>>> things go?
>>>>>>
>>>>>> Hopefully we could figure out a solution and include it in the next
>>>>>> release of MET.
>>>>>>
>>>>>> If you'd be willing to work with you on this, could you please send me a
>>>>>> couple of your forecast files and corresponding sets of ASCII
>>>>>> observations? I'll use them to test out the code I write before
>>>>>> sending it to you. Otherwise, if you'd like to do it yourself, I could
>>>>>> provide you with some more info on how to do it.
>>>>>>
>>>>>> Thanks and please let me know,
>>>>>> John Halley Gotway
> --
> -----------------------------------------------------------------------
> Víctor Homar Santaner
> Grup de Meteorologia
>
> Edif. Mateu Orfila Tel: +34 971 17 1376
> Universitat de les Illes Balears Fax: +34 971 17 3426
> 07122 Palma de Mallorca (SPAIN) Email: Victor.Homar at uib.es
>
> Knowledge is contagious. Infect truth.
> -----------------------------------------------------------------------
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: met_stats.cc
Type: text/x-c++src
Size: 82320 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20080129/888fc98f/met_stats-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Observations_INM_rain_30h_mm.nc
Type: application/x-netcdf
Size: 294172 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/met_help/attachments/20080129/888fc98f/Observations_INM_rain_30h_mm-0001.nc
More information about the Met_help
mailing list