[Met_help] soil verification in MET's point_stat

Tue May 12 12:52:52 MDT 2009

John,

This is a fantastic response time on your part!  I think this is perhaps what I like the most about MET -- the responsiveness of your team to inquiries.  Just let me know if ever become too much of a hassle!  ;-)

I hope to be able to get to it this week because after that, I'll be on travel a lot through the end of May.  I'll be sure to get feedback to you as soon as I can.

Thanks!
Jonathan

> -----Original Message-----
> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
> Sent: Tuesday, May 12, 2009 1:31 PM
> To: Case, Jonathan (MSFC-VP61)[Other]
> Cc: Crosson, William L. (MSFC-VP61); Blankenship, Clay B. (MSFC-
> VP61)[University Space Research Association]; met_help
> Subject: Re: soil verification in MET's point_stat
>
> Jonathan,
>
> I took a look at your data, and after making a few code changes, was able
> to get Point-Stat to behave as we'd like - I think.
>
> Please take a look at the attached tar file.  Copy it into your METv2.0
> directory, unzip and untar it.  It'll overwrite 2 files in the
> "lib/vx_met_util" directory and 1 file in the "src/point_stat"
> directory.
>
> In METv2.0, we've only allowed verification on vertical levels to be at the
> surface.  We inherited a very simple approach for matching observations to
> the surface from the NCEP verification tools, and that approach is based on
> message type.  So any observations with a message type of "ADPSFC" or
> "SFCSHP" is considered to be at the surface and is matched to surface
> forecasts (really any forecast defined as a vertical level such as "TMP/Z2"
> - the Z means vertical level).
>
> Here's how I've modified this matching logic... If we are comparing a
> forecast on a vertical level to an observation with message type "ADPSFC"
> or "SFCSHP", do as before and match the observation to the forecast value.
> However, if we are comparing a forecast on a vertical level to an
> observation that is NOT one of those surface message types, check to see if
> the observation's level value falls within the specified range of levels.
> If so, use that observation, but if not, don't use it.
>
> So in your case, you should use a message type that is not ADPSFC or
> SFCSHP.  Not sure which one applies best from this list:
> http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_1.htm
>
> When you set up your ASCII files as input to ASCII2NC, use that message
> type.  And when you run Point-Stat, setup your configuration file to use
> that same message type.  Also, select the forecast field to be verified
> with something like: fcst_field = [ "TSOIL/Z0-10" ]
>
> Now suppose you're verifying soil temperature between 0 and 10 cm.  All
> observations with a level value between (and including) 0 and 10 should
> match it, but one with a level value of 12, for example, should not.
> Please note that there is no vertical interpolation being done here.  All
> observations falling within that range are used and compared against the
> single value defined for that forecast layer.
>
> Please try these changes out and see if it does what you'd like.  If so,
> I'll need to do more testing to make sure I haven't broken anything else
> with these changes!
>
> Thanks,
> John
>
>
> Case, Jonathan (MSFC-VP61)[Other] wrote:
> > Hi John,
> >
> >
> >
> > I uploaded to your ftp site a sample GRIB-1 file at the model analysis
> time of 03z 1 June 2008.  I also uploaded 3 sample files of the SCAN soil
> obs (sorry for the hideous way the data are stored!) from June 2008.
> Unfortunately, instead of columns continuing into a long row, the OBS
> archive re-starts the data from obs#1 into a new set of rows/columns.  But
> don't worry -- I'll write a conversion script so that it can be shoved
> through ascii2nc.
> >
> >
> >
> > I suppose for what you need to do, you can simply pull a line of data
> valid at 03z 1 June 2008, and use those obs in ascii2nc and point_stat for
> testing your code changes.
> >
> >
> >
> > The problem we're dealing with is to verify soil LAYER data in the model
> (at 0-10cm, 10-40cm, 40-100cm, and 100-200cm) against data valid at
> specific LEVELS in the OBS.
> >
> >
> >
> > The variables we have from the obs are:
> >
> > *         C1SMV = soil moisture percent at  2 cm depth
> >
> > *         C2SMV = soil moisture percent at  4 cm depth
> >
> > *         C3SMV = soil moisture percent at  8 cm depth
> >
> > *         C4SMV = soil moisture percent at 20 cm depth
> >
> > *         C5SMV = soil moisture percent at 40 cm depth
> >
> >
> >
> > Similarly,
> >
> > *         C1TMP = soil temperature at  2 cm depth
> >
> > *         C2TMP = soil temperature at  4 cm depth
> >
> > *         C3TMP = soil temperature at  8 cm depth
> >
> > *         C4TMP = soil temperature at 20 cm depth
> >
> > *         C5TMP = soil temperature at 40 cm depth
> >
> >
> >
> > I might be inclined to average the 2, 4, and 8 cm obs either straight up,
> or in a weighted average to compare to the 0-10 cm model layer in Noah/WRF.
> And similarly, I might average out the 20 cm and 40 cm OBS to compare to
> the model's 10-40 cm layer.
> >
> >
> >
> > Let's see what we can best figure out because I'm sure there will be
> others who would like to verify soil quantities using MET (including my 2
> co-workers whom I CC'd).
> >
> >
> >
> > Jonathan
> >
> >
> >
> >
> >
> >> -----Original Message-----
> >
> >> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
> >
> >> Sent: Friday, May 08, 2009 2:00 PM
> >
> >> To: Case, Jonathan (MSFC-VP61)[Other]; met_help
> >
> >> Subject: Re: soil verification in point_stat
> >
> >
> >> Jonathan,
> >
> >
> >> Since I've never run MET on layers like this, I'm really not sure
> either.
> >
> >
> >> Looking closer at the code, MET is only really set up to verify
> >> forecasts
> >
> >> as the surface: precip at surface, 2-meter temp, 10-meter winds.  And
> >> we're
> >
> >> matching to observations based on message type -
> >
> >> all observations with a level type of ADPSFC or SFCSHP are matched to
> >> those
> >
> >> surface forecasts.
> >
> >
> >> I think we'll need to modify the code slightly to handle your case.
> >> In
> >
> >> particular, we'll need to adjust the logic in the
> "GCPairData::add_obs()"
> >
> >> routine in the file METv2.0/lib/vx_met_util/met_stats.cc.
> >
> >
> >> Would you be able to send me a sample GRIB1 file along with a handful
> >> - 10
> >
> >> would be fine - of fake ASCII observations?  I could take a look and
> >> see
> >
> >> what changes would need to be made to get MET
> >
> >> working the way you'd want.
> >
> >
> >> You could post them to RAL's anonymous ftp site:
> >
> >
> >> ftp ftp.rap.ucar.edu
> >
> >> username = anonymous
> >
> >> password = "your email address"
> >
> >> cd incoming/irap/johnhg
> >
> >> put "your files"
> >
> >> bye
> >
> >
> >> Thanks,
> >
> >> John
> >
> >
> >> Case, Jonathan (MSFC-VP61)[Other] wrote:
> >
> >>> John,
> >
> >
> >
> >
> >>> I figured that the vertical interpolation would best be handled
> >>> outside
> >
> >> of MET, so that is what I plan to do.  I will assign the OBS to the
> >> valid
> >
> >> layer in the model.
> >
> >
> >
> >
> >>> With that, I hinted at my WRF configuration.  I'm running with the
> >>> Noah
> >
> >> LSM with soil layers of 0-10, 10-40, 40-100, and 100-200 cm.  Given
> >> those
> >
> >> layers, how should I go about assigning the OBS to the model layer in
> >> the
> >
> >> ascii2nc process?
> >
> >
> >
> >
> >>> Here's what in the WRF GRIB-1 file:
> >
> >
> >
> >
> >>> lc2:/usr/people/casejl/tmp/seus_2008062103/control > wgrib
> >
> >> WRFPRS00.tm00.d01 | grep SOILW
> >
> >
> >
> >> 86:13184188:d=08062103:SOILW:kpds5=144:kpds6=112:kpds7=10:TR=0:P1=0:P
> >> 2=0:Ti
> >
> >> meU=1:0-10 cm down:anl:NAve=0
> >
> >
> >
> >>
> 88:13482750:d=08062103:SOILW:kpds5=144:kpds6=112:kpds7=2600:TR=0:P1=0:P2=0:
> >
> >> TimeU=1:10-40 cm down:anl:NAve=0
> >
> >
> >
> >> 90:13781312:d=08062103:SOILW:kpds5=144:kpds6=112:kpds7=10340:TR=0:P1=
> >> 0:P2=0
> >
> >> :TimeU=1:40-100 cm down:anl:NAve=0
> >
> >
> >
> >> 92:14079874:d=08062103:SOILW:kpds5=144:kpds6=112:kpds7=25800:TR=0:P1=
> >> 0:P2=0
> >
> >> :TimeU=1:100-200 cm down:anl:NAve=0
> >
> >
> >
> >
> >>> lc2:/usr/people/casejl/tmp/seus_2008062103/control > wgrib
> >
> >> WRFPRS00.tm00.d01 | grep TSOIL
> >
> >
> >
> >> 85:13005070:d=08062103:TSOIL:kpds5=85:kpds6=112:kpds7=10:TR=0:P1=0:P2
> >> =0:Tim
> >
> >> eU=1:0-10 cm down:anl:NAve=0
> >
> >
> >
> >> 87:13303632:d=08062103:TSOIL:kpds5=85:kpds6=112:kpds7=2600:TR=0:P1=0:
> >> P2=0:T
> >
> >> imeU=1:10-40 cm down:anl:NAve=0
> >
> >
> >
> >>
> 89:13602194:d=08062103:TSOIL:kpds5=85:kpds6=112:kpds7=10340:TR=0:P1=0:P2=0:
> >
> >> TimeU=1:40-100 cm down:anl:NAve=0
> >
> >
> >
> >>
> 91:13900756:d=08062103:TSOIL:kpds5=85:kpds6=112:kpds7=25800:TR=0:P1=0:P2=0:
> >
> >> TimeU=1:100-200 cm down:anl:NAve=0
> >
> >
> >
> >
> >>> Thanks much!
> >
> >>> Jonathan
> >
> >
> >
> >
> >>>> -----Original Message-----
> >
> >
> >>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
> >
> >
> >>>> Sent: Friday, May 08, 2009 12:16 PM
> >
> >
> >>>> To: Case, Jonathan (MSFC-VP61)[Other]
> >
> >
> >>>> Cc: met_help
> >
> >
> >>>> Subject: Re: soil verification in point_stat
> >
> >
> >
> >>>> Jonathan,
> >
> >
> >
> >>>> As a short answer I'd say, yes, you could probably use Point-Stat
> >>>> to
> >
> >> verify
> >
> >
> >>>> soil moisture and temperature observations from WRF.
> >
> >
> >
> >>>> ...BUT... I don't think you'll be able to do any vertical
> >>>> interpolation
> >
> >
> >>>> without some code changes.  Right now, Point-Stat is only set up to
> >>>> do
> >
> >
> >>>> vertical interpolation for pressure levels... and that's
> >
> >
> >>>> done linear in the log of pressure.
> >
> >
> >
> >>>> Here's what I'd recommend...
> >
> >
> >
> >>>> (1) Take a look at the soil moisture and temperature fields you
> >>>> have
> >
> >> from
> >
> >
> >>>> WRF.  Depending on which "surface scheme" (I think that's right)
> >>>> you're
> >
> >
> >>>> using, the soil moisture fields may be at a single
> >
> >
> >>>> levels or a layer of levels.  If it's in layers, you probably
> >>>> wouldn't
> >
> >> want
> >
> >
> >>>> to interpolate in the vertical.  You'd just like to match the
> >
> >> observation
> >
> >
> >>>> to the forecast value for the layer in which the
> >
> >
> >>>> observation falls.  If it's single levels, you may want to consider
> >
> >
> >>>> interpolating.
> >
> >
> >
> >>>> (2) Take a look at the observation levels.  Are they all at some
> >>>> set of
> >
> >
> >>>> "standard" depths or do the depths vary greatly?
> >
> >
> >
> >>>> Here's one approach that may be pretty easy:
> >
> >
> >>>> - Assuming that your forecasts are in single levels.
> >
> >
> >>>> - Assuming that your observations are at some set of "standard"
> depths.
> >
> >
> >>>> - When you pre-process the observations to prepare them for
> >>>> ASCII2NC,
> >
> >> just
> >
> >
> >>>> round the level value for the observation to the nearest depth you
> >>>> have
> >
> >> in
> >
> >
> >>>> your model output.
> >
> >
> >>>> - Then run Point-Stat to verify at each of your model depths (i.e.
> >>>> no
> >
> >
> >>>> vertical interpolation).
> >
> >
> >>>> - You could even dump out the matched pair (MPR) line type for all
> >>>> of
> >
> >> the
> >
> >
> >>>> levels and use the Stat-Analysis tool to aggregate across levels
> >
> >
> >
> >>>> I've never done this before, but you could *probably* do that
> >>>> approach
> >
> >> with
> >
> >
> >>>> no code changes.  However, if you'd like to do more sophisticated
> >
> >> vertical
> >
> >
> >>>> interpolation, I may be able to point you in the
> >
> >
> >>>> right direction as for what code to modify.
> >
> >
> >
> >>>> Hope that helps.
> >
> >
> >
> >>>> John
> >
> >
> >
> >
> >>>> Case, Jonathan (MSFC-VP61)[Other] wrote:
> >
> >
> >>>>> Thank you John!
> >
> >
> >
> >>>>> I have another question for you regarding point_stat.
> >
> >
> >
> >>>>> I would like to use observations from SCAN (Soil Climate Analysis
> >
> >
> >>>> Network) to verify soil moisture and temperature observations in
> >>>> the WRF
> >
> >
> >>>> model.  Is there a way we could use point_stat for
> >
> >> processing/interpolating
> >
> >
> >>>> data for soil verification?  I don't have the exact specs yet on
> >>>> the
> >
> >
> >>>> observations, but it would probably require a vertical
> >>>> interpolation or
> >
> >
> >>>> nearest-neighbor assignment of a soil layer to the sensor obs
> >>>> depth.  I
> >
> >
> >>>> figure that if could use ascii2nc to pre-process the data into a
> >>>> netcdf
> >
> >
> >>>> file with the same variable naming convention as the WRF GRIB
> >>>> output,
> >
> >> then
> >
> >
> >>>> I might be able to pull it off.
> >
> >
> >
> >>>>> Thanks in advance,
> >
> >
> >>>>> Jonathan
> >
> >
> >
> >>>>>> -----Original Message-----
> >
> >
> >>>>>> From: John Halley Gotway [mailto:johnhg at rap.ucar.edu]
> >
> >
> >>>>>> Sent: Friday, May 08, 2009 8:47 AM
> >
> >
> >>>>>> To: Case, Jonathan (MSFC-VP61)[Other]; met_help
> >
> >
> >>>>>> Subject: Re: formatted output in mode_analysis
> >
> >
> >
> >>>>>> Jonathan,
> >
> >
> >
> >>>>>> Take a look in METv2.0/lib/vx_analysis_util/mode_job.cc.  In
> >>>>>> lines 458
> >
> >
> >>>>>> to 485, change all instances of "%.2f" to "%.3f".  That should
> >
> >> increase
> >
> >
> >>>>>> your precision by 1.
> >
> >
> >
> >>>>>> Thanks,
> >
> >
> >>>>>> John
> >
> >
> >
> >>>>>> Case, Jonathan (MSFC-VP61)[Other] wrote:
> >
> >
> >>>>>>> John,
> >
> >
> >
> >>>>>>> I would like to output an addition decimal place in the
> >>>>>>> mode_analysis
> >
> >
> >>>>>> -summary output to have 3 decimal places.
> >
> >
> >>>>>>> What line(s)/file(s) should I modify in order to get 1 more
> >>>>>>> decimal
> >
> >
> >>>>>> point precision?
> >
> >
> >>>>>>> Thanks much!
> >
> >
> >>>>>>> Jonathan
> >
> >
> >
> >>>>>>> ***********************************************************
> >
> >
> >>>>>>> Jonathan Case, ENSCO, Inc.
> >
> >
> >>>>>>> Aerospace Sciences & Engineering Division
> >
> >
> >>>>>>> Short-term Prediction Research and Transition Center
> >
> >
> >>>>>>> 320 Sparkman Drive, Room 3062
> >
> >
> >>>>>>> Huntsville, AL 35805-1912
> >
> >
> >>>>>>> Voice: (256) 961-7504   Fax: (256) 961-7788
> >
> >
> >>>>>>> Emails: Jonathan.Case-1 at nasa.gov
> >
> >
> >>>>>>>              case.jonathan at ensco.com
> >
> >
> >>>>>>> ***********************************************************
> >
> >
> >
> >