[Met_help] point_stat config question
Ruifang Li
lir at ucar.edu
Fri Aug 21 14:26:20 MDT 2009
John,
Thanks for help and info.
Ruifang
On Fri, Aug 21, 2009 at 2:13 PM, John Halley Gotway <johnhg at rap.ucar.edu>wrote:
> Ruifang,
>
> Thresholds are not applied when computing continuous (CNT) statistics.
> That's why those columns (FCST_THRESH and OBS_THRESH) contain NA's in the
> CNT lines of the MET output.
>
> Let me refer you to two places for more information about the verification
> statistics that MET generates. The first is Appendix C of the MET User's
> Guide. In there we give a description of how the stats are computed and
> what each means. And here's a good link about the definition of
> verification scores:
> http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html
>
> Hopefully these resources will answer your questions.
>
> But if you have a specific question about how to interpret one of the
> stats, go ahead and write MET-Help, and a statistician will get back to
> you.
>
> Thanks,
> John
>
> > John,
> > I looked at CTC table and thresholds are applied. However I could not
> see
> > thresholds in CNT table. I guess thresholds should be also applied in CNT
> > to
> > get user desired statistic values (for example bias, rmse), right?
> >
> > Thanks,
> > Ruifang
> >
> > On Fri, Aug 21, 2009 at 8:06 AM, John Halley Gotway
> > <johnhg at rap.ucar.edu>wrote:
> >
> >> Ruifang,
> >>
> >> The behavior you're seeing is correct. The MPR output would be the
> >> same.
> >> However, the values in the contingency tables (CTC line type, for
> >> example)
> >> should be different when choosing different
> >> thresholds.
> >>
> >> Here's what's going on... When you run Point-Stat, you can think of it
> >> as
> >> setting up a bucket for each combination of forecast field/masking
> >> region/interpolation method. So for example, suppose you
> >> configure Point-Stat to verify 2-meter temperature and 500mb temperature
> >> over 3 masking regions, your FULL domain and then two subdomains. And
> >> you're using the nearest neighbor interpolation method.
> >> In this case, we'd be verifying 2 fields over 3 regions using 1
> >> interpolation method. So Point-Stat will set up 6 "buckets", 3 for
> >> 2-meter
> >> temp and 3 for 500mb temp.
> >>
> >> Next, Point-Stat looks at each one of the point observation values
> >> you've
> >> passed to it. If the observation type (i.e. 2-m temp) matches the
> >> forecast
> >> type, and it's in the correct masking region,
> >> it'll interpolate the forecast values to the observation location to
> >> create
> >> a matched pair. Then it throws that matched fcst-obs pair into the
> >> appropriate bucket. Point-Stat processes through all of
> >> the point observations in this way, computing matched pairs, and
> >> throwing
> >> them in the right buckets.
> >>
> >> After that's finished, it'll compute whatever statistics you've
> >> requested
> >> using whatever threshold values you've set. For example, if you set
> >> threshold values of ">273.0" and ">283.0" for 2-m temp,
> >> it'll apply those to all of the matched pairs in the "bucket" and
> >> compute
> >> contingency tables and stats. However, the choice of threshold has no
> >> effect on which matched pairs make it into the bucket
> >> to begin with.
> >>
> >> The matched pair (MPR) output consists of all the matched pairs that are
> >> in
> >> the buckets. So the MPR output does NOT depend on any choice of
> >> threshold
> >> value.
> >>
> >> Does that make sense?
> >>
> >> Thanks,
> >> John
> >>
> >> Ruifang Li wrote:
> >> > Hi John,
> >> > I run point_stat tool and did two test with different fcst_thresh[] in
> >> > config file. I expect the different MPR output, but they create same
> >> MPR.
> >> I
> >> > could not figure out why. In my understanding, if fcst_thresh eq 0,
> >> fcst
> >> > value in MPR should be 0. if fcst_thresh le 100 fcst value in MPR
> >> should
> >> be
> >> > le 100, right?
> >> >
> >> > Here are two fcst_thresh:
> >> >
> >> > fcst_field[] =
> >> >
> >>
> ["UGRD/Z10","UGRD/P1000","UGRD/P925","UGRD/P850","UGRD/P700","UGRD/P500","UGRD/P400","UGRD/P300","UGRD/P250","UGRD/P200","UGRD/P150","UGRD/P100","UGRD/P70","UGRD/P50","UGRD/P30","UGRD/P20","UGRD/P10"
> >> > ];
> >> >
> >> > //fcst_thresh[] = [ "le100",
> >> >
> >>
> "le10","le10","le100","le100","le100","le100","le100","le100","le100","le100","le100",
> >> > "le100","le100", "le100","le100","le100" ];
> >> > fcst_thresh[] = [ "eq0",
> >> > "eq0","eq0","eq0","eq0","eq0","eq0","le10","eq0","eq0","eq0","eq0",
> >> > "eq0","eq0", "eq0","eq0","eq0" ];
> >> >
> >> > Here is MPR file:
> >> >
> >> > /ptmp/lir/test/2009_t8_15km/verify_met2.0/test% ls -l
> >> > /ptmp/lir/data/2009_t8_15km/verify_met/gsi/point_stat/UGRD/2007081512/
> >> > total 14208
> >> > -rw-r--r-- 1 lir ncar 786984 Aug 20 16:05
> >> > point_stat_000000L_20070815_120000V.stat
> >> > -rw-r--r-- 1 lir ncar 652568 Aug 20 16:05
> >> > point_stat_000000L_20070815_120000V_mpr.txt
> >> > -rw-r--r-- 1 lir ncar 698412 Aug 20 16:05
> >> > point_stat_120000L_20070816_000000V.stat
> >> > -rw-r--r-- 1 lir ncar 579124 Aug 20 16:05
> >> > point_stat_120000L_20070816_000000V_mpr.txt
> >> > -rw-r--r-- 1 lir ncar 761574 Aug 20 16:05
> >> > point_stat_240000L_20070816_120000V.stat
> >> > -rw-r--r-- 1 lir ncar 631498 Aug 20 16:05
> >> > point_stat_240000L_20070816_120000V_mpr.txt
> >> > -rw-r--r-- 1 lir ncar 740520 Aug 20 16:05
> >> > point_stat_360000L_20070817_000000V.stat
> >> > -rw-r--r-- 1 lir ncar 614040 Aug 20 16:05
> >> > point_stat_360000L_20070817_000000V_mpr.txt
> >> > -rw-r--r-- 1 lir ncar 791340 Aug 20 16:05
> >> > point_stat_480000L_20070817_120000V.stat
> >> > -rw-r--r-- 1 lir ncar 656180 Aug 20 16:05
> >> > point_stat_480000L_20070817_120000V_mpr.txt
> >> >
> >> > Script:
> >> > /ptmp/lir/test/2009_t8_15km/verify_met2.0/test% point_stat_gsi.ksh
> >> > *** Running POINT_STAT on WRF OUTPUT ***
> >> > *** UGRD ***
> >> > GSL_RNG_TYPE=mt19937
> >> > GSL_RNG_SEED=2412285258
> >> > Forecast File:
> >> >
> >>
> /ptmp/lir/data/2009_t8_15km/verify_met2.0/gsi/wpp/2007081512/postprd/wrfprs_d01.000
> >> > Climatology File: none
> >> > Configuration File: ./config/PointStatConfig_GSI.UGRD
> >> > Observation File:
> >> > /ptmp/lir/data/2009_t8_15km/verify_met2.0/ob_nc/2007081512/ob.nc.qc2
> >> >
> >> > ----------------------------------------
> >> >
> >> > Reading records for UGRD/Z10.
> >> > For UGRD/Z10 found 1 forecast levels and 0 climatology levels.
> >> >
> >> >
> >> > Thanks for your support,
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------
> >> >
> >> > _______________________________________________
> >> > Met_help mailing list
> >> > Met_help at mailman.ucar.edu
> >> > http://mailman.ucar.edu/mailman/listinfo/met_help
> >>
> >
> >
> >
> > --
> > Ruifang
> >
> > Mesoscale & Microscale Meteorology Division
> > Phone: 303-497-8938
> > Office: FL3-3085
> > Email: lir at ucar.edu
> >
>
>
>
--
Ruifang
Mesoscale & Microscale Meteorology Division
Phone: 303-497-8938
Office: FL3-3085
Email: lir at ucar.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/met_help/attachments/20090821/9edbae10/attachment-0001.html
More information about the Met_help
mailing list