[Met_help] Verification for the HWT

[Tressa Fowler]

Ken,

This is great news. All of our neatest methods are for grid to grid  
comparison.

We have a software package called MET (designed for verifying WRF)  
that can implement a lot of methods for you. It can be used on other  
gridded forecasts if they are in either grib1 or netCDF format. The  
software, tutorial, and documentation are available at: http://www.dtcenter.org/met/users

We have a MET tutorial this July in Boulder if you want to send  
someone to learn how to use it. There is also an online tutorial  
linked to the page above. http://www.mmm.ucar.edu/events/tutorial_709/index.php

In this package, we have several spatial verification methods and some  
basic probabilistic verification.

How do you verify PoP? In MET, we produce some information about  
reliability that can be used to calibrate probability forecasts.

For your QPF, you might want to start with our simpler spatial  
approach and then try out the object and wavelet based verification.
The neighborhood methods in our grid verification tool (grid_stat) are  
pretty easy to implement and understand. Basically, you smooth the  
forecasts and obs over some area, then figure the usual statistics  
plus a few neighborhood specific skill scores.

If you want to get more complicated, you can use MODE (a component of  
MET), to create and match up objects. We have lots examples of this  
(see papers by Chris Davis), but it is a bit computer intensive so not  
easy to implement for real time use. (Though it can be done.) It is  
really fabulous for case studies or post-analysis.

We also have a wavelet decomposition tool in MET that can help you  
find the spatial scales of your errors.

I am happy to give advice and answer questions if you think you would  
like to try any of these things.

Maybe after you look this over, you could decide where you would like  
to start and I can offer more detailed advice or recommendations.

Tressa

On May 8, 2009, at 3:56 PM, Ken Pomeroy wrote:

> We are solely doing gridded verification. However, our current  
> methods are not spatial in nature. They just compare observed precip  
> to forecast precip at each grid box and determine if the forecast is  
> "close enough". It's a binary yes/no decision for each grid box.
>
> Ken
>
> Tressa Fowler wrote:
>> Ken,
>>
>> I would be very happy to share. Could you give me a little more  
>> background about what you do now?
>>
>> Unfortunately, the verification methods available are completely  
>> dependent on whether you have gridded or point forecasts and  
>> observations. Which do you use?
>>
>> Thanks,
>>
>> Tressa
>>
>> On May 8, 2009, at 11:13 AM, Ken Pomeroy wrote:
>>
>>> Hi Tressa,
>>>
>>> Andy Edman spoke to me about your presentation on verification at  
>>> the USWRP Workshop. Here at WRH, we are trying to do verification  
>>> on QPF and PoP forecasts at our WFO's but have found it difficult  
>>> to come up with meaningful data to help forecasters improve. I was  
>>> wondering if you would be willing to share some of your methods  
>>> for verification.
>>>
>>> Thanks!
>>> Ken Pomeroy
>>> WRH Scientific Services Division
>>