[Met_help] [rt.rap.ucar.edu #62012] History for Unable to get RMSE

[Tressa Fowler via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi Paul and John. 
I am verifying rainfall for a season (say 86 days) the following categories using pointstat. 
the categories are 
gt0.0, ge2.5, ge7.6, ge35.6, ge64.5 and so on. 
I am using following command to get the aggregate statistics for RMSE. 

/oprn/model/wrf3/utils/met/METv3.0/bin/stat_analysis \
  -lookin statfiles \
  -out agst-f-rmse-ge35.6 -v 4 -job aggregate_stat -line_type SL1L2 -out_line_type CNT \

~                                                                                        
SO I have got the o/p files 
-rw-r--r--    1 model    staff          1372 Jun 27 15:52 agst-f-rmse-gt0.0
-rw-r--r--    1 model    staff          1372 Jun 27 16:49 agst-f-rmse-ge2.5
-rw-r--r--    1 model    staff          1372 Jun 27 16:51 agst-f-rmse-ge7.6
drwxr-xr-x    2 model    staff          8192 Jun 27 16:52 statfiles
-rwxr-xr-x    1 model    staff           319 Jun 27 16:52 agst.sh
-rw-r--r--    1 model    staff          1372 Jun 27 16:52 agst-f-rmse-ge35.6
but I find that there is no difference between the files. 
 
bash-3.2$ diff agst-f-rmse-ge2.5 agst-f-rmse-ge7.6                   
bash-3.2$ diff agst-f-rmse-ge7.6 agst-f-rmse-ge35.6 
bash-3.2$ 

bash-3.2$ 
 
Help me. I have seen that my stat files are all different. 
Pls help. 
Geeta

 		 	   		  

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #62012] Unable to get RMSE
From: Julie Prestopnik
Time: Thu Jun 27 09:34:32 2013

Hi Geeta.  Paul no longer works here, and John is out through next
week.  I am filling in on met_help and have limited experience with
the
software, but will try to help you as best I can. Could you please
send
your statfiles and the config file(s) that you are used?

Thank you,
Julie

On 2013-06-27 05:32, Geeta Geeta via RT wrote:
> Thu Jun 27 05:32:26 2013: Request 62012 was acted upon.
> Transaction: Ticket created by geeta124 at hotmail.com
>        Queue: met_help
>      Subject: Unable to get RMSE
>        Owner: Nobody
>   Requestors: geeta124 at hotmail.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62012
> >
>
>
> Hi Paul and John.
> I am verifying rainfall for a season (say 86 days) the following
> categories using pointstat.
> the categories are
> gt0.0, ge2.5, ge7.6, ge35.6, ge64.5 and so on.
> I am using following command to get the aggregate statistics for
> RMSE.
>
> /oprn/model/wrf3/utils/met/METv3.0/bin/stat_analysis \
>   -lookin statfiles \
>   -out agst-f-rmse-ge35.6 -v 4 -job aggregate_stat -line_type SL1L2
> -out_line_type CNT \
>
> ~
>
> SO I have got the o/p files
> -rw-r--r--    1 model    staff          1372 Jun 27 15:52
> agst-f-rmse-gt0.0
> -rw-r--r--    1 model    staff          1372 Jun 27 16:49
> agst-f-rmse-ge2.5
> -rw-r--r--    1 model    staff          1372 Jun 27 16:51
> agst-f-rmse-ge7.6
> drwxr-xr-x    2 model    staff          8192 Jun 27 16:52 statfiles
> -rwxr-xr-x    1 model    staff           319 Jun 27 16:52 agst.sh
> -rw-r--r--    1 model    staff          1372 Jun 27 16:52
> agst-f-rmse-ge35.6
> but I find that there is no difference between the files.
>
> bash-3.2$ diff agst-f-rmse-ge2.5 agst-f-rmse-ge7.6
> bash-3.2$ diff agst-f-rmse-ge7.6 agst-f-rmse-ge35.6
> bash-3.2$
>
> bash-3.2$
>
> Help me. I have seen that my stat files are all different.
> Pls help.
> Geeta


------------------------------------------------
Subject: Unable to get RMSE
From: Tressa Fowler
Time: Wed Jul 03 10:03:14 2013

Hi Geeta,

John is on vacation, but I can help you today.

RMSE is a continuous statistic, applied to actual values of forecasts
and observations according to the formula in Appendix C of the MET
Users' Guide. No thresholding is performed on this measure, therefore
you should be getting the same RMSE values in all of your files.

Thresholds are applied _only_ for categorical statistics to produce
contingency tables and statistics like probability of detection and
equitable threat score. Again, see Appendix C for details.

I would encourage you not to use RMSE for precipitation verification.
The statistic tends to be overwhelmed by all the non-rainy 0 values in
both the forecast and observations. Equitable threat score, frequency
bias, and probability of detection are much more useful, in my
opinion.

Please let me know if you have further questions.

Tressa


On Thu Jun 27 05:32:26 2013, geeta124 at hotmail.com wrote:
> Hi Paul and John.
> I am verifying rainfall for a season (say 86 days) the following
> categories using pointstat.
> the categories are
> gt0.0, ge2.5, ge7.6, ge35.6, ge64.5 and so on.
> I am using following command to get the aggregate statistics for
RMSE.
>
> /oprn/model/wrf3/utils/met/METv3.0/bin/stat_analysis \
>   -lookin statfiles \
>   -out agst-f-rmse-ge35.6 -v 4 -job aggregate_stat -line_type SL1L2
> -out_line_type CNT \
>
> ~
> SO I have got the o/p files
> -rw-r--r--    1 model    staff          1372 Jun 27 15:52 agst-f-
rmse-
> gt0.0
> -rw-r--r--    1 model    staff          1372 Jun 27 16:49 agst-f-
rmse-
> ge2.5
> -rw-r--r--    1 model    staff          1372 Jun 27 16:51 agst-f-
rmse-
> ge7.6
> drwxr-xr-x    2 model    staff          8192 Jun 27 16:52 statfiles
> -rwxr-xr-x    1 model    staff           319 Jun 27 16:52 agst.sh
> -rw-r--r--    1 model    staff          1372 Jun 27 16:52 agst-f-
rmse-
> ge35.6
> but I find that there is no difference between the files.
>
> bash-3.2$ diff agst-f-rmse-ge2.5 agst-f-rmse-ge7.6
> bash-3.2$ diff agst-f-rmse-ge7.6 agst-f-rmse-ge35.6
> bash-3.2$
>
> bash-3.2$
>
> Help me. I have seen that my stat files are all different.
> Pls help.
> Geeta
>



------------------------------------------------