[Met_help] A problem

John Halley Gotway johnhg at rap.ucar.edu
Mon Jul 6 11:25:35 MDT 2009 

Great, I'm glad to hear that you got the lat/lon's figured out, and are able to run MODE!

As for your questions on thresholding, let me explain a little...

The TS or ETS is computed by the Grid-Stat and Point-Stat tools.  The user selects a threshold value of interest, and that threshold is applied to forecast/observation matched pairs to determine hits,
misses, false alarms, and correct negatives.  The ETS (and several other contingency table statistics) are derived from those values.

In MODE, we use a two-step method to define objects.  First, we apply a convolution filter to the raw fields.  The user decides how much smoothing is to be done by setting the "fcst_conv_radius" and
"obs_conv_radius" parameters.  Second, we threshold the convolved field using the "fcst_conv_thresh" and "obs_conv_thresh" parameters.  And from that thresholded field, we define the objects.

To answer your question directly, I believe the general answer is NO, those thresholds aren't doing exactly the same thing.  Grid-Stat and Point-Stat are thresholding the RAW fcst/obs values.  Whereas
MODE is thresholding the convolved, smoothed fields.  So in general they won't result in exactly the same grid points being turned on.

HOWEVER, if you set the convolution radii in MODE equal to 0, no convolution will be done, and you will be thresholding the raw field.  But that generally isn't recommended because you'll likely end
up with a large number of very small objects that'll be difficult to work with.

Hope that helps.

John

zhxubinchaoshan wrote:
> Thanks for your suggestions! And I have figure out the problem by your suggestion. The problem is that my lat's and lon's switched.
> In the calculating of TS or ETS, there is a parameter 'threshold'. And in the MODE tool, there is a parameter 'obs_conv_thresh'. Are they the same? 
> As we know, in the forecast system, we generally make contrast between different threshold TS or ETS. So if I can make contrast between MODE results with different threshold, just by changing the parameter 'obs_conv_thresh'?
> Thank you very much!
> 
> 
> 
> 
> ÔÚ2009-07-02£¬"John Halley Gotway" <johnhg at rap.ucar.edu> дµÀ£º
>> I took a look at the data file files you sent.  The problem is clearly in the generation of the observation file.  The forecast file is fine... the only reason the image looks funny in the PostScript
>> file you sent is that you're masking the bad data values from the observation field onto the forecast.  Try setting "mask_missing_flag = 0" in the MODE config file, rerun the data, and you'll see that
>> the forecast field is fine.
>>
>> So there's some problem in how you're generating the observation file.  I see this type of problem all the time in when I'm converting data.  You just have some indexing out of order when you're
>> creating the observation NetCDF file.  I'd suggest playing around with the tool you're using to create the NetCDF file and look at how you're ordering the data.  Perhaps you have your lat's and lon's
>> switched... or you have x where you should have nx - x.... or something like that.
>>
>> You can check how the observation NetCDF file looks by viewing it using "ncview".  Once it looks good in "ncview" it should work fine for MODE.
>>
>> Good luck.  I'm guessing it's a minor issue in how you're ordering the data, and once you figure it out, it'll all work fine.
>>
>> Thanks,
>> John
>>
>> zhxubinchaoshan wrote:
>>> Thanks for your reply!
>>> When I run MODE, I am comparing a forecast netCDF file to an observed netCDF file. The observed netCDF file is produced according to the pcp_combine output format.
>>> I send the MODE output to you, can you help me check how the problem come out?
>>> Thank you very much!
>>>
>>>  
>>>
>>>
>>> ÔÚ2009-07-01£¬"John Halley Gotway" <johnhg at rap.ucar.edu> дµÀ£º
>>>> When you run MODE are you comparing a forecast GRIB file to an observed GRIB file?  If that is the case, you can run the "copygb" utility to interpolate GRIB files from one grid to another.
>>>>
>>>> The "copygb" utility is included as part of the WRF-PostProcessor or may be downloaded/compiled separately.
>>>>
>>>> Here's some info about copygb:
>>>> http://www.cpc.noaa.gov/products/wesley/copygb.html
>>>> Slides 24-29 of the PDF file: http://www.mmm.ucar.edu/wrf/users/tutorial/200901/wpp.pdf
>>>>
>>>> And I've also attached the copygb documentation file that's included with the code.
>>>>
>>>> You can use it to regrid one dataset to another so that they're on the same grid when running MODE.  Hope that helps.
>>>>
>>>> John
>>>>
>>>> zhxubinchaoshan wrote:
>>>>> Hello, I have interpolate my precipitation data from station observations to grid using the Cressman method. But the Cressman method I used is form GRADS. So the grid interpolated is not consistent with the one in WRF model. So when I run the MODE, I found the raw observation graphic is wrong. 
>>>>> So my question is : Do I need to interpolate my precipitation data in consistent with the WRF model grid?
>>>>> And if I need to do that, is there any good method you can provide for me to do it?
>>>>> Thanks very much!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ÔÚ2009-06-30£¬"John Halley Gotway" <johnhg at rap.ucar.edu> дµÀ£º
>>>>>> Hello,
>>>>>>
>>>>>> You are correct.  The 2 most important parameters in the MODE configuration file are the convolution radius and the convolution threshold.  They determine how the objects are defined from the raw field.
>>>>>>
>>>>>> And actually, there are 4 entries in the config file for these - 2 for the forecast field and 2 for the observation field:
>>>>>> fcst_conv_radius
>>>>>> obs_conv_radius
>>>>>> fcst_conv_thresh
>>>>>> obs_conv_thresh
>>>>>>
>>>>>> But if you're comparing the same field between the fcst and obs, just set the obs settings to the same values as the fcst.
>>>>>>
>>>>>> To address your question... unfortunately, there is no one answer based on the model resolution.  Instead, it's up to the user to decide what scale of objects to analyze.
>>>>>>
>>>>>> But here are some guidelines:
>>>>>> - Setting the convolution radius lower will make the objects less smooth and more detailed.
>>>>>> - Setting the convolution radius higher will make the objects smoother.
>>>>>> - Setting the threshold lower will make the objects bigger.
>>>>>> - Setting the threshold higher will make the objects smaller.
>>>>>>
>>>>>> It's up to you to decide how you'd like to define the objects based on what type of features you'd like to extract from the field.  For example, when running MODE on precip, you may define objects
>>>>>> that are pretty large and smooth to represent large systems or MCS's.  Or you may define them to be pretty small and detailed to capture convection.  You need to ask yourself the question, what
>>>>>> features am I trying to analyze in my output?
>>>>>>
>>>>>> You'll just need to play around with running MODE to get a sense of how the settings work.  Choose a setting for the convolution radius (5, 10, 15, 20 grid squares?).  Just pick one and set it.  Then
>>>>>> set the convolution threshold to some value - for precip, just try "gt0.0".  Then run MODE.  And bring up the output PostScript file in a window.  Next, adjust the convolution threshold, rerun MODE,
>>>>>> and observe how the objects change.  Once you get a sense for how the convolution threshold works, try fixing that one and playing with the convolution radius, moving it up and down, and observing the
>>>>>> changes in the PostScript output.  At first, I'd suggest keeping radius or threshold fixed, while you adjust the other one.
>>>>>>
>>>>>> Sorry I can't provide more help.  You'll need to play around with it to get a sense of what it's doing.  If you try to run MODE to define a lot of objects (more than about 15 in each field), it'll
>>>>>> take a lot longer to run since there's more calculations to perform for each object.
>>>>>>
>>>>>> Once you do find some settings you like, feel free to send some sample data to us for advice on interpreting the output.  If you do send some output, be sure to send the MODE configuration file you
>>>>>> used and the PostScript plots.
>>>>>>
>>>>>> Thanks,
>>>>>> John Halley Gotway
>>>>>> johnhg at ucar.edu
>>>>>>
>>>>>> zhxubinchaoshan wrote:
>>>>>>> Hello,I am using the MODE tool in MET for verification. 
>>>>>>> And I have some problems about the parameters in it. In the WrfModeConfig, there are two important parameters.They are obs_conv_radius and obs_conv_thresh.I have read some referances, and I found that how to specify these two parameters is dependent on my model resolution. If my model resolution is 12km, how can I specify these two parameters? 
>>>>>>> Thank you very much!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ÔÚ2009-05-13£¬"John Halley Gotway" <johnhg at rap.ucar.edu> дµÀ£º
>>>>>>>> Great.  Glad that did the trick for you.
>>>>>>>>
>>>>>>>> John
>>>>>>>>
>>>>>>>> zhxubinchaoshan wrote:
>>>>>>>>> Thanks for your reply!It can work now! 
>>>>>>>>> The problem is the "fcst_var". By your suggestion, I change "APCP_6" to "APCP_06", and then it can work successfully! 
>>>>>>>>> Thank you! 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ÔÚ2009-05-11£¬"John Halley Gotway" <johnhg at rap.ucar.edu> дµÀ£º
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> What this warning message is telling you is that the Stat-Analysis tool is
>>>>>>>>>> looking in the directory "out/rain" for *.stat files, but it is not
>>>>>>>>>> finding any lines in those files that EXACTLY match the job you've
>>>>>>>>>> specified.
>>>>>>>>>>
>>>>>>>>>> When I experience a problem like this, my usual approach is to take a step
>>>>>>>>>> back and under-specify the job just to make sure I'm matching some lines. 
>>>>>>>>>> Then I take a look at the STAT lines that went into the job and adjust
>>>>>>>>> >from there.  For example, you may try the following:
>>>>>>>>>> (1) Run this Stat-Analysis job on the command line:
>>>>>>>>>> ../bin/stat_analysis -lookin ./out/rain \
>>>>>>>>>> -job aggregate_stat -line_type FHO -out_line_type CTS \
>>>>>>>>>> -dump_row tmp.stat
>>>>>>>>>>
>>>>>>>>>> (2) Did this job run successfully? Did it find any input FHO lines to use?
>>>>>>>>>> If not, then maybe there aren't any FHO lines in your STAT files.  If it
>>>>>>>>>> did find FHO lines, and the job ran fine, proceed to the next step.
>>>>>>>>>>
>>>>>>>>>> (3) Take a look in the file "tmp.stat".  How would you like to filter this
>>>>>>>>>> data down more?  Do you see "APCP_6" in the column for "fcst_var"?  Or
>>>>>>>>>> maybe it's actually "APCP_06"?  Try adding that to the job:
>>>>>>>>>>
>>>>>>>>>> ../bin/stat_analysis -lookin ./out/rain \
>>>>>>>>>> -job aggregate_stat -line_type FHO -fcst_var APCP_6 -out_line_type CTS \
>>>>>>>>>> -dump_row tmp.stat
>>>>>>>>>>
>>>>>>>>>> (4) Did this job run successfully, or was there a problem with how you
>>>>>>>>>> specified the "fcst_var".  Now check the "tmp.stat" file again, and adjust
>>>>>>>>>> your job as necessary.  One hint though, when running Stat-Analysis jobs
>>>>>>>>>> like this on the command line, you'll usually need to put a backslash
>>>>>>>>>> before special characters like this: -fcst_thresh "\>0.000"
>>>>>>>>>>
>>>>>>>>>> So I'd suggest approaching the problem like that.  If you can't figure it
>>>>>>>>>> out, feel free to send along all of the STAT files in your "out/rain"
>>>>>>>>>> directory, the Stat-Analysis config file you're using, and the command
>>>>>>>>>> line you're using... and I'll try to figure out what's going on.
>>>>>>>>>>
>>>>>>>>>> Good luck.
>>>>>>>>>>
>>>>>>>>>> John Halley Gotway
>>>>>>>>>> johnhg at ucar.edu
>>>>>>>>>>
>>>>>>>>>>> When I run the Stat_analysis Tool, I get the error below:
>>>>>>>>>>> WARNING: do_job_aggr_stat() -> no matching STAT lines found for job: -job
>>>>>>>>>>> aggregate_stat -fcst_var APCP_6 -fcst_thresh >0.000 -line_type FHO
>>>>>>>>>>> -dump_row ./out/stat_analysis/job_aggregate_stat_FHO_CTS.stat
>>>>>>>>>>> -out_line_type CTS -out_alpha 0.050000
>>>>>>>>>>>
>>>>>>>>>>> And my command is :
>>>>>>>>>>>  ../bin/stat_analysis -config ./STATAnalysisConfig_07 -lookin ./out/rain
>>>>>>>>>>> -out ./out/stat_analysis/stat_analysis.out
>>>>>>>>>>>
>>>>>>>>>>> Can you tell me how to figure out the problem above?
>>>>>>>>>>> Thank you very much!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Met_help mailing list
>>>>>>>>>>> Met_help at mailman.ucar.edu
>>>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/met_help
>>>>>>>>>>>
>

More information about the Met_help mailing list