[Met_help] More MET questions

[John Halley Gotway]

Dan,

I realized that you're not using MET but wanted to let you know about the update.

But you raise a good point.  Grid-Stat is currently not being as discerning as it could be when computing counts and statistics.  We are checking the contents of the output_flag prior to computing
things but have grouped several tasks together for coding convenience.  We could conceivably split things out to a finer scale so that when you request only counts, that's really all the code would be
doing.

Thank for the suggestion.

John

Daniel Schaffer wrote:
> Thanks John
> 
> I do need to remind you of the FYI I included in my previous email that due to the short time fuse I'm on, I had to move ahead without MET. So I probably won't try your fix at this time.  However, as I also said, it may be appropriate down the road to come back to MET and compare my results with those from MET.  This would be contingent on a solution to the performance problem associated with a large number of points in the region polygons.  
> 
> Also, I think a subtext to GSL issue is the apparent fact that there was no way I could configure MET to avoid computing stats at all.  In my case, as I mentioned, I just needed the counts.  So, something else to think about for your long term planning purposes.
> 
> Dan
> 
> -------------------------------------------------------------------------------
> 
> Dan Schaffer, Research Associate            e-mail: Daniel.S.Schaffer at noaa.gov
> NOAA/OAR/Earth System Research Lab         phone:  (303) 497-7252
> Aviation Branch                             fax:    (303) 497-6301
> R/GSD5   325 Broadway                       
> Boulder, CO 80305
> 
> www-ad.fsl.noaa.gov/ac/schaffer.html
> 
> On Wed, 29 Oct 2008, John Halley Gotway wrote:
> 
>> Dan,
>>
>> I looked more closely at the bug you're seeing when running Grid-Stat over the 1830x918 domain.
>>
>> I narrowed it down to a GSL library routine that's choking on a the large number of grid points.  For whatever reason, the GSL routine "gsl_cdf_chisq_Pinv", which MET uses in computing normal
>> confidence intervals, is choking when the degrees of freedom passed to it are greater than 1,263,131.  For your data, we're passing it a value of 1,679,940.
>>
>> I tried downloading and linking to the latest version of GSL, version 1.11, released 3/30/2008.  But the bug is still present in that version.  So I've submitted a bug report to GSL.
>>
>> As n -> infinity, the value returned by this routine -> infinity as well.  As a crude workaround, I've added a check to the wrapper for this GSL routine to check for n being too large.  If it is, the
>> routine now returns a value of n*0.998.  With n so large anyway, this should have a negligible affect on the confidence intervals computed.
>>
>> I tried this using your sample data files, and it does now work.
>>
>> The attached file "gsl_cdf.cc" which includes the workaround replaces the file "METv1.1/lib/vx_gsl_prob/gsl_cdf.cc".  After replacing that file, please rebuild MET being sure to do a "make clean" first.
>>
>> Thanks,
>> John
>>
>> Daniel Schaffer wrote:
>>> Hey John
>>>
>>> FYI.  Given the difficulties I ran into as we have been discussing and a short time fuse I face, I have gone ahead and written my own code that computes counts for various neighborhood widths.  So at this moment it is not super-critical to resolve these issues.  However, it would be interesting to compare the counts I generate with MET's answers.
>>>
>>> I've attached the files that you requested.  Note that I only attached a sample forecast file since the behavior is the same if the forecast and obs files are identical.
>>>
>>> As for your response below; I had already seen that.  I'm not clear how it's relevant to the problem I'm having with the GSL gamma function.
>>>
>>>>> Regarding GSL, please see the entry about GSL on the known issue page of the MET website:
>>>>> http://www.dtcenter.org/met/users/support/known_issues/METv1.1/index.php
>>>>> Look in the section on Grid-Stat.
>>>>>
>>> Thanks for your continued support.
>>> Dan
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Met_help mailing list
>>> Met_help at mailman.ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/met_help