[Met_help] mode_analysis output question
John Halley Gotway
johnhg at ucar.edu
Thu Nov 5 07:39:15 MST 2009
Jon,
Thanks for letting us know. I see why this is an insufficient fix!
At this point, I'd suggest modifying the directory structure you're using. For example, if you want to stratify by accumulation interval, I'd suggest putting all of the files related to 1-hour
accumulations into a separate subdirectory. And then put the other accumulation intervals into separate subdirectories as well.
When you call the mode_analysis tool then you could pass it just that subdirectory for the 1-hour accumulation using the "-lookin" option. Please note that you can use the "-lookin" option multiple
times to specify multiple subdirectories in which to look for MODE output files. Since the fix extracts the valid time from the MODE output file names, the valid time for any MODE output file you
pass to MODE-Analysis will be used. So you'll basically need to do the filtering manually by how you set up the directory structure.
I know this is inconvenient, but at this point, I'd say that'd be the simplest approach.
Another possibility that occurred to me would be to have MODE-Analysis read the "_cts.txt" output files from MODE as well. These can be created regardless of the number of objects defined. We could
read those header lines, do the filtering requested, and extract the valid times meeting the filtering criteria. However, this isn't a simple change and would require some larger coding changes.
Hope that helps.
John
Case, Jonathan (MSFC-VP61)[Other] wrote:
> Hello again John,
>
> It looks like the change you made was both successful and unsuccessful, according to what I need.
> First, it was successful in that it output all 0's for valid times that had no matching or unmatching objects.
> However, it was unsuccessful in that it no longer excludes times/files at different accumulation threshold.
>
> Here's my sample output with your code change in mode_job.cc. As you'll see, it outputs all the same times, when I only have MODE output from hours 12-24 for the 1-hour threshold. It appears to have parsed the data, but it still includes all the valid times, regardless of the fcst_accum threshold.
>
> Any ideas what to do next?
> Thanks,
> Jonathan
>
> lc2:/raid1/casejl/MET/MODEANALYSIS > mode_analysis -lookin /raid1/casejl/MET/MODE/seus_control_ge10mm_interest_lt_80km/JUN/2008060103 -bycase -single -simple -fcst_accum 01
>
>
>
> Total mode lines read = 397
> Total mode lines kept = 137
>
> Fcst Valid Time Area Matched Area Unmatched # Fcst Matched # Fcst Unmatched # Obs Matched # Obs Unmatched
> ---------------------- ------------ -------------- -------------- ---------------- ------------- ---------------
> Jun 1, 2008 06:00:00 0 0 0 0 0 0
> Jun 1, 2008 09:00:00 0 0 0 0 0 0
> Jun 1, 2008 12:00:00 0 0 0 0 0 0
> Jun 1, 2008 15:00:00 0 0 0 0 0 0
> Jun 1, 2008 16:00:00 0 10 0 1 0 0
> Jun 1, 2008 17:00:00 0 21 0 1 0 0
> Jun 1, 2008 18:00:00 0 109 0 1 0 3
> Jun 1, 2008 19:00:00 22 177 1 3 1 3
> Jun 1, 2008 20:00:00 80 279 1 5 1 4
> Jun 1, 2008 21:00:00 0 989 0 10 0 10
> Jun 1, 2008 22:00:00 172 1794 3 12 2 11
> Jun 1, 2008 23:00:00 181 1648 3 13 1 6
> Jun 2, 2008 00:00:00 794 452 2 7 2 7
> Jun 2, 2008 01:00:00 888 337 2 3 2 4
> Jun 2, 2008 02:00:00 0 852 0 6 0 3
> Jun 2, 2008 03:00:00 0 373 0 3 0 0
> Jun 2, 2008 06:00:00 0 0 0 0 0 0
> lc2:/raid1/casejl/MET/MODEANALYSIS > mode_analysis -lookin /raid1/casejl/MET/MODE/seus_control_ge10mm_interest_lt_80km/JUN/2008060103 -bycase -single -simple -fcst_accum 03
>
>
>
> Total mode lines read = 397
> Total mode lines kept = 152
>
> Fcst Valid Time Area Matched Area Unmatched # Fcst Matched # Fcst Unmatched # Obs Matched # Obs Unmatched
> ---------------------- ------------ -------------- -------------- ---------------- ------------- ---------------
> Jun 1, 2008 06:00:00 0 582 0 7 0 2
> Jun 1, 2008 09:00:00 0 40 0 1 0 0
> Jun 1, 2008 12:00:00 0 880 0 3 0 1
> Jun 1, 2008 15:00:00 0 237 0 3 0 3
> Jun 1, 2008 16:00:00 0 0 0 0 0 0
> Jun 1, 2008 17:00:00 0 0 0 0 0 0
> Jun 1, 2008 18:00:00 0 266 0 4 0 5
> Jun 1, 2008 19:00:00 0 0 0 0 0 0
> Jun 1, 2008 20:00:00 0 0 0 0 0 0
> Jun 1, 2008 21:00:00 983 1974 7 22 6 12
> Jun 1, 2008 22:00:00 0 0 0 0 0 0
> Jun 1, 2008 23:00:00 0 0 0 0 0 0
> Jun 2, 2008 00:00:00 3102 3447 6 14 7 19
> Jun 2, 2008 01:00:00 0 0 0 0 0 0
> Jun 2, 2008 02:00:00 0 0 0 0 0 0
> Jun 2, 2008 03:00:00 551 2702 4 6 3 9
> Jun 2, 2008 06:00:00 111 607 1 3 1 3
>
> -----Original Message-----
> From: John Halley Gotway [mailto:johnhg at ucar.edu]
> Sent: Wednesday, November 04, 2009 9:31 AM
> To: Case, Jonathan (MSFC-VP61)[Other]
> Cc: Randy Bullock; met_help
> Subject: Re: FW: You have been unsubscribed from the Met_help mailing list
>
> Jon,
>
> Unfortunately, this is a bit messier than I'd like. The MODE-Analysis tool simply parses MODE output lines and extracts the valid times from those output lines. However, if no objects are identified in a MODE run, it'll create an output file that contains no lines (other than the header line). And the MODE-Analysis tool won't have anything to parse.
>
> The only alternative I see is to have the MODE-Analysis tool also parse the valid times from the MODE output file name itself. However, this is something we always try to avoid. We don't like relying on information being stored in filenames as they can easily be changed.
>
> I've made a quick and dirty fix for this and have attached the file you'll need. But we'll discuss how we'd like to handle this in the next release of MET. Hopefully we can come up with a more robust way of handling this case without relying on info in the filename.
>
> Please place the attached file "mode_job.cc" into the directory METv2.0/lib/vx_analysis_util. Then from the top-level directory, do a "make clean", and then rebuild MET.
>
> Listed below are the lines I added for this fix. Please let me know if you have any more problems with this.
>
> Thanks,
> John
>
> +++++++++++++++++++++++++++++++++++++
>
> char tmp_str[16];
>
> //
> // parse valid time from the filename
> //
>
> c = mode_filename + strlen(mode_filename) - 32;
>
> strncpy(tmp_str, c, 15);
>
> tmp_str[15] = '\0';
>
> t = (int) timestring_to_unix(tmp_str);
>
> if ( !(valid_times.has(t)) ) {
>
> valid_times.add(t);
> }
>
> +++++++++++++++++++++++++++++++++++++
>
> Case, Jonathan (MSFC-VP61)[Other] wrote:
>> Take your time. All I need is you to point us to where the code is to make that change. It should be trivial, but I figure you know where in the code that write statement is a lot faster than if I searched the files!
>>
>> Jonathan
>>
>> ----------------------------------------------------------------------
>> ----------------------------
>> Jonathan Case, ENSCO Inc.
>> NASA Short-term Prediction Research and Transition Center (aka SPoRT
>> Center) 320 Sparkman Drive, Room 3062 Huntsville, AL 35805
>> Voice: 256.961.7504
>> Fax: 256.961.7788
>> Emails: Jonathan.Case-1 at nasa.gov / case.jonathan at ensco.com
>> ----------------------------------------------------------------------
>> ----------------------------
>>
>> "Whether the weather is cold, or whether the weather is hot, we'll weather
>> the weather whether we like it or not!"
>>
>>
>> -----Original Message-----
>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>> Sent: Wednesday, November 04, 2009 8:35 AM
>> To: Case, Jonathan (MSFC-VP61)[Other]
>> Cc: Randy Bullock
>> Subject: Re: FW: You have been unsubscribed from the Met_help mailing
>> list
>>
>> Jon,
>>
>> We were all out of town at a work retreat over the last 2 days, but I'm going through the met_help emails now. I'll take a look at that and get back to you.
>>
>> John
>>
>> Case, Jonathan (MSFC-VP61)[Other] wrote:
>>> John,
>>>
>>> I fully understand. I just wasn't sure what happened because I received an unsubscribing email without explanation.
>>>
>>> If possible, did you have a chance to look into my request for modifying the mode_analysis code slightly to output all 0's for forecast times when no forecast/observed objects occur? This would greatly benefit us in working up a side-by-side dataset for doing a student t significance test.
>>>
>>> Many thanks!
>>> Jonathan
>>>
>>> ---------------------------------------------------------------------
>>> -----------------------------
>>> Jonathan Case, ENSCO Inc.
>>> NASA Short-term Prediction Research and Transition Center (aka SPoRT
>>> Center) 320 Sparkman Drive, Room 3062 Huntsville, AL 35805
>>> Voice: 256.961.7504
>>> Fax: 256.961.7788
>>> Emails: Jonathan.Case-1 at nasa.gov / case.jonathan at ensco.com
>>> ---------------------------------------------------------------------
>>> -----------------------------
>>>
>>> "Whether the weather is cold, or whether the weather is hot, we'll weather
>>> the weather whether we like it or not!"
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway [mailto:johnhg at ucar.edu]
>>> Sent: Wednesday, November 04, 2009 8:29 AM
>>> To: Case, Jonathan (MSFC-VP61)[Other]
>>> Cc: tressa at ucar.edu
>>> Subject: Re: FW: You have been unsubscribed from the Met_help mailing
>>> list
>>>
>>> Jon,
>>>
>>> We're in the process of transitioning our tools for met_help support.
>>> We're moving from a mailing list setup to a ticketing system, which
>>> will help us track the support requests better. I'm checking into what options we have for "publishing" the support requests so that they're open for others to view. I apologize for the inconvenience. We're hoping this change will enable us to have multiple people working to provide met_help support.
>>>
>>> Thanks,
>>> John
>>>
>>> Case, Jonathan (MSFC-VP61)[Other] wrote:
>>>> Hello John and Tressa,
>>>>
>>>> I just noticed the email I received below on 19 OCT, unsubscribing me from the met_help list.
>>>> Just so you know, I didn't do anything to unsubscribe, so I'm wondering what may have happened?
>>>>
>>>> If you could, please re-subscribe me to your met_help list.
>>>>
>>>> Many thanks!
>>>> Jon
>>>>
>>>> --------------------------------------------------------------------
>>>> ------------------------------
>>>> Jonathan Case, ENSCO Inc.
>>>> NASA Short-term Prediction Research and Transition Center (aka SPoRT
>>>> Center) 320 Sparkman Drive, Room 3062 Huntsville, AL 35805
>>>> Voice: 256.961.7504
>>>> Fax: 256.961.7788
>>>> Emails: Jonathan.Case-1 at nasa.gov / case.jonathan at ensco.com
>>>> --------------------------------------------------------------------
>>>> ------------------------------
>>>>
>>>> "Whether the weather is cold, or whether the weather is hot, we'll weather
>>>> the weather whether we like it or not!"
>>>>
>>>> From: met_help-bounces at mailman.ucar.edu
>>>> [mailto:met_help-bounces at mailman.ucar.edu]
>>>> Sent: Monday, October 19, 2009 10:42 AM
>>>> To: Case, Jonathan (MSFC-VP61)[Other]
>>>> Subject: You have been unsubscribed from the Met_help mailing list
>>>>
>>>>
>>>>
More information about the Met_help
mailing list