[Met_help] [rt.rap.ucar.edu #67872] History for stat-analysis valid hours question

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Dear MET team
 
Hope you are doing well at your end.

 
I would like to ask for your help/hints/advice with the following:
 
I want to obtain the performance of some variables in an interval of hours for several days. The "several days" can be a week and probably a month or even longer. That is, to aggregate statistics, say from  18 UTC to 23 UTC for all days in the given simulation period, but can be any reasonable hourly interval:
 
  day1_P_180000
  day1_P_190000
  .
  day1 P _230000
   .
   .
  day2_P_180000
  day2_P_190000
  .
  dayN_P_230000.. 
 
Here "P" denotes the period of interest, and "N" the corresponding days within that period
 
 I have included the "fcst_valid_hour H1 .... fcst_valid_hour H2 .... fcst_valid_hour Hn" flag (where "n" is the final hour in the interval of interest) into the configuration file of stat_analysis, and so far I have obtained some results. This allowed to extract the required hours for all days in that period. Part of the objective is to also obtain the performance by station, so that these flags are in combination with the "-by" flag.
 
One thing to note is that this analysis is conjunction with the one for the entire period. That is, I will be doing it as a separate one.
 
 
My questions are:
 
1) Nevertheless I have some results, I want to know if this procedure is correct so that am not overlooking something that could be important regarding the use of the configuration flags. I read some posts but do not have it so clear at all.
 
 
2) I want to know if the following approach is valid: Instead of using the "fcst_valid_hour" flag, I am thinking in using a combination of wild cards/globbing to work directly in the required files for processing. That is: after the required combination of commands, to end up with something like (the notation is similar as above (P, period; D, day; N, # days; n, # hrs)):
 
    point_stat..L_YYYY_P_D1_H1....stat
    point_stat..L_YYYY_P_D1_H2....stat
    point_stat..L_YYYY_P_D1_Hn....stat
     .
    point_stat..L_YYYY_P_DN_H1....stat
    point_stat..L_YYYY_P_DN_Hn....stat
 
and then make some links to tell stat_analysis where to find these files and further  aggregate the stats just for these files
 
Probably it may help in CPU time since it does not have to read all the files for the original simulation period.
 
Is it equivalent/valid ?
 
 
 
THANK you very much for your time and help
 
Best
 
Victor

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: stat-analysis valid hours question
From: John Halley Gotway
Time: Tue Jun 24 13:03:49 2014

Hello Victor,

I took your recent email and turned it into a new met_help ticket.  In
the future, please send new questions to the "met_help at ucar.edu" email
address.  That will automatically create a new support ticket.  For
existing issues, just reply to the emails you receive keeping the
subject line intact.  The ticketing system keys off the strings in the
subject line - for example, [rt.rap.ucar.edu #67862].

In answer to your questions, yes specifying multiple forecast valid
hours is how I would suggest stratifying your data in order to compute
statistics between 18 and 23Z.  That makes sense to me.

But you're right, you could also stratify the data yourself and store
them in separate files.  Then when you run stat-anlysis, you could
hand it the subset of data you'd like to use.

Both approaches are reasonable.  The second one should run faster
since stat-analysis has less data to process, however, the second one
is also more error-prone.  But either approach should be able to get
the job done.

If there is something more specific you'd like me to look at, please
send the stat-anlysis config file and some sample MET output files
that demonstrate the question or problem you're having.

Thanks,
John Halley Gotway
met_help at ucar.edu

------------------------------------------------
Subject: stat-analysis valid hours question
From: Victor Almanza
Time: Tue Jun 24 18:35:38 2014

Dear John

I sincerely do not know what happened, since I sent a new email for
this new question, I mean, not replying to previous ones and changing
the subject. I remember having rejected emails though. Probably these
previous emails are still there. In any case, apologies for that.

I think this doubt is clarified.

Thank you so much for your time and help.


Best !

Victor


________________________________
 De: John Halley Gotway via RT <met_help at ucar.edu>
Para: halmanset at yahoo.com
Enviado: Martes, 24 de junio, 2014 12:03 P.M.
Asunto: [rt.rap.ucar.edu #67872] stat-analysis valid hours question


Hello Victor,

I took your recent email and turned it into a new met_help ticket.  In
the future, please send new questions to the "met_help at ucar.edu" email
address.  That will automatically create a new support ticket.  For
existing issues, just reply to the emails you receive keeping the
subject line intact.  The ticketing system keys off the strings in the
subject line - for example, [rt.rap.ucar.edu #67862].

In answer to your questions, yes specifying multiple forecast valid
hours is how I would suggest stratifying your data in order to compute
statistics between 18 and 23Z.  That makes sense to me.

But you're right, you could also stratify the data yourself and store
them in separate files.  Then when you run stat-anlysis, you could
hand it the subset of data you'd like to use.

Both approaches are reasonable.  The second one should run faster
since stat-analysis has less data to process, however, the second one
is also more error-prone.  But either approach should be able to get
the job done.

If there is something more specific you'd like me to look at, please
send the stat-anlysis config file and some sample MET output files
that demonstrate the question or problem you're having.

Thanks,
John Halley Gotway
met_help at ucar.edu

------------------------------------------------