<p><b>demirtas</b> 2007-01-18 14:20:10 -0700 (Thu, 18 Jan 2007)</p><p>The first draft version of suite document.<br>
MD<br>
</p><hr noshade><pre><font color="gray">Modified: trunk/wrfvar/technote/suites.tex
===================================================================
--- trunk/wrfvar/technote/suites.tex 2007-01-17 23:23:37 UTC (rev 34)
+++ trunk/wrfvar/technote/suites.tex 2007-01-18 21:20:10 UTC (rev 35)
@@ -1,4 +1,502 @@
\chapter{WRF Suite Scripts}
\label{suites}
-Barker, Demirtas
+Barker, Demirtas\\
+
+An introduction to the WRF Suite Scripts System is given in this chapter.
+The overview section highlights the basis for this system.
+The following sections are organized to give details of the script suite.
+
+\section{Overview}
+
+The suite scripting software allows data assimilation cycles to be run with minimal setup.
+It is designed to use a specific type of directory structure for WRF system
+code and data.
+
+A sample code directory structure might look like:\\
+\$CODE\_DIR
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item wrfvar
+\item wrf
+\item wrfplus
+\item wps
+\item update\_bc
+\item obsproc
+\item wps
+\end{itemize}
+With directory structure for a particular region composed of reference files for all runs,
+and directories for each individual experiment.
+
+A sample data directory structure might look like: \\
+\$DAT\_DIR
+
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item model\_input\_data: ncep, gfs
+\item test\_domain: con200
+\item subdirs for experiment type: noda, nobs, coldstart, cycling, update, and etc.
+\item subdir for re\_occurring data: rc
+\item subdir for forecast results: fc
+\item subdir(s) for gen\_be results: be
+\item subdir(s) for WPS geo results: geo
+\end{itemize}
+
+The script suite system have two categories of scripts: control scripts and components of main suite script. A wrapper script may be needed to set up some environment variables to specify the user's experiment. This script calls {\it da\_run\_job} which creates {\it job.ksh} script on the fly.
+The {\it job.ksh} executes the main {\it da\_run\_suite} script. Depending on what kind of specifications
+defined in the user's wrapper script, the {\it da\_run\_suite} script calls its component scripts
+listed below:
+
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item \it da\_restore\_data\_ncep.ksh
+\item \it da\_restore\_data\_rtobs.ksh
+\item \it da\_run\_wps.ksh
+\item \it da\_run\_real.ksh
+\item \it da\_run\_obsproc.ksh
+\item \it da\_run\_wrfvar.ksh
+\item \it da\_run\_wrf.ksh
+\end{itemize}
+\\
+In a flow-daigram of the script suite can be shown as:\\
+\\
+{\it A wrapper script}--${>}{\it da\_run\_job} --${>}{\it job.ksh}--${>}{\it da\_run\_suite}--${>}{\it components\_of\_suite script} \\
+\\
+The following section provides detailed information about these scripts.
+
+\section{Suite Scripts}
+
+Suite scripts are organized in two categories; control scripts and a suite script which executes run scripts of various components of the WRF system.
+
+\subsection{Control Scripts}
+
+There are two main control scripts:
+
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item {\bf {A Wrapper Script}}\\
+
+A wrapper script may be needed to set up some environment variables to specify the user's experiment. This script calls {\it da\_run\_job} script.\\
+
+\item {\bf {da\_run\_job.ksh}}\\
+
+This script contains the logic for submitting jobs to various queuing systems, and creates {\it job.ksh} script on the fly.
+The {\it job.ksh} script understands IBM's Loadleveller and LSF queuing systems. The {\it da\_run\_job.ksh} script may need tailoring
+for the user's local machine. The {\it job.ksh} executes the main {\it da\_run\_suite} script - which is explained in the following section.\\
+
+\end{itemize}
+
+\subsection{Main Suite Script and Its Components}
+
+The control scripts set required environmental variables and the main suite script, {\bf da\_run\_suite.ksh},
+executes various run scripts of WRF system.
+
+
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item {\bf {da\_run\_suite.ksh}}\\
+
+This script runs data assimilation suites, with WPS, REAL, WRF, WRFVAR, OBSPROC and UPDATEBC components.
+These are given below:\\
+
+\item {\bf {da\_get\_date\_range.ksh}}\\
+
+This script helps for setting some environment variables such as:
+\\
+START\_YEAR\\
+START\_MONTH\\
+START\_DAY\\
+START\_HOUR\\
+END\_YEAR\\
+END\_MONTH\\
+END\_DAY\\
+END\_HOUR\\
+
+\item {\bf {da\_run\_wps.ksh}}\\
+
+This script runs the WPS preprocessing software.
+(The WRF Preprocessing System (WPS) is a replacement for the WRF Standard Initialization (SI) software.)\\
+
+\item {\bf {da\_run\_real.ksh}}\\
+
+This script runs the WRF REAL software.\\
+
+\item {\bf {da\_run\_wrf.ksh}}\\
+
+ This script runs the WRF Model.\\
+
+\item {\bf {da\_run\_obsproc.ksh}}\\
+
+This script runs OBSPROC observation preprocessing software.\\
+
+\item {\bf {da\_run\_wrfvar.ksh}}\\
+
+This script runs the WRFVAR data assimilation software.\\
+
+\item {\bf {da\_update\_bc.ksh}}\\
+
+This script runs the WRF Boundary Conditions update mechanism, used after a regional
+WRFVAR run to combine the analysis with an external lateral boundary conditions file.\\
+
+\item {\bf {da\_restore\_data\_ncep.ksh}}\\
+
+This script restores NCEP data from NCAR's MSS.
+(It is suitable for NCAR users.)\\
+
+\item {\bf {da\_restore\_data\_rtobs.ksh}}\\
+
+This script restores RTOBS (an archive of conventional data) from NCAR's MSS.
+(It is suitable for NCAR users.)\\
+\end{itemize}
+
+\subsection{Ways to Check Suite Runs}
+
+The {\it job.ksh} script produces {\it html} based log files. Utilities such as
+{\it lynx} and {\it mozilla} can be used to display {\it html} files produced
+by the user's run. When using {\it lynx}, some instructions appear on the screen.
+A user may simply look into anything that is bold.\\
+\\
+{\bf Examples for {\it lynx} and {\it mozilla}:}\\
+\\
+{\it lynx \$/data/con200/trunk\_rsl\_xlf\_bluesky\_noobs\_1\\
+\\
+mozilla \$/data/con200/trunk\_rsl\_xlf\_bluesky\_noobs\_1}\\
+\\
+where trunk\_rsl\_xlf\_bluesky\_noobs\_1 is the directory that contains the user's run.\\
+
+\section{Run Types}
+
+The suite scripts may be used to run the following types of runs:
+
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item No data assimilation (NODA) runs
+\item Cold start runs
+\item Cycling runs
+\end{itemize}
+
+\subsection{NO DA runs}
+
+This type of run contains: WPS, REAL and WRF.\\
+\\
+A wrapper script may be set to contain following settings:\\
+RUN\_WPS=\$\{RUN\_WPS:-true\}\\
+RUN\_REAL=\$\{RUN\_REAL:-true\}\\
+RUN\_WRF=\$\{RUN\_WRF:-true\}\\
+\\
+{\bf WPS:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item Terrain data: wps\_geog (see WPS)
+\item Model data: NCEP's various data sets
+\item namelist.wps
+\end{itemize}
+{\bf Output files created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item geogrid: /data/test\_domain/geo/geo\_em.d01.nc
+\item ungrib: /data/test\_domain/name\_of\_experiment/wps/working/FILE:datestamp
+\item metgrid: /data/test\_domain/rc/met\_em.d01\_datestamp}
+\end{itemize}
+{\bf WRF\_Real:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item namelist.input
+\item /data/test\_domain/rc/met\_em.d01\_datestamp
+\end{itemize}
+ {\bf Output files created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item /data/test\_domain/rc/DATE/wrfinput (initial conditions for WRF )
+\item /data/test\_domain/rc/DATE/wrfbdy (lateral boundary conditions for WRF)
+\end{itemize}
+\\
+{\bf WRF:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item namelist.input
+\item Various physics tables/files
+\item /data/test\_domain/rc/DATE/wrfinput (initial conditions for WRF)
+\item /data/test\_domain/rc/DATE/wrfbdy (lateral boundary conditions for WRF)
+\end{itemize}
+\\
+ {\bf Output files created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item /data/test\_domain/name\_of\_experiment/fc/DATE/wrfout\_d01\_datestamp
+\end{itemize}
+\\
+\subsection{COLD START runs}
+
+These runs involve: WPS, REAL, OBSPROC, WRF-VAR and UPDATE\_BC.\\
+\\
+A wrapper script may be set to contain following settings:\\
+RUN\_WPS=\$\{RUN\_WPS:-true\}\\
+RUN\_REAL=\$\{RUN\_REAL:-true\}\\
+RUN\_OBSPROC=\$\{RUN\_OBSPROC:-true\}\\
+RUN\_WRFVAR=\$\{RUN\_WRFVAR:-true\}\\
+RUN\_UPDATE\_BC=\$\{RUN\_UPDATE\_BC:-true\}\\
+\\
+WPS and REAL runs are as in no obs runs. \\
+\\
+{\bf OBSPROC:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item namelist (namelist.3dvar\_obs)
+\item /data/test\_domain/ob/DATE/obs.2006080806
+\end{itemize}
+\\
+{\bf Output files created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item /data/test\_domain/ob/DATE/ob.ascii
+\end{itemize}
+\\
+{\bf WRF-Var:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item namelist.input
+\item DA\_FIRST\_GUESS /rc/2006080806/wrfinput\_d01 (output from REAL)\\
+ (wrfinput\_d01 linked as /rc/2003050100/wrfinput\_d01)\\
+ (fg01 linked as /rc/2006080806/wrfinput\_d01)
+\item ob01.ascii linked as /ob/2003050100/ob.ascii (output from OBS\_PROC)
+\item DA\_BACK\_ERRORS /be/gen\_be.NMC.dat (output from "gen\_be")\\
+ (fort.34 linked as /be/gen\_be.NMC.dat)
+\end{itemize}
+\\
+{\bf Output files created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_ANALYSIS /fc/2006080806/analysis
+\item Text type of output files: cost\_fn, grad\_fn, statistics
+\end{itemize}
+\\
+{\bf UPDATE\_BC:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item parame.in
+\item DA\_ANALYSIS /fc/2006080806/analysis\\
+ (wrfvar\_output linked as /fc/2003050100/analysis)
+\item DA\_FIRST\_GUESS /rc/2006080806/wrfinput\_d01
+\item BDYIN /rc/2006080806/wrfbdy\_d01
+\end{itemize}
+\\
+{\bf Output files created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item BDYOUT /fc/2006080806/wrfbdy\_d01
+\end{itemize}
+\\
+\subsection{CYCLING runs}
+
+These runs involve: WPS, REAL, OBSPROC, WRF-VAR, UPDATE\_BC and WRF.\\
+\\
+A wrapper script may be set to contain following settings:\\
+RUN\_WPS=\$\{RUN\_WPS:-true\}\\
+RUN\_REAL=\$\{RUN\_REAL:-true\}\\
+RUN\_OBSPROC=\$\{RUN\_OBSPROC:-true\}\\
+RUN\_WRFVAR=\$\{RUN\_WRFVAR:-true\}\\
+RUN\_UPDATE\_BC=\$\{RUN\_UPDATE\_BC:-true\}\\
+RUN\_WRF=\$\{RUN\_WRF:-true\}\\
+CYCLING=\$\{CYCLING:-true\}\\
+\\
+{\bf \It First Step:}\\
+WPS, REAL and OBSPROC runs are as in cold start runs. \\
+{\bf WRF-VAR:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_FIRST\_GUESS /rc/2006080806/wrfinput\_d01 (output from REAL)\\
+ (wrfinput\_d01 linked as /rc/2006080806/wrfinput\_d01)
+\item DA\_BACK\_ERRORS /be/gen\_be.NMC.dat (output from "gen\_be")
+\item OB\_DIR /ob/date\_stamp/ob.ascii (output from OBS\_PROC)
+\end{itemize}
+\\
+{\bf Output data created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_ANALYSIS /fc/2006080806/analysis
+\end{itemize}
+\\
+{\bf UPDATE\_BC:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_ANALYSIS: /fc/2006080806/analysis (output from WRF-VAR)
+\item DA\_FIRST\_GUESS: /rc/2006080806/wrfinput\_d01 (output from REAL)
+\item BDYIN: /rc/2006080806/wrfbdy\_d01 (output from REAL)
+\end{itemize}
+{\bf Output data created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item BDYOUT: /fc/2006080806/wrfbdy\_d01 (output from UPDATE\_BC)
+\end{itemize}
+\\
+{\bf WRF:}\\
+It is important to note that in cycling runs {\It wrfinput} and {\It wrfinput} files are NOT obtained from REAL.\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item WRF\_INPUT /fc/2006080806/analysis (output from WRF-VAR)
+\item WRF\_BDY /fc/2006080806/wrfbdy\_d01 (output from UPDATE\_BC)
+\end{itemize}
+\\
+{\bf Output data created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item wrfout\_d01\_datestamp
+\end{itemize}
+\\
+{\bf \It Other Steps:}\\
+ \\
+{\bf WRF-VAR:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_FIRST\_GUESS /rc/2006080806/wrfout\_d01\_datestamp (output from WRF)\\
+\item DA\_BACK\_ERRORS /be/gen\_be.NMC.dat (output from "gen\_be")
+\item OB\_DIR /ob/date\_stamp/ob.ascii (output from OBS\_PROC)
+\end{itemize}
+\\
+{\bf Output data created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_ANALYSIS /fc/2006080806/analysis
+\end{itemize}
+\\
+{\bf UPDATE\_BC:}\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item DA\_ANALYSIS: /fc/2006080806/analysis (output from WRF-VAR)
+\item DA\_FIRST\_GUESS: /rc/2006080806/wrfout\_d01\_datestamp (output from WRF)
+\item BDYIN: /rc/2006080806/wrfbdy\_d01 (output from REAL)
+\end{itemize}
+{\bf Output data created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item BDYOUT: /fc/2006080806/wrfbdy\_d01 (output from UPDATE\_BC)
+\end{itemize}
+\\
+{\bf WRF:}\\
+It is importnant to note that in cycling runs {\It wrfinput} and {\It wrfinput} files are NOT obtained from REAL.\\
+\\
+{\bf Input files required:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item WRF\_INPUT /fc/2006080806/analysis (output from WRF-VAR)
+\item WRF\_BDY /fc/2006080806/wrfbdy\_d01 (output from UPDATE\_BC)
+\end{itemize}
+\\
+{\bf Output data created:}
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item wrfout\_d01\_datestamp
+\end{itemize}
+\\
+{\bf Final Results of Cycling Run:}
+ /fc/2006080806
+\begin{itemize}\setlength{\parskip}{-4pt}
+\item analysis (output from WRF-VAR)
+\item wrfbdy\_d01 (output from UP\_DATE)
+\item wrfout\_d01\_2006-08-08\_06:00:00 (output from WRF)
+\item wrfout\_d01\_2006-08-08\_12:00:00 (output from WRF)
+\end{itemize}
+
+\section{Environment Variables}
+
+The suites are controlled by environment variables.
+\\
+% Requires the booktabs if the memoir class is not being used
+\begin{table}[htbp]
+% \centering
+ %\topcaption{Table captions are better up top} % requires the topcapt package
+ \begin{tabular}{@{} lll @{}} % Column formatting, @{} suppresses leading/trailing space
+ \toprule
+ \multicolumn{2}{c}{\bf Environment Variables} \\
+ \\
+ \cmidrule % Partial rule. (r) trims the line a little bit on the right; (l) & (lr) also possible
+ {\bf Variable} & {\bf Default} & {\bf Description}\\
+ \\
+\midrule
+ REL\_DIR & \$HOME/trunk & Top level for\\
+ & & a collection of code builds\\
+ WRFVAR\_DIR & \$REL\_DIR/wrfvar & Top level for wrfvar build\\
+ WRF\_DIR & \$REL\_DIR/wrf &Top level for WRF build\\
+ WRF\_NL\_DIR & \$REL\_DIR/wrf\_nl & Top level for WRF NL build\\
+ WPS\_DIR & \$REL\_DIR/wps &Top level for WPS reconfiguration tool\\
+ DATE & 2003010112 & Current date\\
+ INITIAL\_DATE & 2003010100 & Date a suite starts\\
+ FINAL\_DATE & 2003020100 & Date a suite ends\\
+ START\_DATE & 2003010109 & Start of a date range used locally. \\
+ & & No user control\\
+ END\_DATE & 2003010115 & End of a date range used locally.\\
+ & & No user control\\
+ PREV\_DATE & 2003010106 & Previous data in cycle. No user control\\
+ NEXT\_DATE & 2003010118 & Next date in cycle. No user control\\
+ DAT\_DIR & \$HOME/data & Top level for all test data\\
+ REGION & con200 & Region of the globe\\
+ REG\_DIR & \$DAT\_DIR/\$REGION& Top level for test data for a particular region\\
+ OB\_DIR & \$REG\_DIR/ob & Observation directory\\
+ BE\_DIR & \$REG\_DIR/be & Background error directory\\
+ RC\_DIR & \$REG\_DIR/rc & Model data reconfigured with WPS, REAL,\\
+ & & often static\\
+ FC\_DIR & \$EXP\_DIR/fc & Model data from WRFVAR and WRF\\
+ WPS\_INPUT\_DIR & \$RC\_DIR & Input for WPS\\
+ WRF\_NAMELIST & \$WRF\_DIR & Force a particular namelist for WRF\\
+ & /test/em\_real/namelist.input &\\
+ NCEP\_DIR & \$DAT\_DIR/ncep & NCEP global forecasts\\
+ DA\_FIRST\_GUESS & \$FC\_DIR & wrfvar "first guess" input\\
+ & /$DATE/wrfinput\_d01 &\\
+ DA\_ANALYSIS & \$FC\_DIR/$DATE/analysis & wrfvar analysis\\
+ DA\_BOUNDARIES & \$FC\_DIR & wrfvar boundaries input\\
+ & /$DATE/wrfbdy\_d01 &\\
+ DA\_BACK\_ERRORS & \$BE\_DIR & wrfvar background errors\\
+ & /gen\_be.NMC.dat &\\
+ \bottomrule
+ \end{tabular}
+% \caption{Environment variables.}
+% \label{tab:booktabs}
+\end{table}
+
+% Requires the booktabs if the memoir class is not being used
+\begin{table}[htbp]
+% \centering
+ %\topcaption{Table captions are better up top} % requires the topcapt package
+ \begin{tabular}{@{} lll @{}} % Column formatting, @{} suppresses leading/trailing space
+ \toprule
+ \multicolumn{2}{c}{\bf Environment Variables} \\
+ \\
+ \cmidrule % Partial rule. (r) trims the line a little bit on the right; (l) & (lr) also possible
+ {\bf Variable} & {\bf Default} & {\bf Description}\\
+ \\
+
+ RTTOV & \$HOME & RTTOV\\
+ & /rttov/rttov85\_{compiler} &\\
+ CRTM & \$HOME & CRTM\\
+ & /crtm/crtm\_{compiler} &\\
+ DA\_RTTOV\_COEFFS & \$RTTOV/rtcoef\_rttov7 & \\
+ NUM\_PROCS & 1 & Number of processors for parallel jobs\\
+ EXPT & test & Identifier of experiment for this region\\
+ EXP\_DIR & \$REG\_DIR/\$EXPT & Top level directory for experimental run.\\
+ & & Does not change\\
+ RUN\_DIR & \$EXP\_DIR/\$DATE/{wrfvar} & Top level for each component.
+ & & Changes during a suite run.\\
+ WORK\_DIR & \$RUN\_DIR/working & Working directory for run,\\
+ & & can be removed after run using CLEAN\\
+ CLEAN & false & Remove \$WORK\_DIR after run\\
+ DUMMY & false & A bogus run test.\\
+ RUN\_CMD & mpirun -np \$NUM\_PROCS & Command to run executable.\\
+ & -nolocal -machinefile \$HOSTS} & Leave blank for non mpi.\\
+ HOSTS & \$PWD/hosts & hosts file for MPI runs\\
+ NL\_* & & Registry produced namelist variables\\
+ CHECK\_SVNVERSION & true & Check subversion release,
+ & & slows down job submission,\\
+ & & but is useful to document runs\\
+ LL\_* & & Loadleveller control\\
+ LL\_NODE\_USAGE & shared & Whether to share nodes with other jobs\\
+ & & (shared/non\_shared).\\
+ LL\_PTILE & 8 & Number of processors per node.\\
+ & & Set to 16 to take advantage\\
+ & & of SMT on bluevista\\
+ LSF\_* & & LSF control\\
+ LSF\_MAX_RUNTIME & 01:30:00 & Maximum wallclock time.\\
+ LSF\_EXCLUSIVE & -x & Whether to share nodes with other jobs\\
+ & & (-x to use, blank to not)\\
+ POE & false & Use the poe job submission system.\\
+ MP\_SHARED\_MEMORY & YES & IBM MPI optimisation\\
+ \bottomrule
+ \end{tabular}
+% \caption{Environment variables.}
+% \label{tab:booktabs}
+\end{table}
+
+
+
</font>
</pre>