<p><b>kavulich@ucar.edu</b> 2013-05-09 20:52:55 -0600 (Thu, 09 May 2013)</p><p>-rm obsolete file be_modeling.tex<br>
-include radar chapter <br>
-Additions covering parallelization, background error, and general data assimilation stuff<br>
</p><hr noshade><pre><font color="gray">Modified: trunk/wrfvar/3DVAR_technote/3dvar.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/3dvar.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/3dvar.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -15,16 +15,109 @@
Metgrid interpolates meteorological data to the WRF domain. First, wind components are rotated to the WRF domain, then interpolated to their native grid (for staggering).
There are several possible interpolation schemes. 4-point bilinear, 16-point overlapping parabolic, 4- or 16-point average, etc (See the WRF User's Guide for more information \citep{WangEA2013})
+\subsection{Background error statistics}
+\label{be}
+The background error covariance matrix (${\bf B_0}$) describes the probability density function (PDF) of forecast errors for a given model setup. Ideally, it represents the covariance of the difference between the model forecast and the true atmospheric state \citep{TempRizvi2010}. Providing a good background error is an essential part of successful data assimilation.
+
+\subsubsection{Background error options}
+\label{be-options}
+There are three background error options available in WRFDA, which can be chosen by changing the namelist variable \texttt{cv\_options} in \texttt{\&wrfvar7}:
+
+\begin{enumerate}
+\item \texttt{cv\_options=3}; the generic NCEP Background Error model
+\item \texttt{cv\_options=5}; the default and recommended NCAR Background Error model
+\item \texttt{cv\_options=6}; the option for Multivariate Background Error (MBE) statistics
+\end{enumerate}
+
+There are other options corresponding to the missing numbers, but they are of no interest to our applications: Option 4 is equivalent to Option 5 except it is used for the Global WRF, and so horizontal correlations are computed in spectral space. Options 1 and 2 are BE models used with MM5 and very early versions of WRFDA. All of these options are now considered obsolete \citep{SkamarockEA2005,TempRizvi2003}.
+
+Option 3, the generic NCEP Background Error model, is quite flexible in that it can be used immediately upon the installation of WRFDA with any domain. The background error statistics for this method were generated using the NMC method applied to the 48–24 hour differences of GFS forecasts over the course of one year \citep{TempWu2003}. These statistics are packaged with the WRFDA system in the binary file \texttt{WRFDA/var/run/be.dat.cv3}. Although this background error generation method is the easiest, it is recommended that it only be used for testing and debugging purposes, for several reasons:
+
+\begin{enumerate}
+\item the 48\textendash 24 hour NMC method is known to introduce diurnal biases \citep{NEED_CITATION}
+\item interpolation from the native grid to the user’s domain will undoubtedly introduce errors
+\item the ideal background error statistics are known to change over the course of a year, so BE statistics are best generated for their representative season.
+\end{enumerate}
+
+However, as a quick check of one’s experiment, this method is useful as it is domain-independent and computationally inexpensive.
+
+Option 5, the NCAR Background Error model, is the default background error generation method.
+
+\subsubsection{gen\_be}
+\label{gen_be}
+
+\texttt{gen\_be} is a built-in WRFDA utility intended to derive background error statistics for WRFDA experiments. It is broken down into several stages:
+
+Stage 0: Read the input data set. This data set consists of model outputs chosen by the user either at consecutive times with regular intervals or as a set of ensemble forecasts. From these forecasts a difference field is computed for each of 5 control variables: the streamfunction (psi), potential velocity (chi), temperature, relative humidity, and surface pressure \citep{TempGuo2005}.
+
+Stage 1: Create bins based on latitude and remove bias from the difference fields
+
+Stage 2: Calculate the regression coefficients for the balanced potential velocity, temperature, and surface pressure variables with the streamfunction for each bin, then produce the unbalanced components of the potential velocity, temperature, and surface pressure.
+
+Stage 3: Compute the local (per bin) and global eigenvectors and eigenvalues for the control variables (the streamfunction, relative humidity, and unbalanced versions of potential velocity, temperature, and surface pressure).
+
+Stage 4: Compute the horizontal scale length used in the recursive filter for regional applications, or the power spectrum for the global applications, for each of the control variables.
+
+Diags: Gather the results from stages 2–4, write the background error file (\texttt{be.dat}).
+
+
+
\section{Minimization}
\label{minim}
+
+As mentioned earlier, the essential problem of WRFDA is minimizing a cost function, which purports to represent the difference between the first guess and the true atmospheric state. The cost function listed earlier is actually a simplification of the full cost function we are attempting to solve; the full cost function is as follows:
+
+\begin{equation}
+ J\left[{\bf x}\left(t_{0}\right)\right] = \frac{1}{2} \left[{\bf x}\left(t_{0}\right) - {\bf x^b}\left(t_{0}\right)\right]^{\textrm{T}}
+ {\bf B}^{-1}_{0}\left[{\bf x}\left(t_{0}\right) - {\bf x^b}\left(t_{0}\right)\right] +
+ \frac{1}{2}\sum_{i=0}^n\left[{\bf y}_{i} - {\bf y^o}_{i}\right]^{\textrm{T}}{\bf R}^{-1}_{i}\left[{\bf y}_{i} - {\bf y^o}_{i}\right]
+\end{equation}
+
+\begin{equation}
+ J({\bf x}) = J_b({\bf x}) + J_o({\bf x}) = \frac{1}{2}({\bf x} - {\bf x^b})^T{\bf B}^{-1}({\bf x} - {\bf x^b}) +
+ ({\bf y} - {\bf y^o})^T({\bf E} + {\bf F})^{-1}({\bf y} - {\bf y^o})
+\end{equation}
+
+The following sections describe each of the terms in the above equation in detail. \citep{SkamarockEA2008,IdeEA1997}
+
+The full cost function is actually a combination of two cost functions that we want to minimize: $J_b$ is the background cost function, or the cost function associated with errors in the background (first guess) state. $J_o$ is the observation cost function, or the cost function associated with errors in the observations and the errors accumulated from transforming them onto the model grid. Mathematically, $J_b$ measures the weighted sum of squares of distances between the analysis state and the background, and $J_o$ measures the weighted sum of squares of distances between the observations (${\bf y^o}$) and the equivalent values in model space (${\bf y}$) \citep{IdeEA1997}.
+
+%%%THIS SECTION NEEDS CLEANUP AND EXPANSION
+
+The analysis is the unknown of the cost function; this is the model state which we are trying to approach in our cost function minimization. Ideally, this should be the for which , but this may not be possible, and in practice is impractically expensive from a computational standpoint. So in practice is the model state for which is sufficiently minimized.
+
+. The background or first guess is the initial state of the model before data assimilation. For WRFDA this is either a WRF input file as prepared by WPS and real.exe, or a WRF output file from a WRF forecast.
+
+. The background error covariance matrix is
+
+Representivity error is an estimate of inaccuracies introduced by the observation operator used to transform the gridded analysis ()
+
+%%%
+
+WRFDA minimizes the cost function using the conjugate gradient method (CGM). This method is described in an easily understood yet appropriately detailed summary in \citet{Shewchuk1994}.
+
+
+The matrix is positive semi-definite, and thus its eigenvalues are all non-negative. This is essential to the minimization process, as when is applied in an iterative process, it will continually decrease in value.
+
+
+%%%THIS PARAGRAPH MUST BE UPDATED (PERHAPS REMOVED ALTOGETHER?): BUDDY CHECK IS OBSOLETE
Unlike the QNM technique, the CGM method restricts WRF-Var's inner loop to be completely linear. This limitation is dealt with through the inclusion of an outer loop in WRF-Var, the purpose of which is to iterate towards nonlinear solutions (e.g., observation operators, balance constraints, and the forecast itself in 4D-Var) using the WRF-Var analysis from the previous iteration as new first guess. The outer loop is also used as a form of variational quality control as follows: observations are rejected if the magnitude of the observation minus first guess differences are larger than a specified threshold (typically several times the observation error standard deviation). This {\it errormax} test implicitly assumes the first guess is accurate. However, in cases when this assumption breaks down (i.e. in areas of large forecast error), there is a danger that good observations might be rejected in areas where they are most valuable. The outer loop alleviates this effect by allo
wing observations rejected in previous iterations to be accepted if their updated observation minus analysis differences pass the errormax QC check in in subsequent outer loops. The assimilation of nearby observations in previous iterations essentially provides a ``buddy check" to the observation in question.
+
\section{Output}
\label{output}
We should detail ALL the output files and what the options mean
+\section{Cycling mode}
+\label{cycling}
+
+\subsection{update\_bc}
+\label{update_bc}
+
+Boundary conditions are 5-gridpoints deep, nudge the interior 4 gridpoints
+
\section{Radar Data Assimilation}
\label{radar}
Modified: trunk/wrfvar/3DVAR_technote/acronyms.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/acronyms.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/acronyms.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -12,6 +12,8 @@
\\ 4DVAR & Four-dimensional Variational data assimilation &
\\ NCAR & National Center for Atmospheric Research & ``A federally funded research and development center devoted to service, research and
education in the atmospheric and related sciences'' (\url{http://ncar.ucar.edu/})
+\\ NMC & National Meteorological Center & In the context of background error, the ``NMC method'' refers to the background error generation
+ method described in \citet{Parrish_Derber1992}
\\ netCDF & network Common Data Form & A machine-independent file format for storing scientific data.
\url{http://www.unidata.ucar.edu/software/netcdf}
\\ MM4 & Fourth-Generation NCAR / Penn State Mesoscale Model & MM5's predecessor model \citep{AnthesEA1987}
Modified: trunk/wrfvar/3DVAR_technote/be.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/be.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/be.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -23,7 +23,7 @@
thus permitting the matrix ${\bf B}$ to be block-diagonalized. The major change in WRF-Var V3.0
is to simplify the control variable transform ${\rm U_p}$ to perform a simple statistical regression as described in subsection
(\ref{var-b}) below. Testing in numerous applications has shown
-a general improvement of forecasts scores using this definition of balance, as compared to the dynamical geostrophic//cyclostrophic balance constraint defined in \citet{barker03}.
+a general improvement of forecasts scores using this definition of balance, as compared to the dynamical geostrophic//cyclostrophic balance constraint defined in \citet{BarkerEA2003}.
\section{Overview}
@@ -158,7 +158,7 @@
\subsection{gen\_be\_stage3 - Eigenvectors/values of Vertical Error Covariances}
-The gen\_be\_stage3 program calculates the statistics required for the vertical component of the control variable transform of WRF-Var. This involves the projection of 3D fields on model-levels onto empirical orthogonal functions (EOFs) of the vertical component of background error covariances \cite{barker04}.
+The gen\_be\_stage3 program calculates the statistics required for the vertical component of the control variable transform of WRF-Var. This involves the projection of 3D fields on model-levels onto empirical orthogonal functions (EOFs) of the vertical component of background error covariances \cite{BarkerEA2004}.
The {\it gen$\_$be} code calculates both domain-averaged and local
@@ -174,7 +174,7 @@
statistics are included in the dataset supplied to WRF-Var, allowing
the choice between homogeneous (domain-averaged) or local
(inhomogeneous) background error variances and vertical correlations
-to be chosen at run time \cite{barker04}.
+to be chosen at run time \cite{BarkerEA2004}.
Having calculated and stored eigenvectors and eigenvalues, the final
part of {\it gen$\_$be$\_$stage3} is to project the entire sequence of
@@ -194,11 +194,11 @@
In global applications (gen\_be\_stage4\_global). a spectral decomposition of the grid-point data is performed, and power spectra computed for each variable and vertical mode. Details of the spectral technique used to project gridpoint fields to spectral modes are given in Appendix B together with a description of the power spectra computed from the spectral modes.
-In regional applications (gen\_be\_stage4\_regional), a recursive filter is used to provide the horizontal correlations (\cite{barker04}). The error covariance inputs to the recursive filter is a correlation lengthscale (provided by gen\_be\_stage4\_regional), and a WRF-Var namelist parameters to define the correlations shape (the number of passes through the recursive filter). The calculation of horizontal lengthscales in gen\_be\_stage4\_regional is described in section 8c of (\cite{barker04}). Note: This is the most expensive part of the entire gen\_be process.
+In regional applications (gen\_be\_stage4\_regional), a recursive filter is used to provide the horizontal correlations (\cite{BarkerEA2004}). The error covariance inputs to the recursive filter is a correlation lengthscale (provided by gen\_be\_stage4\_regional), and a WRF-Var namelist parameters to define the correlations shape (the number of passes through the recursive filter). The calculation of horizontal lengthscales in gen\_be\_stage4\_regional is described in section 8c of (\cite{BarkerEA2004}). Note: This is the most expensive part of the entire gen\_be process.
Input: Projected 3D (i,j,m) control variable fields: $\psi$, $\chi_u$, $T_u$, and $r$, and $p_{su}$(i,j). (output from stage 3 and 2a)
-Processing (regional): Perform linear regression of horizontal correlations to calculate recursive filter lengthscales (see \cite{barker04}).
+Processing (regional): Perform linear regression of horizontal correlations to calculate recursive filter lengthscales (see \cite{BarkerEA2004}).
Processing (global): Perform horizontal spectral decomposition, and compute power spectra for each field/mode (see Appendix B).
Deleted: trunk/wrfvar/3DVAR_technote/be_modeling.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/be_modeling.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/be_modeling.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -1,4 +0,0 @@
-\chapter{Background Error Covariance Modeling}
-\label{be_modeling}
-
-Barker, Rizvi
\ No newline at end of file
Modified: trunk/wrfvar/3DVAR_technote/description.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/description.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/description.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -85,6 +85,7 @@
\include{obs}
\include{be}
\include{3dvar}
+\include{radar}
\include{se}
%\include{asf}
%
Modified: trunk/wrfvar/3DVAR_technote/intro.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/intro.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/intro.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -1,16 +1,16 @@
\chapter{Introduction}
\label{intro}
-This overview supplements the original description of the three-dimensional variational (3D-Var) algorithm found in \citet{BarkerEA04}.
+This overview supplements the original description of the three-dimensional variational (3D-Var) algorithm found in \citet{BarkerEA2004}.
Data assimilation is the technique by which \textbf{observations} are combined with an NWP product (the \textbf{first guess} or background forecast) and their respective error statistics to provide an improved estimate (the \textbf{analysis}) of the atmospheric (or oceanic, Jovian, etc.) state. Variational (Var) data assimilation achieves this through the iterative minimization of a prescribed cost (or penalty) function. Differences between the analysis and observations/first guess are penalized (damped) according to their perceived error.
The MMM Division of NCAR supports a unified (global/regional, multi-model, 3/4D-Var) model-space data assimilation system (WRFDA) for use by the NCAR staff and collaborators, and is also freely available to the general community, together with further documentation, test results, plans etc., from the WRFDA web-page (\url{http://www.mmm.ucar.edu/wrf/users/wrfda/index.html}).
-Various components of the WRFDA system are shown in blue in the sketch below, to-gether with their relationship with the rest of the WRF system.
+Various components of the WRFDA system are shown in blue in the sketch below, together with their relationship with the rest of the WRF system.
\section{Overview of Variational Data Assimilation}
\label{overview}
-The basic goal of any variational data assimilation system is to produce an optimal estimate of the true atmospheric state at analysis time through iterative solution of a prescribed cost-function \citep{ide97}:
+The basic goal of any variational data assimilation system is to produce an optimal estimate of the true atmospheric state at analysis time through iterative solution of a prescribed cost-function \citep{IdeEA1997}:
\begin{equation}
J({\bf x}) = J_b({\bf x}) + J_o({\bf x}) = \frac{1}{2
@@ -20,15 +20,15 @@
\label{cost_function}
\end{equation}
-The variational problem can be summarized as the iterative minimization of (\ref{cost_function}) to find the analysis state ${\bf x}$ that minimizes $J({\bf x})$. This solution represents the {\it a posteriori} maximum likelihood (minimum variance) estimate of the true state of the atmosphere given the two sources of {\it a priori} data: the first guess (or background) ${\bf x^{b}}$ and observations ${\bf y^{o}}$ \citep{lorenc86}. The fit to individual data points is weighted by estimates of their errors: ${\bf B}$, ${\bf E}$, and ${\bf F}$ are the background, observation (instrumental), and representivity error covariance matrices, respectively. Representivity error is an estimate of inaccuracies introduced in the observation operator $H$ used to transform the gridded analysis ${\bf x}$ to observation space ${\bf y}=H({\bf x})$ for comparison against observations. This error will be resolution dependent and may also include a contribution from approximations (e.g., lineariz
ations) in $H$.
+The variational problem can be summarized as the iterative minimization of (\ref{cost_function}) to find the analysis state ${\bf x}$ that minimizes $J({\bf x})$. This solution represents the {\it a posteriori} maximum likelihood (minimum variance) estimate of the true state of the atmosphere given the two sources of {\it a priori} data: the first guess (or background) ${\bf x^{b}}$ and observations ${\bf y^{o}}$ \citep{lorenc86}. The fit to individual data points is weighted by estimates of their errors: ${\bf B}$, ${\bf E}$, and ${\bf F}$ are the background, observation (instrumental), and representivity error covariance matrices, respectively. Representivity error is an estimate of inaccuracies introduced in the observation operator $H$ used to transform the gridded analysis ${\bf x}$ to observation space ${\bf y}=H({\bf x})$ for comparison against observations. This error will be resolution dependent and may also include a contribution from approximations (e.g., lineariz
ation) in $H$.
\section{Overview of WRFDA}
\label{overview-wrfda}
-As described in \citet{barker04}, the particular variational data assimilation algorithm adopted in WRF-Var is a model-space, incremental formulation of the variational problem. In this approach, observations, previous forecasts, their errors, and physical laws are combined to produce analysis increments ${\bf x^{a'}}$, which are added to the first guess ${\bf x^{b}}$ to provide an updated analysis.
+As described in \citet{BarkerEA2004}, the particular variational data assimilation algorithm adopted in WRF-Var is a model-space, incremental formulation of the variational problem. In this approach, observations, previous forecasts, their errors, and physical laws are combined to produce analysis increments ${\bf x^{a'}}$, which are added to the first guess ${\bf x^{b}}$ to provide an updated analysis.
-Figure \ref{var-sketch} illustrates the relationship between WRF-Var, the various datasets, and the other components of a typical NWP system (here ARW). The WRF-Var assimilation proceeds as described in \citet{barker04}. A number of recent upgrades to the WRF-Var algorithm will be described in Section \ref{var-upgrade}.
+Figure \ref{var-sketch} illustrates the relationship between WRF-Var, the various datasets, and the other components of a typical NWP system (here ARW). The WRF-Var assimilation proceeds as described in \citet{BarkerEA2004}. A number of recent upgrades to the WRF-Var algorithm will be described in Section \ref{var-upgrade}.
%
% Figure ?.? WRFDA flowchart
@@ -48,13 +48,13 @@
\vspace{0.5cm}
-b) Observations ${\bf y^{o}}$--- In the current version of WRF-Var, observations may be supplied either in PREPBUFR format ({\it ob\_format=1}) or an ASCII "little\_r" format ({\it ob\_format=2}). An observation preprocessor (3DVAR$\_$OBSPROC) is supplied with the code release to perform basic quality control, assign "total" observation errors (${\bf R = E+F}$ in Fig. \ref{var-sketch}), and reformat observations from the MM5 {\it little$\_$r} text format into 3D-Var's own text format. Details can be found in \citet{barker03, barker04}.
+b) Observations ${\bf y^{o}}$--- In the current version of WRF-Var, observations may be supplied either in PREPBUFR format ({\it ob\_format=1}) or an ASCII "little\_r" format ({\it ob\_format=2}). An observation preprocessor (3DVAR$\_$OBSPROC) is supplied with the code release to perform basic quality control, assign "total" observation errors (${\bf R = E+F}$ in Fig. \ref{var-sketch}), and reformat observations from the MM5 {\it little$\_$r} text format into 3D-Var's own text format. Details can be found in \citet{BarkerEA2003,BarkerEA2004}.
\vspace{0.5cm}
c) Background error covariances ${\bf B}$--- used to define the spatial and multivariate response of the analysis to an observation. In variational systems, these covariances are typically calculated off-line, and significant tuning is required to optimize performance for a particular application (e.g., \citet{ingleby01, wu02}). The amount of work required to do this satisfactorily is significant, and should not be underestimated. In order to assist the user, WRF developers supply the following: i) a default set of statistics used for the initial set up of a domain; ii) a utility {\it gen$\_$be} (described in Section \ref{var-be}) to process ensembles of forecasts into the appropriate control variable space; and iii) diagnostic routines to assess the accuracy of observation and background error statistics. These routines include both innovation vector-based approaches \citep{hollingsworth86} and variational tuning approaches \citep{desroziers01}.
-Following assimilation of all data, an analysis ${\bf x^{a}}$ is produced that must be merged with the existing lateral boundary conditions ${\bf x^{lbc}}$ in the {\it WRF\_BC} utility (\citet{barker03}). At this stage, the {\it wrfbdy} lateral boundary condition files (${\bf x^{lbc}}$) output of WPS/real is updated to make the lateral boundaries consistent with the analysis, and surface fields (e.g. SST) are also updated in the {\it wrfinput} analysis file.
+Following assimilation of all data, an analysis ${\bf x^{a}}$ is produced that must be merged with the existing lateral boundary conditions ${\bf x^{lbc}}$ in the {\it WRF\_BC} utility (\citet{BarkerEA2003}). At this stage, the {\it wrfbdy} lateral boundary condition files (${\bf x^{lbc}}$) output of WPS/real is updated to make the lateral boundaries consistent with the analysis, and surface fields (e.g. SST) are also updated in the {\it wrfinput} analysis file.
\section{Basic outline of WRFDA}
\label{basic-outline}
Modified: trunk/wrfvar/3DVAR_technote/refs.bib
===================================================================
--- trunk/wrfvar/3DVAR_technote/refs.bib 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/refs.bib 2013-05-10 02:52:55 UTC (rev 431)
@@ -3,6 +3,7 @@
author = {Richard Anthes and Eirh-Yu Hsie and Ying-Hwa Kuo},
series = {NCAR Technical Note},
publisher = {University Corporation for Atmospheric Research},
+ institution = {National Center for Atmospheric Research},
year = {1987},
LCCN = {03174},
language = {en},
@@ -30,6 +31,7 @@
series = {NCAR Technical Note},
abstract = {This report describes the three-dimensional variational (3DVAR) data assimilation system designed and built in the MMM Division of NCAR for use with the MM5 modeling system. This, and additional, documentation can be found online at the MM5 3DVAR web-site: http://www.mmm.ucar.edu/3dvar The 3DVAR system described here was also adopted in June 2001 as the starting point for 3DVAR development for the Weather Research Forecast (WRF) model. This version of the technical documentation focuses on the use of 3DVAR within the MM5 modeling environment.},
publisher = {University Corporation for Atmospheric Research},
+ institution = {National Center for Atmospheric Research},
year = {2003},
LCCN = {03535},
language = {en},
@@ -57,6 +59,7 @@
Sasha Madronich and Francis S. Binkowski and Nelson L. Seaman and David R. Stauffer},
series = {Acidic Deposition: State of Science and Technology},
publisher = {National Acid Precipitation Assessment Program},
+ institution = {The National Acid Precipitation Assessment Program},
volume = {I},
year = {1990},
month = {December},
@@ -70,6 +73,7 @@
series = {NCAR Technical Note},
abstract = {The MM4 system is a series of FORTRAN programs and accompanying C-shell scripts that runs under UNICOS on the Cray Y-MP8/864 at NCAR. Each of these shells is a standalone script which must be submitted in a specific sequence to the Cray. The MM4 package is a flexible modeling system, implying that a good deal of user interaction is required. Each of the major components of the modeling system has a section in this document describing its standard usage; the initial forecast for any particular case involves running them all. Chapters include TERRAIN, DATAGRID, RAWINS, INTERP, GRAPH, MM4, INIT, TRAJEC, VERIFY. Appendix A: Modeling System Data Formats; Appendix B: Modeling System Input Files; Appendix C: Glossary.},
publisher = {University Corporation for Atmospheric Research},
+ institution = {National Center for Atmospheric Research},
year = {1992},
LCCN = {03295},
language = {en},
@@ -84,6 +88,7 @@
series = {NCAR Technical Note},
abstract = {This technical report describes the fifth generation Penn State/NCAR Mesoscale Model, or MM5. It is intended to provide scientific and technical documentation of the model for users. Source code documentation is available as a separate Technical Note (NCAR/TN-392) by Haagenson et al. (1994). The document structure is as follows: In section 2 we describe the governing equations, algorithms, and boundary conditions. This will include the finite difference algorithms and time splitting techniques of both the hydrostatic and the nonhydrostatic equations of motion (hydrostatic and nonhydrostatic solver). All subsequent sections will describe features common to both solvers. Section 3 discusses the mesh-refinement scheme, section 4 the four-dimensional data-assimilation technique, and section 5 focuses on the various physics options.},
publisher = {University Corporation for Atmospheric Research},
+ institution = {National Center for Atmospheric Research},
year = {1994},
LCCN = {03386},
language = {en},
@@ -97,6 +102,7 @@
author = {Eirh-Yu Hsie},
series = {NCAR Technical Note},
publisher = {University Corporation for Atmospheric Research},
+ institution = {National Center for Atmospheric Research},
year = {1987},
LCCN = {03178},
language = {en},
@@ -119,6 +125,18 @@
url = {http://journals.ametsoc.org/doi/pdf/10.1175/2008MWR2577.1},
}
+@article{IdeEA1997,
+ title = {Unified Notation for Data Assimilation: Operational, Sequential and Variational},
+ author = {K. Ide and P. Courtier and M. Ghil and A. C. Lorenc},
+ journal = {Journal of the Meteorological Society of Japan},
+ year = {1997},
+ month = {May},
+ volume = {75},
+ number = {1B},
+ pages = {181--189},
+ url = {http://www.atmos.ucla.edu/~kayo/data/publication/ide_etal_jmsj97.pdf}
+}
+
@article { Parrish_Derber1992,
title = {The National Meteorological Center's Spectral Statistical-Interpolation Analysis System},
author = {David F. Parrish and John C. Derber},
@@ -132,6 +150,18 @@
url = {http://journals.ametsoc.org/doi/abs/10.1175/1520-0493%281992%29120%3C1747%3ATNMCSS%3E2.0.CO%3B2},
}
+@techreport{ Shewchuk1994,
+ author = {Jonathan R. Shewchuk},
+ title = {An Introduction to the Conjugate Gradient Method Without the Agonizing Pain},
+ year = {1994},
+ month = {August},
+ edition = {1 1/4},
+ source = {http://www.cs.cmu.edu/~jrs/jrspapers.html},
+ publisher = {Carnegie Mellon University},
+ institution = {Carnegie Mellon University},
+ address = {Pittsburgh, PA, USA},
+}
+
@techreport { SkamarockEA2005,
title = { A Description of the Advanced Research WRF Version 2},
author = {William Skamarock and Joseph Klemp and Jimy Dudhia and David Gill and Dale Barker and Wei Wang},
@@ -201,6 +231,23 @@
howpublished = {\url{http://web.archive.org/web/20051216013621/http://www.mmm.ucar.edu/wrf/WG4/wrfvar.htm}},
}
+@misc{ TempDelVento2012,
+ author = {Davide Del Vento},
+ title = {Introduction to Parallel Computing with OpenMP and MPI},
+ month = {May},
+ year = {2012},
+ publisher = {NCAR CISL},
+ howpublished = {\url{https://www2.cisl.ucar.edu/sites/default/files/ParallelProgramming-20120524_0.pdf}},
+}
+
+@misc{ TempGuo2005,
+ author = {Yong-Run Guo},
+ title = {Document for Stage0 (WRF model) in Background Error Statistics Calculation},
+ month = {February},
+ year = {2005},
+ howpublished = {\url{http://www.mmm.ucar.edu/wrf/users/wrfda/Docs/NMC_Stage0_for_WRF.pdf}},
+}
+
@misc{ TempHuangEA2005,
author = {Xiang-Yu Huang and Qingnong Xiao and Wei Huang and Dale Barker and Ying-Hwa Kuo and John Michalakes and Zaizhong Ma},
title = {The Weather Research and Forecasting Model Based 4-dimensional Variational Data Assimilation System [WRF-(4D)Var]},
@@ -224,3 +271,24 @@
year = {2003},
howpublished = {\url{http://www.mmm.ucar.edu/mm5/mm5v3/little_rv3.html}},
}
+
+@misc{ TempRizvi2003,
+ author = {S. R. H. Rizvi},
+ title = {Recent Developments in Data Assimilation at NCAR},
+ year = {2003?},
+ howpublished = {\url{http://www.docstoc.com/docs/83548375/Recent-Developments-in-Data-Assimilation-at-NCAR}},
+}
+
+@misc{ TempRizvi2010,
+ author = {S. R. H. Rizvi},
+ title = {WRFDA Background Error Estimation },
+ year = {2010},
+ howpublished = {\url{http://www.mmm.ucar.edu/wrf/users/wrfda/Tutorials/2010_Aug/docs/WRFDA_BE.pdf}},
+}
+
+@misc{ TempWu2003,
+ author = {Wan-Shu Wu},
+ title = {Documentation on CV\_OPTIONS=3 in WRF 3D Var},
+ year = {2003},
+ howpublished = {\url{http://www.mmm.ucar.edu/wrf/WG4/wrf3dvar_cvoptions3.htm}},
+}
Modified: trunk/wrfvar/3DVAR_technote/se.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/se.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/se.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -1,6 +1,63 @@
\chapter{Software Engineering}
\label{se}
+\section{WRF/WRFDA Registry}
+\label{registry}
+WRFDA is integrated into the WRF Software Framework (WSF). The WSF is highly compartmentalized to allow for easy third-party development and to ease portability across different platforms. It can be thought of as being built in three layers: the Driver Layer, the Mediation Layer, and the Model Layer.
+
+WRF, and consequentially WRFDA, uses a registry consisting of an ``active data dictionary'' for managing data structures. This is a database which describes different attributes of the model state, intermediate data formats, and configuration data. It keeps track of, among other things:
+
+\begin{itemize}
+\item Dimensionality, time levels, grid staggering
+\item Associations with physics parameterization
+\item Input/Output (I/O) classification (e.g., history, restart, initial, boundary)
+\item Configuration lists (e.g. Namelists)
+\end{itemize}
+
+At compilation time, a large amount of the WRFDA code is generated dynamically from this registry. Much of this code is for the Driver and Mediation Layer interfaces, inter-processor communications, and run-time configuration information.
+
+WRFDA is built in this hierarchical structure for several reasons. It automates repetitive, error-prone coding which would otherwise have to be done by hand at a great cost of man-hours. It insulates programmers and code from package dependencies which change between different WRFDA versions, minimizing chances for code regressions. In this way, it allows for rapid development of WRFDA and its associated packages. It also serves as additional documentation for different WRFDA processes.
+
+\section{Parallelization}
+\label{parallelization}
+
+Data assimilation is a computationally expensive process, even when extreme measures of simplification are taken. Thus it is practical, nay, desired to run WRFDA using the resources of multiple machines; this process is called parallelization.
+Terminology in parallel computing is difficult to follow, constantly changing, and at times inconsistent, but it is set forth as clearly as possible in this section. \citep{TempDelVento2012}
+
+If you are not interested in the intimate details of parallelization, feel free to skip the following sections.
+
+\subsection{Serial computation}
+\label{serial}
+
+Ultimately, all computing is performed on one or more \textit{processors}, or \textit{central processing units} (CPUs). A processor is defined as ``a device that reads and executes instructions in sequence from a memory device, producing results that are written back to a memory device''. Most modern CPUs are actually composed of several processing units on one chip; these processors are often referred to as cores, hence the marketing terms you may have heard, referencing ``dual-core'' or ``quad-core'' processors. However, in essence these cores behave as independent processors, and so from here on ``processor'' will refer to one processing unit.
+
+The simplest way to run any computer program is in sequence on a single processor. A program is read one instruction at a time by the processor, which computes the result and writes it back to memory before reading and executing the next command in the program. This is known as \textit{serial computation}.
+
+\subsection{Shared-memory parallel computation}
+\label{smpar}
+A node is defined as “a memory device connected to one or more processors”. Although the term “node” refers to many different concepts in computing, in the context of a computer network this is the most common definition, and will be used from here onwards.
+
+Having multiple independent processors introduces the possibility of parallel execution of a program, where each processor can evaluate a different command from the same program concurrently, thus reducing the overall time it takes to execute the full program. Multiple processors on the same node share the same memory, therefore this method of parallelization is known as \textit{shared-memory parallel computation}, or smpar.
+
+Parallelization is not as simple as distributing commands evenly among the available processors, however. Because calculations performed by one processor might change the values of variables used for subsequent calculations, distribution of computation to different processors must be carefully managed. This is achieved using OpenMP, which is an \textit{application programming interface} (API; i.e. an interface to be used by different software components to communicate with each other) defined by an agreed-upon standard by most major computer hardware and software manufacturers. This is important because it fits in with WRFDA’s mission of portability, as OpenMP works with almost any hardware and operating system.
+
+The basic unit of computing is a \textit{thread}; which is essentially a stream of instructions as read from a program in memory, to be executed successively. OpenMP works by the process of \textit{multithreading}, which breaks down a \textit{master thread} into a number of \textit{slave threads}, which can run concurrently. The program must be written to specify which sections of code may be run in parallel, but beyond that much of the distribution of tasks to different processors is handled automatically by the runtime environment (define?).
+
+\subsection{Distributed-memory parallel computation}
+\label{dmpar}
+Due to various hardware issues, it is not practical to have more than a dozen or so processors in a node. For such a computationally expensive application as data assimilation, it is often desirable to have hundreds or even thousands of processors. This is achieved by running the program on a \textit{cluster}, which is a group of nodes connected by a \textit{network}. The only difference between a cluster of single-processor nodes and a node with multiple processors is that processors on different nodes do not share memory with each other. For processors on different nodes to work on the same program, the different nodes must communicate with each other to regulate which node will execute which section of a program. This type of parallelization is known as \textit{distributed-memory parallel computation}, or dmpar.
+
+Because OpenMP is a shared-memory only API, a different parallelization system is needed for distributed-memory applications. This is achieved by using the \textit{Message Passing Interface} (MPI), which is a simple, but most importantly, language-independent communications protocol. MPI was developed beginning from a call to action at the Supercomputing ’92 conference, after which a working group was created, from which MPI-1.0 released in May 1994. Unlike OpenMP, MPI is not standardized by any governing body; however, its utility has allowed it to become the de facto standard for distributed-memory systems, and WRFDA is no different.
+
+\subsection{Distributed memory with shared memory parallel computation}
+\label{sm+dm}
+If running an application on a network with multi-processor nodes, it is possible to combine the two methods of parallelization using both MPI and OpenMP. This is known as \textit{distributed memory with shared memory parallel computation} (dm+sm).
+
+\subsection{Parallelization in WRFDA}
+\label{wrfda-par}
+While WRF supports all four run types, WRFDA does not support shared memory, so is only capable of compiling to run in serial or dmpar.
+
+
\section{Introduction}
\label{se-intro}
@@ -122,12 +179,9 @@
Statistics for the cost and gradient functions, and observation statistics are written
to separate files.
-\section{Parallelization}
-\label{parallelization}
-
\section{Run Directory}
\label{se-run-directory}
Modified: trunk/wrfvar/3DVAR_technote/var.tex
===================================================================
--- trunk/wrfvar/3DVAR_technote/var.tex 2013-05-08 01:07:09 UTC (rev 430)
+++ trunk/wrfvar/3DVAR_technote/var.tex 2013-05-10 02:52:55 UTC (rev 431)
@@ -11,7 +11,7 @@
\label{var-upgrade}
The latest version of WRF-Var (V3.0) contains a number of improvements relative
-to that described in the MM5 3DVAR technical note (\citet{barker03}). These are described below.
+to that described in the MM5 3DVAR technical note (\citet{BarkerEA2003}). These are described below.
It should also be noted that the public release of WRF-Var V3.0 contains only a subset of
the capabilities of the full WRF-Var system. In particular, the direct assimilation of radiances,
hybrid variational/ensemble data assimilation technique, and 4D-Var will be released once
@@ -263,7 +263,7 @@
control variable transform. This calculation involves the projection
of 3D fields on model-levels onto empirical orthogonal functions
(EOFs) of the vertical component of background error covariances
-\citet{barker04}. For each 3D control variable ($\psi$, $\chi_u$,
+\citet{BarkerEA2004}. For each 3D control variable ($\psi$, $\chi_u$,
$T_u$, and $r$), the vertical component of ${\bf B}$, is calculated
and an eigenvector decomposition performed. The resulting eigenvectors
${\bf E}$ and eigenvalues $\Lambda$ are saved for use in WRF-Var.
@@ -277,7 +277,7 @@
statistics are included in the dataset supplied to WRF-Var, allowing
the choice between homogeneous (domain-averaged) or local
(inhomogeneous) background error variances and vertical correlations
-to be chosen at run time \citet{barker04}.
+to be chosen at run time \citet{BarkerEA2004}.
Having calculated and stored eigenvectors and eigenvalues, the final
part of {\it gen$\_$be$\_$stage3} is to project the entire sequence of
3D control variable fields into EOF space ${\bf v_v}=U_{v}^{-1}{\bf
@@ -298,7 +298,7 @@
variable $p_{su}$ data. In contrast, in regional mode, horizontal
correlations are computed between grid-points of each 2D field, binned
as a function of distance. A Gaussian curve is then fitted to the data
-as described in \citet{barker04} to provide correlation lengthscales
+as described in \citet{BarkerEA2004} to provide correlation lengthscales
for use in the recursive filter algorithm.
\section{WRF-Var V3.0 Software Engineering Improvements}
</font>
</pre>