<p><b>pwjones@lanl.gov</b> 2011-09-02 12:23:32 -0600 (Fri, 02 Sep 2011)</p><p><br>
Initial skeleton and draft for the testing document - may want to put this elsewhere soon.<br>
</p><hr noshade><pre><font color="gray">Added: branches/ocean_projects/performance/MPASOceanTesting.pdf
===================================================================
(Binary files differ)
Property changes on: branches/ocean_projects/performance/MPASOceanTesting.pdf
___________________________________________________________________
Added: svn:mime-type
+ application/octet-stream
Added: branches/ocean_projects/performance/MPASOceanTesting.tex
===================================================================
--- branches/ocean_projects/performance/MPASOceanTesting.tex (rev 0)
+++ branches/ocean_projects/performance/MPASOceanTesting.tex 2011-09-02 18:23:32 UTC (rev 977)
@@ -0,0 +1,211 @@
+\documentclass[11pt]{report}
+
+\usepackage{epsf,amsmath,amsfonts}
+\usepackage{graphicx}
+\usepackage{moreverb}
+
+\begin{document}
+
+\title{MPAS Ocean Testing and Testbed Plan: \\
+Requirements and Design}
+\author{Doug Jacobsen \and Phil Jones \and Todd Ringler \and Bill Large
+ \and Gokhan Danabosoglu \and others to be added}
+
+\maketitle
+\tableofcontents
+
+%-----------------------------------------------------------------------
+
+\chapter{Summary}
+
+This document describes an overall test plan for verifying and validating
+the MPAS Ocean model and to a lesser extent, ocean models in general. A
+hierarchy of such tests is described. This document should evolve as
+more tests are defined and when new failure modes are identified, related
+tests should be added to this test suite.
+
+%-----------------------------------------------------------------------
+
+\chapter{Requirements}
+
+\section{Unit Tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Phil Jones) \\
+
+When possible, specific modules or operators should have a defined unit
+test that can test the behavior against an analytic or known result. While
+this may be difficult for more complex operators or parameterizations, it
+should be possible for much of the infrastructure routines and well-defined
+operations like the equation of state for which reference values are available.
+
+\section{Intermediate tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Well-defined test cases should be defined for partial or simpler configurations
+that test more than one aspect of the code base. Examples include advection
+tests and shallow water test cases. Results from these tests may not be
+simple objective evaluations (e.g. tradeoffs between dispersion, diffusion
+and monotonicity in advection), but should provide a framework and quantitative
+measures within which the configuration can be judged.
+
+\section{System tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Full system tests are required to provide a validation of the model
+against observational data.
+
+\section{Regression Tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Some subset of the above test, including all of the unit tests, should be
+included in an automated test suite with a script that can perform and
+evaluate each tests on a regular (e.g. nightly) basis. Such tests require
+a method for a simple pass/fail evaluation of each included test.
+
+%-----------------------------------------------------------------------
+
+\chapter{Design and Implementation}
+
+\section{Unit Tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Unit tests can be defined for the following modules.
+
+\subsection{Halo Updates}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+A test case for testing the validity of values in the halo region for halo updates should be defined. This test should include a variety of grids and
+decompositions to cover any previous failure modes and potentially problematic
+combinations of grids and decompositions.
+
+\subsection{Global Reductions}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+A test case for ensuring accurate computation of global reductions of a
+known field should be defined. Tests for all supported interfaces
+(e.g. integer, real, double, logical fields) and all supported reduction
+operators (sum, min, max, masked sum) should be included. If a
+bit-reproducible implementation is available, bit-reproducibility across
+different node/processor counts should be tested.
+
+\subsection{I/O}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Tests of the I/O system should validate whether fields are accurately written and read to a file. Multiple grid and processor configurations should be
+included.
+
+\subsection{Equation of State}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+A unit test for returning the proper values of the ocean equation of
+state should be included. Reference density values for a few specific
+p,T,S combinations are known and published for the typical equation of state
+implementations.
+
+%-----------------------------------------------------------------------
+
+\section{Intermediate Tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Several intermediate tests are either published or commonly used. Some
+have analytic results while others must be compared to reference
+simulations, typically at high resolution.
+
+\subsection{Advection Tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Advection algorithms are often tested by advecting a shape around the
+surface of the Earth. Such a test should be included. Add details
+on specific test case...
+
+\subsection{Shallow Water Test Cases}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Add specifics here
+
+\subsection{Channel Test Case}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+A channel configuration is a good test of eddy processes and ACC flow.
+A standard configuration is defined as (dimensions, forcing)....
+
+\subsubsection{N. Atlantic variant}
+
+Variants of the channel case are useful. For example, adding
+a North Atlantic basin (simple box) can be used.
+
+\subsection{Double-gyre}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+The double-gyre configuration is a commonly explored case in the
+ocean community and can be used to test both boundary interactions
+and viscosity formulations. A variety of configurations can be
+used.
+
+\subsubsection{Double-gyre variant 1}
+\subsubsection{Double-gyre variant 2}
+
+\subsection{Mixed layer tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+Vertical mixing and mixed/boundary layer parameterizations can be
+tested using a simple 2-d configuration with depth as one dimension
+and space/time as the second. A variation in surface buoyancy
+forcing can be imposed in the space/time direction to mimic either
+spatial variability or diurnal cycling.
+
+\subsection{Equatorial Pacific}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+An equatorial Pacific configuration is also a good test of vertical
+mixing in a configuration that includes both wind and buoyancy forcing
+as well as realistic ocean conditions. Specifics to be added...
+
+\subsection{Overflows}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+An overflow test case is useful to explore both vertical coordinate
+choices as well as any specific overflow parameterizations. More
+details here (DOME configuration?).
+
+%-----------------------------------------------------------------------
+
+\section{Full System Tests}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+\subsection{CORE forced case}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+The Common Ocean Reference Experiment (CORE) defines a useful test case
+for a full ocean simulation and comparisons with existing models can
+be performed. More detail...
+
+\subsection{Variable resolution tests?}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+\subsection{High resolution tests?}
+Date last modified: 2011/09/02 \\
+Contributors: (Doug Jacobsen, Phil Jones) \\
+
+%-----------------------------------------------------------------------
+
+\end{document}
</font>
</pre>