From nancy at ucar.edu Fri Apr 1 11:35:25 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 01 Apr 2011 11:35:25 -0600 Subject: [Dart-dev] [4835] DART/trunk/diagnostics/threed_sphere/obs_diag.html: Better mention of the ncview rank histogram instructions, Message-ID: Revision: 4835 Author: thoar Date: 2011-04-01 11:35:25 -0600 (Fri, 01 Apr 2011) Log Message: ----------- Better mention of the ncview rank histogram instructions, clarified the example. Modified Paths: -------------- DART/trunk/diagnostics/threed_sphere/obs_diag.html -------------- next part -------------- Modified: DART/trunk/diagnostics/threed_sphere/obs_diag.html =================================================================== --- DART/trunk/diagnostics/threed_sphere/obs_diag.html 2011-03-31 23:12:11 UTC (rev 4834) +++ DART/trunk/diagnostics/threed_sphere/obs_diag.html 2011-04-01 17:35:25 UTC (rev 4835) @@ -78,8 +78,8 @@ is set to 80 (for example); the first 80 ensemble members prior and posterior "expected" values of the observation are also included. In this case, the obs_seq.final file contains enough - information to calculate a rank histogram, while the ensemble means are still - used for all the other calculations. + information to calculate a rank histograms, verify forecasts, etc. + The ensemble means are still used for many other calculations. From nancy at ucar.edu Thu Apr 14 10:27:04 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 14 Apr 2011 10:27:04 -0600 Subject: [Dart-dev] [4852] DART/trunk/observations/observations.html: Adding a link to David Dowell's radar preprocessing code. Message-ID: Revision: 4852 Author: thoar Date: 2011-04-14 10:27:04 -0600 (Thu, 14 Apr 2011) Log Message: ----------- Adding a link to David Dowell's radar preprocessing code. (Here and in the DART:Miscellany:Useful Software tab) On Feb 22 Nancy wrote: "last week in the wrf/dart meeting david d gave a presentation on his radar preprocessing code. he's uploaded the software and constructed doc pages and whatnot. the web link is: http://code.google.com/p/opaws/ he's got a pointer to dart on that page, and we said we'd add a pointer to his page in our stuff. i'll add one on our developer's wiki page in the wrf section, but if there's a place on our image web site where it might fit that would be good to add, too." Modified Paths: -------------- DART/trunk/observations/observations.html -------------- next part -------------- Modified: DART/trunk/observations/observations.html =================================================================== --- DART/trunk/observations/observations.html 2011-04-12 17:57:58 UTC (rev 4851) +++ DART/trunk/observations/observations.html 2011-04-14 16:27:04 UTC (rev 4852) @@ -81,7 +81,7 @@ distances appropriately. Two of the ones most commonly used are for data in a 1D system and for data in a 3D spherical coordinate system. All the programs here assume the -location/threed_sphere/location_mod.f90 3D sphere +location/threed_sphere/location_mod.f90 3D sphere location module is being used.

@@ -109,7 +109,15 @@

PROGRAMS

-  +

From nancy at ucar.edu Thu Apr 14 11:45:20 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 14 Apr 2011 11:45:20 -0600 Subject: [Dart-dev] [4853] DART/trunk/models/tiegcm/model_mod.f90: Now a linear interpolation of the electron density to the obs height Message-ID: Revision: 4853 Author: tmatsuo Date: 2011-04-14 11:45:20 -0600 (Thu, 14 Apr 2011) Log Message: ----------- Now a linear interpolation of the electron density to the obs height is applied in a straightforward manner (it used to take log(Ne) before linearly interpolating values) Modified Paths: -------------- DART/trunk/models/tiegcm/model_mod.f90 -------------- next part -------------- Modified: DART/trunk/models/tiegcm/model_mod.f90 =================================================================== --- DART/trunk/models/tiegcm/model_mod.f90 2011-04-14 16:27:04 UTC (rev 4852) +++ DART/trunk/models/tiegcm/model_mod.f90 2011-04-14 17:45:20 UTC (rev 4853) @@ -521,12 +521,15 @@ endif -if ((obs_kind == KIND_ELECTRON_DENSITY) .or. & - (obs_kind == KIND_PRESSURE) ) then +if (obs_kind == KIND_PRESSURE) then val = exp(frac_lev * log(val_bottom) + (1.0 - frac_lev) * log(val_top)) -else +else +!KIND_ELECTRON_DENSITY +!KIND_TEMPERATURE +!KIND_MOLEC_OXYGEN_MIXING_RATIO +!KIND_ATOMIC_OXYGEN_MIXING_RATIO val = frac_lev * val_bottom + (1.0 - frac_lev) * val_top From nancy at ucar.edu Thu Apr 14 16:04:07 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 14 Apr 2011 16:04:07 -0600 Subject: [Dart-dev] [4855] DART/trunk/models/cam: Kevin prefers the entry point to be model_mod .html, so Message-ID: Revision: 4855 Author: thoar Date: 2011-04-14 16:04:06 -0600 (Thu, 14 Apr 2011) Log Message: ----------- Kevin prefers the entry point to be model_mod.html, so we are renaming the index.html to be something more appropriate. It is not really the MAIN page someone should be looking at. Modified Paths: -------------- DART/trunk/models/cam/model_mod.html Added Paths: ----------- DART/trunk/models/cam/doc/cam_guidelines.html Removed Paths: ------------- DART/trunk/models/cam/doc/index.html -------------- next part -------------- Copied: DART/trunk/models/cam/doc/cam_guidelines.html (from rev 4851, DART/trunk/models/cam/doc/index.html) =================================================================== --- DART/trunk/models/cam/doc/cam_guidelines.html (rev 0) +++ DART/trunk/models/cam/doc/cam_guidelines.html 2011-04-14 22:04:06 UTC (rev 4855) @@ -0,0 +1,550 @@ + + + +DART-CAM setup + + + + + + + +
+CALLING TREE / +SET-UP / +FILE CONTENTS / +OUTPUT DIRECTORY / +HINTS / +SPACE / +TERMS OF USE +
+ +

DART-CAM OVERVIEW

+

$Id$

+ +

+The up-to-date overview will always be available at + +http://www.image.ucar.edu/DAReS/DART/cgd_cam.shtml +

+For the + Jamaica release +the async=3 (.../input.nml:filter_nml) option +is no longer offered, and has been replaced by async=4. +This new option runs an MPI filter, which can use either single threaded or MPI CAM. +The single threaded option will run CAM for 1 ensemble member on +each processor (up to the lesser of the number of ensemble members +or the number of processors). +The MPI CAM option will run CAM for each ensemble member in succession, +using all the available processors. +It's not possible (yet) to run several MPI CAMs at the same time, +each using a subset of the processors. +

+This new option allows users to control the assimilation through a single script, +'job_mpi.csh', +except for modifications for machines on which DART-CAM hasn't been tested yet. +Job_mpi.csh has a section of user set parameters that define many aspects of +the assimilation. +It uses those parameters to create a series of batch scripts, +one for each obs_seq.out file that will be assimilated. +Their names have the form Exper_obsseq#.script, +the parts of which are defined in job_mpi.csh. +It submits all of those scripts to the batch scheduler, +where they run in succession as resources become available. +Each obs_seq batch script executes the programs layed out in the calling tree (below). +

+The async=2 option (non-MPI filter and non-MPI CAM) is still available. +

+These options have been tested for DART-CAM in batch submission environments +PBS and LSF on Linux clusters and IBM AIX. +

+There are, no doubt, things missing from these lists, so don't struggle too long +before contacting raeder'at'ucar.edu. +

+The sequence of operations for assimilating a single obs_seq.out file follows. +The functionality of each operation has been restricted to one "domain". +A script/program is specific to: +a machine where the experiment is run; +a model version used in the assimilation; +the filter version; +or the experiment being conducted using the choices for the previous 3. +

+Go to cam/model_mod page +

+ + +
+

CALLING TREE

+

+The calling tree for the scripts and fortran executables when running under +async=4 is: +

+ +
@@ -127,7 +127,7 @@

obs_diag - is not explicitly designed to take explore OSSE's. In general, it is + is not explicitly designed to explore OSSE's. In general, it is used for 'real' observations and looks through the metadata for the observation sequence to identify which 'copy' is labeled 'observation'. It is THAT copy that is used as the noisy estimate of the truth. @@ -147,7 +147,7 @@ files and calculates the following quantities (in no particular order) for an arbitrary number of regions and levels. It is necessary to query the CopyMetaData variable to determine the storage order - ( i.e. "which copy is what?"). + (i.e. "which copy is what?").

@@ -733,6 +733,16 @@ IFF it is possible to calculate a rank histogram in the first place. + +
+ + + + + + +
Instructions for viewing the rank histogram with ncview.
Instructions for viewing the rank histogram with Matlab.
+ @@ -742,8 +752,9 @@

USAGE

-Obs_diag is built in .../DART/models/your_model/work, in the same way -as the other DART components. +obs_diag is built in +.../DART/models/your_model/work, +in the same way as the other DART components.

multiple observation sequence files

@@ -786,14 +797,17 @@ -

Example: creating a rank histogram from observation sequence files -spanning 30 days.

- - +

Example: observation sequence files spanning 30 days.

+
In this example, we are generating an obs_diag_output.nc - file that will have exactly ONE timestep in it - for the entire globe - and we - will NOT be using outlier observations in the rank histogram. Lets presume - that all your obs_seq.final files are in alphabetically-nice - directories:
+
In this example, we will be accumulating metrics for 30 days over the + entire globe. The obs_diag_output.nc file will have + exactly ONE timestep in it (so it won't be much use for the + plot_evolution functions) - but the + plot_profile functions and the + plot_rank_histogram function will be used to + explore the assimilation. By way of an example, we will NOT be using + outlier observations in the rank histogram. Lets presume that all your + obs_seq.final files are in alphabetically-nice directories:   
@@ -878,8 +892,10 @@

Now that you have the obs_diag_output.nc, you can explore it with +plot_profile.m, plot_bias_xxx_profile.m, or plot_rmse_xxx_profile.m, +and look at the rank histograms with ncview or -plot_rank_histogram.m. +plot_rank_histogram.m.

From thoar at ucar.edu Fri Apr 1 16:00:41 2011 From: thoar at ucar.edu (Tim Hoar) Date: Fri, 1 Apr 2011 16:00:41 -0600 Subject: [Dart-dev] DART documentation needs your help. Message-ID: I am adding to the DART www-pages and need suggestions from you. In particular, I have added to the Documentation:Was the Assimilation Effective? section (the direct link follows:) http://test.www.image.ucar.edu/DAReS/DART/DART_Documentation.php#DidItWork Please read it over and let me know what you think. The intent is to guide someone who is just starting with DART. I would like to keep it model-agnostic, but nice examples (model-specific) are always a good idea. I also added to the Documentation:State Space Diagnostics section http://test.www.image.ucar.edu/DAReS/DART/DART_Documentation.php#ss_diagnostics and realize how little I know of how people analyze their state-space results! I need ideas and examples on how people diagnose things from the Posterior_Diag.nc and Prior_Diag.nc files. In the coming days, I will add examples on checking inflation fields and values through time (with ncview), but I quickly run out of ideas. I will be adding example graphics to this section to illustrate what each of the matlab scripts does - so consider that to be a given. (Most of these will be WRF examples, BTW.) I have also added to the Documentation:non-Matlab Diagnostics section. Please let me know what you think. http://test.www.image.ucar.edu/DAReS/DART/DART_Documentation.php#alt_diagnostics Thanks -- Tim Tim Hoar, Associate Scientist National Center for Atmospheric Research thoar at ucar.edu 303 497 1708 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ucar.edu/pipermail/dart-dev/attachments/20110401/cef279ab/attachment.html From thoar at ucar.edu Fri Apr 1 16:58:57 2011 From: thoar at ucar.edu (Tim Hoar) Date: Fri, 1 Apr 2011 16:58:57 -0600 Subject: [Dart-dev] DART documentation needs your help. In-Reply-To: References: Message-ID: Doh! If you are INSIDE the UCAR network, the links below should work. Everyone: if you remove the "test." in the links, they should work from any network. My apologies. Tim On Apr 1, 2011, at 4:00 PM, Tim Hoar wrote: > I am adding to the DART www-pages and need suggestions from you. > > In particular, I have added to the > > Documentation:Was the Assimilation Effective? section > > (the direct link follows:) > http://test.www.image.ucar.edu/DAReS/DART/DART_Documentation.php#DidItWork > > Please read it over and let me know what you think. The intent is > to guide someone who is just starting with DART. I would like to > keep it model-agnostic, but nice examples (model-specific) are > always a good idea. > > > I also added to the > Documentation:State Space Diagnostics section > > http://test.www.image.ucar.edu/DAReS/DART/DART_Documentation.php#ss_diagnostics > > and realize how little I know of how people analyze their state-space results! I need ideas and examples on how people diagnose things from > the Posterior_Diag.nc and Prior_Diag.nc files. In the coming days, I will > add examples on checking inflation fields and values through time (with ncview), > but I quickly run out of ideas. I will be adding example graphics to this > section to illustrate what each of the matlab scripts does - so consider that > to be a given. (Most of these will be WRF examples, BTW.) > > > I have also added to the > Documentation:non-Matlab Diagnostics section. Please let me know what you think. > > http://test.www.image.ucar.edu/DAReS/DART/DART_Documentation.php#alt_diagnostics > > > Thanks -- Tim > > Tim Hoar, Associate Scientist > National Center for Atmospheric Research > thoar at ucar.edu > 303 497 1708 > Tim Hoar, Associate Scientist National Center for Atmospheric Research thoar at ucar.edu 303 497 1708 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ucar.edu/pipermail/dart-dev/attachments/20110401/d978d502/attachment.html From nancy at ucar.edu Mon Apr 4 15:13:21 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Mon, 04 Apr 2011 15:13:21 -0600 Subject: [Dart-dev] [4841] DART/trunk/models/cam: New program that converts a cam initial file (caminput.nc) to Message-ID: Revision: 4841 Author: thoar Date: 2011-04-04 15:13:20 -0600 (Mon, 04 Apr 2011) Log Message: ----------- New program that converts a cam initial file (caminput.nc) to a DART ics file and uses the time from the netCDF file as opposed to some external source. The input and output filenames for cam_to_dart.f90 are namelist controlled, so there is a new namelist (cam_to_dart_nml) in input.nml. There is still a model_mod:static_init_model() dependency on cam_phis.nc, the name of which is now part of the model_nml namelist. Modified Paths: -------------- DART/trunk/models/cam/model_mod.f90 DART/trunk/models/cam/work/input.nml Added Paths: ----------- DART/trunk/models/cam/cam_to_dart.f90 DART/trunk/models/cam/work/mkmf_cam_to_dart DART/trunk/models/cam/work/path_names_cam_to_dart -------------- next part -------------- Added: DART/trunk/models/cam/cam_to_dart.f90 =================================================================== --- DART/trunk/models/cam/cam_to_dart.f90 (rev 0) +++ DART/trunk/models/cam/cam_to_dart.f90 2011-04-04 21:13:20 UTC (rev 4841) @@ -0,0 +1,95 @@ +! DART software - Copyright \xA9 2004 - 2010 UCAR. This open source software is +! provided by UCAR, "as is", without charge, subject to all terms of use at +! http://www.image.ucar.edu/DAReS/DART/DART_download + +program cam_to_dart + +! +! $URL$ +! $Id$ +! $Revision$ +! $Date$ + +!---------------------------------------------------------------------- +! purpose: interface between CAM and DART +! +! method: Read CAM 'initial' file (netCDF format) for model state and time. +! Reform fields into a DART state vector. +! Write out state vector in "proprietary" format for DART +! +! author: Tim Hoar 4/4/2011 +! +!---------------------------------------------------------------------- + +use types_mod, only : r8 +use utilities_mod, only : initialize_utilities, finalize_utilities, do_output, & + check_namelist_read, find_namelist_in_file, nmlfileunit +use model_mod, only : model_type, init_model_instance, end_model_instance, & + prog_var_to_vector, read_cam_init +use assim_model_mod, only : static_init_assim_model, get_model_size, & + open_restart_write, awrite_state_restart, close_restart +use time_manager_mod, only : time_type + +implicit none + +! version controlled file description for error handling, do not edit +character(len=128), parameter :: & + source = "$URL$", & + revision = "$Revision$", & + revdate = "$Date$" + + +!----------------------------------------------------------------------- +! namelist parameters with default values. +!----------------------------------------------------------------------- + +character (len = 128) :: cam_to_dart_input_file = 'caminput.nc' +character (len = 128) :: cam_to_dart_output_file = 'temp_ud' + +namelist /cam_to_dart_nml/ cam_to_dart_input_file, cam_to_dart_output_file + +! allocatable storage to read in a native format for cam state +real(r8), allocatable :: statevector(:) +type(model_type) :: var +type(time_type) :: model_time +integer :: iunit, x_size, io + +call initialize_utilities('cam_to_dart') + +! Read the namelist entry +call find_namelist_in_file("input.nml", "cam_to_dart_nml", iunit) +read(iunit, nml = cam_to_dart_nml, iostat = io) +call check_namelist_read(iunit, io, "cam_to_dart_nml") + +! Record the namelist values +if (do_output()) write(nmlfileunit, nml=cam_to_dart_nml) +if (do_output()) write( * , nml=cam_to_dart_nml) + +! Static init assim model sets the output file format (binary/ascii) +! and calls static_init_model +call static_init_assim_model() + +! Allocate the local state vector +x_size = get_model_size() +allocate(statevector(x_size)) + +! Allocate the instance of the cam model type for storage +! I'll just point to the space I need, not; +call init_model_instance(var) + +! Read the file cam state fragments into var; +! transform fields into state vector for DART + +call read_cam_init(cam_to_dart_input_file, var, model_time) + +call prog_var_to_vector(var, statevector) +call end_model_instance(var) + +! write out state vector in "proprietary" format +iunit = open_restart_write(cam_to_dart_output_file) +call awrite_state_restart(model_time, statevector, iunit) +call close_restart(iunit) + +call finalize_utilities() + +end program cam_to_dart Property changes on: DART/trunk/models/cam/cam_to_dart.f90 ___________________________________________________________________ Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Modified: DART/trunk/models/cam/model_mod.f90 =================================================================== --- DART/trunk/models/cam/model_mod.f90 2011-04-01 22:53:29 UTC (rev 4840) +++ DART/trunk/models/cam/model_mod.f90 2011-04-04 21:13:20 UTC (rev 4841) @@ -244,11 +244,10 @@ use types_mod, only : r8, MISSING_I, MISSING_R8, gravity_const => gravity ! add after verification against Hui's tests; gas_constant_v,gas_constant,ps0,PI,DEG2RAD -use time_manager_mod, only : time_type, set_time, print_time, set_calendar_type, & - THIRTY_DAY_MONTHS, JULIAN, GREGORIAN, NOLEAP, NO_CALENDAR +use time_manager_mod, only : time_type, set_time, set_date, print_time, print_date, set_calendar_type use utilities_mod, only : open_file, close_file, find_namelist_in_file, check_namelist_read, & register_module, error_handler, file_exist, E_ERR, E_WARN, E_MSG, & - logfileunit, nmlfileunit, do_output, nc_check + logfileunit, nmlfileunit, do_output, nc_check, get_unit use mpi_utilities_mod, only : my_task_id, task_count !------------------------------------------------------------------------- @@ -526,7 +525,7 @@ ! by numerical stability concerns for repeated restarting in leapfrog. integer :: Time_step_seconds = 21600, Time_step_days = 0 -namelist /model_nml/ output_state_vector , model_version , model_config_file & +namelist /model_nml/ output_state_vector , model_version , cam_phis, model_config_file & ,state_num_0d ,state_num_1d ,state_num_2d ,state_num_3d & ,state_names_0d ,state_names_1d ,state_names_2d ,state_names_3d & , which_vert_1d ,which_vert_2d ,which_vert_3d & @@ -556,8 +555,8 @@ data ens_member /0/ logical :: do_out -! common error string used by many subroutines -character(len=129) :: errstring +! common message string used by many subroutines +character(len=129) :: msgstring ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - integer :: nflds ! # fields to read @@ -657,17 +656,15 @@ ! name netcdf file. integer :: iunit, io, topog_lons, topog_lats, i, num_lons, num_lats, ncfileid -! calendar types listed in time_manager_mod.f90 -integer :: calendar_type = GREGORIAN integer :: max_levs - ! Register the module call register_module(source, revision, revdate) ! setting calendar type +! calendar types listed in time_manager_mod.f90 ! this information is NOT passed to CAM; it must be set in the CAM namelist -call set_calendar_type(calendar_type) +call set_calendar_type('GREGORIAN') ! Read the namelist entry call find_namelist_in_file("input.nml", "model_nml", iunit) @@ -677,14 +674,14 @@ ! set the printed output logical variable to reduce printed output; ! depends on whether this is being called by trans_... (read ens member # from file 'element' ) ! or by filter (multiple processes, printout controlled by do_output()) + if (file_exist('element')) then -! debug; fix this ugliness - open(unit = 99, file='element', form = 'formatted') - read(99,*) ens_member - close(99) + iunit = get_unit() + open(unit = iunit, file='element', form = 'formatted') + read(iunit,*) ens_member + close(iunit) do_out = .false. if (ens_member == 1) do_out = .true. - !write(*,*) 'do_out = ',do_out else do_out = do_output() !write(*,*) 'do_out = ',do_out @@ -786,20 +783,20 @@ if (do_out) write(*, *) 'file_name for surface geopotential height is ', trim(cam_phis) call read_topog_size(ncfileid, topog_lons, topog_lats) - ! debug + if (do_out) write(*,*) 'topog_lons, _lats = ',topog_lons, topog_lats num_lons = dim_sizes(find_name('lon ',dim_names)) num_lats = dim_sizes(find_name('lat ',dim_names)) if (topog_lons /= num_lons .or. topog_lats /= num_lats) then - write(errstring,'(A,4I4)') 'horizontal dimensions mismatch of initial files and topog ' & + write(msgstring,'(A,4I4)') 'horizontal dimensions mismatch of initial files and topog ' & ,num_lons, topog_lons, num_lats, topog_lats - call error_handler(E_ERR, 'static_init_model', trim(errstring), source, revision, revdate) + call error_handler(E_ERR, 'static_init_model', trim(msgstring), source, revision, revdate) end if else - write(errstring,'(2A)') trim(cam_phis),' is missing; find a CAM history file (h0) to provide PHIS' - call error_handler(E_ERR, 'static_init_model', trim(errstring), source, revision, revdate) + write(msgstring,'(2A)') trim(cam_phis),' is missing; find a CAM history file (h0) to provide PHIS' + call error_handler(E_ERR, 'static_init_model', trim(msgstring), source, revision, revdate) end if ! Read surface geopotential from cam_phis for use in vertical interpolation in height. @@ -1106,8 +1103,8 @@ end do Alldim1 if ( s_dim_1d(i) == 0 ) then - write(errstring, '(A,I3,A)') ' state 1d dimension(',i,') was not assigned and = 0' - call error_handler(E_ERR, 'trans_coord',trim(errstring), source, revision, revdate) + write(msgstring, '(A,I3,A)') ' state 1d dimension(',i,') was not assigned and = 0' + call error_handler(E_ERR, 'trans_coord',trim(msgstring), source, revision, revdate) end if end do @@ -1481,8 +1478,8 @@ end do if (nfld .ne. nflds) then - write(errstring, *) 'nfld = ',nfld,', nflds = ',nflds,' must be equal ' - call error_handler(E_ERR, 'order_state_fields', errstring, source, revision, revdate) + write(msgstring, *) 'nfld = ',nfld,', nflds = ',nflds,' must be equal ' + call error_handler(E_ERR, 'order_state_fields', msgstring, source, revision, revdate) elseif (do_out) then write(logfileunit,'(/A/)') 'State vector is composed of ' ! write(logfileunit,'((8(A8,1X)))') (cflds(i),i=1,nflds) @@ -1588,19 +1585,23 @@ ! Module I/O to/from DART and files - subroutine read_cam_init(file_name, var) + subroutine read_cam_init(file_name, var, model_time) !======================================================================= -! subroutine read_cam_init(file_name, var) +! subroutine read_cam_init(file_name, var, model_time) ! -character(len = *), intent(in) :: file_name -type(model_type), intent(out) :: var +character(len = *), intent(in) :: file_name +type(model_type), intent(out) :: var +type(time_type), optional, intent(out) :: model_time ! Local workspace integer :: i, k, n, m, ifld ! grid and constituent indices -integer :: ncfileid, ncfldid +integer :: ncfileid, ncfldid, dimid, varid, dimlen real(r8), allocatable :: temp_3d(:,:,:), temp_2d(:,:) +integer :: iyear, imonth, iday, ihour, imin, isec, rem +integer, allocatable, dimension(:) :: datetmp, datesec + !---------------------------------------------------------------------- ! read CAM 'initial' file domain info call nc_check(nf90_open(path = trim(file_name), mode = nf90_write, ncid = ncfileid), & @@ -1720,6 +1721,58 @@ end if end do +! Read the time of the current state. +! All the caminput.nc files I have seen have two variables of +! length 'time' (the unlimited dimension): date, datesec +! The rest of the routine presumes there is but one time in the file - +! print warning message if this is not the case. + +if (present( model_time)) then + + call nc_check(nf90_inq_dimid(ncfileid, 'time', dimid), & + 'read_cam_init', 'inq_dimid time '//trim(file_name)) + call nc_check(nf90_inquire_dimension(ncfileid, dimid, len=dimlen), & + 'read_cam_init', 'inquire_dimension time '//trim(file_name)) + + if (dimlen /= 1) then + write(msgstring,*)'UNUSUAL - ',trim(file_name),' has',dimlen,'times. Expected 1.' + call error_handler(E_MSG, 'read_cam_init', msgstring, source, revision, revdate) + endif + + allocate(datetmp(dimlen), datesec(dimlen)) + + call nc_check(nf90_inq_varid(ncfileid, 'date', varid), & + 'read_cam_init', 'inq_varid date '//trim(file_name)) + call nc_check(nf90_get_var(ncfileid, varid, values=datetmp), & + 'read_cam_init', 'get_var date '//trim(file_name)) + + call nc_check(nf90_inq_varid(ncfileid, 'datesec', varid), & + 'read_cam_init', 'inq_varid datesec '//trim(file_name)) + call nc_check(nf90_get_var(ncfileid, varid, values=datesec), & + 'read_cam_init', 'get_var datesec '//trim(file_name)) + + ! The 'date' is YYYYMMDD ... datesec is 'current seconds of current day' + iyear = datetmp(dimlen) / 10000 + rem = datetmp(dimlen) - iyear*10000 + imonth = rem / 100 + iday = rem - imonth*100 + + ihour = datesec(dimlen) / 3600 + rem = datesec(dimlen) - ihour*3600 + imin = rem / 60 + isec = rem - imin*60 + + model_time = set_date(iyear,imonth,iday,ihour,imin,isec) + + if (do_out) then + call print_date(model_time,'read_cam_init:CAM input date') + call print_time(model_time,'read_cam_init:CAM input time') + endif + + deallocate(datetmp, datesec) + +endif + call nc_check(nf90_close(ncfileid), 'read_cam_init', 'closing '//trim(file_name)) deallocate (temp_3d,temp_2d) @@ -2047,9 +2100,9 @@ ! This will malfunction for fields that are filled with MISSING_r8 for lat_val or lon_val. if (lon_val == MISSING_r8 .or. lat_val == MISSING_r8 ) then - write(errstring, *) 'Field ',cflds(nfld),' has no lon or lat dimension. ', & + write(msgstring, *) 'Field ',cflds(nfld),' has no lon or lat dimension. ', & 'What should be specified for it in the call to location?' - call error_handler(E_ERR, 'get_state_meta_data', errstring, source, revision, revdate) + call error_handler(E_ERR, 'get_state_meta_data', msgstring, source, revision, revdate) else location = set_location(lon_val, lat_val, lev_val, which_vert) endif @@ -2152,10 +2205,10 @@ ! More dimensions, variables and attributes will be added in this routine. !------------------------------------------------------------------------------- -write(errstring,*) 'ncFileID', ncFileID +write(msgstring,*) 'ncFileID', ncFileID call nc_check(nf90_Inquire(ncFileID, nDimensions, nVariables, nAttributes, unlimitedDimID), & - 'nc_write_model_atts', 'Inquire '//trim(errstring)) -call nc_check(nf90_Redef(ncFileID), 'nc_write_model_atts', 'Redef '//trim(errstring)) + 'nc_write_model_atts', 'Inquire '//trim(msgstring)) +call nc_check(nf90_Redef(ncFileID), 'nc_write_model_atts', 'Redef '//trim(msgstring)) !------------------------------------------------------------------------------- ! We need the dimension ID for the number of copies @@ -2167,8 +2220,8 @@ 'nc_write_model_atts', 'inq_dimid time') if ( TimeDimID /= unlimitedDimId ) then - write(errstring,*)'Time dimension ID ',TimeDimID,'must match Unlimited Dimension ID ',unlimitedDimId - call error_handler(E_ERR,'nc_write_model_atts', errstring, source, revision, revdate) + write(msgstring,*)'Time dimension ID ',TimeDimID,'must match Unlimited Dimension ID ',unlimitedDimId + call error_handler(E_ERR,'nc_write_model_atts', msgstring, source, revision, revdate) end if !------------------------------------------------------------------------------- @@ -3070,8 +3123,8 @@ ! if (abs((val - pressure)/val) > 1.0E-12) then ! ! We're looking for a pressure on a model level, which is exactly what p_col provides, !! NOT HERE; that happens in get_val_level -! write(errstring, *) 'val /= pressure = ',val,pressure,' when val is a P obs ' -! call error_handler(E_WARN, 'get_val_pressure', errstring, source, revision, revdate) +! write(msgstring, *) 'val /= pressure = ',val,pressure,' when val is a P obs ' +! call error_handler(E_WARN, 'get_val_pressure', msgstring, source, revision, revdate) ! end if else ! Pobs end @@ -3326,9 +3379,9 @@ do nf= 1, state_num_3d ! if (do_out) then -! write(errstring, '(A,4I5)') 'fld, nlons, nlats, nlevs ',nf & +! write(msgstring, '(A,4I5)') 'fld, nlons, nlats, nlevs ',nf & ! ,s_dim_3d(2,nf),s_dim_3d(3,nf),s_dim_3d(1,nf) -! call error_handler(E_MSG, 'prog_var_to_vector', errstring, source, revision, revdate) +! call error_handler(E_MSG, 'prog_var_to_vector', msgstring, source, revision, revdate) ! endif do i=1,s_dim_3d(3,nf) !lats @@ -3343,8 +3396,8 @@ ! Temporary check if (indx /= model_size) then - write(errstring, *) 'indx ',indx,' model_size ',model_size,' must be equal ' - call error_handler(E_ERR, 'prog_var_to_vector', errstring, source, revision, revdate) + write(msgstring, *) 'indx ',indx,' model_size ',model_size,' must be equal ' + call error_handler(E_ERR, 'prog_var_to_vector', msgstring, source, revision, revdate) end if end subroutine prog_var_to_vector @@ -3392,9 +3445,9 @@ ! 3D fields; see comments in prog_var_to_vect do nf = 1, state_num_3d ! if (do_out) then -! write(errstring, '(A,4I5)') 'fld, nlons, nlats, nlevs ',nf & +! write(msgstring, '(A,4I5)') 'fld, nlons, nlats, nlevs ',nf & ! ,s_dim_3d(2,nf),s_dim_3d(3,nf),s_dim_3d(1,nf) -! call error_handler(E_MSG, 'vector_to_prog_var', errstring, source, revision, revdate) +! call error_handler(E_MSG, 'vector_to_prog_var', msgstring, source, revision, revdate) ! end if do i = 1, s_dim_3d(3,nf) do j = 1, s_dim_3d(2,nf) @@ -3408,8 +3461,8 @@ ! Temporary check if (indx /= model_size) then - write(errstring, *) 'indx ',indx,' model_size ',model_size,' must be equal ' - call error_handler(E_ERR, 'vector_to_prog_var', errstring, source, revision, revdate) + write(msgstring, *) 'indx ',indx,' model_size ',model_size,' must be equal ' + call error_handler(E_ERR, 'vector_to_prog_var', msgstring, source, revision, revdate) end if end subroutine vector_to_prog_var @@ -3599,9 +3652,9 @@ ! proceed else ! make this a fatal error - there should be no other options for vert. - write(errstring,'(''obs at '',3(F9.5,1x),I2,'' has bad vertical type'')') & + write(msgstring,'(''obs at '',3(F9.5,1x),I2,'' has bad vertical type'')') & old_array, old_which - call error_handler(E_ERR, 'convert_vert', errstring,source,revision,revdate) + call error_handler(E_ERR, 'convert_vert', msgstring,source,revision,revdate) end if ! Find the nfld of this dart-KIND @@ -3785,26 +3838,26 @@ if (top_lev == 1 .and. old_array(3) > model_h(1)) then ! above top of model frac = 1.0_r8 - write(errstring, *) 'ob height ',old_array(3),' above CAM levels at ' & + write(msgstring, *) 'ob height ',old_array(3),' above CAM levels at ' & ,old_array(1) ,old_array(2) ,' for ob type',dart_kind - call error_handler(E_MSG, 'convert_vert', errstring,source,revision,revdate) + call error_handler(E_MSG, 'convert_vert', msgstring,source,revision,revdate) else if (bot_lev <= num_levs) then ! within model levels frac = (old_array(3) - model_h(bot_lev)) / (model_h(top_lev) - model_h(bot_lev)) else ! below bottom of model frac = 0.0_r8 - write(errstring, *) 'ob height ',old_array(3),' below CAM levels at ' & + write(msgstring, *) 'ob height ',old_array(3),' below CAM levels at ' & ,old_array(1) ,old_array(2) ,' for ob type',dart_kind - call error_handler(E_MSG, 'convert_vert', errstring,source,revision,revdate) + call error_handler(E_MSG, 'convert_vert', msgstring,source,revision,revdate) endif new_array(3) = (1.0_r8 - frac) * p_col(bot_lev) + frac * p_col(top_lev) new_which = 2 else - write(errstring, *) 'model which_vert = ',old_which,' not handled in convert_vert ' - call error_handler(E_ERR, 'convert_vert', errstring,source,revision,revdate) + write(msgstring, *) 'model which_vert = ',old_which,' not handled in convert_vert ' + call error_handler(E_ERR, 'convert_vert', msgstring,source,revision,revdate) end if return @@ -4204,8 +4257,8 @@ resol = ilev%resolution else ! should not happen; fatal error. - write(errstring, *) 'unexpected dim_name, ', trim(dim_name) - call error_handler(E_ERR, 'coord_index', errstring,source,revision,revdate) + write(msgstring, *) 'unexpected dim_name, ', trim(dim_name) + call error_handler(E_ERR, 'coord_index', msgstring,source,revision,revdate) end if ! further check? for blunders check that coord(1) - val is smaller than coord(2) - coord(1), etc. Modified: DART/trunk/models/cam/work/input.nml =================================================================== --- DART/trunk/models/cam/work/input.nml 2011-04-01 22:53:29 UTC (rev 4840) +++ DART/trunk/models/cam/work/input.nml 2011-04-04 21:13:20 UTC (rev 4841) @@ -81,21 +81,28 @@ &assim_model_nml write_binary_restart_files = .true., netCDF_large_file_support = .false. - / + / +&cam_to_dart_nml + / + + # also a common setup: -# model_version = '3.5.06', -# state_names_3d = 'T','US','VS','Q','CLDLIQ','CLDICE' +# model_version = '3.5.06', +# model_version = '4.0.1', +# state_names_3d = 'T','US','VS','Q','CLDLIQ','CLDICE' +# model_version = '3.1', +# state_names_3d = 'T','U','V','Q','CLDLIQ','CLDICE' &model_nml output_state_vector = .false., - model_version = '3.1', + model_version = '4.0.1', model_config_file = 'caminput.nc', state_num_0d = 0, state_num_1d = 0, state_num_2d = 1, state_num_3d = 6, state_names_2d = 'PS' - state_names_3d = 'T','U','V','Q','CLDLIQ','CLDICE' + state_names_3d = 'T','US','VS','Q','CLDLIQ','CLDICE' which_vert_1d = 0, which_vert_2d = -1, which_vert_3d = 6*1, @@ -134,8 +141,6 @@ input_obs_def_mod_file = '../../../obs_def/DEFAULT_obs_def_mod.F90', output_obs_def_mod_file = '../../../obs_def/obs_def_mod.f90', input_files = '../../../obs_def/obs_def_gps_mod.f90', - '../../../obs_def/obs_def_QuikSCAT_mod.f90', - '../../../obs_def/obs_def_GWD_mod.f90', '../../../obs_def/obs_def_altimeter_mod.f90', '../../../obs_def/obs_def_reanalysis_bufr_mod.f90' / Added: DART/trunk/models/cam/work/mkmf_cam_to_dart =================================================================== --- DART/trunk/models/cam/work/mkmf_cam_to_dart (rev 0) +++ DART/trunk/models/cam/work/mkmf_cam_to_dart 2011-04-04 21:13:20 UTC (rev 4841) @@ -0,0 +1,18 @@ +#!/bin/csh +# +# DART software - Copyright \xA9 2004 - 2010 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# $Id$ + +../../../mkmf/mkmf -p cam_to_dart -t ../../../mkmf/mkmf.template -c "-Duse_netCDF" \ + -a "../../.." path_names_cam_to_dart + +exit $status + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/cam/work/mkmf_cam_to_dart ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Added: DART/trunk/models/cam/work/path_names_cam_to_dart =================================================================== --- DART/trunk/models/cam/work/path_names_cam_to_dart (rev 0) +++ DART/trunk/models/cam/work/path_names_cam_to_dart 2011-04-04 21:13:20 UTC (rev 4841) @@ -0,0 +1,11 @@ +assim_model/assim_model_mod.f90 +common/types_mod.f90 +location/threed_sphere/location_mod.f90 +models/cam/cam_to_dart.f90 +models/cam/model_mod.f90 +mpi_utilities/null_mpi_utilities_mod.f90 +obs_kind/obs_kind_mod.f90 +random_nr/random_nr_mod.f90 +random_seq/random_seq_mod.f90 +time_manager/time_manager_mod.f90 +utilities/utilities_mod.f90 From nancy at ucar.edu Mon Apr 4 16:11:13 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Mon, 04 Apr 2011 16:11:13 -0600 Subject: [Dart-dev] [4842] DART/trunk/models/cam: Created the documentation and unified the namelist defaults. Message-ID: Revision: 4842 Author: thoar Date: 2011-04-04 16:11:13 -0600 (Mon, 04 Apr 2011) Log Message: ----------- Created the documentation and unified the namelist defaults. Modified Paths: -------------- DART/trunk/models/cam/cam_to_dart.f90 DART/trunk/models/cam/model_mod.nml Added Paths: ----------- DART/trunk/models/cam/cam_to_dart.html -------------- next part -------------- Modified: DART/trunk/models/cam/cam_to_dart.f90 =================================================================== --- DART/trunk/models/cam/cam_to_dart.f90 2011-04-04 21:13:20 UTC (rev 4841) +++ DART/trunk/models/cam/cam_to_dart.f90 2011-04-04 22:11:13 UTC (rev 4842) @@ -44,7 +44,7 @@ !----------------------------------------------------------------------- character (len = 128) :: cam_to_dart_input_file = 'caminput.nc' -character (len = 128) :: cam_to_dart_output_file = 'temp_ud' +character (len = 128) :: cam_to_dart_output_file = 'dart_ics' namelist /cam_to_dart_nml/ cam_to_dart_input_file, cam_to_dart_output_file Added: DART/trunk/models/cam/cam_to_dart.html =================================================================== --- DART/trunk/models/cam/cam_to_dart.html (rev 0) +++ DART/trunk/models/cam/cam_to_dart.html 2011-04-04 22:11:13 UTC (rev 4842) @@ -0,0 +1,219 @@ + + + +program cam_to_dart + + + + + +

PROGRAM cam_to_dart

+ + + + + + +
+ DART project logo + +

Jump to DART Documentation Main Index
+ version information for this file:
+ + $Id$ +

+ +NAMELIST / +MODULES / +FILES / +REFERENCES / +ERRORS / +PLANS / +TERMS OF USE + +

Overview

+ +

+ cam_to_dart is the program that reads a CAM + restart file (usually caminput.nc) and creates + a single DART output/restart file + (e.g. perfect_ics, filter_ics, ... ). If you have + multiple input files, you will need to rename the output files as you create them. +
+
+ The list of variables extracted from the CAM netCDF + file and conveyed to DART is controlled by the set of input.nml + &model_nml:state_names_* variables. The date and + datesec variables in the CAM netcdf file are used to specify + the valid time of the state vector. The time may be changed with the + restart_file_tool if desired. +
+
+ Conditions required for successful execution of cam_to_dart: +

+ +
    +
  • a valid input.nml namelist file for DART
    • +
  • a CAM 'phis' netCDF file [default: cam_phis.nc]
    • +
  • a CAM restart file [default: caminput.nc].
    • +
+ +

+Since this program is called repeatedly for every ensemble member, +we have found it convenient to link the CAM restart files +to the default input filename (caminput.nc). +The default DART output filename is dart_ics - +this may be moved or linked as necessary. +

+ + + + + + +
+

NAMELIST

+

We adhere to the F90 standard of starting a namelist with an ampersand +'&' and terminating with a slash '/' for all our namelist input. +Character strings that contain a '/' must be +enclosed in quotes to prevent them from prematurely terminating the namelist. +The namelist declaration is: +

+
+
+namelist / cam_to_dart_nml /  cam_to_dart_input_file, &
+                                       cam_to_dart_output_file
+
+
+ +
+ + +

+This namelist is always read from a file called input.nml. +

+ + + + + + + + + + + + + + +
Contents Type Description
cam_to_dart_input_file   character(len=128)   The name of the DART file containing the CAM state. + [default: 'caminput.nc']
cam_to_dart_output_file   character(len=128)   The name of the DART file containing the model state + derived from the CAM restart file. + [default: 'dart_ics']
+ +
+
+ + + + +
+

MODULES USED

+
+assim_model_mod.f90
+types_mod.f90
+threed_sphere/location_mod.f90
+model_mod.f90
+null_mpi_utilities_mod.f90
+obs_kind_mod.f90
+random_nr_mod.f90
+random_seq_mod.f90
+time_manager_mod.f90
+utilities_mod.f90
+
+ + + + + + +
+

FILES Read

+
    +
  • DART namelist file; input.nml
    • +
  • CAM restart file; caminput.nc
    • +
  • CAM "phis" file specified in model_nml&cam_phis + (normally cam_phis.nc)
    • +
+ +

FILES Written

+
  • DART initial conditions/restart file; e.g. dart_ics
    • +
+ + + + + + +
+

REFERENCES

+

+none +

+ + + + + + +
+

ERROR CODES and CONDITIONS

+

+none - all error messages come from modules that have their own documentation. +

+ +

KNOWN BUGS

+

+none +

+ + + + + + +
+

FUTURE PLANS

+

+None. +

+ + + + + + +
+

Terms of Use

+ +

+DART software - Copyright © 2004 - 2010 UCAR.
+This open source software is provided by UCAR, "as is",
+without charge, subject to all terms of use at
+ +http://www.image.ucar.edu/DAReS/DART/DART_download +

+ + + + + + + +
Contact: DART core group
Revision: $Revision$
Source: $URL$
Change Date: $Date$
Change history:  try "svn log" or "svn diff"
+ + + + + Property changes on: DART/trunk/models/cam/cam_to_dart.html ___________________________________________________________________ Added: svn:mime-type + text/html Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Modified: DART/trunk/models/cam/model_mod.nml =================================================================== --- DART/trunk/models/cam/model_mod.nml 2011-04-04 21:13:20 UTC (rev 4841) +++ DART/trunk/models/cam/model_mod.nml 2011-04-04 22:11:13 UTC (rev 4842) @@ -2,6 +2,7 @@ output_state_vector = .false., model_version = '3.0', model_config_file = 'caminput.nc', + cam_phis = 'cam_phis.nc', state_num_0d = 0, state_num_1d = 0, state_num_2d = 1, @@ -20,5 +21,7 @@ highest_state_pressure_mb = 150.0, max_obs_lat_degree = 90.0, Time_step_seconds = 21600, - Time_step_days = 0 / + Time_step_days = 0, + impact_only_same_kind = ' ', + / From nancy at ucar.edu Tue Apr 5 08:49:30 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 05 Apr 2011 08:49:30 -0600 Subject: [Dart-dev] [4843] DART/trunk/models/cam: Added capability to read CAM restart files that have dates that preceed the Message-ID: Revision: 4843 Author: thoar Date: 2011-04-05 08:49:29 -0600 (Tue, 05 Apr 2011) Log Message: ----------- Added capability to read CAM restart files that have dates that preceed the start of the Gregorian calendar. Simply adding 1601 to the year for all years that are before 1601. The resulting DART ics file can be modified with the restart_file_tool. Modified Paths: -------------- DART/trunk/models/cam/cam_to_dart.html DART/trunk/models/cam/model_mod.f90 -------------- next part -------------- Modified: DART/trunk/models/cam/cam_to_dart.html =================================================================== --- DART/trunk/models/cam/cam_to_dart.html 2011-04-04 22:11:13 UTC (rev 4842) +++ DART/trunk/models/cam/cam_to_dart.html 2011-04-05 14:49:29 UTC (rev 4843) @@ -50,6 +50,14 @@ restart_file_tool if desired.

+ Some CAM restart files are from climatological runs and have a valid time that + predates the use of the Gregorian calendar. In such instances, the year component + of the original date is changed to be a valid Gregorian year (by adding 1601). + A warning is issued to the screen and to the logfile. Please use the + restart_file_tool to change + this time. +
+
Conditions required for successful execution of cam_to_dart:

Modified: DART/trunk/models/cam/model_mod.f90 =================================================================== --- DART/trunk/models/cam/model_mod.f90 2011-04-04 22:11:13 UTC (rev 4842) +++ DART/trunk/models/cam/model_mod.f90 2011-04-05 14:49:29 UTC (rev 4843) @@ -556,7 +556,7 @@ logical :: do_out ! common message string used by many subroutines -character(len=129) :: msgstring +character(len=129) :: msgstring, string2 ! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - integer :: nflds ! # fields to read @@ -1762,11 +1762,27 @@ imin = rem / 60 isec = rem - imin*60 + ! some cam files are from before the start of the gregorian calendar. + ! since these are 'arbitrary' years, just change the offset. + + if (iyear < 1601) then + write(logfileunit,*)' ' + write( * ,*)' ' + write(msgstring,*)'WARNING - ',trim(file_name),' changing year from ',iyear,'to',iyear+1601 + call error_handler(E_MSG, 'read_cam_init', msgstring, source, revision, & + revdate, text2='to make it a valid Gregorian date.') + write(logfileunit,*)' ' + write( * ,*)' ' + iyear = iyear + 1601 + endif + model_time = set_date(iyear,imonth,iday,ihour,imin,isec) if (do_out) then - call print_date(model_time,'read_cam_init:CAM input date') - call print_time(model_time,'read_cam_init:CAM input time') + call print_date(model_time,' read_cam_init ... input date') + call print_time(model_time,' read_cam_init ... input time') + call print_date(model_time,' read_cam_init ... input date',logfileunit) + call print_time(model_time,' read_cam_init ... input time',logfileunit) endif deallocate(datetmp, datesec) From nancy at ucar.edu Tue Apr 5 13:06:00 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 05 Apr 2011 13:06:00 -0600 Subject: [Dart-dev] [4844] DART/trunk/models/wrf/matlab/map_wrf.m: Simplified pcolor image of any field in either a wrfinput_d0x or Prior/ Posterior Message-ID: Revision: 4844 Author: thoar Date: 2011-04-05 13:06:00 -0600 (Tue, 05 Apr 2011) Log Message: ----------- Simplified pcolor image of any field in either a wrfinput_d0x or Prior/Posterior DART netCDF file. has examples on how to overlay coastlines, observation locations - faithfully preserves the shape of the WRF domain. Modified Paths: -------------- DART/trunk/models/wrf/matlab/map_wrf.m -------------- next part -------------- Modified: DART/trunk/models/wrf/matlab/map_wrf.m =================================================================== --- DART/trunk/models/wrf/matlab/map_wrf.m 2011-04-05 14:49:29 UTC (rev 4843) +++ DART/trunk/models/wrf/matlab/map_wrf.m 2011-04-05 19:06:00 UTC (rev 4844) @@ -1,17 +1,42 @@ -function h = map_wrf(fname, varname, copystring, levelindx, timeindx ) -%% map_wrf creates a map of a WRF field - please check what is supposed to -% happen with the staggering. +function h = map_wrf(fname, varname, levelindx, timeindx, copystring ) +%% map_wrf creates an image of a WRF field without using the mapping toolbox. % -% (rather slow) Example: -% fname = 'Prior_Diag.nc'; +% Example using a DART-style diagnostic file, i.e.: +% +% fname = '/glade/proj3/image/hliu/200812new/cwb_icbc/12_01/Prior_Diag.nc'; % varname = 'U_d01'; % copystring = 'ensemble mean'; % levelindx = 10; % timeindx = 1; +% map_wrf(fname, varname, levelindx, timeindx, copystring); +% worldmap; +% axis off; % -% map_wrf(fname, varname, copystring, levelindx, timeindx ); +% +% Example using a WRF netCDF file: +% +% fname = '/glade/proj3/image/hliu/ICBC_from_cwb/wrfinput_d01'; +% levelindx = 10; +% timeindx = 1; +% map_wrf(fname, 'U', levelindx, timeindx); +% worldmap; % superimpose some low-res coastlines +% axis off; +% +% layer on the locations of some observations: +% +% obsfile = 'obs_epoch_001.nc'; +% ObsTypeString = 'RADIOSONDE_U_WIND_COMPONENT'; +% region = [0 360 -90 90 -Inf Inf]; +% CopyString = 'observation'; +% QCString = 'DART quality control'; +% verbose = 1; % anything > 0 == 'true' +% obs = read_obs_netcdf(obsfile, ObsTypeString, region, CopyString, QCString, verbose); +% hold on; +% plot(obs.lons, obs.lats, 'kd', 'MarkerFaceColor','k') +% -%% DART software - Copyright \xA9 2004 - 2010 UCAR. This open source software is + +%% DART software - Copyright ? 2004 - 2010 UCAR. This open source software is % provided by UCAR, "as is", without charge, subject to all terms of use at % http://www.image.ucar.edu/DAReS/DART/DART_download % @@ -30,51 +55,21 @@ error('Wrong number of arguments ... must have 3,4, or 5') end +% Check to ensure the file and the desired variable exist + if (exist(fname,'file') ~= 2) error('%s does not exist',fname) end -varexist(fname,{varname,'time','XLON_d01','XLAT_d01','level_d01', ... - 'TRUELAT1','TRUELAT2', 'CEN_LAT', 'CEN_LON', 'MAP_PROJ'}) +varexist(fname,{varname}) -copyindex = get_copy_index(fname,copystring); +%% Get the dimension information. -%% Read map metadata -times = nc_varget(fname, 'time'); Ntimes = size(times, 1); -xlon = nc_varget(fname, 'XLON_d01'); we = size( xlon, 2); -xlat = nc_varget(fname, 'XLAT_d01'); sn = size( xlat, 1); -level = nc_varget(fname,'level_d01'); bt = size(level, 1); +xy = GetWRFlatlon( fname, varname); +levels = GetWRFlevels( fname, varname); +times = GetWRFtimes( fname, varname); +timestrings = datestr(times,31); -calendar = nc_attget(fname, 'time','calendar'); -timeunits = nc_attget(fname, 'time','units'); -timebase = sscanf(timeunits,'%*s%*s%d%*c%d%*c%d'); % YYYY MM DD -timeorigin = datenum(timebase(1),timebase(2),timebase(3)); -timestrings = datestr(times + timeorigin, 0); - -stdlat1 = nc_varget(fname, 'TRUELAT1'); -stdlat2 = nc_varget(fname, 'TRUELAT2'); -cen_lat = nc_varget(fname, 'CEN_LAT'); -cen_lon = nc_varget(fname, 'CEN_LON'); -mp = nc_varget(fname, 'MAP_PROJ')+1; % convert [0,N-1] to [1,N] -map_proj = nc_attget(fname, 'MAP_PROJ','units'); -inds = find((map_proj == ',') | (map_proj == '=')); -map_proj(inds) = ' '; -[a b] = strread(map_proj,'%s %s','delimiter',' '); -map_proj = lower(b); - -%% Check to make sure longitudes are [0, 360] -inds = xlon < 0; -xlon(inds) = xlon(inds) + 360; -minlat = min(xlat(:)); maxlat = max(xlat(:)); -minlon = min(xlon(:)); maxlon = max(xlon(:)); -hunk.xlat = xlat; -hunk.xlon = xlon; - -%% Set up map projection -% may have to translate their projections to matlabs keywords - -fprintf('map projection is %s\n',map_proj{mp}) - %% Get the variable units, description, etc. % Determine variable to plot ... @@ -93,15 +88,14 @@ end end -% iso = [0.5:1:5]; -% iso = [0.01:0.01:0.1]; -% iso = [50:50:300]; -% iso = [0.5:0.5:5]; -% iso = [100:100:600]; -% iso = [0.0001:0.0001:0.001]; -% iso = [0.00001:0.00001:0.0001]; -% iso = [0.00001:0.00001:0.0001]; +copydim = find(strncmp('copy',varinfo.Dimension,length('copy'))); +if (isempty(copydim)) + copyindex = NaN; +else + copyindex = get_copy_index(fname,copystring); +end + for itime = timeindx plot_title = {fname, ... @@ -110,72 +104,23 @@ %% Determine the hyperslab indexing myinfo.diagn_file = fname; - myinfo.copyindex = copyindex; + if (isfinite(copyindex)), myinfo.copyindex = copyindex; end myinfo.levelindex = levelindx; myinfo.timeindex = timeindx; [start, count] = GetNCindices(myinfo, 'diagn', varname); % Extract field - stag_field = nc_varget(fname, varname, start, count); - field = zeros(sn,we); - - switch lower(varname) - case {'u','u10_d01','u_d01'} - for iy = 1:sn - for ix = 1:we - field(iy,ix) = (stag_field(iy,ix) + stag_field(iy,ix+1))/2.0; - end - end - case {'v','v10_d01','v_d01'} - for iy = 1:sn - for ix = 1:we - field(iy,ix) = (stag_field(iy,ix) + stag_field(iy+1,ix))/2.0; - end - end - otherwise - field = stag_field; - end + datmat = double(nc_varget(fname, varname, start, count)); - % Plot field + clf; + h = pcolor(xy.lonmat, xy.latmat, datmat); + set(h,'LineStyle','none'); - % subplot(m,m,pane); + title(plot_title,'Interpreter','none') + h2 = colorbar('vert'); + set(get(h2,'YLabel'),'String',varinfo.units) - switch lower(map_proj{mp}) - case 'mercator' - axesm(map_proj{mp}, ... - 'MapParallels',[stdlat1], ... - 'MapLatLimit',[minlat maxlat], ... - 'MapLonLimit',[minlon maxlon]); - h = gridm('on'); - % mlabel on; - % plabel on; - - otherwise - fprintf('projection %s is not yet supported\n',map_proj{mp}) - end - -% states = shaperead('usastatelo', 'UseGeoCoords', true); -% geoshow([states.Lat],[states.Lon],'Color','black'); - - landareas = shaperead('landareas', 'UseGeoCoords', true); - geoshow([landareas.Lat],[landareas.Lon],'Color','black'); - - if min(field(:)) ~= max(field(:)) - -% disp('contouring (slow) ...') -% [C h]=contourm(xlat,xlon,field) ; -% h = clabelm(C,h,'labelspacing',288); set(h,'Fontsize',9); - - h1 = surfm(xlat,xlon,field); - - title(plot_title,'Interpreter','none') - h2 = colorbar('vert'); - set(get(h2,'YLabel'),'String',varinfo.units) - else - error('field is all uniform value') - end - end @@ -190,7 +135,7 @@ for i = 1:nvars gotone(i) = nc_isvar(filename,varnames{i}); if ( ~ gotone(i) ) - fprintf('\n%s is not a variable in %s\n',varnames{i},filename) + warning('\n%s is not a variable in %s\n',varnames{i},filename) end end @@ -198,3 +143,61 @@ error('missing required variable ... exiting') end + + +function xy = GetWRFlatlon(fname, varname); +%% Each of the WRF variables has a 'coordinate' attribute signifying which +% of the 6 possible lat/lon variables is appropriate. + +coordinates{1} = sscanf(nc_attget(fname,varname,'coordinates'),'%s %*s'); +coordinates{2} = sscanf(nc_attget(fname,varname,'coordinates'),'%*s %s'); + +latcoord = find(strncmp('XLAT', coordinates, length('XLAT')) > 0); +loncoord = find(strncmp('XLON', coordinates, length('XLON')) > 0); +xy.latmat = nc_varget(fname, coordinates{latcoord}); +xy.lonmat = nc_varget(fname, coordinates{loncoord}); +xy.latunits = nc_attget(fname, coordinates{latcoord},'units'); +xy.lonunits = nc_attget(fname, coordinates{latcoord},'units'); + +inds = (xy.lonmat < 0); % Convert to 0,360 to minimize dateline probs. +xy.lonmat(inds) = xy.lonmat(inds) + 360.0; + + + +function levels = GetWRFlevels(fname, varname); +%% Return the appropriate vertical indices. + +varinfo = nc_getvarinfo(fname,varname); +lvlind = find(strncmp('bottom_top',varinfo.Dimension,length('bottom_top'))); +nlevels = varinfo.Size(lvlind); +levels = 1:nlevels; + + + +function times = GetWRFtimes(fname, varname); +%% Return the appropriate time coordinates. +% DART files use 'time' as a real array +% WRF files use 'Time' as a character string + +varinfo = nc_getvarinfo(fname,varname); +timeind = find(strncmp('time',varinfo.Dimension,length('time'))); + +if (isempty(timeind)) % WRF flavor Time variable. + timeind = find(strncmp('Time',varinfo.Dimension,length('Time'))); + if (isempty(timeind)), error('%s has no time information. Dying.'); end + timestr = nc_varget(fname, 'Times'); % Note the plural ... seeth. + timebase = sscanf(timestr,'%d%*c%d%*c%d%*c%d%*c%d%*c%d'); % YYYY MM DD HH MM SS + times = datenum(timebase'); + +else + times = nc_varget(fname, 'time'); + calendar = nc_attget(fname, 'time','calendar'); + timeunits = nc_attget(fname, 'time','units'); + timebase = sscanf(timeunits,'%*s%*s%d%*c%d%*c%d'); % YYYY MM DD + timeorigin = datenum(timebase(1),timebase(2),timebase(3)); + times = times + timeorigin; + +end + + + From thoar at ucar.edu Tue Apr 5 13:12:47 2011 From: thoar at ucar.edu (Tim Hoar) Date: Tue, 5 Apr 2011 13:12:47 -0600 Subject: [Dart-dev] [4844] DART/trunk/models/wrf/matlab/map_wrf.m: Simplified pcolor image of any field in either a wrfinput_d0x or Prior/ Posterior In-Reply-To: References: Message-ID: Just to demonstrate what this function does ... this graphic is the result of running the example (from 'help map_wrf') The function does NOT require a Mapping Toolbox license. removing the coastline bits outside the WRF domain is a bit tricky ... but if you work in the CONUS domain, I have a gift ... just ask. Tim On Apr 5, 2011, at 1:06 PM, nancy at ucar.edu wrote: > Revision: 4844 > Author: thoar > Date: 2011-04-05 13:06:00 -0600 (Tue, 05 Apr 2011) > Log Message: > ----------- > Simplified pcolor image of any field in either a wrfinput_d0x or Prior/Posterior > DART netCDF file. has examples on how to overlay coastlines, observation locations - > faithfully preserves the shape of the WRF domain. > > Modified Paths: > -------------- > DART/trunk/models/wrf/matlab/map_wrf.m > > _______________________________________________ > Dart-dev mailing list > Dart-dev at mailman.ucar.edu > http://mailman.ucar.edu/mailman/listinfo/dart-dev Tim Hoar, Associate Scientist National Center for Atmospheric Research thoar at ucar.edu 303 497 1708 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ucar.edu/pipermail/dart-dev/attachments/20110405/4004a6ca/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: map_wrf_demo.png Type: image/png Size: 200991 bytes Desc: not available Url : http://mailman.ucar.edu/pipermail/dart-dev/attachments/20110405/4004a6ca/attachment-0001.png From nancy at ucar.edu Tue Apr 5 13:35:55 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 05 Apr 2011 13:35:55 -0600 Subject: [Dart-dev] [4845] DART/trunk/diagnostics/matlab: Changed the examples to use current default filenames. Message-ID: Revision: 4845 Author: thoar Date: 2011-04-05 13:35:55 -0600 (Tue, 05 Apr 2011) Log Message: ----------- Changed the examples to use current default filenames. obs_seq_to_netcdf has default filenames like 'obs_epoch_xxx.nc' plot_rmse_xxx_profile.m and two_experiments_profile.m are beautification changes ... Modified Paths: -------------- DART/trunk/diagnostics/matlab/plot_obs_netcdf.m DART/trunk/diagnostics/matlab/plot_obs_netcdf_diffs.m DART/trunk/diagnostics/matlab/plot_rmse_xxx_profile.m DART/trunk/diagnostics/matlab/read_obs_netcdf.m DART/trunk/diagnostics/matlab/two_experiments_profile.m -------------- next part -------------- Modified: DART/trunk/diagnostics/matlab/plot_obs_netcdf.m =================================================================== --- DART/trunk/diagnostics/matlab/plot_obs_netcdf.m 2011-04-05 19:06:00 UTC (rev 4844) +++ DART/trunk/diagnostics/matlab/plot_obs_netcdf.m 2011-04-05 19:35:55 UTC (rev 4845) @@ -8,7 +8,7 @@ %-------------------------------------------------- % EXAMPLE 1: plotting just one type of observation %-------------------------------------------------- -% fname = 'obs_sequence_001.nc'; +% fname = 'obs_epoch_001.nc'; % ObsTypeString = 'RADIOSONDE_U_WIND_COMPONENT'; % region = [0 360 -90 90 -Inf Inf]; % CopyString = 'NCEP BUFR observation'; @@ -22,7 +22,7 @@ %-------------------------------------------------- % EXAMPLE 2: plotting all the observation types %-------------------------------------------------- -% fname = 'obs_sequence_001.nc'; +% fname = 'obs_epoch_001.nc'; % ObsTypeString = 'ALL'; % region = [0 360 -90 90 -Inf Inf]; % CopyString = 'WOD observation'; Modified: DART/trunk/diagnostics/matlab/plot_obs_netcdf_diffs.m =================================================================== --- DART/trunk/diagnostics/matlab/plot_obs_netcdf_diffs.m 2011-04-05 19:06:00 UTC (rev 4844) +++ DART/trunk/diagnostics/matlab/plot_obs_netcdf_diffs.m 2011-04-05 19:35:55 UTC (rev 4845) @@ -28,7 +28,7 @@ % The 'copies' are recorded in the netCDF 'CopyMetaData' variable - % the observation types are recorded in the 'ObsTypesMetaData' variable, % and the QC strings of interest are recorded in QCMetaData - so -% ncdump -v CopyMetaData,ObsTypesMetaData,QCMetaData obs_sequence_001.nc +% ncdump -v CopyMetaData,ObsTypesMetaData,QCMetaData obs_epoch_001.nc % is a useful endeavor. % % $Id$ @@ -38,7 +38,7 @@ % and the actual observation value - while rejecting any obs that had % a DART QC greater than 3 ( prior forward operator failed ... or worse) %-------------------------------------------------- -% fname = 'obs_sequence_001.nc'; +% fname = 'obs_epoch_001.nc'; % ObsTypeString = 'RADIOSONDE_U_WIND_COMPONENT'; % region = [0 360 -90 90 -Inf Inf]; % CopyString1 = 'NCEP BUFR observation'; Modified: DART/trunk/diagnostics/matlab/plot_rmse_xxx_profile.m =================================================================== --- DART/trunk/diagnostics/matlab/plot_rmse_xxx_profile.m 2011-04-05 19:06:00 UTC (rev 4844) +++ DART/trunk/diagnostics/matlab/plot_rmse_xxx_profile.m 2011-04-05 19:35:55 UTC (rev 4845) @@ -18,7 +18,6 @@ % copystring = 'totalspread'; % 'copy' string == quantity of interest % plotdat = plot_rmse_xxx_profile(fname,copystring); - %% DART software - Copyright ? 2004 - 2010 UCAR. This open source software is % provided by UCAR, "as is", without charge, subject to all terms of use at % http://www.image.ucar.edu/DAReS/DART/DART_download Modified: DART/trunk/diagnostics/matlab/read_obs_netcdf.m =================================================================== --- DART/trunk/diagnostics/matlab/read_obs_netcdf.m 2011-04-05 19:06:00 UTC (rev 4844) +++ DART/trunk/diagnostics/matlab/read_obs_netcdf.m 2011-04-05 19:35:55 UTC (rev 4845) @@ -3,7 +3,7 @@ %% read_obs_netcdf reads in the netcdf flavor observation sequence file % and returns a subsetted structure. % -% fname = 'obs_sequence_001.nc'; +% fname = 'obs_epoch_001.nc'; % ObsTypeString = 'RADIOSONDE_U_WIND_COMPONENT'; % or 'ALL' ... % region = [0 360 -90 90 -Inf Inf]; % CopyString = 'NCEP BUFR observation'; @@ -14,7 +14,7 @@ % % The return variable 'obs' is a structure. As an example ... % -% fname: 'obs_sequence_001.nc' +% fname: 'obs_epoch_001.nc' % ObsTypeString: 'RADIOSONDE_U_WIND_COMPONENT' % region: [0 360 -90 90 -Inf Inf] % CopyString: 'NCEP BUFR observation' Modified: DART/trunk/diagnostics/matlab/two_experiments_profile.m =================================================================== --- DART/trunk/diagnostics/matlab/two_experiments_profile.m 2011-04-05 19:06:00 UTC (rev 4844) +++ DART/trunk/diagnostics/matlab/two_experiments_profile.m 2011-04-05 19:35:55 UTC (rev 4845) @@ -21,7 +21,7 @@ % prpo = 'forecast'; % [guess, forecast, prior ] == prior % % two_experiments_profile(files, titles, varnames, qtty, prpo) -% print -dpsc -painters myplot.ps +% print -dpdf myplot.pdf % % From nancy at ucar.edu Tue Apr 5 13:40:07 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 05 Apr 2011 13:40:07 -0600 Subject: [Dart-dev] [4846] DART/trunk/models/wrf/matlab/map_wrf.m: better example syntax Message-ID: Revision: 4846 Author: thoar Date: 2011-04-05 13:40:07 -0600 (Tue, 05 Apr 2011) Log Message: ----------- better example syntax Modified Paths: -------------- DART/trunk/models/wrf/matlab/map_wrf.m -------------- next part -------------- Modified: DART/trunk/models/wrf/matlab/map_wrf.m =================================================================== --- DART/trunk/models/wrf/matlab/map_wrf.m 2011-04-05 19:35:55 UTC (rev 4845) +++ DART/trunk/models/wrf/matlab/map_wrf.m 2011-04-05 19:40:07 UTC (rev 4846) @@ -1,7 +1,7 @@ function h = map_wrf(fname, varname, levelindx, timeindx, copystring ) %% map_wrf creates an image of a WRF field without using the mapping toolbox. % -% Example using a DART-style diagnostic file, i.e.: +%% Example using a DART-style diagnostic file, i.e.: % % fname = '/glade/proj3/image/hliu/200812new/cwb_icbc/12_01/Prior_Diag.nc'; % varname = 'U_d01'; @@ -13,7 +13,7 @@ % axis off; % % -% Example using a WRF netCDF file: +%% Example using a WRF netCDF file: % % fname = '/glade/proj3/image/hliu/ICBC_from_cwb/wrfinput_d01'; % levelindx = 10; @@ -22,7 +22,7 @@ % worldmap; % superimpose some low-res coastlines % axis off; % -% layer on the locations of some observations: +%% layer on the locations of some observations: % % obsfile = 'obs_epoch_001.nc'; % ObsTypeString = 'RADIOSONDE_U_WIND_COMPONENT'; From nancy at ucar.edu Wed Apr 6 09:19:44 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 06 Apr 2011 09:19:44 -0600 Subject: [Dart-dev] [4849] DART/trunk/observations/var: updates from glen Message-ID: Revision: 4849 Author: nancy Date: 2011-04-06 09:19:43 -0600 (Wed, 06 Apr 2011) Log Message: ----------- updates from glen Modified Paths: -------------- DART/trunk/observations/var/3DVAR_OBSPROC/README DART/trunk/observations/var/work/path_names_gts_to_dart -------------- next part -------------- Modified: DART/trunk/observations/var/3DVAR_OBSPROC/README =================================================================== --- DART/trunk/observations/var/3DVAR_OBSPROC/README 2011-04-05 20:01:52 UTC (rev 4848) +++ DART/trunk/observations/var/3DVAR_OBSPROC/README 2011-04-06 15:19:43 UTC (rev 4849) @@ -1,19 +1,31 @@ # DART software - Copyright \xA9 2004 - 2010 UCAR. This open source software is # provided by UCAR, "as is", without charge, subject to all terms of use at # http://www.image.ucar.edu/DAReS/DART/DART_download -# -# DART $Id$ -This directory needs to have the following source code -files from the WRF-Var release, the var/obsproc directory: +Before building the executables in the work directory you +will need to copy some of the WRF distribution files into +this directory. The files listed below are what is needed +with WRFDA v3.2.1 (the current release at this time). + +From WRFDA/var/obsproc/MAP_plot/Dir_map, copy to here: + +DA_Constants.f90 DA_Define_Structures.f90 -DA_Constants.f90 -module_obs.f90 +DA_Read_Obs.inc +DA_Read_Obs_Info.inc +DA_Setup_Obs_Structures.inc +module_obs.F90 -Copy them here before trying to build the executables in -the work directory. +As well as from WRFDA/var/obsproc/MAP_plot/include, copy to here: +maps.incl +nestdmn.incl + + See here for more info on this code: http://www.mmm.ucar.edu/wrf/WG4/wrfvar/3dvar_obsproc.htm + + +# DART $Id$ Modified: DART/trunk/observations/var/work/path_names_gts_to_dart =================================================================== --- DART/trunk/observations/var/work/path_names_gts_to_dart 2011-04-05 20:01:52 UTC (rev 4848) +++ DART/trunk/observations/var/work/path_names_gts_to_dart 2011-04-06 15:19:43 UTC (rev 4849) @@ -2,7 +2,7 @@ observations/var/gts_dart_mod.f90 observations/var/3DVAR_OBSPROC/DA_Constants.f90 observations/var/3DVAR_OBSPROC/DA_Define_Structures.f90 -observations/var/3DVAR_OBSPROC/module_obs.f90 +observations/var/3DVAR_OBSPROC/module_obs.F90 obs_sequence/obs_sequence_mod.f90 obs_def/obs_def_mod.f90 assim_model/assim_model_mod.f90 From nancy at ucar.edu Fri Apr 8 16:33:23 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 08 Apr 2011 16:33:23 -0600 Subject: [Dart-dev] [4850] DART/trunk/location/threed_sphere/location_mod.html: Add Jeff' s discussion of computing the minimium longitudes at any given Message-ID: Revision: 4850 Author: nancy Date: 2011-04-08 16:33:23 -0600 (Fri, 08 Apr 2011) Log Message: ----------- Add Jeff's discussion of computing the minimium longitudes at any given latitude which are within a given radius. Also fix a typo in an href down in the namelist. minor other reformatting, remove extra

, etc. Modified Paths: -------------- DART/trunk/location/threed_sphere/location_mod.html -------------- next part -------------- Modified: DART/trunk/location/threed_sphere/location_mod.html =================================================================== --- DART/trunk/location/threed_sphere/location_mod.html 2011-04-06 15:19:43 UTC (rev 4849) +++ DART/trunk/location/threed_sphere/location_mod.html 2011-04-08 22:33:23 UTC (rev 4850) @@ -43,12 +43,11 @@ read and write location information to files. DART isolates all this location information into separate modules so that the main algorithms can operate with the same code independent -of whether the model uses latitude/longitude/height, one-d unit +of whether the model uses latitude/longitude/height, 1D unit sphere coordinates, cylindrical coordinates, etc. DART provides about half a dozen possible coordinate systems, and others can be added. The most common one for geophysical models is this one: threed_sphere. -

-

+

This location module provides a representation of a physical location on a 3-D spherical shell, using latitude and longitude plus a vertical component with choices of vertical coordinate type such as pressure or @@ -69,8 +68,7 @@ is controlled by which source file name is specified in the path_names_xxx file, which is used by the mkmf_xxx scripts. -

-

+

All types of location modules define the same Fortran 90 derived type location_type. Programs that need to pass location information to subroutines @@ -92,8 +90,7 @@ a single observation and items in the state vector, and also between a single observation and other nearby observations. However, any source for locations is supported. -

-

+

In simpler location modules (like the oned version) there is no need for anything other than a brute force search between the base location and all available state vector locations. However in @@ -103,7 +100,7 @@ The location code pre-processes all locations into a set of bins and then only needs to search the lists of locations in nearby bins when looking for locations that are within a specified distance. - +

The expected calling sequence of the get_close routines is as follows: 

@@ -124,15 +121,13 @@
 compute the exact separation distance from the reference location.
 The last routine deallocates the space.  See the documentation 
 below for the specific details for each routine.
-

-

+
 

 All 4 of these routines must be present in every location module
 but in most other versions
 all but get_close_obs() are stubs.
 In this threed_sphere version
 of the locations module all are fully implemented.
 

-

 
Interaction with model_mod.f90 code

 
 

@@ -146,8 +141,7 @@
 from the location_mod module.
 Thus they 'pass through' the model_mod but the user does
 not need to provide a subroutine or any code for them.
-

-

+
 

 However, if the model interface code wants to intercept and alter
 the default behavior of the get_close routines, it is able to.
 Typically the model_mod still calls the location_mod routines and
@@ -171,8 +165,7 @@
 through to the code in the location_mod, but the model_mod must supply
 a get_close_obs() subroutine.  When it wants to call the code in the 
 location_mod it calls location_get_close_obs().
-

-

+
 

 One use pattern is for the model_mod to call the location get_close_obs()
 routine without the dist argument.  This returns a list
 of any potentially close locations without computing the exact distance
@@ -190,57 +183,34 @@
 particular unit type, this is the place where the conversion to that
 vertical unit should be done.
 

-
-
-
-
-
-
-

-
NAMELIST

-
We adhere to the F90 standard of starting a namelist with an ampersand
-'&' and terminating with a slash '/' for all our namelist input.
-Consider yourself forewarned that character strings that contain a '/' must be
-enclosed in quotes to prevent them from prematurely terminating the namelist.
-The namelist declaration (i.e. what follows) has a different syntax, naturally.
-

-

-
-namelist / location_nml /  &
-   horiz_dist_only, vert_normalization_pressure, vert_normalization_height, &
-   vert_normalization_level, vert_normalization_scale_height, &
-   approximate_distance, nlon, nlat, output_box_info, maintain_original_vert
-

-

-
-

-
-
+
Horizontal Distance Only

 

-Items in this namelist either control the way in which distances 
-are computed and/or influence the code performance.
-


-If horiz_distance_only is .false.
+If horiz_distance_only is .true. in the namelist, then the
+vertical coordinate is ignored and only the great-circle distance
+between the two locations is computed, as if they were both on the
+surface of the sphere. 
+
 

+If horiz_distance_only is .false. in the namelist
 then the appropriate normalization constant determines the relative impact of
 vertical and horizontal separation.  Since only a single localization
 distance is specified, and the vertical scales might have very different
 distance characteristics, the vert_normalization_xxx values can be used
 to scale the vertical appropriately to control the desired influence 
 of observations in the vertical.
-


+

+
Precomputation for Run-time Search Efficiency

+

 For search efficiency all locations are pre-binned. The surface of the
 sphere is divided up into nlon by nlat boxes and the
 index numbers of all items (both state vector entries and observations)
 are stored in the appropriate box.  
 To locate all points close to a given location, only the locations
 listed in the boxes within the search radius must be checked.  
-
-


+
 

 The default values have given good performance on many of our
 existing model runs, but for tuning purposes the box counts
 have been added to the namelist to allow adjustment.  
@@ -260,7 +230,7 @@
 distributed across boxes. 
 The absolute numbers for best performance will certainly
 vary from case to case.
-


+
 

 For latitude, the nlat boxes are distributed evenly across the
 actual extents of the data. (Locations are in radians, so the
 maximum limits are the poles at -PI/2 and +PI/2).   
@@ -280,7 +250,91 @@
 across the entire surface of the sphere.  
 For local or regional models, the boxes
 are distributed only across the the extent of the local grid.
-


+
 

+For efficiency in the case where the boxes span less than half the globe,
+the 3D location module needs to be able to determine the greatest longitude 
+difference between a base point at latitude φs
+and all points that are separated from that point by a central 
+angle of θ. 
+We might also want to know the latitude, φf , 
+at which the largest separation occurs. Note also that an intermediate 
+form below allows the computation of the maximum longitude difference 
+at a particular latitude.
+
 

+The central angle between a point at latitude φs
+and a second point at latitude φf that are separated 
+in longitude by Δλ is 
 

+
+  &theta = cos-1(sinφssinφf + 
+   cosφscos&phifcosΔλ) 
+ 
 

+Taking the cos of both sides gives 
 

+
+  cosθ = (sinφssinφf +
+   cosφscosφfcosΔλ)
+ 
 

+Solving for cosΔλ gives 
 

+
+  cosΔλ 
+    = (a - b sin&phif)/(c cosφf)
+    = a/c secφf - 
+      b/c tanφf 
+ 
 

+where a = cosθ , 
+b = sinφs , 
+and c = cos&phis . 
+We want to maximize Δλ which implies minimizing   
+cosΔλ subject to constraints. Taking the 
+derivative with respect to φf gives  
 

+
+  (d cosΔλ)/(dφf) = 
+   a/c secφf tanφf
+ - b/c sec2φf = 0
+ 
 

+Factoring out secφf which can never be 0 and using the
+definitions of sec and tan gives 
 

+
+  (a sinφf)/(c cosφf) -
+b/(c cosφf) = 0
+ 
 

+Solving in the constrained range from 0 to PI/2 gives 
 

+
+   sinφf = b/a = 
+   sinφs/cosθ
+ 
 

+Note that if the angle between the base point and a pole is less than 
+or equal to the central angle, all longitude differences will occur 
+as the pole is approached.
+

+
+
+
+
+
+
+

+
NAMELIST

+
We adhere to the F90 standard of starting a namelist with an ampersand
+'&' and terminating with a slash '/' for all our namelist input.
+Consider yourself forewarned that character strings that contain a '/' must be
+enclosed in quotes to prevent them from prematurely terminating the namelist.
+The namelist declaration (i.e. what follows) has a different syntax, naturally.
+

+

+
+namelist / location_nml /  &
+   horiz_dist_only, vert_normalization_pressure, vert_normalization_height, &
+   vert_normalization_level, vert_normalization_scale_height, &
+   approximate_distance, nlon, nlat, output_box_info, maintain_original_vert
+

+

+
+

+
+
+

+Items in this namelist either control the way in which distances 
+are computed and/or influence the code performance.
 This namelist is read in a file called input.nml .
 

 
@@ -311,7 +365,7 @@
maintain_original_vert logical If using per-type specialized localization distances - (see assim_tools_mod) by default the vertical distances are altered also. To maintain the original vertical scaling while changing the horizontal, set this to true. Default: .false.
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SCRIPT DOMAIN LOCATION OR NOTES
Exper_obsseq#.scriptexperimentexperiment central directory where I/O and execution are organized.
   -> mpirun filter executable
filter version local disc on a compute node/processor, or a work directory + in the central directory.
      -> advance_model.csh
 model pre-existing work subdirectory of the central directory.
         -> trans_time executable
 model Translates DART time format into CAM time format
         -> trans_sv_pv executable
 model Inserts DART state vector into CAM initial file
         -> run_pc.csh
 model Modified form of run script from CAM, now in DART 
            -> bld-namelist
model uses namelistin and results of trans_time to make CAM namelist 
            -> advance of CAM
 model Single threaded or MPI CAM 
         -> trans_pv_sv executable
 Translates CAM fields in to DART state vector
   -> qsub auto_re2ms.csh
 machine Central/Experiment/obs_seq_output_directory(i.e. 01_01)
+ + + + + + + + +
+

EXPERIMENT SET-UP

+

+Instructions for setting up a DART-CAM assimilation. +

+ +
    +
  1. DART +
      +
    1. In .../DART/mkmf link mkmf.template to + the mkmf.template.xxxx which is appropriate for your computer + (or make one of your own).
      2. +
    2. Since you have this file you've already checked out DART.
      + Edit ...DART/models/cam/work/input.nml:preprocess_nml:input_files + to choose obs_def source code files to load via the preprocessor. + The default file (obs_def_reanlysis_bufr_mod.f90) will handle + observations from NCEP reanalysis BUFR files. (Warning; assimilating the + obs in the example GPS obs_seq.out file available from the + DART large file site + requires loading more than just the obs_def_gsp.mod.f90.)
      3. +
    3. Script DART/models/cam/work/workshop_setup.csh is recommended for compiling + the package. + It is set up to compile and run the preprocessor, + compile filter and wakeup_filter as MPI processes (async=4), + and compile all other executables as single threaded. + If you want to use async=2, remove the -mpi flag from the calls + to mkmf_filter and mkmf_wakeup_filter.
      4. +
    2. + +
  2. CAM +
      +
    1. Put the DART modifications + (.../DART/models/cam/Cam[version#]_DART_mods/*), + and any other CAM modifications you have, + in the CAM directory of user-provided modifications. + CAM's "configure" will use these during compilation. + Configure and compile CAM at the resolution/dynamical core desired. + Do this in a directory where all your CAM versions will reside, + here called CamCentral.
      2. + +
    2. Put the cam executable and config_cache.xml in a subdirectory of + the standard cam executable directory (CamCentral/CAM_version/models/atm/cam/bld), + which I'll call CAM_config_1-mpi (leave off the -mpi for single threaded CAM). + Job_mpi.csh has a variable "CAMsrc" that should point to that location.
      3. +
    3. Build a CAM namelist and call it 'namelistin' containing (among the other/default + variables defined by the CAM build-namelist): + 
      +          &camexp
+          ncdata         = 'caminput.nc'
+          caseid         = 'whatever_you_want'
+          nsrest         = 0
+          calendar       = 'GREGORIAN'
+          inithist       = 'ENDOFRUN'
+          /
+          &clmexp
+          finidat        = 'clminput.nc'
+          /
+
      + and NOT containing ... + 
      +          >  nhtfrq         = 4368
+          >  start_ymd      = 20020901
+          >  start_tod      = 0
+          >  stop_ymd       = 20021201
+          >  stop_tod       = 0
+
      + The CAM build-namelist script will use this to make a new namelist with the correct + forecast parameters, named 'namelist'. + Put this in CamCentral/CAM_version/models/atm/cam/bld/CAM_config_1-mpi.
      4. + +
    4. Confirm that you have access to all the CAM input files listed in namelistin, + or suitable replacements.
      5. + +
    5. Put an appropriate CAM initial file in CAM_config_1-mpi and call it caminput.nc. + Put a matching CLM initial file there and call it clminput.nc. + Only the grid information is used from these files, so their dates don't matter. +
      6. +
    3. + +
  3. Set up an experiment central directory ("Central" here) where there's + enough space for output.
    4. + +
  4. Copy the DART namelist file (DART/models/cam/work/input.nml) + into one called "input_1.nml" in Central
    5. + +
  5. Copy DART/models/cam/shell_scripts/job_mpi.csh to Central
    6. + +
  6. EXPERIMENT +
      +
    1. Edit job_mpi.csh. It has more detailed instructions about how to: +
      • define experiment output directory names
        • +
      • tell it whether CAM is MPI
        • +
      • provide directory name of CAM executable and associated files
        • +
      • define which obs_seq.out files to use. + Some pre-made examples can be found + here.
        • +
      • find and link to the obs_seq.out files
        • +
      • find and link to filter_ic[.#] files. + Such an ensemble can be created from CAM initial files using + + .../DART/models/cam/trans_pv_sv_time0.f90.
        • +
      • define which CAM and CLM initial files to use. + Some initial and filter_ic files are available from + DART large file site
        • +
      • define which DART version to use
        • +
      • define resources to be requested by each of the obs_seq_#.script scripts
        • +
      +
      2. + +
    2. Edit input_1.nml to configure the assimilation of the first obs_seq.out. + Be sure that +
        +
      • filenames listed in it agree with what's required by job_mpi.csh + and what is or will be available in the Central directory.
        • +
      • start from restart is .true. if you have an initial ensemble, + .false. otherwise.
        • +
      • init_time_days is the first Gregorian day of your assimilation. + This can be obtained from the program + + ...DART/models/cam/trans_date_to_dart.f90
        • +
      • init_time_seconds is the first second of your assimilation (usually 0).
        • +
      +
      3. + +
    3. Copy input_1.nml to input_n.nml and edit input_n.nml. + Set start_from_restart to .true. + Change init_time_days = -1, init_time_seconds = -1.
      4. +
    4. If you need to make up synthetic observations get create_obs_sequence, + create_fixed_network_seq and perfect_model_obs. + Otherwise, use the obs_seq.out files provided + here
      5. +
    5. Run the experiment by executing job_mpi.csh, + either through the batch queue or interactively.
      6. +
    7. +
+ + + + +
+

FILE CONTENTS

+

+See the Jamaica release notes +for diagrams and flowcharts showing DART under the various async options. + +The contents of some of the files which appear there are listed here. +# refers to the list of ensemble members. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FILE CONTENTS or PURPOSE
assim_model_state_ic#the state vectors to be used as initial conditions + for the next model advance. Contains the state vector time, the target + time, and the state vector.
assim_model_state_ud#the updated state vectors returned by the model + advance. Contain the state vector time (was the target time) and the + state vector for one ensemble member.
filter_ic_old#s the initial conditions to be used by the filter for the + next assimilation of a single obs_seq.out file. There may be one of + these, or one for each ensemble member, named filter_ic_old.####, + where the #### means a 4 digit number such as 0001.
filter_ic_new#s same as filter_ic_new#s, except that it/they are produced + at the end of the assimilation, for use by the next assimilation.
input.nml the filter namelist file, containing the namelists for all the + necessary modules of the filter.
caminput.nc model initial file provides information about the model + to filter, such as state vector size, etc.
caminput_#.nc CAM has more fields in its initial files than + we use in the DART state vector. It's useful to carry these fields along from + advance to advance so that they don't need to spin-up as much at the beginning + of each advance. trans_sv_pv replaces the state vector fields in these + "shells" with the contents of assim_model_state_ic and leaves the other + fields alone.
namelists the forecast model may need namelist(s) to define its advance.
obs_seq.final the innovations in observation space which result from the + assimilation of all the chosen obs in obs_seq.out.
obs_seq.out the set of observations to be assimilated. How the observations + are distributed in time defines when the model advances happen.
Posterior_Diag.nc the state vector in model space after each + assimilation defined by the obs times in obs_seq.out.
Prior_Diag.nc the state vector in model space before each assimilation + defined by the obs times in obs_seq.out. It results from the previous model advance. +
True_State.nc the state vector in model space resulting from an execution + of perfect_model_obs. These are the model forecast values from which identity obs + are derived.
+ + + +
+

OUTPUT DIRECTORY

+

+Organization of output directories: +

+ + + + + + + + + + + + + + + + + + + + + + + +
DIRECTORY CREATOR CONTENTS and PURPOSE 
Central directory
User + Location of scripts and pass-through point for files during execution. + Typically named according defining characteristics of a *set* of + experiments; resolution, model, obs being assimilated, unique model state + variables, etc.
    Experiment
job_mpi.csh + Location of subdirectories of output and some diagnostics files. + Typically where the obs-space diagnostics are calculated using obs_diag. 
       Obsseq_#
Exper_obsseq#.script + Each holds the obs-space and model-space output from assimilating one obs_seq.out file. + It should be named according to the need for + obs_diag to see a name with the 2 digit month, underscore, and the + number within the series of obs_seq.out files, + i.e. 01_02 for the second obs_seq.final of a January case. 
           DART
Exper_obsseq#.script + Holds the filter restart files (named filter_ic[.#]) created at the end of + the filter run for this obs_seq.out. They're used by the next obs_seq.out + file to restart.
           CAM
 Exper_obsseq#.script + Holds the CAM initial file shells which carry along model fields which are not + DART state vector fields (preventing the repeated re-spin-up of those variables)
           CLM
Exper_obsseq#.script + Same as CAM, but for Community Land Model initial files.
+ +
+A typical pathname for a restart file in my case would be:
+/scratch/cluster/raeder/T21x80/Taper1/01_03/DART/filter_ic#
+                        |      |      |     |    restart file(s)
+                        |      |      |     DART restart file directory
+			|      |      Obs_seq (3rd obs_seq.out of a Jan case)
+                        |      Experiment (reduced influence of obs above 150 hPa)
+                        Central directory (resolution x num_ens_members)
+
+

+You may also want to make a subdirectory within Experiment +for each set of obs_space postscript and .dat files created by obs_diag and matlab. +

+ + + +
+

HELPFUL HINTS

+

+For async=2 use the ensemble size, available compute nodes, and processors/node to figure how many +nodes to request. Make this request in job_mpi.csh. For example, on a machine with +2 processors/node, and running an assimilation with a +typical ensemble of 20 members, it's efficient to request 5 nodes. +This will advance CAM in 2 batches of 10 (1 CAM/processor). + +

+ +Each batch of restart data can be saved to a mass store using (a modified) +auto_re2ms and retrieved using .../ms2restart. Execute the +commands with no arguments to see instructions. They package files of each ensemble +member together, and then bundle batches of ensemble members together for efficient +storage in a directory named similarly to the one where they exist on the cluster. + +

+ + + +Modify and use alias 'rmtemp' to remove the temporary files from +the central directory where the experiment is run, after a run bombs and +before running another experiment. +

+
+alias rmtemp 'rm *_ud* *_ic[0-9]* cam_*_temp* c[al]minput_[1-9]*.nc  \
+              *control filter_ic_old* obs_seq.out times'
+
+

+Needless to say, be careful that you don't name files you want to +keep in such a way that they'll be deleted by this. +

+ + + +
+

SPACE REQUIREMENTS

+

+Space requirements (per ensemble member) for several CAM resolutions. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
Resolution filter_ic CAM initial CLM initial Diagnostic (assuming no ensemble members)
T50.16 Mb0.3 Mb0.15 Mb1.3 Mb + obs_seq.final
T212.5 Mb4.5 Mb1.4 Mb21 Mb + obs_seq.final
T4210 Mb18 Mb4.5 Mb57 Mb + "
T8541 Mb74 Mb15 Mb342 Mb + "
+ +

+obs_seq.final typically ranges from 50-150 Mb, independent of model resolution. +

+ + + + + + +
+

Terms of Use

+ +

+DART software - Copyright © 2004 - 2010 UCAR.
+This open source software is provided by UCAR, "as is",
+without charge, subject to all terms of use at
+ +http://www.image.ucar.edu/DAReS/DART/DART_download +

+ + + + + + + +
Contact: Kevin Raeder
Revision: $Revision$
Source: $URL$
Change Date: $Date$
Change history:  try "svn log" or "svn diff"
+ + + + + Deleted: DART/trunk/models/cam/doc/index.html =================================================================== --- DART/trunk/models/cam/doc/index.html 2011-04-14 22:00:42 UTC (rev 4854) +++ DART/trunk/models/cam/doc/index.html 2011-04-14 22:04:06 UTC (rev 4855) @@ -1,550 +0,0 @@ - - - -DART-CAM setup - - - - - - - -
-CALLING TREE / -SET-UP / -FILE CONTENTS / -OUTPUT DIRECTORY / -HINTS / -SPACE / -TERMS OF USE -
- -

DART-CAM OVERVIEW

-

$Id$

- -

-The up-to-date overview will always be available at - -http://www.image.ucar.edu/DAReS/DART/cgd_cam.shtml -

-For the - Jamaica release -the async=3 (.../input.nml:filter_nml) option -is no longer offered, and has been replaced by async=4. -This new option runs an MPI filter, which can use either single threaded or MPI CAM. -The single threaded option will run CAM for 1 ensemble member on -each processor (up to the lesser of the number of ensemble members -or the number of processors). -The MPI CAM option will run CAM for each ensemble member in succession, -using all the available processors. -It's not possible (yet) to run several MPI CAMs at the same time, -each using a subset of the processors. -

-This new option allows users to control the assimilation through a single script, -'job_mpi.csh', -except for modifications for machines on which DART-CAM hasn't been tested yet. -Job_mpi.csh has a section of user set parameters that define many aspects of -the assimilation. -It uses those parameters to create a series of batch scripts, -one for each obs_seq.out file that will be assimilated. -Their names have the form Exper_obsseq#.script, -the parts of which are defined in job_mpi.csh. -It submits all of those scripts to the batch scheduler, -where they run in succession as resources become available. -Each obs_seq batch script executes the programs layed out in the calling tree (below). -

-The async=2 option (non-MPI filter and non-MPI CAM) is still available. -

-These options have been tested for DART-CAM in batch submission environments -PBS and LSF on Linux clusters and IBM AIX. -

-There are, no doubt, things missing from these lists, so don't struggle too long -before contacting raeder'at'ucar.edu. -

-The sequence of operations for assimilating a single obs_seq.out file follows. -The functionality of each operation has been restricted to one "domain". -A script/program is specific to: -a machine where the experiment is run; -a model version used in the assimilation; -the filter version; -or the experiment being conducted using the choices for the previous 3. -

-Go to cam/model_mod page -

- - -
-

CALLING TREE

-

-The calling tree for the scripts and fortran executables when running under -async=4 is: -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
SCRIPT DOMAIN LOCATION OR NOTES
Exper_obsseq#.scriptexperimentexperiment central directory where I/O and execution are organized.
   -> mpirun filter executable
filter version local disc on a compute node/processor, or a work directory - in the central directory.
      -> advance_model.csh
 model pre-existing work subdirectory of the central directory.
         -> trans_time executable
 model Translates DART time format into CAM time format
         -> trans_sv_pv executable
 model Inserts DART state vector into CAM initial file
         -> run_pc.csh
 model Modified form of run script from CAM, now in DART 
            -> bld-namelist
model uses namelistin and results of trans_time to make CAM namelist 
            -> advance of CAM
 model Single threaded or MPI CAM 
         -> trans_pv_sv executable
 Translates CAM fields in to DART state vector
   -> qsub auto_re2ms.csh
 machine Central/Experiment/obs_seq_output_directory(i.e. 01_01)
- - - - - - - - -
-

EXPERIMENT SET-UP

-

-Instructions for setting up a DART-CAM assimilation. -

- -
    -
  1. DART -
      -
    1. In .../DART/mkmf link mkmf.template to - the mkmf.template.xxxx which is appropriate for your computer - (or make one of your own).
      2. -
    2. Since you have this file you've already checked out DART.
      - Edit ...DART/models/cam/work/input.nml:preprocess_nml:input_files - to choose obs_def source code files to load via the preprocessor. - The default file (obs_def_reanlysis_bufr_mod.f90) will handle - observations from NCEP reanalysis BUFR files. (Warning; assimilating the - obs in the example GPS obs_seq.out file available from the - DART large file site - requires loading more than just the obs_def_gsp.mod.f90.)
      3. -
    3. Script DART/models/cam/work/workshop_setup.csh is recommended for compiling - the package. - It is set up to compile and run the preprocessor, - compile filter and wakeup_filter as MPI processes (async=4), - and compile all other executables as single threaded. - If you want to use async=2, remove the -mpi flag from the calls - to mkmf_filter and mkmf_wakeup_filter.
      4. -
    2. - -
  2. CAM -
      -
    1. Put the DART modifications - (.../DART/models/cam/Cam[version#]_DART_mods/*), - and any other CAM modifications you have, - in the CAM directory of user-provided modifications. - CAM's "configure" will use these during compilation. - Configure and compile CAM at the resolution/dynamical core desired. - Do this in a directory where all your CAM versions will reside, - here called CamCentral.
      2. - -
    2. Put the cam executable and config_cache.xml in a subdirectory of - the standard cam executable directory (CamCentral/CAM_version/models/atm/cam/bld), - which I'll call CAM_config_1-mpi (leave off the -mpi for single threaded CAM). - Job_mpi.csh has a variable "CAMsrc" that should point to that location.
      3. -
    3. Build a CAM namelist and call it 'namelistin' containing (among the other/default - variables defined by the CAM build-namelist): - 
      -          &camexp
-          ncdata         = 'caminput.nc'
-          caseid         = 'whatever_you_want'
-          nsrest         = 0
-          calendar       = 'GREGORIAN'
-          inithist       = 'ENDOFRUN'
-          /
-          &clmexp
-          finidat        = 'clminput.nc'
-          /
-
      - and NOT containing ... - 
      -          >  nhtfrq         = 4368
-          >  start_ymd      = 20020901
-          >  start_tod      = 0
-          >  stop_ymd       = 20021201
-          >  stop_tod       = 0
-
      - The CAM build-namelist script will use this to make a new namelist with the correct - forecast parameters, named 'namelist'. - Put this in CamCentral/CAM_version/models/atm/cam/bld/CAM_config_1-mpi.
      4. - -
    4. Confirm that you have access to all the CAM input files listed in namelistin, - or suitable replacements.
      5. - -
    5. Put an appropriate CAM initial file in CAM_config_1-mpi and call it caminput.nc. - Put a matching CLM initial file there and call it clminput.nc. - Only the grid information is used from these files, so their dates don't matter. -
      6. -
    3. - -
  3. Set up an experiment central directory ("Central" here) where there's - enough space for output.
    4. - -
  4. Copy the DART namelist file (DART/models/cam/work/input.nml) - into one called "input_1.nml" in Central
    5. - -
  5. Copy DART/models/cam/shell_scripts/job_mpi.csh to Central
    6. - -
  6. EXPERIMENT -
      -
    1. Edit job_mpi.csh. It has more detailed instructions about how to: -
      • define experiment output directory names
        • -
      • tell it whether CAM is MPI
        • -
      • provide directory name of CAM executable and associated files
        • -
      • define which obs_seq.out files to use. - Some pre-made examples can be found - here.
        • -
      • find and link to the obs_seq.out files
        • -
      • find and link to filter_ic[.#] files. - Such an ensemble can be created from CAM initial files using - - .../DART/models/cam/trans_pv_sv_time0.f90.
        • -
      • define which CAM and CLM initial files to use. - Some initial and filter_ic files are available from - DART large file site
        • -
      • define which DART version to use
        • -
      • define resources to be requested by each of the obs_seq_#.script scripts
        • -
      -
      2. - -
    2. Edit input_1.nml to configure the assimilation of the first obs_seq.out. - Be sure that -
        -
      • filenames listed in it agree with what's required by job_mpi.csh - and what is or will be available in the Central directory.
        • -
      • start from restart is .true. if you have an initial ensemble, - .false. otherwise.
        • -
      • init_time_days is the first Gregorian day of your assimilation. - This can be obtained from the program - - ...DART/models/cam/trans_date_to_dart.f90
        • -
      • init_time_seconds is the first second of your assimilation (usually 0).
        • -
      -
      3. - -
    3. Copy input_1.nml to input_n.nml and edit input_n.nml. - Set start_from_restart to .true. - Change init_time_days = -1, init_time_seconds = -1.
      4. -
    4. If you need to make up synthetic observations get create_obs_sequence, - create_fixed_network_seq and perfect_model_obs. - Otherwise, use the obs_seq.out files provided - here
      5. -
    5. Run the experiment by executing job_mpi.csh, - either through the batch queue or interactively.
      6. -
    7. -
- - - - -
-

FILE CONTENTS

-

-See the Jamaica release notes -for diagrams and flowcharts showing DART under the various async options. - -The contents of some of the files which appear there are listed here. -# refers to the list of ensemble members. -

- - - - - -
FILE CONTENTS or PURPOSE
assim_model_state_ic#the state vectors to be used as initial conditions - for the next model advance. Contains the state vector time, the target @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Thu Apr 14 16:24:50 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 14 Apr 2011 16:24:50 -0600 Subject: [Dart-dev] [4856] DART/trunk/models/cam: Moved the trans_*html codes to be 'next to' the Message-ID: Revision: 4856 Author: thoar Date: 2011-04-14 16:24:49 -0600 (Thu, 14 Apr 2011) Log Message: ----------- Moved the trans_*html codes to be 'next to' the corresponding/matching Fortran codes. Changed the references that I could find so as not to break the links. Added the snappy new DART documentation header and moved the namelist to be 'first'. Modified Paths: -------------- DART/trunk/models/cam/doc/cam_guidelines.html DART/trunk/models/cam/model_mod.html Added Paths: ----------- DART/trunk/models/cam/trans_date_to_dart.html DART/trunk/models/cam/trans_pv_sv_time0.html Removed Paths: ------------- DART/trunk/models/cam/doc/trans_date_to_dart_instr.html DART/trunk/models/cam/doc/trans_pv_sv_time0_instr.html -------------- next part -------------- Modified: DART/trunk/models/cam/doc/cam_guidelines.html =================================================================== --- DART/trunk/models/cam/doc/cam_guidelines.html 2011-04-14 22:04:06 UTC (rev 4855) +++ DART/trunk/models/cam/doc/cam_guidelines.html 2011-04-14 22:24:49 UTC (rev 4856) @@ -262,7 +262,7 @@
  • find and link to the obs_seq.out files
  • find and link to filter_ic[.#] files. Such an ensemble can be created from CAM initial files using - + .../DART/models/cam/trans_pv_sv_time0.f90.
  • define which CAM and CLM initial files to use. Some initial and filter_ic files are available from @@ -281,7 +281,7 @@ .false. otherwise.
  • init_time_days is the first Gregorian day of your assimilation. This can be obtained from the program - + ...DART/models/cam/trans_date_to_dart.f90
  • init_time_seconds is the first second of your assimilation (usually 0).
    • Deleted: DART/trunk/models/cam/doc/trans_date_to_dart_instr.html =================================================================== --- DART/trunk/models/cam/doc/trans_date_to_dart_instr.html 2011-04-14 22:04:06 UTC (rev 4855) +++ DART/trunk/models/cam/doc/trans_date_to_dart_instr.html 2011-04-14 22:24:49 UTC (rev 4856) @@ -1,60 +0,0 @@ - - - -trans_date_to_dart - - - - - -
    -TERMS OF USE -
    - -

    Program trans_date_to_dart

    -

    $Id$

    - -

    -trans_date_to_dart translates a standard date and time into Gregorian and DART date/time pairs. -It will be compiled automatically by .../DART/models/cam/work/workshop_setup.csh. -It opens a pre-existing file called 'date_greg' which must be in the run directory -and which must contain a date and time on the first line, in the form -

    -
    -yyyy/mm/dd hh:mm:ss
-
    -

    -It will write out the gregorian day and time onto line 2 of the same file. -It will also write out the date in the format the CAM wants to see onto the 3rd line. -It does not require any namelist input from input.nml. -

    - - - - - - -
    -

    Terms of Use

    - -

    -DART software - Copyright © 2004 - 2010 UCAR.
    -This open source software is provided by UCAR, "as is",
    -without charge, subject to all terms of use at
    - -http://www.image.ucar.edu/DAReS/DART/DART_download -

    - - - - - - - -
    Contact: Kevin Raeder
    Revision: $Revision$
    Source: $URL$
    Change Date: $Date$
    Change history:  try "svn log" or "svn diff"
    - - - - - Deleted: DART/trunk/models/cam/doc/trans_pv_sv_time0_instr.html =================================================================== --- DART/trunk/models/cam/doc/trans_pv_sv_time0_instr.html 2011-04-14 22:04:06 UTC (rev 4855) +++ DART/trunk/models/cam/doc/trans_pv_sv_time0_instr.html 2011-04-14 22:24:49 UTC (rev 4856) @@ -1,153 +0,0 @@ - - - -trans_pv_sv_time0 - - - - - -
    -NAMELIST / -FILES / -REFERENCES / -ERRORS / -PLANS / -TERMS OF USE -
    - -

    Program trans_pv_sv_time0

    -

    $Id$

    - -

    -You can make filter_ic files from caminput_#.nc files using the program -.../DART/models/cam/trans_pv_sv_time0.f90. -This is compiled in .../DART/models/cam/work by default if you use workshop_setup.csh. -trans_pv_sv_time0 needs to be run for each caminput file. -So far there is not a robust script for running trans_pv_sv_time0, -but you can write your own. -

    -You'll need to have several files in the directory where trans_pv_sv_time0 is run: -

    - -
      -
    • input.nml containing perfect_model_nml, which provides the Gregorian - timestamp for the filter_ic file in variables - 
      -           init_time_days    = 146827,
-           init_time_seconds = 0,
-
      - This date can be generated using - - .../DART/models/cam/trans_date_to_dart.f90. -
      • -
    • 'topog_file.nc' for your resolution. - A copy of this should be in with the rest of the CAM - input files that are listed in the CAM "namelistin" in variable bnd_topo. - It will be named similarly to - .../something/else/more/topo-from-cami_0000-09-01_64x128_L26_c030918.nc -
      • -
    • link or copy your CAM initial file caminput_#.nc to 'caminput.nc', - which is the name trans_pv_sv_time0 wants to see. -
      • -
    - -

    -Each run of trans_pv_sv_time0 will create a 'temp_ud' file, -which should be renamed filter_ic.####, -where #### = 1,...,num_ens (padded with 0s) -is taken from the caminput_#.nc used as input. -You might want to put these new filter_ic files in a directory that has MPI -in the name, so that you know whether they work with Jamaica (new state vector order) -or Iceland (old). -

    - - - - - - -
    -

    NAMELIST

    -

    -There is no namelist unique to this program. -

    - - - - - - -
    -

    FILES

    -

    -still working on this part -

    - - - - - - -
    -

    REFERENCES

    -
      -
    • none
      • -
    - - - - - - -
    -

    ERROR CODES and CONDITIONS

    -

    -still working on this part -

    - -

    KNOWN BUGS

    -

    -none at this time -

    - - - - - - -
    -

    FUTURE PLANS

    -

    -none at this time -

    - - - - - - -
    -

    Terms of Use

    - -

    -DART software - Copyright © 2004 - 2010 UCAR.
    -This open source software is provided by UCAR, "as is",
    -without charge, subject to all terms of use at
    - -http://www.image.ucar.edu/DAReS/DART/DART_download -

    - - - - - - - -
    Contact: Kevin Raeder
    Revision: $Revision$
    Source: $URL$
    Change Date: $Date$
    Change history:  try "svn log" or "svn diff"
    - - - - - Modified: DART/trunk/models/cam/model_mod.html =================================================================== --- DART/trunk/models/cam/model_mod.html 2011-04-14 22:04:06 UTC (rev 4855) +++ DART/trunk/models/cam/model_mod.html 2011-04-14 22:24:49 UTC (rev 4856) @@ -3,24 +3,37 @@ module model_mod (CAM) - + -
    +

    MODULE model_mod (CAM)

    + + + + + + +
    + DART project logo + +

    Jump to DART Documentation Main Index
    + version information for this file:
    + + $Id$     +

    + +NAMELIST / INTERFACES / -NAMELIST / FILES / REFERENCES / ERRORS / PLANS / PRIVATE COMPONENTS / TERMS OF USE -
    -

    MODULE model_mod (CAM)

    -

    $Id$

    +

    Overview

    The 17 public interfaces in model_mod @@ -88,7 +101,161 @@ + + + +

    +

    NAMELIST

    +

    We adhere to the F90 standard of starting a namelist with an ampersand +'&' and terminating with a slash '/' for all our namelist input. +The declarations have a different syntax, naturally. +

    +
    +
    +namelist / model_nml /  &
+     output_state_vector,  model_version,  model_config_file,          &  
+       state_num_0d,   state_num_1d,   state_num_2d,   state_num_3d,   &  
+     state_names_0d, state_names_1d, state_names_2d, state_names_3d,   &
+                      which_vert_1d,  which_vert_2d,  which_vert_3d,   &
+     pert_names, pert_sd, pert_base_vals,                              &
+     highest_obs_pressure_mb, highest_state_pressure_mb,               &
+     max_obs_lat_degree, Time_step_seconds, Time_step_days
+
    +
    + +
    + + +

    +This namelist is read in a file called input.nml +The specification of lists of names and numbers for the various dimensions +enables the very flexible definition of the state vector. It can be +done via the namelist, instead of recompiling DART for each different set. +One hurdle that still remains is that distinct filter_ics are +necessary for each separate set of fields which compose the state vector. +

    + +

    +The dimension of these lists is currently hardwired to size 100. +If more variables types need to be assimilated (e.g. many chemical species), +look for the integer parameter MAX_STATE_NAMES in the source code +and change it to a long enough value and recompile DART. +Longer term we intend to investigate using 2 different nameslists +inside model_mod; one for setting the length of the lists and another +to actually read in the data which fills the lists. +

    + +

    +The values for which_vert_#d is described in +DART/location/threed_sphere/location_mod.html. +

    + +

    +The names of the fields to put into the state vector come from the CAM initial +file field names. +

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Contents Type Description
    output_state_vector logical Controls the output to netCDF files. + If .true., output the raw dart state vector. If .false., + output the prognostic flavor (gridded data) for easier + plotting (recommended). Defaults to .false.
    model_version character(len=128) The number of the CAM version being + used, i.e. '3.0.7'. (no letters allowed, so rename cam3_0_p1) + Default = '3.0' .
    model_config_file character(len=128) CAM initial file used to provide + configuration information. Default 'caminput.nc'.
    state_num_#d, #=0,1,2,3 integer Numbers of fields of various dimensions to put + into the state vector. + Defaults state_num_0d=0, _1d=0, _2d=1, _3d=4 .
    state_names_#d, #=0,1,2,3 character(len=8), dimension(100) Names of fields of various dimensions to put + into the state vector. Defaults +
    state_names_0d='        ', state_names_1d='        ',
+state_names_2d='PS      '
+state_names_3d='T       ','U       ','V       ','Q       '
    +
    which_vert_#d, #=1,2,3 integer, dimension(100) Vertical location types of fields in + state_names_#d. Defaults +
    • which_vert_1d = state_num_1d*-2, +
    • which_vert_2d = state_num_2d*-1, +
    • which_vert_3d = state_num_3d* 1
    pert_names character(len=8), dimension(100) List of field(s) whose values need to be changed at or before + the beginning of an assimilation. To make filter generate + an ensemble from a single model state by randomly perturbing it, + list the fields to be perturbed here. To make trans_pv_sv_pert0 + reset a single field to a constant value, list that field here. + Trans_pv_sv_pert0 would be run as many times as there are ensemble + members (see pert_base_vals), in order to provide spread to that + variable. + Defaults
    + 
    pert_names='        '
    pert_sd real(r8), dimension(100) If positive, it's the standard deviation of the + perturbation for each field in the pert_names list (filter). + If negative, then pert_names can contain only one entry, and that field + will be set to a different constant value for each ensemble member + (trans_pv_sv_pert0). Those values come from pert_base_vals. + Defaults to a MISSING real value and + unused unless pert_names is set.
    pert_base_vals real(r8), dimension(100) If pert_sd (yes, 'sd') is negative, this is the list of + values to use for each ensemble member when perturbing the single + field named in pert_names. + Otherwise, it's the list of values to which the fieldS listed in + pert_names will be reset if filter is told to create an ensemble from + a single state vector. + Defaults to a MISSING real value and unused unless pert_names + is set and pert_base_vals is /= this missing value (-888888.0d0).
    max_obs_lat_degree real(r8) Observations closer to the poles than this + latitude will be ignored. + Default 90 degrees
    highest_obs_pressure_mb real(r8) Observations higher than this pressure are ignored. + Default 150.0 hPa
    highest_state_pressure_mb real(r8) Influence of all obs on model points higher than + this is reduced. + Default 150.0 hPa
    Time_step_seconds real(r8) Forecast duration (the part < 1 day) + Default 21600 sec
    Time_step_days real(r8) Forecast duration (the part > 24*3600 sec) + Default 0 days
    + +
    +
    + + + +

    OTHER MODULES USED

    @@ -608,10 +775,10 @@

    -This subroutine is also used by trans_pv_sv_pert0 for the off-line +This subroutine is also used by trans_pv_sv_pert0 for the off-line perturbation of fields the model considers to be parameters. In this use a new random sequence seed must be provided via an input file ("ens_member") -for each ensemble member, since trans_pv_sv_pert0 is executed once for each +for each ensemble member, since trans_pv_sv_pert0 is executed once for each member and can't keep a series of different seeds intact.

    @@ -743,157 +910,6 @@
    - - - -
    -

    NAMELIST

    -

    We adhere to the F90 standard of starting a namelist with an ampersand -'&' and terminating with a slash '/' for all our namelist input. -The declarations have a different syntax, naturally. -

    -
    -
    -namelist / model_nml /  &
-     output_state_vector,  model_version,  model_config_file,          &  
-       state_num_0d,   state_num_1d,   state_num_2d,   state_num_3d,   &  
-     state_names_0d, state_names_1d, state_names_2d, state_names_3d,   &
-                      which_vert_1d,  which_vert_2d,  which_vert_3d,   &
-     pert_names, pert_sd, pert_base_vals,                              &
-     highest_obs_pressure_mb, highest_state_pressure_mb,               &
-     max_obs_lat_degree, Time_step_seconds, Time_step_days
-
    -
    - -
    - - -

    -This namelist is read in a file called input.nml -The specification of lists of names and numbers for the various dimensions -enables the very flexible definition of the state vector. It can be -done via the namelist, instead of recompiling DART for each different set. -One hurdle that still remains is that distinct filter_ics are -necessary for each separate set of fields which compose the state vector. -

    - -

    -The dimension of these lists is currently hardwired to size 100. -If more variables types need to be assimilated (e.g. many chemical species), -look for the integer parameter MAX_STATE_NAMES in the source code -and change it to a long enough value and recompile DART. -Longer term we intend to investigate using 2 different nameslists -inside model_mod; one for setting the length of the lists and another -to actually read in the data which fills the lists. -

    - -

    -The values for which_vert_#d is described in -DART/location/threed_sphere/location_mod.html. -

    - -

    -The names of the fields to put into the state vector come from the CAM initial -file field names. -

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    Contents Type Description
    output_state_vector logical Controls the output to netCDF files. - If .true., output the raw dart state vector. If .false., - output the prognostic flavor (gridded data) for easier - plotting (recommended). Defaults to .false.
    model_version character(len=128) The number of the CAM version being - used, i.e. '3.0.7'. (no letters allowed, so rename cam3_0_p1) - Default = '3.0' .
    model_config_file character(len=128) CAM initial file used to provide - configuration information. Default 'caminput.nc'.
    state_num_#d, #=0,1,2,3 integer Numbers of fields of various dimensions to put - into the state vector. - Defaults state_num_0d=0, _1d=0, _2d=1, _3d=4 .
    state_names_#d, #=0,1,2,3 character(len=8), dimension(100) Names of fields of various dimensions to put - into the state vector. Defaults -
    state_names_0d='        ', state_names_1d='        ',
-state_names_2d='PS      '
-state_names_3d='T       ','U       ','V       ','Q       '
    -
    which_vert_#d, #=1,2,3 integer, dimension(100) Vertical location types of fields in - state_names_#d. Defaults -
    • which_vert_1d = state_num_1d*-2, -
    • which_vert_2d = state_num_2d*-1, -
    • which_vert_3d = state_num_3d* 1
    pert_names character(len=8), dimension(100) List of field(s) whose values need to be changed at or before - the beginning of an assimilation. To make filter generate - an ensemble from a single model state by randomly perturbing it, - list the fields to be perturbed here. To make trans_pv_sv_pert0 - reset a single field to a constant value, list that field here. - Trans_pv_sv_pert0 would be run as many times as there are ensemble - members (see pert_base_vals), in order to provide spread to that - variable. - Defaults
    - 
    pert_names='        '
    pert_sd real(r8), dimension(100) If positive, it's the standard deviation of the - perturbation for each field in the pert_names list (filter). - If negative, then pert_names can contain only one entry, and that field - will be set to a different constant value for each ensemble member - (trans_pv_sv_pert0). Those values come from pert_base_vals. - Defaults to a MISSING real value and - unused unless pert_names is set.
    pert_base_vals real(r8), dimension(100) If pert_sd (yes, 'sd') is negative, this is the list of - values to use for each ensemble member when perturbing the single - field named in pert_names. - Otherwise, it's the list of values to which the fieldS listed in - pert_names will be reset if filter is told to create an ensemble from - a single state vector. - Defaults to a MISSING real value and unused unless pert_names - is set and pert_base_vals is /= this missing value (-888888.0d0).
    max_obs_lat_degree real(r8) Observations closer to the poles than this - latitude will be ignored. - Default 90 degrees
    highest_obs_pressure_mb real(r8) Observations higher than this pressure are ignored. - Default 150.0 hPa
    highest_state_pressure_mb real(r8) Influence of all obs on model points higher than - this is reduced. - Default 150.0 hPa
    Time_step_seconds real(r8) Forecast duration (the part < 1 day) - Default 21600 sec
    Time_step_days real(r8) Forecast duration (the part > 24*3600 sec) - Default 0 days
    - -
    -
    - Copied: DART/trunk/models/cam/trans_date_to_dart.html (from rev 4851, DART/trunk/models/cam/doc/trans_date_to_dart_instr.html) =================================================================== --- DART/trunk/models/cam/trans_date_to_dart.html (rev 0) +++ DART/trunk/models/cam/trans_date_to_dart.html 2011-04-14 22:24:49 UTC (rev 4856) @@ -0,0 +1,60 @@ + + + +trans_date_to_dart + + + + + +
    +TERMS OF USE +
    + +

    Program trans_date_to_dart

    +

    $Id$

    + +

    +trans_date_to_dart translates a standard date and time into Gregorian and DART date/time pairs. +It will be compiled automatically by .../DART/models/cam/work/workshop_setup.csh. +It opens a pre-existing file called 'date_greg' which must be in the run directory +and which must contain a date and time on the first line, in the form +

    +
    +yyyy/mm/dd hh:mm:ss
+
    +

    +It will write out the gregorian day and time onto line 2 of the same file. +It will also write out the date in the format the CAM wants to see onto the 3rd line. +It does not require any namelist input from input.nml. +

    + + + + + + +
    +

    Terms of Use

    + +

    +DART software - Copyright © 2004 - 2010 UCAR.
    +This open source software is provided by UCAR, "as is",
    +without charge, subject to all terms of use at
    + +http://www.image.ucar.edu/DAReS/DART/DART_download +

    + + + + + + + +
    Contact: Kevin Raeder
    Revision: $Revision$
    Source: $URL$
    Change Date: $Date$
    Change history:  try "svn log" or "svn diff"
    + + + + + Copied: DART/trunk/models/cam/trans_pv_sv_time0.html (from rev 4851, DART/trunk/models/cam/doc/trans_pv_sv_time0_instr.html) =================================================================== --- DART/trunk/models/cam/trans_pv_sv_time0.html (rev 0) +++ DART/trunk/models/cam/trans_pv_sv_time0.html 2011-04-14 22:24:49 UTC (rev 4856) @@ -0,0 +1,153 @@ + + + +trans_pv_sv_time0 + + + + + +
    +NAMELIST / +FILES / +REFERENCES / +ERRORS / +PLANS / +TERMS OF USE +
    + +

    Program trans_pv_sv_time0

    +

    $Id$

    + +

    +You can make filter_ic files from caminput_#.nc files using the program +.../DART/models/cam/trans_pv_sv_time0.f90. +This is compiled in .../DART/models/cam/work by default if you use workshop_setup.csh. +trans_pv_sv_time0 needs to be run for each caminput file. +So far there is not a robust script for running trans_pv_sv_time0, +but you can write your own. +

    +You'll need to have several files in the directory where trans_pv_sv_time0 is run: +

    + +
      +
    • input.nml containing perfect_model_nml, which provides the Gregorian + timestamp for the filter_ic file in variables + 
      +           init_time_days    = 146827,
+           init_time_seconds = 0,
+
      + This date can be generated using + + .../DART/models/cam/trans_date_to_dart.f90. +
      • +
    • 'topog_file.nc' for your resolution. + A copy of this should be in with the rest of the CAM + input files that are listed in the CAM "namelistin" in variable bnd_topo. + It will be named similarly to + .../something/else/more/topo-from-cami_0000-09-01_64x128_L26_c030918.nc +
      • +
    • link or copy your CAM initial file caminput_#.nc to 'caminput.nc', + which is the name trans_pv_sv_time0 wants to see. +
      • +
    + +

    +Each run of trans_pv_sv_time0 will create a 'temp_ud' file, +which should be renamed filter_ic.####, +where #### = 1,...,num_ens (padded with 0s) +is taken from the caminput_#.nc used as input. +You might want to put these new filter_ic files in a directory that has MPI +in the name, so that you know whether they work with Jamaica (new state vector order) +or Iceland (old). +

    + + + + + + +
    +

    NAMELIST

    +

    +There is no namelist unique to this program. +

    + + + + + + +
    +

    FILES

    +

    +still working on this part +

    + + + + + + +
    +

    REFERENCES

    +
      +
    • none
      • +
    + + + + + + +
    +

    ERROR CODES and CONDITIONS

    +

    +still working on this part +

    + +

    KNOWN BUGS

    +

    +none at this time +

    + + + + + + +
    +

    FUTURE PLANS

    +

    +none at this time +

    + + + + + + +
    +

    Terms of Use

    + +

    +DART software - Copyright © 2004 - 2010 UCAR.
    +This open source software is provided by UCAR, "as is",
    +without charge, subject to all terms of use at
    + +http://www.image.ucar.edu/DAReS/DART/DART_download +

    + + + + + + + +
    Contact: Kevin Raeder
    Revision: $Revision$
    Source: $URL$
    Change Date: $Date$
    Change history:  try "svn log" or "svn diff"
    + + + + + From nancy at ucar.edu Thu Apr 14 16:52:50 2011 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 14 Apr 2011 16:52:50 -0600 Subject: [Dart-dev] [4857] DART/trunk/models/cam: using the snappy new DART documentation intro Message-ID: Revision: 4857 Author: thoar Date: 2011-04-14 16:52:50 -0600 (Thu, 14 Apr 2011) Log Message: ----------- using the snappy new DART documentation intro Modified Paths: -------------- DART/trunk/models/cam/trans_date_to_dart.html DART/trunk/models/cam/trans_pv_sv_time0.html -------------- next part -------------- Modified: DART/trunk/models/cam/trans_date_to_dart.html =================================================================== --- DART/trunk/models/cam/trans_date_to_dart.html 2011-04-14 22:24:49 UTC (rev 4856) +++ DART/trunk/models/cam/trans_date_to_dart.html 2011-04-14 22:52:50 UTC (rev 4857) @@ -2,24 +2,40 @@ "http://www.w3.org/TR/html4/strict.dtd"> -trans_date_to_dart - +program trans_date_to_dart + -
    +

    PROGRAM trans_date_to_dart

    + + + + + + +
    + DART project logo + +

    Jump to DART Documentation Main Index
    + version information for this file:
    + + $Id$     +

    + TERMS OF USE -
    -

    Program trans_date_to_dart

    -

    $Id$

    +

    Overview

    -trans_date_to_dart translates a standard date and time into Gregorian and DART date/time pairs. -It will be compiled automatically by .../DART/models/cam/work/workshop_setup.csh. -It opens a pre-existing file called 'date_greg' which must be in the run directory -and which must contain a date and time on the first line, in the form +trans_date_to_dart translates a standard date and time into +Gregorian and DART date/time pairs. It will be compiled automatically by +.../DART/models/cam/work/quickbuild.csh or it can be built +in the usual way. +It opens a pre-existing file called 'date_greg' +which must be in the run directory and which must contain a date and time on the +first line, in the form 

     yyyy/mm/dd hh:mm:ss
@@ -27,7 +43,7 @@
 
    
 It will write out the gregorian day and time onto line 2 of the same file.
 It will also write out the date in the format the CAM wants to see onto the 3rd line.
-It does not require any namelist input from input.nml.
+It does not require any namelist input from input.nml .
 
    
 
 

Modified: DART/trunk/models/cam/trans_pv_sv_time0.html
===================================================================
--- DART/trunk/models/cam/trans_pv_sv_time0.html	2011-04-14 22:24:49 UTC (rev 4856)
+++ DART/trunk/models/cam/trans_pv_sv_time0.html	2011-04-14 22:52:50 UTC (rev 4857)
@@ -1,66 +1,85 @@
-
+
 
 
-trans_pv_sv_time0 
-
+program trans_pv_sv_time0
+
 
 
 
 
-
    
+
    PROGRAM trans_pv_sv_time0
    
+
+
+
+    
+    
+
+
    
+    DART project logo
+    
+       
    Jump to DART Documentation Main Index
    
+          version information for this file: 
    
+          
+          $Id$    
+       
    
+
 NAMELIST /
 FILES /
 REFERENCES /
 ERRORS /
 PLANS /
 TERMS OF USE
-
    
 
+
    Overview
    
+
 
    Program trans_pv_sv_time0
    
-
    $Id$
    
 
 
    
-You can make filter_ic files from caminput_#.nc files using the program
-.../DART/models/cam/trans_pv_sv_time0.f90.
-This is compiled in .../DART/models/cam/work by default if you use workshop_setup.csh.
-trans_pv_sv_time0 needs to be run for each caminput file. 
-So far there is not a robust script for running trans_pv_sv_time0, 
+You can make filter_ic files from caminput_#.nc 
+files using the program .../DART/models/cam/trans_pv_sv_time0.f90.
+This is compiled in .../DART/models/cam/work by default if you use 
+quickbuild.csh or it can be compiled standalone in the usual way.
+trans_pv_sv_time0 needs to be run for each caminput file. 
+So far there is not a robust script for running trans_pv_sv_time0, 
 but you can write your own. 
 

    
-You'll need to have several files in the directory where trans_pv_sv_time0 is run:
+You'll need to have several files in the directory where trans_pv_sv_time0 is run:
 
    
 
 
      
-   
    •  input.nml containing perfect_model_nml, which provides the Gregorian 
-        timestamp for the filter_ic file in variables
-        
      +   
    • input.nml containing perfect_model_nml, 
+        which provides the Gregorian timestamp for filter_ic 
+        in variables
+        
      +        &perfect_model_nml
            init_time_days    = 146827,
-           init_time_seconds = 0,
-        
      
+           init_time_seconds = 0,
      • 
+        
+        
      
         This date can be generated using 
-	 
-	.../DART/models/cam/trans_date_to_dart.f90.  
-   
      • 
-   
    •  'topog_file.nc' for your resolution. 
+	.../DART/models/cam/trans_date_to_dart.f90.  
+   

      • 
+   
    •  'topog_file.nc' for your resolution. 
         A copy of this should be in with the rest of the CAM
 	input files that are listed in the CAM "namelistin" in variable bnd_topo.  
 	It will be named similarly to 
-        .../something/else/more/topo-from-cami_0000-09-01_64x128_L26_c030918.nc 
+        .../something/else/more/topo-from-cami_0000-09-01_64x128_L26_c030918.nc
+   

      • 
+   
    • link or copy your CAM initial file caminput_#.nc 
+       to 'caminput.nc', which is the name 
+       trans_pv_sv_time0 requires.
    
      • 
-   
    •  link or copy your CAM initial file caminput_#.nc to 'caminput.nc',
-        which is the name trans_pv_sv_time0 wants to see.
-   
      • 
 
    
 
 
    
 Each run of trans_pv_sv_time0 will create a 'temp_ud' file, 
-which should be renamed filter_ic.####, 
-where #### = 1,...,num_ens (padded with 0s)
+which should be renamed filter_ic.####, 
+where #### = 1,...,num_ens (padded with 0s)
 is taken from the caminput_#.nc used as input.
-You might want to put these new filter_ic files in a directory that has MPI
+.
 
    
 
 

From nancy at ucar.edu  Fri Apr 15 11:03:41 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Fri, 15 Apr 2011 11:03:41 -0600
Subject: [Dart-dev] [4858] DART/trunk: Using a more robust form of the URL
 for the doi: citations.
Message-ID: 

Revision: 4858
Author:   thoar
Date:     2011-04-15 11:03:41 -0600 (Fri, 15 Apr 2011)
Log Message:
-----------
Using a more robust form of the URL for the doi: citations.
The citations can include some characters that do not play well
with certain browsers.

Modified Paths:
--------------
    DART/trunk/adaptive_inflate/adaptive_inflate_mod.html
    DART/trunk/assim_tools/assim_tools_mod.html
    DART/trunk/doc/html/boilerplate.html
    DART/trunk/filter/filter.html
    DART/trunk/filter/wakeup_filter.html

-------------- next part --------------
Modified: DART/trunk/adaptive_inflate/adaptive_inflate_mod.html
===================================================================
--- DART/trunk/adaptive_inflate/adaptive_inflate_mod.html	2011-04-14 22:52:50 UTC (rev 4857)
+++ DART/trunk/adaptive_inflate/adaptive_inflate_mod.html	2011-04-15 17:03:41 UTC (rev 4858)
@@ -723,12 +723,12 @@
 
 
    
 
    REFERENCES
    
-
      
+
        
 
      • Anderson, J. L., 2007:
 An adaptive covariance inflation error correction algorithm for ensemble filters.
 Tellus A,
 59, 210-224.
        
-
 doi: 10.1111/j.1600-0870.2006.00216.x 
        • 
 
        
@@ -740,7 +740,7 @@
 target="_blank" >
 doi: 10.1111/j.1600-0870.2008.00361.x
 
        
-
      
+
    
 
 
 

Modified: DART/trunk/assim_tools/assim_tools_mod.html
===================================================================
--- DART/trunk/assim_tools/assim_tools_mod.html	2011-04-14 22:52:50 UTC (rev 4857)
+++ DART/trunk/assim_tools/assim_tools_mod.html	2011-04-15 17:03:41 UTC (rev 4858)
@@ -583,17 +583,17 @@
 An Ensemble Adjustment Kalman Filter for Data Assimilation.
 Mon. Wea. Rev.,
 129, 2884-2903.
    
-
-doi: 10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2 
+doi: 10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2 
 
    
 
  • Anderson, J. L., 2003:
 A Local Least Squares Framework for Ensemble Filtering.
 Mon. Wea. Rev.,
 131, 634-642.
    
-
-doi: 10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2
    • 
+doi: 10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2
 
    
 
  • Anderson, J., Collins, N., 2007:
 Scalable Implementations of Ensemble Filter Algorithms for Data Assimilation.

Modified: DART/trunk/doc/html/boilerplate.html
===================================================================
--- DART/trunk/doc/html/boilerplate.html	2011-04-14 22:52:50 UTC (rev 4857)
+++ DART/trunk/doc/html/boilerplate.html	2011-04-15 17:03:41 UTC (rev 4858)
@@ -421,7 +421,7 @@
     The Data Assimilation Research Testbed: A Community Facility. 
     Bull. Amer. Meteor. Soc.,
     90, 1283-1296.
    
-    DOI: 10.1175/2009BAMS2618.1
    • 
+    DOI: 10.1175/2009BAMS2618.1
 
 
      
 
    •  none 
      • 

Modified: DART/trunk/filter/filter.html
===================================================================
--- DART/trunk/filter/filter.html	2011-04-14 22:52:50 UTC (rev 4857)
+++ DART/trunk/filter/filter.html	2011-04-15 17:03:41 UTC (rev 4858)
@@ -1064,17 +1064,17 @@
 An Ensemble Adjustment Kalman Filter for Data Assimilation.
 Mon. Wea. Rev.,
 129, 2884-2903.
      
-
-doi: 10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2 
+doi: 10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2 
 
      
 
    • Anderson, J. L., 2003:
 A Local Least Squares Framework for Ensemble Filtering.
 Mon. Wea. Rev.,
 131, 634-642.
      
-
-doi: 10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2
      • 
+doi: 10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2
 
      
 
    • Anderson, J. L., 2007: 
 An adaptive covariance inflation error correction algorithm for ensemble filters.

Modified: DART/trunk/filter/wakeup_filter.html
===================================================================
--- DART/trunk/filter/wakeup_filter.html	2011-04-14 22:52:50 UTC (rev 4857)
+++ DART/trunk/filter/wakeup_filter.html	2011-04-15 17:03:41 UTC (rev 4858)
@@ -99,7 +99,7 @@
     The Data Assimilation Research Testbed: A Community Facility. 
     Bull. Amer. Meteor. Soc.,
     90, 1283-1296.
      
-    DOI: 10.1175/2009BAMS2618.1
      • 
+    DOI: 10.1175/2009BAMS2618.1
 
    
 
 

From nancy at ucar.edu  Fri Apr 15 13:19:53 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Fri, 15 Apr 2011 13:19:53 -0600
Subject: [Dart-dev] [4860] DART/trunk/assim_tools/assim_tools_mod.f90:
 proposed update to assim_tools_mod. differen
Message-ID: 

Revision: 4860
Author:   nancy
Date:     2011-04-15 13:19:53 -0600 (Fri, 15 Apr 2011)
Log Message:
-----------
proposed update to assim_tools_mod.  different method to compute variance & covariance; _r8 on constants, additional limit tests for values

Modified Paths:
--------------
    DART/trunk/assim_tools/assim_tools_mod.f90

-------------- next part --------------
Modified: DART/trunk/assim_tools/assim_tools_mod.f90
===================================================================
--- DART/trunk/assim_tools/assim_tools_mod.f90	2011-04-15 19:16:58 UTC (rev 4859)
+++ DART/trunk/assim_tools/assim_tools_mod.f90	2011-04-15 19:19:53 UTC (rev 4860)
@@ -236,7 +236,7 @@
       write(errstring, '(A,I10,A)') 'Using adaptive localization, threshold ', &
          adaptive_localization_threshold, ' obs'
       call error_handler(E_MSG,'assim_tools_init:', errstring)
-      if(adaptive_cutoff_floor > 0) then
+      if(adaptive_cutoff_floor > 0.0_r8) then
          write(errstring, '(A,F18.6)') 'Minimum cutoff will not go below ', &
             adaptive_cutoff_floor
          call error_handler(E_MSG,'assim_tools_init:', 'Using adaptive localization cutoff floor.', &
@@ -248,6 +248,10 @@
       call error_handler(E_MSG,'assim_tools_init:', 'Writing localization diagnostics to file:')
       call error_handler(E_MSG,'assim_tools_init:', trim(localization_diagnostics_file))
    endif
+
+   if(sampling_error_correction) then
+      call error_handler(E_MSG,'assim_tools_init:', 'Using Sampling Error Correction')
+   endif
 endif
 
 end subroutine assim_tools_init
@@ -580,8 +584,9 @@
       grp_bot = grp_beg(group)
       grp_top = grp_end(group)
       obs_prior_mean(group) = sum(obs_prior(grp_bot:grp_top)) / grp_size
-      obs_prior_var(group)  = sum(obs_prior(grp_bot:grp_top) * obs_prior(grp_bot:grp_top)) - &
-         grp_size * obs_prior_mean(group)**2
+      obs_prior_var(group) = sum((obs_prior(grp_bot:grp_top) - obs_prior_mean(group))**2) / &
+         (grp_size - 1)
+      if (obs_prior_var(group) < 0.0_r8) obs_prior_var(group) = 0.0_r8
    end do
 
    ! Need to get obs density first in case of adaptive localization
@@ -738,7 +743,7 @@
                obs_prior_var(group), obs_inc(grp_bot:grp_top), &
                ens_handle%copies(grp_bot:grp_top, state_index), grp_size, &
                increment(grp_bot:grp_top), reg_coef(group), net_a(group), correl(group))
-            else
+         else
             call update_from_obs_inc(obs_prior(grp_bot:grp_top), obs_prior_mean(group), &
                obs_prior_var(group), obs_inc(grp_bot:grp_top), &
                ens_handle%copies(grp_bot:grp_top, state_index), grp_size, &
@@ -888,7 +893,7 @@
                 num_close_obs_buffered + num_close_states_buffered
       if (num_close_obs_buffered+num_close_obs_calls_made+ &
           num_close_states_buffered+num_close_states_calls_made > 0) then 
-         print *, "Percent saved: ", 100. * &
+         print *, "Percent saved: ", 100.0_r8 * &
                    (real(num_close_obs_buffered+num_close_states_buffered, r8) /  &
                    (num_close_obs_calls_made+num_close_obs_buffered+ &
                     num_close_states_calls_made+num_close_states_buffered))
@@ -1422,15 +1427,15 @@
 real(r8),           intent(inout) :: net_a
 real(r8), optional, intent(inout) :: correl_out
 
-real(r8) :: t(ens_size), obs_state_cov
+real(r8) :: obs_state_cov
 real(r8) :: restoration_inc(ens_size), state_mean, state_var, correl
 real(r8) :: factor, exp_true_correl, mean_factor
 
 ! For efficiency, just compute regression coefficient here unless correl is needed
-t = obs - obs_prior_mean
-obs_state_cov = sum(t * state)
+state_mean = sum(state) / ens_size
+obs_state_cov = sum( (state - state_mean) * (obs - obs_prior_mean) ) / (ens_size - 1)
 
-if (obs_prior_var /= 0.0_r8) then
+if (obs_prior_var > 0.0_r8) then
    reg_coef = obs_state_cov/obs_prior_var
 else
    reg_coef = 0.0_r8
@@ -1439,8 +1444,9 @@
 ! If correl_out is present, need correl for adaptive inflation
 ! Also needed for file correction below
 if(present(correl_out) .or. sampling_error_correction) then
-   state_var = sum(state * state) - sum(state)**2 / ens_size
-   if(obs_prior_var * state_var <= 0.0_r8) then
+   state_var = sum((state - state_mean)**2) / (ens_size - 1)
+   if (state_var < 0.0_r8) state_var = 0.0_r8
+   if ((obs_prior_var <= 0.0_r8) .or. (state_var <= 0.0_r8)) then
       correl = 0.0_r8
    else
       correl = obs_state_cov / sqrt(obs_prior_var * state_var)
@@ -1455,7 +1461,6 @@
 ! Get the expected actual correlation and the regression weight reduction factor
 if(sampling_error_correction) then
    call get_correction_from_file(ens_size, correl, mean_factor, exp_true_correl)
-   !write(*, *) correl, exp_true_correl, mean_factor
    ! Watch out for division by zero; if correl is really small regression is safely 0
    if(abs(correl) > 0.001) then
       reg_coef = reg_coef * (exp_true_correl / correl) * mean_factor
@@ -1488,7 +1493,7 @@
    ! the reciprocal of the ensemble_size (slope = 0.80 / ens_size). These are empirical
    ! for now. See also README in spread_restoration_paper documentation.
    !!!factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) * (0.8_r8 / ens_size)) - 1.0_r8
-   factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) / (-2.4711 + 1.6386 * ens_size)) - 1.0_r8
+   factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) / (-2.4711_r8 + 1.6386_r8 * ens_size)) - 1.0_r8
    !!!factor = 1.0_r8 / (1.0_r8 + (net_a**2 - 1.0_r8) * (-0.0111_r8 + .8585_r8 / ens_size)) - 1.0_r8
 
    ! Variance restoration
@@ -1497,6 +1502,15 @@
    state_inc = state_inc + restoration_inc
 endif
 
+!! NOTE:  if requested to be returned, correl_out is set further up in the
+!! code, before the sampling error correction, if enabled, is applied.
+!! this means it's returning a different larger value than the correl 
+!! being returned here.  it's used by the adaptive inflation and so the
+!! inflation will see a slightly different correlation value.  it isn't
+!! clear that this is a bad thing; it means the inflation might be a bit
+!! larger than it would otherwise.  before we move any code this would
+!! need to be studied to see what the real impact would be.
+
 end subroutine update_from_obs_inc
 
 
@@ -1554,8 +1568,7 @@
    first_get_correction = .false.
 endif
 
-
-! First quick modification of correl to expected true correl for test (should interp)
+! Interpolate to get values of expected correlation and mean_factor
 if(scorrel < -1.0_r8) then
    correl = -1.0_r8
    mean_factor = 1.0_r8
@@ -1563,29 +1576,42 @@
    correl = 1.0_r8
    mean_factor = 1.0_r8
 else if(scorrel <= -0.995_r8) then
-   fract = (scorrel + 1.0_r8) / 0.05_r8
+   fract = (scorrel + 1.0_r8) / 0.005_r8
    correl = (exp_true_correl(1) + 1.0_r8) * fract - 1.0_r8
    mean_factor = (alpha(1) - 1.0_r8) * fract + 1.0_r8
 else if(scorrel >= 0.995_r8) then
-   fract = (scorrel - 0.995_r8) / 0.05_r8
+   fract = (scorrel - 0.995_r8) / 0.005_r8
    correl = (1.0_r8 - exp_true_correl(200)) * fract + exp_true_correl(200)
    mean_factor = (1.0_r8 - alpha(200)) * fract + alpha(200)
 else
-   low_indx = floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8)
-   low_correl = -0.995_r8 + (low_indx - 1) * 0.01
+   !low_indx = floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8)
+   !if(low_indx < 1) low_indx = 1
+   !if(low_indx > 199) low_indx = 199
+   low_indx = min(floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8), 199)
+   low_correl = -0.995_r8 + (low_indx - 1) * 0.01_r8
    low_exp_correl = exp_true_correl(low_indx)
    low_alpha = alpha(low_indx)
    high_indx = low_indx + 1
-   high_correl = low_correl + 0.01
+   high_correl = low_correl + 0.01_r8
    high_exp_correl = exp_true_correl(high_indx)
-   high_alpha = alpha(low_indx)
+   high_alpha = alpha(high_indx)
    fract = (scorrel - low_correl) / (high_correl - low_correl)
    correl = (high_exp_correl - low_exp_correl) * fract + low_exp_correl
    mean_factor = (high_alpha - low_alpha) * fract + low_alpha
 endif
 
-expected_true_correl = correl
+expected_true_correl = correl 
 
+! Don't want Monte Carlo interpolation problems to put us outside of a
+! ratio between 0 and 1 for expected_true_correl / sample_correl
+! If they have different signs, expected should just be 0
+if(expected_true_correl * scorrel <= 0.0_r8) then
+   expected_true_correl = 0.0_r8
+else if(abs(expected_true_correl) > abs(scorrel)) then
+   ! If same sign, expected should not be bigger in absolute value
+   expected_true_correl = scorrel
+endif 
+
 end subroutine get_correction_from_file
 
 

From nancy at ucar.edu  Fri Apr 15 13:24:37 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Fri, 15 Apr 2011 13:24:37 -0600
Subject: [Dart-dev] [4861] DART/trunk/assim_tools/assim_tools_mod.f90: undo
 last commit. intended
Message-ID: 

Revision: 4861
Author:   nancy
Date:     2011-04-15 13:24:37 -0600 (Fri, 15 Apr 2011)
Log Message:
-----------
undo last commit.  intended to commit to branch, not trunk; not tested on other platforms yet

Modified Paths:
--------------
    DART/trunk/assim_tools/assim_tools_mod.f90

-------------- next part --------------
Modified: DART/trunk/assim_tools/assim_tools_mod.f90
===================================================================
--- DART/trunk/assim_tools/assim_tools_mod.f90	2011-04-15 19:19:53 UTC (rev 4860)
+++ DART/trunk/assim_tools/assim_tools_mod.f90	2011-04-15 19:24:37 UTC (rev 4861)
@@ -236,7 +236,7 @@
       write(errstring, '(A,I10,A)') 'Using adaptive localization, threshold ', &
          adaptive_localization_threshold, ' obs'
       call error_handler(E_MSG,'assim_tools_init:', errstring)
-      if(adaptive_cutoff_floor > 0.0_r8) then
+      if(adaptive_cutoff_floor > 0) then
          write(errstring, '(A,F18.6)') 'Minimum cutoff will not go below ', &
             adaptive_cutoff_floor
          call error_handler(E_MSG,'assim_tools_init:', 'Using adaptive localization cutoff floor.', &
@@ -248,10 +248,6 @@
       call error_handler(E_MSG,'assim_tools_init:', 'Writing localization diagnostics to file:')
       call error_handler(E_MSG,'assim_tools_init:', trim(localization_diagnostics_file))
    endif
-
-   if(sampling_error_correction) then
-      call error_handler(E_MSG,'assim_tools_init:', 'Using Sampling Error Correction')
-   endif
 endif
 
 end subroutine assim_tools_init
@@ -584,9 +580,8 @@
       grp_bot = grp_beg(group)
       grp_top = grp_end(group)
       obs_prior_mean(group) = sum(obs_prior(grp_bot:grp_top)) / grp_size
-      obs_prior_var(group) = sum((obs_prior(grp_bot:grp_top) - obs_prior_mean(group))**2) / &
-         (grp_size - 1)
-      if (obs_prior_var(group) < 0.0_r8) obs_prior_var(group) = 0.0_r8
+      obs_prior_var(group)  = sum(obs_prior(grp_bot:grp_top) * obs_prior(grp_bot:grp_top)) - &
+         grp_size * obs_prior_mean(group)**2
    end do
 
    ! Need to get obs density first in case of adaptive localization
@@ -743,7 +738,7 @@
                obs_prior_var(group), obs_inc(grp_bot:grp_top), &
                ens_handle%copies(grp_bot:grp_top, state_index), grp_size, &
                increment(grp_bot:grp_top), reg_coef(group), net_a(group), correl(group))
-         else
+            else
             call update_from_obs_inc(obs_prior(grp_bot:grp_top), obs_prior_mean(group), &
                obs_prior_var(group), obs_inc(grp_bot:grp_top), &
                ens_handle%copies(grp_bot:grp_top, state_index), grp_size, &
@@ -893,7 +888,7 @@
                 num_close_obs_buffered + num_close_states_buffered
       if (num_close_obs_buffered+num_close_obs_calls_made+ &
           num_close_states_buffered+num_close_states_calls_made > 0) then 
-         print *, "Percent saved: ", 100.0_r8 * &
+         print *, "Percent saved: ", 100. * &
                    (real(num_close_obs_buffered+num_close_states_buffered, r8) /  &
                    (num_close_obs_calls_made+num_close_obs_buffered+ &
                     num_close_states_calls_made+num_close_states_buffered))
@@ -1427,15 +1422,15 @@
 real(r8),           intent(inout) :: net_a
 real(r8), optional, intent(inout) :: correl_out
 
-real(r8) :: obs_state_cov
+real(r8) :: t(ens_size), obs_state_cov
 real(r8) :: restoration_inc(ens_size), state_mean, state_var, correl
 real(r8) :: factor, exp_true_correl, mean_factor
 
 ! For efficiency, just compute regression coefficient here unless correl is needed
-state_mean = sum(state) / ens_size
-obs_state_cov = sum( (state - state_mean) * (obs - obs_prior_mean) ) / (ens_size - 1)
+t = obs - obs_prior_mean
+obs_state_cov = sum(t * state)
 
-if (obs_prior_var > 0.0_r8) then
+if (obs_prior_var /= 0.0_r8) then
    reg_coef = obs_state_cov/obs_prior_var
 else
    reg_coef = 0.0_r8
@@ -1444,9 +1439,8 @@
 ! If correl_out is present, need correl for adaptive inflation
 ! Also needed for file correction below
 if(present(correl_out) .or. sampling_error_correction) then
-   state_var = sum((state - state_mean)**2) / (ens_size - 1)
-   if (state_var < 0.0_r8) state_var = 0.0_r8
-   if ((obs_prior_var <= 0.0_r8) .or. (state_var <= 0.0_r8)) then
+   state_var = sum(state * state) - sum(state)**2 / ens_size
+   if(obs_prior_var * state_var <= 0.0_r8) then
       correl = 0.0_r8
    else
       correl = obs_state_cov / sqrt(obs_prior_var * state_var)
@@ -1461,6 +1455,7 @@
 ! Get the expected actual correlation and the regression weight reduction factor
 if(sampling_error_correction) then
    call get_correction_from_file(ens_size, correl, mean_factor, exp_true_correl)
+   !write(*, *) correl, exp_true_correl, mean_factor
    ! Watch out for division by zero; if correl is really small regression is safely 0
    if(abs(correl) > 0.001) then
       reg_coef = reg_coef * (exp_true_correl / correl) * mean_factor
@@ -1493,7 +1488,7 @@
    ! the reciprocal of the ensemble_size (slope = 0.80 / ens_size). These are empirical
    ! for now. See also README in spread_restoration_paper documentation.
    !!!factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) * (0.8_r8 / ens_size)) - 1.0_r8
-   factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) / (-2.4711_r8 + 1.6386_r8 * ens_size)) - 1.0_r8
+   factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) / (-2.4711 + 1.6386 * ens_size)) - 1.0_r8
    !!!factor = 1.0_r8 / (1.0_r8 + (net_a**2 - 1.0_r8) * (-0.0111_r8 + .8585_r8 / ens_size)) - 1.0_r8
 
    ! Variance restoration
@@ -1502,15 +1497,6 @@
    state_inc = state_inc + restoration_inc
 endif
 
-!! NOTE:  if requested to be returned, correl_out is set further up in the
-!! code, before the sampling error correction, if enabled, is applied.
-!! this means it's returning a different larger value than the correl 
-!! being returned here.  it's used by the adaptive inflation and so the
-!! inflation will see a slightly different correlation value.  it isn't
-!! clear that this is a bad thing; it means the inflation might be a bit
-!! larger than it would otherwise.  before we move any code this would
-!! need to be studied to see what the real impact would be.
-
 end subroutine update_from_obs_inc
 
 
@@ -1568,7 +1554,8 @@
    first_get_correction = .false.
 endif
 
-! Interpolate to get values of expected correlation and mean_factor
+
+! First quick modification of correl to expected true correl for test (should interp)
 if(scorrel < -1.0_r8) then
    correl = -1.0_r8
    mean_factor = 1.0_r8
@@ -1576,42 +1563,29 @@
    correl = 1.0_r8
    mean_factor = 1.0_r8
 else if(scorrel <= -0.995_r8) then
-   fract = (scorrel + 1.0_r8) / 0.005_r8
+   fract = (scorrel + 1.0_r8) / 0.05_r8
    correl = (exp_true_correl(1) + 1.0_r8) * fract - 1.0_r8
    mean_factor = (alpha(1) - 1.0_r8) * fract + 1.0_r8
 else if(scorrel >= 0.995_r8) then
-   fract = (scorrel - 0.995_r8) / 0.005_r8
+   fract = (scorrel - 0.995_r8) / 0.05_r8
    correl = (1.0_r8 - exp_true_correl(200)) * fract + exp_true_correl(200)
    mean_factor = (1.0_r8 - alpha(200)) * fract + alpha(200)
 else
-   !low_indx = floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8)
-   !if(low_indx < 1) low_indx = 1
-   !if(low_indx > 199) low_indx = 199
-   low_indx = min(floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8), 199)
-   low_correl = -0.995_r8 + (low_indx - 1) * 0.01_r8
+   low_indx = floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8)
+   low_correl = -0.995_r8 + (low_indx - 1) * 0.01
    low_exp_correl = exp_true_correl(low_indx)
    low_alpha = alpha(low_indx)
    high_indx = low_indx + 1
-   high_correl = low_correl + 0.01_r8
+   high_correl = low_correl + 0.01
    high_exp_correl = exp_true_correl(high_indx)
-   high_alpha = alpha(high_indx)
+   high_alpha = alpha(low_indx)
    fract = (scorrel - low_correl) / (high_correl - low_correl)
    correl = (high_exp_correl - low_exp_correl) * fract + low_exp_correl
    mean_factor = (high_alpha - low_alpha) * fract + low_alpha
 endif
 
-expected_true_correl = correl 
+expected_true_correl = correl
 
-! Don't want Monte Carlo interpolation problems to put us outside of a
-! ratio between 0 and 1 for expected_true_correl / sample_correl
-! If they have different signs, expected should just be 0
-if(expected_true_correl * scorrel <= 0.0_r8) then
-   expected_true_correl = 0.0_r8
-else if(abs(expected_true_correl) > abs(scorrel)) then
-   ! If same sign, expected should not be bigger in absolute value
-   expected_true_correl = scorrel
-endif 
-
 end subroutine get_correction_from_file
 
 

From nancy at ucar.edu  Mon Apr 18 14:56:26 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Mon, 18 Apr 2011 14:56:26 -0600
Subject: [Dart-dev] [4865] DART/trunk/observations/text: Revamped the html
 file quite a bit; added sections with more advice on
Message-ID: 

Revision: 4865
Author:   nancy
Date:     2011-04-18 14:56:25 -0600 (Mon, 18 Apr 2011)
Log Message:
-----------
Revamped the html file quite a bit; added sections with more advice on 
choosing types, calendars, and in particular setting observation error values.
Changed only a single comment line in the .f90 file; no code changes.

Modified Paths:
--------------
    DART/trunk/observations/text/text_to_obs.f90
    DART/trunk/observations/text/text_to_obs.html

-------------- next part --------------
Modified: DART/trunk/observations/text/text_to_obs.f90
===================================================================
--- DART/trunk/observations/text/text_to_obs.f90	2011-04-15 23:21:25 UTC (rev 4864)
+++ DART/trunk/observations/text/text_to_obs.f90	2011-04-18 20:56:25 UTC (rev 4865)
@@ -117,7 +117,8 @@
    !  location: lat, lon, and height in pressure or meters
    !  time: when the observation was taken
    !  type: from the DART list of obs types
-   !  error: very important - the instrument error plus model error
+   !  error: very important - the instrument error plus representativeness error
+   !        (see html file for more info)
 
    ! assume here a line is a type (1/2), location, time, value, obs error
 

Modified: DART/trunk/observations/text/text_to_obs.html
===================================================================
--- DART/trunk/observations/text/text_to_obs.html	2011-04-15 23:21:25 UTC (rev 4864)
+++ DART/trunk/observations/text/text_to_obs.html	2011-04-18 20:56:25 UTC (rev 4865)
@@ -2,26 +2,41 @@
           "http://www.w3.org/TR/html4/strict.dtd">
 
 
-MADIS Data
+program text_to_obs
 
+
 
 
 
 
-
    
-OVERVIEW /
+
    PROGRAM text_to_obs
    
+
+
+
+    
+    
+
+
    
+    DART project logo
+    
+       
    Jump to DART Documentation Main Index
    
+          version information for this file: 
    
+          
+          $Id$    
+       
    
+
 DATA SOURCES /
 PROGRAMS /
-KNOWN BUGS /
-FUTURE PLANS /
+DECISIONS /
+REFERENCES /
+ERRORS /
+PLANS / 
 TERMS OF USE
-
    
 
-
    Simple Text File to DART Converter
    
-
    
-$Id$
-
    
+
    Overview
    
 
+
    Text File to DART Converter
    
+
 
    
 If you have observations in spreadsheet or column format, in text, 
 with a single line per observation, then the files this directory
@@ -29,20 +44,26 @@
 into a format suitable for DART use.
 
    
 
    
-The workflow is you collect the needed information about each
+The workflow is usually: 
+
      
+
    • read in the needed information about each
 observation - location, time, data value, observation type - from 
-your data source, and then
-call a series of DART library routines to construct a derived type that
-contains all the observation information.  Then another set of DART library
-routines puts them into a time-sorted series.  And finally, the last
-routine is a write subroutine that writes out the series to a file.
+a data source (usually a file)
      • 
+
    • call a series of DART library routines to construct a derived type that
+contains all the information about a single observation
      • 
+
    • call another set of DART library
+routines to put it into a time-sorted series
      • 
+
    • repeat the last 2 steps until all observations are processed
      • 
+
    • finally, call a write subroutine that writes out the entire series to a file in 
+a format that DART can read in
      • 
+
    
 It is not recommended that you try to mimic the ascii file format
 by other means; the format is subject to change and the library routines
 will continue to be supported even if the physical format changes.
 
    
 
    
 If your input data is in some kind of format like netCDF or HDF,
-then one of the other converters (the MADIS ones for netCDF) might
+then one of the other converters (e.g. the MADIS ones for netCDF) might
 be a better starting place for adapting code.
 
    
 
@@ -54,10 +75,12 @@
 
    DATA SOURCES
    
 
 
    
-This part is up to you.  You will need a location, a value,
-a type, and some kind of error estimate.  The error estimate
-can be hardcoded in the converter if it's not available in
-the input data.
+This part is up to you.  
+For each observation you will need a location, a data value,
+a type, a time, and some kind of error estimate.  The error 
+estimate can be hardcoded in the converter if they are not 
+available in the input data.  See below for more details on 
+selecting an appropriate error value.
 
    
 
 
@@ -66,29 +89,33 @@
 
    
 
    PROGRAMS
    
 
    
-The text_to_obs.f90 file is the main converter.
-Look at the source code where it reads the sample data file.  You will
-almost certainly need to change the read to match your data.  The
-sample code provided reads in each text line into a character buffer
-and then uses reads from that buffer to parse up the data items.
-
    
-
    
+The text_to_obs.f90 file is the source
+for the main converter program.
+Look at the source code where it reads the example data file.  You will
+almost certainly need to change the "read" statement to match your data
+format.  The example code 
+reads each text line into a character buffer
+and then reads from that buffer to parse up the data items.
+
     
    
 To compile and test,
 go into the work subdirectory and run the quickbuild.csh
 script to build the converter and a couple of general purpose utilities.
-advance_time helps compute times, and
+advance_time helps with calendar and time computations,
+and the
 obs_sequence_tool manipulates DART observation files
 once they have been created.
 
    
 
    
-To add different observation types, look in the 
+To change the observation types, look in the 
 DART/obs_def directory.  If you can
 find an obs_def_XXX_mod.f90 file with an appropriate set
-of observations, put that name in the input.nml
-namelist file, in the &preprocess_nml namelist, 'input_files'.
-Multiples can be listed.  Then run quickbuild.csh again to remake
-the table of supported observation types before trying to recompile
-the source code.
+of observation types, change the 'use' lines in the converter
+source to include those types.  Then add that filename in the 
+input.nml namelist file
+to the &preprocess_nml namelist, the 'input_files' variable.
+Multiple files can be listed.  Then run quickbuild.csh again.
+It remakes the table of supported observation types before 
+trying to recompile the source code.
 
    
 
    
 An example script for converting batches of files is
@@ -102,6 +129,117 @@
 
    
 
 
+
+
+
    
+
    DECISIONS YOU MIGHT NEED TO MAKE
    
+
+
    Time
    
+
+
    
+The basic time type in DART is a pair of integers; one for
+the day number and one for the number of seconds.  For some
+of the simple models which aren't direct geophysical models
+time can start at day 0 and increment in any appropriate
+number of seconds or days.  For a model of a real-world system
+it is likely to have observations with a year/month/day,
+hour/min/seconds timestamp.  There are routines in DART
+to convert back and forth between the (day-number/seconds) 
+format and a variety of (year/month/day) calendars. 
+See here
+for more details on how DART stores time information and
+the types of available calendars.
+
    
+
+
    Error
    
+
+
    
+Each observation must specify an associated error.
+There is an error value per individual observation.
+It can be a constant value for all observations
+of that type, or it
+can vary by location, by height, by magnitude of the
+observed value, etc.  
+This value is the expected instrument error plus
+the representativeness error of the model.  
+The model error includes deficiencies
+in the equations representing the processes of the system
+as well as errors introduced by representing a continuous
+system as a series of discrete points.
+While the instrument error and the representativeness error
+could be specified separately, they each have the same
+impact on the assimilation and can be difficult to determine
+with any real accuracy.  For simplicity, in DART (and
+most current assimilation software) they are
+combined and specified as a single value.
+
    
+The instrument error is generally supplied by the instrument
+maker.  Sadly, however, it is frequently surprisingly
+difficult to find these values.
+For the representativeness error, a set of artificial
+observations could be generated with the 
+perfect_model_obs
+program and an assimilation experiment could be run to
+generate an estimate of the error in the model.
+In practice however most people make an educated guess
+on the values of the error and then start with a larger than
+expected value and decrease it based on the results of
+running some test assimilations.
+For these tests the namelist for the
+outlier threshold 
+should be disabled by setting it to -1 (the default value is 3). 
+This value controls whether the observation is rejected
+because the observed value is too far from the ensemble mean.
+
    
+If the diagnostics show that the difference between the mean of
+the forward operators and the observed value is consistently 
+smaller than the specified observation error, then the error 
+is probably too large.  
+A too-large error reduces the impact of an
+observation on the state.
+If the specified observation error is too small it is
+likely the observation will be rejected when the outlier
+threshold is enabled, and the observation will not be 
+assimilated.  It is important
+to look at the output observation sequence files after an assimilation
+to see how many observations were assimilated or rejected, and at
+the error and spread of the ensemble mean compared to the observation
+values.
+
    
+One last recommendation: if possible, the Prior forward operator values
+should be compared against the observations after several assimilation
+cycles.  If you plot results using the Posterior values
+it is always possible for the assimilation to overfit the
+observations and look good on the diagnostic plots.  But the
+actual test is to then advance the model and look at how the
+forecast of the state compares to the observations.
+
    
+
+
    Types
    
+
+
    
+All observations have to have a specific 'type'.  
+There are namelist controls
+to turn on and off the assimilation of observations at run-time
+by type, or to only evaluate the forward operator for an observation
+but have no impact on the state.  
+Several of the diagnostics also group observations by
+type to give aggregate statistics after an assimilation.
+Generally types are based on both the observing platform or
+instrument as well as the kind of observation, 
+e.g. RADIOSONDE_TEMPERATURE, ARGO_SALINITY, etc.
+Each type is associated with a single underlying generic 'kind',
+which controls what forward operator code is called inside the
+model, e.g. KIND_TEMPERATURE, KIND_DENSITY, etc.
+
    
+See here for more
+details on how to use and add new DART types.
+The DART obs_kind_mod.f90 defines a list of already defined
+observation kinds, and users can either use existing observation
+types in 'obs_def_xxx_mod.f90' files, or define their own.
+
    
+
+
 
 
 

From nancy at ucar.edu  Mon Apr 18 15:02:04 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Mon, 18 Apr 2011 15:02:04 -0600
Subject: [Dart-dev] [4866] DART/trunk/observations/text/text_to_obs.html:
 Forgot a short section on locations.
Message-ID: 

Revision: 4866
Author:   nancy
Date:     2011-04-18 15:02:04 -0600 (Mon, 18 Apr 2011)
Log Message:
-----------
Forgot a short section on locations.

Modified Paths:
--------------
    DART/trunk/observations/text/text_to_obs.html

-------------- next part --------------
Modified: DART/trunk/observations/text/text_to_obs.html
===================================================================
--- DART/trunk/observations/text/text_to_obs.html	2011-04-18 20:56:25 UTC (rev 4865)
+++ DART/trunk/observations/text/text_to_obs.html	2011-04-18 21:02:04 UTC (rev 4866)
@@ -239,6 +239,20 @@
 types in 'obs_def_xxx_mod.f90' files, or define their own.
 
    
 
+
    Locations
    
+
+
    
+The two most common choices for specifying the location of an
+observation are the 
+threed_sphere
+and the
+oned
+locations.  For observations of a real-world system, the 3D Sphere
+is generally the best choice.  For low-order, 1D models, the oned
+locations are the most commonly used.  The observation locations
+need to match the type of locations used in the model.
+
     
+
 
 
 

From nancy at ucar.edu  Tue Apr 19 16:57:13 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Tue, 19 Apr 2011 16:57:13 -0600
Subject: [Dart-dev] [4869] DART/trunk/assim_tools/assim_tools_mod.f90:
 Several critical fixes for those users who redefine R8 to be R4.
Message-ID: 

Revision: 4869
Author:   nancy
Date:     2011-04-19 16:57:13 -0600 (Tue, 19 Apr 2011)
Log Message:
-----------
Several critical fixes for those users who redefine R8 to be R4.

1. Compute variance and covariance in more numerically stable ways
if running with single precision reals.  The previous code was fine
when running with real*8, but the updated code retains more accuracy
when running real*4.

2. Fix minor issues with the Sampling Error Correction; a few of
the 

Modified Paths:
--------------
    DART/trunk/assim_tools/assim_tools_mod.f90

-------------- next part --------------
Modified: DART/trunk/assim_tools/assim_tools_mod.f90
===================================================================
--- DART/trunk/assim_tools/assim_tools_mod.f90	2011-04-19 22:40:49 UTC (rev 4868)
+++ DART/trunk/assim_tools/assim_tools_mod.f90	2011-04-19 22:57:13 UTC (rev 4869)
@@ -236,7 +236,7 @@
       write(errstring, '(A,I10,A)') 'Using adaptive localization, threshold ', &
          adaptive_localization_threshold, ' obs'
       call error_handler(E_MSG,'assim_tools_init:', errstring)
-      if(adaptive_cutoff_floor > 0) then
+      if(adaptive_cutoff_floor > 0.0_r8) then
          write(errstring, '(A,F18.6)') 'Minimum cutoff will not go below ', &
             adaptive_cutoff_floor
          call error_handler(E_MSG,'assim_tools_init:', 'Using adaptive localization cutoff floor.', &
@@ -248,6 +248,10 @@
       call error_handler(E_MSG,'assim_tools_init:', 'Writing localization diagnostics to file:')
       call error_handler(E_MSG,'assim_tools_init:', trim(localization_diagnostics_file))
    endif
+
+   if(sampling_error_correction) then
+      call error_handler(E_MSG,'assim_tools_init:', 'Using Sampling Error Correction')
+   endif
 endif
 
 end subroutine assim_tools_init
@@ -580,8 +584,9 @@
       grp_bot = grp_beg(group)
       grp_top = grp_end(group)
       obs_prior_mean(group) = sum(obs_prior(grp_bot:grp_top)) / grp_size
-      obs_prior_var(group)  = sum(obs_prior(grp_bot:grp_top) * obs_prior(grp_bot:grp_top)) - &
-         grp_size * obs_prior_mean(group)**2
+      obs_prior_var(group) = sum((obs_prior(grp_bot:grp_top) - obs_prior_mean(group))**2) / &
+         (grp_size - 1)
+      if (obs_prior_var(group) < 0.0_r8) obs_prior_var(group) = 0.0_r8
    end do
 
    ! Need to get obs density first in case of adaptive localization
@@ -738,7 +743,7 @@
                obs_prior_var(group), obs_inc(grp_bot:grp_top), &
                ens_handle%copies(grp_bot:grp_top, state_index), grp_size, &
                increment(grp_bot:grp_top), reg_coef(group), net_a(group), correl(group))
-            else
+         else
             call update_from_obs_inc(obs_prior(grp_bot:grp_top), obs_prior_mean(group), &
                obs_prior_var(group), obs_inc(grp_bot:grp_top), &
                ens_handle%copies(grp_bot:grp_top, state_index), grp_size, &
@@ -888,7 +893,7 @@
                 num_close_obs_buffered + num_close_states_buffered
       if (num_close_obs_buffered+num_close_obs_calls_made+ &
           num_close_states_buffered+num_close_states_calls_made > 0) then 
-         print *, "Percent saved: ", 100. * &
+         print *, "Percent saved: ", 100.0_r8 * &
                    (real(num_close_obs_buffered+num_close_states_buffered, r8) /  &
                    (num_close_obs_calls_made+num_close_obs_buffered+ &
                     num_close_states_calls_made+num_close_states_buffered))
@@ -1422,15 +1427,15 @@
 real(r8),           intent(inout) :: net_a
 real(r8), optional, intent(inout) :: correl_out
 
-real(r8) :: t(ens_size), obs_state_cov
+real(r8) :: obs_state_cov
 real(r8) :: restoration_inc(ens_size), state_mean, state_var, correl
 real(r8) :: factor, exp_true_correl, mean_factor
 
 ! For efficiency, just compute regression coefficient here unless correl is needed
-t = obs - obs_prior_mean
-obs_state_cov = sum(t * state)
+state_mean = sum(state) / ens_size
+obs_state_cov = sum( (state - state_mean) * (obs - obs_prior_mean) ) / (ens_size - 1)
 
-if (obs_prior_var /= 0.0_r8) then
+if (obs_prior_var > 0.0_r8) then
    reg_coef = obs_state_cov/obs_prior_var
 else
    reg_coef = 0.0_r8
@@ -1439,8 +1444,9 @@
 ! If correl_out is present, need correl for adaptive inflation
 ! Also needed for file correction below
 if(present(correl_out) .or. sampling_error_correction) then
-   state_var = sum(state * state) - sum(state)**2 / ens_size
-   if(obs_prior_var * state_var <= 0.0_r8) then
+   state_var = sum((state - state_mean)**2) / (ens_size - 1)
+   if (state_var < 0.0_r8) state_var = 0.0_r8
+   if ((obs_prior_var <= 0.0_r8) .or. (state_var <= 0.0_r8)) then
       correl = 0.0_r8
    else
       correl = obs_state_cov / sqrt(obs_prior_var * state_var)
@@ -1455,7 +1461,6 @@
 ! Get the expected actual correlation and the regression weight reduction factor
 if(sampling_error_correction) then
    call get_correction_from_file(ens_size, correl, mean_factor, exp_true_correl)
-   !write(*, *) correl, exp_true_correl, mean_factor
    ! Watch out for division by zero; if correl is really small regression is safely 0
    if(abs(correl) > 0.001) then
       reg_coef = reg_coef * (exp_true_correl / correl) * mean_factor
@@ -1488,7 +1493,7 @@
    ! the reciprocal of the ensemble_size (slope = 0.80 / ens_size). These are empirical
    ! for now. See also README in spread_restoration_paper documentation.
    !!!factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) * (0.8_r8 / ens_size)) - 1.0_r8
-   factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) / (-2.4711 + 1.6386 * ens_size)) - 1.0_r8
+   factor = 1.0_r8 / (1.0_r8 + (net_a - 1.0_r8) / (-2.4711_r8 + 1.6386_r8 * ens_size)) - 1.0_r8
    !!!factor = 1.0_r8 / (1.0_r8 + (net_a**2 - 1.0_r8) * (-0.0111_r8 + .8585_r8 / ens_size)) - 1.0_r8
 
    ! Variance restoration
@@ -1497,6 +1502,15 @@
    state_inc = state_inc + restoration_inc
 endif
 
+!! NOTE: if requested to be returned, correl_out is set further up in the
+!! code, before the sampling error correction, if enabled, is applied.
+!! this means it's returning a different larger value than the correl 
+!! being returned here.  it's used by the adaptive inflation and so the
+!! inflation will see a slightly different correlation value.  it isn't
+!! clear that this is a bad thing; it means the inflation might be a bit
+!! larger than it would otherwise.  before we move any code this would
+!! need to be studied to see what the real impact would be.
+
 end subroutine update_from_obs_inc
 
 
@@ -1554,8 +1568,7 @@
    first_get_correction = .false.
 endif
 
-
-! First quick modification of correl to expected true correl for test (should interp)
+! Interpolate to get values of expected correlation and mean_factor
 if(scorrel < -1.0_r8) then
    correl = -1.0_r8
    mean_factor = 1.0_r8
@@ -1563,29 +1576,47 @@
    correl = 1.0_r8
    mean_factor = 1.0_r8
 else if(scorrel <= -0.995_r8) then
-   fract = (scorrel + 1.0_r8) / 0.05_r8
+   fract = (scorrel + 1.0_r8) / 0.005_r8
    correl = (exp_true_correl(1) + 1.0_r8) * fract - 1.0_r8
    mean_factor = (alpha(1) - 1.0_r8) * fract + 1.0_r8
 else if(scorrel >= 0.995_r8) then
-   fract = (scorrel - 0.995_r8) / 0.05_r8
+   fract = (scorrel - 0.995_r8) / 0.005_r8
    correl = (1.0_r8 - exp_true_correl(200)) * fract + exp_true_correl(200)
    mean_factor = (1.0_r8 - alpha(200)) * fract + alpha(200)
 else
+   ! given the ifs above, the floor() computation below for low_indx 
+   ! should always result in a value in the range 1 to 199.  but if this
+   ! code is compiled with r8=r4 (single precision reals) it turns out
+   ! to be possible to get values a few bits below 0 which results in
+   ! a very large negative integer.  the limit tests below ensure the
+   ! index stays in a legal range.
    low_indx = floor((scorrel + 0.995_r8) / 0.01_r8 + 1.0_r8)
-   low_correl = -0.995_r8 + (low_indx - 1) * 0.01
+   if (low_indx <   1) low_indx =   1
+   if (low_indx > 199) low_indx = 199
+   low_correl = -0.995_r8 + (low_indx - 1) * 0.01_r8
    low_exp_correl = exp_true_correl(low_indx)
    low_alpha = alpha(low_indx)
    high_indx = low_indx + 1
-   high_correl = low_correl + 0.01
+   high_correl = low_correl + 0.01_r8
    high_exp_correl = exp_true_correl(high_indx)
-   high_alpha = alpha(low_indx)
+   high_alpha = alpha(high_indx)
    fract = (scorrel - low_correl) / (high_correl - low_correl)
    correl = (high_exp_correl - low_exp_correl) * fract + low_exp_correl
    mean_factor = (high_alpha - low_alpha) * fract + low_alpha
 endif
 
-expected_true_correl = correl
+expected_true_correl = correl 
 
+! Don't want Monte Carlo interpolation problems to put us outside of a
+! ratio between 0 and 1 for expected_true_correl / sample_correl
+! If they have different signs, expected should just be 0
+if(expected_true_correl * scorrel <= 0.0_r8) then
+   expected_true_correl = 0.0_r8
+else if(abs(expected_true_correl) > abs(scorrel)) then
+   ! If same sign, expected should not be bigger in absolute value
+   expected_true_correl = scorrel
+endif 
+
 end subroutine get_correction_from_file
 
 

From nancy at ucar.edu  Fri Apr 22 08:20:52 2011
From: nancy at ucar.edu (nancy at ucar.edu)
Date: Fri, 22 Apr 2011 08:20:52 -0600
Subject: [Dart-dev] [4870]
 DART/trunk/models/MITgcm_ocean/work/path_names_column_rand: Add missing
 dependency on obs_kind_mod, now needed by the locations mod.
Message-ID: 

Revision: 4870
Author:   nancy
Date:     2011-04-22 08:20:52 -0600 (Fri, 22 Apr 2011)
Log Message:
-----------
Add missing dependency on obs_kind_mod, now needed by the locations mod.

Modified Paths:
--------------
    DART/trunk/models/MITgcm_ocean/work/path_names_column_rand

-------------- next part --------------
Modified: DART/trunk/models/MITgcm_ocean/work/path_names_column_rand
===================================================================
--- DART/trunk/models/MITgcm_ocean/work/path_names_column_rand	2011-04-19 22:57:13 UTC (rev 4869)
+++ DART/trunk/models/MITgcm_ocean/work/path_names_column_rand	2011-04-22 14:20:52 UTC (rev 4870)
@@ -1,6 +1,7 @@
 common/types_mod.f90
 mpi_utilities/null_mpi_utilities_mod.f90
 models/MITgcm_ocean/column_rand.f90
+obs_kind/obs_kind_mod.f90
 random_nr/random_nr_mod.f90
 random_seq/random_seq_mod.f90
 location/threed_sphere/location_mod.f90