From nancy at ucar.edu Tue Oct 8 11:15:24 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 08 Oct 2013 11:15:24 -0600 Subject: [Dart-dev] [6498] DART/trunk/cov_cutoff/cov_cutoff_mod.html: fix a missing and add a 'summary' tag to Message-ID: Revision: 6498 Author: nancy Date: 2013-10-08 11:15:24 -0600 (Tue, 08 Oct 2013) Log Message: ----------- fix a missing and add a 'summary' tag to the tables so the 'tidy' html checker is happier. Modified Paths: -------------- DART/trunk/cov_cutoff/cov_cutoff_mod.html -------------- next part -------------- Modified: DART/trunk/cov_cutoff/cov_cutoff_mod.html =================================================================== --- DART/trunk/cov_cutoff/cov_cutoff_mod.html 2013-09-30 16:17:15 UTC (rev 6497) +++ DART/trunk/cov_cutoff/cov_cutoff_mod.html 2013-10-08 17:15:24 UTC (rev 6498) @@ -74,12 +74,13 @@ Item Type - Description + Description + select_localization - integer
+ integer Selects the localization function. - + + @@ -116,7 +118,7 @@
[top]

PUBLIC INTERFACES

- +
use cov_factor_mod, only : comp_cov_factor
@@ -162,7 +164,7 @@ functions.

- +
@@ -178,7 +180,7 @@ - +
var Weighting factor.
z_inLocation of target.
target_kind Generic kind of target.
localization_override  
localization_override Controls localization type if present. Same values as for namelist control.
@@ -193,7 +195,7 @@
[top]

FILES

- +
@@ -219,7 +221,7 @@
[top]

ERROR CODES and CONDITIONS

-
filename purpose
input.nml
+
@@ -277,7 +279,7 @@ - +
RoutineMessageComment
comp_cov_factor
Revision: $Revision$
Source: $URL$
Change Date: $Date$
Change history:  try "svn log" or "svn diff"
Change history: try "svn log" or "svn diff"
From nancy at ucar.edu Tue Oct 8 16:38:57 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 08 Oct 2013 16:38:57 -0600 Subject: [Dart-dev] [6499] DART/trunk/doc/html/boilerplate: update all the formatting for these programs, and add a couple Message-ID: Revision: 6499 Author: nancy Date: 2013-10-08 16:38:57 -0600 (Tue, 08 Oct 2013) Log Message: ----------- update all the formatting for these programs, and add a couple more for other kinds of repetative sections. also updated the template and boilerplate to be more consistent with how we're formatting the html files in the system now. this is all because i found a module with no docs at all and i didn't want to type it all in from scratch. so to save that time i spent a couple hours updating these programs. :/ now i'll go back and create the html doc page that's missing. Modified Paths: -------------- DART/trunk/doc/html/boilerplate/boilerplate.html DART/trunk/doc/html/boilerplate/html_function.f90 DART/trunk/doc/html/boilerplate/html_namelist.f90 DART/trunk/doc/html/boilerplate/html_subroutine.f90 DART/trunk/doc/html/boilerplate/template.html Added Paths: ----------- DART/trunk/doc/html/boilerplate/html_functoc.f90 DART/trunk/doc/html/boilerplate/html_type.f90 -------------- next part -------------- Modified: DART/trunk/doc/html/boilerplate/boilerplate.html =================================================================== --- DART/trunk/doc/html/boilerplate/boilerplate.html 2013-10-08 17:15:24 UTC (rev 6498) +++ DART/trunk/doc/html/boilerplate/boilerplate.html 2013-10-08 22:38:57 UTC (rev 6499) @@ -2,23 +2,22 @@ "http://www.w3.org/TR/html4/strict.dtd"> -module xxxxxxx - - - +PROGRAM OR MODULE name_of_thing + + -

MODULE xxxxxxx

+

PROGRM OR MODULE name_of_thing

- +
- DART project logo + DART project logo -

Jump to DART Documentation Main Index
+

Jump to DART Documentation Main Index
version information for this file:
$Id$
@@ -27,6 +26,7 @@

NAMELIST / +MODULES USED / INTERFACES / FILES / REFERENCES / @@ -40,99 +40,71 @@

This is the place for the general description of the module or program or whatever. -

+

+

overview issues, how/when/why to use this thing, etc. -

- Atmospheric driver for the dry B-grid dynamical core and - Held-Suarez benchmark (aka simple_physics).

+

+ more stuff about code, usage, etc. +

-
+
[top]

NAMELIST

+

+DART namelists are +always read from file input.nml. +

We adhere to the F90 standard of starting a namelist with an ampersand '&' and terminating with a slash '/' for all our namelist input. -Consider yourself forewarned that character strings that contain a '/' must be -enclosed in quotes to prevent them from prematurely terminating the namelist. -The namelist declaration (i.e. what follows) has a different syntax, naturally. +Character strings that contain a '/' must be +enclosed in quotes to prevent them from +prematurely terminating the namelist.

+
-namelist / model_nml /  &
-    ens_size, spin_up_steps, num_steps, output_start, ens_spin_up_steps, &
-    obs_freq, cov_inflate, mean_inflate, restart, add_on, noise_amp
+&NAMELIST_NML 
+   name=value,
+   name=value, 
+   name=value
+/
 
-
+
+
-

-This namelist is read in a file called input.nml -

+
+ + + + + + + -
Item Type Description
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + +
Contents Type Description
ens_size integer [default: 0]The number of "i" and "j" rows processed each time the - modular physics is called.
spin_up_steps integer the month
num_steps real(r8) the day-of-month
output_start real(r4) the hour-of-day
ens_spin_up_steps real(r4) the minute
obs_freq, real(r4) the second
cov_inflate real(r8) uniformly monotonic time
mean_inflate character(len=80) the type of calendar being used
restart integer the type of calendar being used
add_on integer the type of calendar being used
noise_amp integer the type of calendar being used
nametype(often multi-line) description +
-
+
- -
+
[top]

OTHER MODULES USED

 types_mod
@@ -150,10 +122,11 @@
 
 
 
-
+ +
[top]

PUBLIC INTERFACES

- +
@@ -167,8 +140,9 @@ [like this].

- + +
@@ -186,96 +160,35 @@ If we don't know where we are, how do we know which exit to take?

-
use xxxxxxx, only : yyypubtype1
 yyyroutine1
+
+ + + + + - - - - - - + + + + +
Component Description
Component Description
x is the nondimensional distance [0,1]
x is the nondimensional distance [0,1]

- - - -
-
-
-type time_type
-   private
-   integer           :: year
-   integer           :: month
-   integer           :: day
-   integer           :: hour
-   integer           :: minute
-   integer           :: second
-   real(r8)          :: umt
-   character(len=80) :: calendar
-end type time_type
-
-
- -
- -

The time type is ubiquitous throughout the module. Given the value -of calendar, everything should be pretty clear. -calendar may take on several values: -julian, -nondimensional, -uniformleap, -threesixty. -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Component Description
year the (non-negative) year
month the month
day the day-of-month
hour the hour-of-day
minute the minute
second the second
umt uniformly monotonic time
calendar the type of calendar being used
- -
-
-
-call yyyroutine1( var1, var2, var3 +subroutine yyyroutine1( var1, var2, var3 [, global])
 type(time_type),          intent(in)    ::  var1 
 real(r8), dimension(:),   intent(inout) ::  var2 
 real(r8), dimension(:,:), intent(out)   ::  var3 
-real(r8), OPTIONAL,       intent(in)    ::  global 
+real(r8), optional,       intent(in)    ::  global 
 
@@ -290,24 +203,25 @@ slices, dices and makes julienne fries. But wait! There's more!

- - - - - - +
var1is the number of spark plugs in a '67 Cuda.
var2
+ + + + + - - + spotted gobies per square hectare at that level. + + - - - - + feathers on said fowl, naturally. + + + + +
var1is the number of spark plugs in a '67 Cuda.
var2 is used for both input and output. Upon input, this contains the furlongs per weekday for every vertical level. Upon exit, we now have the number of - spotted gobies per square hectare at that level.
var3
var3 The leftmost dimension pertains to the number of feet in an orange-footed brush fowl, indiginous to Australia. The next dimension is the number of - feathers on said fowl, naturally.
globalis really, REALLY useful in certain situations.
globalis really, REALLY useful in certain situations.

@@ -322,14 +236,14 @@

- var = yyyroutine2( var1, var2, var3, + function yyyroutine2( var1, var2, var3, [,bob] )
 type(time_type),          intent(in)  ::  var1 
 real(r8), dimension(:),   intent(in)  ::  var2 
 real(r8), dimension(:,:), intent(in)  ::  var3 
-real(r8), OPTIONAL,       intent(in)  ::  bob 
-integer, dimension(size(var2))        ::  var 
+real(r8), optional,       intent(in)  ::  bob 
+integer, dimension(size(var2))        ::  yyyroutine2 
 
@@ -345,24 +259,24 @@ deep fries.

- - - +
var1  
+ + - + - + - + - + - +
var1 the first time you changed your oil.
var2
var2 miles between every oil change you've ever done. Don't lie.
var3
var3 the distances you've ridden. Each row corresponds to the hour-of-day, each column is a different day-of-the-week.
bob
bob mean time between failures. in msec.
var
yyroutine2 number of gray hairs as a function of time. in kilohairs.

@@ -377,16 +291,21 @@ -


+
[top]

FILES

This is the place to discuss the files that are associated with this module. They could be input files, output files, data files, shell scripts ... anything.

- - - +
filenamepurpose
+ + + + + + + @@ -400,6 +319,7 @@ +
filenamepurpose
inputfile1 to read some input
input.nml
dart_log.nml [default name] the record of all the namelists actually USED - contains the default values
@@ -407,12 +327,11 @@ -
+
[top]

REFERENCES

    -
  • Anderson, J., T. Hoar, K. Raeder, - H. Liu, N. Collins, R. Torn, - and A. Arellano, 2009:
    +
  • Anderson, J., T. Hoar, K. Raeder, H. Liu, + N. Collins, R. Torn, and A. Arellano, 2009:
    The Data Assimilation Research Testbed: A Community Facility. Bull. Amer. Meteor. Soc., 90, 1283-1296.
    @@ -427,10 +346,10 @@ -
    +
    [top]

    ERROR CODES and CONDITIONS

    - +
    @@ -455,7 +374,7 @@ -
    +
    [top]

    FUTURE PLANS

    none at this time @@ -466,7 +385,7 @@ -


    +
    [top]

    PRIVATE COMPONENTS

    N/A @@ -490,7 +409,7 @@ -


    +
    [top]

    Terms of Use

    @@ -506,7 +425,7 @@

    - +
    RoutineMessageComment
    xxxx
    Revision: $Revision$
    Source: $URL$
    Change Date: $Date$
    Change history:  try "svn log" or "svn diff"
    Change history: try "svn log" or "svn diff"
    Modified: DART/trunk/doc/html/boilerplate/html_function.f90 =================================================================== --- DART/trunk/doc/html/boilerplate/html_function.f90 2013-10-08 17:15:24 UTC (rev 6498) +++ DART/trunk/doc/html/boilerplate/html_function.f90 2013-10-08 22:38:57 UTC (rev 6499) @@ -12,8 +12,10 @@ integer :: num, num_args, i, max_len, len, j character(len = 299) :: c1, c2, function_name, function_type, spaces character(len = 299), allocatable :: arg_name(:), arg_type(:) +logical :: ispointer write(22, *) '' +write(22, *) '' spaces = ' ' @@ -25,7 +27,7 @@ read(*, *) arg_name(0) write(22, *) '' -write(22, *) '


    ' +write(22, *) '
    ' write(22, *) '
    ' ! Loop to read in the argument names @@ -41,23 +43,26 @@ end do ! Write out the command line -write(22, *) ' var = ', trim(arg_name(0)), '(', trim(c1), ') ' +write(22, *) ' function ', trim(arg_name(0)), '(', trim(c1), ') ' write(22, *) '
    '
     
     max_len = 0
     do i = 0, num_args
        write(*, *) 'input the type for ', trim(arg_name(i))
        read(*, *) arg_type(i)
    -   write(*, *) 'input 1 if allocatable'
    -   read(*, *) j
    -   if(j == 1) arg_type(i) = trim(arg_type(i)) // ', allocatable'
    -   write(*, *) 'input 1 if optional'
    -   read(*, *) j
    -   if(j == 1) arg_type(i) = trim(arg_type(i)) // ', optional'
    +   if (i > 0) then
    +      write(*, *) 'input 1 if allocatable'
    +      read(*, *) j
    +      if(j == 1) arg_type(i) = trim(arg_type(i)) // ', allocatable'
    +      write(*, *) 'input 1 if optional'
    +      read(*, *) j
    +      if(j == 1) arg_type(i) = trim(arg_type(i)) // ', optional'
    +   endif
        write(*, *) 'input 1 if pointer'
        read(*, *) j
    -   if(j == 1) arg_type(i) = trim(arg_type(i)) // ', pointer'
    -   write(*, *) 'input dimesnion (0, 1, 2, 3)'
    +   ispointer = (j == 1)
    +   if(ispointer) arg_type(i) = trim(arg_type(i)) // ', pointer'
    +   write(*, *) 'input dimension (0, 1, 2, 3)'
        read(*, *) j
        if(j == 1) then
           arg_type(i) = trim(arg_type(i)) // ', dimension(:)'
    @@ -66,14 +71,16 @@
        else if (j == 3) then
           arg_type(i) = trim(arg_type(i)) // ', dimension(:, :, :)'
        endif
    -   write(*, *) '1=intent(in); 2=intent(out); 3=intent(inout)'
    -   read(*, *) j
    -   if(j == 1) then
    -      arg_type(i) = trim(arg_type(i)) // ', intent(in)'
    -   elseif(j == 2) then
    -      arg_type(i) = trim(arg_type(i)) // ', intent(out)'
    -   elseif(j == 3) then
    -      arg_type(i) = trim(arg_type(i)) // ', intent(inout)'
    +   if (i > 0 .or. ispointer) then   ! cannot give intent on pointer args
    +      write(*, *) '1=intent(in); 2=intent(out); 3=intent(inout)'
    +      read(*, *) j
    +      if(j == 1) then
    +         arg_type(i) = trim(arg_type(i)) // ', intent(in)'
    +      elseif(j == 2) then
    +         arg_type(i) = trim(arg_type(i)) // ', intent(out)'
    +      elseif(j == 3) then
    +         arg_type(i) = trim(arg_type(i)) // ', intent(inout)'
    +      endif
        endif
     
        len = len_trim(arg_type(i))
    @@ -85,16 +92,21 @@
        write(22, *) c1(1:max_len) // ' :: ' // trim(arg_name(i)) // ''
     end do
     
    -write(22, *) '
    ' +write(22, *) '' +write(22, *) '
    ' +write(22, *) ' ' +write(22, *) '
    ' +write(22, *) '' -write(22, *) '

    Description

    ' - +write(22, *) ' ' write(22, *) '

    ' -write(22, *) 'input module description here ????????????????????' +write(22, *) 'input function description here ' write(22, *) '

    ' +write(22, *) ' ' ! Write table header -write(22, *) '' +write(22, *) '
    ' +write(22, *) '' ! Write the table of argument descriptions @@ -102,17 +114,27 @@ write(*, *) 'input description of ', trim(arg_name(i)) read(*, 31) c1 31 format(a299) - write(22, *) '' - write(22, *) ' ' + write(22, *) '' + if (i == 0) then + write(22, *) ' ' + else + write(22, *) ' ' + endif + write(22, *) '' end do !End of table +write(22, *) '' write(22, *) '
    ' // trim(arg_name(i)) // & - '    ' // trim(c1) // '
    ' // trim(arg_name(i)) // & + ' Function return value. ' // trim(c1) // '' // trim(c1) // '
    ' -write(22, *) '

    ' +write(22, *) '

    ' +write(22, *) '
    ' +write(22, *) '' write(22, *) '' +write(*,*) 'output file is fort.22 -- move or rename before running program again' + end program html_function ! Added: DART/trunk/doc/html/boilerplate/html_functoc.f90 =================================================================== --- DART/trunk/doc/html/boilerplate/html_functoc.f90 (rev 0) +++ DART/trunk/doc/html/boilerplate/html_functoc.f90 2013-10-08 22:38:57 UTC (rev 6499) @@ -0,0 +1,65 @@ +! DART software - Copyright 2004 - 2013 UCAR. This open source software is +! provided by UCAR, "as is", without chiteme, subject to all terms of use at +! http://www.image.ucar.edu/DAReS/DART/DART_download +! +! $Id$ + +program html_functoc + +implicit none + +! Interactive creation of html for interface list documentation for DART +integer :: num, num_items, i, max_len, len, j +character(len = 299) :: c1, c2, module_name, item_name, spaces + +write(22, *) '' +write(22, *) '' + +spaces = ' ' + +! get the name of the module +write(*, *) 'input the name of this module' +read(*, *) module_name + +! Put in the standard header stuff +write(22, *) '' +write(22, *) '
    [top]

    ' +write(22, *) '

    ' +write(22, *) '' +write(22, *) "" +write(22, *) '' + +write(*, *) 'input the names of the subroutines and functions in this module' +write(*, *) 'input "end" when done' +readloop: do i=1, 10000 + write(*, *) 'next routine name ("end" to finish)' + read(*, *) item_name + if (trim(item_name) == "end") exit readloop + if (i==1) then + write(22, *) ' ' + else + write(22, *) '' + endif +end do readloop + +write(22, *) '
    use ' // trim(module_name), ', only : '//trim(item_name)//'
      '//trim(item_name)//'
    ' + +write(22, *) '' +write(22, *) '

    ' +write(22, *) ' A note about documentation style.' +write(22, *) ' Optional arguments are enclosed in brackets' +write(22, *) ' [like this].' +write(22, *) '

    ' +write(22, *) '' + +write(22, *) '' + +write(*,*) 'output file is fort.22 -- move or rename before running program again' + +end program html_functoc + +! +! $URL$ +! $Id$ +! $Revision$ +! $Date$ Property changes on: DART/trunk/doc/html/boilerplate/html_functoc.f90 ___________________________________________________________________ Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Modified: DART/trunk/doc/html/boilerplate/html_namelist.f90 =================================================================== --- DART/trunk/doc/html/boilerplate/html_namelist.f90 2013-10-08 17:15:24 UTC (rev 6498) +++ DART/trunk/doc/html/boilerplate/html_namelist.f90 2013-10-08 22:38:57 UTC (rev 6499) @@ -8,21 +8,30 @@ implicit none -! Interactive creation of html for function documentation for DART +! Interactive creation of html for namelist documentation for DART integer :: num, num_args, i, max_len, len, j -character(len = 299) :: c1, c2, function_name, function_type, spaces, default_value +character(len = 299) :: c1, c2, function_name, function_type, spaces character(len = 299), allocatable :: arg_name(:), arg_type(:) -write(22, *) '' +write(22, *) '' +write(22, *) '' spaces = ' ' ! Put in the standard header stuff write(22, *) '' -write(22, *) '


    ' -write(22, *) '

    We adhere to the F90 standard of starting a namelist with an ampersand' -write(22, *) "'&' and terminating with a slash '/'." -write(22, *) "

    "
    +write(22, *) '
    [top]

    ' +write(22, *) '

    NAMELIST

    ' +write(22, *) '

    ' +write(22, *) 'This namelist is read from the file input.nml.' +write(22, *) 'Namelists start with an ampersand' +write(22, *) "'&' and terminate with a slash '/'." +write(22, *) "Character strings that contain a '/' must be" +write(22, *) 'enclosed in quotes to prevent them from' +write(22, *) 'prematurely terminating the namelist.' +write(22, *) '

    ' +write(22, *) "
    " +write(22, *) "
    "
     
     ! Now write the namelist sequence
     write(*, *) 'input the number of namelist entries'
    @@ -31,63 +40,62 @@
     write(*, *) 'input the namelist name'
     read(*, *) arg_name(0)
     
    +! Write out the namelist start
    +write(22, *) '&' , trim(arg_name(0))
    +
     ! Loop to read in the argument names
     do i = 1, num_args
    -   write(*, *) 'input namelist entry name ', i
    -   read(*, *) arg_name(i)
    +   write(*, *) 'input namelist entry ', i, ' as name = value '
    +   read(*, 31) c1
    +   read(c1, *) arg_name(i)
    +   write(22, *) '   ' // trim(c1) // ', '
     end do
     
    -c1 = ''
    -do i = 1, num_args
    -   c1 = trim(c1) // trim(arg_name(i))
    -   if(i /= num_args) c1 = trim(c1) // ', '
    -end do
    -
    -! Write out the command line if possible
    -write(22, *) 'namelist / ' , trim(arg_name(0)), ' / '
    -write(22, *) c1
    +write(22, *) '/'
     write(22, *) '
    ' -write(22, *) '

    Discussion

    ' write(22, *) '

    ' -write(22, *) 'put in general discussion ????????' -write(22, *) '' +write(22, *) 'put in any general namelist discussion information' write(22, *) '

    ' +write(22, *) '

    ' -write(22, *) '

    This namelist is read in a file called input.nml' -write(22, *) '

    ' - - ! Prepare the table header -write(22, *) '' -write(22, *) '' -write(22, *) ' ' -write(22, *) ' ' - +write(22, *) '
    ' +write(22, *) '
    Contents Type Description
    ' +write(22, *) '' +write(22, *) '' +write(22, *) ' ' +write(22, *) ' ' +write(22, *) '' +write(22, *) '' +write(22, *) '' do i = 1, num_args write(*, *) 'input the type for ', trim(arg_name(i)) read(*, *) arg_type(i) end do do i = 1, num_args - write(*, *) 'input default value for ', trim(arg_name(i)) - read(*, *) default_value write(*, *) 'input description for ', trim(arg_name(i)) read(*, 31) c1 31 format(a299) - c1 = trim(c1) // ' Default: ' // trim(default_value) - write(22, *) '' - write(22, *) ' ' - write(22, *) '' + write(22, *) '' + write(22, *) ' ' + write(22, *) ' ' + write(22, *) '' end do ! Write the end of the table +write(22, *) '' write(22, *) '
    Item Type Description
    ' // trim(arg_name(i)) // '' // trim(arg_type(i)) // '' // trim(c1) // '
    ' // trim(arg_name(i)) // ' ' // trim(arg_type(i)) // ' ' // trim(c1) // '
    ' +write(22, *) '
    ' -write(22, *) '

    ' +write(22, *) '

    ' + write(22, *) '' +write(*,*) 'output file is fort.22 -- move or rename before running program again' + end program html_namelist ! Modified: DART/trunk/doc/html/boilerplate/html_subroutine.f90 =================================================================== --- DART/trunk/doc/html/boilerplate/html_subroutine.f90 2013-10-08 17:15:24 UTC (rev 6498) +++ DART/trunk/doc/html/boilerplate/html_subroutine.f90 2013-10-08 22:38:57 UTC (rev 6499) @@ -8,12 +8,13 @@ implicit none -! Interactive creation of html for function documentation for DART +! Interactive creation of html for subroutine documentation for DART integer :: num, num_args, i, max_len, len, j character(len = 299) :: c1, c2, function_name, function_type, spaces character(len = 299), allocatable :: arg_name(:), arg_type(:) -write(22, *) '' +write(22, *) '' +write(22, *) '' spaces = ' ' @@ -25,7 +26,7 @@ read(*, *) arg_name(0) write(22, *) '' -write(22, *) '


    ' +write(22, *) '
    ' write(22, *) '
    ' ! Loop to read in the argument names @@ -40,8 +41,8 @@ if(i /= num_args) c1 = trim(c1) // ', ' end do -! Write out the command line if possible -write(22, *) ' call ', trim(arg_name(0)), '(', trim(c1), ') ' +! Write out the command line +write(22, *) ' subroutine ', trim(arg_name(0)), '(', trim(c1), ') ' write(22, *) '
    '
     
     max_len = 0
    @@ -57,7 +58,7 @@
        write(*, *) 'input 1 if pointer'
        read(*, *) j
        if(j == 1) arg_type(i) = trim(arg_type(i)) // ', pointer'
    -   write(*, *) 'input dimesnion (0, 1, 2, 3)'
    +   write(*, *) 'input dimension (0, 1, 2, 3)'
        read(*, *) j
        if(j == 1) then
           arg_type(i) = trim(arg_type(i)) // ', dimension(:)'
    @@ -86,16 +87,21 @@
        write(22, *) c1(1:max_len) // ' :: ' // trim(arg_name(i)) // ''
     end do
     
    -write(22, *) '
    ' +write(22, *) '' +write(22, *) '' +write(22, *) ' ' +write(22, *) '
    ' +write(22, *) '' -write(22, *) '

    Description

    ' - +write(22, *) ' ' write(22, *) '

    ' -write(22, *) 'input module description here ????????????????????' +write(22, *) 'input subroutine description here ' write(22, *) '

    ' +write(22, *) ' ' ! Write table header -write(22, *) '' +write(22, *) '
    ' +write(22, *) '' ! Write the table of argument descriptions @@ -103,17 +109,23 @@ write(*, *) 'input description of ', trim(arg_name(i)) read(*, 31) c1 31 format(a299) - write(22, *) '' - write(22, *) ' ' + write(22, *) '' + write(22, *) ' ' + write(22, *) '' end do !End of table +write(22, *) '' write(22, *) '
    ' // trim(arg_name(i)) // & - '    ' // trim(c1) // '
    ' // trim(arg_name(i)) // & + ' ' // trim(c1) // '
    ' -write(22, *) '

    ' +write(22, *) '

    ' +write(22, *) '
    ' +write(22, *) '' write(22, *) '' +write(*,*) 'output file is fort.22 -- move or rename before running program again' + end program html_subroutine ! Added: DART/trunk/doc/html/boilerplate/html_type.f90 =================================================================== --- DART/trunk/doc/html/boilerplate/html_type.f90 (rev 0) +++ DART/trunk/doc/html/boilerplate/html_type.f90 2013-10-08 22:38:57 UTC (rev 6499) @@ -0,0 +1,96 @@ +! DART software - Copyright 2004 - 2013 UCAR. This open source software is +! provided by UCAR, "as is", without chiteme, subject to all terms of use at +! http://www.image.ucar.edu/DAReS/DART/DART_download +! +! $Id$ + +program html_type + +implicit none + +! Interactive creation of html for derived type documentation for DART +integer :: num, num_items, i, max_len, len, j +character(len = 299) :: c1, c2, type_name, spaces +character(len = 299), allocatable :: item_name(:), item_type(:) + +write(22, *) '' +write(22, *) '' + +spaces = ' ' + +! get the count and name of the type +write(*, *) 'input the number of items in this type' +read(*, *) num_items +allocate(item_name(0:num_items), item_type(0:num_items)) +write(*, *) 'input the derived type name' +read(*, *) item_name(0) +write(*, *) 'enter 1 if contents are private' +read(*, *) j + +! Put in the standard header stuff +write(22, *) '' +write(22, *) '
    ' +write(22, *) '
    '
    +write(22, *) 'type ' // trim(item_name(0)) // ''
    +if (j==1) write(22, *) '   private'
    +
    +! Loop to read in the item names
    +do i = 1, num_items
    +   write(*, *) 'input name for item ', i
    +   read(*, *) item_name(i)
    +   write(*, *) 'input the type for ', trim(item_name(i))
    +   read(*, *) item_type(i)
    +   write(22, *) '   ' // trim(item_type(i)) // ' :: ' // trim(item_name(i))
    +end do
    +
    +write(22, *) 'end type ' // trim(item_name(0))
    +write(22, *) '
    ' +write(22, *) '
    ' + +write(22, *) '' +write(22, *) '
    ' +write(22, *) '' +write(22, *) '' +write(22, *) '

    ' +write(22, *) 'put in any general type discussion information' +write(22, *) '

    ' + +! Prepare the table header +write(22, *) '' +write(22, *) '' +write(22, *) '' +write(22, *) ' ' +write(22, *) '' +write(22, *) '' +write(22, *) '' +write(22, *) '' +do i = 1, num_items + write(*, *) 'input description for ', trim(item_name(i)) + read(*, 31) c1 + 31 format(a299) + + write(22, *) '' + write(22, *) ' ' + write(22, *) '' +end do + + +! Write the end of the table +write(22, *) '' +write(22, *) '
    Component Description
    ' // trim(item_name(i)) // ' ' // trim(c1) // '
    ' +write(22, *) '' +write(22, *) '
    ' +write(22, *) '
    ' +write(22, *) '' + +write(22, *) '' + +write(*,*) 'output file is fort.22 -- move or rename before running program again' + +end program html_type + +! +! $URL$ +! $Id$ +! $Revision$ +! $Date$ Property changes on: DART/trunk/doc/html/boilerplate/html_type.f90 ___________________________________________________________________ Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Modified: DART/trunk/doc/html/boilerplate/template.html =================================================================== --- DART/trunk/doc/html/boilerplate/template.html 2013-10-08 17:15:24 UTC (rev 6498) +++ DART/trunk/doc/html/boilerplate/template.html 2013-10-08 22:38:57 UTC (rev 6499) @@ -18,7 +18,7 @@

    PROGRM OR MODULE name_of_thing

    - +
    DART project logo @@ -56,8 +56,10 @@ + + -
    +
    [top]

    NAMELIST

    DART namelists are @@ -68,40 +70,50 @@ Character strings that contain a '/' must be enclosed in quotes to prevent them from prematurely terminating the namelist.

    +
    -&bob_nml
    -  bob1 = 2
    -  bob3 = 4,
    +&NAMELIST_NML 
    +   name=value,
    +   name=value, 
    +   name=value
     /
     
    -
    - -

    Any comments about the namelist as a whole.

    - - - - +
    +
    - - - +
    +
    Contents Type Description
    bob1integerwhat it does, what valid values are. - Default value is XX
    + + + + + + + + + + + +
    Item Type Description
    nametype(often multi-line) description +
    +
    +
    -
    +
    [top]

    MODULES USED

     types_mod
    @@ -112,39 +124,35 @@
     
    - - -
    +
    [top]

    PUBLIC INTERFACES

    - + - - - - +
    use adaptive_inflate_mod, only : subr/function name
     name2
    + + + + +
    use this_module_name_mod, only : subr/function name
      name2
      name3

    -Optional namelist interface -&bob_nml -may be read from file input.nml. -

    - -

    - A note about documentation style. - Optional arguments are enclosed in brackets + A note about documentation style. + Optional arguments are enclosed in brackets [like this].

    + + -
    +
    - call subroutine1(arg1, [, arg2]) @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Tue Oct 8 16:40:01 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 08 Oct 2013 16:40:01 -0600 Subject: [Dart-dev] [6500] DART/trunk/doc/html/boilerplate/Makefile: make a simple makefile for these programs. Message-ID: Revision: 6500 Author: nancy Date: 2013-10-08 16:40:00 -0600 (Tue, 08 Oct 2013) Log Message: ----------- make a simple makefile for these programs. Added Paths: ----------- DART/trunk/doc/html/boilerplate/Makefile -------------- next part -------------- Added: DART/trunk/doc/html/boilerplate/Makefile =================================================================== --- DART/trunk/doc/html/boilerplate/Makefile (rev 0) +++ DART/trunk/doc/html/boilerplate/Makefile 2013-10-08 22:40:00 UTC (rev 6500) @@ -0,0 +1,25 @@ + +include ../../../mkmf/mkmf.template + +ALL = html_namelist html_function html_subroutine html_type html_functoc + +default: $(ALL) + +html_namelist : html_namelist.f90 + $(FC) $(FFLAGS) $< -o $@ + +html_function: html_function.f90 + $(FC) $(FFLAGS) $< -o $@ + +html_subroutine: html_subroutine.f90 + $(FC) $(FFLAGS) $< -o $@ + +html_type: html_type.f90 + $(FC) $(FFLAGS) $< -o $@ + +html_functoc: html_functoc.f90 + $(FC) $(FFLAGS) $< -o $@ + + +clean: + rm -f $(ALL) fort.22 Property changes on: DART/trunk/doc/html/boilerplate/Makefile ___________________________________________________________________ Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native From nancy at ucar.edu Wed Oct 9 14:42:12 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 14:42:12 -0600 Subject: [Dart-dev] [6502] DART/trunk/models/CESM/shell_scripts: make all the time tests in the assimilate scripts be a positive Message-ID: Revision: 6502 Author: nancy Date: 2013-10-09 14:42:12 -0600 (Wed, 09 Oct 2013) Log Message: ----------- make all the time tests in the assimilate scripts be a positive test, and return in the else clause instead of a negative test. add a perfect_model script which is modeled on the assimilate script. it calls 3 perfect_model scripts which are to be written. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh Added Paths: ----------- DART/trunk/models/CESM/shell_scripts/perfect_model.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh 2013-10-09 20:29:18 UTC (rev 6501) +++ DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh 2013-10-09 20:42:12 UTC (rev 6502) @@ -9,9 +9,10 @@ # This block is an attempt to localize all the machine-specific # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# assimilate. echo "`date` -- BEGIN CAM_ASSIMILATE" -echo "custom version assimilates at 0,6,12,18Z" set nonomatch # suppress "rm" warnings if wildcard does not match anything @@ -82,12 +83,13 @@ # If not, return before assimilating. #------------------------------------------------------------------------- -if ( $ATM_HOUR != 0 && $ATM_HOUR != 6 && $ATM_HOUR != 12 && $ATM_HOUR != 18) then +## TIMECHECK: +if ( $ATM_HOUR == 0 || $ATM_HOUR == 6 || $ATM_HOUR == 12 || $ATM_HOUR == 18) then + echo "Hour is $ATM_HOUR so we are assimilating the atmosphere" +else echo "Hour is not 0,6,12 or 18Z so we are skipping the atmosphere assimilation" echo "`date` -- END CAM_ASSIMILATE" exit 0 -else - echo "Hour is $ATM_HOUR so we are assimilating the atmosphere" endif #------------------------------------------------------------------------- Modified: DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh 2013-10-09 20:29:18 UTC (rev 6501) +++ DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh 2013-10-09 20:42:12 UTC (rev 6502) @@ -9,9 +9,10 @@ # This block is an attempt to localize all the machine-specific # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# assimilate. echo "`date` -- BEGIN CLM_ASSIMILATE" -echo "this version assimilates only when hour is 0Z" set nonomatch # suppress "rm" warnings if wildcard does not match anything @@ -82,21 +83,21 @@ # If not, return before assimilating. #------------------------------------------------------------------------- -if ( $LND_HOUR != 0 && $LND_HOUR != 6 && $LND_HOUR != 12 && $LND_HOUR != 18) then +## TIMECHECK: +if ( $LND_HOUR == 0 || $LND_HOUR == 6 || $LND_HOUR == 12 || $LND_HOUR == 18 ) then + echo "Hour is $LND_HOUR so we are assimilating the land" +else echo "Hour is not 0,6,12 or 18Z so we are skipping the land assimilation" echo "`date` -- END CLM_ASSIMILATE" exit 0 -else - echo "Hour is $LND_HOUR so we are assimilating the land" endif -# if you want to only assimilate at 0Z each day, substitute the -# following two lines for the longer test above. we have flux tower -# observations which need to be assimilated every 6 hours; if you -# aren't going to assimilate them and are only assimilating some obs type -# that is only available once a day, you can change to this test instead. -#if ( $LND_HOUR != 0 ) then -# echo "Hour is not 0Z so we are skipping the land assimilation" +# we have flux tower observations which can be assimilated every 6 hours, +# but if you want to only assimilate at 0Z each day, change the LND_HOUR +# check to this one: +# if ( $LND_HOUR == 0 ) then +# and the message in the else clause to this: +# echo "Hour is not 0Z so we are skipping the land assimilation" #------------------------------------------------------------------------- # Create temporary working directory for the assimilation and go there Added: DART/trunk/models/CESM/shell_scripts/perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/perfect_model.csh (rev 0) +++ DART/trunk/models/CESM/shell_scripts/perfect_model.csh 2013-10-09 20:42:12 UTC (rev 6502) @@ -0,0 +1,30 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. + +echo "`date` -- BEGIN CESM PERFECT MODEL" + +${CASEROOT}/cam_perfect_model.csh +if ( $status != 0 ) exit $status +${CASEROOT}/pop_perfect_model.csh +if ( $status != 0 ) exit $status +${CASEROOT}/clm_perfect_model.csh +if ( $status != 0 ) exit $status + +echo "`date` -- END CESM PERFECT MODEL" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/CESM/shell_scripts/perfect_model.csh ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Modified: DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh 2013-10-09 20:29:18 UTC (rev 6501) +++ DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh 2013-10-09 20:42:12 UTC (rev 6502) @@ -9,9 +9,10 @@ # This block is an attempt to localize all the machine-specific # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# assimilate. echo "`date` -- BEGIN POP_ASSIMILATE" -echo "this version assimilates only when hour is 0Z" set nonomatch # suppress "rm" warnings if wildcard does not match anything @@ -84,8 +85,15 @@ # restart configuration for pop (data_assim or rest). #------------------------------------------------------------------------- +## TIMECHECK: ${REMOVE} ${CASEROOT}/user_nl_pop2_*back -if ( $OCN_HOUR != 0 ) then +if ( $OCN_HOUR == 0 ) then + echo "Hour is $OCN_HOUR so we are assimilating the ocean" + foreach nml ( ${CASEROOT}/user_nl_pop2_* ) + ${MOVE} $nml ${nml}.back + sed -e "s/'rest'/'data_assim'/" ${nml}.back >! $nml + end +else echo "Hour is not 0Z so we are skipping the ocean assimilation" foreach nml ( ${CASEROOT}/user_nl_pop2_* ) ${MOVE} $nml ${nml}.back @@ -93,12 +101,6 @@ end echo "`date` -- END POP_ASSIMILATE" exit 0 -else - echo "Hour is $OCN_HOUR so we are assimilating the ocean" - foreach nml ( ${CASEROOT}/user_nl_pop2_* ) - ${MOVE} $nml ${nml}.back - sed -e "s/'rest'/'data_assim'/" ${nml}.back >! $nml - end endif #------------------------------------------------------------------------- From nancy at ucar.edu Wed Oct 9 15:08:32 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 15:08:32 -0600 Subject: [Dart-dev] [6503] DART/trunk/models/CESM/shell_scripts: minor tweek to cam_assimilate to be consistent with Message-ID: Revision: 6503 Author: nancy Date: 2013-10-09 15:08:31 -0600 (Wed, 09 Oct 2013) Log Message: ----------- minor tweek to cam_assimilate to be consistent with how we generally form instance numbers. also add tim's script which must be called when we stop CESM but aren't doing a cam assimilation. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh Added Paths: ----------- DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh 2013-10-09 20:42:12 UTC (rev 6502) +++ DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh 2013-10-09 21:08:31 UTC (rev 6503) @@ -512,11 +512,11 @@ set member = 1 while ( ${member} <= ${ensemble_size} ) - set n4 = `printf %04d $member` + set inst_string = `printf _%04d $member` - set ATM_INITIAL_FILENAME = `printf ${MYCASE}.cam_%04d.i.${ATM_DATE_EXT}.nc ${member}` + set ATM_INITIAL_FILENAME = ${MYCASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc - ${LINK} ${ATM_INITIAL_FILENAME} cam_initial_${n4}.nc || exit -9 + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 @ member++ Added: DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh (rev 0) +++ DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh 2013-10-09 21:08:31 UTC (rev 6503) @@ -0,0 +1,71 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. + +echo "`date` -- BEGIN CAM_NO_ASSIMILATE" + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set LINK = '/usr/local/bin/ln -fvs' + breaksw + + default: + set LINK = 'ln -fvs' + breaksw +endsw + +set ensemble_size = ${NINST_ATM} + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name of first member +# of the form "./${CASE}.cam_${ensemble_member}.i.2000-01-06-00000.nc" +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.atm_0001` +set FILE = $FILE:t +set FILE = $FILE:r +set MYCASE = `echo $FILE | sed -e "s#\..*##"` +set ATM_DATE_EXT = `echo $FILE:e` + +#========================================================================= +# As implemented, the input filenames are static in the CESM namelists. +# We must link the new uniquely-named files to static names so that when +# the short-term archiver 'restores' the CESM files, the links are right. +#========================================================================= + +cd ${RUNDIR} + +set member = 1 +while ( ${member} <= ${ensemble_size} ) + + set inst_string = `printf _%04d $member` + + set ATM_INITIAL_FILENAME = ${MYCASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc + + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 + + @ member++ + +end + +echo "`date` -- END CAM_NO_ASSIMILATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native From nancy at ucar.edu Wed Oct 9 15:10:49 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 15:10:49 -0600 Subject: [Dart-dev] [6504] DART/trunk/models/CESM/shell_scripts/assimilate.csh: add note that if you are skipping the CAM assimilation for any Message-ID: Revision: 6504 Author: nancy Date: 2013-10-09 15:10:49 -0600 (Wed, 09 Oct 2013) Log Message: ----------- add note that if you are skipping the CAM assimilation for any reason, you still have to call a script to link the right initial files for each ensemble member. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/assimilate.csh 2013-10-09 21:08:31 UTC (rev 6503) +++ DART/trunk/models/CESM/shell_scripts/assimilate.csh 2013-10-09 21:10:49 UTC (rev 6504) @@ -10,6 +10,11 @@ # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. +# if you are skipping the POP or CLM assimilations you can simply +# comment out the calls to the xxx_assimilate.csh scripts. however +# if you are skipping the CAM assimilation you must replace the +# call below with a call to ${CASEROOT}/cam_no_assimilate.csh + echo "`date` -- BEGIN CESM ASSIMILATE" ${CASEROOT}/cam_assimilate.csh From nancy at ucar.edu Wed Oct 9 15:20:06 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 15:20:06 -0600 Subject: [Dart-dev] [6505] DART/trunk/models/cam/shell_scripts/assimilate.csh: this script no longer needs the DARTDIR for anything; Message-ID: Revision: 6505 Author: nancy Date: 2013-10-09 15:20:06 -0600 (Wed, 09 Oct 2013) Log Message: ----------- this script no longer needs the DARTDIR for anything; the sampling error correction file is copied over to the CASEROOT dir by the CESM_DART_config script and that was the final reference to the dart source code tree. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/assimilate.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/assimilate.csh 2013-10-09 21:10:49 UTC (rev 6504) +++ DART/trunk/models/cam/shell_scripts/assimilate.csh 2013-10-09 21:20:06 UTC (rev 6505) @@ -26,7 +26,6 @@ set REMOVE = '/usr/local/bin/rm -fr' set BASEOBSDIR = /glade/proj3/image/Observations/ACARS - set DARTDIR = ${HOME}/svn/DART/trunk set LAUNCHCMD = mpirun.lsf breaksw @@ -38,7 +37,6 @@ set REMOVE = 'rm -fr' set BASEOBSDIR = /glade/p/image/Observations/ACARS - set DARTDIR = ${HOME}/svn/DART/trunk set LAUNCHCMD = mpirun.lsf breaksw @@ -50,7 +48,6 @@ set REMOVE = 'rm -fr' set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS - set DARTDIR = ${HOME}/trunk set LAUNCHCMD = "aprun -n $NTASKS" breaksw endsw @@ -142,7 +139,7 @@ set SECSTRING = `echo $MYSTRING[2] | tr '[:upper:]' '[:lower:]'` if ( $SECSTRING == true ) then - set SAMP_ERR_FILE = ${DARTDIR}/system_simulation/final_full_precomputed_tables/final_full.${ensemble_size} + set SAMP_ERR_FILE = ${CASEROOT}/final_full.${ensemble_size} if ( -e ${SAMP_ERR_FILE} ) then ${COPY} ${SAMP_ERR_FILE} . else From nancy at ucar.edu Wed Oct 9 16:18:51 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 16:18:51 -0600 Subject: [Dart-dev] [6506] DART/trunk/models/CESM/shell_scripts: first stab at perfect model scripts when running the fully coupled Message-ID: Revision: 6506 Author: nancy Date: 2013-10-09 16:18:50 -0600 (Wed, 09 Oct 2013) Log Message: ----------- first stab at perfect model scripts when running the fully coupled CESM model. untested as of yet. Added Paths: ----------- DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh -------------- next part -------------- Added: DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh (rev 0) +++ DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh 2013-10-09 22:18:50 UTC (rev 6506) @@ -0,0 +1,256 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# run. + +echo "`date` -- BEGIN GENERATE CAM TRUE STATE" + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + set BASEOBSDIR = /glade/proj3/image/Observations/ACARS + breaksw + + case ys*: + # NCAR "yellowstone" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /glade/p/image/Observations/ACARS + breaksw + + default: + # NERSC "hopper" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS + breaksw +endsw + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name of first member +# of the form "./${CASE}.cam_${ensemble_member}.i.2000-01-06-00000.nc" +# +# Piping stuff through 'bc' strips off any preceeding zeros. +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.atm_0001` +set FILE = $FILE:t +set FILE = $FILE:r +set ATM_DATE_EXT = `echo $FILE:e` +set ATM_DATE = `echo $FILE:e | sed -e "s#-# #g"` +set ATM_YEAR = `echo $ATM_DATE[1] | bc` +set ATM_MONTH = `echo $ATM_DATE[2] | bc` +set ATM_DAY = `echo $ATM_DATE[3] | bc` +set ATM_SECONDS = `echo $ATM_DATE[4] | bc` +set ATM_HOUR = `echo $ATM_DATE[4] / 3600 | bc` + +echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_SECONDS (seconds)" +echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_HOUR (hours)" + +#------------------------------------------------------------------------- +# Determine if current time is a perfect model time. +# If not, return before doing anything. +#------------------------------------------------------------------------- + +## TIMECHECK: +if ( $ATM_HOUR == 0 || $ATM_HOUR == 6 || $ATM_HOUR == 12 || $ATM_HOUR == 18) then + echo "Hour is $ATM_HOUR so we are generating perfect obs for the atmosphere" +else + echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the atmosphere" + echo "`date` -- END CAM_ASSIMILATE" + exit 0 +endif + +#------------------------------------------------------------------------- +# Create temporary working directory for the perfect model and go there +#------------------------------------------------------------------------- + +set temp_dir = pmo_cam +echo "temp_dir is $temp_dir" + +if ( -d $temp_dir ) then + ${REMOVE} $temp_dir/* +else + mkdir -p $temp_dir +endif +cd $temp_dir + +#----------------------------------------------------------------------------- +# Get observation sequence file ... or die right away. +# The observation file names have a time that matches the stopping time of CAM. +#----------------------------------------------------------------------------- + +set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` +set OBSFNAME = `printf obs_seq%04d%02d%02d%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY} ${ATM_HOUR}` +set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}_6H/${OBSFNAME} + +if ( -e ${OBS_FILE} ) then + ${LINK} ${OBS_FILE} obs_seq.in +else + echo "ERROR ... no observation file $OBS_FILE" + echo "ERROR ... no observation file $OBS_FILE" + exit -1 +endif + +#========================================================================= +# Block 1: Populate a run-time directory with the input needed to run DART. +#========================================================================= + +echo "`date` -- BEGIN COPY BLOCK" + +if ( -e ${CASEROOT}/cam_input.nml ) then + ${COPY} ${CASEROOT}/cam_input.nml input.nml +else + echo "ERROR ... DART required file ${CASEROOT}/cam_input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/cam_input.nml not found ... ERROR" + exit -2 +endif + +echo "`date` -- END COPY BLOCK" + +#========================================================================= +# Block 2: Convert 1 CAM restart file to DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from pointer file ../rpointer.atm_0001 +# +# REQUIRED DART namelist settings: +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &cam_to_dart_nml: cam_to_dart_output_file = 'dart_ics' +#========================================================================= + +echo "`date` -- BEGIN CAM-TO-DART" + +set member = 1 + + set MYTEMPDIR = member_${member} + mkdir -p $MYTEMPDIR + cd $MYTEMPDIR + + # Turns out the .h0. files are timestamped with the START of the + # run, which is *not* ATM_DATE_EXT ... I just link to a whatever + # is convenient (since the info is static). + # make sure there are no old output logs hanging around + $REMOVE output.${member}.cam_to_dart + + set ATM_INITIAL_FILENAME = "../../${CASE}.cam.i.${ATM_DATE_EXT}.nc" + set ATM_HISTORY_FILENAME = `ls -1t ../../${CASE}.cam*.h0.* | head -n 1` + set DART_IC_FILENAME = `printf filter_ics.%04d ${member}` + set DART_RESTART_FILE = `printf filter_restart.%04d ${member}` + + sed -e "s#dart_ics#../${DART_IC_FILENAME}#" \ + -e "s#dart_restart#../${DART_RESTART_FILE}#" < ../input.nml >! input.nml + + ${LINK} $ATM_INITIAL_FILENAME caminput.nc + ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc + + ${EXEROOT}/cam_to_dart >! output.${member}.cam_to_dart + + if ($status != 0) then + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + exit -6 +endif + +echo "`date` -- END CAM-TO-DART" + +#========================================================================= +# Block 3: Advance the model and harvest the synthetic observations. +# Will result in a single file : 'perfect_restart' which we don't need +# for a perfect model experiment with CESM. +# +# DART namelist settings required: +# &perfect_model_obs_nml: async = 0, +# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' +# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' +# &perfect_model_obs_nml: init_time_days = -1, +# &perfect_model_obs_nml: init_time_seconds = -1, +# &perfect_model_obs_nml: first_obs_days = -1, +# &perfect_model_obs_nml: first_obs_seconds = -1, +# &perfect_model_obs_nml: last_obs_days = -1, +# &perfect_model_obs_nml: last_obs_seconds = -1, +# +#========================================================================= + +# CAM:static_init_model() always needs a caminput.nc and a cam_phis.nc +# for geometry information, etc. + +set ATM_INITIAL_FILENAME = ../${CASE}.cam.i.${ATM_DATE_EXT}.nc +set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` + +${LINK} $ATM_INITIAL_FILENAME caminput.nc +${LINK} $ATM_HISTORY_FILENAME cam_phis.nc + +echo "`date` -- BEGIN CAM PERFECT_MODEL_OBS" +${LAUNCHCMD} ${EXEROOT}/perfect_model_obs_cam || exit -7 +echo "`date` -- END CAM PERFECT_MODEL_OBS" + +${MOVE} Prior_Diag.nc ../cam_Prior_Diag.${ATM_DATE_EXT}.nc +${MOVE} Posterior_Diag.nc ../cam_Posterior_Diag.${ATM_DATE_EXT}.nc +${MOVE} obs_seq.final ../cam_obs_seq.${ATM_DATE_EXT}.final +${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out + +#========================================================================= +# Block 4: Update the cam restart file +#========================================================================= + +# not needed ... perfect_model_obs does not update the model state. + +#========================================================================= +# Block 5: Link the next cam file to the static name needed here. +#========================================================================= + +cd ${RUNDIR} + + set ATM_INITIAL_FILENAME = ${CASE}.cam.i.${ATM_DATE_EXT}.nc + + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial.nc || exit -9 + +end + +#------------------------------------------------------------------------- +# Cleanup +# we (DART) do not need these files, and CESM does not need them either +# to continue a run. if we remove them here they do not get moved to +# the short-term archiver. +#------------------------------------------------------------------------- + +# ${REMOVE} ${RUNDIR}/*.rh0.* +# ${REMOVE} ${RUNDIR}/*.rs1.* + +echo "`date` -- END GENERATE CAM TRUE STATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Added: DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh (rev 0) +++ DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh 2013-10-09 22:18:50 UTC (rev 6506) @@ -0,0 +1,290 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. + +echo "`date` -- BEGIN GENERATE CLM TRUE STATE" + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + set BASEOBSDIR = /glade/proj3/image/Observations/FluxTower + breaksw + + case ys*: + # NCAR "yellowstone" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /glade/p/image/Observations/land + breaksw + + case lone*: + # UT lonestar + set MOVE = '/bin/mv -fv' + set COPY = '/bin/cp -fv --preserve=timestamps' + set LINK = '/bin/ln -fvs' + set REMOVE = '/bin/rm -fr' + + set BASEOBSDIR = ${WORK}/DART/observations/snow/work/obs_seqs + breaksw + + case la*: + # LBNL "lawrencium" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /your/observation/directory/here + breaksw + + default: + # NERSC "hopper" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS + breaksw +endsw + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name of first member +# of the form "./${CASE}.clm2.r.2000-01-06-00000.nc" +# +# Piping stuff through 'bc' strips off any preceeding zeros. +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.lnd` +set FILE = $FILE:r +set LND_DATE_EXT = `echo $FILE:e` +set LND_DATE = `echo $FILE:e | sed -e "s#-# #g"` +set LND_YEAR = `echo $LND_DATE[1] | bc` +set LND_MONTH = `echo $LND_DATE[2] | bc` +set LND_DAY = `echo $LND_DATE[3] | bc` +set LND_SECONDS = `echo $LND_DATE[4] | bc` +set LND_HOUR = `echo $LND_DATE[4] / 3600 | bc` + +echo "valid time of model is $LND_YEAR $LND_MONTH $LND_DAY $LND_SECONDS (seconds)" +echo "valid time of model is $LND_YEAR $LND_MONTH $LND_DAY $LND_HOUR (hours)" + +#------------------------------------------------------------------------- +# Determine if current time is an perfect model time. +# If not, return before doing anything. +#------------------------------------------------------------------------- + +## TIMECHECK: +if ( $LND_HOUR == 0 || $LND_HOUR == 6 || $LND_HOUR == 12 || $LND_HOUR == 18 ) then + echo "Hour is $LND_HOUR so we are generating perfect obs for the land" +else + echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the land" + echo "`date` -- END GENERATE TRUE STATE" + exit 0 +endif + +# we have flux tower observations which can be processed every 6 hours, +# but if you want to only processed at 0Z each day, change the LND_HOUR +# check to this one: +# if ( $LND_HOUR == 0 ) then +# and the message in the else clause to this: +# echo "Hour is not 0Z so we are skipping generating perfect obs for the land" + +#------------------------------------------------------------------------- +# Create temporary working directory for the perfect model and go there +#------------------------------------------------------------------------- + +set temp_dir = pmo_clm +echo "temp_dir is $temp_dir" + +if ( -d $temp_dir ) then + ${REMOVE} $temp_dir/* +else + mkdir -p $temp_dir +endif +cd $temp_dir + +#----------------------------------------------------------------------------- +# Get observation sequence file ... or die right away. +# The observation file names have a time that matches the stopping time of CLM. +# +# The CLM observations are stowed in two sets of directories. +# If you are stopping every 24 hours or more, the obs are in directories like YYYYMM. +# In all other situations the observations come from directories like YYYYMM_6H. +# The only ugly part here is if the first advance and subsequent advances are +# not the same length. The observations _may_ come from different directories. +# +# The contents of the file must match the history file contents if one is using +# the obs_def_tower_mod or could be the 'traditional' +/- 12Z ... or both. +# Since the history file contains the previous days' history ... so must the obs file. +#----------------------------------------------------------------------------- + +if ($STOP_N >= 24) then + set OBSDIR = `printf %04d%02d ${LND_YEAR} ${LND_MONTH}` +else + set OBSDIR = `printf %04d%02d_6H ${LND_YEAR} ${LND_MONTH}` +endif + +set OBS_FILE = ${BASEOBSDIR}/${OBSDIR}/obs_seq.${LND_DATE_EXT} + +if ( -e ${OBS_FILE} ) then + ${COPY} ${OBS_FILE} obs_seq.in +else + echo "ERROR ... no observation file $OBS_FILE" + echo "ERROR ... no observation file $OBS_FILE" + exit -1 +endif + +#========================================================================= +# Block 1: Populate a run-time directory with the input needed to run DART. +#========================================================================= + +echo "`date` -- BEGIN COPY BLOCK" + +if ( -e ${CASEROOT}/clm_input.nml ) then + ${COPY} ${CASEROOT}/clm_input.nml input.nml +else + echo "ERROR ... DART required file ${CASEROOT}/clm_input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/clm_input.nml not found ... ERROR" + exit -2 +endif + +echo "`date` -- END COPY BLOCK" + +#========================================================================= +# Block 2: convert 1 clm restart file to a DART initial conditions file. +# At the end of the block, we have a DART restart file perfect_ics +# that came from the pointer file ../rpointer.lnd_0001 +# +# DART namelist settings appropriate/required: +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &clm_to_dart_nml: clm_to_dart_output_file = 'dart_ics' +#========================================================================= + +echo "`date` -- BEGIN CLM-TO-DART" + +set member = 1 + + set MYTEMPDIR = member_${member} + mkdir -p $MYTEMPDIR + cd $MYTEMPDIR + + # make sure there are no old output logs hanging around + $REMOVE output.${member}.clm_to_dart + + set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc + set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc + set OBS1_HISTORY_FILENAME = ${CASE}.clm2.h1.${LND_DATE_EXT}.nc + set OBS2_HISTORY_FILENAME = ${CASE}.clm2_0001.h1.${LND_DATE_EXT}.nc + set DART_IC_FILENAME = perfect_ics + + sed -e "s/dart_ics/..\/${DART_IC_FILENAME}/" < ../input.nml >! input.nml + + ${LINK} ../../$LND_RESTART_FILENAME clm_restart.nc + ${LINK} ../../$LND_HISTORY_FILENAME clm_history.nc + + if (-e $OBS1_HISTORY_FILENAME) then + ${LINK} ../../$OBS1_HISTORY_FILENAME $OBS2_HISTORY_FILENAME + endif + + # patch the CLM restart files to ensure they have the proper + # _FillValue and missing_value attributes. +# ncatted -O -a _FillValue,frac_sno,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a missing_value,frac_sno,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a _FillValue,DZSNO,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a missing_value,DZSNO,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a _FillValue,H2OSOI_LIQ,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a missing_value,H2OSOI_LIQ,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a _FillValue,H2OSOI_ICE,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a missing_value,H2OSOI_ICE,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a _FillValue,T_SOISNO,o,d,1.0e+36 clm_restart.nc +# ncatted -O -a missing_value,T_SOISNO,o,d,1.0e+36 clm_restart.nc + + ${EXEROOT}/clm_to_dart >! output.${member}.clm_to_dart + + if ($status != 0) then + echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" + echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" + exit -3 + endif + + cd .. + +echo "`date` -- END CLM-TO-DART" + +#========================================================================= +# Block 3: Advance the model and harvest the synthetic observations. +# Will result in a single file : 'perfect_restart' which we don't need +# for a perfect model experiment with CESM. +# +# DART namelist settings required: +# &perfect_model_obs_nml: async = 0, +# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' +# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' +# &perfect_model_obs_nml: init_time_days = -1, +# &perfect_model_obs_nml: init_time_seconds = -1, +# &perfect_model_obs_nml: first_obs_days = -1, +# &perfect_model_obs_nml: first_obs_seconds = -1, +# &perfect_model_obs_nml: last_obs_days = -1, +# &perfect_model_obs_nml: last_obs_seconds = -1, +# +#========================================================================= + +# clm always needs a clm_restart.nc, clm_history.nc for geometry information, etc. + +set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc +set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc + +${LINK} ../$LND_RESTART_FILENAME clm_restart.nc +${LINK} ../$LND_HISTORY_FILENAME clm_history.nc + +echo "`date` -- BEGIN CLM PERFECT_MODEL_OBS" +${EXEROOT}/perfect_model_obs_clm || exit -4 +echo "`date` -- END CLM PERFECT_MODEL_OBS" + +${MOVE} True_State.nc ../clm_True_State.${LND_DATE_EXT}.nc +${MOVE} obs_seq.out ../clm_obs_seq.${LND_DATE_EXT}.out +${MOVE} dart_log.out ../clm_dart_log.${LND_DATE_EXT}.out + +#========================================================================= +# Block 4: Update the clm restart files. +#========================================================================= + +# not needed ... perfect_model_obs does not update the model state. + +#------------------------------------------------------------------------- +# Cleanup +#------------------------------------------------------------------------- + +echo "`date` -- END GENERATE CLM TRUE STATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Copied: DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh (from rev 6501, DART/trunk/models/CESM/shell_scripts/run_perfect_model_obs.csh) =================================================================== --- DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh (rev 0) +++ DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-09 22:18:50 UTC (rev 6506) @@ -0,0 +1,237 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# evaluate obs. + +echo "`date` -- BEGIN GENERATE POP TRUE STATE" + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + set BASEOBSDIR = /glade/proj3/image/Observations/WOD09 + breaksw + + case ys*: + # NCAR "yellowstone" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /glade/p/image/Observations/WOD09 + breaksw + + default: + # NERSC "hopper" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS + breaksw +endsw + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name of first member +# of the form "./${CASE}.pop.r.2000-01-06-00000.nc" +# +# Piping stuff through 'bc' strips off any preceeding zeros. +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.ocn.restart` +set FILE = $FILE:t +set FILE = $FILE:r +set OCN_DATE_EXT = `echo $FILE:e` +set OCN_DATE = `echo $FILE:e | sed -e "s#-# #g"` +set OCN_YEAR = `echo $OCN_DATE[1] | bc` +set OCN_MONTH = `echo $OCN_DATE[2] | bc` +set OCN_DAY = `echo $OCN_DATE[3] | bc` +set OCN_SECONDS = `echo $OCN_DATE[4] | bc` +set OCN_HOUR = `echo $OCN_DATE[4] / 3600 | bc` + +echo "valid time of model is $OCN_YEAR $OCN_MONTH $OCN_DAY $OCN_SECONDS (seconds)" +echo "valid time of model is $OCN_YEAR $OCN_MONTH $OCN_DAY $OCN_HOUR (hours)" + +#------------------------------------------------------------------------- +# Determine if current time is 0Z - if so, generate perfect obs. +# If not, return before doing anything. +#------------------------------------------------------------------------- + +## TIMECHECK: +if ( $OCN_HOUR == 0 ) then + echo "Hour is $OCN_HOUR so we are generating perfect obs for the ocean" +else + echo "Hour is not 0Z so we are skipping generating perfect obs for the ocean" + echo "`date` -- END GENERATE TRUE STATE" + exit 0 +endif + +#------------------------------------------------------------------------- +# Create temporary working directory for the perfect model and go there +#------------------------------------------------------------------------- + +set temp_dir = pmo_pop +echo "temp_dir is $temp_dir" + +if ( -d $temp_dir ) then + ${REMOVE} $temp_dir/* +else + mkdir -p $temp_dir +endif +cd $temp_dir + +#----------------------------------------------------------------------------- +# Get observation sequence file ... or die right away. +# The observation file names have a time that matches the stopping time of POP. +#----------------------------------------------------------------------------- + +set YYYYMM = `printf %04d%02d ${OCN_YEAR} ${OCN_MONTH}` +set OBSFNAME = `printf obs_seq.0Z.%04d%02d%02d ${OCN_YEAR} ${OCN_MONTH} ${OCN_DAY}` +set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}/${OBSFNAME} + +if ( -e ${OBS_FILE} ) then + ${COPY} ${OBS_FILE} obs_seq.in +else + echo "ERROR ... no observation file $OBS_FILE" + echo "ERROR ... no observation file $OBS_FILE" + exit -1 +endif + +#========================================================================= +# Block 1: Populate a run-time directory with the input needed to run DART. +#========================================================================= + +echo "`date` -- BEGIN COPY BLOCK" + +if ( -e ${CASEROOT}/input.nml ) then + ${COPY} ${CASEROOT}/input.nml . +else + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + exit -2 +endif + +echo "`date` -- END COPY BLOCK" + +# Eat the cookie regardless +${REMOVE} ../pop_inflation_cookie + +#========================================================================= +# Block 2: convert 1 pop restart file to a DART initial conditions file. +# At the end of the block, we have a DART restart file perfect_ics +# that came from the pointer file ../rpointer.ocn.restart +# +# DART namelist settings appropriate/required: +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &pop_to_dart_nml: pop_to_dart_output_file = 'dart_ics' +#========================================================================= + +echo "`date` -- BEGIN POP-TO-DART" + +set member = 1 + + set MYTEMPDIR = member_${member} + mkdir -p $MYTEMPDIR + cd $MYTEMPDIR + + # make sure there are no old output logs hanging around + $REMOVE output.${member}.pop_to_dart + + set OCN_RESTART_FILENAME = ${CASE}.pop.r.${OCN_DATE_EXT}.nc + set OCN_NML_FILENAME = pop2_in + set DART_IC_FILENAME = perfect_ics + set DART_RESTART_FILE = perfect_restart + + sed -e "s#dart_ics#../${DART_IC_FILENAME}#" \ + -e "s#dart_restart#../${DART_RESTART_FILE}#" < ../input.nml >! input.nml + + ${LINK} ../../$OCN_RESTART_FILENAME pop.r.nc + ${LINK} ../../$OCN_NML_FILENAME pop_in + + ${EXEROOT}/pop_to_dart >! output.${member}.pop_to_dart + + if ($status != 0) then + echo "ERROR ... DART died in 'pop_to_dart' ... ERROR" + echo "ERROR ... DART died in 'pop_to_dart' ... ERROR" + exit -3 + endif + + cd .. + +echo "`date` -- END POP-TO-DART" + +#========================================================================= +# Block 3: Advance the model and harvest the synthetic observations. +# Will result in a single file : 'perfect_restart' which we don't need +# for a perfect model experiment with CESM. +# +# DART namelist settings required: +# &perfect_model_obs_nml: async = 0, +# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' +# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' +# &perfect_model_obs_nml: init_time_days = -1, +# &perfect_model_obs_nml: init_time_seconds = -1, +# &perfect_model_obs_nml: first_obs_days = -1, +# &perfect_model_obs_nml: first_obs_seconds = -1, +# &perfect_model_obs_nml: last_obs_days = -1, +# &perfect_model_obs_nml: last_obs_seconds = -1, +# +#========================================================================= + +# POP always needs a pop_in and a pop.r.nc to start. + +set OCN_RESTART_FILENAME = ${CASE}.pop.r.${OCN_DATE_EXT}.nc + +${LINK} ../$OCN_RESTART_FILENAME pop.r.nc +${LINK} ../pop2_in pop_in + +echo "`date` -- BEGIN PERFECT_MODEL_OBS" +${EXEROOT}/perfect_model_obs || exit -4 +echo "`date` -- END PERFECT_MODEL_OBS" + +${MOVE} True_State.nc ../pop_True_State.${OCN_DATE_EXT}.nc +${MOVE} obs_seq.out ../pop_obs_seq.${OCN_DATE_EXT}.out +${MOVE} dart_log.out ../pop_dart_log.${OCN_DATE_EXT}.out + +#========================================================================= +# Block 4: Update the pop restart files. +#========================================================================= + +# not needed ... perfect_model_obs does not update the model state. + +#------------------------------------------------------------------------- +# Cleanup +#------------------------------------------------------------------------- + +echo "`date` -- END GENERATE POP TRUE STATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + From nancy at ucar.edu Wed Oct 9 16:31:00 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 16:31:00 -0600 Subject: [Dart-dev] [6507] DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh: no need for a separate MYCASE since we're called from Message-ID: Revision: 6507 Author: nancy Date: 2013-10-09 16:31:00 -0600 (Wed, 09 Oct 2013) Log Message: ----------- no need for a separate MYCASE since we're called from the CESM scripts which already define CASE for is. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh 2013-10-09 22:18:50 UTC (rev 6506) +++ DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh 2013-10-09 22:31:00 UTC (rev 6507) @@ -36,7 +36,6 @@ set FILE = `head -n 1 rpointer.atm_0001` set FILE = $FILE:t set FILE = $FILE:r -set MYCASE = `echo $FILE | sed -e "s#\..*##"` set ATM_DATE_EXT = `echo $FILE:e` #========================================================================= @@ -52,7 +51,7 @@ set inst_string = `printf _%04d $member` - set ATM_INITIAL_FILENAME = ${MYCASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc + set ATM_INITIAL_FILENAME = ${CASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 From nancy at ucar.edu Wed Oct 9 16:37:21 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 09 Oct 2013 16:37:21 -0600 Subject: [Dart-dev] [6508] DART/trunk/ensemble_manager/ensemble_manager_mod.html: change
    to
    to make html checker happy. Message-ID: Revision: 6508 Author: nancy Date: 2013-10-09 16:37:21 -0600 (Wed, 09 Oct 2013) Log Message: ----------- change
    to
    to make html checker happy. ditto with adding a 'summary' tag to all s. also removed unneeded  s to make the routine argument tables more legible. doesn't seem to affect the resulting output in any way. Modified Paths: -------------- DART/trunk/ensemble_manager/ensemble_manager_mod.html -------------- next part -------------- Modified: DART/trunk/ensemble_manager/ensemble_manager_mod.html =================================================================== --- DART/trunk/ensemble_manager/ensemble_manager_mod.html 2013-10-09 22:31:00 UTC (rev 6507) +++ DART/trunk/ensemble_manager/ensemble_manager_mod.html 2013-10-09 22:37:21 UTC (rev 6508) @@ -11,7 +11,7 @@

    MODULE ensemble_manager_mod

    -
    +
    DART project logo @@ -225,7 +225,7 @@
    -
    +
    @@ -296,7 +296,7 @@ -
    +
     type ensemble_type
    @@ -400,12 +400,12 @@
     
    -
    +
    -
    +
    call init_ensemble_manager(ens_handle, num_copies, num_vars [, distribution_type_in] @@ -427,7 +427,7 @@ size descriptions in the ensemble_type are initialized.

    - +
    @@ -445,12 +445,12 @@
    ens_handle Handle for the ensemble being initialized
    num_copies
    -
    +
    -
    +
    call read_ensemble_restart(ens_handle, start_copy, end_copy, start_from_restart, file_name @@ -488,33 +488,33 @@ directly and independently.

    - - +
    ens_handle  
    + - + - + - + - + - + - +
    ens_handle Handle of ensemble.
    start_copy  
    start_copy Global index of first of continguous set of copies to be read.
    end_copy  
    end_copy Global index of last of contiguous set of copies to be read, copies(start_copy:end_copy).
    start_from_restart  
    start_from_restart If true, read all copies from file. If false, read one copy and perturb to get required number.
    file_name  
    file_name Name of file from which to read.
    init_time  
    init_time If present, set time of all copies read to this value.
    force_single_file  
    force_single_file If present and true, force the read to be from a single file which contains all copies.
    -
    +
    -
    +
    call write_ensemble_restart(ens_handle, file_name, start_copy, end_copy [, force_single_file]) @@ -545,28 +545,30 @@ directly and independently.

    - - - - - - +
    ens_handle  Handle of ensemble.
    file_name  Name of file from which to read.
    start_copy  
    + + + + + + + - + - +
    ens_handle Handle of ensemble.
    file_name Name of file from which to read.
    start_copy Global index of first of continguous set of copies to be written.
    end_copy  
    end_copy Global index of last of contiguous set of copies to be written, copies(start_copy:end_copy).
    force_single_file  
    force_single_file If present and true, force the write to be to a single file which contains all copies.
    -
    +
    -
    +
    call get_copy(receiving_pe, ens_handle, copy, vars [, mtime]) @@ -591,30 +593,30 @@ Data is only returned in vars on the receiving PE; vars on all other PEs is unset.

    - - +
    receiving_pe  
    + - + - + - + - +
    receiving_pe This process ends up with the requested copy of the state vector.
    ens_handle  
    ens_handle Handle for ensemble.
    copy  
    copy The global index of the copy of the state vector that is to be retrieved.
    vars  
    vars One dimensional array in which the requested copy of the state vector is returned. Data is only returned in vars on the receiving PE; vars on all other PEs is unset.
    mtime  
    mtime If present returns the time of the requested copy.
    -
    +
    -
    +
    call put_copy(sending_pe, ens_handle, copy, vars [, mtime]) @@ -641,29 +643,29 @@ is processed; vars on all other PEs is ignored.

    - - +
    sending_pe  
    + - + - + - + - +
    sending_pe This process sends the copy of the state vector.
    ens_handle  
    ens_handle Handle for ensemble.
    copy  
    copy The global index of the copy of the state vector that is to be sent.
    vars  
    vars One dimensional array in which the requested copy of the state vector is located. Only the data in vars on the sending PE is processed; vars on all other PEs is ignored.
    mtime  
    mtime If present send the time of the copy.
    -
    +
    -
    +
    call broadcast_copy(ens_handle, copy, arraydata)
    @@ -685,24 +687,24 @@
     all processes in the job.
     

    - - +
    ens_handle  
    + - + - +
    ens_handle Handle for ensemble.
    copy  
    copy The global index of the copy of the state vector that is to be sent.
    arraydata  
    arraydata One dimensional array into which the requested copy of the state vector will be copied on all PEs, including the sending PE.
    -
    +
    -
    +
    call set_ensemble_time(ens_handle, indx, mtime)
    @@ -725,23 +727,23 @@
     and to get the local index number for that copy.
     

    - - +
    ens_handle  
    + - + - +
    ens_handle Handle for ensemble.
    indx  
    indx The local index of the copy of the state vector that is to be set.
    mtime  
    mtime The time to set for this copy.
    -
    +
    -
    +
    call get_ensemble_time(ens_handle, indx, mtime)
    @@ -764,18 +766,18 @@
     get_copy_time() which uses global copy numbers. -->
     

    - - +
    ens_handle  
    + - + - +
    ens_handle Handle for ensemble.
    indx  
    indx The local index of the copy to retrieve the time from.
    mtime  
    mtime The returned time value.
    -
    +
    @@ -870,7 +872,7 @@ -
    +
    call end_ensemble_manager(ens_handle)
    @@ -885,18 +887,18 @@
     Frees up storage associated with an ensemble.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for an ensemble.
    -
    +
    -
    +
    call duplicate_ens(ens1, ens2, duplicate_time)
    @@ -925,17 +927,17 @@
             Data from the vars array will be replicated.
     ens2
         Ensemble handle of ensemble into which ens1 vars data will be copied.
    -duplicate_time   
    +duplicate_time  
         If true, copy the times from ens1 into ens2, else leave ens2 times unchanged.
     
     
     
    -
    +
    -
    +
    var = get_my_num_copies(ens_handle)
    @@ -954,21 +956,21 @@
     with only a single process.
     

    - - +
    var  
    + - +
    var Returns the number of copies stored by this process when storing all variables for a subset of copies.
    ens_handle  
    ens_handle Handle for an ensemble.
    -
    +
    -
    +
    var = get_my_num_vars(ens_handle)
    @@ -986,21 +988,21 @@
     with only a single process.
     

    - - +
    var  
    + - +
    var Returns the number of vars stored by this process when storing all copies of a subset of variables.
    ens_handle  
    ens_handle Handle for an ensemble.
    -
    +
    -
    +
    call get_my_copies(ens_handle, copies)
    @@ -1017,21 +1019,21 @@
     when storing subset of copies of all variables.
     

    - - +
    ens_handle  
    + - +
    ens_handle Handle for an ensemble.
    copies  
    copies List of all copies stored by this process when storing subset of copies of all variables.
    -
    +
    -
    +
    call get_my_vars(ens_handle, vars)
    @@ -1048,21 +1050,21 @@
     when storing all copies of a subset of variables.
     

    - - +
    ens_handle  
    + - +
    ens_handle Handle for an ensemble.
    vars  
    vars List of all variables stored on this process when storing all copies of a subset of variables.
    -
    +
    -
    +
    call get_copy_owner_index(copy_number, owner, owners_index)
    @@ -1081,23 +1083,23 @@
     the local storage index for this copy on that process.
     

    - +
    - +
    copy_number Global index of a copy from an ensemble.
    owner Process Element (PE) that stores this copy when each has all variables of a subset of copies.
    owners_index   
    owners_index   Local storage index for this copy on the owning process.
    -
    +
    -
    +
    call get_var_owner_index(var_number, owner, owners_index)
    @@ -1116,23 +1118,23 @@
     the local storage index for this variable on that process.
     

    - +
    - +
    var_number Global index of a variable in the vector from an ensemble.
    owner Process Element (PE) that stores this variable when each has all copies of subset of variables.
    owners_index   
    owners_index   Local storage index for this variable on the owning process.
    -
    +
    -
    +
    call all_vars_to_all_copies(ens_handle, label)
    @@ -1167,21 +1169,21 @@
     vars arrays, so either can be used as a data source.
     

    - - +
    ens_handle  
    + - +
    ens_handle The handle of the ensemble being transposed.
    label  
    label A character string label. If present, a timestamp with this label is printed at the start and end of the transpose.
    -
    +
    -
    +
    call all_copies_to_all_vars(ens_handle, label)
    @@ -1217,21 +1219,21 @@
     

    - - +
    ens_handle  
    + - +
    ens_handle The handle of the ensemble being transposed.
    label  
    label A character string label. If present, a timestamp with this label is printed at the start and end of the transpose.
    -
    +
    -
    +
    call compute_copy_mean(ens_handle, start_copy, end_copy, mean_copy)
    @@ -1256,24 +1258,24 @@
     array data is more current and the vars data is stale.
     

    - - +
    ens_handle  
    + - + - + - +
    ens_handle Handle for an ensemble.
    start_copy  
    start_copy Global index of first copy in mean and sd computation.
    end_copy  
    end_copy Global index of last copy in mean and sd computation.
    mean_copy  
    mean_copy Global index of copy into which mean is written.
    -
    +
    -
    +
    call compute_copy_mean_sd(ens_handle, start_copy, end_copy, mean_copy, sd_copy)
    @@ -1300,26 +1302,26 @@
     

    - - +
    ens_handle  
    + - + - + - + - +
    ens_handle Handle for an ensemble.
    start_copy  
    start_copy Global index of first copy in mean and sd computation.
    end_copy  
    end_copy Global index of last copy in mean and sd computation.
    mean_copy  
    mean_copy Global index of copy into which mean is written.
    sd_copy  
    sd_copy Global index of copy into which standard deviation is written.
    -
    +
    -
    +
    call compute_copy_mean_var(ens_handle, start_copy, end_copy, mean_copy, var_copy)
    @@ -1345,26 +1347,26 @@
     array data is more current and the vars data is stale.
     

    - - +
    ens_handle  
    + - + - + - + - +
    ens_handle Handle for an ensemble.
    start_copy  
    start_copy Global index of first copy in mean and sd computation.
    end_copy  
    end_copy Global index of last copy in mean and sd computation.
    mean_copy  
    mean_copy Global index of copy into which mean is written.
    var_copy  
    var_copy Global index of copy into which variance is written.
    -
    +
    -
    +
    call prepare_to_update_vars(ens_handle)
    @@ -1403,18 +1405,18 @@
     the representations.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for the ensemble being accessed directly.
    -
    +
    -
    +
    call prepare_to_update_copies(ens_handle)
    @@ -1453,18 +1455,18 @@
     the representations.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for the ensemble being accessed directly.
    -
    +
    -
    +
    call prepare_to_read_from_vars(ens_handle)
    @@ -1503,18 +1505,18 @@
     the representations.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for the ensemble being accessed directly.
    -
    +
    -
    +
    call prepare_to_read_from_copies(ens_handle)
    @@ -1553,18 +1555,18 @@
     the representations.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for the ensemble being accessed directly.
    -
    +
    -
    +
    call prepare_to_write_to_vars(ens_handle)
    @@ -1606,18 +1608,18 @@
     the representations.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for the ensemble being accessed directly.
    -
    +
    -
    +
    call prepare_to_write_to_copies(ens_handle)
    @@ -1659,13 +1661,13 @@
     the representations.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for the ensemble being accessed directly.
    -
    +
    @@ -1690,7 +1692,7 @@ -
    +
    var = get_max_num_copies(num_copies)
    @@ -1709,21 +1711,21 @@
     Used to get size for creating storage to receive a list of the copies on a PE.
     

    - - +
    var  
    + - +
    var Returns the largest number of copies any an individual PE when var complete.
    num_copies  
    num_copies Total number of copies in the ensemble.
    -
    +
    -
    +
    var = get_max_num_vars(num_vars)
    @@ -1742,21 +1744,21 @@
     Used to get size for creating storage to receive a list of the vars on a PE.
     

    - - +
    var  
    + - +
    var Returns the largest number of vars any an individual PE when copy complete.
    num_copies  
    num_copies Total number of vars in an ensemble vector.
    -
    +
    -
    +
    call set_up_ens_distribution(ens_handle)
    @@ -1775,18 +1777,18 @@
     total number of processes. 
     

    - - +
    ens_handle  
    +
    ens_handle Handle for an ensemble.
    -
    +
    -
    +
    call get_var_list(num_vars, pe, var_list, pes_num_vars)
    @@ -1807,28 +1809,28 @@
     Depends on distribution_type with only option 1 currently implemented.
     

    - - +
    num_vars  
    + - + - + - +
    num_vars
    pe  
    pe
    var_list(:)  
    var_list(:)
    pes_num_vars  
    pes_num_vars
    -
    +
    -
    +
    call get_copy_list(num_copies, pe, copy_list, pes_num_copies)
    @@ -1849,28 +1851,28 @@
     Depends on distribution_type with only option 1 currently implemented.
     

    - - +
    num_copies  
    + - + - + - +
    num_copies
    pe  
    pe
    copy_list(:)  
    copy_list(:)
    pes_num_copies  
    pes_num_copies
    -
    +
    -
    +
    call timestamp_message(msg [, sync] @@ -1892,16 +1894,16 @@ default is only task 0 prints a timestamp.

    - - +
    msg  
    + - + - +
    msg character string to prepend to the time info
    sync  
    sync if present and true, execute an MPI_Barrier() to sync all MPI tasks before printing the time. this means the time will be the value of the slowest of the tasks to reach this point.
    alltasks  
    alltasks if present and true, have all tasks print out a timestamp. the default is for just task 0 to print. the usual combination is either sync=true and alltasks=false, or sync=false and alltasks=true. @@ -1909,12 +1911,12 @@
    -
    +
    -
    +
    call print_ens_handle(ens_handle, force, label)
    @@ -1935,26 +1937,26 @@
     provide context for the output.
     

    - - +
    ens_handle  
    + - + - +
    ens_handle The derived type to print information about.
    force  
    force If the debug namelist item is false, set this to true to enable printing.
    label  
    label Optional string label to print to provide context for the output.
    -
    +
    -
    +
    call assign_tasks_to_pes(ens_handle, nEns_members, layout_type)
    @@ -1977,24 +1979,24 @@
       2. Round-robin on the nodes
     

    - - +
    ens_handle  
    + - + - +
    ens_handle Handle for an ensemble.
      
      
    -
    +
    -
    +
    call round_robin(ens_handle)
    @@ -2013,18 +2015,18 @@
     place to start. Test with the smoother.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for an ensemble.
    -
    +
    -
    +
    call create_pe_to_task_list(ens_handle)
    @@ -2041,18 +2043,18 @@
     otherwise this routine will just return nonsense.
     

    - - +
    ens_handle  
    +
    ens_handle Handle for an ensemble.
    -
    +
    -
    +
    call calc_tasks_on_each_node(nodes, last_node_task_number)
    @@ -2072,22 +2074,22 @@
     mpi tasks = 17
     

    - - +
    nodes  
    + - +
    nodes
    last_node_task_number  
    last_node_task_number
    -
    +
    -
    +
    call simple_layout(ens_handle, n)
    @@ -2104,21 +2106,21 @@
     where my_pe = my_task_id()
     

    - - +
    ens_handle  
    + - +
    ens_handle Handle for an ensemble.
    n  
    n
    -
    +
    @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Fri Oct 11 15:35:38 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 11 Oct 2013 15:35:38 -0600 Subject: [Dart-dev] [6509] DART/trunk/models/cam/work/input.nml: commit a version that has a perfect_model_obs namelist. Message-ID: Revision: 6509 Author: nancy Date: 2013-10-11 15:35:38 -0600 (Fri, 11 Oct 2013) Log Message: ----------- commit a version that has a perfect_model_obs namelist. Modified Paths: -------------- DART/trunk/models/cam/work/input.nml -------------- next part -------------- Modified: DART/trunk/models/cam/work/input.nml =================================================================== --- DART/trunk/models/cam/work/input.nml 2013-10-09 22:37:21 UTC (rev 6508) +++ DART/trunk/models/cam/work/input.nml 2013-10-11 21:35:38 UTC (rev 6509) @@ -44,6 +44,27 @@ ! ! inf_flavor is 0:none, 1:obs space, 2: varying state space, 3: fixed state_space &perfect_model_obs_nml + start_from_restart = .true., + output_restart = .false., + async = 0, + init_time_days = -1, + init_time_seconds = -1, + first_obs_days = -1, + first_obs_seconds = -1, + last_obs_days = -1, + last_obs_seconds = -1, + output_interval = 1, + trace_execution = .false., + restart_in_file_name = "perfect_ics", + restart_out_file_name = "perfect_restart", + obs_seq_in_file_name = "obs_seq.in", + obs_seq_out_file_name = "obs_seq.out", + adv_ens_command = "no_advance_script" + output_timestamps = .false., + trace_execution = .false., + output_forward_op_errors = .false., + print_every_nth_obs = -1, + silence = .false., / &smoother_nml From nancy at ucar.edu Fri Oct 11 15:37:04 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 11 Oct 2013 15:37:04 -0600 Subject: [Dart-dev] [6510] DART/trunk/models/cam/work/input.nml: make the output obs_seq file be obs_seq.perfect - that makes Message-ID: Revision: 6510 Author: nancy Date: 2013-10-11 15:37:04 -0600 (Fri, 11 Oct 2013) Log Message: ----------- make the output obs_seq file be obs_seq.perfect - that makes it more obvious that these are synthetic obs and not real ones. i found this used in another model's namelist and i'm starting to like it more and more. Modified Paths: -------------- DART/trunk/models/cam/work/input.nml -------------- next part -------------- Modified: DART/trunk/models/cam/work/input.nml =================================================================== --- DART/trunk/models/cam/work/input.nml 2013-10-11 21:35:38 UTC (rev 6509) +++ DART/trunk/models/cam/work/input.nml 2013-10-11 21:37:04 UTC (rev 6510) @@ -58,7 +58,7 @@ restart_in_file_name = "perfect_ics", restart_out_file_name = "perfect_restart", obs_seq_in_file_name = "obs_seq.in", - obs_seq_out_file_name = "obs_seq.out", + obs_seq_out_file_name = "obs_seq.perfect", adv_ens_command = "no_advance_script" output_timestamps = .false., trace_execution = .false., From nancy at ucar.edu Fri Oct 11 16:31:22 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 11 Oct 2013 16:31:22 -0600 Subject: [Dart-dev] [6511] DART/trunk/models/CESM/shell_scripts: cam version is working, and i believe the others Message-ID: Revision: 6511 Author: nancy Date: 2013-10-11 16:31:22 -0600 (Fri, 11 Oct 2013) Log Message: ----------- cam version is working, and i believe the others are very close. testing now, but want to get versions into the repository before the weekend. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh 2013-10-11 21:37:04 UTC (rev 6510) +++ DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh 2013-10-11 22:31:22 UTC (rev 6511) @@ -52,13 +52,13 @@ endsw #------------------------------------------------------------------------- -# Determine time of model state ... from file name of first member -# of the form "./${CASE}.cam_${ensemble_member}.i.2000-01-06-00000.nc" +# Determine time of model state ... from file name +# of the form "./${CASE}.cam.i.2000-01-06-00000.nc" # # Piping stuff through 'bc' strips off any preceeding zeros. #------------------------------------------------------------------------- -set FILE = `head -n 1 rpointer.atm_0001` +set FILE = `head -n 1 rpointer.atm` set FILE = $FILE:t set FILE = $FILE:r set ATM_DATE_EXT = `echo $FILE:e` @@ -82,7 +82,7 @@ echo "Hour is $ATM_HOUR so we are generating perfect obs for the atmosphere" else echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the atmosphere" - echo "`date` -- END CAM_ASSIMILATE" + echo "`date` -- END GENERATE CAM TRUE STATE" exit 0 endif @@ -118,25 +118,10 @@ endif #========================================================================= -# Block 1: Populate a run-time directory with the input needed to run DART. -#========================================================================= - -echo "`date` -- BEGIN COPY BLOCK" - -if ( -e ${CASEROOT}/cam_input.nml ) then - ${COPY} ${CASEROOT}/cam_input.nml input.nml -else - echo "ERROR ... DART required file ${CASEROOT}/cam_input.nml not found ... ERROR" - echo "ERROR ... DART required file ${CASEROOT}/cam_input.nml not found ... ERROR" - exit -2 -endif - -echo "`date` -- END COPY BLOCK" - -#========================================================================= -# Block 2: Convert 1 CAM restart file to DART initial conditions file. +# Block 1: Populate a run-time directory with the input needed to run DART, +# and Block 2: Convert 1 CAM restart file to DART initial conditions file. # At the end of the block, we have DART initial condition file perfect_ics -# that came from pointer file ../rpointer.atm_0001 +# that came from pointer file ../rpointer.atm # # REQUIRED DART namelist settings: # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' @@ -145,51 +130,49 @@ echo "`date` -- BEGIN CAM-TO-DART" -set member = 1 + if ( ! -e ${CASEROOT}/cam_input.nml ) then + echo "ERROR ... DART required file ${CASEROOT}/cam_input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/cam_input.nml not found ... ERROR" + exit -2 + endif - set MYTEMPDIR = member_${member} - mkdir -p $MYTEMPDIR - cd $MYTEMPDIR - # Turns out the .h0. files are timestamped with the START of the # run, which is *not* ATM_DATE_EXT ... I just link to a whatever # is convenient (since the info is static). # make sure there are no old output logs hanging around - $REMOVE output.${member}.cam_to_dart + $REMOVE output.cam_to_dart - set ATM_INITIAL_FILENAME = "../../${CASE}.cam.i.${ATM_DATE_EXT}.nc" - set ATM_HISTORY_FILENAME = `ls -1t ../../${CASE}.cam*.h0.* | head -n 1` - set DART_IC_FILENAME = `printf filter_ics.%04d ${member}` - set DART_RESTART_FILE = `printf filter_restart.%04d ${member}` + set ATM_INITIAL_FILENAME = "../${CASE}.cam.i.${ATM_DATE_EXT}.nc" + set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` + set DART_IC_FILENAME = perfect_ics - sed -e "s#dart_ics#../${DART_IC_FILENAME}#" \ - -e "s#dart_restart#../${DART_RESTART_FILE}#" < ../input.nml >! input.nml + sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/cam_input.nml >! input.nml ${LINK} $ATM_INITIAL_FILENAME caminput.nc ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc - ${EXEROOT}/cam_to_dart >! output.${member}.cam_to_dart + ${EXEROOT}/cam_to_dart >! output.cam_to_dart if ($status != 0) then - echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" - echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" - exit -6 -endif + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + exit -6 + endif echo "`date` -- END CAM-TO-DART" #========================================================================= -# Block 3: Advance the model and harvest the synthetic observations. -# Will result in a single file : 'perfect_restart' which we don't need -# for a perfect model experiment with CESM. +# Block 3: Run perfect_model_obs and harvest the synthetic observations +# and diagnostic files. # # DART namelist settings required: # &perfect_model_obs_nml: async = 0, -# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", +# &perfect_model_obs_nml: adv_ens_command = "no_advance_script", +# &perfect_model_obs_nml: output_restart = .false., # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' +# &perfect_model_obs_nml: restart_out_file_name = 'not_created' # &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' -# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' # &perfect_model_obs_nml: init_time_days = -1, # &perfect_model_obs_nml: init_time_seconds = -1, # &perfect_model_obs_nml: first_obs_days = -1, @@ -199,22 +182,16 @@ # #========================================================================= -# CAM:static_init_model() always needs a caminput.nc and a cam_phis.nc -# for geometry information, etc. -set ATM_INITIAL_FILENAME = ../${CASE}.cam.i.${ATM_DATE_EXT}.nc -set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` - -${LINK} $ATM_INITIAL_FILENAME caminput.nc -${LINK} $ATM_HISTORY_FILENAME cam_phis.nc - echo "`date` -- BEGIN CAM PERFECT_MODEL_OBS" -${LAUNCHCMD} ${EXEROOT}/perfect_model_obs_cam || exit -7 +${EXEROOT}/perfect_model_obs_cam || exit -7 echo "`date` -- END CAM PERFECT_MODEL_OBS" + +${MOVE} True_State.nc ../cam_True_State.${ATM_DATE_EXT}.nc ${MOVE} Prior_Diag.nc ../cam_Prior_Diag.${ATM_DATE_EXT}.nc ${MOVE} Posterior_Diag.nc ../cam_Posterior_Diag.${ATM_DATE_EXT}.nc -${MOVE} obs_seq.final ../cam_obs_seq.${ATM_DATE_EXT}.final +${MOVE} obs_seq.perfect ../cam_obs_seq.${ATM_DATE_EXT}.perfect ${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out #========================================================================= @@ -233,7 +210,6 @@ ${LINK} ${ATM_INITIAL_FILENAME} cam_initial.nc || exit -9 -end #------------------------------------------------------------------------- # Cleanup Modified: DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh 2013-10-11 21:37:04 UTC (rev 6510) +++ DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh 2013-10-11 22:31:22 UTC (rev 6511) @@ -9,6 +9,8 @@ # This block is an attempt to localize all the machine-specific # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# run. echo "`date` -- BEGIN GENERATE CLM TRUE STATE" @@ -90,7 +92,7 @@ echo "valid time of model is $LND_YEAR $LND_MONTH $LND_DAY $LND_HOUR (hours)" #------------------------------------------------------------------------- -# Determine if current time is an perfect model time. +# Determine if current time is a perfect model time. # If not, return before doing anything. #------------------------------------------------------------------------- @@ -99,7 +101,7 @@ echo "Hour is $LND_HOUR so we are generating perfect obs for the land" else echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the land" - echo "`date` -- END GENERATE TRUE STATE" + echo "`date` -- END GENERATE CLM TRUE STATE" exit 0 endif @@ -148,7 +150,7 @@ set OBS_FILE = ${BASEOBSDIR}/${OBSDIR}/obs_seq.${LND_DATE_EXT} if ( -e ${OBS_FILE} ) then - ${COPY} ${OBS_FILE} obs_seq.in + ${LINK} ${OBS_FILE} obs_seq.in else echo "ERROR ... no observation file $OBS_FILE" echo "ERROR ... no observation file $OBS_FILE" @@ -156,57 +158,36 @@ endif #========================================================================= -# Block 1: Populate a run-time directory with the input needed to run DART. -#========================================================================= - -echo "`date` -- BEGIN COPY BLOCK" - -if ( -e ${CASEROOT}/clm_input.nml ) then - ${COPY} ${CASEROOT}/clm_input.nml input.nml -else - echo "ERROR ... DART required file ${CASEROOT}/clm_input.nml not found ... ERROR" - echo "ERROR ... DART required file ${CASEROOT}/clm_input.nml not found ... ERROR" - exit -2 -endif - -echo "`date` -- END COPY BLOCK" - -#========================================================================= -# Block 2: convert 1 clm restart file to a DART initial conditions file. -# At the end of the block, we have a DART restart file perfect_ics -# that came from the pointer file ../rpointer.lnd_0001 +# Block 1: Populate a run-time directory with the input needed to run DART, +# and Block 2: Convert 1 CLM restart file to DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from pointer file ../rpointer.lnd # -# DART namelist settings appropriate/required: +# REQUIRED DART namelist settings: # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' # &clm_to_dart_nml: clm_to_dart_output_file = 'dart_ics' #========================================================================= echo "`date` -- BEGIN CLM-TO-DART" -set member = 1 + if ( ! -e ${CASEROOT}/clm_input.nml ) then + echo "ERROR ... DART required file ${CASEROOT}/clm_input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/clm_input.nml not found ... ERROR" + exit -2 + endif - set MYTEMPDIR = member_${member} - mkdir -p $MYTEMPDIR - cd $MYTEMPDIR - # make sure there are no old output logs hanging around - $REMOVE output.${member}.clm_to_dart + $REMOVE output.clm_to_dart - set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc - set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc - set OBS1_HISTORY_FILENAME = ${CASE}.clm2.h1.${LND_DATE_EXT}.nc - set OBS2_HISTORY_FILENAME = ${CASE}.clm2_0001.h1.${LND_DATE_EXT}.nc + set LND_RESTART_FILENAME = ../${CASE}.clm2.r.${LND_DATE_EXT}.nc + set LND_HISTORY_FILENAME = ../${CASE}.clm2.h0.${LND_DATE_EXT}.nc set DART_IC_FILENAME = perfect_ics - sed -e "s/dart_ics/..\/${DART_IC_FILENAME}/" < ../input.nml >! input.nml + sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/clm_input.nml >! input.nml - ${LINK} ../../$LND_RESTART_FILENAME clm_restart.nc - ${LINK} ../../$LND_HISTORY_FILENAME clm_history.nc + ${LINK} $LND_RESTART_FILENAME clm_restart.nc + ${LINK} $LND_HISTORY_FILENAME clm_history.nc - if (-e $OBS1_HISTORY_FILENAME) then - ${LINK} ../../$OBS1_HISTORY_FILENAME $OBS2_HISTORY_FILENAME - endif - # patch the CLM restart files to ensure they have the proper # _FillValue and missing_value attributes. # ncatted -O -a _FillValue,frac_sno,o,d,1.0e+36 clm_restart.nc @@ -220,30 +201,28 @@ # ncatted -O -a _FillValue,T_SOISNO,o,d,1.0e+36 clm_restart.nc # ncatted -O -a missing_value,T_SOISNO,o,d,1.0e+36 clm_restart.nc - ${EXEROOT}/clm_to_dart >! output.${member}.clm_to_dart + ${EXEROOT}/clm_to_dart >! output.clm_to_dart if ($status != 0) then echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" - exit -3 + exit -6 endif - cd .. - echo "`date` -- END CLM-TO-DART" #========================================================================= -# Block 3: Advance the model and harvest the synthetic observations. -# Will result in a single file : 'perfect_restart' which we don't need -# for a perfect model experiment with CESM. +# Block 3: Run perfect_model_obs and harvest the synthetic observations +# and diagnostic files. # # DART namelist settings required: # &perfect_model_obs_nml: async = 0, -# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", +# &perfect_model_obs_nml: adv_ens_command = "no_advance_script", +# &perfect_model_obs_nml: output_restart = .false., # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' +# &perfect_model_obs_nml: restart_out_file_name = 'not_created' # &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' -# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' # &perfect_model_obs_nml: init_time_days = -1, # &perfect_model_obs_nml: init_time_seconds = -1, # &perfect_model_obs_nml: first_obs_days = -1, @@ -253,28 +232,25 @@ # #========================================================================= -# clm always needs a clm_restart.nc, clm_history.nc for geometry information, etc. -set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc -set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc - -${LINK} ../$LND_RESTART_FILENAME clm_restart.nc -${LINK} ../$LND_HISTORY_FILENAME clm_history.nc - echo "`date` -- BEGIN CLM PERFECT_MODEL_OBS" -${EXEROOT}/perfect_model_obs_clm || exit -4 +${EXEROOT}/perfect_model_obs_clm || exit -7 echo "`date` -- END CLM PERFECT_MODEL_OBS" -${MOVE} True_State.nc ../clm_True_State.${LND_DATE_EXT}.nc -${MOVE} obs_seq.out ../clm_obs_seq.${LND_DATE_EXT}.out -${MOVE} dart_log.out ../clm_dart_log.${LND_DATE_EXT}.out +${MOVE} True_State.nc ../clm_True_State.${LND_DATE_EXT}.nc +${MOVE} Prior_Diag.nc ../clm_Prior_Diag.${LND_DATE_EXT}.nc +${MOVE} Posterior_Diag.nc ../clm_Posterior_Diag.${LND_DATE_EXT}.nc +${MOVE} obs_seq.perfect ../clm_obs_seq.${LND_DATE_EXT}.perfect +${MOVE} dart_log.out ../clm_dart_log.${LND_DATE_EXT}.out + #========================================================================= -# Block 4: Update the clm restart files. +# Block 4: Update the clm restart file #========================================================================= # not needed ... perfect_model_obs does not update the model state. + #------------------------------------------------------------------------- # Cleanup #------------------------------------------------------------------------- Modified: DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-11 21:37:04 UTC (rev 6510) +++ DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-11 22:31:22 UTC (rev 6511) @@ -10,7 +10,7 @@ # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. # Search below for TIMECHECK to see what times this script will -# evaluate obs. +# run. echo "`date` -- BEGIN GENERATE POP TRUE STATE" @@ -82,7 +82,7 @@ echo "Hour is $OCN_HOUR so we are generating perfect obs for the ocean" else echo "Hour is not 0Z so we are skipping generating perfect obs for the ocean" - echo "`date` -- END GENERATE TRUE STATE" + echo "`date` -- END GENERATE POP TRUE STATE" exit 0 endif @@ -110,7 +110,7 @@ set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}/${OBSFNAME} if ( -e ${OBS_FILE} ) then - ${COPY} ${OBS_FILE} obs_seq.in + ${LINK} ${OBS_FILE} obs_seq.in else echo "ERROR ... no observation file $OBS_FILE" echo "ERROR ... no observation file $OBS_FILE" @@ -118,57 +118,37 @@ endif #========================================================================= -# Block 1: Populate a run-time directory with the input needed to run DART. -#========================================================================= - -echo "`date` -- BEGIN COPY BLOCK" - -if ( -e ${CASEROOT}/input.nml ) then - ${COPY} ${CASEROOT}/input.nml . -else - echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" - echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" - exit -2 -endif - -echo "`date` -- END COPY BLOCK" - -# Eat the cookie regardless -${REMOVE} ../pop_inflation_cookie - -#========================================================================= -# Block 2: convert 1 pop restart file to a DART initial conditions file. -# At the end of the block, we have a DART restart file perfect_ics -# that came from the pointer file ../rpointer.ocn.restart +# Block 1: Populate a run-time directory with the input needed to run DART, +# and Block 2: Convert 1 POP restart file to DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from pointer file ../rpointer.ocn.restart # -# DART namelist settings appropriate/required: +# REQUIRED DART namelist settings: # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' # &pop_to_dart_nml: pop_to_dart_output_file = 'dart_ics' #========================================================================= echo "`date` -- BEGIN POP-TO-DART" -set member = 1 + if ( ! -e ${CASEROOT}/pop_input.nml ) then + echo "ERROR ... DART required file ${CASEROOT}/pop_input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/pop_input.nml not found ... ERROR" + exit -2 + endif - set MYTEMPDIR = member_${member} - mkdir -p $MYTEMPDIR - cd $MYTEMPDIR - # make sure there are no old output logs hanging around - $REMOVE output.${member}.pop_to_dart + $REMOVE output.pop_to_dart - set OCN_RESTART_FILENAME = ${CASE}.pop.r.${OCN_DATE_EXT}.nc - set OCN_NML_FILENAME = pop2_in + set OCN_RESTART_FILENAME = ../${CASE}.pop.r.${OCN_DATE_EXT}.nc + set OCN_NML_FILENAME = ../pop2_in set DART_IC_FILENAME = perfect_ics - set DART_RESTART_FILE = perfect_restart - sed -e "s#dart_ics#../${DART_IC_FILENAME}#" \ - -e "s#dart_restart#../${DART_RESTART_FILE}#" < ../input.nml >! input.nml + sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/pop_input.nml >! input.nml - ${LINK} ../../$OCN_RESTART_FILENAME pop.r.nc - ${LINK} ../../$OCN_NML_FILENAME pop_in + ${LINK} $OCN_RESTART_FILENAME pop.r.nc + ${LINK} $OCN_NML_FILENAME pop_in - ${EXEROOT}/pop_to_dart >! output.${member}.pop_to_dart + ${EXEROOT}/pop_to_dart >! output.pop_to_dart if ($status != 0) then echo "ERROR ... DART died in 'pop_to_dart' ... ERROR" @@ -181,17 +161,17 @@ echo "`date` -- END POP-TO-DART" #========================================================================= -# Block 3: Advance the model and harvest the synthetic observations. -# Will result in a single file : 'perfect_restart' which we don't need -# for a perfect model experiment with CESM. +# Block 3: Run perfect_model_obs and harvest the synthetic observations +# and diagnostic files. # # DART namelist settings required: # &perfect_model_obs_nml: async = 0, -# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", +# &perfect_model_obs_nml: adv_ens_command = "no_advance_script", +# &perfect_model_obs_nml: output_restart = .false., # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' +# &perfect_model_obs_nml: restart_out_file_name = 'not_created' # &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' -# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' # &perfect_model_obs_nml: init_time_days = -1, # &perfect_model_obs_nml: init_time_seconds = -1, # &perfect_model_obs_nml: first_obs_days = -1, @@ -201,27 +181,25 @@ # #========================================================================= -# POP always needs a pop_in and a pop.r.nc to start. -set OCN_RESTART_FILENAME = ${CASE}.pop.r.${OCN_DATE_EXT}.nc +echo "`date` -- BEGIN POP PERFECT_MODEL_OBS" +${EXEROOT}/perfect_model_obs_pop || exit -7 +echo "`date` -- END POP PERFECT_MODEL_OBS" -${LINK} ../$OCN_RESTART_FILENAME pop.r.nc -${LINK} ../pop2_in pop_in -echo "`date` -- BEGIN PERFECT_MODEL_OBS" -${EXEROOT}/perfect_model_obs || exit -4 -echo "`date` -- END PERFECT_MODEL_OBS" +${MOVE} True_State.nc ../pop_True_State.${OCN_DATE_EXT}.nc +${MOVE} Prior_Diag.nc ../pop_Prior_Diag.${OCN_DATE_EXT}.nc +${MOVE} Posterior_Diag.nc ../pop_Posterior_Diag.${OCN_DATE_EXT}.nc +${MOVE} obs_seq.perfect ../pop_obs_seq.${OCN_DATE_EXT}.perfect +${MOVE} dart_log.out ../pop_dart_log.${OCN_DATE_EXT}.out -${MOVE} True_State.nc ../pop_True_State.${OCN_DATE_EXT}.nc -${MOVE} obs_seq.out ../pop_obs_seq.${OCN_DATE_EXT}.out -${MOVE} dart_log.out ../pop_dart_log.${OCN_DATE_EXT}.out - #========================================================================= # Block 4: Update the pop restart files. #========================================================================= # not needed ... perfect_model_obs does not update the model state. + #------------------------------------------------------------------------- # Cleanup #------------------------------------------------------------------------- From nancy at ucar.edu Fri Oct 11 16:33:10 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 11 Oct 2013 16:33:10 -0600 Subject: [Dart-dev] [6512] DART/trunk/models/CESM/shell_scripts: remove redundant script variable $MYCASE, since the calling cesm Message-ID: Revision: 6512 Author: nancy Date: 2013-10-11 16:33:10 -0600 (Fri, 11 Oct 2013) Log Message: ----------- remove redundant script variable $MYCASE, since the calling cesm script sets $CASE to the same thing, and we inherit that environment. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh 2013-10-11 22:31:22 UTC (rev 6511) +++ DART/trunk/models/CESM/shell_scripts/cam_assimilate.csh 2013-10-11 22:33:10 UTC (rev 6512) @@ -66,7 +66,6 @@ set FILE = `head -n 1 rpointer.atm_0001` set FILE = $FILE:t set FILE = $FILE:r -set MYCASE = `echo $FILE | sed -e "s#\..*##"` set ATM_DATE_EXT = `echo $FILE:e` set ATM_DATE = `echo $FILE:e | sed -e "s#-# #g"` set ATM_YEAR = `echo $ATM_DATE[1] | bc` @@ -366,8 +365,8 @@ # make sure there are no old output logs hanging around $REMOVE output.${member}.cam_to_dart - set ATM_INITIAL_FILENAME = `printf ../../${MYCASE}.cam_%04d.i.${ATM_DATE_EXT}.nc ${member}` - set ATM_HISTORY_FILENAME = `ls -1t ../../${MYCASE}.cam*.h0.* | head -n 1` + set ATM_INITIAL_FILENAME = `printf ../../${CASE}.cam_%04d.i.${ATM_DATE_EXT}.nc ${member}` + set ATM_HISTORY_FILENAME = `ls -1t ../../${CASE}.cam*.h0.* | head -n 1` set DART_IC_FILENAME = `printf filter_ics.%04d ${member}` set DART_RESTART_FILE = `printf filter_restart.%04d ${member}` @@ -420,8 +419,8 @@ # CAM:static_init_model() always needs a caminput.nc and a cam_phis.nc # for geometry information, etc. -set ATM_INITIAL_FILENAME = ../${MYCASE}.cam_0001.i.${ATM_DATE_EXT}.nc -set ATM_HISTORY_FILENAME = `ls -1t ../${MYCASE}.cam*.h0.* | head -n 1` +set ATM_INITIAL_FILENAME = ../${CASE}.cam_0001.i.${ATM_DATE_EXT}.nc +set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` ${LINK} $ATM_INITIAL_FILENAME caminput.nc ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc @@ -514,7 +513,7 @@ set inst_string = `printf _%04d $member` - set ATM_INITIAL_FILENAME = ${MYCASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc + set ATM_INITIAL_FILENAME = ${CASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 Modified: DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh 2013-10-11 22:31:22 UTC (rev 6511) +++ DART/trunk/models/CESM/shell_scripts/clm_assimilate.csh 2013-10-11 22:33:10 UTC (rev 6512) @@ -66,7 +66,6 @@ set FILE = `head -n 1 rpointer.lnd_0001` set FILE = $FILE:t set FILE = $FILE:r -set MYCASE = `echo $FILE | sed -e "s#\..*##"` set LND_DATE_EXT = `echo $FILE:e` set LND_DATE = `echo $FILE:e | sed -e "s#-# #g"` set LND_YEAR = `echo $LND_DATE[1] | bc` Modified: DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh 2013-10-11 22:31:22 UTC (rev 6511) +++ DART/trunk/models/CESM/shell_scripts/pop_assimilate.csh 2013-10-11 22:33:10 UTC (rev 6512) @@ -66,7 +66,6 @@ set FILE = `head -n 1 rpointer.ocn_0001.restart` set FILE = $FILE:t set FILE = $FILE:r -set MYCASE = `echo $FILE | sed -e "s#\..*##"` set OCN_DATE_EXT = `echo $FILE:e` set OCN_DATE = `echo $FILE:e | sed -e "s#-# #g"` set OCN_YEAR = `echo $OCN_DATE[1] | bc` @@ -375,7 +374,7 @@ # make sure there are no old output logs hanging around $REMOVE output.${member}.pop_to_dart - set OCN_RESTART_FILENAME = `printf ${MYCASE}.pop_%04d.r.${OCN_DATE_EXT}.nc ${member}` + set OCN_RESTART_FILENAME = `printf ${CASE}.pop_%04d.r.${OCN_DATE_EXT}.nc ${member}` set OCN_NML_FILENAME = `printf pop2_in_%04d ${member}` set DART_IC_FILENAME = `printf filter_ics.%04d ${member}` set DART_RESTART_FILE = `printf filter_restart.%04d ${member}` From nancy at ucar.edu Fri Oct 11 16:38:32 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 11 Oct 2013 16:38:32 -0600 Subject: [Dart-dev] [6513] DART/trunk/models: fix up perfect model sections to work with the CESM scripts. Message-ID: Revision: 6513 Author: nancy Date: 2013-10-11 16:38:32 -0600 (Fri, 11 Oct 2013) Log Message: ----------- fix up perfect model sections to work with the CESM scripts. Modified Paths: -------------- DART/trunk/models/POP/work/input.nml DART/trunk/models/clm/work/input.nml -------------- next part -------------- Modified: DART/trunk/models/POP/work/input.nml =================================================================== --- DART/trunk/models/POP/work/input.nml 2013-10-11 22:33:10 UTC (rev 6512) +++ DART/trunk/models/POP/work/input.nml 2013-10-11 22:38:32 UTC (rev 6513) @@ -1,7 +1,7 @@ &perfect_model_obs_nml start_from_restart = .true., - output_restart = .true., - async = 2, + output_restart = .false., + async = 0, init_time_days = -1, init_time_seconds = -1, first_obs_days = -1, @@ -13,8 +13,8 @@ restart_in_file_name = "perfect_ics", restart_out_file_name = "perfect_restart", obs_seq_in_file_name = "obs_seq.in", - obs_seq_out_file_name = "obs_seq.out", - adv_ens_command = "./advance_model.csh" + obs_seq_out_file_name = "obs_seq.perfect", + adv_ens_command = "no_advance_script" output_timestamps = .false., trace_execution = .false., output_forward_op_errors = .false., Modified: DART/trunk/models/clm/work/input.nml =================================================================== --- DART/trunk/models/clm/work/input.nml 2013-10-11 22:33:10 UTC (rev 6512) +++ DART/trunk/models/clm/work/input.nml 2013-10-11 22:38:32 UTC (rev 6513) @@ -1,6 +1,6 @@ &perfect_model_obs_nml start_from_restart = .true., - output_restart = .true., + output_restart = .false., async = 0, init_time_days = -1, init_time_seconds = -1, @@ -13,16 +13,15 @@ restart_in_file_name = "perfect_ics", restart_out_file_name = "perfect_restart", obs_seq_in_file_name = "obs_seq.in", - obs_seq_out_file_name = "obs_seq.out", - adv_ens_command = "./ha_ha_ha.csh" + obs_seq_out_file_name = "obs_seq.perfect", + adv_ens_command = "no_advance_script" output_timestamps = .false., trace_execution = .false., - output_forward_op_errors = .true., + output_forward_op_errors = .false., print_every_nth_obs = -1, silence = .false., / - &filter_nml async = 0, adv_ens_command = "./no_advance_model.csh", From nancy at ucar.edu Fri Oct 11 16:40:22 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 11 Oct 2013 16:40:22 -0600 Subject: [Dart-dev] [6514] DART/trunk/models/CESM/shell_scripts: perfect model makes a True_State.nc file, but it doesn't Message-ID: Revision: 6514 Author: nancy Date: 2013-10-11 16:40:22 -0600 (Fri, 11 Oct 2013) Log Message: ----------- perfect model makes a True_State.nc file, but it doesn't make Prior or Posterior Diag files. stop trying to copy them after PMO is done. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh 2013-10-11 22:38:32 UTC (rev 6513) +++ DART/trunk/models/CESM/shell_scripts/cam_perfect_model.csh 2013-10-11 22:40:22 UTC (rev 6514) @@ -189,8 +189,6 @@ ${MOVE} True_State.nc ../cam_True_State.${ATM_DATE_EXT}.nc -${MOVE} Prior_Diag.nc ../cam_Prior_Diag.${ATM_DATE_EXT}.nc -${MOVE} Posterior_Diag.nc ../cam_Posterior_Diag.${ATM_DATE_EXT}.nc ${MOVE} obs_seq.perfect ../cam_obs_seq.${ATM_DATE_EXT}.perfect ${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out Modified: DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh 2013-10-11 22:38:32 UTC (rev 6513) +++ DART/trunk/models/CESM/shell_scripts/clm_perfect_model.csh 2013-10-11 22:40:22 UTC (rev 6514) @@ -239,8 +239,6 @@ ${MOVE} True_State.nc ../clm_True_State.${LND_DATE_EXT}.nc -${MOVE} Prior_Diag.nc ../clm_Prior_Diag.${LND_DATE_EXT}.nc -${MOVE} Posterior_Diag.nc ../clm_Posterior_Diag.${LND_DATE_EXT}.nc ${MOVE} obs_seq.perfect ../clm_obs_seq.${LND_DATE_EXT}.perfect ${MOVE} dart_log.out ../clm_dart_log.${LND_DATE_EXT}.out Modified: DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-11 22:38:32 UTC (rev 6513) +++ DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-11 22:40:22 UTC (rev 6514) @@ -188,8 +188,6 @@ ${MOVE} True_State.nc ../pop_True_State.${OCN_DATE_EXT}.nc -${MOVE} Prior_Diag.nc ../pop_Prior_Diag.${OCN_DATE_EXT}.nc -${MOVE} Posterior_Diag.nc ../pop_Posterior_Diag.${OCN_DATE_EXT}.nc ${MOVE} obs_seq.perfect ../pop_obs_seq.${OCN_DATE_EXT}.perfect ${MOVE} dart_log.out ../pop_dart_log.${OCN_DATE_EXT}.out From nancy at ucar.edu Mon Oct 14 10:56:29 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Mon, 14 Oct 2013 10:56:29 -0600 Subject: [Dart-dev] [6515] DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh: remove ' cd ..' which shouldn't have been left there. Message-ID: Revision: 6515 Author: nancy Date: 2013-10-14 10:56:29 -0600 (Mon, 14 Oct 2013) Log Message: ----------- remove 'cd ..' which shouldn't have been left there. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-11 22:40:22 UTC (rev 6514) +++ DART/trunk/models/CESM/shell_scripts/pop_perfect_model.csh 2013-10-14 16:56:29 UTC (rev 6515) @@ -156,8 +156,6 @@ exit -3 endif - cd .. - echo "`date` -- END POP-TO-DART" #========================================================================= From nancy at ucar.edu Wed Oct 16 17:10:19 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 16 Oct 2013 17:10:19 -0600 Subject: [Dart-dev] [6522] DART/trunk/models/tiegcm/shell_scripts: These changes are approved by Tomoko in support of tiegcm V 1.95 Message-ID: Revision: 6522 Author: thoar Date: 2013-10-16 17:10:19 -0600 (Wed, 16 Oct 2013) Log Message: ----------- These changes are approved by Tomoko in support of tiegcm V 1.95 Modified Paths: -------------- DART/trunk/models/tiegcm/shell_scripts/advance_model.csh DART/trunk/models/tiegcm/shell_scripts/run_filter.csh DART/trunk/models/tiegcm/shell_scripts/run_filter_async4.csh DART/trunk/models/tiegcm/shell_scripts/run_perfect_model_obs.csh -------------- next part -------------- Modified: DART/trunk/models/tiegcm/shell_scripts/advance_model.csh =================================================================== --- DART/trunk/models/tiegcm/shell_scripts/advance_model.csh 2013-10-16 23:00:15 UTC (rev 6521) +++ DART/trunk/models/tiegcm/shell_scripts/advance_model.csh 2013-10-16 23:10:19 UTC (rev 6522) @@ -67,7 +67,7 @@ set tierestart = `printf "tiegcm_restart_p.nc.%04d" $ensemble_member` set tieinp = `printf "tiegcm.nml.%04d" $ensemble_member` - ln -sf ../$input_file temp_ic || exit 2 + cp -p ../$input_file temp_ic || exit 2 cp -p ../$tiesecond tiegcm_s.nc || exit 2 cp -p ../$tierestart tiegcm_restart_p.nc || exit 2 cp -p ../$tieinp tiegcm.nml || exit 2 @@ -90,8 +90,6 @@ set secstop = " SECSTOP = "`head -4 namelist_update | tail -1` set hist = " HIST = "`head -5 namelist_update | tail -1` set sechist = " SECHIST = "`head -5 namelist_update | tail -1` - set save = " SAVE = "`head -5 namelist_update | tail -1` - set secsave = " SECSAVE = "`head -5 namelist_update | tail -1` set f107value = `head -6 namelist_update | tail -1` set f107 = " F107 = ""$f107value" @@ -104,11 +102,9 @@ -e 's/'"`grep 'START' tiegcm.nml.original | head -4 | tail -1`"'/'"$start"'/' \ -e 's/'"`grep 'STOP' tiegcm.nml.original | head -1`"'/'"$stop"'/' \ -e 's/'"`grep 'HIST' tiegcm.nml.original | head -1`"'/'"$hist"'/' \ - -e 's/'"`grep 'SAVE' tiegcm.nml.original | head -1`"'/'"$save"'/' \ -e 's/'"`grep 'SECSTART' tiegcm.nml.original`"'/'"$secstart"'/' \ -e 's/'"`grep 'SECSTOP' tiegcm.nml.original`"'/'"$secstop"'/' \ -e 's/'"`grep 'SECHIST' tiegcm.nml.original`"'/'"$sechist"'/' \ - -e 's/'"`grep 'SECSAVE' tiegcm.nml.original`"'/'"$secsave"'/' \ -e 's/'"`grep 'F107' tiegcm.nml.original | head -1`"'/'"$f107"'/' \ tiegcm.nml.original >! tiegcm.nml.update @@ -121,11 +117,9 @@ -e 's/'"`grep 'START' tiegcm.nml.original | head -4 | tail -1`"'/'"$start"'/' \ -e 's/'"`grep 'STOP' tiegcm.nml.original | head -1`"'/'"$stop"'/' \ -e 's/'"`grep 'HIST' tiegcm.nml.original | head -1`"'/'"$hist"'/' \ - -e 's/'"`grep 'SAVE' tiegcm.nml.original | head -1`"'/'"$save"'/' \ -e 's/'"`grep 'SECSTART' tiegcm.nml.original`"'/'"$secstart"'/' \ -e 's/'"`grep 'SECSTOP' tiegcm.nml.original`"'/'"$secstop"'/' \ -e 's/'"`grep 'SECHIST' tiegcm.nml.original`"'/'"$sechist"'/' \ - -e 's/'"`grep 'SECSAVE' tiegcm.nml.original`"'/'"$secsave"'/' \ tiegcm.nml.original >! tiegcm.nml.update endif Modified: DART/trunk/models/tiegcm/shell_scripts/run_filter.csh =================================================================== --- DART/trunk/models/tiegcm/shell_scripts/run_filter.csh 2013-10-16 23:00:15 UTC (rev 6521) +++ DART/trunk/models/tiegcm/shell_scripts/run_filter.csh 2013-10-16 23:10:19 UTC (rev 6522) @@ -33,8 +33,8 @@ # #BSUB -J filter #BSUB -o filter.%J.log -#BSUB -P 35071364 -#BSUB -q regular +#BSUB -P P3507xxxx +#BSUB -q economy #BSUB -n 64 #BSUB -R "span[ptile=64]" #BSUB -W 3:00 @@ -84,7 +84,7 @@ # Make a unique, (empty, clean) temporary directory. #---------------------------------------------------------------------- -setenv TMPDIR /ptmp/${user}/DART/${JOBNAME}/job_${JOBID} +setenv TMPDIR /glade/scratch/${user}/DART/${JOBNAME}/job_${JOBID} mkdir -p ${TMPDIR} cd ${TMPDIR} @@ -119,9 +119,9 @@ # Set variables containing various directory names where we will GET things #----------------------------------------------------------------------------- -set DARTDIR = /blhome/tmatsuo/DART/models/tiegcm -set TIEGCMDIR = /blhome/tmatsuo/DART/models/tiegcm/tiegcm_files -set EXPERIMENT = /ptmp/tmatsuo/DART/tiegcm/2002_03_28 +set DARTDIR = /glade/u/home/tmatsuo/DART/models/tiegcm +set TIEGCMDIR = /glade/u/home/tmatsuo/DART/models/tiegcm/tiegcm_files +set EXPERIMENT = /glade/scratch/tmatsuo/2002_03_28_tiegcm #----------------------------------------------------------------------------- # Get the DART executables, scripts, and input files Modified: DART/trunk/models/tiegcm/shell_scripts/run_filter_async4.csh =================================================================== --- DART/trunk/models/tiegcm/shell_scripts/run_filter_async4.csh 2013-10-16 23:00:15 UTC (rev 6521) +++ DART/trunk/models/tiegcm/shell_scripts/run_filter_async4.csh 2013-10-16 23:10:19 UTC (rev 6522) @@ -25,13 +25,10 @@ ##============================================================================= #BSUB -J filter #BSUB -o filter.%J.log -#BSUB -P 35071364 +#BSUB -P P3507xxxx #BSUB -q debug #BSUB -n 4 #BSUB -W 0:30 -##BSUB -q regular -##BSUB -n 20 -##BSUB -W 2:00 #BSUB -N -u ${USER}@ucar.edu # ##============================================================================= Modified: DART/trunk/models/tiegcm/shell_scripts/run_perfect_model_obs.csh =================================================================== --- DART/trunk/models/tiegcm/shell_scripts/run_perfect_model_obs.csh 2013-10-16 23:00:15 UTC (rev 6521) +++ DART/trunk/models/tiegcm/shell_scripts/run_perfect_model_obs.csh 2013-10-16 23:10:19 UTC (rev 6522) @@ -33,10 +33,10 @@ # #BSUB -J perfect #BSUB -o perfect.%J.log -#BSUB -P 35071364 -#BSUB -q debug +#BSUB -P P3507xxxx +#BSUB -q economy #BSUB -n 1 -#BSUB -W 6:00 +#BSUB -W 1:00 #BSUB -N -u ${USER}@ucar.edu #---------------------------------------------------------------------- @@ -83,7 +83,7 @@ # Make a unique, (empty, clean) temporary directory. #---------------------------------------------------------------------- -setenv TMPDIR /ptmp/${user}/DART/${JOBNAME}/job_${JOBID} +setenv TMPDIR /glade/scratch/${user}/DART/${JOBNAME}/job_${JOBID} mkdir -p ${TMPDIR} cd ${TMPDIR} @@ -118,9 +118,9 @@ # Set variables containing various directory names where we will GET things #----------------------------------------------------------------------------- -set DARTDIR = /blhome/tmatsuo/DART/models/tiegcm -set TIEGCMDIR = /blhome/tmatsuo/DART/models/tiegcm/tiegcm_files -set EXPERIMENT = /ptmp/tmatsuo/DART/tiegcm/2002_03_28 +set DARTDIR = /glade/u/home/tmatsuo/DART/models/tiegcm +set TIEGCMDIR = /glade/u/home/tmatsuo/DART/models/tiegcm/tiegcm_files +set EXPERIMENT = /glade/scratch/tmatsuo/2002_03_28_tiegcm #----------------------------------------------------------------------------- # Get the DART executables, scripts, and input files @@ -154,7 +154,7 @@ # Remove all the comments that follow (;) symbol from tiegcm.nml namelist file #----------------------------------------------------------------------------- -sed -e 's/;.*//' -e '/^$/ d' tiegcm.nml.original >! tiegcm.nml +grep -v "^;" tiegcm.nml.original >! tiegcm.nml #----------------------------------------------------------------------------- # Check that everything moved OK, and the table is set. From nancy at ucar.edu Thu Oct 17 10:20:30 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 17 Oct 2013 10:20:30 -0600 Subject: [Dart-dev] [6523] DART/trunk/models/cam/shell_scripts: Removing mention of the ' development' branch of DART - these Message-ID: Revision: 6523 Author: thoar Date: 2013-10-17 10:20:29 -0600 (Thu, 17 Oct 2013) Log Message: ----------- Removing mention of the 'development' branch of DART - these should use the 'trunk' (since 'development' is "gone") Moved the bit about modifying the input.nml number of instances from the assimilate scripts to the setup scripts. This is consistent with the way forward and so if anyone wanted to revive these, they would be more likely to work. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM_setup_startup_hopper.csh DART/trunk/models/cam/shell_scripts/CESM_setup_zagar.csh DART/trunk/models/cam/shell_scripts/CESM_setup_zagar_pmo.csh DART/trunk/models/cam/shell_scripts/assimilate.Fzagar.csh DART/trunk/models/cam/shell_scripts/assimilate.hopper.csh DART/trunk/models/cam/shell_scripts/assimilate.ned.csh DART/trunk/models/cam/shell_scripts/assimilate.zagar.csh DART/trunk/models/cam/shell_scripts/setup_cesm_case.ned.csh DART/trunk/models/cam/shell_scripts/setup_cesm_case.pmo.csh -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM_setup_startup_hopper.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM_setup_startup_hopper.csh 2013-10-16 23:10:19 UTC (rev 6522) +++ DART/trunk/models/cam/shell_scripts/CESM_setup_startup_hopper.csh 2013-10-17 16:20:29 UTC (rev 6523) @@ -387,6 +387,16 @@ ${COPY} ${DARTroot}/models/cam/shell_scripts/assimilate.csh . ${COPY} ${DARTroot}/models/cam/work/input.nml . +# Ensure that the input.nml ensemble size matches the number of instances. +# WARNING: the output files contain ALL ensemble members ==> BIG + +ex input.nml < BIG - -ex input.nml < BIG - -ex input.nml < BIG - -ex input.nml < BIG - -ex input.nml < Revision: 6524 Author: thoar Date: 2013-10-17 10:31:35 -0600 (Thu, 17 Oct 2013) Log Message: ----------- These scripts are now officially deprecated. Added Paths: ----------- DART/trunk/models/cam/deprecated/CESM_setup_startup_hopper.csh DART/trunk/models/cam/deprecated/CESM_setup_zagar.csh DART/trunk/models/cam/deprecated/CESM_setup_zagar_pmo.csh DART/trunk/models/cam/deprecated/assimilate.Fzagar.csh DART/trunk/models/cam/deprecated/assimilate.hopper.csh DART/trunk/models/cam/deprecated/assimilate.ned.csh DART/trunk/models/cam/deprecated/assimilate.zagar.csh DART/trunk/models/cam/deprecated/auto_diagPOP2hpss_LSF.csh DART/trunk/models/cam/deprecated/clm.cpl7.template DART/trunk/models/cam/deprecated/clm_FillValue_vars.csh DART/trunk/models/cam/deprecated/input.normal.nml DART/trunk/models/cam/deprecated/input.pert0.nml DART/trunk/models/cam/deprecated/job_mpi_blueice_fragment.csh DART/trunk/models/cam/deprecated/pmo.ned.csh DART/trunk/models/cam/deprecated/setup_cesm_case.hopper.csh DART/trunk/models/cam/deprecated/setup_cesm_case.ned.csh DART/trunk/models/cam/deprecated/setup_cesm_case.pmo.csh Removed Paths: ------------- DART/trunk/models/cam/full_experiment/auto_diagPOP2hpss_LSF.csh DART/trunk/models/cam/full_experiment/clm_FillValue_vars.csh DART/trunk/models/cam/full_experiment/job_mpi_blueice_fragment.csh DART/trunk/models/cam/shell_scripts/CESM_setup_startup_hopper.csh DART/trunk/models/cam/shell_scripts/CESM_setup_zagar.csh DART/trunk/models/cam/shell_scripts/CESM_setup_zagar_pmo.csh DART/trunk/models/cam/shell_scripts/assimilate.Fzagar.csh DART/trunk/models/cam/shell_scripts/assimilate.hopper.csh DART/trunk/models/cam/shell_scripts/assimilate.ned.csh DART/trunk/models/cam/shell_scripts/assimilate.zagar.csh DART/trunk/models/cam/shell_scripts/clm.cpl7.template DART/trunk/models/cam/shell_scripts/input.normal.nml DART/trunk/models/cam/shell_scripts/input.pert0.nml DART/trunk/models/cam/shell_scripts/pmo.ned.csh DART/trunk/models/cam/shell_scripts/setup_cesm_case.hopper.csh DART/trunk/models/cam/shell_scripts/setup_cesm_case.ned.csh DART/trunk/models/cam/shell_scripts/setup_cesm_case.pmo.csh -------------- next part -------------- Copied: DART/trunk/models/cam/deprecated/CESM_setup_startup_hopper.csh (from rev 6523, DART/trunk/models/cam/shell_scripts/CESM_setup_startup_hopper.csh) =================================================================== --- DART/trunk/models/cam/deprecated/CESM_setup_startup_hopper.csh (rev 0) +++ DART/trunk/models/cam/deprecated/CESM_setup_startup_hopper.csh 2013-10-17 16:31:35 UTC (rev 6524) @@ -0,0 +1,676 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# --------------------- +# Purpose +# --------------------- +# +# This script is designed to configure and build a multi-instance CESM model +# that has CAM, CLM, and CICE as active components over a single data ocean, +# and will use DART to assimilate observations at regular intervals. +# This script does not build DART. + +# --------------------- +# How to set up the script +# --------------------- +# -- Either edit and run this script in the $DART/models/cam/shell_scripts +# directory where it first appears, +# or copy it to somewhere that it will be preserved and run it there. +# It will create a 'case' directory, where the model will be built, +# and an execution directory, where each forecast and assimilation will +# take place. The short term archiver will use a third directory for +# storage of model output until it can be moved to long term storage (HPSS) +# -- Examin the whole script to identify things to change for your experiments. +# -- Provide any initial files needed by your run: +# inflation +# sampling error correction +# CAM/CLM/CICE initial ensemble +# ... +# -- Run this script. +# -- Edit the DART input.nml that appears in the $caseroot directory. +# -- Submit the job using $caseroot/${case}.${mach}.submit +# ($mach may not be needed for cesm releases after cesm1_1_beta04) +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime +# settings, you need to delete everything and start the run from scratch. +# +# ./${CASENAME}.*.clean_build +# ./configure -cleanall +# +# ==================================================================== +# === IMPORTANT modifications to the distribution code before ANYTHING +# ==================================================================== + +# The cesm1_1_beta04 lt_archive script did not create parent dirs +# if they did not already exist. To fix the script, edit: +# ${cesmroot}/scripts/ccsm_utils/Tools/lt_archive.csh +# and change 'mkdir' to 'mkdir -p'. This is fixed in more recent +# versions of the code. + +# ==================================================================== +# ==== Set case options +# ==================================================================== + +# case will be used many ways; +# directory and file names, both locally and on HPSS, and +# script names; so consider it's length and information content. +# num_instances: Number of ensemble members +# reuse_existing_case: +# false; Remove $caseroot and $exeroot and rebuild +# true; configure -cleannamelist + +setenv case Exp1 +setenv compset F_2000 +setenv resolution f09_f09 +setenv cesmtag cesm1_1_beta04 +setenv num_instances 4 +setenv reuse_existing_case false + +# ==================================================================== +# define machines and directories +# +# mach Computer name +# cesm_datadir Root path of the public CESM data files +# cesmroot Location of the cesm code base +# For cesm1_1_beta04 on hopper, MUST use 'nscollin' value provided. +# DARTroot Location of DART code tree. +# Executables, scripts and input in $DARTroot/models/cam/... +# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# Preferably not a frequently scrubbed location. +# caseroot will be deleted if reuse_existing_case is false (below) +# So this script, and other useful things should be kept elsewhere. +# exeroot (Future) Run-time directory; scrubbable, large amount of space needed. +# archdir (Future) Short-term archive directory +# until the long-term archiver moves it to permanent storage. +# note: +# hopper organizes the scratch space into 26 subdirs based on the +# first letter of the username, and then below that has the actual +# user's scratch dir. $firstlet below is the first letter of the user name. +# ==================================================================== + +setenv mach hopp2 +setenv cesm_datadir /project/projectdirs/ccsm1/inputdata +setenv cesmroot /global/u1/n/nscollin/${cesmtag} + +set firstlet = `echo $USER | cut -c1` + +setenv DARTroot /global/u1/${firstlet}/${USER}/DART/development + +setenv caseroot /global/u1/${firstlet}/${USER}/cases/${case} +setenv exeroot /scratch/scratchdirs/${USER}/case} +setenv archdir /scratch/scratchdirs/${USER}/archive/${case} + +# ====================== +# configure settings +# ==================================================================== + +# yyyy-mm-dd +setenv run_startdate 2008-10-31 + +setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc +setenv year_start 1850 +setenv year_end 2010 + +# ==================================================================== +# runtime settings -- How many assimilation steps will be done after this one +# +# stop_option Units for determining the forecast length between assimilations +# Changing stop_option requires changes to user_nl_cam below. +# stop_n Number of time units in the forecast +# ==================================================================== + +setenv resubmit 0 +setenv stop_option nhours +setenv stop_n 6 + +# ==================================================================== +# job settings +# +# timewall can be changed during a series by changing the ${case}.${mach}.run +# queue can be changed during a series by changing the ${case}.${mach}.run +# lrg_ queues are used in order to fit more instances on each node. +# FV 1-degree can comfortably fit 4 instances on 1 lrg_ node (~60 gbyte) +# On bluefire the regular queue (or higher) is probably necessary, +# because it appears that there are not separate queues for the lrg memory +# and the regular memory nodes. So economy jobs requesting smaller numbers +# of processors seem to prevent this lrg_economy job (20 nodes for 1-degree) +# from running for long periods. +# ==================================================================== + +setenv timewall 1:00 +setenv queue regular + +# ==================================================================== +# set these standard commands based on the machine you are running on. +# ==================================================================== + +# NERSC "hopper" +set MOVE = '/bin/mv -fv' +set COPY = '/bin/cp -fv --preserve=timestamps' +set LINK = '/bin/ln -fvs' +set REMOVE = '/bin/rm -fr' + + +# ==================================================================== +# Create the case. +# +# For list of the pre-defined cases: ./create_newcase -list +# To create a variant case, see the CESM documentation and carefully +# incorporate any needed changes into this script. +# ==================================================================== + +if ("${reuse_existing_case}" == "false") then + echo "removing old files from ${caseroot} and ${exeroot}" + ${REMOVE} ${caseroot} + ${REMOVE} ${exeroot} + ${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ + -res ${resolution} -compset ${compset} -skip_rundb + + if ( $status != 0 ) then + echo "ERROR: Case could not be created." + exit 1 + endif +else + cd ${caseroot} + ./configure -cleannamelist +endif + +# ==================================================================== +# Configure the case. +# ==================================================================== + +cd ${caseroot} + +./xmlchange -file env_build.xml -id EXEROOT -val ${exeroot} +./xmlchange -file env_build.xml -id USE_ESMF_LIB -val TRUE +./xmlchange -file env_build.xml -id ESMF_LIBDIR -val /scratch/scratchdirs/nscollin/esmf-mpi + +# num_tasks_per_instance = #tasks_node / #instances_node +# hopper: #tasks_node = 4 with threading of 6/task = 24 processors +# #instances_node = 1-degree: 2 on standard memory nodes +# (using lrg mem nodes requires asking for ALL of them) +set num_tasks_per_node = 4 +set num_tasks_per_instance = 2 +set num_threads = 6 +# This is hard-wiring for the current (1/17/2011) multi-instance CESM restriction +# that all instances must be advanced simultaneously. Work is underway to relax that. +@ total_nt = $num_instances * $num_tasks_per_instance +echo "total MPI tasks requested = $total_nt" +echo "num_threads = $num_threads" + +# Atm gets all the nodes and runs. +# Lnd gets all the nodes and runs. +# The other components divide them up. +# ? ? ? What about sglc? It's a stub, and doesn't matter what pes are assigned to it. +# This algorithm figures out whether there are enough processors requested +# to run each component on whole nodes, or the components need to share some nodes. +# The hard-coded numbers (and ratios between them) are estimates; change them if you +# know better. +@ atm_pes = $total_nt +@ large_small = $total_nt / (8 * $num_tasks_per_node) +if ($large_small > 0) then + # Large_small > 0 means there are at least 8 nodes requested. + # Allot whole nodes to the major components. + # hopp2 doesn't use many 'tasks', so it needs a different distribution than on bluefire. + # In particular, we need to keep num_instances > num_tasks for atm, lnd, and ice. + @ docn_pes = $num_tasks_per_node + @ cpl_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node + @ cice_pes = $total_nt - ($cpl_pes + $docn_pes) + @ lnd_pes = $total_nt +else + # 40% cpl, 40% cice, 20% docn, 100% lnd. These may occupy fractions of nodes. + @ cpl_pes = (2 * $total_nt) / 5 + @ cice_pes = (2 * $total_nt) / 5 + @ docn_pes = $total_nt - ($cpl_pes + $cice_pes) + @ lnd_pes = $total_nt +endif + +echo "task layout" +echo "[0 ......................... ATM ............................. $atm_pes]" + +# first pe of each component (counted from 0) +@ atm_rootpe = 0 +@ lnd_rootpe = 0 +@ cpl_rootpe = 0 +@ cice_rootpe = $cpl_rootpe + $cpl_pes +@ docn_rootpe = $cice_rootpe + $cice_pes +echo "[$cpl_rootpe ... CPL ... $cice_rootpe ... ICE ... $docn_rootpe ... OCN ... $total_nt]" +echo "[$lnd_rootpe ... LND ..................................................... $total_nt]" +echo "" +echo "ATM gets $atm_pes" +echo "ICE gets $cice_pes" +echo "LND gets $lnd_pes" +echo "CPL gets $cpl_pes" +echo "OCN gets $docn_pes" +echo "" + +./xmlchange -file env_mach_pes.xml -id NTASKS_ATM -val $atm_pes +./xmlchange -file env_mach_pes.xml -id NTHRDS_ATM -val $num_threads +./xmlchange -file env_mach_pes.xml -id ROOTPE_ATM -val $atm_rootpe +./xmlchange -file env_mach_pes.xml -id NINST_ATM -val $num_instances + +./xmlchange -file env_mach_pes.xml -id NTASKS_CPL -val $cpl_pes +./xmlchange -file env_mach_pes.xml -id NTHRDS_CPL -val $num_threads +./xmlchange -file env_mach_pes.xml -id ROOTPE_CPL -val $cpl_rootpe + +./xmlchange -file env_mach_pes.xml -id NTASKS_ICE -val $cice_pes +./xmlchange -file env_mach_pes.xml -id NTHRDS_ICE -val $num_threads +./xmlchange -file env_mach_pes.xml -id ROOTPE_ICE -val $cice_rootpe +./xmlchange -file env_mach_pes.xml -id NINST_ICE -val $num_instances + +./xmlchange -file env_mach_pes.xml -id NTASKS_OCN -val $docn_pes +./xmlchange -file env_mach_pes.xml -id NTHRDS_OCN -val $num_threads +./xmlchange -file env_mach_pes.xml -id ROOTPE_OCN -val $docn_rootpe + +./xmlchange -file env_mach_pes.xml -id NTASKS_LND -val $lnd_pes +./xmlchange -file env_mach_pes.xml -id NTHRDS_LND -val $num_threads +./xmlchange -file env_mach_pes.xml -id ROOTPE_LND -val $lnd_rootpe +./xmlchange -file env_mach_pes.xml -id NINST_LND -val $num_instances + +./xmlchange -file env_conf.xml -id RUN_TYPE -val startup +./xmlchange -file env_conf.xml -id RUN_STARTDATE -val $run_startdate +./xmlchange -file env_conf.xml -id DOCN_SSTDATA_FILENAME -val $sst_dataset +./xmlchange -file env_conf.xml -id DOCN_SSTDATA_YEAR_START -val $year_start +./xmlchange -file env_conf.xml -id DOCN_SSTDATA_YEAR_END -val $year_end +./xmlchange -file env_conf.xml -id CLM_CONFIG_OPTS -val '-rtm off' +# The river transport model ON is useful only when using an active ocean or +# land surface diagnostics. + +# Substantial archiving changes exist in the Tools/st_archive.sh script. +# Do not change the CALENDAR or the CONTINUE_RUN +# DOUT_S is to turn on/off the short-term archiving +# DOUT_L_MS is to store to the HPSS (formerly "MSS") + +./xmlchange -file env_run.xml -id CONTINUE_RUN -val FALSE +./xmlchange -file env_run.xml -id RESUBMIT -val $resubmit +./xmlchange -file env_run.xml -id STOP_OPTION -val $stop_option +./xmlchange -file env_run.xml -id STOP_N -val $stop_n +./xmlchange -file env_run.xml -id CALENDAR -val GREGORIAN +./xmlchange -file env_run.xml -id DOUT_S_ROOT -val ${archdir} +./xmlchange -file env_run.xml -id DOUT_S -val TRUE +./xmlchange -file env_run.xml -id DOUT_S_SAVE_INT_REST_FILES -val TRUE +./xmlchange -file env_run.xml -id DOUT_L_MS -val TRUE +./xmlchange -file env_run.xml -id DOUT_L_MSROOT -val "csm/${case}" +./xmlchange -file env_run.xml -id DOUT_L_HTAR -val FALSE + +# ==================================================================== +# Create namelist template: user_nl_cam, user_nl_clm +# ==================================================================== + +cd ${caseroot} + +cat <! user_nl_cam +&camexp + inithist = 'ENDOFRUN' + div24del2flag = 4 + empty_htapes = .true. + fincl1 = 'PHIS:I' + nhtfrq = -$stop_n + iradae = -$stop_n + aerodep_flx_datapath = '${cesm_datadir}/atm/cam/chem/trop_mozart_aero/aero' + aerodep_flx_file = 'aerosoldep_monthly_1849-2006_1.9x2.5_c090803.nc' + aerodep_flx_cycle_yr = 2000 + aerodep_flx_type = 'CYCLICAL' +/ +EOF + +cat <! user_nl_clm +&clmexp + fatmgrid = '${cesm_datadir}/lnd/clm2/griddata/griddata_0.9x1.25_070212.nc' + faerdep = '/scratch/scratchdirs/nscollin/cesm_datafiles/aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc' + outnc_large_files = .true. + hist_empty_htapes = .true. +/ +EOF +# faerdep = '${cesm_datadir}/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc' +# This resolution doesn't exist on hopper yet. + +# ==================================================================== +# Update source files if need be +# Ideally, using DART will not require any source mods. +# Until then, this script accesses source mods from a hard-wired location below. +# Those may eventually be packaged into the DART repository. +# If you have additional source mods, they will need to be merged into any DART +# mods and put in the SourceMods subdirectory found in the 'case' directory. +# ==================================================================== +cp /global/u1/n/nscollin/cesm_mods/seq*F90 $caseroot/SourceMods/src.drv +cp /global/u1/n/nscollin/cesm_mods/hist*F90 $caseroot/SourceMods/src.clm +cp /global/u1/n/nscollin/cesm_mods/ccsm_comp_mod.F90 $caseroot/SourceMods/src.drv + +# this one needs a recursive copy to get all the files in the subdirs +#${COPY} -r ~thoar/${cesmtag}/SourceMods/* ${caseroot}/SourceMods/ +#if ( $status == 0) then +# echo "FYI - Local Source Modifications used for this case:" +# ls -lr ${caseroot}/SourceMods/* +#else +# echo "FYI - No SourceMods for this case" +#endif +# +# ==================================================================== +# Configure +# ==================================================================== + +cd ${caseroot} + +./configure -case + +if ( $status != 0 ) then + echo "ERROR: Case could not be configured." + exit 2 +endif + +# ==================================================================== +# Stage the required parts of DART in the caseroot directory. +# ==================================================================== + +cd ${caseroot} + +if ("${reuse_existing_case}" == "false") then + ${MOVE} Tools/st_archive.sh Tools/st_archive.sh.orig +endif +${COPY} ${DARTroot}/models/cam/shell_scripts/st_archive.sh Tools/ + +# The cesm1_1_beta04 release had an error in that it did not +# provide the lt_archive.csh script, and the one in the repos +# did not have the -p flag, which is a good idea. So, for now ... +${COPY} ${cesmroot}/scripts/ccsm_utils/Tools/lt_archive.csh Tools/ + +${COPY} ${DARTroot}/models/cam/shell_scripts/assimilate.csh . +${COPY} ${DARTroot}/models/cam/work/input.nml . + +# Ensure that the input.nml ensemble size matches the number of instances. +# WARNING: the output files contain ALL ensemble members ==> BIG + +ex input.nml <! add_to_run.txt + +# ------------------------------------------------------------------------- +# START OF DART: if CESM finishes correctly (pirated from ccsm_postrun.csh); +# perform an assimilation with DART. +# ------------------------------------------------------------------------- + +set CplLogFile = `ls -1t cpl.log* | head -n 1` +if ($CplLogFile == "") then + echo 'ERROR: Model did not complete - no cpl.log file present - exiting' + echo 'ERROR: Assimilation will not be attempted.' + exit -4 +endif + +grep 'SUCCESSFUL TERMINATION' $CplLogFile +if ( $status == 0 ) then + ${CASEROOT}/assimilate.csh + + if ( $status == 0 ) then + echo "`date` -- DART HAS FINISHED" + else + echo "`date` -- DART FILTER ERROR - ABANDON HOPE" + exit -5 + endif +endif + +# END OF DART BLOCK +# ------------------------------------------------------------------------- + +"EndOfText" + +# Now that the "here" document is created, +# determine WHERE to insert it -- ONLY IF it is not already there. + +grep "ABANDON HOPE" ${case}.${mach}.run +set STATUSCHECK = $status + +if ( ${STATUSCHECK} == 0 ) then + echo "DART block already present in ${case}.${mach}.run" +else if ( ${STATUSCHECK} == 1 ) then + + set MYSTRING = `grep --line-number "CSM EXECUTION HAS FINISHED" ${case}.${mach}.run` + set MYSTRING = `echo $MYSTRING | sed -e "s#:# #g"` + + @ origlen = `cat ${case}.${mach}.run | wc -l` + @ keep = $MYSTRING[1] + @ lastlines = $origlen - $keep + + mv ${case}.${mach}.run ${case}.${mach}.run.orig + + head -n $keep ${case}.${mach}.run.orig >! ${case}.${mach}.run + cat add_to_run.txt >> ${case}.${mach}.run + tail -n $lastlines ${case}.${mach}.run.orig >> ${case}.${mach}.run + +else + echo "ERROR in grep of ${case}.${mach}.run: aborting" + echo "status was ${STATUSCHECK}" + exit 10 +endif + +# ==================================================================== +# Edit the run script to reflect project, queue, and wallclock +# ==================================================================== + +echo '' +echo 'Updating the run script to set the project number, wallclock time' +echo 'and queue name.' +echo '' + +if ($?proj) then + set PROJ=`grep '^#PBS ' $case.$mach.run | grep -e '-P' ` + sed -e s/$PROJ[3]/$proj/ < $case.$mach.run >! temp + /bin/mv temp $case.$mach.run +endif + +if ($?timewall) then + set TIMEWALL=`grep '^#PBS ' $case.$mach.run | grep walltime ` + sed -e /"${TIMEWALL}"/s/=.\$/=$timewall/ < $case.$mach.run >! temp + /bin/mv temp $case.$mach.run +endif + +if ($?queue) then + set QUEUE=`grep '^#PBS ' $case.$mach.run | grep -e '-q' ` + sed -e s/$QUEUE[3]/$queue/ < $case.$mach.run >! temp + /bin/mv temp $case.$mach.run +endif + +chmod 0755 ${case}.${mach}.run + +# ==================================================================== +# IMPORTANT: All resubmits must be type 'startup'. +# Change Tools/ccsm_postrun.csh line 83 to CONTINUE_RUN -val FALSE' +# ==================================================================== + +cd ${caseroot}/Tools + +echo '' +echo 'Changing Tools/ccsm_postrun.csh such that all the resubmits are "startup",' +echo 'which means CONTINUE_RUN should be FALSE in ccsm_postrun.csh' +echo '' + +ex ccsm_postrun.csh < +# $URL$ +# $Revision$ +# $Date$ + Copied: DART/trunk/models/cam/deprecated/CESM_setup_zagar.csh (from rev 6523, DART/trunk/models/cam/shell_scripts/CESM_setup_zagar.csh) =================================================================== --- DART/trunk/models/cam/deprecated/CESM_setup_zagar.csh (rev 0) +++ DART/trunk/models/cam/deprecated/CESM_setup_zagar.csh 2013-10-17 16:31:35 UTC (rev 6524) @@ -0,0 +1,464 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# --------------------- +# Purpose +# --------------------- +# +# The real purpose of this set of notes is to record what is needed to configure +# and build a CESM instance that has CAM, CLM, and CICE as active components +# in a multi-instance configuration over a single data ocean ... etc. +# Despite looking like a script, it might best be used as a set of notes. +# --------------------- +# How to set the script +# --------------------- +# -- Copy this script into your directory +# -- Choose a case name (by changing "setenv case" ) and save the script as $case.csh +# -- Set the case options at the top of the script +# -- If you have source mods, the script assumes that your mods are in: mods_$case. +# So, if you have source mods, create a subdirectory mods_$case that contains your mods. +# If you don t have any source mods, the script creates an empty subdirectory mods_$case. +# -- If you have namelist mods, you need to add them to the namelist template: user_nl_cam_$case +# Set your namelist variables over there (without modifying the syntax to create user_nl_cam_$case +# -- Now, you are ready to go. Save your script and submit your run with the command: ./$case.csh +# The script creates your case, configure, compile and submit your job. +# -- The script also creates a subdirectory (nml_$case) that contains your namelists. +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime settings, +# you need to delete everything and start the run from scratch. +# +# ./${CASENAME}.*.clean_build +# ./configure -cleanall +# +# ==================================================================== +# === IMPORTANT modifications to the distribution code before ANYTHING +# ==================================================================== + +# had to edit the following to remove the LSB_PJL_... word too long error +# cesm1_1_beta04/scripts/ccsm_utils/Machines/mkbatch.bluefire +# as long as OMP_NUM_THREADS == 1 ... the default is fine. + +# ==================================================================== +# ==== Set case options +# ==================================================================== + +setenv case Fzagar +setenv compset F_2000 +setenv resolution f09_f09 +setenv ccsmtag cesm1_1_beta04 +setenv num_instances 80 +setenv coldbuild false + +# ================================ +# define machines and directories +# ================================ + +setenv mach bluefire ;# machine +setenv DARTdir /glade/home/thoar/svn/DART/trunk ;# DART executables, scripts and input + +setenv cesm_datadir /glade/proj3/cseg/inputdata +setenv cesm_public /glade/proj3/cseg ;# location of public CESM sandbox +setenv ccsmroot ${cesm_public}/collections/${ccsmtag} ;# location of the public cesm code +setenv ccsmroot /glade/home/${USER}/${ccsmtag} ;# location of your personal cesm code +setenv caseroot /glade/user/${USER}/cases/${case} ;# your (future) cesm case directory +setenv rundir /glade/scratch/${USER}/${case} ;# (future) run-time directory +setenv archdir /glade/scratch/${USER}/archive/${case} ;# (future) short-term archive directory + +# ====================== +# configure settings +# ====================== + +setenv run_startdate 2008-08-01 + +setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc +setenv year_start 1850 +setenv year_end 2010 + +# ====================== +# runtime settings +# ====================== + +setenv resubmit 0 +setenv stop_n 12 +setenv stop_option nhours + +# ====================== +# job settings +# ====================== + +setenv proj 93300315 +setenv timewall 2:30 +setenv queue lrg_regular + +# ====================== +# namelist variables +# ====================== +# Create namelist templates that get copied once the case has been created. + +setenv this_dir `pwd` + +cat <! user_nl_cam_${case} +&camexp + inithist = 'ENDOFRUN' + div24del2flag = 4 + aerodep_flx_datapath = '${cesm_datadir}/atm/cam/chem/trop_mozart_aero/aero' + aerodep_flx_file = 'aerosoldep_monthly_1849-2006_1.9x2.5_c090803.nc' + aerodep_flx_cycle_yr = 2000 + aerodep_flx_type = 'CYCLICAL' + iradae = -12 +/ +EOF + +# these cause problems when running with the full cesm: +# empty_htapes = .true. +# nhtfrq = -12 + +cat <! user_nl_clm_${case} +&clmexp + fatmgrid = '${cesm_datadir}/lnd/clm2/griddata/griddata_0.9x1.25_070212.nc' + faerdep = '${cesm_datadir}/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc' + outnc_large_files = .true. +/ +EOF + +# these cause problems when running with the full cesm: +# hist_nhtfrq = -12 +# hist_empty_htapes = .true. + +# ==================================================================== +# Create the case. +# For list of the cases: ./create_newcase -list +# ==================================================================== + +# if coldbuild is false and the directory does not exist, ... + +if ("${coldbuild}" == "true") then + echo "removing old files from ${caseroot} and ${rundir}" + \rm -fr ${caseroot} + \rm -fr ${rundir} + ${ccsmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ + -res ${resolution} -compset ${compset} -skip_rundb + + if ( $status != 0 ) then + echo "ERROR: Case could not be created." + exit 1 + endif +else + cd ${caseroot} + ./configure -cleannamelist +endif + +# ==================================================================== +# Configure the case. +# ==================================================================== + +cd ${caseroot} + +./xmlchange -file env_build.xml -id EXEROOT -val ${rundir} +./xmlchange -file env_build.xml -id USE_ESMF_LIB -val TRUE +#./xmlchange -file env_build.xml -id ESMF_LIBDIR -val ${nancy_scratch}/esmf-mpi + +set num_tasks_per_instance = 16 +set nthreads = 1 +@ total_nt = $num_instances * $num_tasks_per_instance + +./xmlchange -file env_mach_pes.xml -id NTASKS_ATM -val $total_nt +./xmlchange -file env_mach_pes.xml -id NTHRDS_ATM -val $nthreads +./xmlchange -file env_mach_pes.xml -id ROOTPE_ATM -val 0 +./xmlchange -file env_mach_pes.xml -id NINST_ATM -val $num_instances + +./xmlchange -file env_mach_pes.xml -id NTASKS_LND -val $total_nt +./xmlchange -file env_mach_pes.xml -id NTHRDS_LND -val $nthreads +./xmlchange -file env_mach_pes.xml -id ROOTPE_LND -val 0 +./xmlchange -file env_mach_pes.xml -id NINST_LND -val $num_instances + +./xmlchange -file env_mach_pes.xml -id NTASKS_ICE -val $total_nt +./xmlchange -file env_mach_pes.xml -id NTHRDS_ICE -val $nthreads +./xmlchange -file env_mach_pes.xml -id ROOTPE_ICE -val 0 +./xmlchange -file env_mach_pes.xml -id NINST_ICE -val $num_instances + +./xmlchange -file env_conf.xml -id RUN_TYPE -val startup +./xmlchange -file env_conf.xml -id RUN_STARTDATE -val $run_startdate +./xmlchange -file env_conf.xml -id DOCN_SSTDATA_FILENAME -val $sst_dataset +./xmlchange -file env_conf.xml -id DOCN_SSTDATA_YEAR_START -val $year_start +./xmlchange -file env_conf.xml -id DOCN_SSTDATA_YEAR_END -val $year_end + +#./xmlchange -file env_conf.xml -id CLM_CONFIG_OPTS -val '-rtm off' + +./xmlchange -file env_run.xml -id RESUBMIT -val $resubmit +./xmlchange -file env_run.xml -id STOP_OPTION -val $stop_option +./xmlchange -file env_run.xml -id STOP_N -val $stop_n +./xmlchange -file env_run.xml -id CALENDAR -val GREGORIAN + +# Substantial archiving changes exist in the Tools/st_archive.sh script. +# DOUT_S is to turn on/off the short-term archiving +# DOUT_L_MS is to store to the HPSS (formerly "MSS") +./xmlchange -file env_run.xml -id DOUT_S_ROOT -val ${archdir} +./xmlchange -file env_run.xml -id DOUT_S -val TRUE +./xmlchange -file env_run.xml -id DOUT_S_SAVE_INT_REST_FILES -val TRUE +./xmlchange -file env_run.xml -id DOUT_L_MS -val TRUE +./xmlchange -file env_run.xml -id DOUT_L_HTAR -val TRUE + +# ==================================================================== +# Create namelist template: user_nl_cam, user_nl_clm +# ==================================================================== + +\mv -f ${this_dir}/user_nl_cam_{$case} ${caseroot}/user_nl_cam +\mv -f ${this_dir}/user_nl_clm_{$case} ${caseroot}/user_nl_clm + +# ==================================================================== +# Update source files if need be +# ==================================================================== + +\cp -rf ~thoar/${ccsmtag}/SourceMods/* ${caseroot}/SourceMods/ +if ( $status == 0) then + echo "FYI - Local Source Modifications used for this case:" + ls -lr ${caseroot}/SourceMods/* +else + echo "FYI - No SourceMods for this case" +endif + +# ==================================================================== +# Configure +# ==================================================================== + +cd ${caseroot} + +./configure -case + +if ( $status != 0 ) then + echo "ERROR: Case could not be configured." + exit 2 +endif + +# ==================================================================== +# Stage a copy of the DART assimilate.csh script HERE +# ==================================================================== + +cd ${caseroot} + +\mv Tools/st_archive.sh Tools/st_archive.sh.org +\cp -f ${DARTdir}/models/cam/shell_scripts/st_archive.sh Tools/st_archive.sh +\cp -f ${ccsmroot}/scripts/ccsm_utils/Tools/lt_archive.csh Tools/lt_archive.csh +# cesm_1_1_beta04 did not have a lt_archive.sh in the tag ... oops. + +\cp -f ${DARTdir}/models/cam/shell_scripts/assimilate.Fzagar.csh assimilate.csh + +# ==================================================================== +# Update the scripts that build the namelists. +# The active components scripts need to support the multi-instance naming. +# ==================================================================== + +echo '' +echo 'Editing the Buildconf/{cam,cice,clm}.buildnml.csh files' +echo '' + +cd ${caseroot}/Buildconf + +cp -f cam.buildnml.csh cam.buildnml.csh.org +cp -f cice.buildnml.csh cice.buildnml.csh.org +cp -f clm.buildnml.csh clm.buildnml.csh.org + +# The CAM buildnml script only needs changing in one place. + +ex cam.buildnml.csh <! add_to_run.txt + +# ------------------------------------------------------------------------- +# START OF DART: if CESM finishes correctly (pirated from ccsm_postrun.csh); +# perform an assimilation with DART. +# ------------------------------------------------------------------------- + +set CplLogFile = `ls -1t cpl.log* | head -1` +if ($CplLogFile == "") then + echo 'ERROR: Model did not complete - no cpl.log file present - exiting' + echo 'ERROR: Assimilation will not be attempted.' + exit -4 +endif + +grep 'SUCCESSFUL TERMINATION' $CplLogFile +if ( $status == 0 ) then + ${CASEROOT}/assimilate.csh + + if ( $status == 0 ) then + echo "`date` -- DART HAS FINISHED" + else + echo "`date` -- DART FILTER ERROR - ABANDON HOPE" + exit -5 + endif +endif + +# END OF DART BLOCK +# ------------------------------------------------------------------------- + +"EndOfText" + +# Now that the "here" document is created, +# determine WHERE to insert it. + +set MYSTRING = `grep --line-number "CSM EXECUTION HAS FINISHED" ${case}.${mach}.run` +set MYSTRING = `echo $MYSTRING | sed -e "s#:# #g"` + +@ orglen = `cat ${case}.${mach}.run | wc -l` +@ keep = $MYSTRING[1] +@ lastlines = $orglen - $keep + +mv ${case}.${mach}.run ${case}.${mach}.run.orig + +head -$keep ${case}.${mach}.run.orig >! ${case}.${mach}.run +cat add_to_run.txt >> ${case}.${mach}.run +tail -$lastlines ${case}.${mach}.run.orig >> ${case}.${mach}.run + +# ==================================================================== +# IMPORTANT: All resubmits must be type 'startup'. @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Thu Oct 17 10:37:13 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 17 Oct 2013 10:37:13 -0600 Subject: [Dart-dev] [6525] DART/trunk/models: These scripts are now identical. Message-ID: Revision: 6525 Author: thoar Date: 2013-10-17 10:37:13 -0600 (Thu, 17 Oct 2013) Log Message: ----------- These scripts are now identical. Removed an unneeded 'cd rundir' from the cam_no_assimilate.csh ... it already checked for the existence of an rpointer file - which only exist in the rundir. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh Added Paths: ----------- DART/trunk/models/cam/shell_scripts/no_assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh 2013-10-17 16:31:35 UTC (rev 6524) +++ DART/trunk/models/CESM/shell_scripts/cam_no_assimilate.csh 2013-10-17 16:37:13 UTC (rev 6525) @@ -44,8 +44,6 @@ # the short-term archiver 'restores' the CESM files, the links are right. #========================================================================= -cd ${RUNDIR} - set member = 1 while ( ${member} <= ${ensemble_size} ) Added: DART/trunk/models/cam/shell_scripts/no_assimilate.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/no_assimilate.csh (rev 0) +++ DART/trunk/models/cam/shell_scripts/no_assimilate.csh 2013-10-17 16:37:13 UTC (rev 6525) @@ -0,0 +1,68 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. + +echo "`date` -- BEGIN CAM_NO_ASSIMILATE" + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set LINK = '/usr/local/bin/ln -fvs' + breaksw + + default: + set LINK = 'ln -fvs' + breaksw +endsw + +set ensemble_size = ${NINST_ATM} + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name of first member +# of the form "./${CASE}.cam_${ensemble_member}.i.2000-01-06-00000.nc" +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.atm_0001` +set FILE = $FILE:t +set FILE = $FILE:r +set ATM_DATE_EXT = `echo $FILE:e` + +#========================================================================= +# As implemented, the input filenames are static in the CESM namelists. +# We must link the new uniquely-named files to static names so that when +# the short-term archiver 'restores' the CESM files, the links are right. +#========================================================================= + +set member = 1 +while ( ${member} <= ${ensemble_size} ) + + set inst_string = `printf _%04d $member` + + set ATM_INITIAL_FILENAME = ${CASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc + + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 + + @ member++ + +end + +echo "`date` -- END CAM_NO_ASSIMILATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/cam/shell_scripts/no_assimilate.csh ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native From nancy at ucar.edu Thu Oct 17 15:08:59 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 17 Oct 2013 15:08:59 -0600 Subject: [Dart-dev] [6527] DART/trunk/models/CESM/shell_scripts: This script enforces the use of the CESM_DART_config because the CAM Message-ID: Revision: 6527 Author: thoar Date: 2013-10-17 15:08:59 -0600 (Thu, 17 Oct 2013) Log Message: ----------- This script enforces the use of the CESM_DART_config because the CAM strategy requires it. The CESM_DART_config script is now automatically run during CESM1_1_1_hybrid and runs the no_assimilate.csh script to relink the latest CAM initial files to the required static names after each model advance. A new script 'stage_cesm_files' is created in the caseroot directory that can be used to help restart a failed run. The CESM_DART_config script copies the bits required for a perfect_model experiment as well as the no_assimilate experiment. Modified Paths: -------------- DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid DART/trunk/models/CESM/shell_scripts/CESM_DART_config Added Paths: ----------- DART/trunk/models/CESM/shell_scripts/no_assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid =================================================================== --- DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid 2013-10-17 17:40:33 UTC (rev 6526) +++ DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid 2013-10-17 21:08:59 UTC (rev 6527) @@ -26,13 +26,15 @@ # data volume is quite large and you should become comfortable using CESM's restart # capability to re-stage files in your RUN directory # +# CESM/DART requires some modifications to the CESM source code EVEN IF YOU ARE NOT +# assimilating. The modifications for CAM require a modification to the +# CESM ${CASE}.run script. The modifications will invoke a DART script that will +# either assimilate or 'do nothing'. It is stronly recommended that you configure +# the ${CASE}.run script to 'do nothing' initially. +# # CASEROOT/CESM_DART_config will augment the CESM case with the required setup -# and configuration to perform a DART assimilation. CASEROOT/CESM_DART_config -# will insert a few dozen lines into the ${CASE}.run script after it makes a backup -# copy. This, and the required setup, can be run at a later date. e.g. you can -# advance an ensemble from 2004-01-01 to 2004-02-01 and then run -# CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, -# and start assimilating observations when CESM stops at 2004-02-01 06Z ... +# and configuration to perform a DART assimilation and modify the ${CASE}.run script +# as mentioned previously - after it makes a backup copy. # # This script relies heavily on the information in: # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html @@ -59,7 +61,7 @@ # # -- Run this script. # -# -- If you want to run DART; read, understand, and execute ${CASEROOT}/CESM_DART_config +# -- Read, understand, and execute ${CASEROOT}/CESM_DART_config # # -- Submit the job using ${CASEROOT}/${CASE}.submit # @@ -135,11 +137,10 @@ # THIS IS THE LOCATION of the 'reference case'. -set stagedir = /glade/p/image/CESM_initial_ensemble/rest/2004-01-10-00000 +setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/2004-01-10-00000 # ============================================================================== -# runtime settings -- How many assimilation steps will be done after this one -# plus archiving options +# runtime settings # # resubmit How many job steps to run on continue runs (will be 0 initially) # stop_option Units for determining the forecast length between assimilations @@ -270,11 +271,10 @@ ${MOVE} temp.$$ ${caseroot}/CESM_DART_config chmod 755 ${caseroot}/CESM_DART_config else - echo "WARNING: the script to configure for data assimilation is not available." - echo " CESM_DART_config should be present locally or in" - echo " ${dartroot}/models/CESM/shell_scripts/" - echo " You can stage this script later, but you must manually edit it" - echo " to reflect the location of the DART code tree." + echo "ERROR: the script to configure for data assimilation is not available." + echo " CESM_DART_config MUST be present locally or in" + echo " ${dartroot}/models/CESM/shell_scripts/" + exit -2 endif # ============================================================================== @@ -378,6 +378,7 @@ ./xmlchange RUN_REFCASE=$run_refcase ./xmlchange RUN_REFDATE=$run_refdate ./xmlchange RUN_REFTOD=$run_reftod +./xmlchange BRNCH_RETAIN_CASENAME=FALSE ./xmlchange GET_REFCASE=FALSE ./xmlchange EXEROOT=${exeroot} @@ -390,8 +391,16 @@ ./xmlchange PIO_TYPENAME=pnetcdf -# This sets the ocean coupling time to 6 hours. +# COUPLING discussion. +# http://bugs.cgd.ucar.edu/show_bug.cgi?id=1740 +# "In summary, users should ensure that the following is true, +# ATM_NCPL = LND_NCPL = ICE_NCPL >= ROF_NCPL >= OCN_NCPL" +# +# OCN_NCPL == 4 sets the ocean coupling time to 6 hours. # All related namelist settings are based on this value. +# OCN_NCPL is # coupling intervals per NCPL_BASE_PERIOD (default 'day') + +./xmlchange ROF_NCPL=8 ./xmlchange OCN_NCPL=4 # There is no compset that has everything we want, @@ -461,12 +470,18 @@ breaksw endsw +# This is the part that modifies the run script to allow CESM to advance +# correctly given the modifications necessary for CAM. It also copies several +# required DART files to the caseroot directory. + +./CESM_DART_config + # ============================================================================== # Update source files. # Ideally, using DART would not require any modifications to the model source. # Until then, this script accesses sourcemods from a hardwired location. # If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'case' directory. +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. # ============================================================================== if ( -d ~/${cesmtag}/SourceMods ) then @@ -536,20 +551,20 @@ # following the CESM strategy for 'inst_string' set inst_string = `printf _%04d $inst` - # =========================================================================== set fname = "user_nl_cam${inst_string}" # =========================================================================== # For a HOP TEST ... empty_htapes = .false. # For a HOP TEST ... use a default fincl1 - # FIXME ... add documentation for configuring CAM history files + # inithist == 'ENDOFRUN' ensures that CAM writes an initial file every time it stops. + # fincl1,nhtfrq,mfilt all control the history file containing a REQUIRED PHIS field. - echo " inithist = '6-HOURLY'" >> ${fname} - echo " ncdata = 'cam_initial${inst_string}.nc'" >> ${fname} - echo " empty_htapes = .true. " >> ${fname} - echo " fincl1 = 'PHIS:I' " >> ${fname} - echo " nhtfrq = -$assim_n " >> ${fname} - echo " mfilt = 1 " >> ${fname} + echo " inithist = 'ENDOFRUN'" >> ${fname} + echo " ncdata = 'cam_initial${inst_string}.nc'" >> ${fname} + echo " empty_htapes = .true. " >> ${fname} + echo " fincl1 = 'PHIS:I' " >> ${fname} + echo " nhtfrq = -$assim_n " >> ${fname} + echo " mfilt = 1 " >> ${fname} # =========================================================================== set fname = "user_nl_clm${inst_string}" @@ -587,18 +602,21 @@ # =========================================================================== # POP Namelists - # GIVEN: init_ts_option = 'ccsm_hybrid' when RUN_TYPE=hybrid, then - # - # init_ts_suboption = 'data_assim' --> non bit-for-bit restarting (assimilation mode) + # init_ts_suboption = 'data_assim' for non bit-for-bit restarting (assimilation mode) # init_ts_suboption = 'rest' --> default behavior # - # No matter the setting of init_ts_suboption, POP always uses the - # 'data_assim' behavior when RUN_TYPE=hybrid and CONTINUE_RUN=FALSE. - # As soon as CONTINUE_RUN=TRUE, the value becomes important. + # README: + # Configuring the contents of the history files for POP is best explained in + # the section marked "POP2: TAVG Settings" in the cesm1_1_1 pop2 namelist documentation + # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/modelnl/nl_pop2.html # - # For a HOP TEST ... Would like to have restart files every day, not just for end. - # For a HOP TEST (untested)... tavg_file_freq_opt = 'nmonth' 'nday' 'once'" + # and the CESM-specific documentation for the tavg output variables in the pop2 + # online documentation: + # http://www.cesm.ucar.edu/models/cesm1.1/pop2/doc/users/node78.html # + # In CESM1_1_1 keep the values for tavg_file_freq_opt and tavg_freq_opt identical. + # pop2/trunk_tags/cesm_pop_2_1_20130412 explains the issue. + # # DEFAULT values for these are: # tavg_file_freq_opt = 'nmonth' 'nmonth' 'once' # tavg_freq_opt = 'nmonth' 'nday' 'once' @@ -606,14 +624,10 @@ # The second entry indicates we get a monthly average as it is being created. # The third entry indicates we get a daily timeslice # - # IFF values for these are: - # tavg_file_freq_opt = 'nmonth' 'never' 'never' - # tavg_freq_opt = 'nmonth' 'never' 'never' - # The first entry indicates we get a monthly average once a month, and thats all we get.. - + # Default copies of SourceMods/src.pop2/ocn.*.tavg.csh files are provided in the + # DART_SourceMods_cesm1_1_1.tar bundle. + echo "init_ts_suboption = 'data_assim'" >> $fname - echo "tavg_file_freq_opt = 'nmonth' 'never' 'never'" >> $fname - echo "tavg_freq_opt = 'nmonth' 'never' 'never'" >> $fname # =========================================================================== set fname = "user_nl_cice${inst_string}" @@ -639,36 +653,133 @@ # ============================================================================== # Stage the restarts now that the run directory exists # THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. -# POP/CAM/CICE read from pointer files. The others use namelist values initially. +# POP reads from pointer files. The others use namelist values initially. # ============================================================================== -cat << EndOfText >! stage_initial_cesm_files -#!/bin/sh +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. -cd ${rundir} - +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' echo '' -echo 'Copying the restart files from the staging directory' -echo 'into the CESM run directory and relinking the CAM static names.' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' echo '' -${COPY} ${stagedir}/* . +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} -let inst=1 -while ((\$inst <= $num_instances)); do - inst_string=\`printf _%04d \$inst\` +set restart_time = $RUN_REFDATE-$RUN_REFTOD - ${LINK} ${stagedir}/${run_refcase}.cam\${inst_string}.i.${run_refdate}-${run_reftod}.nc cam_initial\${inst_string}.nc +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- - let inst+=1 -done +echo "continue_run is \$CONTINUE_RUN" +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + + # the restarts should be in the short term archive directory + # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + + @ inst=1 + while (\$inst <= $num_instances) + + set inst_string = \`printf _%04d \$inst\` + + echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${CASE}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} + echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} + echo "${CASE}.rtm\${inst_string}.r.\${restart_time}.nc" >! rpointer.rof\${inst_string} + echo "${CASE}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + + echo "${CASE}.pop\${inst_string}.ro.\${restart_time}" >! rpointer.ocn\${inst_string}.ovf + echo "${CASE}.pop\${inst_string}.r.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.restart + echo "RESTART_FMT=nc" >> rpointer.ocn\${inst_string}.restart + + if ( -e rpointer.ocn\${inst_string}.tavg ) then + echo "${CASE}.pop\${inst_string}.rh.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.tavg + endif + if ( -e rpointer.ocn\${inst_string}.tavg.2 ) then + echo "${CASE}.pop\${inst_string}.rh.nday1.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.tavg.2 + endif + + @ inst ++ + end + + endif + + # Now that we're guaranteed to have the cam initial file, link to our hardwired name. + @ inst=1 + while (\$inst <= $num_instances) + set inst_string = \`printf _%04d \$inst\` + ${LINK} ${CASE}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc + @ inst ++ + end + + exit 0 +endif + +#---------------------------------------------------------------------- +# This block prepares the initial run directory for a startup run. +# The CLM, CICE, ROF namelists specify a relative path for the initial file. +# The CAM namelist specifies a static name. +# POP uses pointer files. +# +# The reference case may more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - in a hybrid case - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +set filedir = ${stagedir} + +@ inst=1 +while (\$inst <= $num_instances) + + set inst_string = \`printf _%04d \$inst\` + + echo '' + echo "Staging restarts for instance \$inst of $num_instances" + + ${LINK} \${filedir}/${run_refcase}.clm2\${inst_string}.r.\${restart_time}.nc . + ${LINK} \${filedir}/${run_refcase}.cice\${inst_string}.r.\${restart_time}.nc . + ${LINK} \${filedir}/${run_refcase}.rtm\${inst_string}.r.\${restart_time}.nc . + ${LINK} \${filedir}/${run_refcase}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc + ${LINK} \${filedir}/${run_refcase}.pop\${inst_string}.r.\${restart_time}.nc . + ${LINK} \${filedir}/${run_refcase}.pop\${inst_string}.ro.\${restart_time} . + + echo "${run_refcase}.pop\${inst_string}.ro.\${restart_time}" >! rpointer.ocn\${inst_string}.ovf + echo "${run_refcase}.pop\${inst_string}.r.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.restart + echo "RESTART_FMT=nc" >> rpointer.ocn\${inst_string}.restart + + @ inst ++ +end + exit 0 EndOfText -chmod 0755 stage_initial_cesm_files +chmod 0755 stage_cesm_files -./stage_initial_cesm_files +./stage_cesm_files # ============================================================================== # build @@ -701,25 +812,36 @@ echo ' #BSUB -R "select[scratch_ok > 0]"' echo " around line 9, delete it." echo "" -echo "3) If you want to assimilate 'right away', configure and execute" -echo " the ${caseroot}/CESM_DART_config script." +echo "3) For reasons not clear to me, the very first advance (i.e. STOP_N)" +echo " (CONTINUE_RUN=FALSE) must be longer than *AT LEAST 2 TIMES* the coupling" +echo " frequency between the atmosphere and ocean." +echo " If coupling once a day, the first advance MUST be at least 48 hours." +echo " If coupling 4 times a day, the first advance MUST be at least 12 hours." +echo " After that, STOP_N can be as short as a single coupling frequency." echo "" -echo "3) Verify the contents of env_run.xml and submit the CESM job:" +echo "4) If you want to simply advance CESM without using DART, make sure the" +echo " ${case}.run script references the 'no_assimilate.csh' script." +echo " If you want to assimilate, reference the 'assimilate.csh' script." +echo " The default action is to assimilate." +echo "" +echo "5) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "4) After the job has run, check to make sure it worked and that" +echo "6) After the job has run, check to make sure it worked and that" echo " a: POP is creating netCDF restart files," echo " b: the right restart files exist in the run directory," echo " c: (if you're running DART) the archive dart/hist directory has the DART output," echo " d: everything is working correctly ..." echo "" -echo "5) To extend the run in $assim_n '"$stop_option"' steps," +echo "7) To extend the run in $assim_n '"$stop_option"' steps," echo " change the env_run.xml variables:" echo "" echo " ./xmlchange CONTINUE_RUN=TRUE" echo " ./xmlchange RESUBMIT=" echo " ./xmlchange STOP_N=$assim_n" echo "" +echo " and" +echo " ./${case}.submit" exit 0 Modified: DART/trunk/models/CESM/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/CESM/shell_scripts/CESM_DART_config 2013-10-17 17:40:33 UTC (rev 6526) +++ DART/trunk/models/CESM/shell_scripts/CESM_DART_config 2013-10-17 21:08:59 UTC (rev 6527) @@ -134,7 +134,7 @@ # ============================================================================== # The standard CESM short-term archiving script may need to be altered -# to archive addtional or subsets of things, or to reduce the amount of +# to archive additional or subsets of things, or to reduce the amount of # data that is sent to the long-term archive. Put a version of st_archive.sh # in ${DARTROOT}/models/CESM/shell_scripts when/if necessary @@ -146,12 +146,21 @@ endif ${COPY} ${DARTROOT}/models/CESM/shell_scripts/st_archive.sh Tools/ +${COPY} ${DARTROOT}/shell_scripts/shell_exit.sh . + ${COPY} ${DARTROOT}/models/CESM/shell_scripts/assimilate.csh . -${COPY} ${DARTROOT}/shell_scripts/shell_exit.sh . ${COPY} ${DARTROOT}/models/CESM/shell_scripts/cam_assimilate.csh . ${COPY} ${DARTROOT}/models/CESM/shell_scripts/pop_assimilate.csh . ${COPY} ${DARTROOT}/models/CESM/shell_scripts/clm_assimilate.csh . +${COPY} ${DARTROOT}/models/CESM/shell_scripts/perfect_model.csh . +${COPY} ${DARTROOT}/models/CESM/shell_scripts/cam_perfect_model.csh . +${COPY} ${DARTROOT}/models/CESM/shell_scripts/clm_perfect_model.csh . +${COPY} ${DARTROOT}/models/CESM/shell_scripts/pop_perfect_model.csh . + +${COPY} ${DARTROOT}/models/CESM/shell_scripts/no_assimilate.csh . +${COPY} ${DARTROOT}/models/CESM/shell_scripts/cam_no_assimilate.csh . + # ============================================================================== # Stage the DART executables in the CESM execution root directory: EXEROOT # If you recompile the DART code (maybe to support more observation types) @@ -177,27 +186,31 @@ fi done -${COPY} ${DARTROOT}/models/cam/work/cam_to_dart ${EXEROOT}/. -${COPY} ${DARTROOT}/models/cam/work/dart_to_cam ${EXEROOT}/. -${COPY} ${DARTROOT}/models/cam/work/filter ${EXEROOT}/filter_cam -${COPY} ${DARTROOT}/models/cam/work/input.nml cam_input.nml +${COPY} ${DARTROOT}/models/cam/work/cam_to_dart ${EXEROOT}/. +${COPY} ${DARTROOT}/models/cam/work/dart_to_cam ${EXEROOT}/. +${COPY} ${DARTROOT}/models/cam/work/filter ${EXEROOT}/filter_cam +${COPY} ${DARTROOT}/models/cam/work/perfect_model_obs ${EXEROOT}/perfect_model_obs_cam +${COPY} ${DARTROOT}/models/cam/work/input.nml cam_input.nml -${COPY} ${DARTROOT}/models/clm/work/clm_to_dart ${EXEROOT}/. -${COPY} ${DARTROOT}/models/clm/work/dart_to_clm ${EXEROOT}/. -${COPY} ${DARTROOT}/models/clm/work/filter ${EXEROOT}/filter_clm -${COPY} ${DARTROOT}/models/clm/work/input.nml clm_input.nml +${COPY} ${DARTROOT}/models/clm/work/clm_to_dart ${EXEROOT}/. +${COPY} ${DARTROOT}/models/clm/work/dart_to_clm ${EXEROOT}/. +${COPY} ${DARTROOT}/models/clm/work/filter ${EXEROOT}/filter_clm +${COPY} ${DARTROOT}/models/clm/work/perfect_model_obs ${EXEROOT}/perfect_model_obs_clm +${COPY} ${DARTROOT}/models/clm/work/input.nml clm_input.nml -${COPY} ${DARTROOT}/models/POP/work/pop_to_dart ${EXEROOT}/. -${COPY} ${DARTROOT}/models/POP/work/dart_to_pop ${EXEROOT}/. -${COPY} ${DARTROOT}/models/POP/work/filter ${EXEROOT}/filter_pop -${COPY} ${DARTROOT}/models/POP/work/input.nml pop_input.nml +${COPY} ${DARTROOT}/models/POP/work/pop_to_dart ${EXEROOT}/. +${COPY} ${DARTROOT}/models/POP/work/dart_to_pop ${EXEROOT}/. +${COPY} ${DARTROOT}/models/POP/work/filter ${EXEROOT}/filter_pop +${COPY} ${DARTROOT}/models/POP/work/perfect_model_obs ${EXEROOT}/perfect_model_obs_pop +${COPY} ${DARTROOT}/models/POP/work/input.nml pop_input.nml -${COPY} ${DARTROOT}/models/CESM/work/cesm_to_dart ${EXEROOT}/. -${COPY} ${DARTROOT}/models/CESM/work/dart_to_cesm ${EXEROOT}/. -${COPY} ${DARTROOT}/models/CESM/work/filter ${EXEROOT}/filter_cesm -${COPY} ${DARTROOT}/models/CESM/work/input.nml input.nml +${COPY} ${DARTROOT}/models/CESM/work/cesm_to_dart ${EXEROOT}/. +${COPY} ${DARTROOT}/models/CESM/work/dart_to_cesm ${EXEROOT}/. +${COPY} ${DARTROOT}/models/CESM/work/filter ${EXEROOT}/filter_cesm +${COPY} ${DARTROOT}/models/CESM/work/perfect_model_obs ${EXEROOT}/perfect_model_obs_cesm +${COPY} ${DARTROOT}/models/CESM/work/input.nml input.nml -# The first time this is executed, the update namlist script does not exist. +# The first time this is executed, the update namelist script does not exist. if [[ -x update_dart_namelists ]]; then ./update_dart_namelists @@ -404,7 +417,7 @@ if ( $num_instances == 1 ) then sed -e "s#DARTSCRIPTNAME#perfect_model.csh#" < temp.$$ >! add_to_run.txt else - sed -e "s#DARTSCRIPTNAME#assimilate.csh#" < temp.$$ >! add_to_run.txt + sed -e "s#DARTSCRIPTNAME#no_assimilate.csh#" < temp.$$ >! add_to_run.txt endif # Now that the "here" document is created, @@ -459,6 +472,11 @@ echo "" echo "4) Inflation files? Observation files for the time of interest?" echo "" +echo "5) Make sure the ${CASE}.run script calls 'assimilate.csh' if you desire." +echo " The default behavior is to _not_ invoke DART and simply run CESM." +echo "" +echo "6) You can use ${CASEROOT}/stage_cesm_files" +echo " to stage files to restart a run." exit 0 Added: DART/trunk/models/CESM/shell_scripts/no_assimilate.csh =================================================================== --- DART/trunk/models/CESM/shell_scripts/no_assimilate.csh (rev 0) +++ DART/trunk/models/CESM/shell_scripts/no_assimilate.csh 2013-10-17 21:08:59 UTC (rev 6527) @@ -0,0 +1,31 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# The DART sourcemods for CAM require that the cam 'initial' files are used +# instead of the restart files. The name of the initial file is read from +# the cam namelist file. To avoid having to change the namelist setting, DART +# uses a static filename in the namelist. After each successful model advance, +# the new initial file must be linked to the static file name. +# This is done in ${CASEROOT}/cam_no_assimilate.csh +# +# No action is required for the other model components. + +echo "`date` -- BEGIN CESM-DART NO-ASSIMILATE" + +${CASEROOT}/cam_no_assimilate.csh +if ( $status != 0 ) exit $status + +echo "`date` -- END CESM-DART NO-ASSIMILATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Property changes on: DART/trunk/models/CESM/shell_scripts/no_assimilate.csh ___________________________________________________________________ Added: svn:executable + * Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native From nancy at ucar.edu Fri Oct 18 15:09:24 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 18 Oct 2013 15:09:24 -0600 Subject: [Dart-dev] [6528] DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid: At the moment this works but is not optimal. Message-ID: Revision: 6528 Author: thoar Date: 2013-10-18 15:09:24 -0600 (Fri, 18 Oct 2013) Log Message: ----------- At the moment this works but is not optimal. Must explore removing the rtm namelist section. Must explore reducing the CLM history file information. Must explore using 80 ensemble members for Dec 2008. Must explore using the stage_cesm_files under different scenarios. Added Paths: ----------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid -------------- next part -------------- Added: DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid (rev 0) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid 2013-10-18 21:09:24 UTC (rev 6528) @@ -0,0 +1,868 @@ +#!/bin/csh -f +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id: CESM1_1_1_hybrid 6396 2013-08-12 17:51:26Z thoar $ + +# --------------------- +# Purpose +# --------------------- +# +# This script is designed to set up, stage, and build a multi-instance run of CESM +# using an F compset where CAM and CLM are active. The initial states +# come from a single multi-instance reference case so a CESM hybrid setup is used. +# +# Because the atmosphere assimilations typically occur every 6 hours, the methodology +# here reflects that. All of CESM stops every 6 hours (requiring the ocean to couple +# 4x per day) so that a CAM output file would be available for assimilation. +# +# This script results in a viable setup for a CESM multi-instance experiment. You +# are STRONGLY encouraged to run the multi-instance CESM a few times and experiment +# with different settings BEFORE you try to assimilate observations. The amount of +# data volume is quite large and you should become comfortable using CESM's restart +# capability to re-stage files in your RUN directory +# +# CESM/DART requires some modifications to the CESM source code EVEN IF YOU ARE NOT +# assimilating. The modifications for CAM require a modification to the +# CESM ${CASE}.run script. The modifications will invoke a DART script that will +# either assimilate or 'do nothing'. It is stronly recommended that you configure +# the ${CASE}.run script to 'do nothing' initially. +# +# CASEROOT/CESM_DART_config will augment the CESM case with the required setup +# and configuration to perform a DART assimilation and modify the ${CASE}.run script +# as mentioned previously - after it makes a backup copy. +# +# This script relies heavily on the information in: +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html +# +# --------------------- +# How to use this script. +# --------------------- +# +# -- You will have to read and understand the script in its entirety. +# You will have to modify things outside this script. +# This script sets up a CESM multi-instance run as we understand them and +# it has almost nothing to do with DART. This is intentional. +# +# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory +# or copy it to somewhere that it will be preserved and run it there. +# It will create a CESM 'CASE' directory, where the model will be built, +# and a run directory, where each forecast (and assimilation) will +# take place. The short term archiver will use a third directory for +# storage of model output until it can be moved to long term storage (HPSS) +# +# -- Examine the whole script to identify things to change for your experiments. +# +# -- Provide the CESM initial ensemble needed by your run. +# +# -- Run this script. +# -- Edit the DART input.nml that appears in the $CASEROOT directory, if not done here. +# -- Submit the job using $CASEROOT/${case}.submit +# +# -- Read, understand, and execute ${CASEROOT}/CESM_DART_config +# +# Table of contents: +# Set up case and directory names +# Refined grid mods +# create_newcase +# cd caseroot +# xmlchanges +# cesm_setup +# namelist mods +# preview_namelists +# SourceMods +# case.build +# stage ICs +# update the run script +# Bring in parts of DART; executables, support files,... +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime +# settings, it is safest to delete everything and start the run from scratch. +# For the brave, read +# +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html +# +# and you may be able to salvage something with +# ./cesm_setup -clean +# ./cesm_setup +# ./${case}.clean_build +# ./${case}.build +# +# ============================================================================== +# ==== Set case options +# ============================================================================== + +# the value of "case" will be used many ways; +# directory and file names, both locally and on HPSS, and +# script names; so consider its length and information content. +# num_instances: Number of ensemble members +# BUG 1384 applies here, so ocean and atm/land must be at same resolution. +# 0.9x1.25_0.9x1.25 (f09_f09) +# 1 deg grid for atm/lnd, gx1v6 mask. Normally used for CAM with prescribed lnd, ocn and ice. + +setenv case test_cam +setenv compset F_AMIP_CAM5 +setenv resolution f09_f09 +setenv cesmtag cesm1_1_1 +setenv num_instances 4 + +# ============================================================================== +# define machines and directories +# +# mach Computer name +# cesmroot Location of the cesm code base +# For cesm1_1_1 on yellowstone +# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# Preferably not a frequently scrubbed location. +# This script will delete any existing caseroot, so this script, +# and other useful things should be kept elsewhere. +# rundir (Future) Run-time directory; scrubbable, large amount of space needed. +# exeroot (Future) directory for executables - scrubbable, large amount of space needed. +# archdir (Future) Short-term archive directory +# until the long-term archiver moves it to permanent storage. +# dartroot Location of _your_ DART installation +# This is passed on to the CESM_DART_config script. +# ============================================================================== + +setenv mach yellowstone +setenv cesm_datadir /glade/p/cesm/cseg/inputdata + +setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag +setenv caseroot /glade/p/work/${USER}/cases/${case} +setenv exeroot /glade/scratch/${USER}/${case}/bld +setenv rundir /glade/scratch/${USER}/${case}/run +setenv archdir /glade/scratch/${USER}/archive/${case} +setenv dartroot /glade/u/home/${USER}/svn/DART/trunk + +# ============================================================================== +# configure settings +# The reference case has dates in it. +# For a 'hybrid' start, these may be unrelated to the refyear, refmon, refday. +# ============================================================================== + +setenv run_refcase cesm_hybrid +setenv refyear 2004 +setenv refmon 01 +setenv refday 10 +setenv run_reftod 00000 +setenv run_refdate $refyear-$refmon-$refday + +# THIS IS THE LOCATION of the 'reference case'. + +setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} + +# setenv stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} + +# ============================================================================== +# runtime settings +# +# sst_dataset Data ocean file and +# sst_grid supporting grid file. Must be consistent. +# +# resubmit How many job steps to run on continue runs (will be 0 initially) +# stop_option Units for determining the forecast length between assimilations +# stop_n Number of time units in the first forecast +# +# If the long-term archiver is off, you get a chance to examine the files before +# they get moved to long-term storage. You can always submit $CASE.l_archive +# whenever you want to free up space in the short-term archive directory. +# +# ============================================================================== + +setenv start_year 2004 +setenv start_month 01 +setenv start_day 10 +setenv start_tod 0 +setenv run_startdate ${start_year}-${start_month}-${start_day} +setenv year_start 1850 +setenv year_end 2012 + +# setenv sst_grid ${cesm_datadir}/ocn/docn7/domain.ocn.gx1v6.090206.nc +setenv sst_grid ${cesm_datadir}/share/domains/domain.ocn.fv0.9x1.25_gx1v6.130409.nc +setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc + +setenv short_term_archiver on +setenv long_term_archiver off +setenv resubmit 0 +setenv stop_option nhours +setenv stop_n 6 + +# ============================================================================== +# job settings +# +# queue can be changed during a series by changing the ${case}.run +# timewall can be changed during a series by changing the ${case}.run +# +# TJH: Advancing 30 instances for 6 hours with 900 pes +# took less than 10 minutes on yellowstone. +# ============================================================================== + +setenv ACCOUNT P86850054 +setenv queue premium +setenv timewall 0:20 + +# ============================================================================== +# set these standard commands based on the machine you are running on. +# ============================================================================== + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + breaksw + default: + # NERSC "hopper", NWSC "yellowstone" + set MOVE = '/bin/mv -fv' + set COPY = '/bin/cp -fv --preserve=timestamps' + set LINK = '/bin/ln -fvs' + set REMOVE = '/bin/rm -fr' + + breaksw +endsw + +# ============================================================================== +# Make sure the CESM directories exist. +# VAR is the shell variable name, DIR is the value +# ============================================================================== + +foreach VAR ( cesmroot dartroot stagedir ) + set DIR = `eval echo \${$VAR}` + if ( ! -d $DIR ) then + echo "ERROR: directory '$DIR' not found" + echo " In the setup script check the setting of: $VAR " + exit -1 + endif +end + +# ============================================================================== +# Create the case - this creates the CASEROOT directory. +# +# For list of the pre-defined component sets: ./create_newcase -list +# To create a variant compset, see the CESM documentation and carefully +# incorporate any needed changes into this script. +# ============================================================================== + +# fatal idea to make caseroot the same dir as where this setup script is +# since the build process removes all files in the caseroot dir before +# populating it. try to prevent shooting yourself in the foot. + +if ( $caseroot == `dirname $0` ) then + echo "ERROR: the setup script should not be located in the caseroot" + echo "directory, because all files in the caseroot dir will be removed" + echo "before creating the new case. move the script to a safer place." + exit -1 +endif + +echo "removing old files from ${caseroot}" +echo "removing old files from ${exeroot}" +echo "removing old files from ${rundir}" +${REMOVE} ${caseroot} +${REMOVE} ${exeroot} +${REMOVE} ${rundir} + +${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ + -res ${resolution} -compset ${compset} + +if ( $status != 0 ) then + echo "ERROR: Case could not be created." + exit -1 +endif + +# preserve a copy of this script as it was run +set ThisFileName = $0:t +${COPY} $ThisFileName ${caseroot}/${ThisFileName}.original + +# ============================================================================== +# Record the DARTROOT directory and copy the DART setup script to CASEROOT. +# CESM_DART_config can be run at some later date if desired, but it presumes +# to be run from a CASEROOT directory. If CESM_DART_config does not exist locally, +# then it better exist in the expected part of the DARTROOT tree. +# ============================================================================== + +if ( ! -e CESM_DART_config ) then + ${COPY} ${dartroot}/models/cam/shell_CESM/CESM_DART_config . +endif + +if ( -e CESM_DART_config ) then + sed -e "s#BOGUS_DART_ROOT_STRING#$dartroot#" < CESM_DART_config >! temp.$$ + ${MOVE} temp.$$ ${caseroot}/CESM_DART_config + chmod 755 ${caseroot}/CESM_DART_config +else + echo "ERROR: the script to configure for data assimilation is not available." + echo " CESM_DART_config MUST be present locally or in" + echo " ${dartroot}/models/CESM/shell_scripts/" + exit -2 +endif + +# ============================================================================== +# Configure the case. +# ============================================================================== + +cd ${caseroot} + +source ./Tools/ccsm_getenv || exit -2 + +# MAX_TASKS_PER_NODE comes from $case/Tools/mkbatch.$machine +@ ptile = $MAX_TASKS_PER_NODE / 2 +@ nthreads = 1 + +# Save a copy for debug purposes +foreach FILE ( *xml ) + if ( ! -e ${FILE}.original ) then + ${COPY} $FILE ${FILE}.original + endif +end + +if ($num_instances < 4) then + + # This is only for the purpose of debugging the code. + @ atm_tasks = $ptile * $num_instances * 4 + @ lnd_tasks = $ptile * $num_instances * 4 + @ ice_tasks = $ptile * $num_instances * 4 + @ ocn_tasks = $ptile * $num_instances + @ cpl_tasks = $ptile * $num_instances + @ glc_tasks = $ptile * $num_instances + @ rof_tasks = $ptile * $num_instances + +else + + # This works, but a more efficient layout should be used + # CAM only; 1 node/instance; no cross node communications + # as long as memory is big enough. + @ atm_tasks = $ptile * $num_instances * 4 + @ lnd_tasks = $ptile * $num_instances * 4 + @ ice_tasks = $ptile * $num_instances * 4 + @ ocn_tasks = $ptile * $num_instances + @ cpl_tasks = $ptile * $num_instances + @ glc_tasks = $ptile * $num_instances + @ rof_tasks = $ptile * $num_instances + +endif + +# echo "task partitioning ... perhaps ... atm // ocn // lnd+ice+glc+rof" +# presently, all components run 'serially' - one after another. +# Yellowstone: no large memory nodes, and 15 tasks/node is recommended. +# Edwards says there's no speed up by running non-active components concurrently, +# after ATM has run, so just run all components sequentially. +# BUT, do arrange it so that each member(instance) spans complete nodes: +# modulo(total pe count / number of instances, 15) == 0. + +echo "" +echo "ATM gets $atm_tasks" +echo "LND gets $lnd_tasks" +echo "ICE gets $ice_tasks" +echo "OCN gets $ocn_tasks" +echo "CPL gets $cpl_tasks" +echo "GLC gets $glc_tasks" +echo "ROF gets $rof_tasks" +echo "" + +./xmlchange NTHRDS_ATM=$nthreads,NTASKS_ATM=$atm_tasks,NINST_ATM=$num_instances +./xmlchange NTHRDS_LND=$nthreads,NTASKS_LND=$lnd_tasks,NINST_LND=$num_instances +./xmlchange NTHRDS_ICE=$nthreads,NTASKS_ICE=$ice_tasks,NINST_ICE=$num_instances +./xmlchange NTHRDS_OCN=$nthreads,NTASKS_OCN=$ocn_tasks,NINST_OCN=1 +./xmlchange NTHRDS_CPL=$nthreads,NTASKS_CPL=$cpl_tasks +./xmlchange NTHRDS_GLC=$nthreads,NTASKS_GLC=$glc_tasks,NINST_GLC=1 +./xmlchange NTHRDS_ROF=$nthreads,NTASKS_ROF=$rof_tasks,NINST_ROF=1 +./xmlchange ROOTPE_ATM=0 +./xmlchange ROOTPE_LND=0 +./xmlchange ROOTPE_ICE=0 +./xmlchange ROOTPE_OCN=0 +./xmlchange ROOTPE_CPL=0 +./xmlchange ROOTPE_GLC=0 +./xmlchange ROOTPE_ROF=0 + +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/c1158.html#run_start_stop +# "A hybrid run indicates that CESM is initialized more like a startup, but uses +# initialization datasets from a previous case. This is somewhat analogous to a +# branch run with relaxed restart constraints. A hybrid run allows users to bring +# together combinations of initial/restart files from a previous case (specified +# by $RUN_REFCASE) at a given model output date (specified by $RUN_REFDATE). +# Unlike a branch run, the starting date of a hybrid run (specified by $RUN_STARTDATE) +# can be modified relative to the reference case. In a hybrid run, the model does not +# continue in a bit-for-bit fashion with respect to the reference case. The resulting +# climate, however, should be continuous provided that no model source code or +# namelists are changed in the hybrid run. In a hybrid initialization, the ocean +# model does not start until the second ocean coupling (normally the second day), +# and the coupler does a "cold start" without a restart file." + +# TJH: +# DART's CAM implementation causes a bit more complexity. DART only uses CAM _initial_ +# files, not RESTART files, so there are sourcemods to force a hybrid start for CAM to +# read initial files - even when CONTINUE_RUN = TRUE. + +./xmlchange RUN_TYPE=hybrid +./xmlchange RUN_STARTDATE=$run_startdate +./xmlchange START_TOD=$start_tod +./xmlchange RUN_REFCASE=$run_refcase +./xmlchange RUN_REFDATE=$run_refdate +./xmlchange RUN_REFTOD=$run_reftod +./xmlchange BRNCH_RETAIN_CASENAME=FALSE +./xmlchange GET_REFCASE=FALSE +./xmlchange EXEROOT=${exeroot} + +./xmlchange SSTICE_DATA_FILENAME=$sst_dataset +./xmlchange SSTICE_GRID_FILENAME=$sst_grid +./xmlchange SSTICE_YEAR_ALIGN=$year_start +./xmlchange SSTICE_YEAR_START=$year_start +./xmlchange SSTICE_YEAR_END=$year_end + +# Do not change the CALENDAR or the CONTINUE_RUN + +./xmlchange CALENDAR=GREGORIAN + +./xmlchange STOP_OPTION=$stop_option +./xmlchange STOP_N=$stop_n +./xmlchange CONTINUE_RUN=FALSE +./xmlchange RESUBMIT=$resubmit + +./xmlchange PIO_TYPENAME=pnetcdf + +# COUPLING discussion. F compsets are 'tight' coupling. +# Only change the ATM_NCPL ... everything is based on this one value, +# including CAM physics and dynamics timesteps. +# Default values for coupling are preserved in env_run.xml.original + +./xmlchange NCPL_BASE_PERIOD=day +./xmlchange ATM_NCPL=48 + +# TJH FIXME ... any reason to turn ON the biogeochemistry in CLM when atmos only? +# There is no compset that has everything we want, +# so we are also turning on the CLM biogeochemistry. +#./xmlchange CLM_CONFIG_OPTS='-bgc cn' + +if ($short_term_archiver == 'off') then + ./xmlchange DOUT_S=FALSE +else + ./xmlchange DOUT_S=TRUE + ./xmlchange DOUT_S_ROOT=${archdir} + ./xmlchange DOUT_S_SAVE_INT_REST_FILES=FALSE +endif +if ($long_term_archiver == 'off') then + ./xmlchange DOUT_L_MS=FALSE +else + ./xmlchange DOUT_L_MS=TRUE + ./xmlchange DOUT_L_MSROOT="csm/${case}" + ./xmlchange DOUT_L_HTAR=FALSE +endif + +# The river transport model ON is useful only when using an active ocean or +# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. + +./xmlchange ROF_GRID='null' +./xmlchange RTM_MODE='NULL' + +# level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) +# Early versions (late 2012) of CAM-SE could not handle these being other than FALSE and 0. +./xmlchange DEBUG=TRUE +./xmlchange INFO_DBUG=1 + +# ============================================================================== +# Set up the case. +# This creates the EXEROOT and RUNDIR directories. +# ============================================================================== + +./cesm_setup + +if ( $status != 0 ) then + echo "ERROR: Case could not be set up." + exit -2 +endif + +# ============================================================================== +# Edit the run script to reflect queue and wallclock +# ============================================================================== + +echo '' +echo 'Updating the run script to set wallclock and queue.' +echo '' + +if ( ! -e ${case}.run.original ) then + ${COPY} ${case}.run ${case}.run.original +endif + +source Tools/ccsm_getenv +set BATCH = `echo $BATCHSUBMIT | sed 's/ .*$//'` +switch ( $BATCH ) + case bsub*: + # NCAR "bluefire", "yellowstone" + set TIMEWALL=`grep BSUB ${case}.run | grep -e '-W' ` + set QUEUE=`grep BSUB ${case}.run | grep -e '-q' ` + sed -e "s/$TIMEWALL[3]/$timewall/" \ + -e "s/ptile=[0-9][0-9]*/ptile=$ptile/" \ + -e "s/$QUEUE[3]/$queue/" < ${case}.run >! temp.$$ + ${MOVE} temp.$$ ${case}.run + chmod 755 ${case}.run + breaksw + + default: + + breaksw +endsw + +# This is the part that modifies the run script to allow CESM to advance +# correctly given the modifications necessary for CAM. It also copies several +# required DART files to the caseroot directory. + +./CESM_DART_config + +# ============================================================================== +# Update source files. +# Ideally, using DART would not require any modifications to the model source. +# Until then, this script accesses sourcemods from a hardwired location. +# If you have additional sourcemods, they will need to be merged into any DART +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. +# ============================================================================== + +if ( -d ~/${cesmtag}/SourceMods ) then + ${COPY} -r ~/${cesmtag}/SourceMods/* ${caseroot}/SourceMods/ +else + echo "ERROR - No SourceMods for this case." + echo "ERROR - No SourceMods for this case." + echo "DART requires modifications to several src files." + echo "These files can be downloaded from:" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "untar these into your HOME directory - they will create a" + echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." + exit -4 +endif + +# The CESM multi-instance capability is relatively new and still has a few +# implementation bugs. These are known problems and will be fixed soon. +# this should be removed when the files are fixed: + +echo "REPLACING BROKEN CESM FILES HERE - SHOULD BE REMOVED WHEN FIXED" +echo caseroot is ${caseroot} +if ( -d ~/${cesmtag} ) then + + # preserve the original version of the files + if ( ! -e ${caseroot}/Buildconf/clm.buildnml.csh.original ) then + ${MOVE} ${caseroot}/Buildconf/clm.buildnml.csh \ + ${caseroot}/Buildconf/clm.buildnml.csh.original + endif + if ( ! -e ${caseroot}/Buildconf/rtm.buildnml.csh.original ) then + ${MOVE} ${caseroot}/Buildconf/rtm.buildnml.csh \ + ${caseroot}/Buildconf/rtm.buildnml.csh.original + endif + if ( ! -e ${caseroot}/preview_namelists.original ) then + ${MOVE} ${caseroot}/preview_namelists \ + ${caseroot}/preview_namelists.original + endif + + # patch/replace the broken files + ${COPY} ~/${cesmtag}/clm.buildnml.csh ${caseroot}/Buildconf/. + ${COPY} ~/${cesmtag}/rtm.buildnml.csh ${caseroot}/Buildconf/. + ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. + +endif + +# ============================================================================== +# Modify namelist templates for each instance. This is a bit of a nuisance in +# that we are pulling in restart and initial files from 'all over the place' +# and each model component has a different strategy. +# +# In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): +# +# CAM has been forced to read initial files - specified by namelist var:ncdata +# CICE reads from namelist variable 'ice_ic' +# CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 +# it does not use the instance string. There is a patch for clm.buildnml.csh +# RTM reads from namelist variable 'finidat_rtm', but rtm.buildnml.csh also is buggy. +# +# All of these must later on be staged with these same filenames. +# OR - all these namelists can be changed to match whatever has been staged. +# MAKE SURE THE STAGING SECTION OF THIS SCRIPT MATCHES THESE VALUES. +# ============================================================================== + +@ inst = 1 +while ($inst <= $num_instances) + + # following the CESM strategy for 'inst_string' + set inst_string = `printf _%04d $inst` + + # =========================================================================== + set fname = "user_nl_cam${inst_string}" + # =========================================================================== + # For a HOP TEST ... empty_htapes = .false. + # For a HOP TEST ... use a default fincl1 + # inithist == 'ENDOFRUN' ensures that CAM writes an initial file every time it stops. + # fincl1,nhtfrq,mfilt all control the history file containing a REQUIRED PHIS field. + + echo " inithist = 'ENDOFRUN'" >> ${fname} + echo " ncdata = 'cam_initial${inst_string}.nc'" >> ${fname} + echo " empty_htapes = .true. " >> ${fname} + echo " fincl1 = 'PHIS:I' " >> ${fname} + echo " nhtfrq = -$stop_n " >> ${fname} + echo " mfilt = 1 " >> ${fname} + + # A lot of the files specified here are because the 'default' files only + # contain data through 2005 and we are interested in timeframes after that. + + set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" + + # CAM5 does prognostic aerosols by default. If you want to prescribe them, + # use the following variables with your own settings ... + # Filenames with 'clim' in their names should specify aerodep_flx_type = 'CYCLICAL'. + # Other values are 'CYCLICAL', 'SERIAL', 'INTERP_MISSING_MONTHS', or 'FIXED'. + + echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} + echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} + echo "aerodep_flx_type = 'CYCLICAL' " >> ${fname} + # Set if aerodep_flx_type = 'CYCLICAL' + echo "aerodep_flx_cycle_yr = 2000 " >> ${fname} + + # =========================================================================== + set fname = "user_nl_clm${inst_string}" + # =========================================================================== + + # Customize the land namelists + # The filename is built using the REFCASE/REFDATE/REFTOD information. + # Making a compact .h0. file is a good idea, since the clm restart files + # do not have all the metadata required to reconstruct a gridded field. + # 'TSA' is 2m surface air temperature. + # TJH ... should make monthly average file or something ... + # 1.3 MB is 100x bigger than CAM history file. See page 65 of + # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf + # + # Every 6 hours + # echo "hist_mfilt = 1" >> $fname + # echo "hist_nhtfrq = -$stop_n" >> $fname + # Every month + # echo "hist_mfilt = 1" >> $fname + # echo "hist_nhtfrq = 0" >> $fname + +# KDR; Was the ntapes bug fixed? +# That's where the CLM restart file always has dimension ntapes, +# but it's set = 0 for hist_empty_htapes = .false., +# which prevents the next forecast from starting. +# Kluzek fixed it in some recent (<9/2013) version, but not 1_1_1. +# There's a supposed fix in ~/cesm1_1_1/SourceMods/src.clm/HistFileMod.F90, +# but it only checks whether ntapes = ntapes_on_file, and is_restart() returning 'true'. +# It doesn't check whether the size is 0. +# That solution can't work in hybrid mode. See ~/Homme/notes:9/17/2013 +# KDR + + echo "hist_empty_htapes = .false." >> $fname + echo "hist_fincl1 = 'TSA'" >> $fname + echo "hist_nhtfrq = 0" >> $fname + echo "hist_mfilt = 1" >> $fname + echo "hist_avgflag_pertape = 'A'" >> $fname + + # =========================================================================== + set fname = "user_nl_cice${inst_string}" + # =========================================================================== + # CICE Namelists + + echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname + + # =========================================================================== + # TJH FIXME ... do we need this if we turn off the river runoff model? + set fname = "user_nl_rtm${inst_string}" + # =========================================================================== + # RIVER RUNOFF CAN START FROM AN OLD CLM RESTART FILE + # you can specify the RTM filename here and override the settings from + # RUN_REFCASE/RUN_REFDATE/RUN_REFTOD (something you cannot do with CLM). + + echo "finidat_rtm = '${run_refcase}.rtm${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname + + @ inst ++ +end + +./preview_namelists + +# ============================================================================== +# Stage the restarts now that the run directory exists +# THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. +# ============================================================================== + +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. + +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' +echo '' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' +echo '' + +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} + +set restart_time = $RUN_REFDATE-$RUN_REFTOD + +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- + +echo "continue_run is \$CONTINUE_RUN" + +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + + # the restarts should be in the short term archive directory + # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + + @ inst=1 + while (\$inst <= $num_instances) + + set inst_string = \`printf _%04d \$inst\` + + echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${CASE}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} + echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} + echo "${CASE}.rtm\${inst_string}.r.\${restart_time}.nc" >! rpointer.rof\${inst_string} + echo "${CASE}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + + + @ inst ++ + end + + endif + + # Now that we're guaranteed to have the cam initial file, link to our hardwired name. + @ inst=1 + while (\$inst <= $num_instances) + set inst_string = \`printf _%04d \$inst\` + ${LINK} ${CASE}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc + @ inst ++ + end + + # relink the pointer file for the data ocean + echo "${CASE}.docn.r.\${restart_time}.nc" >! rpointer.ocn + echo "" >> rpointer.ocn + echo "${CASE}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn + echo "" >> rpointer.ocn + + exit 0 +endif + +#---------------------------------------------------------------------- +# This block prepares the initial run directory for a startup run. +# The CLM, CICE, ROF namelists specify a relative path for the initial file. +# The CAM namelist specifies a static name. +# POP uses pointer files. +# +# The reference case may more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - in a hybrid case - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +set filedir = ${stagedir} + +@ inst=1 +while (\$inst <= $num_instances) + + set inst_string = \`printf _%04d \$inst\` + + echo '' + echo "Staging restarts for instance \$inst of $num_instances" + + ${LINK} \${filedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . + ${LINK} \${filedir}/${run_refcase}.cice\${inst_string}.r.${run_refdate}-${run_reftod}.nc . + ${LINK} \${filedir}/${run_refcase}.cam\${inst_string}.i.${run_refdate}-${run_reftod}.nc cam_initial\${inst_string}.nc + + @ inst ++ +end + +exit 0 + +EndOfText +chmod 0755 stage_cesm_files + +./stage_cesm_files + +# ============================================================================== +# build +# ============================================================================== + +echo '' +echo 'Building the case' +echo '' + +./${case}.build + +if ( $status != 0 ) then + echo "ERROR: Case could not be built." + exit -5 +endif + +# ============================================================================== +# What to do next +# ============================================================================== + +echo "" +echo "Time to check the case." +echo "" +echo "1) cd ${rundir}" +echo " and check the compatibility between the namelists/pointer" +echo " files and the files that were staged." +echo "" +echo "2) cd ${caseroot}" +echo " (on yellowstone) If the ${case}.run script still contains:" +echo ' #BSUB -R "select[scratch_ok > 0]"' +echo " around line 9, delete it." +echo "" +echo "3) If you want to simply advance CESM without using DART, make sure the" +echo " ${case}.run script references the 'no_assimilate.csh' script." +echo " If you want to assimilate, reference the 'assimilate.csh' script." +echo " The default action is to assimilate." +echo "" +echo "4) Verify the contents of env_run.xml and submit the CESM job:" +echo " ./${case}.submit" +echo "" +echo "5) After the job has run, check to make sure it worked and that" +echo " a: the right restart files exist in the run directory," +echo " b: (if you're running DART) the archive dart/hist directory has the DART output," +echo " c: everything is working correctly ..." +echo "" +echo "6) To extend the run in $stop_n '"$stop_option"' steps," +echo " change the env_run.xml variables:" +echo "" +echo " ./xmlchange CONTINUE_RUN=TRUE" +echo " ./xmlchange RESUBMIT=" +echo " ./xmlchange STOP_N=$stop_n" +echo "" +echo " and" +echo " ./${case}.submit" +echo "" + +exit 0 + +# +# $URL: https://subversion.ucar.edu/DAReS/DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid $ +# $Revision: 6396 $ +# $Date: 2013-08-12 11:51:26 -0600 (Mon, 12 Aug 2013) $ + Property changes on: DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid ___________________________________________________________________ Added: svn:executable + * From nancy at ucar.edu Fri Oct 18 15:24:33 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 18 Oct 2013 15:24:33 -0600 Subject: [Dart-dev] [6529] DART/trunk/models/cam/shell_scripts/CESM_DART_config: This is the consolidation of the DART-related changes to a CESM case. Message-ID: Revision: 6529 Author: thoar Date: 2013-10-18 15:24:33 -0600 (Fri, 18 Oct 2013) Log Message: ----------- This is the consolidation of the DART-related changes to a CESM case. This script is copied into the CASEROOT directory and subsequently executed. It is REQUIRED to use this script as it modifies the CASE.run script with the bits to relink the latest cam initial files to the expected static filenames. Added Paths: ----------- DART/trunk/models/cam/shell_scripts/CESM_DART_config -------------- next part -------------- Added: DART/trunk/models/cam/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM_DART_config (rev 0) +++ DART/trunk/models/cam/shell_scripts/CESM_DART_config 2013-10-18 21:24:33 UTC (rev 6529) @@ -0,0 +1,410 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id: CESM_DART_config 6392 2013-08-12 16:10:09Z thoar $ + +# --------------------- +# Purpose +# --------------------- +# +# This script integrates DART with a pre-existing CESM multi-instance case. +# It must be run from a valid CASEROOT directory. If the case was created +# using one of the DART scripts, this script should be staged in the +# CASEROOT directory automatically, and DARTROOT is set at that time. +# +# CAM is the only active model component. +# CESM starts and stops to allow for CAM to assimilate every 6 hours. +# +# This script will build the DART executables if they are not found. +# +# There are many CESM binary files in big-endian format, and DART reads +# some of them, so you MUST compile DART accordingly e.g., +# ifort -convert big_endian +# Contact dart at ucar.edu if you want to use another compiler. +# +# --------------------- +# How to set up the script +# --------------------- +# +# -- Ensure DARTROOT references a valid DART directory. +# -- Examine the whole script to identify things to change for your experiments. +# -- Provide any initial files needed by your run: +# inflation +# sampling error correction +# -- Run this script. +# -- Edit the DART input.nml that appears in the ${CASEROOT} directory. +# -- Submit the job using ${CASEROOT}/${CASE}.submit +# +# ============================================================================== +# Get the environment of the case - defines number of instances/ensemble size ... +# Each model component has their own number of instances, but the 'coupled' +# DART parts is going to use the number of instances of CAM. +# ============================================================================== + +if ( ! -e ./Tools/ccsm_getenv ) then + echo "ERROR: $0 must be run from a CASEROOT directory". + exit -1 +endif + +source ./Tools/ccsm_getenv + +set num_instances = $NINST_ATM + +# DARTROOT is set by the DART CESM_configure scripts. Under certain +# situations, you may need to set this manually. It should reference the +# base portion of the DART code tree. + +setenv DARTROOT BOGUS_DART_ROOT_STRING + +# ============================================================================== +# Some +# ============================================================================== + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + breaksw + default: + # NERSC "hopper", NWSC "yellowstone" + set MOVE = '/bin/mv -fv' + set COPY = '/bin/cp -fv --preserve=timestamps' + set LINK = '/bin/ln -fvs' + set REMOVE = '/bin/rm -fr' + + breaksw +endsw + +echo "" + +# ============================================================================== +# make sure the required directories exist +# VAR is the shell variable name, DIR is the value +# ============================================================================== + +foreach VAR ( CASEROOT DARTROOT ) + set DIR = `eval echo \${$VAR}` + if ( ! -d $DIR ) then + echo "ERROR: directory '$DIR' not found" + echo " In the setup script check the setting of: $VAR" + exit -1 + endif +end + +# ============================================================================== +# Make sure the DART executables exist or build them if we can't find them. +# The DART input.nml in the model directory IS IMPORTANT during this part +# because it defines what observation types are supported. +# ============================================================================== + +foreach MODEL ( cam ) + set targetdir = $DARTROOT/models/$MODEL/work + if ( ! -x $targetdir/filter ) then + echo "" + echo "WARNING: executable file 'filter' not found." + echo " Looking for: $targetdir/filter " + echo " Trying to rebuild all executables for $MODEL now ..." + (cd $targetdir; ./quickbuild.csh -mpi) + if ( ! -x $targetdir/filter ) then + echo "ERROR: executable file 'filter' not found." + echo " Unsuccessfully tried to rebuild: $targetdir/filter " + echo " Required DART assimilation executables are not found." + echo " Stopping prematurely." + exit -1 + endif + endif +end + +# ============================================================================== +# Stage the required parts of DART in the CASEROOT directory. +# ============================================================================== + +# The standard CESM short-term archiving script may need to be altered +# to archive additional or subsets of things, or to reduce the amount of +# data that is sent to the long-term archive. Put a version of st_archive.sh +# in ${DARTROOT}/models/CESM/shell_scripts when/if necessary + +if ( ! -e Tools/st_archive.sh.original ) then + ${COPY} Tools/st_archive.sh Tools/st_archive.sh.original +else + echo "Tools/st_archive.sh backup copy already exists." + echo "" +endif + +${COPY} ${DARTROOT}/models/cam/shell_scripts/st_archive.sh Tools/ +${COPY} ${DARTROOT}/models/cam/shell_scripts/no_assimilate.csh . +${COPY} ${DARTROOT}/models/cam/shell_scripts/assimilate.csh . +${COPY} ${DARTROOT}/models/cam/shell_scripts/perfect_model.csh . +${COPY} ${DARTROOT}/shell_scripts/shell_exit.sh . + +# ============================================================================== +# Stage the DART executables in the CESM execution root directory: EXEROOT +# If you recompile the DART code (maybe to support more observation types) +# we're making a script to make it easy to install new DART executables. +# ============================================================================== + +cat << EndOfText >! refresh_dart_files +#!/bin/sh + +# this script copies over the dart executables and namelists to the +# proper directory. if you have to update any dart code or namelists, +# do it in the $DARTROOT directory and then rerun refresh_dart_files +# this script was autogenerated by $0 +# using the variables set in that script + +if [[ -e input.nml ]]; then + if [[ -e input.nml.original ]]; then + echo "input.nml.original already exists - not making another" + else + ${COPY} input.nml input.nml.original + fi +fi + +${COPY} ${DARTROOT}/models/cam/work/cam_to_dart ${EXEROOT}/. +${COPY} ${DARTROOT}/models/cam/work/dart_to_cam ${EXEROOT}/. +${COPY} ${DARTROOT}/models/cam/work/filter ${EXEROOT}/. +${COPY} ${DARTROOT}/models/cam/work/perfect_model_obs ${EXEROOT}/. +${COPY} ${DARTROOT}/models/cam/work/input.nml ${CASEROOT}/. + +# The first time this is executed, the update namelist script does not exist. + +if [[ -x update_dart_namelists ]]; then + ./update_dart_namelists +fi + +exit 0 + +EndOfText +chmod 0755 refresh_dart_files + +./refresh_dart_files + +# ============================================================================== +# Ensure the DART namelists are consistent with the ensemble size, +# suggest settings for num members in the output diagnostics files, etc. +# The user is free to update these after setup and before running. +# ============================================================================== + +cat << EndOfText >! update_dart_namelists +#!/bin/sh + +# this script makes certain namelist settings consistent with the number +# of ensemble members built by the setup script. +# this script was autogenerated by $0 +# using the variables set in that script + +# Ensure that the input.nml ensemble size matches the number of instances. +# WARNING: the output files contain ALL ensemble members ==> BIG + +ex input.nml <! ${RUNDIR}/cam_inflation_cookie + +# ============================================================================== +# Creating the code to add to the *.run script to call DART. +# If there is one instance ... we MUST be running a perfect_model experiment. +# If there are many instances ... we MUST be assimilating. +# The number of instances dictates what DART script will be used. +# ============================================================================== + +cat << "EndOfText" >! temp.$$ + +# ------------------------------------------------------------------------- +# START OF DART: if CESM finishes correctly (pirated from ccsm_postrun.csh); +# perform an assimilation with DART. + +set CplLogFile = `ls -1t cpl.log* | head -n 1` +if ($CplLogFile == "") then + echo 'ERROR: Model did not complete - no cpl.log file present - exiting.' + echo 'ERROR: DART will not be attempted.' + setenv LSB_PJL_TASK_GEOMETRY "{(0)}" + setenv EXITCODE -1 + mpirun.lsf ${CASEROOT}/shell_exit.sh + exit -1 +endif + +grep 'SUCCESSFUL TERMINATION' $CplLogFile +if ( $status == 0 ) then + ${CASEROOT}/DARTSCRIPTNAME + + if ( $status == 0 ) then + echo "`date` -- DART HAS FINISHED" + else + echo "`date` -- DART ERROR - ABANDON HOPE" + setenv LSB_PJL_TASK_GEOMETRY "{(0)}" + setenv EXITCODE -3 + mpirun.lsf ${CASEROOT}/shell_exit.sh + exit -3 + endif +else + echo 'ERROR: Model did not complete successfully - exiting.' + echo 'ERROR: DART will not be attempted.' + setenv LSB_PJL_TASK_GEOMETRY "{(0)}" + setenv EXITCODE -2 + mpirun.lsf ${CASEROOT}/shell_exit.sh + exit -2 +endif + +# END OF DART BLOCK +# ------------------------------------------------------------------------- +"EndOfText" + +# modify the "here" document to call the logical DART script. + +if ( $num_instances == 1 ) then + sed -e "s#DARTSCRIPTNAME#perfect_model.csh#" < temp.$$ >! add_to_run.txt +else + sed -e "s#DARTSCRIPTNAME#no_assimilate.csh#" < temp.$$ >! add_to_run.txt +endif + +# Now that the "here" document is created, +# determine WHERE to insert it -- ONLY IF it is not already there. + +grep "ABANDON HOPE" ${CASE}.run > /dev/null +set STATUSCHECK = $status + +if ( ${STATUSCHECK} == 0 ) then + echo "DART block already present in ${CASE}.run" + echo "" +else if ( ${STATUSCHECK} == 1 ) then + echo "Adding DART block to ${CASE}.run" + echo "" + + set MYSTRING = `grep --line-number "CSM EXECUTION HAS FINISHED" ${CASE}.run` + set MYSTRING = `echo $MYSTRING | sed -e "s#:# #g"` + + @ origlen = `cat ${CASE}.run | wc -l` + @ keep = $MYSTRING[1] + @ lastlines = $origlen - $keep + + head -n $keep ${CASE}.run >! temp.$$ + cat add_to_run.txt >> temp.$$ + tail -n $lastlines ${CASE}.run >> temp.$$ + + ${MOVE} temp.$$ ${CASE}.run + ${REMOVE} add_to_run.txt + +else + echo "ERROR in grep of ${CASE}.run: aborting" + echo "status was ${STATUSCHECK}" + exit -6 +endif + +chmod 0744 ${CASE}.run + +# ============================================================================== +# What to do next +# ============================================================================== + +echo "Time to check the DART configuration:" +echo "" +echo "1) Modify what you need to in the DART namelists. These are " +echo " ${CASEROOT}/input.nml" +echo "" +echo "2) If you have recompiled any part of the DART system, 'refresh_dart_files'" +echo " will copy them into the correct places." +echo "" +echo "3) Make sure the observation directory names in assimilate.csh match" +echo " those on your system, and submit the CESM job in the normal way." +echo "" +echo "4) Inflation files? Observation files for the time of interest?" +echo "" +echo "5) Make sure the ${CASE}.run script calls 'assimilate.csh' if you desire." +echo " The default behavior is to _not_ invoke DART and simply run CESM." +echo "" +echo "6) You can use ${CASEROOT}/stage_cesm_files" +echo " to stage files to restart a run." + +exit 0 + +# +# $URL: https://subversion.ucar.edu/DAReS/DART/trunk/models/CESM/shell_scripts/CESM_DART_config $ +# $Revision: 6392 $ +# $Date: 2013-08-12 10:10:09 -0600 (Mon, 12 Aug 2013) $ + From nancy at ucar.edu Fri Oct 18 16:54:00 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 18 Oct 2013 16:54:00 -0600 Subject: [Dart-dev] [6530] DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid: The RTM stuff is not needed at all, so I removed it. Message-ID: Revision: 6530 Author: thoar Date: 2013-10-18 16:53:59 -0600 (Fri, 18 Oct 2013) Log Message: ----------- The RTM stuff is not needed at all, so I removed it. There is a SourceMod that allows clm to create 'empty' history tapes with one variable in them that does not bomb when CONTINUE_RUN = .true. At the moment, I have not added src.clm2/histFileMod.F90 to the tar file on the web. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid 2013-10-18 21:24:33 UTC (rev 6529) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid 2013-10-18 22:53:59 UTC (rev 6530) @@ -463,6 +463,7 @@ # The river transport model ON is useful only when using an active ocean or # land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. +# If you turn it ON, you will have to stage initial files etc. ./xmlchange ROF_GRID='null' ./xmlchange RTM_MODE='NULL' @@ -555,10 +556,6 @@ ${MOVE} ${caseroot}/Buildconf/clm.buildnml.csh \ ${caseroot}/Buildconf/clm.buildnml.csh.original endif - if ( ! -e ${caseroot}/Buildconf/rtm.buildnml.csh.original ) then - ${MOVE} ${caseroot}/Buildconf/rtm.buildnml.csh \ - ${caseroot}/Buildconf/rtm.buildnml.csh.original - endif if ( ! -e ${caseroot}/preview_namelists.original ) then ${MOVE} ${caseroot}/preview_namelists \ ${caseroot}/preview_namelists.original @@ -566,7 +563,6 @@ # patch/replace the broken files ${COPY} ~/${cesmtag}/clm.buildnml.csh ${caseroot}/Buildconf/. - ${COPY} ~/${cesmtag}/rtm.buildnml.csh ${caseroot}/Buildconf/. ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. endif @@ -582,7 +578,6 @@ # CICE reads from namelist variable 'ice_ic' # CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 # it does not use the instance string. There is a patch for clm.buildnml.csh -# RTM reads from namelist variable 'finidat_rtm', but rtm.buildnml.csh also is buggy. # # All of these must later on be staged with these same filenames. # OR - all these namelists can be changed to match whatever has been staged. @@ -657,11 +652,11 @@ # That solution can't work in hybrid mode. See ~/Homme/notes:9/17/2013 # KDR - echo "hist_empty_htapes = .false." >> $fname - echo "hist_fincl1 = 'TSA'" >> $fname - echo "hist_nhtfrq = 0" >> $fname - echo "hist_mfilt = 1" >> $fname - echo "hist_avgflag_pertape = 'A'" >> $fname + echo "hist_empty_htapes = .true." >> $fname + echo "hist_fincl1 = 'TSA'" >> $fname + echo "hist_nhtfrq = -$stop_n" >> $fname + echo "hist_mfilt = 1" >> $fname + echo "hist_avgflag_pertape = 'A'" >> $fname # =========================================================================== set fname = "user_nl_cice${inst_string}" @@ -670,16 +665,6 @@ echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname - # =========================================================================== - # TJH FIXME ... do we need this if we turn off the river runoff model? - set fname = "user_nl_rtm${inst_string}" - # =========================================================================== - # RIVER RUNOFF CAN START FROM AN OLD CLM RESTART FILE - # you can specify the RTM filename here and override the settings from - # RUN_REFCASE/RUN_REFDATE/RUN_REFTOD (something you cannot do with CLM). - - echo "finidat_rtm = '${run_refcase}.rtm${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname - @ inst ++ end @@ -740,7 +725,6 @@ echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv echo "${CASE}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} - echo "${CASE}.rtm\${inst_string}.r.\${restart_time}.nc" >! rpointer.rof\${inst_string} echo "${CASE}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} From thoar at ucar.edu Tue Oct 22 12:59:50 2013 From: thoar at ucar.edu (Tim Hoar) Date: Tue, 22 Oct 2013 12:59:50 -0600 Subject: [Dart-dev] CESM modelers only - anyone with an opinion, please weigh in Message-ID: <77A451E3-3403-4A04-81C3-831F746F7F1F@ucar.edu> I am trying to be consistent with some scripts and scripting for the CESM model components. If you use CAM, CLM, or POP - please continue reading. If you do not, hit 'delete' now. Functionally, I would like to propose that the shell_scripts directories for each model component have (something like) the following: CESM1_1_1_setup_inital (pull model states from all over) CESM1_1_1_setup_pmo (advance a single model state) CESM1_1_1_setup_hybrid (stage an ensemble from a reference case and carry on) perfect_model_obs.csh assimilate.csh CAM may need additional scripts to provide backward compatibility. CESM1_1_1_setup_startup (run in perpetual startup mode ...) assimilate.csh (to manipulate the value of CONTINUE_RUN ...) Does anyone have an improvement on that? I need to rename the scripts in the CESM directories to be more consistent. This will help the learning curve as well as the documentation. Does anyone have an opinion about making the names a bit more specific? perfect_model_obs.csh --> [cam,clm,pop]_perfect_model_obs.csh assimilate.csh --> [cam,clm,pop]_assimilate.csh Can we call the observations from a perfect model experiment obs_seq.xxxx.perfect instead of obs_seq.out? Can we reserve the obs_seq.out for 'real' observations? Now is the time to provide your opinions. Thanks -- Tim Tim Hoar, Associate Scientist National Center for Atmospheric Research thoar at ucar.edu 303.497.1708 From nancy at ucar.edu Tue Oct 22 14:21:18 2013 From: nancy at ucar.edu (Nancy Collins) Date: Tue, 22 Oct 2013 14:21:18 -0600 Subject: [Dart-dev] CESM modelers only - anyone with an opinion, please weigh in In-Reply-To: <77A451E3-3403-4A04-81C3-831F746F7F1F@ucar.edu> References: <77A451E3-3403-4A04-81C3-831F746F7F1F@ucar.edu> Message-ID: <5266DE3E.4010707@ucar.edu> overall, this looks very good to me. i'm very happy trying to keep the scripts with the same name across cesm components, and have them do similar things. minor quibbles: i'd prefer 'perfect_model' as the script name instead of 'perfect_model_obs' because i've heard it called a 'perfect model' experiment more times than a 'perfect model observation' experiment - but maybe that's a minority opinion. i'd vote that the cam, pop, and clm scripts that are for single component assimilations just have the simpler names (no xxx prepended) and only the fully coupled version would need the component names prepended. again - i can be outvoted on this if everyone else feels differently. for the cam-only ones - i'm not sure setup_startup is a helpful name - can't what it needs to do be part of setup_initial or setup_hybrid? and perhaps you meant no_assimiliate.csh in the cam-only section? you already have an assimilate.csh script listed as something all the components would have. thanks, n. On 10/22/13 12:59 PM, Tim Hoar wrote: > I am trying to be consistent with some scripts and scripting for the CESM model components. > If you use CAM, CLM, or POP - please continue reading. > If you do not, hit 'delete' now. > > Functionally, I would like to propose that the shell_scripts directories for each model component have (something like) the following: > > CESM1_1_1_setup_inital (pull model states from all over) > CESM1_1_1_setup_pmo (advance a single model state) > CESM1_1_1_setup_hybrid (stage an ensemble from a reference case and carry on) > perfect_model_obs.csh > assimilate.csh > > CAM may need additional scripts to provide backward compatibility. > CESM1_1_1_setup_startup (run in perpetual startup mode ...) > assimilate.csh (to manipulate the value of CONTINUE_RUN ...) > > Does anyone have an improvement on that? I need to rename the scripts in the CESM directories to be more consistent. This will help the learning curve as well as the documentation. > > Does anyone have an opinion about making the names a bit more specific? > > perfect_model_obs.csh --> [cam,clm,pop]_perfect_model_obs.csh > assimilate.csh --> [cam,clm,pop]_assimilate.csh > > Can we call the observations from a perfect model experiment obs_seq.xxxx.perfect instead of > obs_seq.out? Can we reserve the obs_seq.out for 'real' observations? > > Now is the time to provide your opinions. > > Thanks -- Tim > > Tim Hoar, Associate Scientist > National Center for Atmospheric Research > thoar at ucar.edu > 303.497.1708 > > _______________________________________________ > Dart-dev mailing list > Dart-dev at mailman.ucar.edu > http://mailman.ucar.edu/mailman/listinfo/dart-dev From raeder at ucar.edu Tue Oct 22 14:35:55 2013 From: raeder at ucar.edu (Kevin Raeder) Date: Tue, 22 Oct 2013 14:35:55 -0600 Subject: [Dart-dev] CESM modelers only - anyone with an opinion, please weigh in In-Reply-To: <77A451E3-3403-4A04-81C3-831F746F7F1F@ucar.edu> References: <77A451E3-3403-4A04-81C3-831F746F7F1F@ucar.edu> Message-ID: <5266E1AB.1050502@ucar.edu> See embedded opinions... On 10/22/13 12:59 PM, Tim Hoar wrote: > > Functionally, I would like to propose that the shell_scripts directories for each model component have (something like) the following: > > CESM1_1_1_setup_inital (pull model states from all over) > CESM1_1_1_setup_pmo (advance a single model state) > CESM1_1_1_setup_hybrid (stage an ensemble from a reference case and carry on) > perfect_model_obs.csh > assimilate.csh Is CESM1_1_1_config missing from this list? Or has it been absorbed into something else? It's good to have the model version in the name. If it's useful, unchanged, for future model versions, we can just have note in each file saying so. I like Nancy's proposal for perfect_model{_obs}.csh. Will st_archive.sh be in here too? Nitpick; 'initial' instead of 'inital' > > CAM may need additional scripts to provide backward compatibility. > CESM1_1_1_setup_startup (run in perpetual startup mode ...) > assimilate.csh (to manipulate the value of CONTINUE_RUN ...) > > Does anyone have an improvement on that? I need to rename the scripts in the CESM directories to be more consistent. This will help the learning curve as well as the documentation. > > Does anyone have an opinion about making the names a bit more specific? > > perfect_model_obs.csh --> [cam,clm,pop]_perfect_model_obs.csh > assimilate.csh --> [cam,clm,pop]_assimilate.csh I'm in favor of either this, but I'd listen to arguments in favor of keeping the generic names, but copying them to new, local names when they might be confused with sibling scripts for other components, like when doing multi-component, or coupled assimilation. Will people be doing assimilations outside of the CESM framework, where the generic name would be slightly more convenient? > > Can we call the observations from a perfect model experiment obs_seq.xxxx.perfect instead of > obs_seq.out? Can we reserve the obs_seq.out for 'real' observations? That would be an improvement. We could also consider 'obs_seq.real' for real obs files that haven't been assimilated yet. 'Obs_seq.final' seems fine for the output from assimilations. > > Now is the time to provide your opinions. > > Thanks -- Tim > > Tim Hoar, Associate Scientist > National Center for Atmospheric Research > thoar at ucar.edu > 303.497.1708 > > _______________________________________________ > Dart-dev mailing list > Dart-dev at mailman.ucar.edu > http://mailman.ucar.edu/mailman/listinfo/dart-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ucar.edu/pipermail/dart-dev/attachments/20131022/23e42df1/attachment.html From nancy at ucar.edu Tue Oct 22 15:50:46 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 22 Oct 2013 15:50:46 -0600 Subject: [Dart-dev] [6534] DART/trunk/models/clm/shell_scripts: Typographical changes only. Message-ID: Revision: 6534 Author: thoar Date: 2013-10-22 15:50:46 -0600 (Tue, 22 Oct 2013) Log Message: ----------- Typographical changes only. misspellings, a bad comment cut&paste, that sort of thing. Modified Paths: -------------- DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial DART/trunk/models/clm/shell_scripts/CESM_DART_config DART/trunk/models/clm/shell_scripts/assimilate.csh DART/trunk/models/clm/shell_scripts/st_archive.sh Property Changed: ---------------- DART/trunk/models/clm/shell_scripts/CESM_DART_config -------------- next part -------------- Modified: DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial 2013-10-22 16:06:00 UTC (rev 6533) +++ DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial 2013-10-22 21:50:46 UTC (rev 6534) @@ -490,7 +490,7 @@ # Ideally, using DART would not require any modifications to the model source. # Until then, this script accesses sourcemods from a hardwired location. # If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'case' directory. +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. # ============================================================================== if ( -d ~/${cesmtag}/SourceMods ) then @@ -752,14 +752,12 @@ echo "3) If you want to assimilate 'right away', configure and execute" echo " the ${caseroot}/CESM_DART_config script." echo "" -echo "3) Verify the contents of env_run.xml and submit the CESM job:" +echo "4) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "4) After the run finishes ... check the contents of the DART observation sequence file" -echo " ${archdir}/dart/hist/obs_seq.YYYY-MM-DD-SSSSS" -echo " to make sure there are good values in the file. (not -888888.)" +echo "5) After the job has run, make sure it worked " echo "" -echo "5) To extend the run in $assim_n '"$stop_option"' steps," +echo "6) To extend the run in $assim_n '"$stop_option"' steps," echo " change the env_run.xml variables:" echo "" echo " ./xmlchange CONTINUE_RUN=TRUE" Modified: DART/trunk/models/clm/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM_DART_config 2013-10-22 16:06:00 UTC (rev 6533) +++ DART/trunk/models/clm/shell_scripts/CESM_DART_config 2013-10-22 21:50:46 UTC (rev 6534) @@ -15,10 +15,8 @@ # using one of the DART scripts, this script should be staged in the # CASEROOT directory automatically, and DARTROOT is set at that time. # -# CAM, POP, and CLM are all active and going to be assimilated separately. -# CESM starts and stops to allow for CAM to assimilate every 6 hours. -# The individual assimilation scripts control how often each model component -# performs its own assimilation. +# CLM is the only active model component. +# CESM starts and stops to allow for CLM to assimilate every 24 hours. # # This script will build the DART executables if they are not found. # @@ -134,7 +132,7 @@ # ============================================================================== # The standard CESM short-term archiving script may need to be altered -# to archive addtional or subsets of things, or to reduce the amount of +# to archive additional or subsets of things, or to reduce the amount of # data that is sent to the long-term archive. Put a version of st_archive.sh # in ${DARTROOT}/models/CESM/shell_scripts when/if necessary @@ -179,7 +177,7 @@ ${COPY} ${DARTROOT}/models/clm/work/perfect_model_obs ${EXEROOT}/. ${COPY} ${DARTROOT}/models/clm/work/input.nml ${CASEROOT}/. -# The first time this is executed, the update namlist script does not exist. +# The first time this is executed, the update namelist script does not exist. if [[ -x update_dart_namelists ]]; then ./update_dart_namelists @@ -233,7 +231,7 @@ # # If it is requested and is not present ... it is an error. # -# The sampling error correction is a lookup table. Each ensemble size +# The sampling error correction is a lookup table. Each ensemble size # has its own (static) file. It is only needed if any # input.nml:&assim_tools_nml:sampling_error_correction = .true., #========================================================================= Property changes on: DART/trunk/models/clm/shell_scripts/CESM_DART_config ___________________________________________________________________ Added: svn:executable + * Modified: DART/trunk/models/clm/shell_scripts/assimilate.csh =================================================================== --- DART/trunk/models/clm/shell_scripts/assimilate.csh 2013-10-22 16:06:00 UTC (rev 6533) +++ DART/trunk/models/clm/shell_scripts/assimilate.csh 2013-10-22 21:50:46 UTC (rev 6534) @@ -16,7 +16,7 @@ # The FORCE options are not optional. # The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following +# some systems don't like the -v option to any of the following switch ("`hostname`") case be*: # NCAR "bluefire" @@ -161,7 +161,8 @@ # Block 2: Stage the files needed for SAMPLING ERROR CORRECTION # # The sampling error correction is a lookup table. -# The tables are stored in the DART distribution. +# The tables were originally in the DART distribution, but should +# have been staged to $CASEROOT at setup time. # Each ensemble size has its own (static) file. # It is only needed if # input.nml:&assim_tools_nml:sampling_error_correction = .true., @@ -308,7 +309,7 @@ # as long as we can have unique namelists for each of them. # # At the end of the block, we have DART initial condition files filter_ics.[1-N] -# that came from pointer files ../rpointer.lnd_[1-N].restart +# that came from pointer files ../rpointer.lnd_[1-N] # # REQUIRED DART namelist settings: # &filter_nml: restart_in_file_name = 'filter_ics' @@ -339,8 +340,8 @@ set DART_IC_FILENAME = `printf filter_ics.%04d ${member}` set DART_RESTART_FILE = `printf filter_restart.%04d ${member}` - sed -e "s/dart_ics/..\/${DART_IC_FILENAME}/" \ - -e "s/dart_restart/..\/${DART_RESTART_FILE}/" < ../input.nml >! input.nml + sed -e "s#dart_ics#../${DART_IC_FILENAME}#" \ + -e "s#dart_restart#../${DART_RESTART_FILE}#" < ../input.nml >! input.nml ${LINK} ../../$LND_RESTART_FILENAME clm_restart.nc ${LINK} ../../$LND_HISTORY_FILENAME clm_history.nc Modified: DART/trunk/models/clm/shell_scripts/st_archive.sh =================================================================== --- DART/trunk/models/clm/shell_scripts/st_archive.sh 2013-10-22 16:06:00 UTC (rev 6533) +++ DART/trunk/models/clm/shell_scripts/st_archive.sh 2013-10-22 21:50:46 UTC (rev 6534) @@ -150,6 +150,7 @@ set *Posterior_Diag.*.nc; dispose ifiles_n ${sta}/dart/hist $* set *obs_seq.*.out; dispose ifiles_n ${sta}/dart/hist $* set *obs_seq.*.final; dispose ifiles_n ${sta}/dart/hist $* +set *obs_seq.*.perfect; dispose ifiles_n ${sta}/dart/hist $* set *pr*inflate_restart*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_n ${sta}/dart/rest $* set *po*inflate_restart*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_n ${sta}/dart/rest $* @@ -159,7 +160,6 @@ do get_inst_suffix $IDX $NINST_ATM set atm${inst_suffix}.log.*; dispose ifiles_n ${sta}/atm/logs $* -# set cam_initial_${IDX}.nc; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_n ${sta}/atm/rest $* set ${CASE}.cam*${inst_suffix}.r.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/atm/rest $* set ${CASE}.cam*${inst_suffix}.rs.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/atm/rest $* set ${CASE}.cam*${inst_suffix}.ra.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/atm/rest $* @@ -195,7 +195,7 @@ set ${CASE}.wrf.h3aux01.*; dispose ifiles_n ${sta}/atm/hist $* set ${CASE}.wrf.h3aux02.*; dispose ifiles_n ${sta}/atm/hist $* set ${CASE}.wrf.h3aux03.*; dispose ifiles_n ${sta}/atm/hist $* - + IDX=`expr $IDX + 1` done @@ -281,7 +281,6 @@ set ${CASE}.pop${inst_suffix}.ro.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/ocn/rest $* set ${CASE}.pop${inst_suffix}.d?*; dispose ifiles_n ${sta}/ocn/hist $* set ${CASE}.pop${inst_suffix}.h*; dispose ifiles_n ${sta}/ocn/hist $* - set ${CASE}.docn${inst_suffix}.r.* ; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/ocn/rest $* set ${CASE}.docn${inst_suffix}.rs* ; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/ocn/rest $* set ${CASE}.docn${inst_suffix}.h.* ; dispose ifiles_n ${sta}/ocn/hist $* From nancy at ucar.edu Tue Oct 22 16:07:52 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 22 Oct 2013 16:07:52 -0600 Subject: [Dart-dev] [6535] DART/trunk/models/clm/shell_scripts/CESM_DART_config: Deleted the svn:executable property ... Message-ID: Revision: 6535 Author: thoar Date: 2013-10-22 16:07:52 -0600 (Tue, 22 Oct 2013) Log Message: ----------- Deleted the svn:executable property ... this file should not have execute permission until it gets copied to the CASEROOT directory. Property Changed: ---------------- DART/trunk/models/clm/shell_scripts/CESM_DART_config -------------- next part -------------- Property changes on: DART/trunk/models/clm/shell_scripts/CESM_DART_config ___________________________________________________________________ Deleted: svn:executable - * From nancy at ucar.edu Tue Oct 22 16:09:44 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 22 Oct 2013 16:09:44 -0600 Subject: [Dart-dev] [6536] DART/trunk/models/clm/shell_scripts: Renaming file to be consistent with other CESM components. Message-ID: Revision: 6536 Author: thoar Date: 2013-10-22 16:09:44 -0600 (Tue, 22 Oct 2013) Log Message: ----------- Renaming file to be consistent with other CESM components. Added Paths: ----------- DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid Removed Paths: ------------- DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial -------------- next part -------------- Deleted: DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial 2013-10-22 22:07:52 UTC (rev 6535) +++ DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial 2013-10-22 22:09:44 UTC (rev 6536) @@ -1,779 +0,0 @@ -#!/bin/csh -f -# -# DART software - Copyright 2004 - 2013 UCAR. This open source software is -# provided by UCAR, "as is", without charge, subject to all terms of use at -# http://www.image.ucar.edu/DAReS/DART/DART_download -# -# DART $Id$ - -# --------------------- -# Purpose -# --------------------- -# -# This script is designed to set up, stage, and build a multi-instance run of CESM -# using an I compset where only CLM active and the ocean and land states are specified -# by data files. -# -# CLM: uses the result of the 'b40.20th.005_ens${instance}' experiments - sort of. -# CLM has changed since then and the CLM restart files needed to be converted -# to the new format. I ran 'interpinic' to (essentially) reformat the files -# and changed the CASENAME to 'cesm_test' and used the multi-instance naming -# convention for the new files. -# -# The initial ensemble can be set by specifying the 'finidat' variable in the -# user_nl_clm_${instance}. A FULL pathname to the file is required. This is nice -# for two reasons - one is that you don't need to copy the files and rename them -# (tedious), the second is that the full pathname provides a means of tracking -# the origin of the initial ensemble. -# -# DATM: We are using an ensemble of data atmospheres. This requires modification of -# the stream text files (and the stream files) for each CESM instance. -# -# DOCN: We are using a single data ocean. -# -# Much of the complexity comes from ensuring compatibility between the namelists -# for each instance and staging of the files. The original experiments were run -# before the multi-instance capacity was developed and the naming convention decided. -# Consequently, there is a lot of manipulation of the 'instance' portion of the -# filenames. -# -# This script results in a viable setup for a CESM multi-instance experiment. You -# are STRONGLY encouraged to run the multi-instance CESM a few times and experiment -# with different settings BEFORE you try to assimilate observations. The amount of -# data volume is quite large and you should become comfortable using CESM's restart -# capability to re-stage files in your RUN directory -# -# ${CASEROOT}/CESM_DART_config will augment the CESM case with the required setup -# and configuration to perform a DART assimilation. CASEROOT/CESM_DART_config -# will insert a few dozen lines into the ${CASE}.run script after it makes a backup -# copy. This, and the required setup, can be run at a later date. e.g. you can -# advance an ensemble from 2004-01-01 to 2004-02-01 and then run -# CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, -# and start assimilating observations when CESM stops at 2004-02-01 06Z ... -# -# This script relies heavily on the information in: -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html -# -# --------------------- -# How to use this script. -# --------------------- -# -# -- You will have to read and understand the script in its entirety. -# You will have to modify things outside this script. -# This script sets up a CESM multi-instance run as we understand them and -# it has almost nothing to do with DART. This is intentional. -# -# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory -# or copy it to somewhere that it will be preserved and run it there. -# It will create a CESM 'CASE' directory, where the model will be built, -# and an execution directory, where each forecast (and assimilation) will -# take place. The short term archiver will use a third directory for -# storage of model output until it can be moved to long term storage (HPSS) -# -# -- Examine the whole script to identify things to change for your experiments. -# -# -- Provide the CESM initial ensemble needed by your run. -# -# -- Run this script. -# -# -- If you want to run DART; read, understand, and execute ${CASEROOT}/CESM_DART_config -# -# -- Submit the job using ${CASEROOT}/${CASE}.submit -# -# --------------------- -# Important features -# --------------------- -# -# If you want to change something in your case other than the runtime -# settings, it is safest to delete everything and start the run from scratch. -# For the brave, read -# -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html -# -# and you may be able to salvage something with -# ./cesm_setup -clean -# ./cesm_setup -# ./${case}.clean_build -# ./${case}.build -# -# ============================================================================== -# ==== Set case options -# ============================================================================== - -# the value of "case" will be used many ways; -# directory and file names, both locally and on HPSS, and -# script names; so consider it's length and information content. -# num_instances: Number of ensemble members - -setenv case clm_hybrid -setenv compset I_2000_CN -setenv resolution f09_f09 -setenv cesmtag cesm1_1_1 -setenv num_instances 80 - -# ============================================================================== -# define machines and directories -# -# mach Computer name -# cesmroot Location of the cesm code base -# For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. -# Preferably not a frequently scrubbed location. -# This script will delete any existing caseroot, so this script, -# and other useful things should be kept elsewhere. -# rundir (Future) Run-time directory; scrubbable, large amount of space needed. -# exeroot (Future) directory for executables - scrubbable, large amount of space needed. -# archdir (Future) Short-term archive directory -# until the long-term archiver moves it to permanent storage. -# dartroot Location of _your_ DART installation -# This is passed on to the CESM_DART_config script. -# ============================================================================== - -setenv mach yellowstone -setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag -setenv caseroot /glade/p/work/${USER}/cases/${case} -setenv exeroot /glade/scratch/${USER}/${case}/bld -setenv rundir /glade/scratch/${USER}/${case}/run -setenv archdir /glade/scratch/${USER}/archive/${case} -setenv dartroot /glade/u/home/${USER}/svn/DART/trunk - -# ============================================================================== -# configure settings -# The reference case has dates in it. -# For a 'hybrid' start, these may be unrelated to the refyear, refmon, refday. -# ============================================================================== - -setenv run_refcase 80_member_freerun_one_degree -setenv refyear 2004 -setenv refmon 01 -setenv refday 01 -setenv run_reftod 00000 -setenv run_refdate $refyear-$refmon-$refday - -setenv stream_year_first 2004 -setenv stream_year_last 2004 -setenv stream_year_align 2004 - -setenv CLM_stagedir /glade/scratch/afox/archive/$run_refcase/rest/$refyear-$refmon-$refday-$run_reftod - -# ============================================================================== -# runtime settings -# -# resubmit How many job steps to run on continue runs (will be 0 initially) -# stop_option DART REQUIRES this to be 'nhours' ... DO NOT CHANGE -# stop_n Specifies number of hours to advance the model in a single execution. -# assim_n Specifies number of hours between assimilations. -# clm_dtime dynamical timestep (in seconds) ... 1800 is the default for CLM -# h1nsteps is the number of time steps to put in a single .h1. file -# DART needs to know this and the only time it is known is during -# this configuration step. The run-time settings can lie. -# -# If the long-term archiver is off, you get a chance to examine the files before -# they get moved to long-term storage. You can always submit $CASE.l_archive -# whenever you want to free up space in the short-term archive directory. -# -# ============================================================================== - -setenv short_term_archiver on -setenv long_term_archiver off -setenv resubmit 0 -setenv stop_option nhours -setenv stop_n 72 -setenv assim_n 24 - -# This specifies the number of time steps available to the forward obs. operator. - -@ clm_dtime = 1800 -@ h1nsteps = $assim_n * 3600 / $clm_dtime - -# ============================================================================== -# job settings -# -# queue can be changed during a series by changing the ${case}.run -# timewall can be changed during a series by changing the ${case}.run -# -# TJH: Advancing 80 instances for 72 hours with 2400 pes (80*15*2) with -# an assimilation step took about 10 minutes on yellowstone. -# ============================================================================== - -setenv ACCOUNT P8685xxxx -setenv queue economy -setenv timewall 0:30 - -# ============================================================================== -# set these standard commands based on the machine you are running on. -# ============================================================================== - -set nonomatch # suppress "rm" warnings if wildcard does not match anything - -# The FORCE options are not optional. -# The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following -switch ("`hostname`") - case be*: - # NCAR "bluefire" - set MOVE = '/usr/local/bin/mv -fv' - set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' - set LINK = '/usr/local/bin/ln -fvs' - set REMOVE = '/usr/local/bin/rm -fr' - - breaksw - default: - # NERSC "hopper", NWSC "yellowstone" - set MOVE = '/bin/mv -fv' - set COPY = '/bin/cp -fv --preserve=timestamps' - set LINK = '/bin/ln -fvs' - set REMOVE = '/bin/rm -fr' - - breaksw -endsw - -# ============================================================================== -# Make sure the CESM directories exist. -# VAR is the shell variable name, DIR is the value -# ============================================================================== - -foreach VAR ( cesmroot dartroot CLM_stagedir ) - set DIR = `eval echo \${$VAR}` - if ( ! -d $DIR ) then - echo "ERROR: directory '$DIR' not found" - echo " In the setup script check the setting of: $VAR " - exit -1 - endif -end - -# ============================================================================== -# Create the case - this creates the CASEROOT directory. -# -# For list of the pre-defined cases: ./create_newcase -list -# To create a variant case, see the CESM documentation and carefully -# incorporate any needed changes into this script. -# ============================================================================== - -# fatal idea to make caseroot the same dir as where this setup script is -# since the build process removes all files in the caseroot dir before -# populating it. try to prevent shooting yourself in the foot. - -if ( $caseroot == `dirname $0` ) then - echo "ERROR: the setup script should not be located in the caseroot" - echo "directory, because all files in the caseroot dir will be removed" - echo "before creating the new case. move the script to a safer place." - exit -1 -endif - -echo "removing old files from ${caseroot}" -echo "removing old files from ${exeroot}" -echo "removing old files from ${rundir}" -${REMOVE} ${caseroot} -${REMOVE} ${exeroot} -${REMOVE} ${rundir} - -${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ - -res ${resolution} -compset ${compset} - -if ( $status != 0 ) then - echo "ERROR: Case could not be created." - exit -1 -endif - -# preserve a copy of this script as it was run -set ThisFileName = $0:t -${COPY} $ThisFileName ${caseroot}/${ThisFileName}.original - -# ============================================================================== -# Record the DARTROOT directory and copy the DART setup script to CASEROOT. -# CESM_DART_config can be run at some later date if desired, but it presumes -# to be run from a CASEROOT directory. If CESM_DART_config does not exist locally, -# then it better exist in the expected part of the DARTROOT tree. -# ============================================================================== - -if ( ! -e CESM_DART_config ) then - ${COPY} ${dartroot}/models/clm/shell_scripts/CESM_DART_config . -endif - -if ( -e CESM_DART_config ) then - sed -e "s#BOGUS_DART_ROOT_STRING#$dartroot#" \ - -e "s#HISTORY_OUTPUT_INTERVAL#$assim_n#" < CESM_DART_config >! temp.$$ - ${MOVE} temp.$$ ${caseroot}/CESM_DART_config - chmod 755 ${caseroot}/CESM_DART_config -else - echo "WARNING: the script to configure for data assimilation is not available." - echo " CESM_DART_config should be present locally or in" - echo " ${dartroot}/models/clm/shell_scripts/" - echo " You can stage this script later, but you must manually edit it" - echo " to reflect the location of the DART code tree." -endif - -# ============================================================================== -# Configure the case. -# ============================================================================== - -cd ${caseroot} - -source ./Tools/ccsm_getenv || exit -2 - -@ ptile = $MAX_TASKS_PER_NODE / 2 -@ nthreads = 1 - -# Save a copy for debug purposes -foreach FILE ( *xml ) - if ( ! -e ${FILE}.original ) then - ${COPY} $FILE ${FILE}.original - endif -end - -if ($num_instances < 10) then - - # This is only for the purpose of debugging the code. - @ atm_tasks = $ptile * $num_instances * 4 - @ lnd_tasks = $ptile * $num_instances * 4 - @ ice_tasks = $ptile * $num_instances - @ ocn_tasks = $ptile * $num_instances - @ cpl_tasks = $ptile * $num_instances - @ glc_tasks = $ptile * $num_instances - @ rof_tasks = $ptile * $num_instances * 4 - -else - - # This works, but a more efficient layout should be used - @ atm_tasks = $ptile * $num_instances * 2 - @ lnd_tasks = $ptile * $num_instances * 2 - @ ice_tasks = $ptile * $num_instances - @ ocn_tasks = $ptile * $num_instances - @ cpl_tasks = $ptile * $num_instances - @ glc_tasks = $ptile * $num_instances - @ rof_tasks = $ptile * $num_instances * 2 - -endif - -# echo "task partitioning ... perhaps ... atm // ocn // lnd+ice+glc+rof" -# presently, all components run 'serially' - one after another. -echo "" -echo "ATM gets $atm_tasks" -echo "LND gets $lnd_tasks" -echo "ICE gets $ice_tasks" -echo "OCN gets $ocn_tasks" -echo "CPL gets $cpl_tasks" -echo "GLC gets $glc_tasks" -echo "ROF gets $rof_tasks" -echo "" - -./xmlchange NTHRDS_ATM=$nthreads,NTASKS_ATM=$atm_tasks,NINST_ATM=$num_instances -./xmlchange NTHRDS_LND=$nthreads,NTASKS_LND=$lnd_tasks,NINST_LND=$num_instances -./xmlchange NTHRDS_ICE=$nthreads,NTASKS_ICE=$ice_tasks,NINST_ICE=1 -./xmlchange NTHRDS_OCN=$nthreads,NTASKS_OCN=$ocn_tasks,NINST_OCN=1 -./xmlchange NTHRDS_CPL=$nthreads,NTASKS_CPL=$cpl_tasks -./xmlchange NTHRDS_GLC=$nthreads,NTASKS_GLC=$glc_tasks,NINST_GLC=1 -./xmlchange NTHRDS_ROF=$nthreads,NTASKS_ROF=$rof_tasks,NINST_ROF=$num_instances -./xmlchange ROOTPE_ATM=0 -./xmlchange ROOTPE_LND=0 -./xmlchange ROOTPE_ICE=0 -./xmlchange ROOTPE_OCN=0 -./xmlchange ROOTPE_CPL=0 -./xmlchange ROOTPE_GLC=0 -./xmlchange ROOTPE_ROF=0 - -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/c1158.html#run_start_stop -# "A hybrid run indicates that CESM is initialized more like a startup, but uses -# initialization datasets from a previous case. This is somewhat analogous to a -# branch run with relaxed restart constraints. A hybrid run allows users to bring -# together combinations of initial/restart files from a previous case (specified -# by $RUN_REFCASE) at a given model output date (specified by $RUN_REFDATE). -# Unlike a branch run, the starting date of a hybrid run (specified by $RUN_STARTDATE) -# can be modified relative to the reference case. In a hybrid run, the model does not -# continue in a bit-for-bit fashion with respect to the reference case. The resulting -# climate, however, should be continuous provided that no model source code or -# namelists are changed in the hybrid run. In a hybrid initialization, the ocean -# model does not start until the second ocean coupling (normally the second day), -# and the coupler does a "cold start" without a restart file." - -# The RUN_REFCASE/REFDATE/REFTOD are used by CLM & RTM to specify the namelist input -# filenames - BUT - their buildnml scripts do not use the INSTANCE, so they all specify -# the same (single) filename. This is remedied by using patched [clm,rtm].buildnml.csh -# scripts that exist in the SourceMods directory. - -./xmlchange RUN_TYPE=hybrid -./xmlchange RUN_STARTDATE=$run_refdate -./xmlchange START_TOD=$run_reftod -./xmlchange RUN_REFCASE=$run_refcase -./xmlchange RUN_REFDATE=$run_refdate -./xmlchange RUN_REFTOD=$run_reftod -./xmlchange GET_REFCASE=FALSE -./xmlchange EXEROOT=${exeroot} - -./xmlchange DATM_MODE=CPLHIST3HrWx -./xmlchange DATM_CPLHIST_CASE=$case -./xmlchange DATM_CPLHIST_YR_ALIGN=$refyear -./xmlchange DATM_CPLHIST_YR_START=$refyear -./xmlchange DATM_CPLHIST_YR_END=$refyear - -./xmlchange CALENDAR=GREGORIAN -./xmlchange STOP_OPTION=$stop_option -./xmlchange STOP_N=$stop_n -./xmlchange CONTINUE_RUN=FALSE -./xmlchange RESUBMIT=$resubmit - -./xmlchange PIO_TYPENAME=pnetcdf - -# The river transport model ON is useful only when using an active ocean or -# land surface diagnostics. Setting ROF_GRID to 'null' turns off the RTM. -# so we are also turning on the CLM biogeochemistry. - -./xmlchange ROF_GRID='null' -./xmlchange CLM_CONFIG_OPTS='-bgc cn' - -if ($short_term_archiver == 'off') then - ./xmlchange DOUT_S=FALSE -else - ./xmlchange DOUT_S=TRUE - ./xmlchange DOUT_S_ROOT=${archdir} - ./xmlchange DOUT_S_SAVE_INT_REST_FILES=FALSE -endif -if ($long_term_archiver == 'off') then - ./xmlchange DOUT_L_MS=FALSE -else - ./xmlchange DOUT_L_MS=TRUE - ./xmlchange DOUT_L_MSROOT="csm/${case}" - ./xmlchange DOUT_L_HTAR=FALSE -endif - -# level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) - -./xmlchange DEBUG=FALSE -./xmlchange INFO_DBUG=0 - -# ============================================================================== -# Set up the case. -# This creates the EXEROOT and RUNDIR directories. -# ============================================================================== - -./cesm_setup - -if ( $status != 0 ) then - echo "ERROR: Case could not be set up." - exit -2 -endif - -# ============================================================================== -# Edit the run script to reflect queue and wallclock -# ============================================================================== - -echo '' -echo 'Updating the run script to set wallclock and queue.' -echo '' - -if ( ! -e ${case}.run.original ) then - ${COPY} ${case}.run ${case}.run.original -endif - -source Tools/ccsm_getenv -set BATCH = `echo $BATCHSUBMIT | sed 's/ .*$//'` -switch ( $BATCH ) - case bsub*: - # NCAR "bluefire", "yellowstone" - set TIMEWALL=`grep BSUB ${case}.run | grep -e '-W' ` - set QUEUE=`grep BSUB ${case}.run | grep -e '-q' ` - sed -e "s/$TIMEWALL[3]/$timewall/" \ - -e "s/ptile=[0-9][0-9]*/ptile=$ptile/" \ - -e "s/$QUEUE[3]/$queue/" < ${case}.run >! temp.$$ - ${MOVE} temp.$$ ${case}.run - chmod 755 ${case}.run - breaksw - - default: - - breaksw -endsw - -# ============================================================================== -# Update source files. -# Ideally, using DART would not require any modifications to the model source. -# Until then, this script accesses sourcemods from a hardwired location. -# If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. -# ============================================================================== - -if ( -d ~/${cesmtag}/SourceMods ) then - ${COPY} -r ~/${cesmtag}/SourceMods/* ${caseroot}/SourceMods/ -else - echo "ERROR - No SourceMods for this case." - echo "ERROR - No SourceMods for this case." - echo "DART requires modifications to several src files." - echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" - echo "untar these into your HOME directory - they will create a" - echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." - exit -4 -endif - -# The CESM multi-instance capability is relatively new and still has a few -# implementation bugs. These are known problems and will be fixed soon. -# this should be removed when the files are fixed: - -echo "REPLACING BROKEN CESM FILES HERE - SHOULD BE REMOVED WHEN FIXED" -echo caseroot is ${caseroot} -if ( -d ~/${cesmtag} ) then - - # preserve the original version of the files - if ( ! -e ${caseroot}/Buildconf/clm.buildnml.csh.original ) then - ${MOVE} ${caseroot}/Buildconf/clm.buildnml.csh \ - ${caseroot}/Buildconf/clm.buildnml.csh.original - endif - if ( ! -e ${caseroot}/preview_namelists.original ) then - ${MOVE} ${caseroot}/preview_namelists \ - ${caseroot}/preview_namelists.original - endif - - # patch/replace the broken files - ${COPY} ~/${cesmtag}/clm.buildnml.csh ${caseroot}/Buildconf/. - ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. - -endif - -# ============================================================================== -# Modify namelist templates for each instance. This is a bit of a nuisance in -# that we are pulling in restart and initial files from 'all over the place' -# and each model component has a different strategy. -# -# In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): -# -# CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 -# it does not use the instance string. There is a patch for clm.buildnml.csh -# -# All of these must later on be staged with these same filenames. -# OR - all these namelists can be changed to match whatever has been staged. -# MAKE SURE THE STAGING SECTION OF THIS SCRIPT MATCHES THESE VALUES. -# ============================================================================== - -@ inst = 1 -while ($inst <= $num_instances) - - # following the CESM strategy for 'inst_string' - set inst_string = `printf _%04d $inst` - - # =========================================================================== - set fname = "user_nl_datm${inst_string}" - # =========================================================================== - - echo "dtlimit = 1.5, 1.5, 1.5" >> $fname - echo "fillalgo = 'nn', 'nn', 'nn'" >> $fname - echo "fillmask = 'nomask','nomask','nomask'" >> $fname - echo "mapalgo = 'bilinear','bilinear','bilinear'" >> $fname - echo "mapmask = 'nomask','nomask','nomask'" >> $fname - echo "streams = 'datm.streams.txt.CPLHIST3HrWx.Solar$inst_string $stream_year_align $stream_year_first $stream_year_last'," >> $fname - echo " 'datm.streams.txt.CPLHIST3HrWx.Precip$inst_string $stream_year_align $stream_year_first $stream_year_last'," >> $fname - echo " 'datm.streams.txt.CPLHIST3HrWx.nonSolarNonPrecip$inst_string $stream_year_align $stream_year_first $stream_year_last'" >> $fname - echo "taxmode = 'cycle','cycle','cycle'" >> $fname - echo "tintalgo = 'coszen','nearest','linear'" >> $fname - echo "restfils = 'unset'" >> $fname - echo "restfilm = 'unset'" >> $fname - - # =========================================================================== - set fname = "user_nl_clm${inst_string}" - # =========================================================================== - - # Customize the land namelists - # The filename is built using the REFCASE/REFDATE/REFTOD information. i.e. - # finidat = ${CLM_stagedir}/${case}.clm2${inst_string}.r.${run_refdate}-${run_reftod}.nc - # - # This is the time to consider how DART and CESM will interact. If you intend - # on assimilating flux tower observations (nominally at 30min intervals), - # then it is required to create a .h1. file with the instantaneous flux - # variables every 30 minutes. Despite being in a namelist, these values - # HAVE NO EFFECT once CONTINUE_RUN = TRUE so now is the time to set these. - # - # See page 65 of: - # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf - # - # DART's forward observation operators for these fluxes just reads them - # from the .h1. file rather than trying to create them from the subset of - # CLM variables that are available in the DART state vector. We have a terrible - # time trying to predict the .h1. filename given only current model time. - # DART does not read the clm namelist input that has this information, and - # since it is in a namelist - it can change during the course of a run - BUT - # as discussed above, only the first settings are important. Tricky. - # - # For a HOP TEST ... hist_empty_htapes = .false. - # For a HOP TEST ... use a default hist_fincl1 - # - # Customize the land namelists: - # The initial ensemble can be set by specifying the 'finidat' variable in the - # user_nl_clm${inst_string}. A FULL pathname to the file is required. This is nice - # for two reasons - one is that you don't need to copy the files and rename them - # (tedious), the second is that the full pathname provides a means of tracking - # the origin of the initial ensemble. - - echo "dtime = $clm_dtime," >> $fname - echo "hist_empty_htapes = .false.," >> $fname - echo "hist_fincl1 = 'NEP'," >> $fname - echo "hist_fincl2 = 'NEP','FSH','EFLX_LH_TOT_R'," >> $fname - echo "hist_nhtfrq = -$assim_n,1," >> $fname - echo "hist_mfilt = 1,$h1nsteps," >> $fname - echo "hist_avgflag_pertape = 'A','A'" >> $fname - - @ inst ++ -end - -# ============================================================================== -# to create custom streamfiles ... -# "To modify the contents of a stream txt file, first use preview_namelists to -# obtain the contents of the stream txt files in CaseDocs, and then place a copy -# of the modified stream txt file in $CASEROOT with the string user_ prepended." -# -# -or- -# -# we copy a template stream txt file from the -# $dartroot/models/clm/shell_scripts directory and modify one for each instance. -# -# ============================================================================== - -./preview_namelists - -# This gives us a stream txt file for each instance that we can -# modify for our own purpose. - -foreach FILE (CaseDocs/*streams*) - set FNAME = $FILE:t - - switch ( ${FNAME} ) - case *presaero*: - echo "Using default prescribed aerosol stream.txt file ${FNAME}" - breaksw - case *diatren*: - echo "Using default runoff stream.txt file ${FNAME}" - breaksw - default: - ${COPY} $FILE user_$FNAME - chmod 644 user_$FNAME - breaksw - endsw - -end - -# Replace each default stream txt file with one that uses the CAM DATM -# conditions for a default year and modify the instance number. - -foreach FNAME (user*streams*) - set name_parse = `echo $FNAME | sed 's/\_/ /g'` - @ instance_index = $#name_parse - @ filename_index = $#name_parse - 1 - set streamname = $name_parse[$filename_index] - set instance = $name_parse[$instance_index] - - if (-e $dartroot/models/clm/shell_scripts/user_$streamname*template) then - - echo "Copying DART template for $FNAME and changing instances, refyear" - - ${COPY} $dartroot/models/clm/shell_scripts/user_$streamname*template $FNAME - - sed s/NINST/$instance/g $FNAME >! out.$$ - sed s/REFYEAR/$refyear/g out.$$ >! $FNAME - \rm -f out.$$ - - else - echo "DIED Looking for a DART stream txt template for $FNAME" - echo "DIED Looking for a DART stream txt template for $FNAME" - exit -3 - endif - -end - -./preview_namelists - -# ============================================================================== -# Stage the restarts now that the run directory exists -# THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. -# POP/CAM/CICE read from pointer files. The others use namelist values initially. -# ============================================================================== - -cat << EndOfText >! stage_initial_cesm_files -#!/bin/sh - -cd ${rundir} - -echo '' -echo 'Copying only the required restart files from the staging directory.' -echo 'With CONTINUE_RUN=FALSE, there are many that are not required.' -echo '' - -# This copies over only the minimum number of instances. -# The reference case may more instances than we need, -# and it certainly has more files than we need. - -let inst=1 -while ((\$inst <= $num_instances)); do - inst_string=\`printf _%04d \$inst\` - - echo '' - echo "Staging restarts for instance \$inst of $num_instances" - - ${COPY} ${CLM_stagedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . - - let inst+=1 -done - -exit 0 - -EndOfText -chmod 0755 stage_initial_cesm_files - -./stage_initial_cesm_files - -# ============================================================================== -# build -# ============================================================================== - -echo '' -echo 'Building the case' -echo '' - -./${case}.build - -if ( $status != 0 ) then - echo "ERROR: Case could not be built." - exit -5 -endif - -# ============================================================================== -# What to do next -# ============================================================================== - -echo "" -echo "Time to check the case." -echo "" -echo "1) cd ${rundir}" -echo " check the compatibility between the namelists and the files that were staged." -echo "" -echo "2) cd ${caseroot}" -echo " (on yellowstone) If the ${case}.run script still contains:" -echo ' #BSUB -R "select[scratch_ok > 0]"' -echo " around line 9, delete it." -echo "" -echo "3) If you want to assimilate 'right away', configure and execute" -echo " the ${caseroot}/CESM_DART_config script." -echo "" -echo "4) Verify the contents of env_run.xml and submit the CESM job:" -echo " ./${case}.submit" -echo "" -echo "5) After the job has run, make sure it worked " -echo "" -echo "6) To extend the run in $assim_n '"$stop_option"' steps," -echo " change the env_run.xml variables:" -echo "" -echo " ./xmlchange CONTINUE_RUN=TRUE" -echo " ./xmlchange RESUBMIT=" -echo " ./xmlchange STOP_N=$assim_n" -echo "" -echo "Check the streams listed in the streams text files. If more or different" -echo 'dates need to be added, then do this in the $CASEROOT/user_*files*' -echo "then invoke 'preview_namelists' so you can check the information in the" -echo "CaseDocs or ${rundir} directories." -echo "" - -exit 0 - -# -# $URL$ -# $Revision$ -# $Date$ - Copied: DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid (from rev 6534, DART/trunk/models/clm/shell_scripts/CESM1_1_1_hybrid_initial) =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid (rev 0) +++ DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-22 22:09:44 UTC (rev 6536) @@ -0,0 +1,779 @@ +#!/bin/csh -f +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# --------------------- +# Purpose +# --------------------- +# +# This script is designed to set up, stage, and build a multi-instance run of CESM +# using an I compset where only CLM active and the ocean and land states are specified +# by data files. +# +# CLM: uses the result of the 'b40.20th.005_ens${instance}' experiments - sort of. +# CLM has changed since then and the CLM restart files needed to be converted +# to the new format. I ran 'interpinic' to (essentially) reformat the files +# and changed the CASENAME to 'cesm_test' and used the multi-instance naming +# convention for the new files. +# +# The initial ensemble can be set by specifying the 'finidat' variable in the +# user_nl_clm_${instance}. A FULL pathname to the file is required. This is nice +# for two reasons - one is that you don't need to copy the files and rename them +# (tedious), the second is that the full pathname provides a means of tracking +# the origin of the initial ensemble. +# +# DATM: We are using an ensemble of data atmospheres. This requires modification of +# the stream text files (and the stream files) for each CESM instance. +# +# DOCN: We are using a single data ocean. +# +# Much of the complexity comes from ensuring compatibility between the namelists +# for each instance and staging of the files. The original experiments were run +# before the multi-instance capacity was developed and the naming convention decided. +# Consequently, there is a lot of manipulation of the 'instance' portion of the +# filenames. +# +# This script results in a viable setup for a CESM multi-instance experiment. You +# are STRONGLY encouraged to run the multi-instance CESM a few times and experiment +# with different settings BEFORE you try to assimilate observations. The amount of +# data volume is quite large and you should become comfortable using CESM's restart +# capability to re-stage files in your RUN directory +# +# ${CASEROOT}/CESM_DART_config will augment the CESM case with the required setup +# and configuration to perform a DART assimilation. CASEROOT/CESM_DART_config +# will insert a few dozen lines into the ${CASE}.run script after it makes a backup +# copy. This, and the required setup, can be run at a later date. e.g. you can +# advance an ensemble from 2004-01-01 to 2004-02-01 and then run +# CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, +# and start assimilating observations when CESM stops at 2004-02-01 06Z ... +# +# This script relies heavily on the information in: +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html +# +# --------------------- +# How to use this script. +# --------------------- +# +# -- You will have to read and understand the script in its entirety. +# You will have to modify things outside this script. +# This script sets up a CESM multi-instance run as we understand them and +# it has almost nothing to do with DART. This is intentional. +# +# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory +# or copy it to somewhere that it will be preserved and run it there. +# It will create a CESM 'CASE' directory, where the model will be built, +# and an execution directory, where each forecast (and assimilation) will +# take place. The short term archiver will use a third directory for +# storage of model output until it can be moved to long term storage (HPSS) +# +# -- Examine the whole script to identify things to change for your experiments. +# +# -- Provide the CESM initial ensemble needed by your run. +# +# -- Run this script. +# +# -- If you want to run DART; read, understand, and execute ${CASEROOT}/CESM_DART_config +# +# -- Submit the job using ${CASEROOT}/${CASE}.submit +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime +# settings, it is safest to delete everything and start the run from scratch. +# For the brave, read +# +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html +# +# and you may be able to salvage something with +# ./cesm_setup -clean +# ./cesm_setup +# ./${case}.clean_build +# ./${case}.build +# +# ============================================================================== +# ==== Set case options +# ============================================================================== + +# the value of "case" will be used many ways; +# directory and file names, both locally and on HPSS, and +# script names; so consider it's length and information content. +# num_instances: Number of ensemble members + +setenv case clm_hybrid +setenv compset I_2000_CN +setenv resolution f09_f09 +setenv cesmtag cesm1_1_1 +setenv num_instances 80 + +# ============================================================================== +# define machines and directories +# +# mach Computer name +# cesmroot Location of the cesm code base +# For cesm1_1_1 on yellowstone +# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# Preferably not a frequently scrubbed location. +# This script will delete any existing caseroot, so this script, +# and other useful things should be kept elsewhere. +# rundir (Future) Run-time directory; scrubbable, large amount of space needed. +# exeroot (Future) directory for executables - scrubbable, large amount of space needed. +# archdir (Future) Short-term archive directory +# until the long-term archiver moves it to permanent storage. +# dartroot Location of _your_ DART installation +# This is passed on to the CESM_DART_config script. +# ============================================================================== + +setenv mach yellowstone +setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag +setenv caseroot /glade/p/work/${USER}/cases/${case} +setenv exeroot /glade/scratch/${USER}/${case}/bld +setenv rundir /glade/scratch/${USER}/${case}/run +setenv archdir /glade/scratch/${USER}/archive/${case} +setenv dartroot /glade/u/home/${USER}/svn/DART/trunk + +# ============================================================================== +# configure settings +# The reference case has dates in it. +# For a 'hybrid' start, these may be unrelated to the refyear, refmon, refday. +# ============================================================================== + +setenv run_refcase 80_member_freerun_one_degree +setenv refyear 2004 +setenv refmon 01 +setenv refday 01 +setenv run_reftod 00000 +setenv run_refdate $refyear-$refmon-$refday + +setenv stream_year_first 2004 +setenv stream_year_last 2004 +setenv stream_year_align 2004 + +setenv CLM_stagedir /glade/scratch/afox/archive/$run_refcase/rest/$refyear-$refmon-$refday-$run_reftod + +# ============================================================================== +# runtime settings +# +# resubmit How many job steps to run on continue runs (will be 0 initially) +# stop_option DART REQUIRES this to be 'nhours' ... DO NOT CHANGE +# stop_n Specifies number of hours to advance the model in a single execution. +# assim_n Specifies number of hours between assimilations. +# clm_dtime dynamical timestep (in seconds) ... 1800 is the default for CLM +# h1nsteps is the number of time steps to put in a single .h1. file +# DART needs to know this and the only time it is known is during +# this configuration step. The run-time settings can lie. +# +# If the long-term archiver is off, you get a chance to examine the files before +# they get moved to long-term storage. You can always submit $CASE.l_archive +# whenever you want to free up space in the short-term archive directory. +# +# ============================================================================== + +setenv short_term_archiver on @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Tue Oct 22 16:13:14 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 22 Oct 2013 16:13:14 -0600 Subject: [Dart-dev] [6537] DART/trunk/models/cam/shell_scripts: Renaming file to be consistent with other CESM components. Message-ID: Revision: 6537 Author: thoar Date: 2013-10-22 16:13:14 -0600 (Tue, 22 Oct 2013) Log Message: ----------- Renaming file to be consistent with other CESM components. Added Paths: ----------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid Removed Paths: ------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid -------------- next part -------------- Deleted: DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid 2013-10-22 22:09:44 UTC (rev 6536) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid 2013-10-22 22:13:14 UTC (rev 6537) @@ -1,852 +0,0 @@ -#!/bin/csh -f -# -# DART software - Copyright 2004 - 2013 UCAR. This open source software is -# provided by UCAR, "as is", without charge, subject to all terms of use at -# http://www.image.ucar.edu/DAReS/DART/DART_download -# -# DART $Id: CESM1_1_1_hybrid 6396 2013-08-12 17:51:26Z thoar $ - -# --------------------- -# Purpose -# --------------------- -# -# This script is designed to set up, stage, and build a multi-instance run of CESM -# using an F compset where CAM and CLM are active. The initial states -# come from a single multi-instance reference case so a CESM hybrid setup is used. -# -# Because the atmosphere assimilations typically occur every 6 hours, the methodology -# here reflects that. All of CESM stops every 6 hours (requiring the ocean to couple -# 4x per day) so that a CAM output file would be available for assimilation. -# -# This script results in a viable setup for a CESM multi-instance experiment. You -# are STRONGLY encouraged to run the multi-instance CESM a few times and experiment -# with different settings BEFORE you try to assimilate observations. The amount of -# data volume is quite large and you should become comfortable using CESM's restart -# capability to re-stage files in your RUN directory -# -# CESM/DART requires some modifications to the CESM source code EVEN IF YOU ARE NOT -# assimilating. The modifications for CAM require a modification to the -# CESM ${CASE}.run script. The modifications will invoke a DART script that will -# either assimilate or 'do nothing'. It is stronly recommended that you configure -# the ${CASE}.run script to 'do nothing' initially. -# -# CASEROOT/CESM_DART_config will augment the CESM case with the required setup -# and configuration to perform a DART assimilation and modify the ${CASE}.run script -# as mentioned previously - after it makes a backup copy. -# -# This script relies heavily on the information in: -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html -# -# --------------------- -# How to use this script. -# --------------------- -# -# -- You will have to read and understand the script in its entirety. -# You will have to modify things outside this script. -# This script sets up a CESM multi-instance run as we understand them and -# it has almost nothing to do with DART. This is intentional. -# -# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory -# or copy it to somewhere that it will be preserved and run it there. -# It will create a CESM 'CASE' directory, where the model will be built, -# and a run directory, where each forecast (and assimilation) will -# take place. The short term archiver will use a third directory for -# storage of model output until it can be moved to long term storage (HPSS) -# -# -- Examine the whole script to identify things to change for your experiments. -# -# -- Provide the CESM initial ensemble needed by your run. -# -# -- Run this script. -# -- Edit the DART input.nml that appears in the $CASEROOT directory, if not done here. -# -- Submit the job using $CASEROOT/${case}.submit -# -# -- Read, understand, and execute ${CASEROOT}/CESM_DART_config -# -# Table of contents: -# Set up case and directory names -# Refined grid mods -# create_newcase -# cd caseroot -# xmlchanges -# cesm_setup -# namelist mods -# preview_namelists -# SourceMods -# case.build -# stage ICs -# update the run script -# Bring in parts of DART; executables, support files,... -# -# --------------------- -# Important features -# --------------------- -# -# If you want to change something in your case other than the runtime -# settings, it is safest to delete everything and start the run from scratch. -# For the brave, read -# -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html -# -# and you may be able to salvage something with -# ./cesm_setup -clean -# ./cesm_setup -# ./${case}.clean_build -# ./${case}.build -# -# ============================================================================== -# ==== Set case options -# ============================================================================== - -# the value of "case" will be used many ways; -# directory and file names, both locally and on HPSS, and -# script names; so consider its length and information content. -# num_instances: Number of ensemble members -# BUG 1384 applies here, so ocean and atm/land must be at same resolution. -# 0.9x1.25_0.9x1.25 (f09_f09) -# 1 deg grid for atm/lnd, gx1v6 mask. Normally used for CAM with prescribed lnd, ocn and ice. - -setenv case test_cam -setenv compset F_AMIP_CAM5 -setenv resolution f09_f09 -setenv cesmtag cesm1_1_1 -setenv num_instances 4 - -# ============================================================================== -# define machines and directories -# -# mach Computer name -# cesmroot Location of the cesm code base -# For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. -# Preferably not a frequently scrubbed location. -# This script will delete any existing caseroot, so this script, -# and other useful things should be kept elsewhere. -# rundir (Future) Run-time directory; scrubbable, large amount of space needed. -# exeroot (Future) directory for executables - scrubbable, large amount of space needed. -# archdir (Future) Short-term archive directory -# until the long-term archiver moves it to permanent storage. -# dartroot Location of _your_ DART installation -# This is passed on to the CESM_DART_config script. -# ============================================================================== - -setenv mach yellowstone -setenv cesm_datadir /glade/p/cesm/cseg/inputdata - -setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag -setenv caseroot /glade/p/work/${USER}/cases/${case} -setenv exeroot /glade/scratch/${USER}/${case}/bld -setenv rundir /glade/scratch/${USER}/${case}/run -setenv archdir /glade/scratch/${USER}/archive/${case} -setenv dartroot /glade/u/home/${USER}/svn/DART/trunk - -# ============================================================================== -# configure settings -# The reference case has dates in it. -# For a 'hybrid' start, these may be unrelated to the refyear, refmon, refday. -# ============================================================================== - -setenv run_refcase cesm_hybrid -setenv refyear 2004 -setenv refmon 01 -setenv refday 10 -setenv run_reftod 00000 -setenv run_refdate $refyear-$refmon-$refday - -# THIS IS THE LOCATION of the 'reference case'. - -setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} - -# setenv stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} - -# ============================================================================== -# runtime settings -# -# sst_dataset Data ocean file and -# sst_grid supporting grid file. Must be consistent. -# -# resubmit How many job steps to run on continue runs (will be 0 initially) -# stop_option Units for determining the forecast length between assimilations -# stop_n Number of time units in the first forecast -# -# If the long-term archiver is off, you get a chance to examine the files before -# they get moved to long-term storage. You can always submit $CASE.l_archive -# whenever you want to free up space in the short-term archive directory. -# -# ============================================================================== - -setenv start_year 2004 -setenv start_month 01 -setenv start_day 10 -setenv start_tod 0 -setenv run_startdate ${start_year}-${start_month}-${start_day} -setenv year_start 1850 -setenv year_end 2012 - -# setenv sst_grid ${cesm_datadir}/ocn/docn7/domain.ocn.gx1v6.090206.nc -setenv sst_grid ${cesm_datadir}/share/domains/domain.ocn.fv0.9x1.25_gx1v6.130409.nc -setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc - -setenv short_term_archiver on -setenv long_term_archiver off -setenv resubmit 0 -setenv stop_option nhours -setenv stop_n 6 - -# ============================================================================== -# job settings -# -# queue can be changed during a series by changing the ${case}.run -# timewall can be changed during a series by changing the ${case}.run -# -# TJH: Advancing 30 instances for 6 hours with 900 pes -# took less than 10 minutes on yellowstone. -# ============================================================================== - -setenv ACCOUNT P86850054 -setenv queue premium -setenv timewall 0:20 - -# ============================================================================== -# set these standard commands based on the machine you are running on. -# ============================================================================== - -set nonomatch # suppress "rm" warnings if wildcard does not match anything - -# The FORCE options are not optional. -# The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following -switch ("`hostname`") - case be*: - # NCAR "bluefire" - set MOVE = '/usr/local/bin/mv -fv' - set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' - set LINK = '/usr/local/bin/ln -fvs' - set REMOVE = '/usr/local/bin/rm -fr' - - breaksw - default: - # NERSC "hopper", NWSC "yellowstone" - set MOVE = '/bin/mv -fv' - set COPY = '/bin/cp -fv --preserve=timestamps' - set LINK = '/bin/ln -fvs' - set REMOVE = '/bin/rm -fr' - - breaksw -endsw - -# ============================================================================== -# Make sure the CESM directories exist. -# VAR is the shell variable name, DIR is the value -# ============================================================================== - -foreach VAR ( cesmroot dartroot stagedir ) - set DIR = `eval echo \${$VAR}` - if ( ! -d $DIR ) then - echo "ERROR: directory '$DIR' not found" - echo " In the setup script check the setting of: $VAR " - exit -1 - endif -end - -# ============================================================================== -# Create the case - this creates the CASEROOT directory. -# -# For list of the pre-defined component sets: ./create_newcase -list -# To create a variant compset, see the CESM documentation and carefully -# incorporate any needed changes into this script. -# ============================================================================== - -# fatal idea to make caseroot the same dir as where this setup script is -# since the build process removes all files in the caseroot dir before -# populating it. try to prevent shooting yourself in the foot. - -if ( $caseroot == `dirname $0` ) then - echo "ERROR: the setup script should not be located in the caseroot" - echo "directory, because all files in the caseroot dir will be removed" - echo "before creating the new case. move the script to a safer place." - exit -1 -endif - -echo "removing old files from ${caseroot}" -echo "removing old files from ${exeroot}" -echo "removing old files from ${rundir}" -${REMOVE} ${caseroot} -${REMOVE} ${exeroot} -${REMOVE} ${rundir} - -${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ - -res ${resolution} -compset ${compset} - -if ( $status != 0 ) then - echo "ERROR: Case could not be created." - exit -1 -endif - -# preserve a copy of this script as it was run -set ThisFileName = $0:t -${COPY} $ThisFileName ${caseroot}/${ThisFileName}.original - -# ============================================================================== -# Record the DARTROOT directory and copy the DART setup script to CASEROOT. -# CESM_DART_config can be run at some later date if desired, but it presumes -# to be run from a CASEROOT directory. If CESM_DART_config does not exist locally, -# then it better exist in the expected part of the DARTROOT tree. -# ============================================================================== - -if ( ! -e CESM_DART_config ) then - ${COPY} ${dartroot}/models/cam/shell_CESM/CESM_DART_config . -endif - -if ( -e CESM_DART_config ) then - sed -e "s#BOGUS_DART_ROOT_STRING#$dartroot#" < CESM_DART_config >! temp.$$ - ${MOVE} temp.$$ ${caseroot}/CESM_DART_config - chmod 755 ${caseroot}/CESM_DART_config -else - echo "ERROR: the script to configure for data assimilation is not available." - echo " CESM_DART_config MUST be present locally or in" - echo " ${dartroot}/models/CESM/shell_scripts/" - exit -2 -endif - -# ============================================================================== -# Configure the case. -# ============================================================================== - -cd ${caseroot} - -source ./Tools/ccsm_getenv || exit -2 - -# MAX_TASKS_PER_NODE comes from $case/Tools/mkbatch.$machine -@ ptile = $MAX_TASKS_PER_NODE / 2 -@ nthreads = 1 - -# Save a copy for debug purposes -foreach FILE ( *xml ) - if ( ! -e ${FILE}.original ) then - ${COPY} $FILE ${FILE}.original - endif -end - -if ($num_instances < 4) then - - # This is only for the purpose of debugging the code. - @ atm_tasks = $ptile * $num_instances * 4 - @ lnd_tasks = $ptile * $num_instances * 4 - @ ice_tasks = $ptile * $num_instances * 4 - @ ocn_tasks = $ptile * $num_instances - @ cpl_tasks = $ptile * $num_instances - @ glc_tasks = $ptile * $num_instances - @ rof_tasks = $ptile * $num_instances - -else - - # This works, but a more efficient layout should be used - # CAM only; 1 node/instance; no cross node communications - # as long as memory is big enough. - @ atm_tasks = $ptile * $num_instances * 4 - @ lnd_tasks = $ptile * $num_instances * 4 - @ ice_tasks = $ptile * $num_instances * 4 - @ ocn_tasks = $ptile * $num_instances - @ cpl_tasks = $ptile * $num_instances - @ glc_tasks = $ptile * $num_instances - @ rof_tasks = $ptile * $num_instances - -endif - -# echo "task partitioning ... perhaps ... atm // ocn // lnd+ice+glc+rof" -# presently, all components run 'serially' - one after another. -# Yellowstone: no large memory nodes, and 15 tasks/node is recommended. -# Edwards says there's no speed up by running non-active components concurrently, -# after ATM has run, so just run all components sequentially. -# BUT, do arrange it so that each member(instance) spans complete nodes: -# modulo(total pe count / number of instances, 15) == 0. - -echo "" -echo "ATM gets $atm_tasks" -echo "LND gets $lnd_tasks" -echo "ICE gets $ice_tasks" -echo "OCN gets $ocn_tasks" -echo "CPL gets $cpl_tasks" -echo "GLC gets $glc_tasks" -echo "ROF gets $rof_tasks" -echo "" - -./xmlchange NTHRDS_ATM=$nthreads,NTASKS_ATM=$atm_tasks,NINST_ATM=$num_instances -./xmlchange NTHRDS_LND=$nthreads,NTASKS_LND=$lnd_tasks,NINST_LND=$num_instances -./xmlchange NTHRDS_ICE=$nthreads,NTASKS_ICE=$ice_tasks,NINST_ICE=$num_instances -./xmlchange NTHRDS_OCN=$nthreads,NTASKS_OCN=$ocn_tasks,NINST_OCN=1 -./xmlchange NTHRDS_CPL=$nthreads,NTASKS_CPL=$cpl_tasks -./xmlchange NTHRDS_GLC=$nthreads,NTASKS_GLC=$glc_tasks,NINST_GLC=1 -./xmlchange NTHRDS_ROF=$nthreads,NTASKS_ROF=$rof_tasks,NINST_ROF=1 -./xmlchange ROOTPE_ATM=0 -./xmlchange ROOTPE_LND=0 -./xmlchange ROOTPE_ICE=0 -./xmlchange ROOTPE_OCN=0 -./xmlchange ROOTPE_CPL=0 -./xmlchange ROOTPE_GLC=0 -./xmlchange ROOTPE_ROF=0 - -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/c1158.html#run_start_stop -# "A hybrid run indicates that CESM is initialized more like a startup, but uses -# initialization datasets from a previous case. This is somewhat analogous to a -# branch run with relaxed restart constraints. A hybrid run allows users to bring -# together combinations of initial/restart files from a previous case (specified -# by $RUN_REFCASE) at a given model output date (specified by $RUN_REFDATE). -# Unlike a branch run, the starting date of a hybrid run (specified by $RUN_STARTDATE) -# can be modified relative to the reference case. In a hybrid run, the model does not -# continue in a bit-for-bit fashion with respect to the reference case. The resulting -# climate, however, should be continuous provided that no model source code or -# namelists are changed in the hybrid run. In a hybrid initialization, the ocean -# model does not start until the second ocean coupling (normally the second day), -# and the coupler does a "cold start" without a restart file." - -# TJH: -# DART's CAM implementation causes a bit more complexity. DART only uses CAM _initial_ -# files, not RESTART files, so there are sourcemods to force a hybrid start for CAM to -# read initial files - even when CONTINUE_RUN = TRUE. - -./xmlchange RUN_TYPE=hybrid -./xmlchange RUN_STARTDATE=$run_startdate -./xmlchange START_TOD=$start_tod -./xmlchange RUN_REFCASE=$run_refcase -./xmlchange RUN_REFDATE=$run_refdate -./xmlchange RUN_REFTOD=$run_reftod -./xmlchange BRNCH_RETAIN_CASENAME=FALSE -./xmlchange GET_REFCASE=FALSE -./xmlchange EXEROOT=${exeroot} - -./xmlchange SSTICE_DATA_FILENAME=$sst_dataset -./xmlchange SSTICE_GRID_FILENAME=$sst_grid -./xmlchange SSTICE_YEAR_ALIGN=$year_start -./xmlchange SSTICE_YEAR_START=$year_start -./xmlchange SSTICE_YEAR_END=$year_end - -# Do not change the CALENDAR or the CONTINUE_RUN - -./xmlchange CALENDAR=GREGORIAN - -./xmlchange STOP_OPTION=$stop_option -./xmlchange STOP_N=$stop_n -./xmlchange CONTINUE_RUN=FALSE -./xmlchange RESUBMIT=$resubmit - -./xmlchange PIO_TYPENAME=pnetcdf - -# COUPLING discussion. F compsets are 'tight' coupling. -# Only change the ATM_NCPL ... everything is based on this one value, -# including CAM physics and dynamics timesteps. -# Default values for coupling are preserved in env_run.xml.original - -./xmlchange NCPL_BASE_PERIOD=day -./xmlchange ATM_NCPL=48 - -# TJH FIXME ... any reason to turn ON the biogeochemistry in CLM when atmos only? -# There is no compset that has everything we want, -# so we are also turning on the CLM biogeochemistry. -#./xmlchange CLM_CONFIG_OPTS='-bgc cn' - -if ($short_term_archiver == 'off') then - ./xmlchange DOUT_S=FALSE -else - ./xmlchange DOUT_S=TRUE - ./xmlchange DOUT_S_ROOT=${archdir} - ./xmlchange DOUT_S_SAVE_INT_REST_FILES=FALSE -endif -if ($long_term_archiver == 'off') then - ./xmlchange DOUT_L_MS=FALSE -else - ./xmlchange DOUT_L_MS=TRUE - ./xmlchange DOUT_L_MSROOT="csm/${case}" - ./xmlchange DOUT_L_HTAR=FALSE -endif - -# The river transport model ON is useful only when using an active ocean or -# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. -# If you turn it ON, you will have to stage initial files etc. - -./xmlchange ROF_GRID='null' -./xmlchange RTM_MODE='NULL' - -# level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) -# Early versions (late 2012) of CAM-SE could not handle these being other than FALSE and 0. -./xmlchange DEBUG=TRUE -./xmlchange INFO_DBUG=1 - -# ============================================================================== -# Set up the case. -# This creates the EXEROOT and RUNDIR directories. -# ============================================================================== - -./cesm_setup - -if ( $status != 0 ) then - echo "ERROR: Case could not be set up." - exit -2 -endif - -# ============================================================================== -# Edit the run script to reflect queue and wallclock -# ============================================================================== - -echo '' -echo 'Updating the run script to set wallclock and queue.' -echo '' - -if ( ! -e ${case}.run.original ) then - ${COPY} ${case}.run ${case}.run.original -endif - -source Tools/ccsm_getenv -set BATCH = `echo $BATCHSUBMIT | sed 's/ .*$//'` -switch ( $BATCH ) - case bsub*: - # NCAR "bluefire", "yellowstone" - set TIMEWALL=`grep BSUB ${case}.run | grep -e '-W' ` - set QUEUE=`grep BSUB ${case}.run | grep -e '-q' ` - sed -e "s/$TIMEWALL[3]/$timewall/" \ - -e "s/ptile=[0-9][0-9]*/ptile=$ptile/" \ - -e "s/$QUEUE[3]/$queue/" < ${case}.run >! temp.$$ - ${MOVE} temp.$$ ${case}.run - chmod 755 ${case}.run - breaksw - - default: - - breaksw -endsw - -# This is the part that modifies the run script to allow CESM to advance -# correctly given the modifications necessary for CAM. It also copies several -# required DART files to the caseroot directory. - -./CESM_DART_config - -# ============================================================================== -# Update source files. -# Ideally, using DART would not require any modifications to the model source. -# Until then, this script accesses sourcemods from a hardwired location. -# If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. -# ============================================================================== - -if ( -d ~/${cesmtag}/SourceMods ) then - ${COPY} -r ~/${cesmtag}/SourceMods/* ${caseroot}/SourceMods/ -else - echo "ERROR - No SourceMods for this case." - echo "ERROR - No SourceMods for this case." - echo "DART requires modifications to several src files." - echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" - echo "untar these into your HOME directory - they will create a" - echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." - exit -4 -endif - -# The CESM multi-instance capability is relatively new and still has a few -# implementation bugs. These are known problems and will be fixed soon. -# this should be removed when the files are fixed: - -echo "REPLACING BROKEN CESM FILES HERE - SHOULD BE REMOVED WHEN FIXED" -echo caseroot is ${caseroot} -if ( -d ~/${cesmtag} ) then - - # preserve the original version of the files - if ( ! -e ${caseroot}/Buildconf/clm.buildnml.csh.original ) then - ${MOVE} ${caseroot}/Buildconf/clm.buildnml.csh \ - ${caseroot}/Buildconf/clm.buildnml.csh.original - endif - if ( ! -e ${caseroot}/preview_namelists.original ) then - ${MOVE} ${caseroot}/preview_namelists \ - ${caseroot}/preview_namelists.original - endif - - # patch/replace the broken files - ${COPY} ~/${cesmtag}/clm.buildnml.csh ${caseroot}/Buildconf/. - ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. - -endif - -# ============================================================================== -# Modify namelist templates for each instance. This is a bit of a nuisance in -# that we are pulling in restart and initial files from 'all over the place' -# and each model component has a different strategy. -# -# In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): -# -# CAM has been forced to read initial files - specified by namelist var:ncdata -# CICE reads from namelist variable 'ice_ic' -# CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 -# it does not use the instance string. There is a patch for clm.buildnml.csh -# -# All of these must later on be staged with these same filenames. -# OR - all these namelists can be changed to match whatever has been staged. -# MAKE SURE THE STAGING SECTION OF THIS SCRIPT MATCHES THESE VALUES. -# ============================================================================== - -@ inst = 1 -while ($inst <= $num_instances) - - # following the CESM strategy for 'inst_string' - set inst_string = `printf _%04d $inst` - - # =========================================================================== - set fname = "user_nl_cam${inst_string}" - # =========================================================================== - # For a HOP TEST ... empty_htapes = .false. - # For a HOP TEST ... use a default fincl1 - # inithist == 'ENDOFRUN' ensures that CAM writes an initial file every time it stops. - # fincl1,nhtfrq,mfilt all control the history file containing a REQUIRED PHIS field. - - echo " inithist = 'ENDOFRUN'" >> ${fname} - echo " ncdata = 'cam_initial${inst_string}.nc'" >> ${fname} - echo " empty_htapes = .true. " >> ${fname} - echo " fincl1 = 'PHIS:I' " >> ${fname} - echo " nhtfrq = -$stop_n " >> ${fname} - echo " mfilt = 1 " >> ${fname} - - # A lot of the files specified here are because the 'default' files only - # contain data through 2005 and we are interested in timeframes after that. - - set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" - - # CAM5 does prognostic aerosols by default. If you want to prescribe them, - # use the following variables with your own settings ... - # Filenames with 'clim' in their names should specify aerodep_flx_type = 'CYCLICAL'. - # Other values are 'CYCLICAL', 'SERIAL', 'INTERP_MISSING_MONTHS', or 'FIXED'. - - echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} - echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} - echo "aerodep_flx_type = 'CYCLICAL' " >> ${fname} - # Set if aerodep_flx_type = 'CYCLICAL' - echo "aerodep_flx_cycle_yr = 2000 " >> ${fname} - - # =========================================================================== - set fname = "user_nl_clm${inst_string}" - # =========================================================================== - - # Customize the land namelists - # The filename is built using the REFCASE/REFDATE/REFTOD information. - # Making a compact .h0. file is a good idea, since the clm restart files - # do not have all the metadata required to reconstruct a gridded field. - # 'TSA' is 2m surface air temperature. - # TJH ... should make monthly average file or something ... - # 1.3 MB is 100x bigger than CAM history file. See page 65 of - # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf - # - # Every 6 hours - # echo "hist_mfilt = 1" >> $fname - # echo "hist_nhtfrq = -$stop_n" >> $fname - # Every month - # echo "hist_mfilt = 1" >> $fname - # echo "hist_nhtfrq = 0" >> $fname - -# KDR; Was the ntapes bug fixed? -# That's where the CLM restart file always has dimension ntapes, -# but it's set = 0 for hist_empty_htapes = .false., -# which prevents the next forecast from starting. -# Kluzek fixed it in some recent (<9/2013) version, but not 1_1_1. -# There's a supposed fix in ~/cesm1_1_1/SourceMods/src.clm/HistFileMod.F90, -# but it only checks whether ntapes = ntapes_on_file, and is_restart() returning 'true'. -# It doesn't check whether the size is 0. -# That solution can't work in hybrid mode. See ~/Homme/notes:9/17/2013 -# KDR - - echo "hist_empty_htapes = .true." >> $fname - echo "hist_fincl1 = 'TSA'" >> $fname - echo "hist_nhtfrq = -$stop_n" >> $fname - echo "hist_mfilt = 1" >> $fname - echo "hist_avgflag_pertape = 'A'" >> $fname - - # =========================================================================== - set fname = "user_nl_cice${inst_string}" - # =========================================================================== - # CICE Namelists - - echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname - - @ inst ++ -end - -./preview_namelists - -# ============================================================================== -# Stage the restarts now that the run directory exists -# THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. -# ============================================================================== - -cat << EndOfText >! stage_cesm_files -#!/bin/csh -f -# This script can be used to help restart an experiment from any previous step. -# The files are copied to the RUN directory. -# Set the 'restart_time' variable below to define when to restart the experiment. -# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine -# the origin of the files. - -echo 'Copying the required files to the run directory.' -echo 'No matter what, make sure the restart_time is set to the desired time.' -echo '' -echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' -echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' -echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' -echo 'just the pointer files need to be updated.' -echo '' - -cd ${caseroot} -source ./Tools/ccsm_getenv || exit -2 -cd ${RUNDIR} - -set restart_time = $RUN_REFDATE-$RUN_REFTOD - -#---------------------------------------------------------------------- -# This block copies over a set of restart files from -# this experiment and exits. Intended use to restart a failed run. -#---------------------------------------------------------------------- - -echo "continue_run is \$CONTINUE_RUN" - -if ( \$CONTINUE_RUN == TRUE ) then - if ( \$DOUT_S == TRUE ) then - - # the restarts should be in the short term archive directory - # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back - ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . - - else - - # All the required files must still be in the run directory. - # Just need to update the pointer files. - - @ inst=1 - while (\$inst <= $num_instances) - - set inst_string = \`printf _%04d \$inst\` - - echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv - echo "${CASE}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} - echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} - echo "${CASE}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} - - - @ inst ++ - end - - endif - - # Now that we're guaranteed to have the cam initial file, link to our hardwired name. - @ inst=1 - while (\$inst <= $num_instances) - set inst_string = \`printf _%04d \$inst\` - ${LINK} ${CASE}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc - @ inst ++ - end - - # relink the pointer file for the data ocean - echo "${CASE}.docn.r.\${restart_time}.nc" >! rpointer.ocn - echo "" >> rpointer.ocn - echo "${CASE}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn - echo "" >> rpointer.ocn - - exit 0 -endif - -#---------------------------------------------------------------------- -# This block prepares the initial run directory for a startup run. -# The CLM, CICE, ROF namelists specify a relative path for the initial file. -# The CAM namelist specifies a static name. -# POP uses pointer files. -# -# The reference case may more instances than we need, -# and it certainly has more files than we need. -# This stages only the minimum number of instances and files. -# -# Remember - in a hybrid case - when CONTINUE_RUN = FALSE, -# the run_refdate and the time in the filenames can be unrelated. -#---------------------------------------------------------------------- - -set filedir = ${stagedir} - -@ inst=1 -while (\$inst <= $num_instances) - - set inst_string = \`printf _%04d \$inst\` - - echo '' - echo "Staging restarts for instance \$inst of $num_instances" - - ${LINK} \${filedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . - ${LINK} \${filedir}/${run_refcase}.cice\${inst_string}.r.${run_refdate}-${run_reftod}.nc . - ${LINK} \${filedir}/${run_refcase}.cam\${inst_string}.i.${run_refdate}-${run_reftod}.nc cam_initial\${inst_string}.nc - - @ inst ++ -end - -exit 0 - -EndOfText -chmod 0755 stage_cesm_files - -./stage_cesm_files - -# ============================================================================== -# build -# ============================================================================== - -echo '' -echo 'Building the case' -echo '' - -./${case}.build - -if ( $status != 0 ) then - echo "ERROR: Case could not be built." - exit -5 -endif - -# ============================================================================== -# What to do next -# ============================================================================== - -echo "" -echo "Time to check the case." -echo "" -echo "1) cd ${rundir}" -echo " and check the compatibility between the namelists/pointer" -echo " files and the files that were staged." -echo "" -echo "2) cd ${caseroot}" -echo " (on yellowstone) If the ${case}.run script still contains:" -echo ' #BSUB -R "select[scratch_ok > 0]"' -echo " around line 9, delete it." -echo "" -echo "3) If you want to simply advance CESM without using DART, make sure the" -echo " ${case}.run script references the 'no_assimilate.csh' script." -echo " If you want to assimilate, reference the 'assimilate.csh' script." -echo " The default action is to assimilate." -echo "" -echo "4) Verify the contents of env_run.xml and submit the CESM job:" -echo " ./${case}.submit" -echo "" -echo "5) After the job has run, check to make sure it worked and that" -echo " a: the right restart files exist in the run directory," -echo " b: (if you're running DART) the archive dart/hist directory has the DART output," -echo " c: everything is working correctly ..." -echo "" -echo "6) To extend the run in $stop_n '"$stop_option"' steps," -echo " change the env_run.xml variables:" -echo "" -echo " ./xmlchange CONTINUE_RUN=TRUE" -echo " ./xmlchange RESUBMIT=" -echo " ./xmlchange STOP_N=$stop_n" -echo "" -echo " and" -echo " ./${case}.submit" -echo "" - -exit 0 - -# -# $URL: https://subversion.ucar.edu/DAReS/DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid $ -# $Revision: 6396 $ -# $Date: 2013-08-12 11:51:26 -0600 (Mon, 12 Aug 2013) $ - Copied: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid (from rev 6530, DART/trunk/models/cam/shell_scripts/CESM1_1_1_hybrid) =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid (rev 0) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-22 22:13:14 UTC (rev 6537) @@ -0,0 +1,852 @@ +#!/bin/csh -f +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id: CESM1_1_1_hybrid 6396 2013-08-12 17:51:26Z thoar $ + +# --------------------- +# Purpose +# --------------------- +# +# This script is designed to set up, stage, and build a multi-instance run of CESM +# using an F compset where CAM and CLM are active. The initial states +# come from a single multi-instance reference case so a CESM hybrid setup is used. +# +# Because the atmosphere assimilations typically occur every 6 hours, the methodology +# here reflects that. All of CESM stops every 6 hours (requiring the ocean to couple +# 4x per day) so that a CAM output file would be available for assimilation. +# +# This script results in a viable setup for a CESM multi-instance experiment. You +# are STRONGLY encouraged to run the multi-instance CESM a few times and experiment +# with different settings BEFORE you try to assimilate observations. The amount of +# data volume is quite large and you should become comfortable using CESM's restart +# capability to re-stage files in your RUN directory +# +# CESM/DART requires some modifications to the CESM source code EVEN IF YOU ARE NOT +# assimilating. The modifications for CAM require a modification to the +# CESM ${CASE}.run script. The modifications will invoke a DART script that will +# either assimilate or 'do nothing'. It is stronly recommended that you configure +# the ${CASE}.run script to 'do nothing' initially. +# +# CASEROOT/CESM_DART_config will augment the CESM case with the required setup +# and configuration to perform a DART assimilation and modify the ${CASE}.run script +# as mentioned previously - after it makes a backup copy. +# +# This script relies heavily on the information in: +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html +# +# --------------------- +# How to use this script. +# --------------------- +# +# -- You will have to read and understand the script in its entirety. +# You will have to modify things outside this script. +# This script sets up a CESM multi-instance run as we understand them and +# it has almost nothing to do with DART. This is intentional. +# +# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory +# or copy it to somewhere that it will be preserved and run it there. +# It will create a CESM 'CASE' directory, where the model will be built, +# and a run directory, where each forecast (and assimilation) will +# take place. The short term archiver will use a third directory for +# storage of model output until it can be moved to long term storage (HPSS) +# +# -- Examine the whole script to identify things to change for your experiments. +# +# -- Provide the CESM initial ensemble needed by your run. +# +# -- Run this script. +# -- Edit the DART input.nml that appears in the $CASEROOT directory, if not done here. +# -- Submit the job using $CASEROOT/${case}.submit +# +# -- Read, understand, and execute ${CASEROOT}/CESM_DART_config +# +# Table of contents: +# Set up case and directory names +# Refined grid mods +# create_newcase +# cd caseroot +# xmlchanges +# cesm_setup +# namelist mods +# preview_namelists +# SourceMods +# case.build +# stage ICs +# update the run script +# Bring in parts of DART; executables, support files,... +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime +# settings, it is safest to delete everything and start the run from scratch. +# For the brave, read +# +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html +# +# and you may be able to salvage something with +# ./cesm_setup -clean +# ./cesm_setup +# ./${case}.clean_build +# ./${case}.build +# +# ============================================================================== +# ==== Set case options +# ============================================================================== + +# the value of "case" will be used many ways; +# directory and file names, both locally and on HPSS, and +# script names; so consider its length and information content. +# num_instances: Number of ensemble members +# BUG 1384 applies here, so ocean and atm/land must be at same resolution. +# 0.9x1.25_0.9x1.25 (f09_f09) +# 1 deg grid for atm/lnd, gx1v6 mask. Normally used for CAM with prescribed lnd, ocn and ice. + +setenv case test_cam +setenv compset F_AMIP_CAM5 +setenv resolution f09_f09 +setenv cesmtag cesm1_1_1 +setenv num_instances 4 + +# ============================================================================== +# define machines and directories +# +# mach Computer name +# cesmroot Location of the cesm code base +# For cesm1_1_1 on yellowstone +# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# Preferably not a frequently scrubbed location. +# This script will delete any existing caseroot, so this script, +# and other useful things should be kept elsewhere. @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Tue Oct 22 17:25:29 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 22 Oct 2013 17:25:29 -0600 Subject: [Dart-dev] [6538] DART/trunk/models/cam/shell_scripts: The best-practices versions of the perfect model scripts. Message-ID: Revision: 6538 Author: thoar Date: 2013-10-22 17:25:29 -0600 (Tue, 22 Oct 2013) Log Message: ----------- The best-practices versions of the perfect model scripts. Added Paths: ----------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh -------------- next part -------------- Added: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo (rev 0) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-22 23:25:29 UTC (rev 6538) @@ -0,0 +1,838 @@ +#!/bin/csh -f +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id: CESM1_1_1_hybrid 6396 2013-08-12 17:51:26Z thoar $ + +# --------------------- +# Purpose +# --------------------- +# +# This script is designed to set up, stage, and build a single-instance run of CESM +# using an F compset where CAM and CLM are active. The initial states +# come from a single multi-instance reference case so a CESM hybrid setup is used. +# +# DOCN: We are using a single data ocean. +# +# This script has a counterpart that is a multi-instance setup for either a free +# run or an assimilation experiment. To make it easy to maintain (and hopefully +# understand), the two scripts are intended to parallel each other. That means this +# script performs a lot of manipulation of the 'instance' portion of the +# filenames, which seems unnecessary initially. +# +# This script results in a viable setup for a CESM single instance experiment. You +# are STRONGLY encouraged to run the single instance CESM a few times and experiment +# with different settings BEFORE you try to generate 'perfect' observations. +# You should become comfortable using CESM's restart capability to re-stage files +# in your RUN directory. +# +# ${CASEROOT}/CESM_DART_config will augment the CESM case with the required setup +# and configuration to use DART to harvest synthetic observations. CESM_DART_config +# will insert a few dozen lines into the ${case}.run script after it makes a backup +# copy. This, and the required setup, can be run at a later date. e.g. you can +# advance an ensemble from 2004-01-01 to 2004-02-01 and then run +# CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, +# and start harvesting synthetic observations when CESM stops at 2004-02-01 06Z ... +# +# This script relies heavily on the information in: +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html +# +# --------------------- +# How to use this script. +# --------------------- +# +# -- You will have to read and understand the script in its entirety. +# You will have to modify things outside this script. +# This script sets up a CESM single instance run as we understand them and +# it has almost nothing to do with DART. This is intentional. +# +# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory +# or copy it to somewhere that it will be preserved and run it there. +# It will create a CESM 'CASE' directory, where the model will be built, +# and an execution directory, where each forecast will +# take place. The short term archiver will use a third directory for +# storage of model output until it can be moved to long term storage (HPSS) +# +# -- Examine the whole script to identify things to change for your experiments. +# +# -- Provide the CESM initial file needed by your run. +# +# -- Run this script. It will modify the $CASEROOT/${case}.run script to +# insert a block to perform the DART operations needed to create synthetic obs. +# +# -- Edit the DART input.nml that appears in the $CASEROOT directory. +# +# -- Submit the job using ${CASEROOT}/${case}.submit +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime +# settings, it is safest to delete everything and start the run from scratch. +# For the brave, read +# +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html +# +# and you may be able to salvage something with +# ./cesm_setup -clean +# ./cesm_setup +# ./${case}.clean_build +# ./${case}.build +# +# ============================================================================== +# ==== Set case options +# ============================================================================== + +# the value of "case" will be used many ways; +# directory and file names, both locally and on HPSS, and +# script names; so consider its length and information content. +# BUG 1384 applies here, so ocean and atm/land must be at same resolution. +# 0.9x1.25_0.9x1.25 (f09_f09) +# 1 deg grid for atm/lnd, gx1v6 mask. Normally used for CAM with prescribed lnd, ocn and ice. + +setenv case cam_pmo +setenv compset F_AMIP_CAM5 +setenv resolution f09_f09 +setenv cesmtag cesm1_1_1 + +# ============================================================================== +# define machines and directories +# +# mach Computer name +# cesmroot Location of the cesm code base +# For cesm1_1_1 on yellowstone +# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# Preferably not a frequently scrubbed location. +# This script will delete any existing caseroot, so this script, +# and other useful things should be kept elsewhere. +# rundir (Future) Run-time directory; scrubbable, large amount of space needed. +# exeroot (Future) directory for executables - scrubbable, large amount of space needed. +# archdir (Future) Short-term archive directory +# until the long-term archiver moves it to permanent storage. +# dartroot Location of _your_ DART installation +# This is passed on to the CESM_DART_config script. +# ============================================================================== + +setenv mach yellowstone +setenv cesm_datadir /glade/p/cesm/cseg/inputdata + +setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag +setenv caseroot /glade/p/work/${USER}/cases/${case} +setenv exeroot /glade/scratch/${USER}/${case}/bld +setenv rundir /glade/scratch/${USER}/${case}/run +setenv archdir /glade/scratch/${USER}/archive/${case} +setenv dartroot /glade/u/home/${USER}/svn/DART/trunk + +# ============================================================================== +# configure settings +# The reference case has dates in it. +# For a 'hybrid' start, these may be unrelated to the refyear, refmon, refday. +# ============================================================================== + +setenv run_refcase cesm_hybrid +setenv refyear 2004 +setenv refmon 01 +setenv refday 10 +setenv run_reftod 00000 +setenv run_refdate $refyear-$refmon-$refday + +# SingleInstanceRefcase: the filenames are fundamentally different for +# a multi-instance CESM run or a single-instance CESM run. A correct setting +# of this variable makes staging the required files easier - that's all. +# 1 means 'true' ... the restart file has a single-instance-like name. +# 0 means 'false' .. the restart file has a multi-instance-like name. +# +# TRUTHinstance: specifies the specific instance you want to define as the TRUTH. +# If you have an initial ensemble size of 80, 1<= instance <= 80. +# If you only have one CLM state ... use 1. This value is also +# used to specify _which_ DATM streamfile to use to force the +# TRUTH run. +# +# CAM_stagedir: specifies the location of the CAM restart file you are defining as +# the truth. As this state evolves in time, it will be used as the +# input to the exact same forward observation operator code that +# will be used during a subsequent assimilation. + +setenv SingleInstanceRefcase 0 +setenv TRUTHinstance 23 +setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} + +# setenv stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} + +# ============================================================================== +# runtime settings +# +# sst_dataset Data ocean file and +# sst_grid supporting grid file. Must be consistent. +# +# resubmit How many job steps to run on continue runs (will be 0 initially) +# stop_option Units for determining the forecast length between assimilations +# stop_n Number of time units in the first forecast +# +# If the long-term archiver is off, you get a chance to examine the files before +# they get moved to long-term storage. You can always submit $case.l_archive +# whenever you want to free up space in the short-term archive directory. +# +# ============================================================================== + +setenv start_year 2004 +setenv start_month 01 +setenv start_day 10 +setenv start_tod 0 +setenv run_startdate ${start_year}-${start_month}-${start_day} +setenv year_start 1850 +setenv year_end 2012 + +# setenv sst_grid ${cesm_datadir}/ocn/docn7/domain.ocn.gx1v6.090206.nc +setenv sst_grid ${cesm_datadir}/share/domains/domain.ocn.fv0.9x1.25_gx1v6.130409.nc +setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc + +setenv short_term_archiver on +setenv long_term_archiver off +setenv resubmit 0 +setenv stop_option nhours +setenv stop_n 6 + +# ============================================================================== +# job settings +# +# queue can be changed during a series by changing the ${case}.run +# timewall can be changed during a series by changing the ${case}.run +# +# TJH: Advancing 1 instance for 6 hours with 60 pes (4 nodes) +# took less than 5 minutes on yellowstone. +# ============================================================================== + +setenv ACCOUNT P86850054 +setenv queue economy +setenv timewall 0:20 + +# ============================================================================== +# set these standard commands based on the machine you are running on. +# ============================================================================== + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + breaksw + default: + # NERSC "hopper", NWSC "yellowstone" + set MOVE = '/bin/mv -fv' + set COPY = '/bin/cp -fv --preserve=timestamps' + set LINK = '/bin/ln -fvs' + set REMOVE = '/bin/rm -fr' + + breaksw +endsw + +# ============================================================================== +# Make sure the CESM directories exist. +# VAR is the shell variable name, DIR is the value +# ============================================================================== + +foreach VAR ( cesmroot dartroot stagedir ) + set DIR = `eval echo \${$VAR}` + if ( ! -d $DIR ) then + echo "ERROR: directory '$DIR' not found" + echo " In the setup script check the setting of: $VAR " + exit -1 + endif +end + +# ============================================================================== +# Create the case - this creates the CASEROOT directory. +# +# For list of the pre-defined component sets: ./create_newcase -list +# To create a variant compset, see the CESM documentation and carefully +# incorporate any needed changes into this script. +# ============================================================================== + +# fatal idea to make caseroot the same dir as where this setup script is +# since the build process removes all files in the caseroot dir before +# populating it. try to prevent shooting yourself in the foot. + +if ( $caseroot == `dirname $0` ) then + echo "ERROR: the setup script should not be located in the caseroot" + echo "directory, because all files in the caseroot dir will be removed" + echo "before creating the new case. move the script to a safer place." + exit -1 +endif + +echo "removing old files from ${caseroot}" +echo "removing old files from ${exeroot}" +echo "removing old files from ${rundir}" +${REMOVE} ${caseroot} +${REMOVE} ${exeroot} +${REMOVE} ${rundir} + +${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ + -res ${resolution} -compset ${compset} + +if ( $status != 0 ) then + echo "ERROR: Case could not be created." + exit -1 +endif + +# preserve a copy of this script as it was run +set ThisFileName = $0:t +${COPY} $ThisFileName ${caseroot}/${ThisFileName}.original + +# ============================================================================== +# Record the DARTROOT directory and copy the DART setup script to CASEROOT. +# CESM_DART_config can be run at some later date if desired, but it presumes +# to be run from a CASEROOT directory. If CESM_DART_config does not exist locally, +# then it better exist in the expected part of the DARTROOT tree. +# ============================================================================== + +if ( ! -e CESM_DART_config ) then + ${COPY} ${dartroot}/models/cam/shell_scripts/CESM_DART_config . +endif + +if ( -e CESM_DART_config ) then + sed -e "s#BOGUS_DART_ROOT_STRING#$dartroot#" < CESM_DART_config >! temp.$$ + ${MOVE} temp.$$ ${caseroot}/CESM_DART_config + chmod 755 ${caseroot}/CESM_DART_config +else + echo "ERROR: the script to configure for data assimilation is not available." + echo " CESM_DART_config MUST be present locally or in" + echo " ${dartroot}/models/CESM/shell_scripts/" + exit -2 +endif + +# ============================================================================== +# Configure the case. +# ============================================================================== + +cd ${caseroot} + +source ./Tools/ccsm_getenv || exit -2 + +# MAX_TASKS_PER_NODE comes from $case/Tools/mkbatch.$machine +@ ptile = $MAX_TASKS_PER_NODE / 2 +@ nthreads = 1 + +# Save a copy for debug purposes +foreach FILE ( *xml ) + if ( ! -e ${FILE}.original ) then + ${COPY} $FILE ${FILE}.original + endif +end + + # This is only for the purpose of debugging the code. + @ atm_tasks = $ptile * 4 + @ lnd_tasks = $ptile * 4 + @ ice_tasks = $ptile * 4 + @ ocn_tasks = $ptile + @ cpl_tasks = $ptile + @ glc_tasks = $ptile + @ rof_tasks = $ptile + +# echo "task partitioning ... perhaps ... atm // ocn // lnd+ice+glc+rof" +# presently, all components run 'serially' - one after another. +echo "" +echo "ATM gets $atm_tasks" +echo "LND gets $lnd_tasks" +echo "ICE gets $ice_tasks" +echo "OCN gets $ocn_tasks" +echo "CPL gets $cpl_tasks" +echo "GLC gets $glc_tasks" +echo "ROF gets $rof_tasks" +echo "" + +./xmlchange NTHRDS_ATM=$nthreads,NTASKS_ATM=$atm_tasks,NINST_ATM=1 +./xmlchange NTHRDS_LND=$nthreads,NTASKS_LND=$lnd_tasks,NINST_LND=1 +./xmlchange NTHRDS_ICE=$nthreads,NTASKS_ICE=$ice_tasks,NINST_ICE=1 +./xmlchange NTHRDS_OCN=$nthreads,NTASKS_OCN=$ocn_tasks,NINST_OCN=1 +./xmlchange NTHRDS_CPL=$nthreads,NTASKS_CPL=$cpl_tasks +./xmlchange NTHRDS_GLC=$nthreads,NTASKS_GLC=$glc_tasks,NINST_GLC=1 +./xmlchange NTHRDS_ROF=$nthreads,NTASKS_ROF=$rof_tasks,NINST_ROF=1 +./xmlchange ROOTPE_ATM=0 +./xmlchange ROOTPE_LND=0 +./xmlchange ROOTPE_ICE=0 +./xmlchange ROOTPE_OCN=0 +./xmlchange ROOTPE_CPL=0 +./xmlchange ROOTPE_GLC=0 +./xmlchange ROOTPE_ROF=0 + +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/c1158.html#run_start_stop +# "A hybrid run indicates that CESM is initialized more like a startup, but uses +# initialization datasets from a previous case. This is somewhat analogous to a +# branch run with relaxed restart constraints. A hybrid run allows users to bring +# together combinations of initial/restart files from a previous case (specified +# by $RUN_REFCASE) at a given model output date (specified by $RUN_REFDATE). +# Unlike a branch run, the starting date of a hybrid run (specified by $RUN_STARTDATE) +# can be modified relative to the reference case. In a hybrid run, the model does not +# continue in a bit-for-bit fashion with respect to the reference case. The resulting +# climate, however, should be continuous provided that no model source code or +# namelists are changed in the hybrid run. In a hybrid initialization, the ocean +# model does not start until the second ocean coupling (normally the second day), +# and the coupler does a "cold start" without a restart file." + +# TJH: +# DART's CAM implementation causes a bit more complexity. DART only uses CAM _initial_ +# files, not RESTART files, so there are sourcemods to force a hybrid start for CAM to +# read initial files - even when CONTINUE_RUN = TRUE. + +./xmlchange RUN_TYPE=hybrid +./xmlchange RUN_STARTDATE=$run_startdate +./xmlchange START_TOD=$start_tod +./xmlchange RUN_REFCASE=$run_refcase +./xmlchange RUN_REFDATE=$run_refdate +./xmlchange RUN_REFTOD=$run_reftod +./xmlchange BRNCH_RETAIN_CASENAME=FALSE +./xmlchange GET_REFCASE=FALSE +./xmlchange EXEROOT=${exeroot} + +./xmlchange SSTICE_DATA_FILENAME=$sst_dataset +./xmlchange SSTICE_GRID_FILENAME=$sst_grid +./xmlchange SSTICE_YEAR_ALIGN=$year_start +./xmlchange SSTICE_YEAR_START=$year_start +./xmlchange SSTICE_YEAR_END=$year_end + +# Do not change the CALENDAR or the CONTINUE_RUN + +./xmlchange CALENDAR=GREGORIAN + +./xmlchange STOP_OPTION=$stop_option +./xmlchange STOP_N=$stop_n +./xmlchange CONTINUE_RUN=FALSE +./xmlchange RESUBMIT=$resubmit + +./xmlchange PIO_TYPENAME=pnetcdf + +# COUPLING discussion. F compsets are 'tight' coupling. +# Only change the ATM_NCPL ... everything is based on this one value, +# including CAM physics and dynamics timesteps. +# Default values for coupling are preserved in env_run.xml.original + +./xmlchange NCPL_BASE_PERIOD=day +./xmlchange ATM_NCPL=48 + +# TJH FIXME ... any reason to turn ON the biogeochemistry in CLM when atmos only? +# There is no compset that has everything we want, +# so we are also turning on the CLM biogeochemistry. +#./xmlchange CLM_CONFIG_OPTS='-bgc cn' + +if ($short_term_archiver == 'off') then + ./xmlchange DOUT_S=FALSE +else + ./xmlchange DOUT_S=TRUE + ./xmlchange DOUT_S_ROOT=${archdir} + ./xmlchange DOUT_S_SAVE_INT_REST_FILES=FALSE +endif +if ($long_term_archiver == 'off') then + ./xmlchange DOUT_L_MS=FALSE +else + ./xmlchange DOUT_L_MS=TRUE + ./xmlchange DOUT_L_MSROOT="csm/${case}" + ./xmlchange DOUT_L_HTAR=FALSE +endif + +# The river transport model ON is useful only when using an active ocean or +# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. +# If you turn it ON, you will have to stage initial files etc. + +./xmlchange ROF_GRID='null' +./xmlchange RTM_MODE='NULL' + +# level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) + +./xmlchange DEBUG=FALSE +./xmlchange INFO_DBUG=0 + +# ============================================================================== +# Set up the case. +# This creates the EXEROOT and RUNDIR directories. +# ============================================================================== + +./cesm_setup + +if ( $status != 0 ) then + echo "ERROR: Case could not be set up." + exit -2 +endif + +# ============================================================================== +# Edit the run script to reflect queue and wallclock +# ============================================================================== + +echo '' +echo 'Updating the run script to set wallclock and queue.' +echo '' + +if ( ! -e ${case}.run.original ) then + ${COPY} ${case}.run ${case}.run.original +endif + +source Tools/ccsm_getenv +set BATCH = `echo $BATCHSUBMIT | sed 's/ .*$//'` +switch ( $BATCH ) + case bsub*: + # NCAR "bluefire", "yellowstone" + set TIMEWALL=`grep BSUB ${case}.run | grep -e '-W' ` + set QUEUE=`grep BSUB ${case}.run | grep -e '-q' ` + sed -e "s/$TIMEWALL[3]/$timewall/" \ + -e "s/ptile=[0-9][0-9]*/ptile=$ptile/" \ + -e "s/$QUEUE[3]/$queue/" < ${case}.run >! temp.$$ + ${MOVE} temp.$$ ${case}.run + chmod 755 ${case}.run + breaksw + + default: + + breaksw +endsw + +# This is the part that modifies the run script to allow CESM to advance +# correctly given the modifications necessary for CAM. It also copies several +# required DART files to the caseroot directory. + +./CESM_DART_config + +# ============================================================================== +# Update source files. +# Ideally, using DART would not require any modifications to the model source. +# Until then, this script accesses sourcemods from a hardwired location. +# If you have additional sourcemods, they will need to be merged into any DART +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. +# ============================================================================== + +if ( -d ~/${cesmtag}/SourceMods ) then + ${COPY} -r ~/${cesmtag}/SourceMods/* ${caseroot}/SourceMods/ +else + echo "ERROR - No SourceMods for this case." + echo "ERROR - No SourceMods for this case." + echo "DART requires modifications to several src files." + echo "These files can be downloaded from:" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "untar these into your HOME directory - they will create a" + echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." + exit -4 +endif + +# The CESM multi-instance capability is relatively new and still has a few +# implementation bugs. These are known problems and will be fixed soon. +# this should be removed when the files are fixed: + +echo "REPLACING BROKEN CESM FILES HERE - SHOULD BE REMOVED WHEN FIXED" +echo caseroot is ${caseroot} +if ( -d ~/${cesmtag} ) then + + # preserve the original version of the files + if ( ! -e ${caseroot}/Buildconf/clm.buildnml.csh.original ) then + ${MOVE} ${caseroot}/Buildconf/clm.buildnml.csh \ + ${caseroot}/Buildconf/clm.buildnml.csh.original + endif + if ( ! -e ${caseroot}/preview_namelists.original ) then + ${MOVE} ${caseroot}/preview_namelists \ + ${caseroot}/preview_namelists.original + endif + + # patch/replace the broken files + ${COPY} ~/${cesmtag}/clm.buildnml.csh ${caseroot}/Buildconf/. + ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. + +endif + +# ============================================================================== +# Modify namelist templates for each instance. This is a bit of a nuisance in +# that we are pulling in restart and initial files from 'all over the place' +# and each model component has a different strategy. +# +# In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): +# +# CAM has been forced to read initial files - specified by namelist var:ncdata +# CICE reads from namelist variable 'ice_ic' +# CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 +# it does not use the instance string. There is a patch for clm.buildnml.csh +# +# All of these must later on be staged with these same filenames. +# OR - all these namelists can be changed to match whatever has been staged. +# MAKE SURE THE STAGING SECTION OF THIS SCRIPT MATCHES THESE VALUES. +# ============================================================================== + +@ inst = 1 +while ($inst <= 1) + + # following the CESM strategy for 'inst_string' + set inst_string = '' + + # =========================================================================== + set fname = "user_nl_cam${inst_string}" + # =========================================================================== + # For a HOP TEST ... empty_htapes = .false. + # For a HOP TEST ... use a default fincl1 + # inithist == 'ENDOFRUN' ensures that CAM writes an initial file every time it stops. + # fincl1,nhtfrq,mfilt all control the history file containing a REQUIRED PHIS field. + + echo " inithist = 'ENDOFRUN'" >> ${fname} + echo " ncdata = 'cam_initial${inst_string}.nc'" >> ${fname} + echo " empty_htapes = .true. " >> ${fname} + echo " fincl1 = 'PHIS:I' " >> ${fname} + echo " nhtfrq = -$stop_n " >> ${fname} + echo " mfilt = 1 " >> ${fname} + + # A lot of the files specified here are because the 'default' files only + # contain data through 2005 and we are interested in timeframes after that. + + set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" + + # CAM5 does prognostic aerosols by default. If you want to prescribe them, + # use the following variables with your own settings ... + # Filenames with 'clim' in their names should specify aerodep_flx_type = 'CYCLICAL'. + # Other values are 'CYCLICAL', 'SERIAL', 'INTERP_MISSING_MONTHS', or 'FIXED'. + + echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} + echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} + echo "aerodep_flx_type = 'CYCLICAL' " >> ${fname} + # Set if aerodep_flx_type = 'CYCLICAL' + echo "aerodep_flx_cycle_yr = 2000 " >> ${fname} + + # =========================================================================== + set fname = "user_nl_clm${inst_string}" + # =========================================================================== + + # Customize the land namelists + # The filename is built using the REFCASE/REFDATE/REFTOD information. + # Making a compact .h0. file is a good idea, since the clm restart files + # do not have all the metadata required to reconstruct a gridded field. + # 'TSA' is 2m surface air temperature. + # TJH ... should make monthly average file or something ... + # 1.3 MB is 100x bigger than CAM history file. See page 65 of + # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf + # + # Every 6 hours + # echo "hist_mfilt = 1" >> $fname + # echo "hist_nhtfrq = -$stop_n" >> $fname + # Every month + # echo "hist_mfilt = 1" >> $fname + # echo "hist_nhtfrq = 0" >> $fname + +# KDR; Was the ntapes bug fixed? +# That's where the CLM restart file always has dimension ntapes, +# but it's set = 0 for hist_empty_htapes = .false., +# which prevents the next forecast from starting. +# Kluzek fixed it in some recent (<9/2013) version, but not 1_1_1. +# There's a supposed fix in ~/cesm1_1_1/SourceMods/src.clm/HistFileMod.F90, +# but it only checks whether ntapes = ntapes_on_file, and is_restart() returning 'true'. +# It doesn't check whether the size is 0. +# That solution can't work in hybrid mode. See ~/Homme/notes:9/17/2013 +# KDR + + echo "hist_empty_htapes = .true." >> $fname + echo "hist_fincl1 = 'TSA'" >> $fname + echo "hist_nhtfrq = -$stop_n" >> $fname + echo "hist_mfilt = 1" >> $fname + echo "hist_avgflag_pertape = 'A'" >> $fname + + # =========================================================================== + set fname = "user_nl_cice${inst_string}" + # =========================================================================== + # CICE Namelists + + echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname + + @ inst ++ +end + +./preview_namelists + +# ============================================================================== +# Stage the restarts now that the run directory exists +# THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. +# ============================================================================== + +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. + +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' +echo '' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' +echo '' + +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} + +set restart_time = $RUN_REFDATE-$RUN_REFTOD + +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- + +echo "continue_run is \$CONTINUE_RUN" + +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + + # the restarts should be in the short term archive directory + # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + + @ inst=1 + while (\$inst <= 1) + + set inst_string = '' + + echo "${case}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${case}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} + echo "${case}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} + echo "${case}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + + + @ inst ++ + end + + endif + + # Now that we're guaranteed to have the cam initial file, link to our hardwired name. + @ inst=1 + while (\$inst <= 1) + set inst_string = '' + ${LINK} ${case}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc + @ inst ++ + end + + # relink the pointer file for the data ocean + echo "${case}.docn.r.\${restart_time}.nc" >! rpointer.ocn + echo "" >> rpointer.ocn + echo "${case}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn + echo "" >> rpointer.ocn + + exit 0 +endif + +#---------------------------------------------------------------------- +# This block prepares the initial run directory for a startup run. +# The CLM, CICE, ROF namelists specify a relative path for the initial file. +# The CAM namelist specifies a static name. +# POP uses pointer files. +# +# The reference case may more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - in a hybrid case - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +set filedir = ${stagedir} + +@ inst=1 +while (\$inst <= 1) + + set inst_string = '' + set true_string = \`printf _%04d $TRUTHinstance\` + + echo '' + echo "Staging restart for instance \$inst" + + ${LINK} \${filedir}/${run_refcase}.clm2\${true_string}.r.${run_refdate}-${run_reftod}.nc \ + ${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc + + ${LINK} \${filedir}/${run_refcase}.cice\${true_string}.r.${run_refdate}-${run_reftod}.nc \ + ${run_refcase}.cice\${inst_string}.r.${run_refdate}-${run_reftod}.nc + + ${LINK} \${filedir}/${run_refcase}.cam\${true_string}.i.${run_refdate}-${run_reftod}.nc \ + cam_initial\${inst_string}.nc + + @ inst ++ +end + +exit 0 + +EndOfText +chmod 0755 stage_cesm_files + +./stage_cesm_files + +# ============================================================================== +# build +# ============================================================================== + +echo '' +echo 'Building the case' +echo '' + +./${case}.build + +if ( $status != 0 ) then + echo "ERROR: Case could not be built." + exit -5 +endif + +# ============================================================================== +# What to do next +# ============================================================================== + +echo "" +echo "Time to check the case." +echo "" +echo "1) cd ${rundir}" +echo " and check the compatibility between the namelists/pointer" +echo " files and the files that were staged." +echo "" +echo "2) cd ${caseroot}" +echo " (on yellowstone) If the ${case}.run script still contains:" +echo ' #BSUB -R "select[scratch_ok > 0]"' +echo " around line 9, delete it." +echo "" +echo "3) If you want to simply advance CESM without using DART, make sure the" +echo " ${case}.run script references the 'no_assimilate.csh' script." +echo " If you want to assimilate, reference the 'assimilate.csh' script." +echo " The default action is to assimilate." +echo "" +echo "4) Verify the contents of env_run.xml and submit the CESM job:" +echo " ./${case}.submit" +echo "" +echo "5) After the job has run, check to make sure it worked and that" +echo " a: the right restart files exist in the run directory," +echo " b: (if you're running DART) the archive dart/hist directory has the DART output," +echo " c: everything is working correctly ..." +echo "" +echo "6) To extend the run in $stop_n '"$stop_option"' steps," +echo " change the env_run.xml variables:" +echo "" +echo " ./xmlchange CONTINUE_RUN=TRUE" +echo " ./xmlchange RESUBMIT=" +echo " ./xmlchange STOP_N=$stop_n" +echo "" +echo " and" +echo " ./${case}.submit" +echo "" + +exit 0 + +# +# $URL: https://subversion.ucar.edu/DAReS/DART/trunk/models/CESM/shell_scripts/CESM1_1_1_hybrid $ +# $Revision: 6396 $ +# $Date: 2013-08-12 11:51:26 -0600 (Mon, 12 Aug 2013) $ + Property changes on: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo ___________________________________________________________________ Added: svn:executable + * Added: DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh (rev 0) +++ DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh 2013-10-22 23:25:29 UTC (rev 6538) @@ -0,0 +1,221 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# run. + +echo "`date` -- BEGIN GENERATE CAM TRUE STATE" + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + set BASEOBSDIR = /glade/proj3/image/Observations/ACARS + breaksw + + case ys*: + # NCAR "yellowstone" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /glade/p/image/Observations/ACARS + breaksw + + default: + # NERSC "hopper" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS + breaksw +endsw + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name +# of the form "./${CASE}.cam.i.2000-01-06-00000.nc" +# +# Piping stuff through 'bc' strips off any preceeding zeros. +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.atm` +set FILE = $FILE:r +set ATM_DATE_EXT = `echo $FILE:e` +set ATM_DATE = `echo $FILE:e | sed -e "s#-# #g"` +set ATM_YEAR = `echo $ATM_DATE[1] | bc` +set ATM_MONTH = `echo $ATM_DATE[2] | bc` +set ATM_DAY = `echo $ATM_DATE[3] | bc` +set ATM_SECONDS = `echo $ATM_DATE[4] | bc` +set ATM_HOUR = `echo $ATM_DATE[4] / 3600 | bc` + +echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_SECONDS (seconds)" +echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_HOUR (hours)" + +#------------------------------------------------------------------------- +# Determine if current time is a perfect model time. +# If not, return before doing anything. +#------------------------------------------------------------------------- + +## TIMECHECK: +if ( $ATM_HOUR == 0 || $ATM_HOUR == 6 || $ATM_HOUR == 12 || $ATM_HOUR == 18) then + echo "Hour is $ATM_HOUR so we are generating perfect obs for the atmosphere" +else + echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the atmosphere" + echo "`date` -- END GENERATE CAM TRUE STATE" + exit 0 +endif + +#------------------------------------------------------------------------- +# Create temporary working directory for the perfect model and go there +#------------------------------------------------------------------------- + +set temp_dir = pmo_cam +echo "temp_dir is $temp_dir" + +if ( -d $temp_dir ) then + ${REMOVE} $temp_dir/* +else + mkdir -p $temp_dir +endif +cd $temp_dir + +#----------------------------------------------------------------------------- +# Get observation sequence file ... or die right away. +# The observation file names have a time that matches the stopping time of CAM. +#----------------------------------------------------------------------------- + +set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` +set OBSFNAME = `printf obs_seq%04d%02d%02d%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY} ${ATM_HOUR}` +set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}_6H/${OBSFNAME} + +if ( -e ${OBS_FILE} ) then + ${LINK} ${OBS_FILE} obs_seq.in +else + echo "ERROR ... no observation file $OBS_FILE" + echo "ERROR ... no observation file $OBS_FILE" + exit -1 +endif + +#========================================================================= +# Block 1: Populate a run-time directory with the input needed to run DART, +# and Block 2: Convert 1 CAM restart file to DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from pointer file ../rpointer.atm +# +# REQUIRED DART namelist settings: +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &cam_to_dart_nml: cam_to_dart_output_file = 'dart_ics' +#========================================================================= + +echo "`date` -- BEGIN CAM-TO-DART" + + if ( ! -e ${CASEROOT}/input.nml ) then + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + exit -2 + endif + + # Turns out the .h0. files are timestamped with the START of the + # run, which is *not* ATM_DATE_EXT ... I just link to a whatever + # is convenient (since the info is static). + # make sure there are no old output logs hanging around + $REMOVE output.cam_to_dart + + set ATM_INITIAL_FILENAME = "../${CASE}.cam.i.${ATM_DATE_EXT}.nc" + set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` + set DART_IC_FILENAME = perfect_ics + @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Tue Oct 22 17:28:49 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 22 Oct 2013 17:28:49 -0600 Subject: [Dart-dev] [6539] DART/trunk/models/cam/shell_scripts: Renaming the script to be consistent across CESM components. Message-ID: Revision: 6539 Author: thoar Date: 2013-10-22 17:28:49 -0600 (Tue, 22 Oct 2013) Log Message: ----------- Renaming the script to be consistent across CESM components. Added Paths: ----------- DART/trunk/models/cam/shell_scripts/perfect_model.csh Removed Paths: ------------- DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh -------------- next part -------------- Copied: DART/trunk/models/cam/shell_scripts/perfect_model.csh (from rev 6538, DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh) =================================================================== --- DART/trunk/models/cam/shell_scripts/perfect_model.csh (rev 0) +++ DART/trunk/models/cam/shell_scripts/perfect_model.csh 2013-10-22 23:28:49 UTC (rev 6539) @@ -0,0 +1,221 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# This block is an attempt to localize all the machine-specific +# changes to this script such that the same script can be used +# on multiple platforms. This will help us maintain the script. +# Search below for TIMECHECK to see what times this script will +# run. + +echo "`date` -- BEGIN GENERATE CAM TRUE STATE" + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + set BASEOBSDIR = /glade/proj3/image/Observations/ACARS + breaksw + + case ys*: + # NCAR "yellowstone" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /glade/p/image/Observations/ACARS + breaksw + + default: + # NERSC "hopper" + set MOVE = 'mv -fv' + set COPY = 'cp -fv --preserve=timestamps' + set LINK = 'ln -fvs' + set REMOVE = 'rm -fr' + + set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS + breaksw +endsw + +#------------------------------------------------------------------------- +# Determine time of model state ... from file name +# of the form "./${CASE}.cam.i.2000-01-06-00000.nc" +# +# Piping stuff through 'bc' strips off any preceeding zeros. +#------------------------------------------------------------------------- + +set FILE = `head -n 1 rpointer.atm` +set FILE = $FILE:r +set ATM_DATE_EXT = `echo $FILE:e` +set ATM_DATE = `echo $FILE:e | sed -e "s#-# #g"` +set ATM_YEAR = `echo $ATM_DATE[1] | bc` +set ATM_MONTH = `echo $ATM_DATE[2] | bc` +set ATM_DAY = `echo $ATM_DATE[3] | bc` +set ATM_SECONDS = `echo $ATM_DATE[4] | bc` +set ATM_HOUR = `echo $ATM_DATE[4] / 3600 | bc` + +echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_SECONDS (seconds)" +echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_HOUR (hours)" + +#------------------------------------------------------------------------- +# Determine if current time is a perfect model time. +# If not, return before doing anything. +#------------------------------------------------------------------------- + +## TIMECHECK: +if ( $ATM_HOUR == 0 || $ATM_HOUR == 6 || $ATM_HOUR == 12 || $ATM_HOUR == 18) then + echo "Hour is $ATM_HOUR so we are generating perfect obs for the atmosphere" +else + echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the atmosphere" + echo "`date` -- END GENERATE CAM TRUE STATE" + exit 0 +endif + +#------------------------------------------------------------------------- +# Create temporary working directory for the perfect model and go there +#------------------------------------------------------------------------- + +set temp_dir = pmo_cam +echo "temp_dir is $temp_dir" + +if ( -d $temp_dir ) then + ${REMOVE} $temp_dir/* +else + mkdir -p $temp_dir +endif +cd $temp_dir + +#----------------------------------------------------------------------------- +# Get observation sequence file ... or die right away. +# The observation file names have a time that matches the stopping time of CAM. +#----------------------------------------------------------------------------- + +set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` +set OBSFNAME = `printf obs_seq%04d%02d%02d%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY} ${ATM_HOUR}` +set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}_6H/${OBSFNAME} + +if ( -e ${OBS_FILE} ) then + ${LINK} ${OBS_FILE} obs_seq.in +else + echo "ERROR ... no observation file $OBS_FILE" + echo "ERROR ... no observation file $OBS_FILE" + exit -1 +endif + +#========================================================================= +# Block 1: Populate a run-time directory with the input needed to run DART, +# and Block 2: Convert 1 CAM restart file to DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from pointer file ../rpointer.atm +# +# REQUIRED DART namelist settings: +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &cam_to_dart_nml: cam_to_dart_output_file = 'dart_ics' +#========================================================================= + +echo "`date` -- BEGIN CAM-TO-DART" + + if ( ! -e ${CASEROOT}/input.nml ) then + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + exit -2 + endif + + # Turns out the .h0. files are timestamped with the START of the + # run, which is *not* ATM_DATE_EXT ... I just link to a whatever + # is convenient (since the info is static). + # make sure there are no old output logs hanging around + $REMOVE output.cam_to_dart + + set ATM_INITIAL_FILENAME = "../${CASE}.cam.i.${ATM_DATE_EXT}.nc" + set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` + set DART_IC_FILENAME = perfect_ics + + sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/input.nml >! input.nml + + ${LINK} $ATM_INITIAL_FILENAME caminput.nc + ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc + + ${EXEROOT}/cam_to_dart >! output.cam_to_dart + + if ($status != 0) then + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + exit -6 + endif + +echo "`date` -- END CAM-TO-DART" + +#========================================================================= +# Block 3: Run perfect_model_obs and harvest the synthetic observations +# and diagnostic files. +# +# DART namelist settings required: +# &perfect_model_obs_nml: async = 0, +# &perfect_model_obs_nml: adv_ens_command = "no_advance_script", +# &perfect_model_obs_nml: output_restart = .false., +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &perfect_model_obs_nml: restart_out_file_name = 'not_created' +# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' +# &perfect_model_obs_nml: init_time_days = -1, +# &perfect_model_obs_nml: init_time_seconds = -1, +# &perfect_model_obs_nml: first_obs_days = -1, +# &perfect_model_obs_nml: first_obs_seconds = -1, +# &perfect_model_obs_nml: last_obs_days = -1, +# &perfect_model_obs_nml: last_obs_seconds = -1, +# +#========================================================================= + +echo "`date` -- BEGIN CAM PERFECT_MODEL_OBS" +${EXEROOT}/perfect_model_obs || exit -7 +echo "`date` -- END CAM PERFECT_MODEL_OBS" + +${MOVE} True_State.nc ../cam_True_State.${ATM_DATE_EXT}.nc +${MOVE} obs_seq.perfect ../cam_obs_seq.${ATM_DATE_EXT}.perfect +${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out + +#========================================================================= +# Block 4: Update the cam restart file +#========================================================================= + +# not needed ... perfect_model_obs does not update the model state. + +#========================================================================= +# Block 5: Link the next cam file to the static name needed here. +#========================================================================= + +cd ${RUNDIR} + + set ATM_INITIAL_FILENAME = ${CASE}.cam.i.${ATM_DATE_EXT}.nc + + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial.nc || exit -9 + + +#------------------------------------------------------------------------- +# Cleanup +#------------------------------------------------------------------------- + +echo "`date` -- END GENERATE CAM TRUE STATE" + +exit 0 + +# +# $URL$ +# $Revision$ +# $Date$ + Deleted: DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh 2013-10-22 23:25:29 UTC (rev 6538) +++ DART/trunk/models/cam/shell_scripts/perfect_model_obs.csh 2013-10-22 23:28:49 UTC (rev 6539) @@ -1,221 +0,0 @@ -#!/bin/csh -# -# DART software - Copyright 2004 - 2013 UCAR. This open source software is -# provided by UCAR, "as is", without charge, subject to all terms of use at -# http://www.image.ucar.edu/DAReS/DART/DART_download -# -# DART $Id$ - -# This block is an attempt to localize all the machine-specific -# changes to this script such that the same script can be used -# on multiple platforms. This will help us maintain the script. -# Search below for TIMECHECK to see what times this script will -# run. - -echo "`date` -- BEGIN GENERATE CAM TRUE STATE" - -set nonomatch # suppress "rm" warnings if wildcard does not match anything - -# The FORCE options are not optional. -# The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following -switch ("`hostname`") - case be*: - # NCAR "bluefire" - set MOVE = '/usr/local/bin/mv -fv' - set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' - set LINK = '/usr/local/bin/ln -fvs' - set REMOVE = '/usr/local/bin/rm -fr' - - set BASEOBSDIR = /glade/proj3/image/Observations/ACARS - breaksw - - case ys*: - # NCAR "yellowstone" - set MOVE = 'mv -fv' - set COPY = 'cp -fv --preserve=timestamps' - set LINK = 'ln -fvs' - set REMOVE = 'rm -fr' - - set BASEOBSDIR = /glade/p/image/Observations/ACARS - breaksw - - default: - # NERSC "hopper" - set MOVE = 'mv -fv' - set COPY = 'cp -fv --preserve=timestamps' - set LINK = 'ln -fvs' - set REMOVE = 'rm -fr' - - set BASEOBSDIR = /scratch/scratchdirs/nscollin/ACARS - breaksw -endsw - -#------------------------------------------------------------------------- -# Determine time of model state ... from file name -# of the form "./${CASE}.cam.i.2000-01-06-00000.nc" -# -# Piping stuff through 'bc' strips off any preceeding zeros. -#------------------------------------------------------------------------- - -set FILE = `head -n 1 rpointer.atm` -set FILE = $FILE:r -set ATM_DATE_EXT = `echo $FILE:e` -set ATM_DATE = `echo $FILE:e | sed -e "s#-# #g"` -set ATM_YEAR = `echo $ATM_DATE[1] | bc` -set ATM_MONTH = `echo $ATM_DATE[2] | bc` -set ATM_DAY = `echo $ATM_DATE[3] | bc` -set ATM_SECONDS = `echo $ATM_DATE[4] | bc` -set ATM_HOUR = `echo $ATM_DATE[4] / 3600 | bc` - -echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_SECONDS (seconds)" -echo "valid time of model is $ATM_YEAR $ATM_MONTH $ATM_DAY $ATM_HOUR (hours)" - -#------------------------------------------------------------------------- -# Determine if current time is a perfect model time. -# If not, return before doing anything. -#------------------------------------------------------------------------- - -## TIMECHECK: -if ( $ATM_HOUR == 0 || $ATM_HOUR == 6 || $ATM_HOUR == 12 || $ATM_HOUR == 18) then - echo "Hour is $ATM_HOUR so we are generating perfect obs for the atmosphere" -else - echo "Hour is not 0,6,12 or 18Z so we are skipping generating perfect obs for the atmosphere" - echo "`date` -- END GENERATE CAM TRUE STATE" - exit 0 -endif - -#------------------------------------------------------------------------- -# Create temporary working directory for the perfect model and go there -#------------------------------------------------------------------------- - -set temp_dir = pmo_cam -echo "temp_dir is $temp_dir" - -if ( -d $temp_dir ) then - ${REMOVE} $temp_dir/* -else - mkdir -p $temp_dir -endif -cd $temp_dir - -#----------------------------------------------------------------------------- -# Get observation sequence file ... or die right away. -# The observation file names have a time that matches the stopping time of CAM. -#----------------------------------------------------------------------------- - -set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` -set OBSFNAME = `printf obs_seq%04d%02d%02d%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY} ${ATM_HOUR}` -set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}_6H/${OBSFNAME} - -if ( -e ${OBS_FILE} ) then - ${LINK} ${OBS_FILE} obs_seq.in -else - echo "ERROR ... no observation file $OBS_FILE" - echo "ERROR ... no observation file $OBS_FILE" - exit -1 -endif - -#========================================================================= -# Block 1: Populate a run-time directory with the input needed to run DART, -# and Block 2: Convert 1 CAM restart file to DART initial conditions file. -# At the end of the block, we have DART initial condition file perfect_ics -# that came from pointer file ../rpointer.atm -# -# REQUIRED DART namelist settings: -# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &cam_to_dart_nml: cam_to_dart_output_file = 'dart_ics' -#========================================================================= - -echo "`date` -- BEGIN CAM-TO-DART" - - if ( ! -e ${CASEROOT}/input.nml ) then - echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" - echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" - exit -2 - endif - - # Turns out the .h0. files are timestamped with the START of the - # run, which is *not* ATM_DATE_EXT ... I just link to a whatever - # is convenient (since the info is static). - # make sure there are no old output logs hanging around - $REMOVE output.cam_to_dart - - set ATM_INITIAL_FILENAME = "../${CASE}.cam.i.${ATM_DATE_EXT}.nc" - set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` - set DART_IC_FILENAME = perfect_ics - - sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/input.nml >! input.nml - - ${LINK} $ATM_INITIAL_FILENAME caminput.nc - ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc - - ${EXEROOT}/cam_to_dart >! output.cam_to_dart - - if ($status != 0) then - echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" - echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" - exit -6 - endif - -echo "`date` -- END CAM-TO-DART" - -#========================================================================= -# Block 3: Run perfect_model_obs and harvest the synthetic observations -# and diagnostic files. -# -# DART namelist settings required: -# &perfect_model_obs_nml: async = 0, -# &perfect_model_obs_nml: adv_ens_command = "no_advance_script", -# &perfect_model_obs_nml: output_restart = .false., -# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &perfect_model_obs_nml: restart_out_file_name = 'not_created' -# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' -# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' -# &perfect_model_obs_nml: init_time_days = -1, -# &perfect_model_obs_nml: init_time_seconds = -1, -# &perfect_model_obs_nml: first_obs_days = -1, -# &perfect_model_obs_nml: first_obs_seconds = -1, -# &perfect_model_obs_nml: last_obs_days = -1, -# &perfect_model_obs_nml: last_obs_seconds = -1, -# -#========================================================================= - -echo "`date` -- BEGIN CAM PERFECT_MODEL_OBS" -${EXEROOT}/perfect_model_obs || exit -7 -echo "`date` -- END CAM PERFECT_MODEL_OBS" - -${MOVE} True_State.nc ../cam_True_State.${ATM_DATE_EXT}.nc -${MOVE} obs_seq.perfect ../cam_obs_seq.${ATM_DATE_EXT}.perfect -${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out - -#========================================================================= -# Block 4: Update the cam restart file -#========================================================================= - -# not needed ... perfect_model_obs does not update the model state. - -#========================================================================= -# Block 5: Link the next cam file to the static name needed here. -#========================================================================= - -cd ${RUNDIR} - - set ATM_INITIAL_FILENAME = ${CASE}.cam.i.${ATM_DATE_EXT}.nc - - ${LINK} ${ATM_INITIAL_FILENAME} cam_initial.nc || exit -9 - - -#------------------------------------------------------------------------- -# Cleanup -#------------------------------------------------------------------------- - -echo "`date` -- END GENERATE CAM TRUE STATE" - -exit 0 - -# -# $URL$ -# $Revision$ -# $Date$ - From nancy at ucar.edu Wed Oct 23 11:24:29 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 23 Oct 2013 11:24:29 -0600 Subject: [Dart-dev] [6540] DART/trunk/models/cam/shell_scripts: Runs a single-instance configuration of CESM by default and Message-ID: Revision: 6540 Author: thoar Date: 2013-10-23 11:24:29 -0600 (Wed, 23 Oct 2013) Log Message: ----------- Runs a single-instance configuration of CESM by default and can be extended to run a perfect model experiment. The short-term archiver now preserves obs_seq.perfect files. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo DART/trunk/models/cam/shell_scripts/CESM_DART_config DART/trunk/models/cam/shell_scripts/no_assimilate.csh DART/trunk/models/cam/shell_scripts/st_archive.sh -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-22 23:28:49 UTC (rev 6539) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-23 17:24:29 UTC (rev 6540) @@ -59,12 +59,12 @@ # # -- Provide the CESM initial file needed by your run. # -# -- Run this script. It will modify the $CASEROOT/${case}.run script to +# -- Run this script. It will modify the $CASEROOT/${CASE}.run script to # insert a block to perform the DART operations needed to create synthetic obs. # # -- Edit the DART input.nml that appears in the $CASEROOT directory. # -# -- Submit the job using ${CASEROOT}/${case}.submit +# -- Submit the job using ${CASEROOT}/${CASE}.submit # # --------------------- # Important features @@ -117,8 +117,8 @@ # ============================================================================== setenv mach yellowstone -setenv cesm_datadir /glade/p/cesm/cseg/inputdata +setenv cesm_datadir /glade/p/cesm/cseg/inputdata setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag setenv caseroot /glade/p/work/${USER}/cases/${case} setenv exeroot /glade/scratch/${USER}/${case}/bld @@ -158,9 +158,10 @@ setenv SingleInstanceRefcase 0 setenv TRUTHinstance 23 -setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} +setenv CAM_stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} -# setenv stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} +# alternative reference case for different times may be available here: +# setenv CAM_stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} # ============================================================================== # runtime settings @@ -186,7 +187,6 @@ setenv year_start 1850 setenv year_end 2012 -# setenv sst_grid ${cesm_datadir}/ocn/docn7/domain.ocn.gx1v6.090206.nc setenv sst_grid ${cesm_datadir}/share/domains/domain.ocn.fv0.9x1.25_gx1v6.130409.nc setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc @@ -202,11 +202,11 @@ # queue can be changed during a series by changing the ${case}.run # timewall can be changed during a series by changing the ${case}.run # -# TJH: Advancing 1 instance for 6 hours with 60 pes (4 nodes) -# took less than 5 minutes on yellowstone. +# TJH: Advancing 1 instance for 6 hours and creating synthetic obs took +# less than 3 minutes with 60 pes (4 nodes) on yellowstone. # ============================================================================== -setenv ACCOUNT P86850054 +setenv ACCOUNT P8685xxxx setenv queue economy setenv timewall 0:20 @@ -243,7 +243,7 @@ # VAR is the shell variable name, DIR is the value # ============================================================================== -foreach VAR ( cesmroot dartroot stagedir ) +foreach VAR ( cesmroot dartroot CAM_stagedir ) set DIR = `eval echo \${$VAR}` if ( ! -d $DIR ) then echo "ERROR: directory '$DIR' not found" @@ -744,7 +744,7 @@ # the run_refdate and the time in the filenames can be unrelated. #---------------------------------------------------------------------- -set filedir = ${stagedir} +set filedir = ${CAM_stagedir} @ inst=1 while (\$inst <= 1) @@ -807,16 +807,11 @@ echo "" echo "3) If you want to simply advance CESM without using DART, make sure the" echo " ${case}.run script references the 'no_assimilate.csh' script." -echo " If you want to assimilate, reference the 'assimilate.csh' script." -echo " The default action is to assimilate." echo "" echo "4) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "5) After the job has run, check to make sure it worked and that" -echo " a: the right restart files exist in the run directory," -echo " b: (if you're running DART) the archive dart/hist directory has the DART output," -echo " c: everything is working correctly ..." +echo "5) After the job has run, check to make sure it worked." echo "" echo "6) To extend the run in $stop_n '"$stop_option"' steps," echo " change the env_run.xml variables:" @@ -829,6 +824,8 @@ echo " ./${case}.submit" echo "" +cat ${caseroot}/DART_instructions.txt + exit 0 # Modified: DART/trunk/models/cam/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM_DART_config 2013-10-22 23:28:49 UTC (rev 6539) +++ DART/trunk/models/cam/shell_scripts/CESM_DART_config 2013-10-23 17:24:29 UTC (rev 6540) @@ -288,7 +288,7 @@ # ============================================================================== # Creating the code to add to the *.run script to call DART. -# If there is one instance ... we MUST be running a perfect_model experiment. +# If there is one instance ... we COULD be running a perfect_model experiment. # If there are many instances ... we MUST be assimilating. # The number of instances dictates what DART script will be used. # ============================================================================== @@ -311,8 +311,14 @@ grep 'SUCCESSFUL TERMINATION' $CplLogFile if ( $status == 0 ) then - ${CASEROOT}/DARTSCRIPTNAME + # If you want to simply run CESM, use 'no_assimilate.csh' + # If you want to use DART , use 'DARTSCRIPTNAME' + # You can only use one of the two. + + ${CASEROOT}/no_assimilate.csh +# ${CASEROOT}/DARTSCRIPTNAME + if ( $status == 0 ) then echo "`date` -- DART HAS FINISHED" else @@ -338,11 +344,13 @@ # modify the "here" document to call the logical DART script. if ( $num_instances == 1 ) then - sed -e "s#DARTSCRIPTNAME#perfect_model.csh#" < temp.$$ >! add_to_run.txt + set DARTscript = perfect_model.csh else - sed -e "s#DARTSCRIPTNAME#no_assimilate.csh#" < temp.$$ >! add_to_run.txt + set DARTscript = assimilate.csh endif +sed -e "s#DARTSCRIPTNAME#${DARTscript}#" < temp.$$ >! add_to_run.txt + # Now that the "here" document is created, # determine WHERE to insert it -- ONLY IF it is not already there. @@ -382,25 +390,39 @@ # What to do next # ============================================================================== -echo "Time to check the DART configuration:" -echo "" -echo "1) Modify what you need to in the DART namelists. These are " -echo " ${CASEROOT}/input.nml" -echo "" -echo "2) If you have recompiled any part of the DART system, 'refresh_dart_files'" -echo " will copy them into the correct places." -echo "" -echo "3) Make sure the observation directory names in assimilate.csh match" -echo " those on your system, and submit the CESM job in the normal way." -echo "" -echo "4) Inflation files? Observation files for the time of interest?" -echo "" -echo "5) Make sure the ${CASE}.run script calls 'assimilate.csh' if you desire." -echo " The default behavior is to _not_ invoke DART and simply run CESM." -echo "" -echo "6) You can use ${CASEROOT}/stage_cesm_files" -echo " to stage files to restart a run." +cat << EndOfText >! DART_instructions.txt + +------------------------------------------------------------ + +Check the DART configuration: + +1) The default behavior is to _not_ invoke DART and simply run CESM. + We recommend that you make sure this works before proceeding. + +2) When you want to run DART, edit the ${CASE}.run script to + reference "$DARTscript" instead of "no_assimilate.csh". + +3) Modify what you need to in the DART namelists. These are + ${CASEROOT}/input.nml + +4) If you have recompiled any part of the DART system, 'refresh_dart_files' + will copy them into the correct places. + +5) Make sure the observation directory names in $DARTscript match + those on your system, and submit the CESM job in the normal way. + +6) Inflation files? Observation files for the time of interest? + +7) You can use ${CASEROOT}/stage_cesm_files + to stage files to restart a run. + +------------------------------------------------------------ + +EndOfText + +cat DART_instructions.txt + exit 0 # Modified: DART/trunk/models/cam/shell_scripts/no_assimilate.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/no_assimilate.csh 2013-10-22 23:28:49 UTC (rev 6539) +++ DART/trunk/models/cam/shell_scripts/no_assimilate.csh 2013-10-23 17:24:29 UTC (rev 6540) @@ -33,8 +33,12 @@ # of the form "./${CASE}.cam_${ensemble_member}.i.2000-01-06-00000.nc" #------------------------------------------------------------------------- -set FILE = `head -n 1 rpointer.atm_0001` -set FILE = $FILE:t +if ( $ensemble_size == 1 ) then + set FILE = `head -n 1 rpointer.atm` +else + set FILE = `head -n 1 rpointer.atm_0001` +endif + set FILE = $FILE:r set ATM_DATE_EXT = `echo $FILE:e` @@ -44,19 +48,20 @@ # the short-term archiver 'restores' the CESM files, the links are right. #========================================================================= -set member = 1 -while ( ${member} <= ${ensemble_size} ) +if ( $ensemble_size == 1 ) then + set inst_string = '' + set ATM_INITIAL_FILENAME = ${CASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 +else + set member = 1 + while ( ${member} <= ${ensemble_size} ) + set inst_string = `printf _%04d $member` + set ATM_INITIAL_FILENAME = ${CASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 + @ member++ + end +endif - set inst_string = `printf _%04d $member` - - set ATM_INITIAL_FILENAME = ${CASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc - - ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 - - @ member++ - -end - echo "`date` -- END CAM_NO_ASSIMILATE" exit 0 Modified: DART/trunk/models/cam/shell_scripts/st_archive.sh =================================================================== --- DART/trunk/models/cam/shell_scripts/st_archive.sh 2013-10-22 23:28:49 UTC (rev 6539) +++ DART/trunk/models/cam/shell_scripts/st_archive.sh 2013-10-23 17:24:29 UTC (rev 6540) @@ -2,6 +2,7 @@ # # This code is part of the CESM distribution, # So it is not protected by the DART copyright agreement. +# # DART $Id$ #short-term archive script - move model output out of run directory @@ -149,6 +150,7 @@ set *Posterior_Diag.*.nc; dispose ifiles_n ${sta}/dart/hist $* set *obs_seq.*.out; dispose ifiles_n ${sta}/dart/hist $* set *obs_seq.*.final; dispose ifiles_n ${sta}/dart/hist $* +set *obs_seq.*.perfect; dispose ifiles_n ${sta}/dart/hist $* set *pr*inflate_restart*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_n ${sta}/dart/rest $* set *po*inflate_restart*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_n ${sta}/dart/rest $* @@ -158,7 +160,6 @@ do get_inst_suffix $IDX $NINST_ATM set atm${inst_suffix}.log.*; dispose ifiles_n ${sta}/atm/logs $* -# set cam_initial_${IDX}.nc; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_n ${sta}/atm/rest $* set ${CASE}.cam*${inst_suffix}.r.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/atm/rest $* set ${CASE}.cam*${inst_suffix}.rs.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/atm/rest $* set ${CASE}.cam*${inst_suffix}.ra.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/atm/rest $* @@ -280,7 +281,6 @@ set ${CASE}.pop${inst_suffix}.ro.*; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/ocn/rest $* set ${CASE}.pop${inst_suffix}.d?*; dispose ifiles_n ${sta}/ocn/hist $* set ${CASE}.pop${inst_suffix}.h*; dispose ifiles_n ${sta}/ocn/hist $* - set ${CASE}.docn${inst_suffix}.r.* ; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/ocn/rest $* set ${CASE}.docn${inst_suffix}.rs* ; latest=`ls -rt $* 2> /dev/null | tail -1`; mv $latest ${sta}/rest/${dname} 2> /dev/null; dispose ifiles_y ${sta}/ocn/rest $* set ${CASE}.docn${inst_suffix}.h.* ; dispose ifiles_n ${sta}/ocn/hist $* From nancy at ucar.edu Wed Oct 23 11:29:54 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 23 Oct 2013 11:29:54 -0600 Subject: [Dart-dev] [6541] DART/trunk/models/cam: Moved to deprecated directory. Message-ID: Revision: 6541 Author: thoar Date: 2013-10-23 11:29:54 -0600 (Wed, 23 Oct 2013) Log Message: ----------- Moved to deprecated directory. This manner of configuring a 'perpetual startup' experiment is not extensible to running multiple CESM model components. Added Paths: ----------- DART/trunk/models/cam/deprecated/CESM_setup_startup.csh Removed Paths: ------------- DART/trunk/models/cam/shell_scripts/CESM_setup_startup.csh -------------- next part -------------- Copied: DART/trunk/models/cam/deprecated/CESM_setup_startup.csh (from rev 6523, DART/trunk/models/cam/shell_scripts/CESM_setup_startup.csh) =================================================================== --- DART/trunk/models/cam/deprecated/CESM_setup_startup.csh (rev 0) +++ DART/trunk/models/cam/deprecated/CESM_setup_startup.csh 2013-10-23 17:29:54 UTC (rev 6541) @@ -0,0 +1,709 @@ +#!/bin/csh +# +# DART software - Copyright 2004 - 2013 UCAR. This open source software is +# provided by UCAR, "as is", without charge, subject to all terms of use at +# http://www.image.ucar.edu/DAReS/DART/DART_download +# +# DART $Id$ + +# --------------------- +# Purpose +# --------------------- +# +# This script is designed to configure and build a multi-instance CESM model +# that has CAM, CLM, and CICE as active components over a single data ocean, +# and will use DART to assimilate observations at regular intervals. +# This script does not build DART. It works best if the appropriate DART +# executables have been built, however. +# +# This script relies heavily on the information in: +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html +# +# --------------------- +# How to set up the script +# --------------------- +# -- Either edit and run this script in the $DART/models/cam/shell_scripts +# directory where it first appears, +# or copy it to somewhere that it will be preserved and run it there. +# It will create a 'case' directory, where the model will be built, +# and an execution directory, where each forecast and assimilation will +# take place. The short term archiver will use a third directory for +# storage of model output until it can be moved to long term storage (HPSS) +# -- Examine the whole script to identify things to change for your experiments. +# -- Provide any initial files needed by your run: +# inflation +# sampling error correction +# CAM/CLM/CICE initial ensemble +# ... +# -- Run this script. +# -- Edit the DART input.nml that appears in the $CASEROOT directory. +# -- Submit the job using $CASEROOT/${case}.submit +# +# --------------------- +# Important features +# --------------------- +# +# If you want to change something in your case other than the runtime +# settings, it is safest to delete everything and start the run from scratch. +# For the brave, read +# +# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html + +# ============================================================================== +# ==== Set case options +# ============================================================================== + +# case will be used many ways; +# directory and file names, both locally and on HPSS, and +# script names; so consider it's length and information content. +# num_instances: Number of ensemble members + +setenv case cam_test +setenv compset F_AMIP_CAM5 +setenv resolution f09_f09 +setenv cesmtag cesm1_1_1 +setenv num_instances 4 + +# ============================================================================== +# define machines and directories +# +# mach Computer name +# cesm_datadir Root path of the public CESM data files +# cesmroot Location of the cesm code base +# i.e. cesm1_1_1 on yellowstone +# DARTroot Location of DART code tree. +# Executables, scripts and input in $DARTroot/models/cam/... +# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# Preferably not a frequently scrubbed location. +# This script will delete any existing caseroot, so this script, +# and other useful things should be kept elsewhere. +# rundir (Future) Run-time directory; scrubbable, large amount of space needed. +# exeroot (Future) directory for executables - scrubbable, large amount of space needed. +# archdir (Future) Short-term archive directory +# until the long-term archiver moves it to permanent storage. +# ============================================================================== + +setenv mach yellowstone +setenv cesm_datadir /glade/p/cesm/cseg/inputdata +setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag +setenv caseroot /glade/p/work/${USER}/cases/${case} +setenv exeroot /glade/scratch/${USER}/${case}/bld +setenv rundir /glade/scratch/${USER}/${case}/run +setenv archdir /glade/scratch/${USER}/archive/${case} + +setenv DARTroot /glade/u/home/${USER}/svn/DART/trunk + +# ============================================================================== +# configure settings ... run_startdate format is yyyy-mm-dd +# ============================================================================== + +setenv run_startdate 2008-11-01 +setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc +setenv year_start 1850 +setenv year_end 2010 + +# ============================================================================== +# runtime settings -- How many assimilation steps will be done after this one +# +# stop_option Units for determining the forecast length between assimilations +# stop_n Number of time units in the forecast +# ============================================================================== + +setenv resubmit 0 +setenv stop_option nhours +setenv stop_n 6 + +# ============================================================================== +# job settings +# +# timewall can be changed during a series by changing the ${case}.run +# queue can be changed during a series by changing the ${case}.run +# lrg_ queues are used in order to fit more instances on each node. +# FV 1-degree can comfortably fit 4 instances on 1 lrg_ node (~60 gbyte) +# On bluefire the regular queue (or higher) is probably necessary, +# because it appears that there are not separate queues for the lrg memory +# and the regular memory nodes. So economy jobs requesting smaller numbers +# of processors seem to prevent this lrg_economy job (20 nodes for 1-degree) +# from running for long periods. +# ============================================================================== + +setenv ACCOUNT P8685nnnn +setenv timewall 0:40 +setenv queue regular +setenv ptile 15 + +# ============================================================================== +# set these standard commands based on the machine you are running on. +# ============================================================================== + +set nonomatch # suppress "rm" warnings if wildcard does not match anything + +# The FORCE options are not optional. +# The VERBOSE options are useful for debugging though +# some systems don't like the -v option to any of the following +switch ("`hostname`") + case be*: + # NCAR "bluefire" + set MOVE = '/usr/local/bin/mv -fv' + set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' + set LINK = '/usr/local/bin/ln -fvs' + set REMOVE = '/usr/local/bin/rm -fr' + + breaksw + default: + # NERSC "hopper", NWSC "yellowstone" + set MOVE = '/bin/mv -fv' + set COPY = '/bin/cp -fv --preserve=timestamps' + set LINK = '/bin/ln -fvs' + set REMOVE = '/bin/rm -fr' + + breaksw +endsw + +# ============================================================================== +# Create the case. +# +# For list of the pre-defined cases: ./create_newcase -list +# To create a variant case, see the CESM documentation and carefully +# incorporate any needed changes into this script. +# ============================================================================== + + echo "removing old files from ${caseroot}" + echo "removing old files from ${exeroot}" + echo "removing old files from ${rundir}" + ${REMOVE} ${caseroot} + ${REMOVE} ${exeroot} + ${REMOVE} ${rundir} + ${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ + -res ${resolution} -compset ${compset} + + if ( $status != 0 ) then + echo "ERROR: Case could not be created." + exit -1 + endif + +# ============================================================================== +# Configure the case - this creates the CASEROOT directory. +# ============================================================================== + +cd ${caseroot} + +# Save a copy for debug purposes +foreach FILE ( *xml ) + if ( ~ -e ${FILE}.original ) then + ${COPY} $FILE ${FILE}.original + endif +end + +# num_tasks_per_instance = #tasks_node / #instances_node +# Bluefire: #tasks_node = 64 using SMT +# #instances_node = 1-degree: 4 on lrg_ nodes, 2 on standard. +# 2-degree: 16 on lrg_ nodes, 8 on standard +# CAM5; to be determined, but no more than listed for CAM4 +set num_tasks_per_node = 15 +set num_tasks_per_instance = 8 +set num_threads = 1 + +# This is hard-wiring for the current (1/17/2011) multi-instance CESM restriction +# that all instances must be advanced simultaneously. Work is underway to relax that. +# @ total_nt = $num_instances * $num_tasks_per_instance +# echo "total MPI tasks requested = $total_nt" + +# Atm gets all the nodes and runs. +# The other components divide them up. +# ? ? ? What about sglc? It's a stub, and doesn't matter what pes are assigned to it. +# This algorithm figures out whether there are enough processors requested +# to run each component on whole nodes, or the components need to share some nodes. +# The hard-coded numbers (and ratios between them) are estimates; change them if you +# know better. +#@ atm_pes = $total_nt +#@ large_small = $total_nt / (8 * $num_tasks_per_node) +#if ($large_small > 0) then +# # Large_small > 0 means there are at least 8 nodes requested. +# # Allot whole nodes to the major components. +# @ cpl_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node +# @ ice_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node +# @ glc_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node +# @ ocn_pes = $num_tasks_per_node +# @ rof_pes = $num_tasks_per_node +# @ lnd_pes = $total_nt - ($cpl_pes + $ice_pes + $ocn_pes + $glc_pes + $rof_pes) +#else + # dummy layout + @ cpl_pes = 8 + @ glc_pes = 8 + @ ocn_pes = 8 + @ ice_pes = 32 + @ rof_pes = 32 + @ lnd_pes = 32 + @ atm_pes = 32 +#endif + +echo "task layout" +echo "" +echo "CPL gets $cpl_pes" +echo "ATM gets $atm_pes" +echo "ICE gets $ice_pes" +echo "LND gets $lnd_pes" +echo "ROF gets $rof_pes" +echo "GLC gets $glc_pes" +echo "OCN gets $ocn_pes" +echo "" + +./xmlchange NTHRDS_CPL=$num_threads,NTASKS_CPL=$cpl_pes +./xmlchange NTHRDS_ATM=$num_threads,NTASKS_ATM=$atm_pes,NINST_ATM=$num_instances +./xmlchange NTHRDS_ICE=$num_threads,NTASKS_ICE=$ice_pes,NINST_ICE=$num_instances +./xmlchange NTHRDS_LND=$num_threads,NTASKS_LND=$lnd_pes,NINST_LND=$num_instances +./xmlchange NTHRDS_ROF=$num_threads,NTASKS_ROF=$rof_pes,NINST_ROF=$num_instances +./xmlchange NTHRDS_GLC=$num_threads,NTASKS_GLC=$glc_pes,NINST_GLC=1 +./xmlchange NTHRDS_OCN=$num_threads,NTASKS_OCN=$ocn_pes,NINST_OCN=1 +./xmlchange ROOTPE_OCN=$ice_pes + +./xmlchange RUN_TYPE=startup +./xmlchange RUN_STARTDATE=$run_startdate +./xmlchange EXEROOT=${exeroot} +./xmlchange PIO_TYPENAME=pnetcdf + +# DOUT_S is to turn on/off the short-term archiving +# DOUT_L_MS is to store to the HPSS (formerly "MSS") + +./xmlchange DOUT_S_ROOT=${archdir} +./xmlchange DOUT_S=TRUE +./xmlchange DOUT_S_SAVE_INT_REST_FILES=TRUE +./xmlchange DOUT_L_MS=FALSE +./xmlchange DOUT_L_MSROOT="csm/${case}" +./xmlchange DOUT_L_HTAR=FALSE + +./xmlchange SSTICE_DATA_FILENAME=$sst_dataset +./xmlchange SSTICE_YEAR_ALIGN=$year_start +./xmlchange SSTICE_YEAR_START=$year_start +./xmlchange SSTICE_YEAR_END=$year_end + +# Do not change the CALENDAR or the CONTINUE_RUN + +./xmlchange CALENDAR=GREGORIAN +./xmlchange STOP_OPTION=$stop_option +./xmlchange STOP_N=$stop_n +./xmlchange CONTINUE_RUN=FALSE +./xmlchange RESUBMIT=$resubmit + +# The river transport model ON is useful only when using an active ocean or +# land surface diagnostics. Setting ROF_GRID to 'null' turns off the RTM. + +./xmlchange ROF_GRID='r05' + +# level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) + +./xmlchange DEBUG=FALSE +./xmlchange INFO_DBUG=0 + +# ============================================================================== +# Set up the case. +# This creates the EXEROOT and RUNDIR directories. +# ============================================================================== + +./cesm_setup + +if ( $status != 0 ) then + echo "ERROR: Case could not be set up." + exit -2 +endif + +# ============================================================================== +# Modify namelist templates for each instance. +# +# hist_empty_htapes = .true. suppresses the creation of all history files +# hist_fincl1 = 'TG', except the first one, which will have one variable +# hist_nhtfrq = -$stop_n, create one every $stop_n HOURS +# hist_mfilt = 1, with precisely one day in it +# hist_avgflag_pertape = 'I' use instantaneous values - no average +# +# ============================================================================== + +set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" + +@ inst = 1 +while ($inst <= $num_instances) + + set instance = `printf %04d $inst` + + # =========================================================================== + set fname = "user_nl_cam_$instance" + # =========================================================================== + # A lot of the files specified here are because the 'default' files only + # contain data through 2005 and we are interested in timeframes after that. + # + # CAM5 does prognostic aerosols by default. If you want to prescribe them, + # use the following variables with your own settings ... +# echo " aerodep_flx_datapath = '${chem_datapath}/aero' " >> ${fname} +# echo " aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc' " >> ${fname} +# echo " aerodep_flx_cycle_yr = 2000 " >> ${fname} +# echo " aerodep_flx_type = 'CYCLICAL' " >> ${fname} + + # This killed CAM5 ... but may work for some other compset +# echo " iradae = -$stop_n " >> ${fname} + + echo " inithist = 'ENDOFRUN' " >> ${fname} + echo " ncdata = 'cam_initial_${instance}.nc' " >> ${fname} + echo " avgflag_pertape = 'A' " >> ${fname} + echo " div24del2flag = 4 " >> ${fname} + echo " empty_htapes = .true. " >> ${fname} + echo " fincl1 = 'PHIS:I' " >> ${fname} + echo " nhtfrq = -$stop_n " >> ${fname} + echo " mfilt = 1 " >> ${fname} + + echo " bndtvghg = '${cesm_datadir}/atm/cam/ggas/ghg_hist_1765-2009_c100902.nc' " >> ${fname} + echo " prescribed_ozone_file = 'ozone_1.9x2.5_L26_1850-2015_rcp45_c101108.nc' " >> ${fname} + echo " tracer_cnst_file = 'oxid_1.9x2.5_L26_1850-2015_rcp45_c101108.nc' " >> ${fname} + + echo " ext_frc_specifier = " >> ${fname} + echo " 'SO2 -> ${chem_datapath}/emis/ar5_mam3_so2_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'bc_a1 -> ${chem_datapath}/emis/ar5_mam3_bc_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a1 -> ${chem_datapath}/emis/ar5_mam3_num_a1_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a2 -> ${chem_datapath}/emis/ar5_mam3_num_a2_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'pom_a1 -> ${chem_datapath}/emis/ar5_mam3_oc_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a1 -> ${chem_datapath}/emis/ar5_mam3_so4_a1_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a2 -> ${chem_datapath}/emis/ar5_mam3_so4_a2_elev_1850-2010_c20100902_v12.nc' " >> ${fname} + + echo " srf_emis_specifier = " >> ${fname} + echo " 'DMS -> ${chem_datapath}/emis/aerocom_mam3_dms_surf_1849-2010_c20100902.nc'," >> ${fname} + echo " 'SO2 -> ${chem_datapath}/emis/ar5_mam3_so2_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'SOAG -> ${chem_datapath}/emis/ar5_mam3_soag_1.5_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'bc_a1 -> ${chem_datapath}/emis/ar5_mam3_bc_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a1 -> ${chem_datapath}/emis/ar5_mam3_num_a1_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a2 -> ${chem_datapath}/emis/ar5_mam3_num_a2_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'pom_a1 -> ${chem_datapath}/emis/ar5_mam3_oc_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a1 -> ${chem_datapath}/emis/ar5_mam3_so4_a1_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a2 -> ${chem_datapath}/emis/ar5_mam3_so4_a2_surf_1850-2010_c20100902_v12.nc' " >> ${fname} + + echo " solar_data_file = '${cesm_datadir}/atm/cam/solar/spectral_irradiance_Lean_1610-2009_ann_c100405.nc' " >> ${fname} + + + # =========================================================================== + set fname = "user_nl_clm_$instance" + # =========================================================================== + # hist_empty_htapes must be false at the moment. Otherwise the CLM restart file + # gets created with ntapes=0 which prevents CLM from restarting. Crazy. + + echo "hist_empty_htapes = .false. " >> ${fname} + echo "finidat = 'clm_restart_${instance}.nc' " >> ${fname} + echo "fpftdyn = '${cesm_datadir}/lnd/clm2/surfdata/surfdata.pftdyn_0.9x1.25_rcp4.5_simyr1850-2100_c100406.nc' " >> ${fname} + + # =========================================================================== + set fname = "user_nl_cice_$instance" + # =========================================================================== + + echo " ice_ic = 'ice_restart_${instance}.nc' " >> ${fname} + + @ inst ++ +end + +# ============================================================================== +# to create custom streamfiles ... +# "To modify the contents of a stream txt file, first use preview_namelists to +# obtain the contents of the stream txt files in CaseDocs, and then place a copy +# of the modified stream txt file in $CASEROOT with the string user_ prepended." +# +# -or- +# +# we copy a template stream txt file from the +# $DARTroot/models/POP/shell_scripts directory and modify one for each instance. +# +# ============================================================================== + +# none needed for CAM + +./preview_namelists + +# ============================================================================== +# Update source files if need be +# Ideally, using DART will not require any source mods. +# Until then, this script accesses source mods from a hard-wired location below. +# Those may eventually be packaged into the DART repository. +# If you have additional source mods, they will need to be merged into any DART +# mods and put in the SourceMods subdirectory found in the 'case' directory. +# ============================================================================== + +if ( -d ~/${cesmtag}/SourceMods ) then + ${COPY} -r ~/${cesmtag}/SourceMods/* ${caseroot}/SourceMods/ +else + echo "ERROR - No SourceMods for this case." + echo "ERROR - No SourceMods for this case." + echo "DART requires modifications to several src.pop2/ files." + echo "These files can be downloaded from:" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "untar these into your HOME directory - they will create a" + echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." + exit -4 +endif + +# ============================================================================== +# build +# ============================================================================== + +echo '' +echo 'Building the case' +echo '' + +./${case}.build + +if ( $status != 0 ) then + echo "ERROR: Case could not be built." + exit -5 +endif + +# ============================================================================== +# Stage the restarts now that the run directory exists +# +# CAM5 ... /glade/p/work/raeder/Exp/Pincus_Cam5/archive/Exp3/rest/2008-11-01-00000 +# ============================================================================== + +set stagedir = /glade/p/work/raeder/Exp/Pincus_Cam5/archive/Exp3/rest/2008-11-01-00000 + +echo '' +echo "Copying the restart files from ${stagedir}" +echo 'into the CESM run directory.' +echo '' + +@ inst = 1 +while ($inst <= $num_instances) + set instance = `printf %04d $inst` + + echo '' + echo "Staging restarts for instance $inst of $num_instances" + + ${COPY} ${stagedir}/cam_initial_${inst}.nc ${rundir}/cam_initial_${instance}.nc + ${COPY} ${stagedir}/clm_restart_${inst}.nc ${rundir}/clm_restart_${instance}.nc + ${COPY} ${stagedir}/ice_restart_${inst}.nc ${rundir}/ice_restart_${instance}.nc + + @ inst ++ +end + +if ( -e ${stagedir}/prior_inflate_restart* ) then + ${COPY} ${stagedir}/prior_inflate_restart* ${rundir}/. +endif + +# ============================================================================== +# Edit the run script to reflect project, queue, and wallclock +# ============================================================================== + +echo '' +echo 'Updating the run script to set wallclock and queue.' +echo '' + +if ( ~ -e ${case}.run.original ) then + ${COPY} ${case}.run ${case}.run.original +endif + +source Tools/ccsm_getenv +set BATCH = `echo $BATCHSUBMIT | sed 's/ .*$//'` +switch ( $BATCH ) + case bsub*: + # NCAR "bluefire", "yellowstone" + set TIMEWALL=`grep BSUB ${case}.run | grep -e '-W' ` + set QUEUE=`grep BSUB ${case}.run | grep -e '-q' ` + sed -e "s/ptile=[0-9][0-9]*/ptile=$ptile/" \ + -e "s/$TIMEWALL[3]/$timewall/" \ + -e "s/$QUEUE[3]/$queue/" < ${case}.run >! temp.$$ + ${MOVE} temp.$$ ${case}.run + breaksw + + default: + + breaksw +endsw + +# ============================================================================== +# IMPORTANT: All resubmits must be type 'startup'. +# ============================================================================== + +echo '' +echo 'Changing the run script such that all the resubmits are "startup",' +echo 'which means CONTINUE_RUN should be FALSE' +echo '' + +ex ${case}.run <! add_to_run.txt + +# ------------------------------------------------------------------------- +# START OF DART: if CESM finishes correctly (pirated from ccsm_postrun.csh); +# perform an assimilation with DART. + +set CplLogFile = `ls -1t cpl.log* | head -n 1` +if ($CplLogFile == "") then + echo 'ERROR: Model did not complete - no cpl.log file present - exiting.' + echo 'ERROR: Assimilation will not be attempted.' + exit -1 +endif + +grep 'SUCCESSFUL TERMINATION' $CplLogFile +if ( $status == 0 ) then + ${CASEROOT}/assimilate.csh + + if ( $status == 0 ) then + echo "`date` -- DART HAS FINISHED" + else + echo "`date` -- DART FILTER ERROR - ABANDON HOPE" + exit -3 + endif +else + echo 'ERROR: Model did not complete successfully - exiting.' + echo 'ERROR: Assimilation will not be attempted.' + exit -2 +endif + +# END OF DART BLOCK +# ------------------------------------------------------------------------- +"EndOfText" + +# Now that the "here" document is created, +# determine WHERE to insert it -- ONLY IF it is not already there. + +grep "ABANDON HOPE" ${case}.run +set STATUSCHECK = $status + +if ( ${STATUSCHECK} == 0 ) then + echo "DART block already present in ${case}.run" +else if ( ${STATUSCHECK} == 1 ) then + + set MYSTRING = `grep --line-number "CSM EXECUTION HAS FINISHED" ${case}.run` + set MYSTRING = `echo $MYSTRING | sed -e "s#:# #g"` + + @ origlen = `cat ${case}.run | wc -l` + @ keep = $MYSTRING[1] + @ lastlines = $origlen - $keep + + head -n $keep ${case}.run >! temp.$$ + cat add_to_run.txt >> temp.$$ + tail -n $lastlines ${case}.run >> temp.$$ + + ${MOVE} temp.$$ ${case}.run + ${REMOVE} add_to_run.txt + +else + echo "ERROR in grep of ${case}.run: aborting" + echo "status was ${STATUSCHECK}" + exit -6 +endif + +chmod 0755 ${case}.run + +# ============================================================================== +# Stage the required parts of DART in the CASEROOT directory. +# ============================================================================== + +# The standard CESM short-term archiving script may need to be altered +# to archive addtional or subsets of things, or to reduce the amount of +# data that is sent to the long-term archive. + +if ( ~ -e Tools/st_archive.sh.original ) then + ${COPY} Tools/st_archive.sh Tools/st_archive.sh.original +endif + +# NOTE: the assimilate.csh script and input.nml must be modified for your +# situation. The script has variables that point to the location of +# the observations sequence files and the DART working directory +# and may be customized for a more efficient PE layout for DART. +# If you are running this, you know what to do with input.nml. +# If you don't, you should give up now. Really. + +${COPY} ${DARTroot}/models/cam/shell_scripts/st_archive.sh Tools/ +${COPY} ${DARTroot}/models/cam/shell_scripts/assimilate.csh assimilate.csh +${COPY} ${DARTroot}/models/cam/work/input.nml input.nml + +# Ensure that the input.nml ensemble size matches the number of instances. +# WARNING: the output files contain ALL ensemble members ==> BIG + +ex input.nml < +# $URL$ +# $Revision$ +# $Date$ + Deleted: DART/trunk/models/cam/shell_scripts/CESM_setup_startup.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM_setup_startup.csh 2013-10-23 17:24:29 UTC (rev 6540) +++ DART/trunk/models/cam/shell_scripts/CESM_setup_startup.csh 2013-10-23 17:29:54 UTC (rev 6541) @@ -1,709 +0,0 @@ -#!/bin/csh -# -# DART software - Copyright 2004 - 2013 UCAR. This open source software is -# provided by UCAR, "as is", without charge, subject to all terms of use at -# http://www.image.ucar.edu/DAReS/DART/DART_download -# -# DART $Id$ - -# --------------------- -# Purpose -# --------------------- -# -# This script is designed to configure and build a multi-instance CESM model -# that has CAM, CLM, and CICE as active components over a single data ocean, -# and will use DART to assimilate observations at regular intervals. -# This script does not build DART. It works best if the appropriate DART -# executables have been built, however. -# -# This script relies heavily on the information in: -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html -# -# --------------------- -# How to set up the script -# --------------------- -# -- Either edit and run this script in the $DART/models/cam/shell_scripts -# directory where it first appears, -# or copy it to somewhere that it will be preserved and run it there. -# It will create a 'case' directory, where the model will be built, -# and an execution directory, where each forecast and assimilation will -# take place. The short term archiver will use a third directory for -# storage of model output until it can be moved to long term storage (HPSS) -# -- Examine the whole script to identify things to change for your experiments. -# -- Provide any initial files needed by your run: -# inflation -# sampling error correction -# CAM/CLM/CICE initial ensemble -# ... -# -- Run this script. -# -- Edit the DART input.nml that appears in the $CASEROOT directory. -# -- Submit the job using $CASEROOT/${case}.submit -# -# --------------------- -# Important features -# --------------------- -# -# If you want to change something in your case other than the runtime -# settings, it is safest to delete everything and start the run from scratch. -# For the brave, read -# -# http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1142.html - -# ============================================================================== -# ==== Set case options -# ============================================================================== - -# case will be used many ways; -# directory and file names, both locally and on HPSS, and -# script names; so consider it's length and information content. -# num_instances: Number of ensemble members - -setenv case cam_test -setenv compset F_AMIP_CAM5 -setenv resolution f09_f09 -setenv cesmtag cesm1_1_1 -setenv num_instances 4 - -# ============================================================================== -# define machines and directories -# -# mach Computer name -# cesm_datadir Root path of the public CESM data files -# cesmroot Location of the cesm code base -# i.e. cesm1_1_1 on yellowstone -# DARTroot Location of DART code tree. -# Executables, scripts and input in $DARTroot/models/cam/... -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. -# Preferably not a frequently scrubbed location. -# This script will delete any existing caseroot, so this script, -# and other useful things should be kept elsewhere. -# rundir (Future) Run-time directory; scrubbable, large amount of space needed. -# exeroot (Future) directory for executables - scrubbable, large amount of space needed. -# archdir (Future) Short-term archive directory -# until the long-term archiver moves it to permanent storage. -# ============================================================================== - -setenv mach yellowstone -setenv cesm_datadir /glade/p/cesm/cseg/inputdata -setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag -setenv caseroot /glade/p/work/${USER}/cases/${case} -setenv exeroot /glade/scratch/${USER}/${case}/bld -setenv rundir /glade/scratch/${USER}/${case}/run -setenv archdir /glade/scratch/${USER}/archive/${case} - -setenv DARTroot /glade/u/home/${USER}/svn/DART/trunk - -# ============================================================================== -# configure settings ... run_startdate format is yyyy-mm-dd -# ============================================================================== - -setenv run_startdate 2008-11-01 -setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc -setenv year_start 1850 -setenv year_end 2010 - -# ============================================================================== -# runtime settings -- How many assimilation steps will be done after this one -# -# stop_option Units for determining the forecast length between assimilations -# stop_n Number of time units in the forecast -# ============================================================================== - -setenv resubmit 0 -setenv stop_option nhours -setenv stop_n 6 - -# ============================================================================== -# job settings -# -# timewall can be changed during a series by changing the ${case}.run -# queue can be changed during a series by changing the ${case}.run -# lrg_ queues are used in order to fit more instances on each node. -# FV 1-degree can comfortably fit 4 instances on 1 lrg_ node (~60 gbyte) -# On bluefire the regular queue (or higher) is probably necessary, -# because it appears that there are not separate queues for the lrg memory -# and the regular memory nodes. So economy jobs requesting smaller numbers -# of processors seem to prevent this lrg_economy job (20 nodes for 1-degree) -# from running for long periods. -# ============================================================================== - -setenv ACCOUNT P8685nnnn -setenv timewall 0:40 -setenv queue regular -setenv ptile 15 - -# ============================================================================== -# set these standard commands based on the machine you are running on. -# ============================================================================== - -set nonomatch # suppress "rm" warnings if wildcard does not match anything - -# The FORCE options are not optional. -# The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following -switch ("`hostname`") - case be*: - # NCAR "bluefire" - set MOVE = '/usr/local/bin/mv -fv' - set COPY = '/usr/local/bin/cp -fv --preserve=timestamps' - set LINK = '/usr/local/bin/ln -fvs' - set REMOVE = '/usr/local/bin/rm -fr' - - breaksw - default: - # NERSC "hopper", NWSC "yellowstone" - set MOVE = '/bin/mv -fv' - set COPY = '/bin/cp -fv --preserve=timestamps' - set LINK = '/bin/ln -fvs' - set REMOVE = '/bin/rm -fr' - - breaksw -endsw - -# ============================================================================== -# Create the case. -# -# For list of the pre-defined cases: ./create_newcase -list -# To create a variant case, see the CESM documentation and carefully -# incorporate any needed changes into this script. -# ============================================================================== - - echo "removing old files from ${caseroot}" - echo "removing old files from ${exeroot}" - echo "removing old files from ${rundir}" - ${REMOVE} ${caseroot} - ${REMOVE} ${exeroot} - ${REMOVE} ${rundir} - ${cesmroot}/scripts/create_newcase -case ${caseroot} -mach ${mach} \ - -res ${resolution} -compset ${compset} - - if ( $status != 0 ) then - echo "ERROR: Case could not be created." - exit -1 - endif - -# ============================================================================== -# Configure the case - this creates the CASEROOT directory. -# ============================================================================== - -cd ${caseroot} - -# Save a copy for debug purposes -foreach FILE ( *xml ) - if ( ~ -e ${FILE}.original ) then - ${COPY} $FILE ${FILE}.original - endif -end - -# num_tasks_per_instance = #tasks_node / #instances_node -# Bluefire: #tasks_node = 64 using SMT -# #instances_node = 1-degree: 4 on lrg_ nodes, 2 on standard. -# 2-degree: 16 on lrg_ nodes, 8 on standard -# CAM5; to be determined, but no more than listed for CAM4 -set num_tasks_per_node = 15 -set num_tasks_per_instance = 8 -set num_threads = 1 - -# This is hard-wiring for the current (1/17/2011) multi-instance CESM restriction -# that all instances must be advanced simultaneously. Work is underway to relax that. -# @ total_nt = $num_instances * $num_tasks_per_instance -# echo "total MPI tasks requested = $total_nt" - -# Atm gets all the nodes and runs. -# The other components divide them up. -# ? ? ? What about sglc? It's a stub, and doesn't matter what pes are assigned to it. -# This algorithm figures out whether there are enough processors requested -# to run each component on whole nodes, or the components need to share some nodes. -# The hard-coded numbers (and ratios between them) are estimates; change them if you -# know better. -#@ atm_pes = $total_nt -#@ large_small = $total_nt / (8 * $num_tasks_per_node) -#if ($large_small > 0) then -# # Large_small > 0 means there are at least 8 nodes requested. -# # Allot whole nodes to the major components. -# @ cpl_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node -# @ ice_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node -# @ glc_pes = ( $total_nt / (3 * $num_tasks_per_node) ) * $num_tasks_per_node -# @ ocn_pes = $num_tasks_per_node -# @ rof_pes = $num_tasks_per_node @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Wed Oct 23 17:02:43 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Wed, 23 Oct 2013 17:02:43 -0600 Subject: [Dart-dev] [6542] DART/trunk/models/cam/shell_scripts: These are the 'best practices' scripts for running a CESM CAM (F compset). Message-ID: Revision: 6542 Author: thoar Date: 2013-10-23 17:02:43 -0600 (Wed, 23 Oct 2013) Log Message: ----------- These are the 'best practices' scripts for running a CESM CAM (F compset). This requires the use of the CAM sourcemods that enable a continuation run instead of a 'perpetual startup' run. The perpetual startup configuration could not be extended to have multiple active model components and so is deprecated at this time. Note: there is a lot of embedded documentation. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid DART/trunk/models/cam/shell_scripts/CESM_DART_config DART/trunk/models/cam/shell_scripts/assimilate.csh -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-23 17:29:54 UTC (rev 6541) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-23 23:02:43 UTC (rev 6542) @@ -14,10 +14,12 @@ # using an F compset where CAM and CLM are active. The initial states # come from a single multi-instance reference case so a CESM hybrid setup is used. # -# Because the atmosphere assimilations typically occur every 6 hours, the methodology -# here reflects that. All of CESM stops every 6 hours (requiring the ocean to couple -# 4x per day) so that a CAM output file would be available for assimilation. +# DOCN: We are using a single data ocean. # +# Because the atmosphere assimilations typically occur every 6 hours, +# the methodology here reflects that. All of CESM stops every 6 hours +# so that a CAM output file would be available for assimilation. +# # This script results in a viable setup for a CESM multi-instance experiment. You # are STRONGLY encouraged to run the multi-instance CESM a few times and experiment # with different settings BEFORE you try to assimilate observations. The amount of @@ -27,8 +29,8 @@ # CESM/DART requires some modifications to the CESM source code EVEN IF YOU ARE NOT # assimilating. The modifications for CAM require a modification to the # CESM ${CASE}.run script. The modifications will invoke a DART script that will -# either assimilate or 'do nothing'. It is stronly recommended that you configure -# the ${CASE}.run script to 'do nothing' initially. +# either assimilate or 'do nothing'. It is strongly recommended that you configure +# the ${CASE}.run script to 'do nothing' initially. This is the default behavior. # # CASEROOT/CESM_DART_config will augment the CESM case with the required setup # and configuration to perform a DART assimilation and modify the ${CASE}.run script @@ -46,37 +48,22 @@ # This script sets up a CESM multi-instance run as we understand them and # it has almost nothing to do with DART. This is intentional. # -# -- Edit and run this script in the $DART/models/CESM/shell_scripts directory -# or copy it to somewhere that it will be preserved and run it there. -# It will create a CESM 'CASE' directory, where the model will be built, -# and a run directory, where each forecast (and assimilation) will -# take place. The short term archiver will use a third directory for -# storage of model output until it can be moved to long term storage (HPSS) -# # -- Examine the whole script to identify things to change for your experiments. # -# -- Provide the CESM initial ensemble needed by your run. +# -- Edit this script in the $DART/models/cam/shell_scripts directory +# or copy it to somewhere where it will be preserved. # -# -- Run this script. -# -- Edit the DART input.nml that appears in the $CASEROOT directory, if not done here. -# -- Submit the job using $CASEROOT/${case}.submit +# -- Run this script. When it is executed, it will create: +# 1) a CESM 'CASE' directory, where the model will be built, +# 2) a run directory, where each forecast (and assimilation) will take place, +# 3) a bld directory for the executables. # -# -- Read, understand, and execute ${CASEROOT}/CESM_DART_config +# This script also executes ${CASEROOT}/CESM_DART_config to modify +# the $CASEROOT/${CASE}.run script so that the SourceMods for CAM +# are effective. CESM_DART_config will also augment the case with all +# the pieces necessary to run DART when the time comes. # -# Table of contents: -# Set up case and directory names -# Refined grid mods -# create_newcase -# cd caseroot -# xmlchanges -# cesm_setup -# namelist mods -# preview_namelists -# SourceMods -# case.build -# stage ICs -# update the run script -# Bring in parts of DART; executables, support files,... +# -- Submit the job using ${CASEROOT}/${CASE}.submit # # --------------------- # Important features @@ -106,19 +93,20 @@ # 0.9x1.25_0.9x1.25 (f09_f09) # 1 deg grid for atm/lnd, gx1v6 mask. Normally used for CAM with prescribed lnd, ocn and ice. -setenv case test_cam +setenv case cam_test setenv compset F_AMIP_CAM5 setenv resolution f09_f09 setenv cesmtag cesm1_1_1 -setenv num_instances 4 +setenv num_instances 30 # ============================================================================== # define machines and directories # # mach Computer name -# cesmroot Location of the cesm code base +# cesm_datadir Location of some supporting CESM data files. +# cesmroot Location of the CESM code base # For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# caseroot Your (future) CESM case directory, where this CESM+DART will be built. # Preferably not a frequently scrubbed location. # This script will delete any existing caseroot, so this script, # and other useful things should be kept elsewhere. @@ -131,8 +119,8 @@ # ============================================================================== setenv mach yellowstone -setenv cesm_datadir /glade/p/cesm/cseg/inputdata +setenv cesm_datadir /glade/p/cesm/cseg/inputdata setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag setenv caseroot /glade/p/work/${USER}/cases/${case} setenv exeroot /glade/scratch/${USER}/${case}/bld @@ -155,9 +143,10 @@ # THIS IS THE LOCATION of the 'reference case'. -setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} +setenv CAM_stagedir /glade/p/image/CESM_initial_ensemble/rest/${run_refdate}-${run_reftod} -# setenv stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} +# alternative reference case for different times may be available here: +# setenv CAM_stagedir /glade/p/work/raeder/Models/CAM_init/${run_refcase}_${run_refdate} # ============================================================================== # runtime settings @@ -170,7 +159,7 @@ # stop_n Number of time units in the first forecast # # If the long-term archiver is off, you get a chance to examine the files before -# they get moved to long-term storage. You can always submit $CASE.l_archive +# they get moved to long-term storage. You can always submit $case.l_archive # whenever you want to free up space in the short-term archive directory. # # ============================================================================== @@ -183,7 +172,6 @@ setenv year_start 1850 setenv year_end 2012 -# setenv sst_grid ${cesm_datadir}/ocn/docn7/domain.ocn.gx1v6.090206.nc setenv sst_grid ${cesm_datadir}/share/domains/domain.ocn.fv0.9x1.25_gx1v6.130409.nc setenv sst_dataset ${cesm_datadir}/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2011_c110307.nc @@ -199,12 +187,12 @@ # queue can be changed during a series by changing the ${case}.run # timewall can be changed during a series by changing the ${case}.run # -# TJH: Advancing 30 instances for 6 hours with 900 pes -# took less than 10 minutes on yellowstone. +# TJH: Advancing 30 instances for 6 hours and assimilating took +# less than 10 minutes on yellowstone using 1800 pes (120 nodes) # ============================================================================== -setenv ACCOUNT P86850054 -setenv queue premium +setenv ACCOUNT P8685xxxx +setenv queue economy setenv timewall 0:20 # ============================================================================== @@ -240,7 +228,7 @@ # VAR is the shell variable name, DIR is the value # ============================================================================== -foreach VAR ( cesmroot dartroot stagedir ) +foreach VAR ( cesmroot dartroot CAM_stagedir ) set DIR = `eval echo \${$VAR}` if ( ! -d $DIR ) then echo "ERROR: directory '$DIR' not found" @@ -295,7 +283,7 @@ # ============================================================================== if ( ! -e CESM_DART_config ) then - ${COPY} ${dartroot}/models/cam/shell_CESM/CESM_DART_config . + ${COPY} ${dartroot}/models/cam/shell_scripts/CESM_DART_config . endif if ( -e CESM_DART_config ) then @@ -441,11 +429,6 @@ ./xmlchange NCPL_BASE_PERIOD=day ./xmlchange ATM_NCPL=48 -# TJH FIXME ... any reason to turn ON the biogeochemistry in CLM when atmos only? -# There is no compset that has everything we want, -# so we are also turning on the CLM biogeochemistry. -#./xmlchange CLM_CONFIG_OPTS='-bgc cn' - if ($short_term_archiver == 'off') then ./xmlchange DOUT_S=FALSE else @@ -469,9 +452,8 @@ ./xmlchange RTM_MODE='NULL' # level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) -# Early versions (late 2012) of CAM-SE could not handle these being other than FALSE and 0. -./xmlchange DEBUG=TRUE -./xmlchange INFO_DBUG=1 +./xmlchange DEBUG=FALSE +./xmlchange INFO_DBUG=0 # ============================================================================== # Set up the case. @@ -517,7 +499,7 @@ endsw # This is the part that modifies the run script to allow CESM to advance -# correctly given the modifications necessary for CAM. It also copies several +# correctly given the modifications of CAM for DART. It also copies several # required DART files to the caseroot directory. ./CESM_DART_config @@ -568,17 +550,17 @@ endif # ============================================================================== -# Modify namelist templates for each instance. This is a bit of a nuisance in -# that we are pulling in restart and initial files from 'all over the place' -# and each model component has a different strategy. +# Modify namelist templates for each instance. +# CAM SourceMods force CAM to ALWAYS read INITIAL files specified by namelist var 'ncdata' # -# In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): +# When CONTINUE_RUN = FALSE (i.e. just starting up): # -# CAM has been forced to read initial files - specified by namelist var:ncdata # CICE reads from namelist variable 'ice_ic' # CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 # it does not use the instance string. There is a patch for clm.buildnml.csh # +# When CONTINUE_RUN = TRUE, CICE and CLM read from pointer files. +# # All of these must later on be staged with these same filenames. # OR - all these namelists can be changed to match whatever has been staged. # MAKE SURE THE STAGING SECTION OF THIS SCRIPT MATCHES THESE VALUES. @@ -605,8 +587,8 @@ echo " nhtfrq = -$stop_n " >> ${fname} echo " mfilt = 1 " >> ${fname} - # A lot of the files specified here are because the 'default' files only - # contain data through 2005 and we are interested in timeframes after that. + # Files specified below are because the 'default' files only + # contain data through 2005 and we may be interested in timeframes after that. set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" @@ -630,8 +612,8 @@ # Making a compact .h0. file is a good idea, since the clm restart files # do not have all the metadata required to reconstruct a gridded field. # 'TSA' is 2m surface air temperature. - # TJH ... should make monthly average file or something ... - # 1.3 MB is 100x bigger than CAM history file. See page 65 of + # TJH ... should make monthly average file or something ... + # 1.3 MB is 100x bigger than CAM history file. See page 65 of # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf # # Every 6 hours @@ -643,7 +625,7 @@ # KDR; Was the ntapes bug fixed? # That's where the CLM restart file always has dimension ntapes, -# but it's set = 0 for hist_empty_htapes = .false., +# but it's set = 0 for hist_empty_htapes = .true. and empty hist_fincl1, # which prevents the next forecast from starting. # Kluzek fixed it in some recent (<9/2013) version, but not 1_1_1. # There's a supposed fix in ~/cesm1_1_1/SourceMods/src.clm/HistFileMod.F90, @@ -710,7 +692,7 @@ # the restarts should be in the short term archive directory # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back - ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . || exit -1 else @@ -722,10 +704,10 @@ set inst_string = \`printf _%04d \$inst\` - echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv - echo "${CASE}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} - echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} - echo "${CASE}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + echo "${case}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${case}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} + echo "${case}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} + echo "${case}.cam\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} @ inst ++ @@ -737,16 +719,16 @@ @ inst=1 while (\$inst <= $num_instances) set inst_string = \`printf _%04d \$inst\` - ${LINK} ${CASE}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc + ${LINK} ${case}.cam\${inst_string}.i.\${restart_time}.nc cam_initial\${inst_string}.nc @ inst ++ end # relink the pointer file for the data ocean - echo "${CASE}.docn.r.\${restart_time}.nc" >! rpointer.ocn + echo "${case}.docn.r.\${restart_time}.nc" >! rpointer.ocn echo "" >> rpointer.ocn - echo "${CASE}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn + echo "${case}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn echo "" >> rpointer.ocn - + exit 0 endif @@ -754,7 +736,7 @@ # This block prepares the initial run directory for a startup run. # The CLM, CICE, ROF namelists specify a relative path for the initial file. # The CAM namelist specifies a static name. -# POP uses pointer files. +# POP uses pointer files. # # The reference case may more instances than we need, # and it certainly has more files than we need. @@ -764,7 +746,7 @@ # the run_refdate and the time in the filenames can be unrelated. #---------------------------------------------------------------------- -set filedir = ${stagedir} +set filedir = ${CAM_stagedir} @ inst=1 while (\$inst <= $num_instances) @@ -821,16 +803,12 @@ echo "" echo "3) If you want to simply advance CESM without using DART, make sure the" echo " ${case}.run script references the 'no_assimilate.csh' script." -echo " If you want to assimilate, reference the 'assimilate.csh' script." -echo " The default action is to assimilate." +echo " This is the default behavior." echo "" echo "4) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "5) After the job has run, check to make sure it worked and that" -echo " a: the right restart files exist in the run directory," -echo " b: (if you're running DART) the archive dart/hist directory has the DART output," -echo " c: everything is working correctly ..." +echo "5) After the job has run, check to make sure it worked." echo "" echo "6) To extend the run in $stop_n '"$stop_option"' steps," echo " change the env_run.xml variables:" @@ -841,8 +819,9 @@ echo "" echo " and" echo " ./${case}.submit" -echo "" +cat ${caseroot}/DART_instructions.txt + exit 0 # Modified: DART/trunk/models/cam/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM_DART_config 2013-10-23 17:29:54 UTC (rev 6541) +++ DART/trunk/models/cam/shell_scripts/CESM_DART_config 2013-10-23 23:02:43 UTC (rev 6542) @@ -267,7 +267,7 @@ end # ============================================================================== -# Initial setup for the default inflation scenario. +# INFLATION : Initial setup for the default inflation scenario. # ============================================================================== # CAM usually uses adaptive state-space prior inflation. The initial settings # are in the filter_nml and ... during an assimilation experiment, the output @@ -283,6 +283,9 @@ # file before it performs the assimilation. After the first assimilation takes # place, the cookie file must be 'eaten' so that subsequent assimilations do not # overwrite whatever _should_ be there. +# +# IMPORTANT: If you stage your own inflation file, you must REMOVE the cookie +# file from the RUNDIR directory. date >! ${RUNDIR}/cam_inflation_cookie @@ -313,7 +316,7 @@ if ( $status == 0 ) then # If you want to simply run CESM, use 'no_assimilate.csh' - # If you want to use DART , use 'DARTSCRIPTNAME' + # If you want to use DART, use 'DARTSCRIPTNAME' # You can only use one of the two. ${CASEROOT}/no_assimilate.csh @@ -393,7 +396,7 @@ cat << EndOfText >! DART_instructions.txt ------------------------------------------------------------- +------------------------------------------------------------------------- Check the DART configuration: @@ -403,21 +406,23 @@ 2) When you want to run DART, edit the ${CASE}.run script to reference "$DARTscript" instead of "no_assimilate.csh". -3) Modify what you need to in the DART namelists. These are - ${CASEROOT}/input.nml +3) Modify what you need to in the DART namelist file, i.e. ${CASEROOT}/input.nml 4) If you have recompiled any part of the DART system, 'refresh_dart_files' will copy them into the correct places. -5) Make sure the observation directory names in $DARTscript match - those on your system, and submit the CESM job in the normal way. +5) If you stage your own inflation files, make sure you read the INFLATION section + in ${CASEROOT}/CESM_DART_config -6) Inflation files? Observation files for the time of interest? +6) Make sure the observation directory names in $DARTscript match + those on your system. -7) You can use ${CASEROOT}/stage_cesm_files +7) Submit the CESM job in the normal way. + +8) You can use ${CASEROOT}/stage_cesm_files to stage files to restart a run. ------------------------------------------------------------- +------------------------------------------------------------------------- EndOfText Modified: DART/trunk/models/cam/shell_scripts/assimilate.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/assimilate.csh 2013-10-23 17:29:54 UTC (rev 6541) +++ DART/trunk/models/cam/shell_scripts/assimilate.csh 2013-10-23 23:02:43 UTC (rev 6542) @@ -16,7 +16,7 @@ # The FORCE options are not optional. # The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following +# some systems don't like the -v option to any of the following switch ("`hostname`") case be*: # NCAR "bluefire" @@ -36,6 +36,9 @@ set LINK = 'ln -fvs' set REMOVE = 'rm -fr' + # BASEOBSDIR needs the whole path name except for the date directory and file name. + # You can set this to point to directories of synthetic observations, too. + # set BASEOBSDIR = /glade/p/image/Observations/Synthetic/UVT_ne30_12H set BASEOBSDIR = /glade/p/image/Observations/ACARS set LAUNCHCMD = mpirun.lsf breaksw @@ -94,6 +97,7 @@ # Get observation sequence file ... or die right away. # The observation file names have a time that matches the stopping time of CAM. #----------------------------------------------------------------------------- +# Make sure the file name structure matches the obs you will be using. set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` set OBSFNAME = `printf obs_seq%04d%02d%02d%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY} ${ATM_HOUR}` @@ -127,7 +131,8 @@ # Block 2: Stage the files needed for SAMPLING ERROR CORRECTION # # The sampling error correction is a lookup table. -# The tables are stored in the DART distribution. +# The tables were originally in the DART distribution, but should +# have been staged to $CASEROOT at setup time. # Each ensemble size has its own (static) file. # It is only needed if # input.nml:&assim_tools_nml:sampling_error_correction = .true., @@ -170,24 +175,28 @@ # files to be as listed above. When being archived, the filenames get a # unique extension (describing the assimilation time) appended to them. # -# The inflation file is essentially a duplicate of the DART model state ... -# For the purpose of this script, they are the output of a previous assimilation, +# The inflation file is essentially a duplicate of the DART model state ... +# For the purpose of this script, they are the output of a previous assimilation, # so they should be named something like prior_inflate_restart.YYYY-MM-DD-SSSSS # # NOTICE: inf_initial_from_restart and inf_sd_initial_from_restart are somewhat # problematic. During the bulk of an experiment, these should be FALSE, since # we want to read existing inflation files. However, the first assimilation # might need these to be TRUE and then subsequently be set to FALSE. -# There are two ways to handle this. -# 1) Create the initial files offline with values of unity by using -# 'fill_inflation_restart' and stage them with the appropriate names -# in the RUNDIR. -# 2) create a cookie file called RUNDIR/make_cam_inflation_cookie +# There are two ways to handle this: +# +# 1) Create the initial files offline (perhaps with 'fill_inflation_restart') +# and stage them with the appropriate names in the RUNDIR. +# You must manually remove the cam_inflation_cookie file +# from the RUNDIR in this case. +# - OR - +# 2) create a cookie file called RUNDIR/cam_inflation_cookie # The existence of this file will cause this script to set the # namelist appropriately. This script will 'eat' the cookie file # to prevent this from happening for subsequent executions. If the # inflation file does not exist for them, and it needs to, this script -# should die. +# should die. The CESM_DART_config script automatically creates a cookie +# file to support this option. # # The strategy is to use the LATEST inflation file from the CESM 'rundir'. # After an assimilation, the new inflation values/files will be moved to @@ -234,14 +243,14 @@ # we are not using an existing inflation file. echo "inf_flavor(1) = $PRIOR_INF, using namelist values." - else if ( -e ../make_cam_inflation_cookie ) then + else if ( -e ../cam_inflation_cookie ) then # We want to use an existing inflation file, but this is # the first assimilation so there is no existing inflation - # file. This is the signal we need to to coerce the namelist + # file. This is the signal we need to to coerce the namelist # to have different values for this execution ONLY. # Since the local namelist comes from CASEROOT each time, we're golden. - set PRIOR_TF = FALSE + set PRIOR_TF = FALSE ex input.nml <! latestfile) > & /dev/null set nfiles = `cat latestfile | wc -l` @@ -277,14 +286,14 @@ # we are not using an existing inflation file. echo "inf_flavor(2) = $POSTE_INF, using namelist values." - else if ( -e ../make_cam_inflation_cookie ) then + else if ( -e ../cam_inflation_cookie ) then # We want to use an existing inflation file, but this is # the first assimilation so there is no existing inflation - # file. This is the signal we need to to coerce the namelist + # file. This is the signal we need to to coerce the namelist # to have different values for this execution ONLY. # Since the local namelist comes from CASEROOT each time, we're golden. - set POSTE_TF = FALSE + set POSTE_TF = FALSE ex input.nml <! input.nml + sed -e "s#dart_ics#../${DART_IC_FILENAME}#" \ + -e "s#dart_restart#../${DART_RESTART_FILE}#" < ../input.nml >! input.nml ${LINK} $ATM_INITIAL_FILENAME caminput.nc ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc @@ -385,19 +408,18 @@ # Will result in a set of files : 'filter_restart.xxxx' # # DART namelist settings required: -# &filter_nml: async = 0, -# &filter_nml: adv_ens_command = "./no_model_advance.csh", -# &filter_nml: restart_in_file_name = 'filter_ics' -# &filter_nml: restart_out_file_name = 'filter_restart' -# &filter_nml: obs_sequence_in_name = 'obs_seq.out' -# &filter_nml: obs_sequence_out_name = 'obs_seq.final' -# &filter_nml: init_time_days = -1, -# &filter_nml: init_time_seconds = -1, -# &filter_nml: first_obs_days = -1, -# &filter_nml: first_obs_seconds = -1, -# &filter_nml: last_obs_days = -1, -# &filter_nml: last_obs_seconds = -1, -# &ensemble_manager_nml: single_restart_file_in = .false. +# &filter_nml: restart_in_file_name = 'filter_ics' +# &filter_nml: restart_out_file_name = 'filter_restart' +# &filter_nml: obs_sequence_in_name = 'obs_seq.out' +# &filter_nml: obs_sequence_out_name = 'obs_seq.final' +# &filter_nml: init_time_days = -1, +# &filter_nml: init_time_seconds = -1, +# &filter_nml: first_obs_days = -1, +# &filter_nml: first_obs_seconds = -1, +# &filter_nml: last_obs_days = -1, +# &filter_nml: last_obs_seconds = -1, +# &ensemble_manager_nml: single_restart_file_in = .false. +# &ensemble_manager_nml: single_restart_file_out = .false. # #========================================================================= @@ -410,23 +432,10 @@ ${LINK} $ATM_INITIAL_FILENAME caminput.nc ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc -# On yellowstone, you can explore task layouts with the following: -if ( $?LSB_PJL_TASK_GEOMETRY ) then - setenv ORIGINAL_LAYOUT "${LSB_PJL_TASK_GEOMETRY}" - - # setenv GEOMETRY_32_1NODE \ - # "{(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31)}"; - # setenv LSB_PJL_TASK_GEOMETRY "${GEOMETRY_32_1NODE}" -endif - echo "`date` -- BEGIN FILTER" ${LAUNCHCMD} ${EXEROOT}/filter || exit -7 echo "`date` -- END FILTER" -if ( $?LSB_PJL_TASK_GEOMETRY ) then - setenv LSB_PJL_TASK_GEOMETRY "${ORIGINAL_LAYOUT}" -endif - ${MOVE} Prior_Diag.nc ../cam_Prior_Diag.${ATM_DATE_EXT}.nc ${MOVE} Posterior_Diag.nc ../cam_Posterior_Diag.${ATM_DATE_EXT}.nc ${MOVE} obs_seq.final ../cam_obs_seq.${ATM_DATE_EXT}.final @@ -434,7 +443,7 @@ # Accomodate any possible inflation files # 1) rename file to reflect current date -# 2) move to CENTRALDIR so the DART INFLATION BLOCK works next time and +# 2) move to RUNDIR so the DART INFLATION BLOCK works next time and # that they can get archived. foreach FILE ( ${PRIOR_INF_OFNAME} ${POSTE_INF_OFNAME} ${PRIOR_INF_DIAG} ${POSTE_INF_DIAG} ) @@ -496,47 +505,17 @@ set member = 1 while ( ${member} <= ${ensemble_size} ) - set n4 = `printf %04d $member` + set inst_string = `printf _%04d $member` - set LND_RESTART_FILENAME = `printf ${MYCASE}.clm2_%04d.r.${ATM_DATE_EXT}.nc ${member}` - set ICE_RESTART_FILENAME = `printf ${MYCASE}.cice_%04d.r.${ATM_DATE_EXT}.nc ${member}` - set ATM_INITIAL_FILENAME = `printf ${MYCASE}.cam_%04d.i.${ATM_DATE_EXT}.nc ${member}` + set ATM_INITIAL_FILENAME = ${MYCASE}.cam${inst_string}.i.${ATM_DATE_EXT}.nc - ${LINK} ${LND_RESTART_FILENAME} clm_restart_${n4}.nc || exit -9 - ${LINK} ${ICE_RESTART_FILENAME} ice_restart_${n4}.nc || exit -9 - ${LINK} ${ATM_INITIAL_FILENAME} cam_initial_${n4}.nc || exit -9 + ${LINK} ${ATM_INITIAL_FILENAME} cam_initial${inst_string}.nc || exit -9 @ member++ end #------------------------------------------------------------------------- -# We have to communicate the current model time to the env_run.xml script -# CAM is running in a 'perpetual startup' mode, so it constantly needs new -# start time information. The other models are running 'restart', so they -# keep track of it internally. -#------------------------------------------------------------------------- - -cd ${CASEROOT} - -set YYYYMMDD = `printf %04d-%02d-%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY}` -set SSSSS = `printf %05d ${ATM_SECONDS}` - -./xmlchange RUN_STARTDATE=${YYYYMMDD} -./xmlchange START_TOD=${SSSSS} - -#------------------------------------------------------------------------- -# we (DART) do not need these files, and CESM does not need them either -# to continue a run. if we remove them here they do not get moved to -# the short-term archiver. -#------------------------------------------------------------------------- - -${REMOVE} ${RUNDIR}/*.rs.* -${REMOVE} ${RUNDIR}/*.rh0.* -${REMOVE} ${RUNDIR}/*.rs1.* -${REMOVE} ${RUNDIR}/PET*ESMF_LogFile - -#------------------------------------------------------------------------- # Cleanup #------------------------------------------------------------------------- From nancy at ucar.edu Thu Oct 24 11:29:55 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 24 Oct 2013 11:29:55 -0600 Subject: [Dart-dev] [6547] DART/trunk/models/cam/shell_scripts: There is a bugfix from cesm1_2_0 that can be used to support truly EMPTY Message-ID: Revision: 6547 Author: thoar Date: 2013-10-24 11:29:54 -0600 (Thu, 24 Oct 2013) Log Message: ----------- There is a bugfix from cesm1_2_0 that can be used to support truly EMPTY history tapes from CLM. Removing the comment/explanation and referencing the tar file that supports the bugfix: http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-24 16:08:21 UTC (rev 6546) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-24 17:29:54 UTC (rev 6547) @@ -519,7 +519,7 @@ echo "ERROR - No SourceMods for this case." echo "DART requires modifications to several src files." echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar" echo "untar these into your HOME directory - they will create a" echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." exit -4 @@ -623,17 +623,6 @@ # echo "hist_mfilt = 1" >> $fname # echo "hist_nhtfrq = 0" >> $fname -# KDR; Was the ntapes bug fixed? -# That's where the CLM restart file always has dimension ntapes, -# but it's set = 0 for hist_empty_htapes = .true. and empty hist_fincl1, -# which prevents the next forecast from starting. -# Kluzek fixed it in some recent (<9/2013) version, but not 1_1_1. -# There's a supposed fix in ~/cesm1_1_1/SourceMods/src.clm/HistFileMod.F90, -# but it only checks whether ntapes = ntapes_on_file, and is_restart() returning 'true'. -# It doesn't check whether the size is 0. -# That solution can't work in hybrid mode. See ~/Homme/notes:9/17/2013 -# KDR - echo "hist_empty_htapes = .true." >> $fname echo "hist_fincl1 = 'TSA'" >> $fname echo "hist_nhtfrq = -$stop_n" >> $fname Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-24 16:08:21 UTC (rev 6546) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-24 17:29:54 UTC (rev 6547) @@ -517,7 +517,7 @@ echo "ERROR - No SourceMods for this case." echo "DART requires modifications to several src files." echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar" echo "untar these into your HOME directory - they will create a" echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." exit -4 @@ -621,17 +621,6 @@ # echo "hist_mfilt = 1" >> $fname # echo "hist_nhtfrq = 0" >> $fname -# KDR; Was the ntapes bug fixed? -# That's where the CLM restart file always has dimension ntapes, -# but it's set = 0 for hist_empty_htapes = .false., -# which prevents the next forecast from starting. -# Kluzek fixed it in some recent (<9/2013) version, but not 1_1_1. -# There's a supposed fix in ~/cesm1_1_1/SourceMods/src.clm/HistFileMod.F90, -# but it only checks whether ntapes = ntapes_on_file, and is_restart() returning 'true'. -# It doesn't check whether the size is 0. -# That solution can't work in hybrid mode. See ~/Homme/notes:9/17/2013 -# KDR - echo "hist_empty_htapes = .true." >> $fname echo "hist_fincl1 = 'TSA'" >> $fname echo "hist_nhtfrq = -$stop_n" >> $fname From nancy at ucar.edu Thu Oct 24 16:09:03 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 24 Oct 2013 16:09:03 -0600 Subject: [Dart-dev] [6548] DART/trunk/models/cam/shell_scripts: The user_nl_cam sections now support the use of prescribed aerosols, Message-ID: Revision: 6548 Author: thoar Date: 2013-10-24 16:09:02 -0600 (Thu, 24 Oct 2013) Log Message: ----------- The user_nl_cam sections now support the use of prescribed aerosols, and are able to run through Dec 2008. FYI: it is still REQUIRED to create a (compact) CLM history file, which these scripts do. The previous commit solved one bug, but caused another. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-24 17:29:54 UTC (rev 6547) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-24 22:09:02 UTC (rev 6548) @@ -452,6 +452,7 @@ ./xmlchange RTM_MODE='NULL' # level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) + ./xmlchange DEBUG=FALSE ./xmlchange INFO_DBUG=0 @@ -587,22 +588,46 @@ echo " nhtfrq = -$stop_n " >> ${fname} echo " mfilt = 1 " >> ${fname} - # Files specified below are because the 'default' files only - # contain data through 2005 and we may be interested in timeframes after that. + # Some of the files specified here are because the default files only + # contain data through 2005 and we are interested in timeframes after that. + # CAM5 does prognostic aerosols by default (which is expensive). + # Specifying the following causes CAM to use prescribed aerosols, which is cheaper. + # Prescribed aerosols on the timescales of interest to assimilation are acceptable. + set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" - # CAM5 does prognostic aerosols by default. If you want to prescribe them, - # use the following variables with your own settings ... - # Filenames with 'clim' in their names should specify aerodep_flx_type = 'CYCLICAL'. - # Other values are 'CYCLICAL', 'SERIAL', 'INTERP_MISSING_MONTHS', or 'FIXED'. + echo " bndtvghg = '${cesm_datadir}/atm/cam/ggas/ghg_hist_1765-2009_c100902.nc'" >> ${fname} + echo " prescribed_ozone_file = 'ozone_1.9x2.5_L26_1850-2015_rcp45_c101108.nc'" >> ${fname} + echo " tracer_cnst_file = 'oxid_1.9x2.5_L26_1850-2015_rcp45_c101108.nc'" >> ${fname} - echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} - echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} - echo "aerodep_flx_type = 'CYCLICAL' " >> ${fname} - # Set if aerodep_flx_type = 'CYCLICAL' - echo "aerodep_flx_cycle_yr = 2000 " >> ${fname} + echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} + echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} + echo "aerodep_flx_type = 'CYCLICAL'" >> ${fname} + echo "aerodep_flx_cycle_yr = 2000" >> ${fname} + echo " ext_frc_specifier = " >> ${fname} + echo " 'SO2 -> ${chem_datapath}/emis/ar5_mam3_so2_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'bc_a1 -> ${chem_datapath}/emis/ar5_mam3_bc_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a1 -> ${chem_datapath}/emis/ar5_mam3_num_a1_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a2 -> ${chem_datapath}/emis/ar5_mam3_num_a2_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'pom_a1 -> ${chem_datapath}/emis/ar5_mam3_oc_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a1 -> ${chem_datapath}/emis/ar5_mam3_so4_a1_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a2 -> ${chem_datapath}/emis/ar5_mam3_so4_a2_elev_1850-2010_c20100902_v12.nc'" >> ${fname} + + echo " srf_emis_specifier = " >> ${fname} + echo " 'DMS -> ${chem_datapath}/emis/aerocom_mam3_dms_surf_1849-2010_c20100902.nc'," >> ${fname} + echo " 'SO2 -> ${chem_datapath}/emis/ar5_mam3_so2_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'SOAG -> ${chem_datapath}/emis/ar5_mam3_soag_1.5_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'bc_a1 -> ${chem_datapath}/emis/ar5_mam3_bc_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a1 -> ${chem_datapath}/emis/ar5_mam3_num_a1_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a2 -> ${chem_datapath}/emis/ar5_mam3_num_a2_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'pom_a1 -> ${chem_datapath}/emis/ar5_mam3_oc_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a1 -> ${chem_datapath}/emis/ar5_mam3_so4_a1_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a2 -> ${chem_datapath}/emis/ar5_mam3_so4_a2_surf_1850-2010_c20100902_v12.nc'" >> ${fname} + + echo " solar_data_file = '${cesm_datadir}/atm/cam/solar/spectral_irradiance_Lean_1610-2009_ann_c100405.nc'" >> ${fname} + # =========================================================================== set fname = "user_nl_clm${inst_string}" # =========================================================================== @@ -617,24 +642,24 @@ # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf # # Every 6 hours - # echo "hist_mfilt = 1" >> $fname - # echo "hist_nhtfrq = -$stop_n" >> $fname + # echo "hist_mfilt = 1" >> ${fname} + # echo "hist_nhtfrq = -$stop_n" >> ${fname} # Every month - # echo "hist_mfilt = 1" >> $fname - # echo "hist_nhtfrq = 0" >> $fname + # echo "hist_mfilt = 1" >> ${fname} + # echo "hist_nhtfrq = 0" >> ${fname} - echo "hist_empty_htapes = .true." >> $fname - echo "hist_fincl1 = 'TSA'" >> $fname - echo "hist_nhtfrq = -$stop_n" >> $fname - echo "hist_mfilt = 1" >> $fname - echo "hist_avgflag_pertape = 'A'" >> $fname + echo "hist_empty_htapes = .true." >> ${fname} + echo "hist_fincl1 = 'TSA'" >> ${fname} + echo "hist_nhtfrq = -$stop_n" >> ${fname} + echo "hist_mfilt = 1" >> ${fname} + echo "hist_avgflag_pertape = 'A'" >> ${fname} # =========================================================================== set fname = "user_nl_cice${inst_string}" # =========================================================================== # CICE Namelists - echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname + echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> ${fname} @ inst ++ end Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-24 17:29:54 UTC (rev 6547) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-24 22:09:02 UTC (rev 6548) @@ -102,9 +102,10 @@ # define machines and directories # # mach Computer name -# cesmroot Location of the cesm code base +# cesm_datadir Location of some supporting CESM data files. +# cesmroot Location of the CESM code base # For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# caseroot Your (future) CESM case directory, where this CESM+DART will be built. # Preferably not a frequently scrubbed location. # This script will delete any existing caseroot, so this script, # and other useful things should be kept elsewhere. @@ -208,7 +209,7 @@ setenv ACCOUNT P8685xxxx setenv queue economy -setenv timewall 0:20 +setenv timewall 0:10 # ============================================================================== # set these standard commands based on the machine you are running on. @@ -421,11 +422,6 @@ ./xmlchange NCPL_BASE_PERIOD=day ./xmlchange ATM_NCPL=48 -# TJH FIXME ... any reason to turn ON the biogeochemistry in CLM when atmos only? -# There is no compset that has everything we want, -# so we are also turning on the CLM biogeochemistry. -#./xmlchange CLM_CONFIG_OPTS='-bgc cn' - if ($short_term_archiver == 'off') then ./xmlchange DOUT_S=FALSE else @@ -497,7 +493,7 @@ endsw # This is the part that modifies the run script to allow CESM to advance -# correctly given the modifications necessary for CAM. It also copies several +# correctly given the modifications of CAM for DART. It also copies several # required DART files to the caseroot directory. ./CESM_DART_config @@ -548,17 +544,17 @@ endif # ============================================================================== -# Modify namelist templates for each instance. This is a bit of a nuisance in -# that we are pulling in restart and initial files from 'all over the place' -# and each model component has a different strategy. +# Modify namelist templates for each instance. +# CAM SourceMods force CAM to ALWAYS read INITIAL files specified by namelist var 'ncdata' # -# In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): +# When CONTINUE_RUN = FALSE (i.e. just starting up): # -# CAM has been forced to read initial files - specified by namelist var:ncdata # CICE reads from namelist variable 'ice_ic' # CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 # it does not use the instance string. There is a patch for clm.buildnml.csh # +# When CONTINUE_RUN = TRUE, CICE and CLM read from pointer files. +# # All of these must later on be staged with these same filenames. # OR - all these namelists can be changed to match whatever has been staged. # MAKE SURE THE STAGING SECTION OF THIS SCRIPT MATCHES THESE VALUES. @@ -585,22 +581,46 @@ echo " nhtfrq = -$stop_n " >> ${fname} echo " mfilt = 1 " >> ${fname} - # A lot of the files specified here are because the 'default' files only + # Some of the files specified here are because the default files only # contain data through 2005 and we are interested in timeframes after that. + # CAM5 does prognostic aerosols by default (which is expensive). + # Specifying the following causes CAM to use prescribed aerosols, which is cheaper. + # Prescribed aerosols on the timescales of interest to assimilation are acceptable. + set chem_datapath = "${cesm_datadir}/atm/cam/chem/trop_mozart_aero" - # CAM5 does prognostic aerosols by default. If you want to prescribe them, - # use the following variables with your own settings ... - # Filenames with 'clim' in their names should specify aerodep_flx_type = 'CYCLICAL'. - # Other values are 'CYCLICAL', 'SERIAL', 'INTERP_MISSING_MONTHS', or 'FIXED'. + echo " bndtvghg = '${cesm_datadir}/atm/cam/ggas/ghg_hist_1765-2009_c100902.nc'" >> ${fname} + echo " prescribed_ozone_file = 'ozone_1.9x2.5_L26_1850-2015_rcp45_c101108.nc'" >> ${fname} + echo " tracer_cnst_file = 'oxid_1.9x2.5_L26_1850-2015_rcp45_c101108.nc'" >> ${fname} - echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} - echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} - echo "aerodep_flx_type = 'CYCLICAL' " >> ${fname} - # Set if aerodep_flx_type = 'CYCLICAL' - echo "aerodep_flx_cycle_yr = 2000 " >> ${fname} + echo "aerodep_flx_datapath = '${chem_datapath}/aero'" >> ${fname} + echo "aerodep_flx_file = 'aerosoldep_rcp4.5_monthly_1849-2104_0.9x1.25_c100407.nc'" >> ${fname} + echo "aerodep_flx_type = 'CYCLICAL'" >> ${fname} + echo "aerodep_flx_cycle_yr = 2000" >> ${fname} + echo " ext_frc_specifier = " >> ${fname} + echo " 'SO2 -> ${chem_datapath}/emis/ar5_mam3_so2_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'bc_a1 -> ${chem_datapath}/emis/ar5_mam3_bc_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a1 -> ${chem_datapath}/emis/ar5_mam3_num_a1_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a2 -> ${chem_datapath}/emis/ar5_mam3_num_a2_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'pom_a1 -> ${chem_datapath}/emis/ar5_mam3_oc_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a1 -> ${chem_datapath}/emis/ar5_mam3_so4_a1_elev_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a2 -> ${chem_datapath}/emis/ar5_mam3_so4_a2_elev_1850-2010_c20100902_v12.nc'" >> ${fname} + + echo " srf_emis_specifier = " >> ${fname} + echo " 'DMS -> ${chem_datapath}/emis/aerocom_mam3_dms_surf_1849-2010_c20100902.nc'," >> ${fname} + echo " 'SO2 -> ${chem_datapath}/emis/ar5_mam3_so2_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'SOAG -> ${chem_datapath}/emis/ar5_mam3_soag_1.5_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'bc_a1 -> ${chem_datapath}/emis/ar5_mam3_bc_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a1 -> ${chem_datapath}/emis/ar5_mam3_num_a1_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'num_a2 -> ${chem_datapath}/emis/ar5_mam3_num_a2_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'pom_a1 -> ${chem_datapath}/emis/ar5_mam3_oc_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a1 -> ${chem_datapath}/emis/ar5_mam3_so4_a1_surf_1850-2010_c20100902_v12.nc'," >> ${fname} + echo " 'so4_a2 -> ${chem_datapath}/emis/ar5_mam3_so4_a2_surf_1850-2010_c20100902_v12.nc'" >> ${fname} + + echo " solar_data_file = '${cesm_datadir}/atm/cam/solar/spectral_irradiance_Lean_1610-2009_ann_c100405.nc'" >> ${fname} + # =========================================================================== set fname = "user_nl_clm${inst_string}" # =========================================================================== @@ -615,24 +635,24 @@ # http://www.cesm.ucar.edu/models/cesm1.1/clm/models/lnd/clm/doc/UsersGuide/clm_ug.pdf # # Every 6 hours - # echo "hist_mfilt = 1" >> $fname - # echo "hist_nhtfrq = -$stop_n" >> $fname + # echo "hist_mfilt = 1" >> ${fname} + # echo "hist_nhtfrq = -$stop_n" >> ${fname} # Every month - # echo "hist_mfilt = 1" >> $fname - # echo "hist_nhtfrq = 0" >> $fname + # echo "hist_mfilt = 1" >> ${fname} + # echo "hist_nhtfrq = 0" >> ${fname} - echo "hist_empty_htapes = .true." >> $fname - echo "hist_fincl1 = 'TSA'" >> $fname - echo "hist_nhtfrq = -$stop_n" >> $fname - echo "hist_mfilt = 1" >> $fname - echo "hist_avgflag_pertape = 'A'" >> $fname + echo "hist_empty_htapes = .true." >> ${fname} + echo "hist_fincl1 = 'TSA'" >> ${fname} + echo "hist_nhtfrq = -$stop_n" >> ${fname} + echo "hist_mfilt = 1" >> ${fname} + echo "hist_avgflag_pertape = 'A'" >> ${fname} # =========================================================================== set fname = "user_nl_cice${inst_string}" # =========================================================================== # CICE Namelists - echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> $fname + echo "ice_ic = '${run_refcase}.cice${inst_string}.r.${run_refdate}-${run_reftod}.nc'" >> ${fname} @ inst ++ end @@ -679,7 +699,7 @@ # the restarts should be in the short term archive directory # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back - ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . || exit -1 else @@ -796,6 +816,7 @@ echo "" echo "3) If you want to simply advance CESM without using DART, make sure the" echo " ${case}.run script references the 'no_assimilate.csh' script." +echo " This is the default behavior." echo "" echo "4) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" @@ -811,7 +832,6 @@ echo "" echo " and" echo " ./${case}.submit" -echo "" cat ${caseroot}/DART_instructions.txt From nancy at ucar.edu Thu Oct 24 17:15:47 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 24 Oct 2013 17:15:47 -0600 Subject: [Dart-dev] [6549] DART/trunk/models/cam/shell_scripts: Put the xmlchanges to turn off the RTM in a more logical spot. Message-ID: Revision: 6549 Author: thoar Date: 2013-10-24 17:15:47 -0600 (Thu, 24 Oct 2013) Log Message: ----------- Put the xmlchanges to turn off the RTM in a more logical spot. Fixed an error in the stage_cesm_files script that put an extra lines in the ocean pointer files. In actuality, the ocean pointer files created by CESM are just REALLY LONG, so it looked like there were 4 lines when in fact there were only two. Modified Paths: -------------- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo -------------- next part -------------- Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-24 22:09:02 UTC (rev 6548) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-24 23:15:47 UTC (rev 6549) @@ -16,8 +16,8 @@ # # DOCN: We are using a single data ocean. # -# Because the atmosphere assimilations typically occur every 6 hours, -# the methodology here reflects that. All of CESM stops every 6 hours +# Because the atmosphere assimilations typically occur every 6 hours, +# the methodology here reflects that. All of CESM stops every 6 hours # so that a CAM output file would be available for assimilation. # # This script results in a viable setup for a CESM multi-instance experiment. You @@ -27,13 +27,13 @@ # capability to re-stage files in your RUN directory # # CESM/DART requires some modifications to the CESM source code EVEN IF YOU ARE NOT -# assimilating. The modifications for CAM require a modification to the -# CESM ${CASE}.run script. The modifications will invoke a DART script that will -# either assimilate or 'do nothing'. It is strongly recommended that you configure +# assimilating. The modifications for CAM require a modification to the +# CESM ${CASE}.run script. The modifications will invoke a DART script that will +# either assimilate or 'do nothing'. It is strongly recommended that you configure # the ${CASE}.run script to 'do nothing' initially. This is the default behavior. # # CASEROOT/CESM_DART_config will augment the CESM case with the required setup -# and configuration to perform a DART assimilation and modify the ${CASE}.run script +# and configuration to perform a DART assimilation and modify the ${CASE}.run script # as mentioned previously - after it makes a backup copy. # # This script relies heavily on the information in: @@ -58,7 +58,7 @@ # 2) a run directory, where each forecast (and assimilation) will take place, # 3) a bld directory for the executables. # -# This script also executes ${CASEROOT}/CESM_DART_config to modify +# This script also executes ${CASEROOT}/CESM_DART_config to modify # the $CASEROOT/${CASE}.run script so that the SourceMods for CAM # are effective. CESM_DART_config will also augment the case with all # the pieces necessary to run DART when the time comes. @@ -421,6 +421,13 @@ ./xmlchange PIO_TYPENAME=pnetcdf +# The river transport model ON is useful only when using an active ocean or +# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. +# If you turn it ON, you will have to stage initial files etc. + +./xmlchange ROF_GRID='null' +./xmlchange RTM_MODE='NULL' + # COUPLING discussion. F compsets are 'tight' coupling. # Only change the ATM_NCPL ... everything is based on this one value, # including CAM physics and dynamics timesteps. @@ -444,13 +451,6 @@ ./xmlchange DOUT_L_HTAR=FALSE endif -# The river transport model ON is useful only when using an active ocean or -# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. -# If you turn it ON, you will have to stage initial files etc. - -./xmlchange ROF_GRID='null' -./xmlchange RTM_MODE='NULL' - # level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) ./xmlchange DEBUG=FALSE @@ -560,7 +560,7 @@ # CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 # it does not use the instance string. There is a patch for clm.buildnml.csh # -# When CONTINUE_RUN = TRUE, CICE and CLM read from pointer files. +# When CONTINUE_RUN = TRUE, CICE and CLM read from pointer files. # # All of these must later on be staged with these same filenames. # OR - all these namelists can be changed to match whatever has been staged. @@ -739,9 +739,7 @@ # relink the pointer file for the data ocean echo "${case}.docn.r.\${restart_time}.nc" >! rpointer.ocn - echo "" >> rpointer.ocn echo "${case}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn - echo "" >> rpointer.ocn exit 0 endif @@ -770,8 +768,8 @@ echo '' echo "Staging restarts for instance \$inst of $num_instances" - ${LINK} \${filedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . - ${LINK} \${filedir}/${run_refcase}.cice\${inst_string}.r.${run_refdate}-${run_reftod}.nc . + ${LINK} \${filedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . + ${LINK} \${filedir}/${run_refcase}.cice\${inst_string}.r.${run_refdate}-${run_reftod}.nc . ${LINK} \${filedir}/${run_refcase}.cam\${inst_string}.i.${run_refdate}-${run_reftod}.nc cam_initial\${inst_string}.nc @ inst ++ Modified: DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-24 22:09:02 UTC (rev 6548) +++ DART/trunk/models/cam/shell_scripts/CESM1_1_1_setup_pmo 2013-10-24 23:15:47 UTC (rev 6549) @@ -414,6 +414,13 @@ ./xmlchange PIO_TYPENAME=pnetcdf +# The river transport model ON is useful only when using an active ocean or +# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. +# If you turn it ON, you will have to stage initial files etc. + +./xmlchange ROF_GRID='null' +./xmlchange RTM_MODE='NULL' + # COUPLING discussion. F compsets are 'tight' coupling. # Only change the ATM_NCPL ... everything is based on this one value, # including CAM physics and dynamics timesteps. @@ -437,13 +444,6 @@ ./xmlchange DOUT_L_HTAR=FALSE endif -# The river transport model ON is useful only when using an active ocean or -# land surface diagnostics. Setting ROF_GRID, RTM_MODE to 'null' turns off the RTM. -# If you turn it ON, you will have to stage initial files etc. - -./xmlchange ROF_GRID='null' -./xmlchange RTM_MODE='NULL' - # level of debug output, 0=minimum, 1=normal, 2=more, 3=too much, valid values: 0,1,2,3 (integer) ./xmlchange DEBUG=FALSE @@ -553,7 +553,7 @@ # CLM builds its own 'finidat' value from the REFCASE variables but in CESM1_1_1 # it does not use the instance string. There is a patch for clm.buildnml.csh # -# When CONTINUE_RUN = TRUE, CICE and CLM read from pointer files. +# When CONTINUE_RUN = TRUE, CICE and CLM read from pointer files. # # All of these must later on be staged with these same filenames. # OR - all these namelists can be changed to match whatever has been staged. @@ -732,9 +732,7 @@ # relink the pointer file for the data ocean echo "${case}.docn.r.\${restart_time}.nc" >! rpointer.ocn - echo "" >> rpointer.ocn echo "${case}.docn.rs1.\${restart_time}.bin" >> rpointer.ocn - echo "" >> rpointer.ocn exit 0 endif From nancy at ucar.edu Fri Oct 25 14:46:25 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Fri, 25 Oct 2013 14:46:25 -0600 Subject: [Dart-dev] [6551] DART/trunk/models/clm/shell_scripts: Better staging scripts and more accurate comments. Message-ID: Revision: 6551 Author: thoar Date: 2013-10-25 14:46:24 -0600 (Fri, 25 Oct 2013) Log Message: ----------- Better staging scripts and more accurate comments. Streamlined perfect_model setup and execution. Modified Paths: -------------- DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_pmo DART/trunk/models/clm/shell_scripts/CESM_DART_config DART/trunk/models/clm/shell_scripts/perfect_model.csh -------------- next part -------------- Modified: DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-25 20:46:12 UTC (rev 6550) +++ DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-25 20:46:24 UTC (rev 6551) @@ -102,7 +102,7 @@ # the value of "case" will be used many ways; # directory and file names, both locally and on HPSS, and -# script names; so consider it's length and information content. +# script names; so consider its length and information content. # num_instances: Number of ensemble members setenv case clm_hybrid @@ -117,7 +117,7 @@ # mach Computer name # cesmroot Location of the cesm code base # For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# caseroot Your (future) CESM case directory, where this CESM+DART will be built. # Preferably not a frequently scrubbed location. # This script will delete any existing caseroot, so this script, # and other useful things should be kept elsewhere. @@ -130,6 +130,7 @@ # ============================================================================== setenv mach yellowstone + setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag setenv caseroot /glade/p/work/${USER}/cases/${case} setenv exeroot /glade/scratch/${USER}/${case}/bld @@ -169,7 +170,7 @@ # this configuration step. The run-time settings can lie. # # If the long-term archiver is off, you get a chance to examine the files before -# they get moved to long-term storage. You can always submit $CASE.l_archive +# they get moved to long-term storage. You can always submit $case.l_archive # whenever you want to free up space in the short-term archive directory. # # ============================================================================== @@ -245,8 +246,8 @@ # ============================================================================== # Create the case - this creates the CASEROOT directory. # -# For list of the pre-defined cases: ./create_newcase -list -# To create a variant case, see the CESM documentation and carefully +# For list of the pre-defined component sets: ./create_newcase -list +# To create a variant compset, see the CESM documentation and carefully # incorporate any needed changes into this script. # ============================================================================== @@ -408,6 +409,7 @@ ./xmlchange DATM_CPLHIST_YR_END=$refyear ./xmlchange CALENDAR=GREGORIAN + ./xmlchange STOP_OPTION=$stop_option ./xmlchange STOP_N=$stop_n ./xmlchange CONTINUE_RUN=FALSE @@ -500,7 +502,7 @@ echo "ERROR - No SourceMods for this case." echo "DART requires modifications to several src files." echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar" echo "untar these into your HOME directory - they will create a" echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." exit -4 @@ -531,9 +533,7 @@ endif # ============================================================================== -# Modify namelist templates for each instance. This is a bit of a nuisance in -# that we are pulling in restart and initial files from 'all over the place' -# and each model component has a different strategy. +# Modify namelist templates for each instance. # # In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): # @@ -683,41 +683,90 @@ # ============================================================================== # Stage the restarts now that the run directory exists # THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. -# POP/CAM/CICE read from pointer files. The others use namelist values initially. # ============================================================================== -cat << EndOfText >! stage_initial_cesm_files -#!/bin/sh +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. -cd ${rundir} - +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' echo '' -echo 'Copying only the required restart files from the staging directory.' -echo 'With CONTINUE_RUN=FALSE, there are many that are not required.' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' echo '' -# This copies over only the minimum number of instances. -# The reference case may more instances than we need, -# and it certainly has more files than we need. +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} -let inst=1 -while ((\$inst <= $num_instances)); do - inst_string=\`printf _%04d \$inst\` +set restart_time = $RUN_REFDATE-$RUN_REFTOD - echo '' - echo "Staging restarts for instance \$inst of $num_instances" +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- - ${COPY} ${CLM_stagedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . +echo "continue_run is \$CONTINUE_RUN" - let inst+=1 -done +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + # the restarts should be in the short term archive directory + # www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . || exit -1 + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + + @ inst=1 + while (\$inst <= $num_instances) + set inst_string = \`printf _%04d \$inst\` + echo "${case}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${case}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} + echo "${case}.datm\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + echo "${case}.datm\${inst_string}.rs1.\${restart_time}.nc" >> rpointer.atm\${inst_string} + @ inst ++ + end + + endif + exit 0 +endif + +#---------------------------------------------------------------------- +# This block prepares the initial run directory for a startup run. +# CLM namelists specify a relative path for the initial file. +# +# The reference case may more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - in a hybrid case - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +@ inst=1 +while (\$inst <= $num_instances) + set inst_string = \`printf _%04d \$inst\` + ${LINK} ${CLM_stagedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc . + @ inst ++ +end + exit 0 EndOfText -chmod 0755 stage_initial_cesm_files +chmod 0755 stage_cesm_files -./stage_initial_cesm_files +./stage_cesm_files # ============================================================================== # build @@ -755,7 +804,7 @@ echo "4) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "5) After the job has run, make sure it worked " +echo "5) After the job has run, check to make sure it worked." echo "" echo "6) To extend the run in $assim_n '"$stop_option"' steps," echo " change the env_run.xml variables:" @@ -764,6 +813,9 @@ echo " ./xmlchange RESUBMIT=" echo " ./xmlchange STOP_N=$assim_n" echo "" +echo " and" +echo " ./${case}.submit" +echo "" echo "Check the streams listed in the streams text files. If more or different" echo 'dates need to be added, then do this in the $CASEROOT/user_*files*' echo "then invoke 'preview_namelists' so you can check the information in the" Modified: DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_pmo 2013-10-25 20:46:12 UTC (rev 6550) +++ DART/trunk/models/clm/shell_scripts/CESM1_1_1_setup_pmo 2013-10-25 20:46:24 UTC (rev 6551) @@ -38,7 +38,7 @@ # # ${CASEROOT}/CESM_DART_config will augment the CESM case with the required setup # and configuration to use DART to harvest synthetic observations. CESM_DART_config -# will insert a few dozen lines into the ${CASE}.run script after it makes a backup +# will insert a few dozen lines into the ${case}.run script after it makes a backup # copy. This, and the required setup, can be run at a later date. e.g. you can # advance an ensemble from 2004-01-01 to 2004-02-01 and then run # CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, @@ -95,7 +95,7 @@ # the value of "case" will be used many ways; # directory and file names, both locally and on HPSS, and -# script names; so consider it's length and information content. +# script names; so consider its length and information content. setenv case clm_pmo setenv compset I_2000_CN @@ -108,7 +108,7 @@ # mach Computer name # cesmroot Location of the cesm code base # For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# caseroot Your (future) CESM case directory, where this CESM+DART will be built. # Preferably not a frequently scrubbed location. # This script will delete any existing caseroot, so this script, # and other useful things should be kept elsewhere. @@ -121,6 +121,7 @@ # ============================================================================== setenv mach yellowstone + setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag setenv caseroot /glade/p/work/${USER}/cases/${case} setenv exeroot /glade/scratch/${USER}/${case}/bld @@ -179,7 +180,7 @@ # this configuration step. The run-time settings can lie. # # If the long-term archiver is off, you get a chance to examine the files before -# they get moved to long-term storage. You can always submit $CASE.l_archive +# they get moved to long-term storage. You can always submit $case.l_archive # whenever you want to free up space in the short-term archive directory. # # ============================================================================== @@ -255,8 +256,8 @@ # ============================================================================== # Create the case - this creates the CASEROOT directory. # -# For list of the pre-defined cases: ./create_newcase -list -# To create a variant case, see the CESM documentation and carefully +# For list of the pre-defined component sets: ./create_newcase -list +# To create a variant compset, see the CESM documentation and carefully # incorporate any needed changes into this script. # ============================================================================== @@ -403,6 +404,7 @@ ./xmlchange DATM_CPLHIST_YR_END=$refyear ./xmlchange CALENDAR=GREGORIAN + ./xmlchange STOP_OPTION=$stop_option ./xmlchange STOP_N=$stop_n ./xmlchange CONTINUE_RUN=FALSE @@ -485,7 +487,7 @@ # Ideally, using DART would not require any modifications to the model source. # Until then, this script accesses sourcemods from a hardwired location. # If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'case' directory. +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. # ============================================================================== if ( -d ~/${cesmtag}/SourceMods ) then @@ -495,7 +497,7 @@ echo "ERROR - No SourceMods for this case." echo "DART requires modifications to several src files." echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar" echo "untar these into your HOME directory - they will create a" echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." exit -4 @@ -526,9 +528,7 @@ endif # ============================================================================== -# Modify namelist templates for each instance. This is a bit of a nuisance in -# that we are pulling in restart and initial files from 'all over the place' -# and each model component has a different strategy. +# Modify namelist templates for each instance. # # In a hybrid run with CONTINUE_RUN = FALSE (i.e. just starting up): # @@ -681,41 +681,98 @@ # ============================================================================== # Stage the restarts now that the run directory exists # THIS IS THE STAGING SECTION - MAKE SURE THIS MATCHES THE NAMELISTS. -# POP/CAM/CICE read from pointer files. The others use namelist values initially. # ============================================================================== -cat << EndOfText >! stage_initial_cesm_files -#!/bin/sh +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. -cd ${rundir} - +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' echo '' -echo 'Copying the required restart file from the staging directory.' -echo 'With CONTINUE_RUN=FALSE, only a single file is required.' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' echo '' -if (( $SingleInstanceRefcase )) -then - ${COPY} ${CLM_stagedir}/${run_refcase}.clm2.r.${run_refdate}-${run_reftod}.nc . +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} + +set restart_time = $RUN_REFDATE-$RUN_REFTOD + +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- + +echo "continue_run is \$CONTINUE_RUN" + +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + + # the restarts should be in the short term archive directory + # www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . || exit -1 + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + # The loop is just to make it similar to the multi-instance case. + + @ inst=1 + while (\$inst <= 1) + set inst_string = '' + echo "${case}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${case}.clm2\${inst_string}.r.\${restart_time}.nc" >! rpointer.lnd\${inst_string} + echo "${case}.datm\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + echo "${case}.datm\${inst_string}.rs1.\${restart_time}.nc" >> rpointer.atm\${inst_string} + @ inst ++ + end + + endif + exit 0 +endif + +#---------------------------------------------------------------------- +# This block prepares the initial run directory for a startup run. +# CLM namelists specify a relative path for the initial file. +# +# The reference case may more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - in a hybrid case - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +if ( $SingleInstanceRefcase ) then + + ${LINK} ${CLM_stagedir}/${run_refcase}.clm2.r.${run_refdate}-${run_reftod}.nc . + else - let inst=$TRUTHinstance + set inst_string = '' + set true_string = `printf _%04d $TRUTHinstance` - inst_string=\`printf _%04d \$inst\` + ${LINK} ${CLM_stagedir}/${run_refcase}.clm2\${true_string}.r.${run_refdate}-${run_reftod}.nc \ + ${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc - echo '' - echo "Staging restart for instance \$inst" +endif - ${COPY} ${CLM_stagedir}/${run_refcase}.clm2\${inst_string}.r.${run_refdate}-${run_reftod}.nc \ - ${run_refcase}.clm2.r.${run_refdate}-${run_reftod}.nc -fi - exit 0 EndOfText -chmod 0755 stage_initial_cesm_files +chmod 0755 stage_cesm_files -./stage_initial_cesm_files +./stage_cesm_files # ============================================================================== # build @@ -750,20 +807,21 @@ echo "3) If you want to create 'perfect obs' 'right away', configure and execute" echo " the ${caseroot}/CESM_DART_config script." echo "" -echo "3) Verify the contents of env_run.xml and submit the CESM job:" +echo "4) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "4) After the run finishes ... check the contents of the DART observation sequence file" -echo " ${archdir}/dart/hist/obs_seq.YYYY-MM-DD-SSSSS" -echo " to make sure there are good values in the file. (not -888888.)" +echo "5) After the job has run, check to make sure it worked." echo "" -echo "5) To extend the run in $assim_n '"$stop_option"' steps," +echo "6) To extend the run in $assim_n '"$stop_option"' steps," echo " change the env_run.xml variables:" echo "" echo " ./xmlchange CONTINUE_RUN=TRUE" echo " ./xmlchange RESUBMIT=" echo " ./xmlchange STOP_N=$assim_n" echo "" +echo " and" +echo " ./${case}.submit" +echo "" echo "Check the streams listed in the streams text files. If more or different" echo 'dates need to be added, then do this in the $CASEROOT/user_*files*' echo "then invoke 'preview_namelists' so you can check the information in the" Modified: DART/trunk/models/clm/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/clm/shell_scripts/CESM_DART_config 2013-10-25 20:46:12 UTC (rev 6550) +++ DART/trunk/models/clm/shell_scripts/CESM_DART_config 2013-10-25 20:46:24 UTC (rev 6551) @@ -294,6 +294,7 @@ grep 'SUCCESSFUL TERMINATION' $CplLogFile if ( $status == 0 ) then + ${CASEROOT}/DARTSCRIPTNAME if ( $status == 0 ) then @@ -321,11 +322,13 @@ # modify the "here" document to call the logical DART script. if ( $num_instances == 1 ) then - sed -e "s#DARTSCRIPTNAME#perfect_model.csh#" < temp.$$ >! add_to_run.txt + set DARTscript = perfect_model.csh else - sed -e "s#DARTSCRIPTNAME#assimilate.csh#" < temp.$$ >! add_to_run.txt + set DARTscript = assimilate.csh endif +sed -e "s#DARTSCRIPTNAME#${DARTscript}#" < temp.$$ >! add_to_run.txt + # Now that the "here" document is created, # determine WHERE to insert it -- ONLY IF it is not already there. @@ -365,20 +368,41 @@ # What to do next # ============================================================================== -echo "Time to check the DART configuration:" -echo "" -echo "1) Modify what you need to in the DART namelists. These are " -echo " ${CASEROOT}/input.nml" -echo "" -echo "2) If you have recompiled any part of the DART system, 'refresh_dart_files'" -echo " will copy them into the correct places." -echo "" -echo "3) Make sure the observation directory names in assimilate.csh match" -echo " those on your system, and submit the CESM job in the normal way." -echo "" -echo "4) Inflation files? Observation files for the time of interest?" -echo "" +cat << EndOfText >! DART_instructions.txt + +------------------------------------------------------------------------- + +Check the DART configuration: + +1) The default behavior is to _not_ invoke DART and simply run CESM. + We recommend that you make sure this works before proceeding. + +2) When you want to run DART, make sure the ${CASE}.run script + references "$DARTscript". + +3) Modify what you need to in the DART namelist file, i.e. ${CASEROOT}/input.nml + +4) If you have recompiled any part of the DART system, 'refresh_dart_files' + will copy them into the correct places. + +5) If you stage your own inflation files, make sure you read the INFLATION section + in ${CASEROOT}/CESM_DART_config + +6) Make sure the observation directory names in $DARTscript match + those on your system. + +7) Submit the CESM job in the normal way. + +8) You can use ${CASEROOT}/stage_cesm_files + to stage files to restart a run. + +------------------------------------------------------------------------- + +EndOfText + +cat DART_instructions.txt + exit 0 # Modified: DART/trunk/models/clm/shell_scripts/perfect_model.csh =================================================================== --- DART/trunk/models/clm/shell_scripts/perfect_model.csh 2013-10-25 20:46:12 UTC (rev 6550) +++ DART/trunk/models/clm/shell_scripts/perfect_model.csh 2013-10-25 20:46:24 UTC (rev 6551) @@ -10,13 +10,13 @@ # changes to this script such that the same script can be used # on multiple platforms. This will help us maintain the script. -echo "`date` -- BEGIN GENERATE TRUE STATE" +echo "`date` -- BEGIN GENERATE CLM TRUE STATE" set nonomatch # suppress "rm" warnings if wildcard does not match anything # The FORCE options are not optional. # The VERBOSE options are useful for debugging though -# some systems don't like the -v option to any of the following +# some systems don't like the -v option to any of the following switch ("`hostname`") case be*: # NCAR "bluefire" @@ -93,7 +93,7 @@ # Create temporary working directory for the assimilation and go there #------------------------------------------------------------------------- -set temp_dir = assimilate_clm +set temp_dir = pmo_clm echo "temp_dir is $temp_dir" if ( -d $temp_dir ) then @@ -113,7 +113,7 @@ # The only ugly part here is if the first advance and subsequent advances are # not the same length. The observations _may_ come from different directories. # -# The contents of the file must match the history file contents if one is using +# The contents of the file must match the history file contents if one is using # the obs_def_tower_mod or could be the 'traditional' +/- 12Z ... or both. # Since the history file contains the previous days' history ... so must the obs file. #----------------------------------------------------------------------------- @@ -127,7 +127,7 @@ set OBS_FILE = ${BASEOBSDIR}/${OBSDIR}/obs_seq.${LND_DATE_EXT} if ( -e ${OBS_FILE} ) then - ${COPY} ${OBS_FILE} obs_seq.in + ${LINK} ${OBS_FILE} obs_seq.in else echo "ERROR ... no observation file $OBS_FILE" echo "ERROR ... no observation file $OBS_FILE" @@ -136,58 +136,54 @@ #========================================================================= # Block 1: Populate a run-time directory with the input needed to run DART. +# +# DART namelist settings required: +# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' +# &perfect_model_obs_nml: init_time_days = -1, +# &perfect_model_obs_nml: init_time_seconds = -1, +# &perfect_model_obs_nml: first_obs_days = -1, +# &perfect_model_obs_nml: first_obs_seconds = -1, +# &perfect_model_obs_nml: last_obs_days = -1, +# &perfect_model_obs_nml: last_obs_seconds = -1, +# &clm_to_dart_nml: clm_to_dart_output_file = 'dart_ics' +# #========================================================================= -echo "`date` -- BEGIN COPY BLOCK" - -if ( -e ${CASEROOT}/input.nml ) then - ${COPY} ${CASEROOT}/input.nml . -else +if ( ! -e ${CASEROOT}/input.nml ) then echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" exit -2 endif -echo "`date` -- END COPY BLOCK" +# DART/CLM routines all need a clm_restart.nc, clm_history.nc, etc. +# The flux tower forward operator looks for a CLM history file with +# an instance number in the filename. -#========================================================================= -# Block 2: convert 1 clm restart file to a DART initial conditions file. -# At the end of the block, we have a DART restart file perfect_ics -# that came from the pointer file ../rpointer.lnd_0001 -# -# DART namelist settings appropriate/required: -# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &clm_to_dart_nml: clm_to_dart_output_file = 'dart_ics' -#========================================================================= +set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc +set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc +set OBS1_HISTORY_FILENAME = ${CASE}.clm2.h1.${LND_DATE_EXT}.nc +set OBS2_HISTORY_FILENAME = ${CASE}.clm2_0001.h1.${LND_DATE_EXT}.nc +set DART_IC_FILENAME = perfect_ics -echo "`date` -- BEGIN CLM-TO-DART" +${LINK} ../$LND_RESTART_FILENAME clm_restart.nc +${LINK} ../$LND_HISTORY_FILENAME clm_history.nc -set member = 1 +if ( -e ../$OBS1_HISTORY_FILENAME) then + ${LINK} ../$OBS1_HISTORY_FILENAME $OBS2_HISTORY_FILENAME +endif - set MYTEMPDIR = member_${member} - mkdir -p $MYTEMPDIR - cd $MYTEMPDIR +sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/input.nml >! input.nml - # make sure there are no old output logs hanging around - $REMOVE output.${member}.clm_to_dart +#========================================================================= +# Block 2: convert 1 clm restart file to a DART initial conditions file. +# At the end of the block, we have a DART file "perfect_ics" +# that came from the contents of the pointer file ../rpointer.lnd +#========================================================================= - set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc - set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc - set OBS1_HISTORY_FILENAME = ${CASE}.clm2.h1.${LND_DATE_EXT}.nc - set OBS2_HISTORY_FILENAME = ${CASE}.clm2_0001.h1.${LND_DATE_EXT}.nc - set DART_IC_FILENAME = perfect_ics - - sed -e "s/dart_ics/..\/${DART_IC_FILENAME}/" < ../input.nml >! input.nml - - ${LINK} ../../$LND_RESTART_FILENAME clm_restart.nc - ${LINK} ../../$LND_HISTORY_FILENAME clm_history.nc - - if (-e $OBS1_HISTORY_FILENAME) then - ${LINK} ../../$OBS1_HISTORY_FILENAME $OBS2_HISTORY_FILENAME - endif - - # patch the CLM restart files to ensure they have the proper - # _FillValue and missing_value attributes. +# patch the CLM restart files to ensure they have the proper +# _FillValue and missing_value attributes. # ncatted -O -a _FillValue,frac_sno,o,d,1.0e+36 clm_restart.nc # ncatted -O -a missing_value,frac_sno,o,d,1.0e+36 clm_restart.nc # ncatted -O -a _FillValue,DZSNO,o,d,1.0e+36 clm_restart.nc @@ -199,67 +195,57 @@ # ncatted -O -a _FillValue,T_SOISNO,o,d,1.0e+36 clm_restart.nc # ncatted -O -a missing_value,T_SOISNO,o,d,1.0e+36 clm_restart.nc - ${EXEROOT}/clm_to_dart >! output.${member}.clm_to_dart +echo "`date` -- BEGIN CLM-TO-DART" - if ($status != 0) then - echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" - echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" - exit -3 - endif +${EXEROOT}/clm_to_dart - cd .. +if ($status != 0) then + echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" + echo "ERROR ... DART died in 'clm_to_dart' ... ERROR" + exit -3 +endif echo "`date` -- END CLM-TO-DART" #========================================================================= # Block 3: Advance the model and harvest the synthetic observations. -# Will result in a single file : 'perfect_restart' which we don't need -# for a perfect model experiment with CESM. -# -# DART namelist settings required: -# &perfect_model_obs_nml: async = 0, -# &perfect_model_obs_nml: adv_ens_command = "./no_model_advance.csh", -# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &perfect_model_obs_nml: restart_out_file_name = 'perfect_restart' -# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' -# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.out' -# &perfect_model_obs_nml: init_time_days = -1, -# &perfect_model_obs_nml: init_time_seconds = -1, -# &perfect_model_obs_nml: first_obs_days = -1, -# &perfect_model_obs_nml: first_obs_seconds = -1, -# &perfect_model_obs_nml: last_obs_days = -1, -# &perfect_model_obs_nml: last_obs_seconds = -1, -# +# output iles are: +# True_state.nc ...... the DART state +# obs_seq.perfect ...... the synthetic observations +# dart_log.out ...... run-time output of all DART routines +# perfect_restart ...... which we don't need #========================================================================= -# clm always needs a clm_restart.nc, clm_history.nc for geometry information, etc. +echo "`date` -- BEGIN PERFECT_MODEL_OBS" -set LND_RESTART_FILENAME = ${CASE}.clm2.r.${LND_DATE_EXT}.nc -set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc +${EXEROOT}/perfect_model_obs -${LINK} ../$LND_RESTART_FILENAME clm_restart.nc -${LINK} ../$LND_HISTORY_FILENAME clm_history.nc +if ($status != 0) then + echo "ERROR ... DART died in 'perfect_model_obs' ... ERROR" + echo "ERROR ... DART died in 'perfect_model_obs' ... ERROR" + exit -4 +endif -echo "`date` -- BEGIN PERFECT_MODEL_OBS" -${EXEROOT}/perfect_model_obs || exit -4 -echo "`date` -- END PERFECT_MODEL_OBS" - ${MOVE} True_State.nc ../clm_True_State.${LND_DATE_EXT}.nc -${MOVE} obs_seq.out ../obs_seq.${LND_DATE_EXT}.out +${MOVE} obs_seq.perfect ../clm_obs_seq.${LND_DATE_EXT}.perfect ${MOVE} dart_log.out ../clm_dart_log.${LND_DATE_EXT}.out +echo "`date` -- END PERFECT_MODEL_OBS" + #========================================================================= # Block 4: Update the clm restart files. #========================================================================= -# not needed ... perfect_model_obs does not update the model state. +# not needed ... perfect_model_obs does not update the CLM model state. #------------------------------------------------------------------------- # Cleanup #------------------------------------------------------------------------- -echo "`date` -- END GENERATE TRUE STATE" +${REMOVE} perfect_ics dart_log.nml +echo "`date` -- END GENERATE CLM TRUE STATE" + exit 0 # From nancy at ucar.edu Tue Oct 29 10:55:28 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 29 Oct 2013 10:55:28 -0600 Subject: [Dart-dev] [6553] DART/trunk/observations/NCEP/prep_bufr/README.developer: obsolete information, removing from distribution. Message-ID: Revision: 6553 Author: nancy Date: 2013-10-29 10:55:27 -0600 (Tue, 29 Oct 2013) Log Message: ----------- obsolete information, removing from distribution. Removed Paths: ------------- DART/trunk/observations/NCEP/prep_bufr/README.developer -------------- next part -------------- Deleted: DART/trunk/observations/NCEP/prep_bufr/README.developer =================================================================== --- DART/trunk/observations/NCEP/prep_bufr/README.developer 2013-10-28 19:52:46 UTC (rev 6552) +++ DART/trunk/observations/NCEP/prep_bufr/README.developer 2013-10-29 16:55:27 UTC (rev 6553) @@ -1,228 +0,0 @@ -# DART software - Copyright 2004 - 2011 UCAR. This open source software is -# provided by UCAR, "as is", without charge, subject to all terms of use at -# http://www.image.ucar.edu/DAReS/DART/DART_download -# -# DART $Id$ - -====================================================================== -WARNING: - -The user documentation for compiling and running the prep_bufr program -is NOT here - read the instructions in the prep_bufr.html file. -====================================================================== - - - - -====================================================================== -Developer comments/info: -====================================================================== - - -How to generate decoded NCEP PREPBUFR intermediate text data -As well as DART observation sequence files (daily) -kdr; only daily? - -Introduction -x kdr; add section about what files we're starting with, and what's necessary - to translate them? Refer to .../docs - - This document describes how to generate decoded NCEP reanalysis PREPBUFR -intermediate text data as well as use of the text files to generate DART daily -observation sequence files. - - The decoding package is available as a tar file, prepbufr.tar. - tar -xvf prepbufr.tar and you get the following directories: - -/src Source code of the NCEP PREPBUFR decoder -/lib NCEP PREPBUFR library -/convert_bufr source code to convert the big endian PREPBUFR files to little - endian files and a script to compile the code. -/install A script to install the NCEP PREPBUFR decoder and the NCEP PREPBUFR library. -/exe Executables of the decoder and converter. -/data Where the NCEP PREPBURE files (prepqm****) are -kdr; "PREPBUFR"? -/work Where we run the script to do the decoding. -/docs Some useful background informations about NCEP PREPBUFR observations. - - -====================================================================== -The decoding program: /src/prepbufr.f -====================================================================== - - The program prepbufr.f is used to decode the NCEP reanalysis BUFR data into -intermediate text files. This program was originally developed by NCEP. It -has been modified to output dry temperature, specific humidity, and wind -components (U/V) of conventional radiosonde, aircraft reports, and satellite -cloud motion derived wind. The NCEP quality control indexes for these -observations based on NCEP forecasts are also output and used in DART -observation sequence files. -kdr; are more types of obs available? (PS, radiances, ?) - How much work to get them? - -Program /convert_bufr/grabbufr.f - - The program grabbufr.f is used to convert the big endian format NCEP PREPBUFR -data into little endian format for use on the LINUX computers only. For SGI, IBM -SP, and Sun computers, this convert program is not needed. - -kdr; is it used automatically, or does the use need to execute it? - - -====================================================================== -How to install the NCEP BUFR decoding program -====================================================================== - -Required compilers - - The NCEP PREPBUFR decoding program is written in Fortran 77 and the grabbufr.f -code is written in Fortran 90. The first code can be successfully compiled on -LINUX computers using pgi90, f77 on SGI computers, and xlf on IBM SP systems. -The 2nd code can be compiled with pgf90. -kdr; Add "See note below about compiling under modules. " - - To compile the PREPBUFR libraries and the decoding program, go to the -install directory /install. Set the CPLAT variable in the install.sh to reflect -the correct platform. Currently CPLAT = linux. - - Execute the install.sh script to complete the compilations for the main -decoding program and the NCEP PREPBUFR library. - - For platforms with little endian format, like linux systems, go to -/convert_bufr/ and run convert_bufr.csh to compile the converting code. - - The executables are placed in /exe/, i.e., prepbufr.x and conv.x). - -Platforms tested: - - The code is tested on Coral, Ocotillo.mmm, Tempest, Blusky. - - Special note for Coral: the default compiler is Intel64. To use the pgf90, the -following module updates are needed: - - module rm intel64 netcdf64 mpich64 - module add pgi32 - - type which pgf90 to see if pgf90 is available. - - -=============================================================== -How to get the NCEP Reanalysis BUFR format data from NCAR MSS? -=============================================================== - - The NCEP PREPBUFR files (prepqm*) can be found within the NCEP reanalysis -dataset, ds090.0 (until August 2003) in NCAR MSS. -kdr; meaning files run through 8/03, or they'll be in ds090.0 until 8/03? - - To find the files: - go to http://dss.ucar.edu/pub/reanalysis/ -kdr; announcement there says - Public (non-restricted) version of 200309-200602 prepqm files are released. - Click "Data Archive Summary" - Click "Complete Tar list of Table of Contents of All VSN's" - Look at the years you want and click on the ds090.0 next to the year mark. - -Find the data set you want by searching for prepqmYYMMDDHH, with YY, MM, DD, HH = -the year, month, day, and hour you want. -The prepqm file will be grouped with others in a weekly tar file. -Each tar file has a unique file number like "A#####". - - For example, for January of 2003, The 4 MSS TAR files are: A21899, A21900, -A21901, A21902. These files are unblocked files. - - The decoding program requires blocked version of the files. The corresponding -blocked files are A21899.blk, A21900.blk, A21901.blk, and A21902.blk. -(These names don't appear in the web page, but are found on the mass store. -Each is about 400 Mb) - - Within subdirectory prepbufr/data use - msrcp mss:/DSS/A*****.blk filename.tar - /contrib/bin/cosconvert -b filename.tar - -to read the NCEP BUFR data tar files from NCAR MSS (on Coral, tempest, Bluesky, etc.) - - Then - tar -xvf filename.tar - - You will get individual 6-hourly NCEP PREPBUFR data files for the -observations in the +/- 3-hours time window of 06Z, 12Z, 18Z, and 00Z of each -day. - -Run the decoding program as follows. - -In prepbufr/work/prepbufr.csh set the appropriate values of the year, month, begin_day, and -last_day of the period you desire. Execute prepbufr.csh in the work directory. -kdr; instructions for running this as a batch job? BSUB is there. QSUB? - Script does not cd to proper directory. Other batch problems? - conv.x (grabbufr) is executed unconditionally; is this script only good for Linux, - which requires translation from big- to little-endian? - -Currently, the script generates daily Decoded PREPBUFR text data which contains the -observations within the time window of 3:00Z of the day to 3:00Z of next day. -These daily output text files are named as temp_obs.yyyymmdd. - - These text PREPBUFR data files can then be read by -DART/observations/NCEP/ascii_to_obs/create_real_obs.f90 to -generate the DART daily observation sequence files. - -======================================================= -How to generate DART daily observation sequence files -======================================================= - Go to DART/observations/NCEP/ascii_to_obs/work - - The create_real_obs.f90 is used to convert the decoded PREPBUFR text data to -DART observation sequence files. Currently, this program produces daily -observation sequence from 3:01Z of the day to 3:00Z of the next day. - - Use './quickbuild.csh' to compile the executables here. - Make appropriate changes to the namelist of input.nml. - - The selection of any combinations of the specific observation kinds (T, Q, -U/V, and surface pressure) and types (radiosonde, aircraft reports, or satellite -wind, etc.) can be done in the namelist "ncepobs_nml". You may also want to change -the location of the decoded PREPBUFR text data files in the namelist. Additional -information about the namelist is available from the "create_real_obs.html". - - Sample namelists are as below: - -&ncepobs_nml - year = 2003, - month = 1, - day = 1, - tot_days = 31, - max_num = 700000, - ObsBase = '/ptmp/hliu/decoded_bufr_data/temp_obs.' - select_obs = 0, - ADPUPA = .true., - AIRCAR = .true., - AIRCFT = .true., - SATEMP = .false., - SFCSHP = .false., - ADPSFC = .false., - SATWND = .true., - obs_U = .true., - obs_V = .true., - obs_T = .true., - obs_PS = .false., - obs_QV = .false., - daily_file = .true./ - -&obs_sequence_nml - write_binary_obs_sequence = .true. / - - This will produces daily (or 12-hourly for CAM) observation sequence files -for the period of Jan. 2003 which have the selected observation types and kinds. -The output files looks like "obs_seqyyyymmdd". - - Please note that the current DART/observations/NCEP/ascii_to_obs/create_real_obs.f90 -generates daily (or 6-hourly) observation sequence files which contains observation in -the time window of 3.01Z of the day to 3.00Z of next day. So, generation of each -such daily DART observation sequence files may require the decoded NCEP BUFR -text data files for that day AND next day as well (since the 2nd BUFR file may -also contain the observations at 3:00Z of the next day). - - For example, to generate the observation sequence for jan 1, 2003, the decoded -NCEP PREPBUFR text files for Jan 1 and 2, 2003 are needed. - -================================================================== - From nancy at ucar.edu Tue Oct 29 11:58:44 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Tue, 29 Oct 2013 11:58:44 -0600 Subject: [Dart-dev] [6554] DART/trunk/observations/NCEP/prep_bufr/prep_bufr.html: updated to remove references to the MSS (now HPSS). Message-ID: Revision: 6554 Author: nancy Date: 2013-10-29 11:58:44 -0600 (Tue, 29 Oct 2013) Log Message: ----------- updated to remove references to the MSS (now HPSS). also added text to help with cosblocking files on yellowstone. Modified Paths: -------------- DART/trunk/observations/NCEP/prep_bufr/prep_bufr.html -------------- next part -------------- Modified: DART/trunk/observations/NCEP/prep_bufr/prep_bufr.html =================================================================== --- DART/trunk/observations/NCEP/prep_bufr/prep_bufr.html 2013-10-29 16:55:27 UTC (rev 6553) +++ DART/trunk/observations/NCEP/prep_bufr/prep_bufr.html 2013-10-29 17:58:44 UTC (rev 6554) @@ -36,9 +36,9 @@

    Overview

    -Translating NCEP BUFR files into DART obs_seq.out files (input +Translating NCEP PREPBUFR files into DART obs_seq.out files (input file to filter) is a 2 stage process. The first stage uses NCEP -software to translate the BUFR file into an "intermediate" text file. +software to translate the PREPBUFR file into an intermediate text file. This is described in this document. The second step is to translate the intermediate files into obs_seq.out files, which is done by create_real_obs, as described in @@ -75,14 +75,15 @@ is available on your system.

    -If you have raw unblocked BUFR files you will need to convert them -to blocked format (what prepbufr expects as input). You will need -to uncomment the blk/ublk section in the script and build that +If you have raw unblocked PREPBUFR files you will need to convert them +to blocked format (what prepbufr expects as input). The blk/ublk +section of the build script compiles the cword.x converter program.

    -If you are running on an Intel (little-endian) based machine, uncomment -the grabbufr section in the script to build that converter program. +If you are running on an Intel (little-endian) based machine you will +need the grabbufr byte swapping program that is +also built by this script.

    One-shot execution

    @@ -96,7 +97,7 @@
    • building the executables.
    • running the blocker if needed (generally not if you have - downloaded the blocked format BUFR files). + downloaded the blocked format PREPBUFR files).
    • running the binary format converter if you are on an Intel (little-endian) machine.
    • linking the input file to a fixed input filename @@ -118,7 +119,7 @@

      -

      Installation of the NCEP BUFR decoding program

      +

      Installation of the NCEP PREPBUFR decoding program

      This package is currently organized into files under the @@ -129,27 +130,27 @@ lib NCEP BUFR library source install.sh A script to install the NCEP PREPBUFR decoder and the NCEP BUFR library. exe Executables of the decoder and converter. -data Where the NCEP BUFR files (prepqm****) could be loaded into +data Where the NCEP PREPBUFR files (prepqm****) could be loaded into from the NCAR Mass Store (the script assumes this is the default location). work Where we run the script to do the decoding. -convert_bufr Source code (grabbufr) to convert the binary big-endian BUFR files to +convert_bufr Source code (grabbufr) to convert the binary big-endian PREPBUFR files to little-endian files, and a script to compile the program. blk_ublk Source code (cwordsh) to convert between blocked and unblocked format. -docs Some background information about NCEP BUFR observations. +docs Some background information about NCEP PREPBUFR observations.

      The decoding program: src/prepbufr.f

      -The program prepbufr.f is used to decode the NCEP reanalysis BUFR data into +The program prepbufr.f is used to decode the NCEP reanalysis PREPBUFR data into intermediate text files. This program was originally developed by NCEP. It has been modified to output surface pressure, dry temperature, specific humidity, and wind components (U/V) of conventional radiosonde, aircraft reports, and satellite cloud motion derived wind. There are additional -observation types on the BUFR files, but using them they would require +observation types on the PREPBUFR files, but using them they would require significant modifications of prepbufr and require detailed knowledge -of the NCEP BUFR files. The NCEP quality control indexes for these +of the NCEP PREPBUFR files. The NCEP quality control indexes for these observations based on NCEP forecasts are also output and used in DART observation sequence files. The NCEP PREPBUFR decoding program is written in Fortran 77 and has been successfully compiled on Linux computers using @@ -172,7 +173,7 @@ set the CPLAT variable in the install.sh script to match the compilers available on your system. CPLAT = linux is the default. Execute the install.sh script to complete the compilations for the main -decoding program and the NCEP BUFR library. +decoding program, the NCEP BUFR library, and the conversion utilities.

      @@ -194,17 +195,19 @@

      For platforms with little-endian binary file format (e.g. Intel, AMD®, and non-MIPS SGI processors) the program grabbufr.f is used to convert the -big-endian format NCEP BUFR data into little-endian format. The grabbufr.f +big-endian format NCEP PREPBUFR data into little-endian format. The grabbufr.f code is written in Fortran 90, and has been compiled can be compiled with the pgf90 compiler on a Linux system, with gfortran on an Intel based -Mac, and the ifort compiler on other Linux machines. Instructions for -building it can be found in convert_bufr/README, but basically go to the +Mac, and the ifort compiler on other Linux machines. More detailed +instructions for +building it can be found in convert_bufr/README, but the base install +script should build this by default. In case of problems, cd into the convert_bufr subdirectory, edit convert_bufr.csh to set your compiler, and run it to compile the converter code (grabbufr).

      -This program reads the whole BUFR file into memory, and needs to know +This program reads the whole PREPBUFR file into memory, and needs to know the size of the file (in bytes). Unfortunately, the system call STAT() returns this size as one number in an array, and the index into that array differs depending on the system and sometimes the word size (32 vs 64) @@ -228,104 +231,104 @@ use links to temporarily give the files the names needed.

      -

      The blocking program blk_ublk/cwordsh

      +

      The blocking program blk_ublk/cword.x

      The prepbufr.x program expects to read a blocked input file, which is generally what is available for download. However, if you have an unblocked file that you need to convert, there is a conversion program. -Change directories into the blk_ublk subdirectory and edit the cwordsh.make -file and edit it to match your system. Execute it and it will build -a cwordsh.x executable, which can be run with the shell-script wrapper -program cwordsh. +The install.sh script will try to build this by default, but in case of +problems you can build it separately. +Change directories into the blk_ublk subdirectory and read the +README_cwordsh file for more help. The cwordsh shell-script wrapper +shows how to run the executable cwordsh.x executable.

      -The shell scripts in this directory actually create both a .c and .f -file (which did not exist before), then compile them, then remove them. If you -have problems with the compile, comment out the remove at the end of the make -script so you can see the files it is actually trying to compile. -

      - -

      Note that if you can get the blocked file formats to begin with, this program is not needed.

      -

      Getting the NCEP Reanalysis BUFR format data from NCAR MSS.

      +

      Getting the NCEP Reanalysis PREPBUFR format data from NCAR HPSS.

      -The NCEP BUFR files (prepqmYYMMDDHH) can be found within the NCEP reanalysis -dataset, ds090.0, on NCAR Mass Store System (MSS). +The NCEP PREPBUFR files (prepqmYYMMDDHH) can be found within the NCEP reanalysis +dataset, ds090.0, on NCAR Mass Store System (HPSS).

      - To find the files:

        -
      • go to the +
      • go to the NCAR/NCEP reanalysis archive.
      • -
      • Click "Data Archive Summary"
      • -
      • Click "Complete Tar list of Table of Contents of All VSN's"
      • -
      • Look at the years you want and click on the ds090.0 next to the - year mark.
      • -
      • Find the data set you want by searching for prepqmYYMMDDHH, with - YY, MM, DD, HH = the year, month, day, and hour you want.
      • +
      • Click on the "Inventories" tab.
      • +
      • Select the year you are interested in.
      • +
      • Search for files with the string "prepqm" in the name.
      • +
      • Depending on the year the format of the filenames change, +but they should contain the year, usually as 2 digits, the month, +and then either the start/stop day for weekly files, or the +letters A and B for semi-monthly files.

      -The prepqm file will be grouped with others in a weekly (monthly after -2003) tar file. Each tar file has a unique file number like "A#####". -For example, for January of 2003, the 4 MSS TAR files are: A21899, A21900, -A21901, A21902. Before 2004 these are unblocked files. The decoding program -requires blocked version of the files. The corresponding blocked files -are A21899.blk, A21900.blk, A21901.blk, and A21902.blk. These names don't -appear in the web page, but are found on the mass store. Each is about -400 Mb. After 2003 the files without ".blk" are blocked and can be used. -They are about 1.7 Gb. Also, after September, 2003, these files include +Depending on the year you select, the prepqm files can be weekly, +monthly, or semi-monthly. Each tar file has a unique dataset +number of the form "A#####". +For example, for January of 2003, the 4 HPSS TAR files are: A21899, A21900, +A21901, A21902. +After September 2003, these files include AIRCRAFT data (airplane readings taken at cruising elevation) but not ACARS data (airplane readings taken during takeoff and landing). There are different datasets which include ACARS data but their use is restricted -and you must contact the DSS group to get access. +and you must contact the RDA group to get access.

      -Change directories into the prep_bufr/data subdirectory. -From a machine with access to the MSS use +If you are running on a machine with direct access to the NCAR HPSS, +then change directories into the prep_bufr/data subdirectory and run: +

      +> hsi get /DSS/A##### rawfile

      -msrcp mss:/DSS/A##### data.tar
      or
      -msrcp mss:/DSS/A#####.blk data.tar
      -where ##### is the data set number, to read the NCEP BUFR data tar files. -The files which include ACARS data have a read -password; to specify that to the 'msrcp' command, add the '-rpwd pass' -argument (where pass is a 1-8 character password). -

      -You must then run the 'cosconvert' program to convert the file into a -readable tar file. +where ##### is the data set number you want.

      -
        -
      • On bluefire: /contrib/bin/cosconvert -b data.tar -
      • On other platforms, download the appropriate version from: -http://www.dss.ucar.edu/libraries/io/cos_blocking/utils/ -

      -Then -

      -tar -xvf data.tar -

      -will yield individual 6-hourly -NCEP BUFR data files for the observations in the +/- 3-hour time windows +These files may be readable tar files, or they may require +running the cosconvert program first. +See if the tar command can read them: +

      +> tar -tvf rawfile
      +
      +If you get a good table of contents then simply rename the file +and untar it: +

      +> mv rawfile data.tar
      +> tar -xvf data.tar
      +
      +However, if you get an error from the tar command you +will need to run the cosconvert program to convert the file into a +readable tar file. On the NCAR machine yellowstone, run: +

      +> /glade/u/home/rdadata/bin/cosconvert -b rawfile data.tar
      +
      +On other platforms, download the appropriate version from: + +http://rda.ucar.edu/libraries/io/cos_blocking/utils/ . +Build and run the converter and +then you should have a tar file you can unpack. +

      +

      +The output of tar should yield individual 6-hourly +NCEP PREPBUFR data files for the observations in the +/- 3-hour time windows of 00Z, 06Z, 12Z, and 18Z of each day. Note that DART obs_seq files are organized such that a 24 hour file with 4 observation times would contain observations from 3:01Z to 3:00Z of the next day, centered on 6Z, 12Z, 18Z -and "24Z". In addition, there are some observations at 3:00Z on the BUFR +and "24Z". In addition, there are some observations at 3:00Z on the PREPBUFR file labelled with 06Z. Then, in order to make a full day intermediate file -incorporating all the required obs from the "next" day, you'll need the BUFR +incorporating all the required obs from the "next" day, you'll need the PREPBUFR files through 6Z of the day after the last day of interest. For example, to generate the observation sequence for Jan 1, 2003, the decoded NCEP -BUFR text files for Jan 1 and 2, 2003 are needed, and hence the BUFR files +PREPBUFR text files for Jan 1 and 2, 2003 are needed, and hence the PREPBUFR files

      • prepqm03010106
      • @@ -338,22 +341,22 @@ are needed.

        -

        Running the NCEP BUFR decoding program

        +

        Running the NCEP PREPBUFR decoding program

        In prep_bufr/work/prepbufr.csh set the appropriate values of the year, month, first day, and last day of the period you desire, and the variable "convert" to control conversion from big- to little-endian. Confirm that -the raw BUFR files are in ../data, or that prepbufr.csh has been changed +the raw PREPBUFR files are in ../data, or that prepbufr.csh has been changed to find them. Execute prepbufr.csh in the work directory. It has code for running in the LSF batch environment, but not PBS.

        -Currently, this script generates decoded BUFR text data each 24 hours +Currently, this script generates decoded PREPBUFR text data each 24 hours which contains the observations within the time window of -3:01 hours to +3:00Z within each six-hour synoptic time. These daily output text -files are named as temp_obs.yyyymmdd. These text BUFR data files can +files are named as temp_obs.yyyymmdd. These text PREPBUFR data files can then be read by DART/observations/NCEP/ascii_to_obs/work/create_real_obs to generate the DART daily observation sequence files. @@ -361,8 +364,8 @@

        There is an alternate section in the script which creates a decoded -BUFR text data file each 6 hours (so they are 1-for-1 with the original -BUFR files). Edit the script prepbufr.csh and look for the commented +PREPBUFR text data file each 6 hours (so they are 1-for-1 with the original +PREPBUFR files). Edit the script prepbufr.csh and look for the commented out code which outputs 4 individual files per day. Note that if you chose this option, you will have to make corresponding changes in the create_obs_seq.csh script in step 2. @@ -500,7 +503,7 @@

        [top]

        FILES

        -
        • input file(s); NCEP BUFR observation files named using ObsBase +
          • input file(s); NCEP PREPBUFR observation files named using ObsBase with the "yymmddhh" date tag on the end. Input to grabbufr if big- to little-endian is to be done. Input to prepbufr if not.
          • intermediate (binary) prepqm.little; output from grabbufr, @@ -516,7 +519,7 @@
            [top]

            REFERENCES

            -DART/observations/NCEP/prep_bufr/docs/* (NCEP text files describing the BUFR files) +DART/observations/NCEP/prep_bufr/docs/* (NCEP text files describing the PREPBUFR files) @@ -526,7 +529,7 @@
            [top]

            ERROR CODES and CONDITIONS

            -this space intentionally left blank +Various, see the source code, doc directory, and README files for more help.

            KNOWN BUGS

            @@ -542,9 +545,16 @@
            [top]

            FUTURE PLANS

            -Further development to get observations directly from original -(undecoded) NCEP BUFR files. +This converter could be combined with the DART library code +to go directly from PREPBUFR files to DART obs_sequence files.

            +

            +The converter should make an output file with a fully qualified date +in the name, so constructing files that start at 21Z and +continue to 3Z the next day are not as difficult to process. +There are currently two separate versions of the converter only +because one deals with the date wrap. This shouldn't be needed. +

            From nancy at ucar.edu Thu Oct 31 14:09:53 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 31 Oct 2013 14:09:53 -0600 Subject: [Dart-dev] [6556] DART/trunk/models/POP/shell_scripts: "Best Practice" implementation of the POP scripts for CESM. Message-ID: Revision: 6556 Author: thoar Date: 2013-10-31 14:09:53 -0600 (Thu, 31 Oct 2013) Log Message: ----------- "Best Practice" implementation of the POP scripts for CESM. Tested by Tim, approved by Alicia, reviewed by Nancy. Modified Paths: -------------- DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_hybrid DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_pmo DART/trunk/models/POP/shell_scripts/CESM_DART_config DART/trunk/models/POP/shell_scripts/perfect_model.csh DART/trunk/models/POP/shell_scripts/st_archive.sh Property Changed: ---------------- DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_pmo DART/trunk/models/POP/shell_scripts/CESM_DART_config -------------- next part -------------- Modified: DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_hybrid =================================================================== --- DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-29 20:04:57 UTC (rev 6555) +++ DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_hybrid 2013-10-31 20:09:53 UTC (rev 6556) @@ -14,6 +14,26 @@ # that has POP as the only active component # and will use DART to assimilate observations at regular intervals. # +# Much of the complexity comes from ensuring compatibility between the namelists +# for each instance and staging of the files. The original experiments were run +# before the multi-instance capacity was developed and the naming convention decided. +# Consequently, there is a lot of manipulation of the 'instance' portion of the +# filenames. +# +# This script results in a viable setup for a CESM multi-instance experiment. You +# are STRONGLY encouraged to run the multi-instance CESM a few times and experiment +# with different settings BEFORE you try to assimilate observations. The amount of +# data volume is quite large and you should become comfortable using CESM's restart +# capability to re-stage files in your RUN directory +# +# ${CASEROOT}/CESM_DART_config will augment the CESM case with the required setup +# and configuration to perform a DART assimilation. CASEROOT/CESM_DART_config +# will insert a few dozen lines into the ${CASE}.run script after it makes a backup +# copy. This, and the required setup, can be run at a later date. e.g. you can +# advance an ensemble from 2004-01-01 to 2004-02-01 and then run +# CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, +# and start assimilating observations when CESM stops at 2004-02-01 06Z ... +# # This script relies heavily on the information in: # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/book1.html # @@ -65,14 +85,14 @@ # the value of "case" will be used many ways; # directory and file names, both locally and on HPSS, and -# script names; so consider it's length and information content. +# script names; so consider its length and information content. # num_instances: Number of ensemble members -setenv case pop_test2 +setenv case pop_hybrid setenv compset GIAF setenv resolution T62_gx1v6 setenv cesmtag cesm1_1_1 -setenv num_instances 4 +setenv num_instances 30 # ============================================================================== # define machines and directories @@ -80,7 +100,7 @@ # mach Computer name # cesmroot Location of the cesm code base # For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# caseroot Your (future) CESM case directory, where this CESM+DART will be built. # Preferably not a frequently scrubbed location. # This script will delete any existing caseroot, so this script, # and other useful things should be kept elsewhere. @@ -93,6 +113,7 @@ # ============================================================================== setenv mach yellowstone + setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag setenv caseroot /glade/p/work/${USER}/cases/${case} setenv exeroot /glade/scratch/${USER}/${case}/bld @@ -119,7 +140,7 @@ # THIS IS THE LOCATION of the 'reference case'. -setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/2004-01-10-00000 +setenv POP_stagedir /glade/p/image/CESM_initial_ensemble/rest/2004-01-10-00000 # ============================================================================== # runtime settings @@ -150,7 +171,6 @@ # # TJH: Advancing 30 instances for 72 hours with 900 pes (30*15*2) with # an assimilation step took less than 7 minutes on yellowstone. -# # ============================================================================== setenv ACCOUNT P8685xxxx @@ -190,7 +210,7 @@ # VAR is the shell variable name, DIR is the value # ============================================================================== -foreach VAR ( cesmroot dartroot stagedir ) +foreach VAR ( cesmroot dartroot POP_stagedir ) set DIR = `eval echo \${$VAR}` if ( ! -d $DIR ) then echo "ERROR: directory '$DIR' not found" @@ -202,8 +222,8 @@ # ============================================================================== # Create the case - this creates the CASEROOT directory. # -# For list of the pre-defined cases: ./create_newcase -list -# To create a variant case, see the CESM documentation and carefully +# For list of the pre-defined component sets: ./create_newcase -list +# To create a variant compset, see the CESM documentation and carefully # incorporate any needed changes into this script. # ============================================================================== @@ -341,7 +361,7 @@ # climate, however, should be continuous provided that no model source code or # namelists are changed in the hybrid run. In a hybrid initialization, the ocean # model does not start until the second ocean coupling (normally the second day), -# and the coupler does a "cold start" without a restart file. +# and the coupler does a "cold start" without a restart file." ./xmlchange RUN_TYPE=hybrid ./xmlchange RUN_STARTDATE=$run_refdate @@ -362,6 +382,7 @@ # The streams files were generated with a NO_LEAP calendar in mind. # We need to test these with a GREGORIAN calendar. ./xmlchange CALENDAR=GREGORIAN + ./xmlchange STOP_OPTION=$stop_option ./xmlchange STOP_N=$stop_n ./xmlchange CONTINUE_RUN=FALSE @@ -369,12 +390,18 @@ ./xmlchange PIO_TYPENAME=pnetcdf -# The river transport model ON is useful only when using an active ocean or -# land surface diagnostics. Setting ROF_GRID to 'null' turns off the RTM. -# TJH - guidance needed from Alicia +# COUPLING discussion. +# http://bugs.cgd.ucar.edu/show_bug.cgi?id=1740 +# "In summary, users should ensure that the following is true, +# ATM_NCPL = LND_NCPL = ICE_NCPL >= ROF_NCPL >= OCN_NCPL" +# ROF_NCPL defaults to $ATM_NCPL (24 in the default config) +# OCN_NCPL defaults to 1 (per day) +# +# OCN_NCPL == 4 sets the ocean coupling time to 6 hours. +# All related CESM namelist settings are based on this value. +# OCN_NCPL is # coupling intervals per NCPL_BASE_PERIOD (default 'day') -# ./xmlchange ROF_GRID='null' -# ./xmlchange ROF_GRID='r05' +./xmlchange OCN_NCPL=1 if ($short_term_archiver == 'off') then ./xmlchange DOUT_S=FALSE @@ -444,7 +471,7 @@ # Ideally, using DART would not require any modifications to the model source. # Until then, this script accesses sourcemods from a hardwired location. # If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'case' directory. +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. # ============================================================================== if ( -d ~/${cesmtag}/SourceMods ) then @@ -454,7 +481,7 @@ echo "ERROR - No SourceMods for this case." echo "DART requires modifications to several src files." echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar" echo "untar these into your HOME directory - they will create a" echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." exit -4 @@ -469,17 +496,12 @@ if ( -d ~/${cesmtag} ) then # preserve the original version of the files - if ( ! -e ${caseroot}/Buildconf/rtm.buildnml.csh.original ) then - ${MOVE} ${caseroot}/Buildconf/rtm.buildnml.csh \ - ${caseroot}/Buildconf/rtm.buildnml.csh.original - endif if ( ! -e ${caseroot}/preview_namelists.original ) then - ${MOVE} ${caseroot}/preview_namelists \ + ${COPY} ${caseroot}/preview_namelists \ ${caseroot}/preview_namelists.original endif # patch/replace the broken files - ${COPY} ~/${cesmtag}/rtm.buildnml.csh ${caseroot}/Buildconf/. ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. endif @@ -526,27 +548,33 @@ # init_ts_suboption = 'data_assim' for non bit-for-bit restarting (assimilation mode) # init_ts_suboption = 'rest' --> default behavior # - # DEFAULT values for these are: - # tavg_file_freq_opt = 'nmonth' 'nmonth' 'once' - # tavg_freq_opt = 'nmonth' 'nday' 'once' - # The first entry indicates we get a monthly average once a month. - # The second entry indicates we get a monthly average as it is being created. - # The third entry indicates we get a daily timeslice + # README: + # Configuring the contents of the history files for POP is best explained in + # the section marked "POP2: TAVG Settings" in the cesm1_1_1 pop2 namelist documentation + # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/modelnl/nl_pop2.html # - # IFF values for these are: - # tavg_file_freq_opt = 'nmonth' 'never' 'never' - # tavg_freq_opt = 'nmonth' 'never' 'never' - # The first entry indicates we get a monthly average once a month, and thats all we get.. + # and the CESM-specific documentation for the tavg output variables in the pop2 + # online documentation: + # http://www.cesm.ucar.edu/models/cesm1.1/pop2/doc/users/node78.html + # + # In CESM1_1_1 keep the values for tavg_file_freq_opt and tavg_freq_opt identical. + # pop2/trunk_tags/cesm_pop_2_1_20130412 explains the issue. + # + # DEFAULT values for these are: + # tavg_file_freq_opt = 'nmonth' 'nmonth' 'once' + # tavg_freq_opt = 'nmonth' 'nday' 'once' + # The first entry indicates we get a monthly average once a month. + # The second entry indicates we get a monthly average as it is being created. + # The third entry indicates we get a daily timeslice + # + # Default copies of SourceMods/src.pop2/ocn.*.tavg.csh files are provided in the + # DART_SourceMods_cesm1_1_1.tar bundle. echo "init_ts_suboption = 'data_assim'" >> $fname - echo "tavg_file_freq_opt = 'nmonth' 'never' 'never'" >> $fname - echo "tavg_freq_opt = 'nmonth' 'never' 'never'" >> $fname @ inst ++ end -# DLND - echo "streams = 'drof.streams.txt.rof.diatren_iaf_rx1" 1 1948 2009"'" >> user_nl_drof # ============================================================================== @@ -621,47 +649,121 @@ # Stage the restarts now that the run directory exists # ============================================================================== -cat << EndOfText >! stage_initial_cesm_files -#!/bin/sh +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. -cd ${rundir} - +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' echo '' -echo 'Copying the restart files from the staging directories' -echo 'into the CESM run directory and creating the pointer files.' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' echo '' -let inst=1 -while ((\$inst <= $num_instances)); do - inst_string=\`printf _%04d \$inst\` +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} +set restart_time = $RUN_REFDATE-$RUN_REFTOD + +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- + +echo "continue_run is \$CONTINUE_RUN" + +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + + # the restarts should be in the short term archive directory + # www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . || exit -1 + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + + @ inst=1 + while (\$inst <= $num_instances) + + set inst_string = \`printf _%04d \$inst\` + echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${CASE}.datm\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + echo "${CASE}.datm\${inst_string}.rs1.\${restart_time}.bin" >> rpointer.atm\${inst_string} + echo "${CASE}.drof\${inst_string}.r.\${restart_time}.nc" >! rpointer.rof\${inst_string} + echo "${CASE}.drof\${inst_string}.rs1.\${restart_time}.bin" >> rpointer.rof\${inst_string} + echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} + + echo "${CASE}.pop\${inst_string}.ro.\${restart_time}" >! rpointer.ocn\${inst_string}.ovf + echo "${CASE}.pop\${inst_string}.r.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.restart + echo "RESTART_FMT=nc" >> rpointer.ocn\${inst_string}.restart + + if ( -e rpointer.ocn\${inst_string}.tavg ) then + echo "${CASE}.pop\${inst_string}.rh.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.tavg + endif + if ( -e rpointer.ocn\${inst_string}.tavg.2 ) then + echo "${CASE}.pop\${inst_string}.rh.nday1.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.tavg.2 + endif + + @ inst ++ + end + endif + + exit 0 +endif + +#---------------------------------------------------------------------- +# THIS IS THE INITIAL STAGING SECTION. +# This block prepares the initial run directory for a startup run. +# The CICE namelist specifies a relative path for the initial file. +# +# POP uses pointer files. +# +# The reference case may contain more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +set filedir = ${POP_stagedir} + +@ inst=1 +while (\$inst <= $num_instances) + + set inst_string = \`printf _%04d \$inst\` + echo '' echo "Staging restarts for instance \$inst of $num_instances" - ${LINK} ${stagedir}/${run_refcase}.pop\${inst_string}.r.2004-01-10-00000.nc . - ${LINK} ${stagedir}/${run_refcase}.pop\${inst_string}.ro.2004-01-10-00000 . -# ${LINK} ${stagedir}/${run_refcase}.pop\${inst_string}.rh.2004-01-10-00000.nc . - #TH: The cice fname must match that in the user_nl_cice file + ${LINK} \${filedir}/${run_refcase}.cice\${inst_string}.r.\${restart_time}.nc . + ${LINK} \${filedir}/${run_refcase}.pop\${inst_string}.r.\${restart_time}.nc . + ${LINK} \${filedir}/${run_refcase}.pop\${inst_string}.ro.\${restart_time} . - ${LINK} ${stagedir}/${run_refcase}.cice\${inst_string}.r.2004-01-10-00000.nc . + echo "${run_refcase}.pop\${inst_string}.ro.\${restart_time}" >! rpointer.ocn\${inst_string}.ovf + echo "${run_refcase}.pop\${inst_string}.r.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.restart + echo "RESTART_FMT=nc" >> rpointer.ocn\${inst_string}.restart - echo "${run_refcase}.pop\${inst_string}.ro.2004-01-10-00000" \ - >| rpointer.ocn\${inst_string}.ovf - echo "${run_refcase}.pop\${inst_string}.r.2004-01-10-00000.nc" \ - >| rpointer.ocn\${inst_string}.restart - echo "RESTART_FMT=nc" \ - >> rpointer.ocn\${inst_string}.restart + @ inst ++ +end - let inst+=1 -done - exit 0 EndOfText -chmod 0755 stage_initial_cesm_files +chmod 0755 stage_cesm_files -./stage_initial_cesm_files +./stage_cesm_files # ============================================================================== # build @@ -693,23 +795,35 @@ echo ' #BSUB -R "select[scratch_ok > 0]"' echo " around line 9, delete it." echo "" -echo "3) If you want to assimilate 'right away', configure and execute" +echo "3) The very first CESM advance (i.e. CONTINUE_RUN=FALSE)" +echo " STOP_N must be longer than *AT LEAST 2 TIMES* the coupling" +echo " frequency between the atmosphere and ocean." +echo " If coupling once a day, the first advance MUST be at least 48 hours." +echo " If coupling 4 times a day, the first advance MUST be at least 12 hours." +echo " After that, STOP_N can be as short as a single coupling frequency." +echo "" +echo "4) If you want to assimilate 'right away', configure and execute" echo " the ${caseroot}/CESM_DART_config script." echo "" -echo "3) Verify the contents of env_run.xml and submit the CESM job:" +echo "5) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "4) After the run finishes ... check the contents of the DART observation sequence file" -echo " ${archdir}/dart/hist/obs_seq.YYYY-MM-DD-SSSSS" -echo " to make sure there are good values in the file. (not -888888.)" +echo "6) After the job has run, check to make sure it worked and that" +echo " a: POP is creating netCDF restart files," +echo " b: the right restart files exist in the run directory," +echo " c: (if you're running DART) the archive dart/hist directory has the DART output," +echo " d: everything is working correctly ..." echo "" -echo "5) To extend the run in $assim_n '"$stop_option"' steps," +echo "7) To extend the run in $assim_n '"$stop_option"' steps," echo " change the env_run.xml variables:" echo "" echo " ./xmlchange CONTINUE_RUN=TRUE" echo " ./xmlchange RESUBMIT=" echo " ./xmlchange STOP_N=$assim_n" echo "" +echo " and" +echo " ./${case}.submit" +echo "" echo "Check the streams listed in the streams text files. If more or different" echo 'dates need to be added, then do this in the $CASEROOT/user_*files*' echo "then invoke 'preview_namelists' so you can check the information in the" Modified: DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_pmo =================================================================== --- DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_pmo 2013-10-29 20:04:57 UTC (rev 6555) +++ DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_pmo 2013-10-31 20:09:53 UTC (rev 6556) @@ -4,7 +4,7 @@ # provided by UCAR, "as is", without charge, subject to all terms of use at # http://www.image.ucar.edu/DAReS/DART/DART_download # -# DART $Id: CESM1_1_setup_hybrid.csh 6282 2013-06-28 22:54:08Z thoar $ +# DART $Id$ # --------------------- # Purpose @@ -28,7 +28,7 @@ # # ${CASEROOT}/CESM_DART_config will augment the CESM case with the required setup # and configuration to use DART to harvest synthetic observations. CESM_DART_config -# will insert a few dozen lines into the ${CASE}.run script after it makes a backup +# will insert a few dozen lines into the ${case}.run script after it makes a backup # copy. This, and the required setup, can be run at a later date. e.g. you can # advance an ensemble from 2004-01-01 to 2004-02-01 and then run # CESM_DART_config to augment the existing run script, modify STOP_N to 6 hours, @@ -85,7 +85,7 @@ # the value of "case" will be used many ways; # directory and file names, both locally and on HPSS, and -# script names; so consider it's length and information content. +# script names; so consider its length and information content. setenv case pop_pmo setenv compset GIAF @@ -98,7 +98,7 @@ # mach Computer name # cesmroot Location of the cesm code base # For cesm1_1_1 on yellowstone -# caseroot Your (future) cesm case directory, where this CESM+DART will be built. +# caseroot Your (future) CESM case directory, where this CESM+DART will be built. # Preferably not a frequently scrubbed location. # This script will delete any existing caseroot, so this script, # and other useful things should be kept elsewhere. @@ -111,6 +111,7 @@ # ============================================================================== setenv mach yellowstone + setenv cesmroot /glade/p/cesm/cseg/collections/$cesmtag setenv caseroot /glade/p/work/${USER}/cases/${case} setenv exeroot /glade/scratch/${USER}/${case}/bld @@ -143,18 +144,18 @@ # # TRUTHinstance: specifies the specific instance you want to define as the TRUTH. # If you have an initial ensemble size of 80, 1<= instance <= 80. -# If you only have one CLM state ... use 1. This value is also +# If you only have one POP state ... use 1. This value is also # used to specify _which_ DATM streamfile to use to force the -# TRUTH run. +# TRUTH run. Do not use leading zeros. # -# CLM_stagedir: specifies the location of the CLM restart file you are defining as +# POP_stagedir: specifies the location of the POP restart file you are defining as # the truth. As this state evolves in time, it will be used as the # input to the exact same forward observation operator code that # will be used during a subsequent assimilation. setenv SingleInstanceRefcase 0 setenv TRUTHinstance 23 -setenv stagedir /glade/p/image/CESM_initial_ensemble/rest/2004-01-10-00000 +setenv POP_stagedir /glade/p/image/CESM_initial_ensemble/rest/2004-01-10-00000 # ============================================================================== # runtime settings @@ -185,7 +186,6 @@ # # TJH: Advancing 1 instance for 72 hours with 60 pes (1*15*4) with # a perfect_model_obs step took less than 2 minutes on yellowstone. -# # ============================================================================== setenv ACCOUNT P8685xxxx @@ -225,7 +225,7 @@ # VAR is the shell variable name, DIR is the value # ============================================================================== -foreach VAR ( cesmroot dartroot stagedir ) +foreach VAR ( cesmroot dartroot POP_stagedir ) set DIR = `eval echo \${$VAR}` if ( ! -d $DIR ) then echo "ERROR: directory '$DIR' not found" @@ -237,8 +237,8 @@ # ============================================================================== # Create the case - this creates the CASEROOT directory. # -# For list of the pre-defined cases: ./create_newcase -list -# To create a variant case, see the CESM documentation and carefully +# For list of the pre-defined component sets: ./create_newcase -list +# To create a variant compset, see the CESM documentation and carefully # incorporate any needed changes into this script. # ============================================================================== @@ -382,6 +382,7 @@ # The streams files were generated with a NO_LEAP calendar in mind. # We need to test these with a GREGORIAN calendar. ./xmlchange CALENDAR=GREGORIAN + ./xmlchange STOP_OPTION=$stop_option ./xmlchange STOP_N=$stop_n ./xmlchange CONTINUE_RUN=FALSE @@ -389,12 +390,18 @@ ./xmlchange PIO_TYPENAME=pnetcdf -# The river transport model ON is useful only when using an active ocean or -# land surface diagnostics. Setting ROF_GRID to 'null' turns off the RTM. -# TJH - guidance needed from Alicia +# COUPLING discussion. +# http://bugs.cgd.ucar.edu/show_bug.cgi?id=1740 +# "In summary, users should ensure that the following is true, +# ATM_NCPL = LND_NCPL = ICE_NCPL >= ROF_NCPL >= OCN_NCPL" +# ROF_NCPL defaults to $ATM_NCPL (24 in the default config) +# OCN_NCPL defaults to 1 (per day) +# +# OCN_NCPL == 4 sets the ocean coupling time to 6 hours. +# All related CESM namelist settings are based on this value. +# OCN_NCPL is # coupling intervals per NCPL_BASE_PERIOD (default 'day') -# ./xmlchange ROF_GRID='null' -# ./xmlchange ROF_GRID='r05' +./xmlchange OCN_NCPL=1 if ($short_term_archiver == 'off') then ./xmlchange DOUT_S=FALSE @@ -464,7 +471,7 @@ # Ideally, using DART would not require any modifications to the model source. # Until then, this script accesses sourcemods from a hardwired location. # If you have additional sourcemods, they will need to be merged into any DART -# mods and put in the SourceMods subdirectory found in the 'case' directory. +# mods and put in the SourceMods subdirectory found in the 'caseroot' directory. # ============================================================================== if ( -d ~/${cesmtag}/SourceMods ) then @@ -474,7 +481,7 @@ echo "ERROR - No SourceMods for this case." echo "DART requires modifications to several src files." echo "These files can be downloaded from:" - echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1.tar" + echo "http://www.image.ucar.edu/pub/DART/CESM/DART_SourceMods_cesm1_1_1_24Oct2013.tar" echo "untar these into your HOME directory - they will create a" echo "~/cesm_1_1_1 directory with the appropriate SourceMods structure." exit -4 @@ -489,17 +496,12 @@ if ( -d ~/${cesmtag} ) then # preserve the original version of the files - if ( ! -e ${caseroot}/Buildconf/rtm.buildnml.csh.original ) then - ${MOVE} ${caseroot}/Buildconf/rtm.buildnml.csh \ - ${caseroot}/Buildconf/rtm.buildnml.csh.original - endif if ( ! -e ${caseroot}/preview_namelists.original ) then - ${MOVE} ${caseroot}/preview_namelists \ + ${COPY} ${caseroot}/preview_namelists \ ${caseroot}/preview_namelists.original endif # patch/replace the broken files - ${COPY} ~/${cesmtag}/rtm.buildnml.csh ${caseroot}/Buildconf/. ${COPY} ~/${cesmtag}/preview_namelists ${caseroot}/. endif @@ -537,7 +539,7 @@ # this is only used for a hybrid start, else rpointers are used. set instance = `printf _%04d $TRUTHinstance` - echo "ice_ic = '${stagedir}/${run_refcase}.cice${instance}.r.2004-01-10-00000.nc'" >> $fname + echo "ice_ic = '${POP_stagedir}/${run_refcase}.cice${instance}.r.2004-01-10-00000.nc'" >> $fname # =========================================================================== set fname = "user_nl_pop2$inst_string" @@ -547,27 +549,33 @@ # init_ts_suboption = 'data_assim' for non bit-for-bit restarting (assimilation mode) # init_ts_suboption = 'rest' --> default behavior # - # DEFAULT values for these are: - # tavg_file_freq_opt = 'nmonth' 'nmonth' 'once' - # tavg_freq_opt = 'nmonth' 'nday' 'once' - # The first entry indicates we get a monthly average once a month. - # The second entry indicates we get a monthly average as it is being created. - # The third entry indicates we get a daily timeslice + # README: + # Configuring the contents of the history files for POP is best explained in + # the section marked "POP2: TAVG Settings" in the cesm1_1_1 pop2 namelist documentation + # http://www.cesm.ucar.edu/models/cesm1.1/cesm/doc/modelnl/nl_pop2.html # - # IFF values for these are: - # tavg_file_freq_opt = 'nmonth' 'never' 'never' - # tavg_freq_opt = 'nmonth' 'never' 'never' - # The first entry indicates we get a monthly average once a month, and thats all we get.. + # and the CESM-specific documentation for the tavg output variables in the pop2 + # online documentation: + # http://www.cesm.ucar.edu/models/cesm1.1/pop2/doc/users/node78.html + # + # In CESM1_1_1 keep the values for tavg_file_freq_opt and tavg_freq_opt identical. + # pop2/trunk_tags/cesm_pop_2_1_20130412 explains the issue. + # + # DEFAULT values for these are: + # tavg_file_freq_opt = 'nmonth' 'nmonth' 'once' + # tavg_freq_opt = 'nmonth' 'nday' 'once' + # The first entry indicates we get a monthly average once a month. + # The second entry indicates we get a monthly average as it is being created. + # The third entry indicates we get a daily timeslice + # + # Default copies of SourceMods/src.pop2/ocn.*.tavg.csh files are provided in the + # DART_SourceMods_cesm1_1_1.tar bundle. echo "init_ts_suboption = 'data_assim'" >> $fname - echo "tavg_file_freq_opt = 'nmonth' 'never' 'never'" >> $fname - echo "tavg_freq_opt = 'nmonth' 'never' 'never'" >> $fname @ inst ++ end -# DLND - echo "streams = 'drof.streams.txt.rof.diatren_iaf_rx1" 1 1948 2009"'" >> user_nl_drof # ============================================================================== @@ -628,9 +636,8 @@ ${COPY} $dartroot/models/POP/shell_scripts/user_$streamname*template $FNAME - sed s/NINST/$instance/g $FNAME >! out.$$ - sed s/REFYEAR/$refyear/g out.$$ >! $FNAME - \rm -f out.$$ + sed s/NINST/$instance/g $FNAME >! out.$$ + ${MOVE} out.$$ $FNAME else echo "DIED Looking for a DART stream txt template for $FNAME" @@ -646,30 +653,112 @@ # Stage the restarts now that the run directory exists # ============================================================================== -cat << EndOfText >! stage_initial_cesm_files -#!/bin/sh +cat << EndOfText >! stage_cesm_files +#!/bin/csh -f +# This script can be used to help restart an experiment from any previous step. +# The files are copied to the RUN directory. +# Set the 'restart_time' variable below to define when to restart the experiment. +# The value of the CESM variable 'env_run.xml:CONTINUE_RUN' is used to determine +# the origin of the files. -cd ${rundir} - +echo 'Copying the required files to the run directory.' +echo 'No matter what, make sure the restart_time is set to the desired time.' echo '' -echo 'Just using the full pathname for the files in the pointer files.' +echo 'With CONTINUE_RUN=FALSE, the files are copied from the initial ensemble directory.' +echo 'When CONTINUE_RUN=TRUE, the files are copied from the short-term archive restart directory.' +echo 'If the short-term archiver is OFF, all the files are already in the RUN directory and' +echo 'just the pointer files need to be updated.' echo '' - inst_string=`printf _%04d $TRUTHinstance` +cd ${caseroot} +source ./Tools/ccsm_getenv || exit -2 +cd ${RUNDIR} - echo "${stagedir}/${run_refcase}.pop\${inst_string}.ro.2004-01-10-00000" > rpointer.ocn.ovf - echo "${stagedir}/${run_refcase}.pop\${inst_string}.r.2004-01-10-00000.nc" > rpointer.ocn.restart - echo "RESTART_FMT=nc" >> rpointer.ocn.restart +set restart_time = $RUN_REFDATE-$RUN_REFTOD - # ${LINK} ${stagedir}/${run_refcase}.cice\${inst_string}.r.2004-01-10-00000.nc \ - # ${run_refcase}.cice.r.2004-01-10-00000.nc +#---------------------------------------------------------------------- +# This block copies over a set of restart files from +# this experiment and exits. Intended use to restart a failed run. +#---------------------------------------------------------------------- +echo "continue_run is \$CONTINUE_RUN" + +if ( \$CONTINUE_RUN == TRUE ) then + if ( \$DOUT_S == TRUE ) then + + # the restarts should be in the short term archive directory + # www.cesm.ucar.edu/models/cesm1.1/cesm/doc/usersguide/x1631.html#running_ccsm_restart_back + + ${COPY} \$DOUT_S_ROOT/rest/\${restart_time}/* . || exit -1 + + else + + # All the required files must still be in the run directory. + # Just need to update the pointer files. + # The loop is just to make it similar to the multi-instance case. + + @ inst=1 + while (\$inst <= 1) + + set inst_string = '' + echo "${CASE}.cpl.r.\${restart_time}.nc" >! rpointer.drv + echo "${CASE}.datm\${inst_string}.r.\${restart_time}.nc" >! rpointer.atm\${inst_string} + echo "${CASE}.datm\${inst_string}.rs1.\${restart_time}.bin" >> rpointer.atm\${inst_string} + echo "${CASE}.drof\${inst_string}.r.\${restart_time}.nc" >! rpointer.rof\${inst_string} + echo "${CASE}.drof\${inst_string}.rs1.\${restart_time}.bin" >> rpointer.rof\${inst_string} + echo "${CASE}.cice\${inst_string}.r.\${restart_time}.nc" >! rpointer.ice\${inst_string} + + echo "${CASE}.pop\${inst_string}.ro.\${restart_time}" >! rpointer.ocn\${inst_string}.ovf + echo "${CASE}.pop\${inst_string}.r.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.restart + echo "RESTART_FMT=nc" >> rpointer.ocn\${inst_string}.restart + + if ( -e rpointer.ocn\${inst_string}.tavg ) then + echo "${CASE}.pop\${inst_string}.rh.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.tavg + endif + if ( -e rpointer.ocn\${inst_string}.tavg.2 ) then + echo "${CASE}.pop\${inst_string}.rh.nday1.\${restart_time}.nc" >! rpointer.ocn\${inst_string}.tavg.2 + endif + + @ inst ++ + end + endif + + exit 0 +endif + +#---------------------------------------------------------------------- +# THIS IS THE INITIAL STAGING SECTION. +# The CICE namelist specified an absolute path for the initial file, +# so there is nothing to do here. +# +# POP uses pointer files. +# +# The reference case may contain more instances than we need, +# and it certainly has more files than we need. +# This stages only the minimum number of instances and files. +# +# Remember - when CONTINUE_RUN = FALSE, +# the run_refdate and the time in the filenames can be unrelated. +#---------------------------------------------------------------------- + +set inst_string = '' + +if ( $SingleInstanceRefcase ) then + set true_string = '' +else + set true_string = `printf _%04d $TRUTHinstance` +endif + +echo "${POP_stagedir}/${run_refcase}.pop\${true_string}.ro.${run_refdate}-${run_reftod}" >! rpointer.ocn\${inst_string}.ovf +echo "${POP_stagedir}/${run_refcase}.pop\${true_string}.r.${run_refdate}-${run_reftod}.nc" >! rpointer.ocn\${inst_string}.restart +echo "RESTART_FMT=nc" >> rpointer.ocn\${inst_string}.restart + exit 0 EndOfText -chmod 0755 stage_initial_cesm_files +chmod 0755 stage_cesm_files -./stage_initial_cesm_files +./stage_cesm_files # ============================================================================== # build @@ -701,23 +790,35 @@ echo ' #BSUB -R "select[scratch_ok > 0]"' echo " around line 9, delete it." echo "" -echo "3) If you want to create 'perfect obs' 'right away', configure and execute" +echo "3) The very first CESM advance (i.e. CONTINUE_RUN=FALSE)" +echo " STOP_N must be longer than *AT LEAST 2 TIMES* the coupling" +echo " frequency between the atmosphere and ocean." +echo " If coupling once a day, the first advance MUST be at least 48 hours." +echo " If coupling 4 times a day, the first advance MUST be at least 12 hours." +echo " After that, STOP_N can be as short as a single coupling frequency." +echo "" +echo "4) If you want to create 'perfect obs' 'right away', configure and execute" echo " the ${caseroot}/CESM_DART_config script." echo "" -echo "3) Verify the contents of env_run.xml and submit the CESM job:" +echo "5) Verify the contents of env_run.xml and submit the CESM job:" echo " ./${case}.submit" echo "" -echo "4) After the run finishes ... check the contents of the DART observation sequence file" -echo " ${archdir}/dart/hist/obs_seq.YYYY-MM-DD-SSSSS" -echo " to make sure there are good values in the file. (not -888888.)" +echo "6) After the job has run, check to make sure it worked and that" +echo " a: POP is creating netCDF restart files," +echo " b: the right restart files exist in the run directory," +echo " c: (if you're running DART) the archive dart/hist directory has the DART output," +echo " d: everything is working correctly ..." echo "" -echo "5) To extend the run in $assim_n '"$stop_option"' steps," +echo "7) To extend the run in $assim_n '"$stop_option"' steps," echo " change the env_run.xml variables:" echo "" echo " ./xmlchange CONTINUE_RUN=TRUE" echo " ./xmlchange RESUBMIT=" echo " ./xmlchange STOP_N=$assim_n" echo "" +echo " and" +echo " ./${case}.submit" +echo "" echo "Check the streams listed in the streams text files. If more or different" echo 'dates need to be added, then do this in the $CASEROOT/user_*files*' echo "then invoke 'preview_namelists' so you can check the information in the" @@ -727,7 +828,7 @@ exit 0 # -# $URL: https://subversion.ucar.edu/DAReS/DART/trunk/models/POP/shell_scripts/CESM1_1_setup_hybrid.csh $ -# $Revision: 6282 $ -# $Date: 2013-06-28 16:54:08 -0600 (Fri, 28 Jun 2013) $ +# $URL$ +# $Revision$ +# $Date$ Property changes on: DART/trunk/models/POP/shell_scripts/CESM1_1_1_setup_pmo ___________________________________________________________________ Added: svn:mime-type + text/plain Added: svn:keywords + Date Rev Author HeadURL Id Added: svn:eol-style + native Modified: DART/trunk/models/POP/shell_scripts/CESM_DART_config =================================================================== --- DART/trunk/models/POP/shell_scripts/CESM_DART_config 2013-10-29 20:04:57 UTC (rev 6555) +++ DART/trunk/models/POP/shell_scripts/CESM_DART_config 2013-10-31 20:09:53 UTC (rev 6556) @@ -4,7 +4,7 @@ # provided by UCAR, "as is", without charge, subject to all terms of use at # http://www.image.ucar.edu/DAReS/DART/DART_download # -# DART $Id: CESM_DART_config 6422 2013-08-23 21:42:33Z thoar $ +# DART $Id$ # --------------------- # Purpose @@ -132,7 +132,7 @@ # ============================================================================== # The standard CESM short-term archiving script may need to be altered -# to archive addtional or subsets of things, or to reduce the amount of +# to archive additional or subsets of things, or to reduce the amount of # data that is sent to the long-term archive. Put a version of st_archive.sh # in ${DARTROOT}/models/CESM/shell_scripts when/if necessary @@ -177,7 +177,7 @@ ${COPY} ${DARTROOT}/models/POP/work/perfect_model_obs ${EXEROOT}/. ${COPY} ${DARTROOT}/models/POP/work/input.nml ${CASEROOT}/. -# The first time this is executed, the update namlist script does not exist. +# The first time this is executed, the update namelist script does not exist. if [[ -x update_dart_namelists ]]; then ./update_dart_namelists @@ -226,7 +226,7 @@ # # If it is requested and is not present ... it is an error. # -# The sampling error correction is a lookup table. Each ensemble size +# The sampling error correction is a lookup table. Each ensemble size # has its own (static) file. It is only needed if any # input.nml:&assim_tools_nml:sampling_error_correction = .true., #========================================================================= @@ -265,7 +265,7 @@ end # ============================================================================== -# Initial setup for the default inflation scenario. +# INFLATION : Initial setup for the default inflation scenario. # ============================================================================== # POP may use adaptive state-space prior inflation. The initial settings # are in the filter_nml and ... during an assimilation experiment, the output @@ -273,7 +273,7 @@ # it is useful to specify an initial file of inflation values for the first # assimilation step. However, I can think of no general way to do this. The # utility that creates the initial inflation values (fill_inflation_restart) -# needs the model size from model_mod. To get that, DART/POP needs +# needs the model size from model_mod. To get that, DART/POP needs # files which we generally don't have at this stage of the game (it exists after # a model advance). So ... until I think of something better ... I am making a # cookie file that indicates this is the very first assimilation. If this @@ -281,6 +281,9 @@ # file before it performs the assimilation. After the first assimilation takes @@ Diff output truncated at 40000 characters. @@ From nancy at ucar.edu Thu Oct 31 15:04:48 2013 From: nancy at ucar.edu (nancy at ucar.edu) Date: Thu, 31 Oct 2013 15:04:48 -0600 Subject: [Dart-dev] [6557] DART/trunk/models: Integrated the best comments from each of the files into all of the files. Message-ID: Revision: 6557 Author: thoar Date: 2013-10-31 15:04:48 -0600 (Thu, 31 Oct 2013) Log Message: ----------- Integrated the best comments from each of the files into all of the files. whitespace and formatting changes only, no executable changes. Modified Paths: -------------- DART/trunk/models/POP/shell_scripts/perfect_model.csh DART/trunk/models/cam/shell_scripts/perfect_model.csh DART/trunk/models/clm/shell_scripts/perfect_model.csh -------------- next part -------------- Modified: DART/trunk/models/POP/shell_scripts/perfect_model.csh =================================================================== --- DART/trunk/models/POP/shell_scripts/perfect_model.csh 2013-10-31 20:09:53 UTC (rev 6556) +++ DART/trunk/models/POP/shell_scripts/perfect_model.csh 2013-10-31 21:04:48 UTC (rev 6557) @@ -50,7 +50,7 @@ endsw #------------------------------------------------------------------------- -# Determine time of model state ... from file name of first member +# Determine time of model state ... from file name # of the form "./${MYCASE}.pop.r.2000-01-06-00000.nc" # # Piping stuff through 'bc' strips off any preceeding zeros. @@ -72,7 +72,7 @@ echo "valid time of model is $OCN_YEAR $OCN_MONTH $OCN_DAY $OCN_HOUR (hours)" #------------------------------------------------------------------------- -# Create temporary working directory for the assimilation and go there +# Create temporary working directory for the perfect model and go there #------------------------------------------------------------------------- set temp_dir = pmo_pop @@ -116,7 +116,6 @@ # &perfect_model_obs_nml: last_obs_days = -1, # &perfect_model_obs_nml: last_obs_seconds = -1, # &pop_to_dart_nml: pop_to_dart_output_file = 'dart_ics' -# #========================================================================= if ( ! -e ${CASEROOT}/input.nml ) then @@ -134,9 +133,9 @@ ${LINK} ../$OCN_NML_FILENAME pop_in #========================================================================= -# Block 2: convert 1 pop restart file to a DART initial conditions file. -# At the end of the block, we have a DART restart file perfect_ics -# that came from the pointer file ../rpointer.ocn.restart +# Block 2: Convert 1 POP restart file to a DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from the contents of the pointer file ../rpointer.ocn.restart #========================================================================= echo "`date` -- BEGIN POP-TO-DART" @@ -153,9 +152,11 @@ #========================================================================= # Block 3: Advance the model and harvest the synthetic observations. -# Will result in a single file : 'perfect_restart' which we don't need -# for a perfect model experiment with CESM. -# +# output files are: +# True_state.nc ...... the DART state +# obs_seq.perfect ...... the synthetic observations +# dart_log.out ...... run-time output of all DART routines +# perfect_restart ...... which we don't need #========================================================================= echo "`date` -- BEGIN POP PERFECT_MODEL_OBS" @@ -175,7 +176,7 @@ echo "`date` -- END POP PERFECT_MODEL_OBS" #========================================================================= -# Block 4: Update the pop restart files. +# Block 4: Update the pop restart file #========================================================================= # not needed ... perfect_model_obs does not update the model state. @@ -186,6 +187,7 @@ # Eat the cookie regardless ${REMOVE} ../pop_inflation_cookie +${REMOVE} perfect_ics dart_log.nml echo "`date` -- END GENERATE POP TRUE STATE" Modified: DART/trunk/models/cam/shell_scripts/perfect_model.csh =================================================================== --- DART/trunk/models/cam/shell_scripts/perfect_model.csh 2013-10-31 20:09:53 UTC (rev 6556) +++ DART/trunk/models/cam/shell_scripts/perfect_model.csh 2013-10-31 21:04:48 UTC (rev 6557) @@ -104,7 +104,7 @@ # The observation file names have a time that matches the stopping time of CAM. #----------------------------------------------------------------------------- -set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` +set YYYYMM = `printf %04d%02d ${ATM_YEAR} ${ATM_MONTH}` set OBSFNAME = `printf obs_seq%04d%02d%02d%02d ${ATM_YEAR} ${ATM_MONTH} ${ATM_DAY} ${ATM_HOUR}` set OBS_FILE = ${BASEOBSDIR}/${YYYYMM}_6H/${OBSFNAME} @@ -117,78 +117,82 @@ endif #========================================================================= -# Block 1: Populate a run-time directory with the input needed to run DART, -# and Block 2: Convert 1 CAM restart file to DART initial conditions file. -# At the end of the block, we have DART initial condition file perfect_ics -# that came from pointer file ../rpointer.atm +# Block 1: Populate a run-time directory with the input needed to run DART. # -# REQUIRED DART namelist settings: +# DART namelist settings required: # &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' +# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' +# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' +# &perfect_model_obs_nml: init_time_days = -1, +# &perfect_model_obs_nml: init_time_seconds = -1, +# &perfect_model_obs_nml: first_obs_days = -1, +# &perfect_model_obs_nml: first_obs_seconds = -1, +# &perfect_model_obs_nml: last_obs_days = -1, +# &perfect_model_obs_nml: last_obs_seconds = -1, # &cam_to_dart_nml: cam_to_dart_output_file = 'dart_ics' #========================================================================= -echo "`date` -- BEGIN CAM-TO-DART" +if ( ! -e ${CASEROOT}/input.nml ) then + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" + exit -2 +endif - if ( ! -e ${CASEROOT}/input.nml ) then - echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" - echo "ERROR ... DART required file ${CASEROOT}/input.nml not found ... ERROR" - exit -2 - endif +sed -e "s#dart_ics#perfect_ics#" < ${CASEROOT}/input.nml >! input.nml - # Turns out the .h0. files are timestamped with the START of the - # run, which is *not* ATM_DATE_EXT ... I just link to a whatever - # is convenient (since the info is static). - # make sure there are no old output logs hanging around - $REMOVE output.cam_to_dart +# Turns out the .h0. files are timestamped with the START of the +# run, which is *not* ATM_DATE_EXT ... I just link to a whatever +# is convenient (since the info is static). - set ATM_INITIAL_FILENAME = "../${CASE}.cam.i.${ATM_DATE_EXT}.nc" - set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` - set DART_IC_FILENAME = perfect_ics +set ATM_INITIAL_FILENAME = "../${CASE}.cam.i.${ATM_DATE_EXT}.nc" +set ATM_HISTORY_FILENAME = `ls -1t ../${CASE}.cam*.h0.* | head -n 1` - sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/input.nml >! input.nml +${LINK} $ATM_INITIAL_FILENAME caminput.nc +${LINK} $ATM_HISTORY_FILENAME cam_phis.nc - ${LINK} $ATM_INITIAL_FILENAME caminput.nc - ${LINK} $ATM_HISTORY_FILENAME cam_phis.nc +#========================================================================= +# Block 2: Convert 1 CAM restart file to a DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics +# that came from the contents of the pointer file ../rpointer.atm +#========================================================================= - ${EXEROOT}/cam_to_dart >! output.cam_to_dart +echo "`date` -- BEGIN CAM-TO-DART" - if ($status != 0) then - echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" - echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" - exit -6 - endif +${EXEROOT}/cam_to_dart +if ($status != 0) then + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + echo "ERROR ... DART died in 'cam_to_dart' ... ERROR" + exit -3 +endif + echo "`date` -- END CAM-TO-DART" #========================================================================= -# Block 3: Run perfect_model_obs and harvest the synthetic observations -# and diagnostic files. -# -# DART namelist settings required: -# &perfect_model_obs_nml: async = 0, -# &perfect_model_obs_nml: adv_ens_command = "no_advance_script", -# &perfect_model_obs_nml: output_restart = .false., -# &perfect_model_obs_nml: restart_in_file_name = 'perfect_ics' -# &perfect_model_obs_nml: restart_out_file_name = 'not_created' -# &perfect_model_obs_nml: obs_sequence_in_name = 'obs_seq.in' -# &perfect_model_obs_nml: obs_sequence_out_name = 'obs_seq.perfect' -# &perfect_model_obs_nml: init_time_days = -1, -# &perfect_model_obs_nml: init_time_seconds = -1, -# &perfect_model_obs_nml: first_obs_days = -1, -# &perfect_model_obs_nml: first_obs_seconds = -1, -# &perfect_model_obs_nml: last_obs_days = -1, -# &perfect_model_obs_nml: last_obs_seconds = -1, -# +# Block 3: Advance the model and harvest the synthetic observations. +# output files are: +# True_state.nc ...... the DART state +# obs_seq.perfect ...... the synthetic observations +# dart_log.out ...... run-time output of all DART routines +# perfect_restart ...... which we don't need #========================================================================= echo "`date` -- BEGIN CAM PERFECT_MODEL_OBS" -${EXEROOT}/perfect_model_obs || exit -7 + +${EXEROOT}/perfect_model_obs + +if ($status != 0) then + echo "ERROR ... DART died in 'perfect_model_obs' ... ERROR" + echo "ERROR ... DART died in 'perfect_model_obs' ... ERROR" + exit -4 +endif + +${MOVE} True_State.nc ../cam_True_State.${ATM_DATE_EXT}.nc +${MOVE} obs_seq.perfect ../cam_obs_seq.${ATM_DATE_EXT}.perfect +${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out + echo "`date` -- END CAM PERFECT_MODEL_OBS" -${MOVE} True_State.nc ../cam_True_State.${ATM_DATE_EXT}.nc -${MOVE} obs_seq.perfect ../cam_obs_seq.${ATM_DATE_EXT}.perfect -${MOVE} dart_log.out ../cam_dart_log.${ATM_DATE_EXT}.out - #========================================================================= # Block 4: Update the cam restart file #========================================================================= @@ -201,15 +205,18 @@ cd ${RUNDIR} - set ATM_INITIAL_FILENAME = ${CASE}.cam.i.${ATM_DATE_EXT}.nc +set ATM_INITIAL_FILENAME = ${CASE}.cam.i.${ATM_DATE_EXT}.nc - ${LINK} ${ATM_INITIAL_FILENAME} cam_initial.nc || exit -9 +${LINK} ${ATM_INITIAL_FILENAME} cam_initial.nc || exit -5 - #------------------------------------------------------------------------- # Cleanup #------------------------------------------------------------------------- +# Eat the cookie regardless +${REMOVE} ../cam_inflation_cookie +${REMOVE} perfect_ics dart_log.nml + echo "`date` -- END GENERATE CAM TRUE STATE" exit 0 Modified: DART/trunk/models/clm/shell_scripts/perfect_model.csh =================================================================== --- DART/trunk/models/clm/shell_scripts/perfect_model.csh 2013-10-31 20:09:53 UTC (rev 6556) +++ DART/trunk/models/clm/shell_scripts/perfect_model.csh 2013-10-31 21:04:48 UTC (rev 6557) @@ -70,7 +70,7 @@ endsw #------------------------------------------------------------------------- -# Determine time of model state ... from file name of first member +# Determine time of model state ... from file name # of the form "./${CASE}.clm2.r.2000-01-06-00000.nc" # # Piping stuff through 'bc' strips off any preceeding zeros. @@ -90,7 +90,7 @@ echo "valid time of model is $LND_YEAR $LND_MONTH $LND_DAY $LND_HOUR (hours)" #------------------------------------------------------------------------- -# Create temporary working directory for the assimilation and go there +# Create temporary working directory for the perfect model and go there #------------------------------------------------------------------------- set temp_dir = pmo_clm @@ -148,7 +148,6 @@ # &perfect_model_obs_nml: last_obs_days = -1, # &perfect_model_obs_nml: last_obs_seconds = -1, # &clm_to_dart_nml: clm_to_dart_output_file = 'dart_ics' -# #========================================================================= if ( ! -e ${CASEROOT}/input.nml ) then @@ -157,6 +156,8 @@ exit -2 endif +sed -e "s#dart_ics#perfect_ics#" < ${CASEROOT}/input.nml >! input.nml + # DART/CLM routines all need a clm_restart.nc, clm_history.nc, etc. # The flux tower forward operator looks for a CLM history file with # an instance number in the filename. @@ -165,7 +166,6 @@ set LND_HISTORY_FILENAME = ${CASE}.clm2.h0.${LND_DATE_EXT}.nc set OBS1_HISTORY_FILENAME = ${CASE}.clm2.h1.${LND_DATE_EXT}.nc set OBS2_HISTORY_FILENAME = ${CASE}.clm2_0001.h1.${LND_DATE_EXT}.nc -set DART_IC_FILENAME = perfect_ics ${LINK} ../$LND_RESTART_FILENAME clm_restart.nc ${LINK} ../$LND_HISTORY_FILENAME clm_history.nc @@ -174,14 +174,14 @@ ${LINK} ../$OBS1_HISTORY_FILENAME $OBS2_HISTORY_FILENAME endif -sed -e "s#dart_ics#${DART_IC_FILENAME}#" < ${CASEROOT}/input.nml >! input.nml - #========================================================================= -# Block 2: convert 1 clm restart file to a DART initial conditions file. -# At the end of the block, we have a DART file "perfect_ics" +# Block 2: Convert 1 CLM restart file to a DART initial conditions file. +# At the end of the block, we have DART initial condition file perfect_ics # that came from the contents of the pointer file ../rpointer.lnd #========================================================================= +echo "`date` -- BEGIN CLM-TO-DART" + # patch the CLM restart files to ensure they have the proper # _FillValue and missing_value attributes. # ncatted -O -a _FillValue,frac_sno,o,d,1.0e+36 clm_restart.nc @@ -195,8 +195,6 @@ # ncatted -O -a _FillValue,T_SOISNO,o,d,1.0e+36 clm_restart.nc # ncatted -O -a missing_value,T_SOISNO,o,d,1.0e+36 clm_restart.nc -echo "`date` -- BEGIN CLM-TO-DART" - ${EXEROOT}/clm_to_dart if ($status != 0) then @@ -209,14 +207,14 @@ #========================================================================= # Block 3: Advance the model and harvest the synthetic observations. -# output iles are: +# output files are: # True_state.nc ...... the DART state # obs_seq.perfect ...... the synthetic observations # dart_log.out ...... run-time output of all DART routines # perfect_restart ...... which we don't need #========================================================================= -echo "`date` -- BEGIN PERFECT_MODEL_OBS" +echo "`date` -- BEGIN CLM PERFECT_MODEL_OBS" ${EXEROOT}/perfect_model_obs @@ -230,10 +228,10 @@ ${MOVE} obs_seq.perfect ../clm_obs_seq.${LND_DATE_EXT}.perfect ${MOVE} dart_log.out ../clm_dart_log.${LND_DATE_EXT}.out -echo "`date` -- END PERFECT_MODEL_OBS" +echo "`date` -- END CLM PERFECT_MODEL_OBS" #========================================================================= -# Block 4: Update the clm restart files. +# Block 4: Update the clm restart file #========================================================================= # not needed ... perfect_model_obs does not update the CLM model state.