[Dart-users] asking for input about DART obs_seq_netcdf files

[Tim Hoar]

I've gotten a request asking to change the storage order of the 'copies' in
the netcdf format version of the DART obs_sequence files. The copy order is
described with strings in the CopyMetaData variable. My strategy has always been
to never assume anything about the storage order - this won't affect me or the matlab
scripts (see the get_copy_index(), get_qc_index() matlab functions).

For people who like to hardwire magic numbers into scripts (and who doesn't!)
this makes a potentially profound change. The NCO operators are also geared
to using magic numbers (as far as I can tell), so the time to speak up is NOW.

Presently, the obs error variance is always 'last'. If 
filter_nml:num_output_obs_members > 0, these ensemble members get preserved
right after the 'mandatory' copies:

NCEP BUFR observation  (for example)                                                                                                         
prior ensemble mean                                                                                                              
posterior ensemble mean                                                                                                          
prior ensemble spread                                                                                                            
posterior ensemble spread                                                                                                        
prior ensemble member      1                                                                                                    
posterior ensemble member      1                                                                                                
prior ensemble member      2                                                                                                    
posterior ensemble member      2                                                                                                
prior ensemble member      3                                                                                                    
posterior ensemble member      3                                                                                                
...
observation error variance
<then the QC values follow>

The proposed change is to make it such that the copy number of the observation_error_variance is invariant to the filter_nml:num_output_obs_members setting:

NCEP BUFR observation  (for example)                                                                                                    
prior ensemble mean                                                                                                              
posterior ensemble mean                                                                                                          
prior ensemble spread                                                                                                            
posterior ensemble spread    
observation error variance                                                                                                   
prior ensemble member      1                                                                                                   
posterior ensemble member      1                                                                                               
prior ensemble member      2                                                                                                   
posterior ensemble member      2                                                                                               
prior ensemble member      3                                                                                                   
posterior ensemble member      3                                                                                               
...
<then the QC values follow>

If you're still reading, keep the following in mind: The observation error
variance is probably ALWAYS present, even if you convert obs_seq.out files to
netCDF. In that case (I believe, but am unwilling to check at Friday at 4:30)
that the storage order is:

NCEP BUFR observation  (for example)                                                                                                 
observation error variance                                                                                           
<then the QC values follow>

So in some ways, I'd like to move the observation error variance to copy 2
for complete consistency. If the prior/posterior means/spread exist, they
get appended after that. This would cause the world as we know it to
cease to exist. I know of lots of "other people's scripts" that REQUIRE the prior_ensemble mean copy 2.

As you may have surmised, I have no opinion on the matter ... I'm happy
to follow the group decision. This is a democracy. I'd like to be a benevolent
dictator, but those pesky minions keep proving me wrong.

Tim Hoar, Associate Scientist
National Center for Atmospheric Research
thoar at ucar.edu
303 497 1708

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/dart-users/attachments/20101105/e5fd74d1/attachment.html