[Dart-users] asking for input about DART obs_seq_netcdf files

[Soyoung Ha]

Tim,

So you give us three options to choose from for the location of obs  
error variance in the obs sequence file.
1. The original way to put it at the end of all available copies.
2. New method to put it right after prior/posterior mean/std but  
before any ensemble prior/posterior mean/std if any.
3. New new method to move it all the way front right after the  
observation.

I guess this issue came up while you were working on the obs sequence  
files for my ensemble forecasting data. (Thanks!)
Given that we will potentially have more interest in expanding our  
ensemble data assimilation to ensemble forecasting and in obs-space  
verification for the real-obs cases,  the current way (i.e., option 1)  
doesn't look ideal anyway.
If it is time we can make a change, it looks to me more natural to put  
the observation error right after the observation regardless of our  
filter design. And I think it's a good thing to update our hard-wired  
scripts in a more generalized way.
Although option 2 may be a safe choice at present as we don't have  
many users who worked with ensemble observations yet, I would be happy  
to modify my scripts (if any) for option 3.

Soyoung


On Nov 5, 2010, at 4:30 PM, Tim Hoar wrote:

> I've gotten a request asking to change the storage order of the  
> 'copies' in
> the netcdf format version of the DART obs_sequence files. The copy  
> order is
> described with strings in the CopyMetaData variable. My strategy has  
> always been
> to never assume anything about the storage order - this won't affect  
> me or the matlab
> scripts (see the get_copy_index(), get_qc_index() matlab functions).
>
> For people who like to hardwire magic numbers into scripts (and who  
> doesn't!)
> this makes a potentially profound change. The NCO operators are also  
> geared
> to using magic numbers (as far as I can tell), so the time to speak  
> up is NOW.
>
> Presently, the obs error variance is always 'last'. If
> filter_nml:num_output_obs_members > 0, these ensemble members get  
> preserved
> right after the 'mandatory' copies:
>
> NCEP BUFR observation  (for example)
> prior ensemble mean
> posterior ensemble mean
> prior ensemble spread
> posterior ensemble spread
> prior ensemble member      1
> posterior ensemble member      1
> prior ensemble member      2
> posterior ensemble member      2
> prior ensemble member      3
> posterior ensemble member      3
> ...
> observation error variance
> <then the QC values follow>
>
> The proposed change is to make it such that the copy number of the  
> observation_error_variance is invariant to the  
> filter_nml:num_output_obs_members setting:
>
> NCEP BUFR observation  (for example)
> prior ensemble mean
> posterior ensemble mean
> prior ensemble spread
> posterior ensemble spread
> observation error variance
> prior ensemble member      1
> posterior ensemble member      1
> prior ensemble member      2
> posterior ensemble member      2
> prior ensemble member      3
> posterior ensemble member      3
> ...
> <then the QC values follow>
>
> If you're still reading, keep the following in mind: The observation  
> error
> variance is probably ALWAYS present, even if you convert obs_seq.out  
> files to
> netCDF. In that case (I believe, but am unwilling to check at Friday  
> at 4:30)
> that the storage order is:
>
> NCEP BUFR observation  (for example)
> observation error variance
> <then the QC values follow>
>
> So in some ways, I'd like to move the observation error variance to  
> copy 2
> for complete consistency. If the prior/posterior means/spread exist,  
> they
> get appended after that. This would cause the world as we know it to
> cease to exist. I know of lots of "other people's scripts" that  
> REQUIRE the prior_ensemble mean copy 2.
>
> As you may have surmised, I have no opinion on the matter ... I'm  
> happy
> to follow the group decision. This is a democracy. I'd like to be a  
> benevolent
> dictator, but those pesky minions keep proving me wrong.
>
> Tim Hoar, Associate Scientist
> National Center for Atmospheric Research
> thoar at ucar.edu
> 303 497 1708
>
> _______________________________________________
> Dart-users mailing list
> Dart-users at mailman.ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/dart-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/dart-users/attachments/20101108/fbf9d737/attachment.html