[Go-essp-tech] Proposed directory structure and filename structure for in-sit stations (time-series and profiles)

[Steve Hankin]

2 cents:  Applause, applause.   But with some concerns.

Existing files of observations often contain multiple variables.  Would 
these be re-writtten for the CMIP5 archives?   If not .... then since 
the DRS syntax includes the variable name "high up"  (3rd from the top 
-- <Activity>/<obs_realm>/<variable>...), presumably this means that 
there may be multiple DRS paths to the same data file.  If so, this 
should perhaps be discussed in the documentation.

Having <start_date>_<end_date> in the naming pattern suggests that we 
are defining a DRS that describes only time series.  Vertical profiles 
and trajectories have different naming needs.  Also, we are increasingly 
seeing (and hope for more) aggregated virtual datasets of in situ 
observations, where the end date is "the present" or "one month ago".  
These may confound the system unless the concepts are worked into the 
system in advance.

     - Steve

=============================================================

On 2/10/2011 7:17 AM, Christensen, Sigurd W. wrote:
> Folks,
>     Several of us worked yesterday and this morning to develop a proposal for a directory structure and filename structure for "in-situ stations (time-series and profiles)".  This is the second type of observational data identified in the third bullet in the current proposal, from in this link sent Tuesday by Luca:
>
> http://oodt.jpl.nasa.gov/wiki/display/CLIMATE/Data+and+Metadata+Requirements+for+CMIP5+Observational+Datasets
>
>     Here is our proposal:
>
>   --------------------------------------------------------------
>
>                   PROPOSAL FOR DATA REFERENCE SYNTAX (DRS)
>     FOR OBSERVATIONAL DATA FROM IN-SITU STATIONS (TIME-SERIES AND PROFILES)
>            (Field Name specification, followed by two examples)
>
>     Directory structure fields:
> <Activity>/<obs_realm>/<variable>/<frequency>/<type>/<agency>/<project>/
> <instrument or data_product>/<published_version>
>     ARM files example: observations/atmos/ta/1hr/in_situ_sites/usdoe/arm/cmbe/v120110101/
> {files}
>     CDIAC files example: observations/atmos/ta/1hr/in_situ_sites/usdoe/ameriflux/L2/v120110101/
> {files}
>
>     File name fields:
> <variable>_<project>_<location>_><instrument or data_product>
> _<processing_level_and product_version>_<start_date>_<end_date>.ext
>     ARM files example:
>   ta_arm_barrow_cmbe_c1v1.0_20110101_20110209.nc
>     CDIAC files example:
>   ta_ameriflux_barrow_tower_L2v1_20110101_20110209.nc
>
>     We hope this proposal will help extend the metadata discussion more to include in-situ data.
>
>     Thanks,
>     Giri Palanisamy, Misha Krassovski, Renata McCoy, and Sigurd Christensen
>