[Go-essp-tech] DRS v1.0 and CMOR2.x

[martin.juckes at stfc.ac.uk]

Hello Sebastien,

The "product" identification would be problematic with the approach you suggest: the product identification will be done through the drslib package, which will hopefully be integrated into the ESG data node. 

In many cases the identification of the product is easy, but there are a handful which are awkward and involve some dependency on the complete range of data being processed and (if the data is a partial modification of a dataset already published, with the intention of retaining some of the already published files) can involve dependency on data which has already been published. I think that integrating this into CMOR would cause more delay than it is worth at this stage.

Cheers,
Martin

> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-
> bounces at ucar.edu] On Behalf Of Sébastien Denvil
> Sent: 10 November 2010 14:51
> To: cmor at lists.llnl.gov; go-essp-tech at ucar.edu
> Subject: [Go-essp-tech] DRS v1.0 and CMOR2.x
> 
> Dear all,
> 
> I have seen that version v1.0 of the DRS has been released
> http://cmip-
> pcmdi.llnl.gov/cmip5/output_req.html?submenuheader=2#req_format
> which is very good.
> 
> Reading it, it's clear that CMOR2.x will output files to a directory
> structure (but not a filename encoding) mapping the old DRS version
> (v0.27).
> 
> My question is, what is the strong rational behind that fact? Can't
> CMOR2.x follow the version v1.0 DRS and be natively compatible with the
> ESGF data node directory structure?
> 
> The version number is apparently the reason.
> <activity>/<product>/<institute>/<model>/<experiment>/<frequency>/<mode
> ling
> realm>/<MIP table>/<ensemble member>/<version number>/<variable name>/
> <CMOR filename>.nc
> 
> But quoting the DRS:
> "Note that the version number assigned to the dataset by ESG is
> supposed
> to reflect the date of ESG publication, but the version will usually be
> assigned by the user so this cannot generally be guaranteed. The user
> will be instructed to provide ESG with the date that appears in the
> ESGF
> data node directory structure for the dataset being published. In many
> cases the directory structure will be generated some days prior to
> publication, so the date will not in fact reflect the date of
> publication, but the date that the directory structure was created."
> 
> In fine, the user will decide/provide the version number. So I think
> the
> user should do that via CMOR2.x.
> 
> CMOR2.x could use the current date as a version number. Or preferably,
> because cmor post-processing can last few days for a MIP_TABLE, be
> assigned to CMOR2.x via an option by the user. This would guarantee
> that
> all variables belonging to the same "stream" belongs to the same
> version
> number.
> 
> As we all hold the presses waiting for the CMOR2.5 release to continue
> the production, it could be a good timing to have CMOR2.x compatible
> with DRS v1.0. Depending on the CMIP5 data distribution system "big
> opening day" I'm willing to rewrite now what we have to be from the
> beginning in line with CMOR2.x, DRS v1.0 and publication aspect.
> 
> Regards.
> Sébastien
> 
> --
> Sébastien Denvil
> IPSL, Pôle de modélisation du climat
> UPMC, Case 101, 4 place Jussieu,
> 75252 Paris Cedex 5
> 
> Tour 45-55 2ème étage Bureau 209
> Tel: 33 1 44 27 21 10
> Fax: 33 1 44 27 39 02
> 
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
-- 
Scanned by iCritical.