[Go-essp-tech] DRS v1.0 and CMOR2.x

[Karl Taylor]

Dear all,

The filenames called for by the DRS and output by CMOR2 (recent 
versions) are the same and unchanged.  Right?

I'm not absolutely certain that the quoted section of the DRS is 
accurate.  My latest understanding is that the "ESG version will usually 
be assigned by the ESG publisher, and it cannot be guaranteed that it 
always will be consistent with version number appearing in the ESGF data 
node directory structure for the data being published.  In many cases 
the directory structure will be generated some days prior to ESG 
publication, so the dates may differ."

Does anyone disagree with this?

In any case I think it's much too late to change the directory structure 
created by CMOR2 (which, by the way, is optional).

Note that drslib has functionality to transform from any directory 
structure whatsoever to the ESGF data node directory structure called 
for by the DRS.  Also note that the ESG publisher doesn't rely on the 
directory structure being consistent with DRS, although we think it is 
important that it be so.

regards,
Karl

On 11/10/10 6:51 AM, Sébastien Denvil wrote:
> Dear all,
>
> I have seen that version v1.0 of the DRS has been released
> http://BLOCKEDcmip-pcmdi.llnl.gov/cmip5/output_req.html?submenuheader=2#req_format
> which is very good.
>
> Reading it, it's clear that CMOR2.x will output files to a directory
> structure (but not a filename encoding) mapping the old DRS version
> (v0.27).

> My question is, what is the strong rational behind that fact? Can't
> CMOR2.x follow the version v1.0 DRS and be natively compatible with the
> ESGF data node directory structure?
>
> The version number is apparently the reason.
> <activity>/<product>/<institute>/<model>/<experiment>/<frequency>/<modeling
> realm>/<MIP table>/<ensemble member>/<version number>/<variable name>/
> <CMOR filename>.nc
>
> But quoting the DRS:
> "Note that the version number assigned to the dataset by ESG is supposed
> to reflect the date of ESG publication, but the version will usually be
> assigned by the user so this cannot generally be guaranteed. The user
> will be instructed to provide ESG with the date that appears in the ESGF
> data node directory structure for the dataset being published. In many
> cases the directory structure will be generated some days prior to
> publication, so the date will not in fact reflect the date of
> publication, but the date that the directory structure was created."
>
> In fine, the user will decide/provide the version number. So I think the
> user should do that via CMOR2.x.
>
> CMOR2.x could use the current date as a version number. Or preferably,
> because cmor post-processing can last few days for a MIP_TABLE, be
> assigned to CMOR2.x via an option by the user. This would guarantee that
> all variables belonging to the same "stream" belongs to the same version
> number.
>
> As we all hold the presses waiting for the CMOR2.5 release to continue
> the production, it could be a good timing to have CMOR2.x compatible
> with DRS v1.0. Depending on the CMIP5 data distribution system "big
> opening day" I'm willing to rewrite now what we have to be from the
> beginning in line with CMOR2.x, DRS v1.0 and publication aspect.
>
> Regards.
> Sébastien
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20101110/5a7bffcc/attachment.html