[Go-essp-tech] DRS v1.0 and CMOR2.x
Sébastien Denvil
sebastien.denvil at ipsl.jussieu.fr
Wed Nov 10 16:24:51 MST 2010
Thanks all for the prompt answer and clarification.
Le 10/11/2010 20:59, Karl Taylor a écrit :
> Dear all,
>
> The filenames called for by the DRS and output by CMOR2 (recent
> versions) are the same and unchanged. Right?
>
> I'm not absolutely certain that the quoted section of the DRS is
> accurate. My latest understanding is that the "ESG version will
> usually be assigned by the ESG publisher, and it cannot be guaranteed
> that it always will be consistent with version number appearing in the
> ESGF data node directory structure for the data being published. In
> many cases the directory structure will be generated some days prior
> to ESG publication, so the dates may differ."
>
> Does anyone disagree with this?
That's clearer. So the DRS needs some adaptation concerning the version
number "issuer" ; the publisher is the real driver. It implies that if
we publish a simulation in several streams so at different dates (many
reason to support that) they will have different version number. I think
loudly just to put myself in situation.
> In any case I think it's much too late to change the directory
> structure created by CMOR2 (which, by the way, is optional).
Ok.
> Note that drslib has functionality to transform from any directory
> structure whatsoever to the ESGF data node directory structure called
> for by the DRS. Also note that the ESG publisher doesn't rely on the
> directory structure being consistent with DRS, although we think it is
> important that it be so.
Yes we think as well it's important that at the filesystem level we
stick to the ESG-F datanode DRS. To organize multi model / multi
experiment analysis by AR5 author and by institute researcher this is
important for every group. We will use drslib in the production system then.
I still have questions for Martin about the different "derived product
class" and their link with "raw" CMIP5 output. I will discuss that first
with Martin "offline" and may get back "online" to enhance my understanding.
Regards.
Sébastien
> regards,
> Karl
>
> On 11/10/10 6:51 AM, Sébastien Denvil wrote:
>> Dear all,
>>
>> I have seen that version v1.0 of the DRS has been released
>> http://BLOCKEDcmip-pcmdi.llnl.gov/cmip5/output_req.html?submenuheader=2#req_format
>> which is very good.
>>
>> Reading it, it's clear that CMOR2.x will output files to a directory
>> structure (but not a filename encoding) mapping the old DRS version
>> (v0.27).
>>
>
>> My question is, what is the strong rational behind that fact? Can't
>> CMOR2.x follow the version v1.0 DRS and be natively compatible with the
>> ESGF data node directory structure?
>>
>> The version number is apparently the reason.
>> <activity>/<product>/<institute>/<model>/<experiment>/<frequency>/<modeling
>> realm>/<MIP table>/<ensemble member>/<version number>/<variable name>/
>> <CMOR filename>.nc
>>
>> But quoting the DRS:
>> "Note that the version number assigned to the dataset by ESG is supposed
>> to reflect the date of ESG publication, but the version will usually be
>> assigned by the user so this cannot generally be guaranteed. The user
>> will be instructed to provide ESG with the date that appears in the ESGF
>> data node directory structure for the dataset being published. In many
>> cases the directory structure will be generated some days prior to
>> publication, so the date will not in fact reflect the date of
>> publication, but the date that the directory structure was created."
>>
>> In fine, the user will decide/provide the version number. So I think the
>> user should do that via CMOR2.x.
>>
>> CMOR2.x could use the current date as a version number. Or preferably,
>> because cmor post-processing can last few days for a MIP_TABLE, be
>> assigned to CMOR2.x via an option by the user. This would guarantee that
>> all variables belonging to the same "stream" belongs to the same version
>> number.
>>
>> As we all hold the presses waiting for the CMOR2.5 release to continue
>> the production, it could be a good timing to have CMOR2.x compatible
>> with DRS v1.0. Depending on the CMIP5 data distribution system "big
>> opening day" I'm willing to rewrite now what we have to be from the
>> beginning in line with CMOR2.x, DRS v1.0 and publication aspect.
>>
>> Regards.
>> Sébastien
>>
>>
--
Sébastien Denvil
IPSL, Pôle de modélisation du climat
UPMC, Case 101, 4 place Jussieu,
75252 Paris Cedex 5
Tour 45-55 2ème étage Bureau 209
Tel: 33 1 44 27 21 10
Fax: 33 1 44 27 39 02
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20101111/4dcc2dbc/attachment-0001.html
More information about the GO-ESSP-TECH
mailing list