[Go-essp-tech] DRS v1.0 and CMOR2.x

[Estanislao Gonzalez]

  Hi Martin,

I agree that the main reason is that it's already too late. But IMHO it 
would have been better to integrate it into cmor. Neither the product 
identification nor the versioning is something that the publisher can 
decide by itself, the former because of the configuration which is known 
by the modelers only; the later because, again, only the modelers may 
call it a new version.

I don't think that normally the person responsible for publishing can 
make those decisions; it's true that the person configuring cmor might 
neither have the required knowledge/decision power, but it's certainly 
nearer to the source.

Thanks,
Estani

On 11/10/2010 04:14 PM, martin.juckes at stfc.ac.uk wrote:
> Hello Sebastien,
>
> The "product" identification would be problematic with the approach you suggest: the product identification will be done through the drslib package, which will hopefully be integrated into the ESG data node.
>
> In many cases the identification of the product is easy, but there are a handful which are awkward and involve some dependency on the complete range of data being processed and (if the data is a partial modification of a dataset already published, with the intention of retaining some of the already published files) can involve dependency on data which has already been published. I think that integrating this into CMOR would cause more delay than it is worth at this stage.
>
> Cheers,
> Martin
>
>> -----Original Message-----
>> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-
>> bounces at ucar.edu] On Behalf Of Sébastien Denvil
>> Sent: 10 November 2010 14:51
>> To: cmor at lists.llnl.gov; go-essp-tech at ucar.edu
>> Subject: [Go-essp-tech] DRS v1.0 and CMOR2.x
>>
>> Dear all,
>>
>> I have seen that version v1.0 of the DRS has been released
>> http://cmip-
>> pcmdi.llnl.gov/cmip5/output_req.html?submenuheader=2#req_format
>> which is very good.
>>
>> Reading it, it's clear that CMOR2.x will output files to a directory
>> structure (but not a filename encoding) mapping the old DRS version
>> (v0.27).
>>
>> My question is, what is the strong rational behind that fact? Can't
>> CMOR2.x follow the version v1.0 DRS and be natively compatible with the
>> ESGF data node directory structure?
>>
>> The version number is apparently the reason.
>> <activity>/<product>/<institute>/<model>/<experiment>/<frequency>/<mode
>> ling
>> realm>/<MIP table>/<ensemble member>/<version number>/<variable name>/
>> <CMOR filename>.nc
>>
>> But quoting the DRS:
>> "Note that the version number assigned to the dataset by ESG is
>> supposed
>> to reflect the date of ESG publication, but the version will usually be
>> assigned by the user so this cannot generally be guaranteed. The user
>> will be instructed to provide ESG with the date that appears in the
>> ESGF
>> data node directory structure for the dataset being published. In many
>> cases the directory structure will be generated some days prior to
>> publication, so the date will not in fact reflect the date of
>> publication, but the date that the directory structure was created."
>>
>> In fine, the user will decide/provide the version number. So I think
>> the
>> user should do that via CMOR2.x.
>>
>> CMOR2.x could use the current date as a version number. Or preferably,
>> because cmor post-processing can last few days for a MIP_TABLE, be
>> assigned to CMOR2.x via an option by the user. This would guarantee
>> that
>> all variables belonging to the same "stream" belongs to the same
>> version
>> number.
>>
>> As we all hold the presses waiting for the CMOR2.5 release to continue
>> the production, it could be a good timing to have CMOR2.x compatible
>> with DRS v1.0. Depending on the CMIP5 data distribution system "big
>> opening day" I'm willing to rewrite now what we have to be from the
>> beginning in line with CMOR2.x, DRS v1.0 and publication aspect.
>>
>> Regards.
>> Sébastien
>>
>> --
>> Sébastien Denvil
>> IPSL, Pôle de modélisation du climat
>> UPMC, Case 101, 4 place Jussieu,
>> 75252 Paris Cedex 5
>>
>> Tour 45-55 2ème étage Bureau 209
>> Tel: 33 1 44 27 21 10
>> Fax: 33 1 44 27 39 02
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech


-- 
Estanislao Gonzalez

Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany

Phone:   +49 (40) 46 00 94-126
E-Mail:  estanislao.gonzalez at zmaw.de