[Go-essp-tech] [IPSL-CMIP5] Re: Dataset versions across CMIP5 Gateways

Mon Jan 23 07:54:22 MST 2012

Hi All,

could we deal with this either on CMIP5-HelpDesk or via email and email lists.  Trying to do both will confuse the helpdesk system (we've already created 2 queries)

I would recommend doing the latter since we are in regular correspondence.  I've deleted a duplicate query and will close the other shortly.

Stephen.

---
Stephen Pascoe  +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK

From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Sébastien Denvil
Sent: 23 January 2012 14:41
To: Estanislao Gonzalez
Cc: ipsl-cmip5 at ipsl.jussieu.fr; go-essp-tech at ucar.edu; CMIP5-Helpdesk
Subject: Re: [Go-essp-tech] [IPSL-CMIP5] Re: Dataset versions across CMIP5 Gateways

Hi Estani,

Le 23/01/2012 12:40, Estanislao Gonzalez a écrit :
Hi Sébastien,

This is a known problem about replicas. I'm removing all replicas from our system (just from the Gateway) hoping this will get solved. This shoudn't inhibit replication via the BDM, but it will forbid discovery... Anyway, I was waiting/hoping for a solution to this, but I see no other option as to retreat them. Should be ready soon...

Ok, thanks for letting me know. What about PCMDI replicated/published datasets?

Regarding your last point:
> Can you confirm that *not* all users authorised to publish at DKRZ are able to modify this dataset?

Well indeed they can, since there's only one person authorized to published to DKRZ (me) and only one person able to change that authorization (myself) I don't think this is a problem...

Ok, that was my supposition. If it's only you then no problem. In the future it could be that other users can publish to the DKRZ gateway. By that time it would be good to change permissions (just to avoid mistakes).

I guess your question goes more on why am I able to "write" IPSL dataset. Well, please remember that we are talking about replicas, so I can't alter IPSL dataset, but  I can publish a replica. Furthermore, I'm even able to publish wrong information, i.e. another datasets or a corrupt one, and mark it as a replica of IPSLs. This is something we don't really want to happen.

Again it's a matter of trust. I'm sure you perform all the necessary checks to avoid publication of corrupted replicas.

On the other hand, IPSL may remove, alter or do whatever it likes with the "original", and that's again something archives don't want, at least not if "our" copy is treated as the "replica".

Up to know we preserve datasets version and we follow the CMIP5 procedures precisely. The benefit for you is that you have time to define the best replicas/publication strategy.

We do have a lot to define regarding replicas. This is an ongoing conversation, so I'll kindly ask or stakeholders to speak their mind.

I believe gateways should expose all dataset version (especially the last one).

Because it's taking time to replicate/publish it would then mean that the latest version may not have been replicated     but should appear as such in gateways.

It would also mean that the best thing the replication software must achieve is to be able to download only what has changed (based on checksums when available) and following the drslib strategy to build link when nothing has changed.

Thanks.
Sébastien

Thanks,
Estani

Am 23.01.2012 10:33, schrieb Sébastien Denvil:
Dear all,

browsing gateways from PCMDI, BADC and DKRZ using the underlying facets of this dataset I observed a strange behaviour. cmip5.output1.IPSL.IPSL-CM5A-LR.piControl.mon.ocean.Omon.r1i1p1

This dataset have 2 versions, v20110324 and v20111010.

Only the BADC gateway display the latest version. The other two gateways display the old one and never mentioned the existence of a new version. I believe this is a major issue due to replication side effects.

Because there isn't any "version" facet it would be important to make visible every version of a dataset in an homogeneous ways across gateways?

http://cmip-gw.badc.rl.ac.uk/dataset/cmip5.output1.IPSL.IPSL-CM5A-LR.piControl.mon.ocean.Omon.r1i1p1.html
http://pcmdi3.llnl.gov/esgcet/dataset/cmip5.output1.IPSL.IPSL-CM5A-LR.piControl.mon.ocean.Omon.r1i1p1.html
http://ipcc-ar5.dkrz.de/dataset/cmip5.output1.IPSL.IPSL-CM5A-LR.piControl.mon.ocean.Omon.r1i1p1.html

Also selecting the administration tab from the DKRZ gateway I can read the following:
Groups authorized for Writing: Users authorized to publish at DKRZ
Gateway Administrators

Can you confirm that *not* all users authorised to publish at DKRZ are able to modify this dataset?

Regards.
Sébastien

_______________________________________________

GO-ESSP-TECH mailing list

GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>

http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

--

Estanislao Gonzalez

Max-Planck-Institut für Meteorologie (MPI-M)

Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre

Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany

Phone:   +49 (40) 46 00 94-126

E-Mail:  gonzalez at dkrz.de<mailto:gonzalez at dkrz.de>

--

Sébastien Denvil

IPSL, Pôle de modélisation du climat

UPMC, Case 101, 4 place Jussieu,

75252 Paris Cedex 5

Tour 45-55 2ème étage Bureau 209

Tel: 33 1 44 27 21 10

Fax: 33 1 44 27 39 02

-- 
Scanned by iCritical.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120123/bea67243/attachment-0001.html 