[Go-essp-tech] Fwd: RE: Request to record checksums for all CMIP5 filesserved by ESG

[Karl Taylor]

Dear all,

Here are some remarks and questions from Jamie Kettleborough which I 
don't have the answers to.  If anyone can provide some feedback, I'd 
appreciate it.

Best regards,
Karl

this is great - thanks for following this up and making it official policy.
I think there are 'project management' type issues arising from this 
aren't there?  You may have these under control - in which case ignore 
me.  But is anyone tracking which centres are updating with checksums, 
and gently prodding (and helping) centres that are finding it hard?  
(Maybe you are relying on annoying users like me to track who is 
updating...).  Is anyone tracking the 'best' way of publishing new data 
sets?  For instance I have a worry that although it may be slow to add 
checksums at time of publishing this may be the best thing to do - as I 
*think* it may avoid some data node, gateway synchronisation issues.


    ------------------------------------------------------------------------
    *From:* go-essp-tech-bounces at ucar.edu
    [mailto:go-essp-tech-bounces at ucar.edu] *On Behalf Of *Karl Taylor
    *Sent:* 27 January 2012 01:48
    *To:* Andreas Hense; Arun Kumar; Bin Wang; Bryan Lawrence; Jones,
    Chris D; David Salas-Melia; Evgeny Volodin; Gary L. Russell; Gavin
    Schmidt; GFDL.Climate.Model.Info at noaa.gov; Greg Flato; HJ Baek;
    Isaac Held; Marika Holland; Jean-Louis Dufresne; Jerry Meehl;
    Duoying Ji; Jozef Syktus; Julia Slingo; Kamal Puri; Larry Horowitz;
    Laurent Terray; Leon Rotstayn; Marco Giorgetta; Mat Collins; Mats
    Bentsen; Max Suarez; Michio Kawamiya; Qingquan Li; Ronald Stouffer;
    Seita Emori; Seung-Ki Min; Ken Lo; Shoji Kusunoki; Silvio Gualdi;
    Stephanie Legutke; Stephen Jeffrey; Stéphane Senesi; Thomas Stocker;
    Tianjun Zhou; Tom Delworth; Tongwen Wu; Tony Hirst; Toru Nozawa;
    Wenjie Dong; Wilco Hazeleger; Wilhelm May; Won-Tae Kwon; YH Byun;
    Ying Xu; Yongjiu Dai; Zaizhi Wang; Bruce Wyman; Chiara Cagnazzo;
    Ayako Abe-Ouche; Masayoshi Ishii; Trond Iversen; Hyo-Shin Lee;
    Masahiro Watanabe; Seiji Yukimoto; Tomoaki Ose; John Scinocca;
    Sandrine Bony; Fangli Qiao; Ben Kirtman; James B White III (Trey);
    Christophe Cassou; Patrick Tripp; Suranjana Saha; go-essp-tech at ucar.edu
    *Subject:* [Go-essp-tech] Request to record checksums for all CMIP5
    filesserved by ESG

    Dear all,

    This message if for those responsible for CMIP5 data.

    We are attempting to replicate at PCMDI and other major data
    archives a considerable fraction of the model output you have made
    available to CMIP5 scientists.  To assure only genuine replicas are
    served, we want to verify that the checksums are the same at all
    sites.  I am therefore writing to request with some urgency that you
    publish checksums.  For datasets that you have already published,
    you can add the checksum following the instructions found at:

    http://www2-pcmdi.llnl.gov/Members/bdrach/.personal/esg-publication-scripts#publishing-checksums-and-other

    Note that you should make sure you have the latest version of the
    publisher.  (We had to add new capability to the publisher for this
    purpose.)  You will have already created mapfiles before, so it
    should not be difficult to simply add the checksum information you need.

    By the way, if you plan to newly publish additional datasets, there
    is an option to create the checksums during the initial publication
    of the data, but that will, of course, slow down the publication
    script substantially.  If you elect to publish new datasets
    initially without checksums, please add checksums within a few days.

    Not only will the checksums help us in replicating data (which will
    provide better access to the users), certain heavy users of CMIP5
    data have been urging us to provide this information for some time
    now.  We thank you for your hard work in improving CMIP5 data services.

    On another matter, a reminder: If you in any way modify a netCDF
    file that has been published to ESG, please make sure that the
    "tracking_id" (one of the netCDF global attributes) in the file is
    also modified before it is republished.

    thanks and best regards,
    Karl

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120127/4d37e0d4/attachment.html