[Go-essp-tech] [esg-node-dev] Re: Documenting CMIP5 data upgrades: the DRSCOMM application

[Estanislao Gonzalez]

Hey Luca,

no it's not. (Ag can correct me if I'm wrong, but in the web page states 
it must be manually added).

You have a point there, and indeed it shouldn't be that difficult to 
implement. The datasets need to have a comment associated to it (and not 
only the latest version as it currently is) and the catalogs will need 
to be recreated. But this might take some time, so I guess this still 
is, in my opinion, a very good interim solution.

Once the esgpublisher can attach a comment to any dataset-version it 
will be quite simple to upload data from DRSCOMM to the esgpublisher DB 
by the corresponding publisher.
The DRSCOMM could also in the meantime "extract" the comments from the 
Catalogs quite easily (the publisher just gives the url of the master 
TDS, and the TDS get's scanned for comments, if found, the appropriate 
entry get's inserted into the DB).
In the end I would expect a service at the Gateway, just as DRSCOMM 
works. And another using the notification system (if a comment gets 
changed, it's emailed at all people who have downloaded a file within them)

That might be a possible "evolution" path. I wouldn't wait until it's 
done though, as users need that information already. But I concur, this 
meta-data belongs in the Catalogs.

Thanks,
Estani


Am 12.09.2011 16:03, schrieb Cinquini, Luca (3880):
> Hi Ag,
> 	I also agree that the application is very, very nice... I am not completely clear where the information is coming from, is it from parsing the THREDDS catalogs, or from the data node databases ? All versioning information should be available from the data node databases...
>
> thanks, Luca
>
> P.S.: as a future possible work... we are also working on a django-based application for managing projects, searches and data, which connects to the ESGF services... maybe there's some good possibilities for integration there ?
>
>
>
> On Sep 12, 2011, at 7:58 AM,<ag.stephens at stfc.ac.uk>  wrote:
>
>> Dear Estani,
>>
>> When we thought about solving the problem for our local node, it seemed to make sense to provide registration that could be extended to the entire ESG federation. Therefore, as a lightweight solution, you are very welcome to make use of the version running at BADC in order to record comments about version changes at your node. It might actually be beneficial (to users) to only run one instance of this application so that they can search the entire federation from a single interface.
>>
>> However, the code is very portable (thanks to Django) if we decide it should be deployed in different places.
>>
>> Kind regards,
>>
>> Ag
>>
>>
>> ________________________________________
>> From: go-essp-tech-bounces at ucar.edu [go-essp-tech-bounces at ucar.edu] on behalf of Estanislao Gonzalez [gonzalez at dkrz.de]
>> Sent: 12 September 2011 14:37
>> To: Pascoe, Stephen (STFC,RAL,RALSP)
>> Cc: Heinrich Widmann; go-essp-tech at ucar.edu; esg-node-dev at lists.llnl.gov; Jörg Wegner
>> Subject: Re: [Go-essp-tech] Documenting CMIP5 data upgrades: the DRSCOMM        application
>>
>> Hi Stephen,
>>
>> this looks great. It's something we will want to embed into the Gateway at some point, but it's definitely a good place to get started.
>> We already were thinking about were to store this information...
>>
>> Thanks,
>> Estani
>>
>> Am 12.09.2011 14:48, schrieb stephen.pascoe at stfc.ac.uk:<mailto:stephen.pascoe at stfc.ac.uk:>
>>
>> Hi all,
>>
>>
>>
>> User feedback and BADC's experience with managing new versions of datasets have convinced us that we need a mechanism for recording when and why data is retracted and upgraded in the CMIP5 system.  To this end Ag Stephen's at BADC has developed a simple web application to record dataset version changes across the federation.
>>
>>
>>
>> Take a look at http://cmip-ingest1.badc.rl.ac.uk/drscomm/
>>
>>
>>
>> The purpose of the application is to provide both data managers and users with a simple interface for adding, finding information and changes to versions based on DRS.  It is a lightweight approach that tries to do the minimum to get the job done with the intention to improve it as needed.  We at BADC will use it to record version upgrades and retractions.  In fact it already contains all our deprecations and the version adjustments announced today.  However, to be most effective it should be used by all the CMIP5 datanodes.  The front page describes a simple wget-based programming API.  Soon we plan to release python scripts and an ESG Publisher plugin to allow datanode administrators to add comments as they publish.
>>
>>
>>
>> Feedback is welcome as ever.
>>
>>
>>
>> Cheers,
>>
>> Stephen Pascoe
>>
>>
>> ---
>> Stephen Pascoe  +44 (0)1235 445980
>> Centre of Environmental Data Archival
>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>>
>>
>>
>> --
>> Scanned by iCritical.
>>
>>
>>
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>
>>
>>
>>
>> --
>> Estanislao Gonzalez
>>
>> Max-Planck-Institut für Meteorologie (MPI-M)
>> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>
>> Phone:   +49 (40) 46 00 94-126
>> E-Mail:  gonzalez at dkrz.de<mailto:gonzalez at dkrz.de>
>>
>> -- 
>> Scanned by iCritical.
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>


-- 
Estanislao Gonzalez

Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany

Phone:   +49 (40) 46 00 94-126
E-Mail:  gonzalez at dkrz.de