[Go-essp-tech] NCC piControl QC L3 checks

Thu Nov 10 04:44:15 MST 2011

  Hi, Bryan, hi, Martin,

I agree that we should point to the data replicas at our three centers 
from our DOI landing page. Because of the delayed publication process 
and since we have a copy of the NCC data at DKRZ, we decided to proceed 
and expect the data to be published at PCMDI and BADC in near future.
We point on the DOI landing page to a concrete version of each 
publication unit in the three gateways. If PCMDI or BADC  point to the 
NCC DN, we have the DKRZ copy to fulfill our contract with our datacite 
registration agency TIB (i.e. guarantee data accessibility). We would 
not alter that links, but advise the user to try another location if he 
gets an error at one of them and trust you at BADC and PCMDI that you 
will get and keep these DOI datasets accessible.

The time for data replication seems to be unclear. I thought it would 
start after QC L2 assignment. We set up the service 'checkedEsgDatasets' 
for the ESGF partners to provide a list of publication units to be 
replicated, since there was no other infrastructure component which 
harvested QC CIM documents as agreed some time ago. We can alter that 
small service if additional information is required.

Cheers,
Martina

On 10.11.2011 12:03, Bryan Lawrence wrote:
> Hi Martin
>
> I totally agree: the DOI should not point to the copy held at the originating centre - that's analogous to the author's preprint ...
>
> I think the DOI should point to replicants in one or more of DKRZ, PCMDI and BADC, since we are the ones that have committed to holding the data long term. I think it's fine, given where we are now, to issue a DOI provided there is a replicant in at least one of the above, but the landing page should be updated as the replicants are made available.
>
> Cheers
> Bryan
>
>> Hello,
>>
>> This is a bit of a digression, but just looking at these URLs, which I haven't really considered before, makes me realise how badly broken our QC/DOI publishing system is.
>>
>> The intention always was that data would be first published from a data node to a gateway, much of the data would then be moved to a QC centre, quality controlled and republished. We have to have a different URL for data which is published to BADC from elsewhere and data which is copied to BADC and republished -- this obviously needs a fairly fundamental change to the system.
>>
>> On the links below, the PCMDI link refers to data held at NCC -- I don't think we want to link to this from the DOI page, as we have no control over or knowledge of their curation policy (though I'm sure it is as good as any other modeling center). Again, I think the intention was clear, that the DOI would point to replicated data, not to the widely distributed archive,
>>
>> cheers,
>> Martin
>>
>> ________________________________
>> From: Pascoe, Stephen (STFC,RAL,RALSP)
>> Sent: 10 November 2011 10:25
>> To: 'Martina Stockhause'
>> Cc: go-essp-tech at ucar.edu; Juckes, Martin (STFC,RAL,RALSP); Lawrence, Bryan (STFC,RAL,RALSP); Frank Toussaint
>> Subject: RE: NCC piControl QC L3 checks
>>
>> Martina,
>>
>> The NorESM1-M piControl data has not been transferred to BADC therefore it will not be a quick fix to make it available through BADC.  We have some historical and rcp45 data on disk but I don't know whether they are ready for publishing as replicas yet.
>>
>> I will liaise with Martin to verify the status of our NorESM1-M data and accelerate transfer of the piControl data.  However, being realistic, I doubt we can publish the replicas of piControl before the end of the year.
>>
>> Cheers,
>> Stephen.
>>
>> ---
>> Stephen Pascoe  +44 (0)1235 445980
>> Centre of Environmental Data Archival
>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>>
>> From: Martina Stockhause [mailto:stockhause at dkrz.de]
>> Sent: 10 November 2011 07:44
>> To: Pascoe, Stephen (STFC,RAL,RALSP)
>> Cc: go-essp-tech at ucar.edu; Juckes, Martin (STFC,RAL,RALSP); Lawrence, Bryan (STFC,RAL,RALSP); Frank Toussaint
>> Subject: NCC piControl QC L3 checks
>>
>> Good morning, Stephen,
>>
>> we have our first author approval from NCC for their piControl data. During QC L3 double and cross-checks we discovered the following problems:
>>
>> 1. NCC piControl data are not accessible at BADC. Here one example publication unit:
>>
>> WDCC/DKRZ:
>> http://ipcc-ar5.dkrz.de/dataset/cmip5.output1.NCC.NorESM1-M.piControl.3hr.atmos.3hr.r1i1p1.html
>> PCMDI:
>> http://pcmdi3.llnl.gov/esgcet/dataset/cmip5.output1.NCC.NorESM1-M.piControl.3hr.atmos.3hr.r1i1p1.html
>> BADC (Error):
>> http://cmip-gw.badc.rl.ac.uk/dataset/cmip5.output1.NCC.NorESM1-M.piControl.3hr.atmos.3hr.r1i1p1.html
>>
>> 2. CIM (http://q.cmip5.ceda.ac.uk/): NCC has confirmed that they have completed the questionnaire, but there is no CIM simulation document in the atom feed. We will leave out the cross-checks between TDS metadata and CIM metadata in our checks. Connected with that we cannot link to CIM metadata from the DOI landing page, even if the CIM viewer or portal will be in place in future.
>>
>> We decided to proceed with the DOI data publication process.
>> Stephen,  could you solve the data access problem at BADC in the meantime, please?
>>
>> For data that we are about to start QC L3 please refer to the 'QCL2_assigned' experiments at: http://cera-www.dkrz.de/WDCC/CMIP5/QCStatus.jsp
>> or for publication units of QC level 2 to the list at:
>> http://cera-www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets
>> (result restriction e.g. to NCC piControl data by: http://cera-www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets?dataset=<http://cera-www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets?dataset=cmip5.output1.IPSL.IPSL-CM5A-LR.aqua4K>cmip5.output1.NCC.NorESM1-M.piControl)<http://cmip-gw.badc.rl.ac.uk/dataset/cmip5.output1.NCC.NorESM1-M.piControl.3hr.atmos.3hr.r1i1p1.html>
>>
>> Thanks,
>> Frank and Martina
>>
>>
> --
> Bryan Lawrence
> University of Reading:  Professor of Weather and Climate Computing.
> National Centre for Atmospheric Science: Director of Models and Data.
> STFC: Director of the Centre for Environmental Data Archival.
> Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech