[Go-essp-tech] NCC piControl QC L3 checks

Thu Nov 10 07:47:02 MST 2011

  Hallo Martin,

we have altogether 440 publication units of QC level 2, which are all 
published at WDCC/DKRZ.
For the names of the publication units see: 
http://cera-www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets

Cheers,
Martina

On 10.11.2011 12:59, martin.juckes at stfc.ac.uk wrote:
> How much data have you passed at QC L2?
>
> Cheers,
> Martin
>
>>> -----Original Message-----
>>> From: Martina Stockhause [mailto:stockhause at dkrz.de]
>>> Sent: 10 November 2011 11:44
>>> To: Lawrence, Bryan (STFC,RAL,RALSP)
>>> Cc: Juckes, Martin (STFC,RAL,RALSP); go-essp-tech at ucar.edu; Frank
>>> Toussaint
>>> Subject: Re: [Go-essp-tech] NCC piControl QC L3 checks
>>>
>>>   Hi, Bryan, hi, Martin,
>>>
>>> I agree that we should point to the data replicas at our three centers
>> >from our DOI landing page. Because of the delayed publication process
>>> and since we have a copy of the NCC data at DKRZ, we decided to
>>> proceed
>>> and expect the data to be published at PCMDI and BADC in near future.
>>> We point on the DOI landing page to a concrete version of each
>>> publication unit in the three gateways. If PCMDI or BADC  point to the
>>> NCC DN, we have the DKRZ copy to fulfill our contract with our
>>> datacite
>>> registration agency TIB (i.e. guarantee data accessibility). We would
>>> not alter that links, but advise the user to try another location if
>>> he
>>> gets an error at one of them and trust you at BADC and PCMDI that you
>>> will get and keep these DOI datasets accessible.
>>>
>>> The time for data replication seems to be unclear. I thought it would
>>> start after QC L2 assignment. We set up the service
>>> 'checkedEsgDatasets'
>>> for the ESGF partners to provide a list of publication units to be
>>> replicated, since there was no other infrastructure component which
>>> harvested QC CIM documents as agreed some time ago. We can alter that
>>> small service if additional information is required.
>>>
>>> Cheers,
>>> Martina
>>>
>>> On 10.11.2011 12:03, Bryan Lawrence wrote:
>>>> Hi Martin
>>>>
>>>> I totally agree: the DOI should not point to the copy held at the
>>> originating centre - that's analogous to the author's preprint ...
>>>> I think the DOI should point to replicants in one or more of DKRZ,
>>> PCMDI and BADC, since we are the ones that have committed to holding
>>> the data long term. I think it's fine, given where we are now, to
>>> issue a DOI provided there is a replicant in at least one of the
>>> above, but the landing page should be updated as the replicants are
>>> made available.
>>>> Cheers
>>>> Bryan
>>>>
>>>>> Hello,
>>>>>
>>>>> This is a bit of a digression, but just looking at these URLs,
>>> which I haven't really considered before, makes me realise how badly
>>> broken our QC/DOI publishing system is.
>>>>> The intention always was that data would be first published from a
>>> data node to a gateway, much of the data would then be moved to a QC
>>> centre, quality controlled and republished. We have to have a
>>> different URL for data which is published to BADC from elsewhere and
>>> data which is copied to BADC and republished -- this obviously needs a
>>> fairly fundamental change to the system.
>>>>> On the links below, the PCMDI link refers to data held at NCC -- I
>>> don't think we want to link to this from the DOI page, as we have no
>>> control over or knowledge of their curation policy (though I'm sure it
>>> is as good as any other modeling center). Again, I think the intention
>>> was clear, that the DOI would point to replicated data, not to the
>>> widely distributed archive,
>>>>> cheers,
>>>>> Martin
>>>>>
>>>>> ________________________________
>>>>> From: Pascoe, Stephen (STFC,RAL,RALSP)
>>>>> Sent: 10 November 2011 10:25
>>>>> To: 'Martina Stockhause'
>>>>> Cc: go-essp-tech at ucar.edu; Juckes, Martin (STFC,RAL,RALSP);
>>> Lawrence, Bryan (STFC,RAL,RALSP); Frank Toussaint
>>>>> Subject: RE: NCC piControl QC L3 checks
>>>>>
>>>>> Martina,
>>>>>
>>>>> The NorESM1-M piControl data has not been transferred to BADC
>>> therefore it will not be a quick fix to make it available through
>>> BADC.  We have some historical and rcp45 data on disk but I don't know
>>> whether they are ready for publishing as replicas yet.
>>>>> I will liaise with Martin to verify the status of our NorESM1-M
>>> data and accelerate transfer of the piControl data.  However, being
>>> realistic, I doubt we can publish the replicas of piControl before the
>>> end of the year.
>>>>> Cheers,
>>>>> Stephen.
>>>>>
>>>>> ---
>>>>> Stephen Pascoe  +44 (0)1235 445980
>>>>> Centre of Environmental Data Archival
>>>>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11
>>> 0QX, UK
>>>>> From: Martina Stockhause [mailto:stockhause at dkrz.de]
>>>>> Sent: 10 November 2011 07:44
>>>>> To: Pascoe, Stephen (STFC,RAL,RALSP)
>>>>> Cc: go-essp-tech at ucar.edu; Juckes, Martin (STFC,RAL,RALSP);
>>> Lawrence, Bryan (STFC,RAL,RALSP); Frank Toussaint
>>>>> Subject: NCC piControl QC L3 checks
>>>>>
>>>>> Good morning, Stephen,
>>>>>
>>>>> we have our first author approval from NCC for their piControl
>>> data. During QC L3 double and cross-checks we discovered the following
>>> problems:
>>>>> 1. NCC piControl data are not accessible at BADC. Here one example
>>> publication unit:
>>>>> WDCC/DKRZ:
>>>>> http://ipcc-ar5.dkrz.de/dataset/cmip5.output1.NCC.NorESM1-
>>> M.piControl.3hr.atmos.3hr.r1i1p1.html
>>>>> PCMDI:
>>>>> http://pcmdi3.llnl.gov/esgcet/dataset/cmip5.output1.NCC.NorESM1-
>>> M.piControl.3hr.atmos.3hr.r1i1p1.html
>>>>> BADC (Error):
>>>>> http://cmip-gw.badc.rl.ac.uk/dataset/cmip5.output1.NCC.NorESM1-
>>> M.piControl.3hr.atmos.3hr.r1i1p1.html
>>>>> 2. CIM (http://q.cmip5.ceda.ac.uk/): NCC has confirmed that they
>>> have completed the questionnaire, but there is no CIM simulation
>>> document in the atom feed. We will leave out the cross-checks between
>>> TDS metadata and CIM metadata in our checks. Connected with that we
>>> cannot link to CIM metadata from the DOI landing page, even if the CIM
>>> viewer or portal will be in place in future.
>>>>> We decided to proceed with the DOI data publication process.
>>>>> Stephen,  could you solve the data access problem at BADC in the
>>> meantime, please?
>>>>> For data that we are about to start QC L3 please refer to the
>>> 'QCL2_assigned' experiments at: http://cera-
>>> www.dkrz.de/WDCC/CMIP5/QCStatus.jsp
>>>>> or for publication units of QC level 2 to the list at:
>>>>> http://cera-www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets
>>>>> (result restriction e.g. to NCC piControl data by: http://cera-
>>> www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets?dataset=<http://cera-
>>> www.dkrz.de/WDCC/CMIP5/checkedEsgDatasets?dataset=cmip5.output1.IPSL.I
>>> PSL-CM5A-LR.aqua4K>cmip5.output1.NCC.NorESM1-M.piControl)<http://cmip-
>>> gw.badc.rl.ac.uk/dataset/cmip5.output1.NCC.NorESM1-
>>> M.piControl.3hr.atmos.3hr.r1i1p1.html>
>>>>> Thanks,
>>>>> Frank and Martina
>>>>>
>>>>>
>>>> --
>>>> Bryan Lawrence
>>>> University of Reading:  Professor of Weather and Climate Computing.
>>>> National Centre for Atmospheric Science: Director of Models and
>>> Data.
>>>> STFC: Director of the Centre for Environmental Data Archival.
>>>> Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

-- 
------------------ DKRZ / Data Management ------------------
Martina Stockhause	
Deutsches Klimarechenzentrum	phone:	+49-40-460094-122
Bundesstr. 45a			FAX:	+49-40-460094-106
D-20146 Hamburg, Germany	e-mail:	stockhause at dkrz.de
------------------------------------------------------------