[Go-essp-tech] CMIP5 Data Coming February 1st (Mark Elkington)
Williams, Dean N.
williams13 at llnl.gov
Tue Jan 18 07:57:49 MST 2011
Dear Colleagues,
I want to first thank you all for working so diligently on preparing for
CMIP5 data. Without all of your efforts, the distribution of CMIP5 would
*not* be possible.
Let's take this time today to take stock of our preparedness and discuss
the most straightforward path to achieving our end goal. Looking forward to
a great discussion...
Best regards,
Dean
On 1/17/11 7:16 AM, "stephen.pascoe at stfc.ac.uk" <stephen.pascoe at stfc.ac.uk>
wrote:
> Hi Dean,
>
> To be fair we've been mentioning that we are receiving data from the Met
> Office, in emails and on telcos, for months now.
>
> I suggest we use tomorrow's telco to do a broad review of our readiness for
> CMIP5 release on Feb 1st. We cam report on QC and the questionaire as well as
> federation testing and replication.
>
>
> We have exposed 1 dataset in our test gateway as a target for replication, the
> rest of the delivered data is
> not in a gateway but is at various stages of our ingestion process.
>
> The test dataset is ...
>
> THREDDS URL:
> http://cmip-dn1.badc.rl.ac.uk/thredds/esgcet/5/cmip5.output1.MOHC.HadGEM2-ES.p
> iControl.6hr.atmos.6hrPlev.r1i1p1.v1.xml
> Gateway: http://cmip-gw2.badc.rl.ac.uk
>
> I admit we've been struggling to get to the point of testing replication with
> this data. I'm juggling as hard as I can.
>
> Stephen.
>
> ---
> Stephen Pascoe +44 (0)1235 445980
> Centre of Environmental Data Archival
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On
> Behalf Of Williams, Dean N.
> Sent: 14 January 2011 21:17
> To: Mark Elkington; go-essp-tech at ucar.edu
> Subject: Re: [Go-essp-tech] CMIP5 Data Coming February 1st (Mark Elkington)
>
> Hi Mark,
>
> This is the first that we have heard that data was being delivered to
> BADC. I would hope that the turn around would be within a few days.
>
> Another question (perhaps for BADC and DKRZ) is the Metafor
> questionnaire. Have most of the modeling centers complete it?
>
> Since you have been delivering data to BADC three months ago, have you
> or BADC been working with DKRZ on QC checks? Since we have really data we
> should look it get this into the system and processed for production use.
>
> Thanks for letting us know,
> Dean
>
>
> On 1/14/11 11:40 AM, "Mark Elkington" <mark.elkington at blueyonder.co.uk>
> wrote:
>
>> Dean
>>
>> Its already started. We started delivering climate data to BADC three months
>> ago, with more going
>> this week. We have a backlog of data at our sire going through final quality
>> checks.
>>
>> What plans are in place for a coordinated release of the data system?. Can
>> we
>> expect to see all of
>> the data that has been delivered accessible at the same time, or at least
>> within a few days of one
>> another?.
>>
>> Mark
>>
>> --------------------------------------------------------------
>> Dr Mark Elkington
>> Met Office Hadley Centre
>>
>>
>> -----Original Message-----
>> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On
>> Behalf Of
>> go-essp-tech-request at ucar.edu
>> Sent: 14 January 2011 19:00
>> To: go-essp-tech at ucar.edu
>> Subject: GO-ESSP-TECH Digest, Vol 23, Issue 12
>>
>> Send GO-ESSP-TECH mailing list submissions to
>> go-essp-tech at ucar.edu
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>> or, via email, send a message with subject or body 'help' to
>> go-essp-tech-request at ucar.edu
>>
>> You can reach the person managing the list at
>> go-essp-tech-owner at ucar.edu
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of GO-ESSP-TECH digest..."
>>
>>
>> Today's Topics:
>>
>> 1. CMIP5 Data Coming February 1st (Williams, Dean N.)
>> 2. Re: CMIP5 Data Coming February 1st (Craig E. Ward)
>> 3. Re: CMIP5 Data Coming February 1st (Cinquini, Luca (3880))
>> 4. Re: CMIP5 Data Coming February 1st (Mehmet Balman)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Fri, 14 Jan 2011 09:04:42 -0800
>> From: "Williams, Dean N." <williams13 at llnl.gov>
>> Subject: [Go-essp-tech] CMIP5 Data Coming February 1st
>> To: "stephen.pascoe at stfc.ac.uk" <stephen.pascoe at stfc.ac.uk>, Eric
>> Nienhouse <ejn at ucar.edu>, "Craig E. Ward" <cward at isi.edu>,
>> "Cinquini,
>> Luca (3880)" <Luca.Cinquini at jpl.nasa.gov>
>> Cc: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
>> Message-ID: <C955C22A.5E8A%williams13 at llnl.gov>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> Dear Colleagues,
>>
>> Karl has informed me that we are expecting to receive CMIP5 model
>> output in 3 weeks, so
>> we must have ESG ready to publish and distribute data to the community by
>> February 1st. At the
>> moment, the targeted model output is CCCma (Canadian Centre for Climate
>> Modelling and Analysis). (I
>> am sure other data sets will quickly follow suit.) To be ready, we must have
>> federation completed.
>> Luca, Stephen, Eric, Craig, and others can you give the status of the
>> federation testing? The last I
>> remember:
>>
>>
>> 1. Stephen was going to include some Hadley data for federation federation
>> testing;
>> 2. Craig was testing replication from multiple sites;
>> 3. Federation security was in place;
>>
>> From what I remember, The federation was mainly completed and mostly
>> tested
>> with a few minor
>> changes needed. Primarily we need to test federation between PCMDI-BADC,
>> PCMDI-DKRZ, PCMDI-ANU,
>> etc.. What else are we missing to be ready for the Febuary 1st deadline?
>>
>> Perhaps this next Tuesday, we can discuss our preparedness for the
>> onslaught of data. How many
>> of us will be around for a Tuesday go-essp meeting at our regular time?
>>
>> Thanks and best regards to all,
>> Dean
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>>
http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110114/ad007262/>>
a
>> ttachment-0001.html
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Fri, 14 Jan 2011 09:48:01 -0800
>> From: "Craig E. Ward" <cward at isi.edu>
>> Subject: Re: [Go-essp-tech] CMIP5 Data Coming February 1st
>> To: "Williams, Dean N." <williams13 at llnl.gov>
>> Cc: "Cinquini, Luca \(3880\)" <Luca.Cinquini at jpl.nasa.gov>,
>> "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
>> Message-ID: <4D308C51.5010307 at ISI.EDU>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> Replication testing has been taken up by Stephen and Estani. Stephen created
>> a
>> breakdown of the test steps on the wiki:
>>
>> http://esgf.org/wiki/Cmip5Status/Tests/ReplicationTests
>>
>> I sent additional information to Stephen about the TDS catalog comparison for
>> incorporation into the page as he sees fit.
>>
>> I expect to be available for a call Tuesday morning (Pacific time).
>>
>> Craig
>>
>> On 1/14/11 9:04 AM, Williams, Dean N. wrote:
>>> Dear Colleagues,
>>>
>>> Karl has informed me that we are expecting to receive CMIP5
>>> model output in 3 weeks,
>> so we must have ESG ready to publish and distribute data to the community by
>> February 1st. At the
>> moment, the targeted model output is CCCma (Canadian Centre for Climate
>> Modelling and Analysis). (I
>> am sure other data sets will quickly follow suit.) To be ready, we must have
>> federation completed.
>> Luca, Stephen, Eric, Craig, and others can you give the status of the
>> federation testing? The last I
>> remember:
>>>
>>>
>>> 1. Stephen was going to include some Hadley data for federation
>>> federation
>>> testing;
>>> 2. Craig was testing replication from multiple sites;
>>> 3. Federation security was in place;
>>>
>>> From what I remember, The federation was mainly completed and mostly
>>> tested with a few minor
>> changes needed. Primarily we need to test federation between PCMDI-BADC,
>> PCMDI-DKRZ, PCMDI-ANU,
>> etc.. What else are we missing to be ready for the Febuary 1st deadline?
>>>
>>> Perhaps this next Tuesday, we can discuss our preparedness for the
>>> onslaught of data. How
>> many of us will be around for a Tuesday go-essp meeting at our regular time?
>>>
>>> Thanks and best regards to all,
>>> Dean
>>>
>>
>> --
>> Craig E. Ward
>> USC Information Sciences Institute
>> 310-448-8271
>> cward at ISI.EDU
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Fri, 14 Jan 2011 09:57:21 -0800
>> From: "Cinquini, Luca (3880)" <Luca.Cinquini at jpl.nasa.gov>
>> Subject: Re: [Go-essp-tech] CMIP5 Data Coming February 1st
>> To: "Williams, Dean N." <williams13 at llnl.gov>
>> Cc: "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
>> Message-ID: <BB3A2EF8-B943-4A3F-8E38-8AEDD93315FF at jpl.nasa.gov>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> Hi Dean,
>> thanks for the notice... indeed it might be a very good idea to focus on
>> CMIP5
>> preparedness on the
>> next GO-ESSP conf call.
>>
>> This is my personal understanding of the situation:
>>
>> o Local system (Data Node + Gateway) are able to collect and deliver data,
>> and
>> register users.
>>
>> o Single Sign On among federation patners is working correctly
>>
>> o An issue has been uncovered which affects authorization when the Data Node
>> is front-ended by a
>> proxy, as it happens for BADC:
>> http://esgf.org/bugzilla/show_bug.cgi?id=11
>>
>> o Another issue has been uncovered recently which affects a user from gateway
>> A downloading data
>> published to gateway B when using wget/curl, when the data is protected by
>> the
>> an attribute
>> controlled by Gateway C (for example, the PCMDI attribute "CMIP5 Research"):
>> the download will work
>> only if the user from Gateway A has already logged into Gateway B. This may
>> not be the most critical
>> case, at this point... See http://esgf.org/bugzilla/show_bug.cgi?id=12
>>
>> o We have not performed extensive testing of metadata exchange yet, although
>> it is generally
>> believed to work.
>>
>> Everybody, please add your understanding of the current status...
>>
>> thanks, Luca
>>
>> On Jan 14, 2011, at 10:04 AM, Williams, Dean N. wrote:
>>
>> Dear Colleagues,
>>
>> Karl has informed me that we are expecting to receive CMIP5 model
>> output in 3 weeks, so
>> we must have ESG ready to publish and distribute data to the community by
>> February 1st. At the
>> moment, the targeted model output is CCCma (Canadian Centre for Climate
>> Modelling and Analysis). (I
>> am sure other data sets will quickly follow suit.) To be ready, we must have
>> federation completed.
>> Luca, Stephen, Eric, Craig, and others can you give the status of the
>> federation testing? The last I
>> remember:
>>
>>
>> 1. Stephen was going to include some Hadley data for federation federation
>> testing;
>> 2. Craig was testing replication from multiple sites;
>> 3. Federation security was in place;
>>
>> From what I remember, The federation was mainly completed and mostly
>> tested
>> with a few minor
>> changes needed. Primarily we need to test federation between PCMDI-BADC,
>> PCMDI-DKRZ, PCMDI-ANU,
>> etc.. What else are we missing to be ready for the Febuary 1st deadline?
>>
>> Perhaps this next Tuesday, we can discuss our preparedness for the
>> onslaught of data. How many
>> of us will be around for a Tuesday go-essp meeting at our regular time?
>>
>> Thanks and best regards to all,
>> Dean
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>>
http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110114/1e6200ab/>>
a
>> ttachment-0001.html
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Fri, 14 Jan 2011 09:58:35 -0800
>> From: Mehmet Balman <mbalman at lbl.gov>
>> Subject: Re: [Go-essp-tech] CMIP5 Data Coming February 1st
>> To: "Williams, Dean N." <williams13 at llnl.gov>
>> Cc: "Cinquini, Luca \(3880\)" <Luca.Cinquini at jpl.nasa.gov>,
>> "go-essp-tech at ucar.edu" <go-essp-tech at ucar.edu>
>> Message-ID: <4D308ECB.3060005 at lbl.gov>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Hi Dean,
>>
>> LBNL site is also ready. We have CMIP3 files on site here at NERSC. We
>> can also publish CMIP5 data over ESG-NERSC(esg.nersc.gov).
>>
>> I tested federation with Stephen and Estani. Federation test results for
>> ESG-NERSC are updated in the wiki page. Only problem is that PCMDI needs
>> to make some minor updates (add esg.nersc.gov ssl cert to truststore and
>> update metadata in db - see wiki page) in order to accept external
>> openId logins from LBNL gateway. LBNL gateway accepts external logins
>> from all other sites including PCMDI. I didnt test replication yet. I
>> will be around on Tuesday to discuss more.
>>
>> -Mehmet
>>
>>
>> On 01/14/2011 09:04 AM, Williams, Dean N. wrote:
>>> Dear Colleagues,
>>>
>>> Karl has informed me that we are expecting to receive
>>> CMIP5 model output in 3 weeks, so we must have ESG ready to publish
>>> and distribute data to the community by February 1st. At the moment,
>>> the targeted model output is CCCma (Canadian Centre for Climate
>>> Modelling and Analysis). (I am sure other data sets will quickly
>>> follow suit.) To be ready, we must have federation completed. Luca,
>>> Stephen, Eric, Craig, and others can you give the status of the
>>> federation testing? The last I remember:
>>>
>>> 1. Stephen was going to include some Hadley data for federation
>>> federation testing;
>>> 2. Craig was testing replication from multiple sites;
>>> 3. Federation security was in place;
>>>
>>>
>>> From what I remember, The federation was mainly completed and
>>> mostly tested with a few minor changes needed. Primarily we need to
>>> test federation between PCMDI-BADC, PCMDI-DKRZ, PCMDI-ANU, etc.. What
>>> else are we missing to be ready for the Febuary 1st deadline?
>>>
>>> Perhaps this next Tuesday, we can discuss our preparedness for the
>>> onslaught of data. How many of us will be around for a Tuesday go-essp
>>> meeting at our regular time?
>>>
>>> Thanks and best regards to all,
>>> Dean
>>>
>>>
>>> _______________________________________________
>>> GO-ESSP-TECH mailing list
>>> GO-ESSP-TECH at ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>>
http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110114/a57aecdf/>>
a
>> ttachment-0001.html
>>
>> ------------------------------
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>
>>
>> End of GO-ESSP-TECH Digest, Vol 23, Issue 12
>> ********************************************
>>
>>
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
> --
> Scanned by iCritical.
More information about the GO-ESSP-TECH
mailing list