[Go-essp-tech] CMIP5 Federation Testing
Eric Nienhouse
ejn at ucar.edu
Fri Dec 3 08:05:39 MST 2010
Hi All,
Thanks Luca, Stephen, Estani and others for working on OpenID Login
testing and updating the matrix. As Luca notes, there are likely
configuration issues at the root of some of these test failures.
We discussed the status of federation testing at the ESG meeting
yesterday. Following are highlights:
1) We agreed to identify a specific CMIP5 Baseline federation test set
of gateways to support federation testing. This provides some
independence from our operational (eg production) gateways which several
test gateway administrators have requested.
2) We also identified the need to establish and distribute a test system
configuration (eg. a complete set of the SQL statements we've been
sending around for the test system) and make that available through the
esgf.org site.
3) Lastly, we identified a need to establish a minimum version
requirement for the test federation gateways and datanodes. There are
recent data node component updates that should be included in the
datanode distribution prior to further testing. At this moment it is
unclear what this version will be labeled, however we need to identify
this as soon as possible.
I have updated the CMIP status page to indicate the participating test
gateways (that I know of.) The update is here:
http://esgf.org/wiki/Cmip5Status
All gateway administrators, please review this list and make changes or
comments. The federation test is open to interested gateways and the
only requirement is to document the gateway and ensure the minimum gw
and datanode versions are installed.
Once the test matrix is up to date, we can derive a configuration file
representative of the test system, distribute it and configure our test
gateways accordingly. This will provide the basis for our testing efforts.
Please note that the 1.2.0 CMIP5 Baseline Release is dependent upon an
acceptable federation system test. The 1.2.0 Release is currently
blocked and is tentatively postponed until Dec 15, 2010.
Thanks,
-Eric
Cinquini, Luca (3880) wrote:
> Thanks Stephen, this is great. I also tested access to BADC and
> NCAR/test and it worked ok with a JPL openid.
> Looks like the ORNL and NCI gateways still don't have the JPL Gateway
> entry in their database, so SSO there fails (I am cc-ing Ross and Alex
> for that).
>
> Ross, Alex - you can find entries for JPL and DKRZ gateways at the
> bottom of this page:
>
> http://www.esgf.org/wiki/Cmip5Status/Tests
>
> thanks, luca
>
>
> On Dec 3, 2010, at 3:23 AM, <stephen.pascoe at stfc.ac.uk
> <mailto:stephen.pascoe at stfc.ac.uk>> <stephen.pascoe at stfc.ac.uk
> <mailto:stephen.pascoe at stfc.ac.uk>> wrote:
>
>> I have just tested OpenID login with a BADC OpenID and filled in the
>> TestResults
>>
>> 1. I tested production and test gateway's and results were consistent.
>> 2. All Gateways passed except PCMDI*
>> 3. I didn't test LBNL or ANU because their Gateway URLs aren't on the
>> Cmup5Status wiki page.
>>
>> * The error was "Error: There were errors during the authentication
>> process. Please verify that the OpenID provided is correct." I think
>> this is
>> the Gateway's OpenID callout to our IdP failing to recognise our SSL
>> certificate.
>>
>> Given that authorisation relies on joining the CMIP5 group on the
>> PCMDI portal I think it is a high priority to fix PCMDI's OpenID SSO.
>>
>> Cheers,
>> Stephen.
>>
>> ---
>> Stephen Pascoe +44 (0)1235 445980
>> Centre of Environmental Data Archival
>> Rutherford Appleton Laboratory, Chilton, Didcot OX11 0QX, UK
>>
>> *From:* Estanislao Gonzalez [mailto:estanislao.gonzalez at zmaw.de]
>> *Sent:* 02 December 2010 16:45
>> *To:* Pascoe, Stephen (STFC,RAL,SSTD)
>> *Cc:* gavin at llnl.gov
>> <mailto:gavin at llnl.gov>; Luca.Cinquini at jpl.nasa.gov
>> <mailto:Luca.Cinquini at jpl.nasa.gov>; ejn at ucar.edu
>> <mailto:ejn at ucar.edu>; ranantha at mcs.anl.gov
>> <mailto:ranantha at mcs.anl.gov>; wilhelmi at ucar.edu
>> <mailto:wilhelmi at ucar.edu>; Kershaw, Philip
>> (STFC,RAL,SSTD); neillm at mcs.anl.gov
>> <mailto:neillm at mcs.anl.gov>; williams13 at llnl.gov
>> <mailto:williams13 at llnl.gov>;don at ucar.edu <mailto:don at ucar.edu>
>> *Subject:* Re: Federation Testing - OpenID Login
>>
>> A good point... I'm testing our test server agains JPL production one
>> as the other is not accessible...
>>
>> But I think we should concentrate on the test servers... as the
>> matrix only have one entry per institution (and please don't even
>> dare to expand that!!)
>>
>> And please put the access data into the current Status wiki
>> page http://www.esgf.org/wiki/Cmip5Status to simplify access.
>>
>> Thanks,
>> Estani
>>
>>
>> On 12/02/2010 05:31 PM, stephen.pascoe at stfc.ac.uk
>> <mailto:stephen.pascoe at stfc.ac.uk> wrote:
>> Yes, I'm stuck in a similar situation. Our production gateway now
>> has a publically-signed certificate but cmip-gw2.badc.rl.ac.uk still
>> has an expired self-signed one. Once that's fixed I'll start filling
>> in the matrix.
>>
>> Can I just check -- these OpenID tests are happening on our TEST
>> gateways, right?
>>
>> Cheers,
>> Stephen.
>>
>> ---
>> Stephen Pascoe +44 (0)1235 445980
>> Centre of Environmental Data Archival
>> Rutherford Appleton Laboratory, Chilton, Didcot OX11 0QX, UK
>>
>>
>>
>> *From:* Gavin M. Bell [mailto:gavin at llnl.gov]
>> *Sent:* 02 December 2010 16:28
>> *To:* Cinquini, Luca (3880)
>> *Cc:* Estanislao Gonzalez; Eric Nienhouse; Rachana Ananthakrishnan;
>> Nathan Wilhelmi; Kershaw, Philip (STFC,RAL,SSTD); Pascoe, Stephen
>> (STFC,RAL,SSTD); Neill Miller; Williams, Dean N.; Don Middleton
>> *Subject:* Re: Federation Testing - OpenID Login
>>
>> As soon as I slog through this key stuff, we'll joint the party ;-)!
>>
>> On 12/2/10 8:22 AM, Cinquini, Luca (3880) wrote:
>> Yes, that worked too. WDCC and JPL are officially friends...
>> thanks, Luca
>>
>> On Dec 2, 2010, at 8:52 AM, Estanislao Gonzalez wrote:
>>
>>
>> Sorry I was in a meeting...
>>
>> It's done now. I've tested WDCC2->JPL with the same results... I'll be
>>
>> testing the other way around now (JPL to WDCC).
>>
>>
>>
>> Thanks,
>>
>> Estani
>>
>>
>>
>>
>>
>>
>>
>> On 12/02/2010 04:27 PM, Cinquini, Luca (3880) wrote:
>>
>> FYI a little good news: after I inserted the WDCC gateway into the JPL database, I was able to register at WDCC and use that openid to log in at JPL.
>>
>> Estani, please let me know when you have the JPL Gateway in your system so I can try the reverse.
>>
>> thanks, Luca
>>
>>
>>
>> On Dec 2, 2010, at 7:14 AM, Estanislao Gonzalez wrote:
>>
>>
>>
>> Hi Eric,
>>
>>
>>
>> As I said quite some times now, the data of our gateway is in no other
>>
>> GW DB. AFAIK there's no procedure for setting this. I've already sent it
>>
>> to BADC but it doesn't appear to have been include:
>>
>>
>>
>> ERROR: Consumer error; nested exception is
>>
>> org.springframework.security.ui.openid.OpenIDConsumerException:
>>
>> Unrecognized Gateway for name:ESG-WDCC IdP endpoint:
>>
>> https://ipcc-ar5.dkrz.de/openid/provider.htm
>>
>>
>>
>> If no other GW add this to their DB I'm afraid we won't be able to
>>
>> participate on the tests... So... please?
>>
>>
>>
>>
>>
>> -- Albedo2 Test gateway (External)
>>
>> INSERT INTO metadata.gateway (id, name, description, base_url,
>>
>> base_secure_url, attributes_service_url, oai_repository_url, identity,
>>
>> administrator_personal, administrator_email, myproxy_endpoint,
>>
>> idp_endpoint) VALUES ('a1bed020-835e-4fdb-8fbc-0be206191027',
>>
>> 'ESG-WDCC2', 'Earth System Grid gateway at the World Data Center for
>>
>> Climate', 'http://albedo2.dkrz.de/esgcet',
>>
>> 'https://albedo2.dkrz.de/esgcet',
>>
>> 'https://albedo2.dkrz.de/esgcet/saml/soap/secure/attributeService.htm',
>>
>> 'https://albedo2.dkrz.de/esgcet/oai/repository.htm',
>>
>> 'CN=albedo2.dkrz.de, OU=WDCC, O=DKRZ, C=DE', 'ESG Gateway
>>
>> Administrator', 'estanislao.gonzalez at zmaw.de <mailto:estanislao.gonzalez at zmaw.de>', 'albedo2.dkrz.de:7512',
>>
>> 'https://albedo2.dkrz.de/esgcet/openid/provider.htm');
>>
>>
>>
>> -- ipcc-ar5 gateway
>>
>> INSERT INTO metadata.gateway (id, name, description, base_url,
>>
>> base_secure_url, attributes_service_url, oai_repository_url, identity,
>>
>> administrator_personal, administrator_email, myproxy_endpoint,
>>
>> idp_endpoint) VALUES ('1bcca350-835e-4fdb-8fbc-0be206191027',
>>
>> 'ESG-WDCC', 'Earth System Grid gateway at the World Data Center for
>>
>> Climate', 'http://ipcc-ar5.dkrz.de', 'https://ipcc-ar5.dkrz.de',
>>
>> 'https://ipcc-ar5.dkrz.de/saml/soap/secure/attributeService.htm',
>>
>> 'https://ipcc-ar5.dkrz.de/oai/repository.htm', 'CN=ipcc-ar5.dkrz.de,
>>
>> OU=WDCC, O=DKRZ, C=DE', 'ESG Gateway Administrator',
>>
>> 'estanislao.gonzalez at zmaw.de <mailto:estanislao.gonzalez at zmaw.de>', 'ipcc-ar5.dkrz.de:7512',
>>
>> 'https://ipcc-ar5.dkrz.de/openid/provider.htm');
>>
>>
>>
>>
>>
>> Thanks,
>>
>> estani
>>
>>
>>
>> On 12/02/2010 03:00 PM, Eric Nienhouse wrote:
>>
>> Hi All,
>>
>>
>>
>> I've started doing some basic testing of federation login (tests #2,
>>
>> #3 in our Integration Test outline.) I've run into a number of
>>
>> problems which are outlined below. I've tested login with with three
>>
>> of my own OpenIDs (NCAR Test GW, BADC/CEDA, PCMDI) at the five
>>
>> gateways noted below.
>>
>>
>>
>> In general, federated gateway login is failing in my experience.
>>
>> Further, the results seems to differ based on the user performing the
>>
>> test. (For example, I can login to the NCAR Test GW with my PCMDI
>>
>> OpenID, however others like Dean Williams cannot. Phil has noted
>>
>> similar problems.)
>>
>>
>>
>> First, could others try similar login tests and report back? (Eg.
>>
>> test GW login with your OpenID to PCMDI, BADC, JPL, NCAR Test GW). I
>>
>> would like to try to characterize the extent of these issues to help
>>
>> form our approach to fixing the system.
>>
>>
>>
>> I suspect the failures we're experiencing are due to several causes:
>>
>>
>>
>> 1) Trust store is missing some GW certs.
>>
>> 2) Gateway database table missing federated GW entries.
>>
>> 3) Expired SSL certificates.
>>
>>
>>
>> I noticed the BADC production gateway's ssl certificate appears to
>>
>> have expired recently (12/1 ~2:00 am.) PCMDI's ssl certificate is
>>
>> expired as well (updating it is in progress.)
>>
>>
>>
>> System wide federation login is critical to furthering our integration
>>
>> testing work and we need to work out these problems quickly to
>>
>> continue testing toward system acceptance.
>>
>>
>>
>> Thanks everyone for working on this!
>>
>>
>>
>> -Eric
>>
>>
>>
>> #
>>
>> # Simple Federation Login Test Results
>>
>> #
>>
>>
>>
>> Please see following for GW URLS: http://esgf.org/wiki/Cmip5Status
>>
>>
>>
>> Gateway OpenID Login test results:
>>
>>
>>
>> NCAR Test GW OpenID: https://esg.prototype.ucar.edu/myopenid/enienhouse
>>
>>
>>
>> PCMDI: Success
>>
>> JPL: Success
>>
>> BADC Prod: Fail
>>
>> NCAR Test GW: Success (home gw)
>>
>> DKRZ Prod: Fail
>>
>>
>>
>> PCMDI OpenID: https://pcmdi3.llnl.gov/esgcet/myopenid/nienhouse
>>
>>
>>
>> PCMDI: Success (home gw)
>>
>> JPL: Success
>>
>> NCAR Test GW: Success
>>
>> BADC Prod: Fail
>>
>> DKRZ Prod: Fail
>>
>>
>>
>> BADC OpenID: https://ceda.ac.uk/openid/Eric.Nienhouse
>>
>>
>>
>> PCMDI: Fail
>>
>> JPL: Success
>>
>> NCAR Test GW: Fail
>>
>> BADC Prod: Success
>>
>> DKRZ Prod: Success
>>
>>
>>
>>
>>
>>
>>
>> Eric Nienhouse wrote:
>>
>> Hi Rachana, All,
>>
>>
>>
>> Thanks for all the effort on these documents and related discussions
>>
>> including the go-essp call last week! Sorry I missed much of this as
>>
>> I was out over the US Thanksgiving holiday.
>>
>>
>>
>> Rachana notes a need to wait on the data node update pending
>>
>> developer branch merging prior to proceeding on immediate integration
>>
>> tests.
>>
>>
>>
>> Is there a time-line for this data node update? (I may have missed
>>
>> the details on this - apologies if so. Are we still waiting?)
>>
>>
>>
>> I'd like to get a sense of when we can begin this testing work as
>>
>> basic acceptance is blocking the gateway 1.2 release. Once
>>
>> clarified, we should include those at LBNL and ANL in the testing
>>
>> discussions as well.
>>
>>
>>
>> I imagine a few of the "gateway only" tests can be accomplished in
>>
>> the meantime (eg: Integrations tests 1-5, such as register at
>>
>> gateway, request group CMIP5 group membership, etc.) Has anyone been
>>
>> testing these areas?
>>
>>
>>
>> I will spend some time adding testing step details to the Integration
>>
>> Tests document today.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> -Eric
>>
>>
>>
>>
>>
>> Rachana Ananthakrishnan wrote:
>>
>> I have merged both documents to the ESGF wiki. Top page remains the
>>
>> same: http://esgf.org/wiki/Cmip5Status/Tests.
>>
>>
>>
>> For immediate work, as determined by Eric's document, please see:
>>
>> http://esgf.org/wiki/Cmip5Status/Tests/TestResults. Only the
>>
>> Integration Tests section with "Priority" marked as "1" was agreed
>>
>> for immediate testing. By immediate I do mean we wait for the data
>>
>> node update roll out after the many developer branches are merged.
>>
>>
>>
>> I still need to figure out how to get table of contents in a page,
>>
>> make some pages like my heading markups, make a title of anchor show
>>
>> up nicely, and format the Interface Tests page: apparently the joys
>>
>> of learning yet another wiki format syntax! But the structure and
>>
>> edits will show what I propose as format for maintaining these tests
>>
>> and their results. Feel free to make your comments and edits on the
>>
>> wiki directly, I am guessing there is history to revert if needed.
>>
>>
>>
>> The security document from Argonne only has some deployment matrix
>>
>> and notes at the end, that has not been moved to this document. Once
>>
>> that is done (I need to find a good place), we can remove that
>>
>> document. It is linked in reference for now.
>>
>>
>>
>> For everyone in the US, hope you have a good Thanksgiving break!
>>
>> Rachana
>>
>>
>>
>> Rachana Ananthakrishnan
>>
>> Argonne National Lab | University of Chicago
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Estanislao Gonzalez
>>
>>
>>
>> Max-Planck-Institut für Meteorologie (MPI-M)
>>
>> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>>
>> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>
>>
>>
>> Phone: +49 (40) 46 00 94-126
>>
>> E-Mail: estanislao.gonzalez at zmaw.de <mailto:estanislao.gonzalez at zmaw.de>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Estanislao Gonzalez
>>
>>
>>
>> Max-Planck-Institut für Meteorologie (MPI-M)
>>
>> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>>
>> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>
>>
>>
>> Phone: +49 (40) 46 00 94-126
>>
>> E-Mail: estanislao.gonzalez at zmaw.de <mailto:estanislao.gonzalez at zmaw.de>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Gavin M. Bell
>> --
>>
>> "Never mistake a clear view for a short distance."
>> -Paul Saffo
>>
>>
>>
>> --
>> Scanned by iCritical.
>>
>>
>>
>>
>>
>> --
>> Estanislao Gonzalez
>>
>> Max-Planck-Institut für Meteorologie (MPI-M)
>> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
>> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>>
>> Phone: +49 (40) 46 00 94-126
>> E-Mail: estanislao.gonzalez at zmaw.de <mailto:estanislao.gonzalez at zmaw.de>
>>
>> --
>> Scanned by iCritical.
>>
>>
>>
>
More information about the GO-ESSP-TECH
mailing list