[Go-essp-tech] [esg-gateway-dev] [ESG-CET] CMIP5 Federation Testing Outline
Eric Nienhouse
ejn at ucar.edu
Thu Nov 18 09:44:26 MST 2010
Hi Estani, Phil, Luca,
Thanks for all these thoughts and ideas (and Estani's action to place
the doc on the ESGF wiki.)
I agree that version control on the document is important. Is the
version history tracked on the wiki?
If not, can we place a revision history block at the top of the document
and agree to utilize it?
Phil, thanks for noting the security document and attaching it for all.
I hesitated adding a link to that as it is not broadly accessible.
Indeed, much of the content there is very pertinent to this pre-release
testing work.
However, given the small amount of time we have to perform federation
testing I'd like to scope our collaborative work to what we feel is of
the highest importance (Eg: critical or blocking tests.) I think the
document should represent this as we work on it (tho there may be some
differing opinions on priority and I'll try to focus on resolving them
if so.)
I've sent a request to Gavin for editor acess on the wiki. Let's try
out collaborating on this on the ESGF wiki. Who else needs access?
Note: I've dropped esg-cet from the cc: list to lower the volume on this
thread.
Thanks,
-Eric
Estanislao Gonzalez wrote:
> Hi,
>
> I've already took the liberty to move it into esgf. The page can be
> found here: http://esgf.org/wiki/Cmip5Status/Tests (Phil I haven't
> gone through your doc yet)
>
> I also modified some things, please check this is ok. (GridFTP from
> federation node is not possible, is it? Maybe something else was
> meant, please check.)
>
> I think it's better to keep everything on the same "page". We could
> link every test to a page only viewable by Editors or any registered
> user, as you like. We could use Jira or ESG's bugzilla (if it's ready)
> for tracking specific problems, but I do think that storing the
> results on tickets is not the best option, as with time they get
> obfuscated by new tickets... A table with the results, or at least
> with the positive results, should be IMO better. What what really
> matters is to gather all links at the sameplace and I think esgf wiki
> does the job.
>
> Until we have a plugin for managing users in place I need to know your
> names to add you to the editor group. Please send them two
> me/Gavin/Luca... or any other Admin(http://esgf.org/wiki/AdminGroup).
>
> Thanks,
> Estani
>
>
> On 11/18/2010 05:05 PM, philip.kershaw at stfc.ac.uk wrote:
>> Hi Eric,
>>
>> There's actually also the document we started at the ANL security meeting in Sept. :) ...
>>
>> https://docs.google.com/document/d/17PcZqDs0zONyUa4A3Z835Rt-_ePsEBKHWWP8XkawEow/edit?hl=en#
>>
>> This is not visible to all so I've attached a PDF also. We could lift what we have in there to whatever format doc we want to use to share with: wiki etc.
>>
>> My only reservation about a wiki approach is that you don't lose the concept of some kind of change control. I think it's important to keep track of versions with a document like this.
>>
>> I think we could also prepare some test tools for system wide tests. I've attempted to kick this off by preparing a Python test script to probe the SSL endpoints in the federation and check for correct configuration:
>>
>> http://proj.badc.rl.ac.uk/ndg/browser/TI12-security/trunk/esg_system_tests/esg/security/test/system
>>
>> Cheers,
>> Phil
>>
>>
>>> -----Original Message-----
>>> From: esg-gateway-dev-bounces at mailman.earthsystemgrid.org [mailto:esg-
>>> gateway-dev-bounces at mailman.earthsystemgrid.org] On Behalf Of Eric
>>> Nienhouse
>>> Sent: 18 November 2010 15:52
>>> To: Cinquini, Luca (3880)
>>> Cc: ESG-CET Mail List; go-essp-tech at ucar.edu; esg-gateway-
>>> dev at earthsystemgrid.org
>>> Subject: Re: [esg-gateway-dev] [ESG-CET] CMIP5 Federation Testing
>>> Outline
>>>
>>> Hi Luca, All,
>>>
>>> Thanks for this input. What I would propose are two "documents" to
>>> support this testing work:
>>>
>>> 1) An outline of the tests with some detail about how to perform them,
>>> and what constitutes and acceptable outcome.
>>> 2) A tracking document to capture the tests themselves with comments,
>>> dates and test related details as you note below.
>>>
>>> I think a wiki is ideal for the outline (1) above. I agree emailing a
>>> document around is too cumbersome.
>>>
>>> I think an issue tracker is ideal for (2) above. I propose we use the
>>> existing NCAR Jira system for this. Once we have a good outline
>>> organized we can then create the necessary Jira tasks to track progress
>>> and issues.
>>>
>>> Does this sound reasonable?
>>>
>>> Thanks,
>>>
>>> -Eric
>>>
>>> Cinquini, Luca (3880) wrote:
>>>
>>>> Hi Eric,
>>>> thanks for starting this document, the list of tests to perform
>>>>
>>> looks like a good start - we might want expand it in a little more
>>> detail, like for example distinguishing between:
>>>
>>>> 1) http file download via browser started at the gateway
>>>> 2) http file download via browser started at the TDS page
>>>> 3) http multi-files download via wget script (directly to the TDS)
>>>>
>>>> for both home data node and federated data node.
>>>>
>>>> We might also want to have a matrix table where we list the result of
>>>>
>>> conducting each test between institutions (A,B), with an attached date
>>> when the test was last performed.
>>>
>>>> As for where the document should live, I personally favor the esgf
>>>>
>>> wiki option, although I could also go with 3) and 4). Exchanging the
>>> document by email is kind of cumbersome IMO.
>>>
>>>> thanks, Luca
>>>>
>>>> On Nov 18, 2010, at 7:25 AM, Eric Nienhouse wrote:
>>>>
>>>>
>>>>
>>>>> Dear ESG Gateway Collaborators,
>>>>>
>>>>> We discussed the need for federation testing in our recent GO-ESSP
>>>>> call. I would like to continue our test planning discussion and
>>>>> collaboratively develop a document to support this effort.
>>>>>
>>>>> I welcome your input and insights, in particular for testing
>>>>>
>>> scenarios
>>>
>>>>> related to security and access control of CMIP5 datasets. I plan to
>>>>> serve to coordinate our testing efforts, however, I need help from
>>>>> others to develop and document them.
>>>>>
>>>>> I have an draft outline of these key federation tests at the link
>>>>>
>>> below
>>>
>>>>> and I would like your input:
>>>>>
>>>>>
>>>>>
>>> https://wiki.ucar.edu/display/esgcet/CMIP5+Federation+Testing+Task+Matr
>>> ix
>>>
>>>>> Several options exist to enable collaborative access to this
>>>>>
>>> document.
>>>
>>>>> I'd like suggestions from others on the best approach. Some options
>>>>>
>>> follow:
>>>
>>>>> 1) Exchange this document by email and merge edits (we've done this
>>>>>
>>> in
>>>
>>>>> the past)
>>>>> 2) Move the document to the ESGF wiki (Stephen and Estani have
>>>>> suggested this already.)
>>>>> 3) Keep it on wiki.ucar.edu and provide logins to all
>>>>>
>>> collaborators.
>>>
>>>>> 4) Move it to Google Docs.
>>>>>
>>>>> There is momentum to use the ESGF wiki as at least one supporting
>>>>> document may already be there (CMIP5 Status of Gateways and
>>>>>
>>> Datanodes.)
>>>
>>>>> What do others think would be best?
>>>>>
>>>>> Thanks for your input on this.
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> -Eric
>>>>>
>>>>> Eric Nienhouse wrote:
>>>>>
>>>>>
>>>>>> Dear ESG Gateway Collaborators,
>>>>>>
>>>>>> We're quickly approaching the 1.2.0 ESG Gateway software release.
>>>>>>
>>> This
>>>
>>>>>> version will provide the basis for the upcoming CMIP5 activity data
>>>>>> management effort.
>>>>>>
>>>>>> The Gateway 1.2.0 Release Candidate 1 (RC1) is now available for
>>>>>> inter-gateway federation testing. We encourage all who can to
>>>>>>
>>> upgrade
>>>
>>>>>> to this version as soon as is reasonably possible. (In particular,
>>>>>>
>>> if
>>>
>>>>>> your gateway is on the list of "Key Participating Gateways" below.)
>>>>>> For installation and update details, see the included email below.
>>>>>>
>>>>>> Please note that the Gateway 1.2.0 RC1 requires a corresponding
>>>>>>
>>> Data
>>>
>>>>>> Node version 1.0.4.0 or greater. This Gateway upgrade may require
>>>>>>
>>> an
>>>
>>>>>> update to your Data Node components as well.
>>>>>>
>>>>>> We're in need of significant inter-gateway (eg: federation)
>>>>>>
>>> testing.
>>>
>>>>>> In order to begin this process, it is important that all
>>>>>>
>>> participating
>>>
>>>>>> gateways are running a consistent pre-release 1.2.0 version. The
>>>>>> 1.2.0 RC1 is the recommended version for federation related
>>>>>>
>>> testing.
>>>
>>>>>> Following are the Key Participating Gateways. Several have already
>>>>>> upgraded to the 1.2.0 RC1 version. If you have not yet upgraded to
>>>>>> the 1.2.0 RC1 version, please advise as to when this may be
>>>>>>
>>> reasonably
>>>
>>>>>> possible. (Please send any details or questions to:
>>>>>> esg-gateway-dev at earthsystemgrid.org)
>>>>>>
>>>>>> ANU
>>>>>> BADC
>>>>>> DKRZ
>>>>>> JPL
>>>>>> LBNL
>>>>>> NCAR
>>>>>> ORNL
>>>>>> PCMDI
>>>>>>
>>>>>> Please note the NCAR testing site (http://esg.prototype.ucar.edu)
>>>>>>
>>> is
>>>
>>>>>> currently running the 1.2.0 RC1 version for federation testing
>>>>>>
>>> purposes.
>>>
>>>>>> I would like to begin discussing the top priority federation tests
>>>>>>
>>> and
>>>
>>>>>> would like to hear your input. In particular, testing scenarios
>>>>>> related to security and access control of CMIP5 datasets are of key
>>>>>> importance. I will send a follow up email regarding this testing
>>>>>>
>>> effort.
>>>
>>>>>> Thank you for your involvement in this effort!
>>>>>>
>>>>>> Kind Regards,
>>>>>>
>>>>>> -Eric
>>>>>>
>>>>>> On Sun, 14 Nov 2010 20:15:48 -0700
>>>>>> Nathan Wilhelmi <wilhelmi at ucar.edu> wrote:
>>>>>>
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> We have made a new release of the gateway available, 1.2.0-RC1.
>>>>>>>
>>> The
>>>
>>>>>>> release resolves a number of issues that were identified in the
>>>>>>>
>>> BETA3
>>>
>>>>>>> release.
>>>>>>>
>>>>>>> The new BETA release can be downloaded directly from here:
>>>>>>>
>>>>>>>
>>> https://vets.development.ucar.edu/nexus/content/repositories/releases/s
>>> gf/gateway/1.2.0-RC1/
>>>
>>>>>>> For this release the the new installation and upgrade from 1.1.0
>>>>>>>
>>> guides
>>>
>>>>>>> remain the same.
>>>>>>>
>>>>>>> If you already have 1.2.0-BETAx installed upgrade instructions can
>>>>>>>
>>> be
>>>
>>>>>>> found here:
>>>>>>>
>>>>>>>
>>> https://wiki.ucar.edu/display/esgcet/Gateway+Upgrade+Instructions+%28ve
>>> rsion+1.2.0-BETA-1.2.0-BETA2+to+version+1.2.0-BETA3%29
>>>
>>>>>>>
>>>>>>> Please provide feedback (good or bad!) to:
>>>>>>> esg-gateway-dev at earthsystemgrid.org
>>>>>>>
>>>>>>> -or-
>>>>>>>
>>>>>>> If you have Jira Issue Tracking access, please enter a trouble
>>>>>>>
>>> ticket
>>>
>>>>>>> against version "1.2.0-RC1" under the "Affects Version/s" drop-
>>>>>>>
>>> down.
>>>
>>>>>>> Thanks!!!
>>>>>>>
>>>>>>> -Nate
>>>>>>>
>>>>>>>
>>>>> _______________________________________________
>>>>> ESG-CET mailing list
>>>>> ESG-CET at earthsystemgrid.org
>>>>> http://mailman.ucar.edu/mailman/listinfo/esg-cet
>>>>>
>>>>>
>>>>
>>> _______________________________________________
>>> esg-gateway-dev mailing list
>>> esg-gateway-dev at mailman.earthsystemgrid.org
>>> http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev
>>>
>>
>>
>> _______________________________________________
>> esg-gateway-dev mailing list
>> esg-gateway-dev at mailman.earthsystemgrid.org
>> http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev
>
>
> --
> Estanislao Gonzalez
>
> Max-Planck-Institut für Meteorologie (MPI-M)
> Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
> Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
>
> Phone: +49 (40) 46 00 94-126
> E-Mail: estanislao.gonzalez at zmaw.de
>
More information about the GO-ESSP-TECH
mailing list