[Go-essp-tech] [esg-gateway-dev] [ESG-CET] CMIP5 Federation Testing Outline

philip.kershaw at stfc.ac.uk philip.kershaw at stfc.ac.uk
Fri Nov 19 05:09:46 MST 2010 

Hi Eric and all,

Firstly, thank you for raising the issue of testing and putting it up the agenda.

I can understand your concerns about scope of the testing and time available.

My concern though would be to keep this activity within the scope of what we already agreed.

At the September meeting at ANL one of the express purposes was to lay out a plan for federation wide testing of the security system.  The document I referenced was an output from that.  It's not complete and to a large extent just sets out a framework. 

I'm not saying that we should slavishly keep to it if it's not going to address our needs.  That said, because it was what we worked on together I think we should at the very least review it and take it into account.  It could be that the outcome of that is we say, it's not suitable - and that would be fine.  If we don't follow this process though, I think there's a danger of us not being consistent in our approach: perhaps missing out important items for testing our repeating or reinventing work that has already been done.

We can still have a test plan which encompasses a broad scope even if given time constraints, all those can't be met.  The tests can be listed and against each we agree at what stage we do them in our process of completing and rolling out the system.  It could be that some tests can't be done in the time - fine, but at least we have captured and recorded them.  If we go with an alternative organic wiki approach to the plan there's a danger that important things are omitted.

I had more of a think about using a wiki for the plan and I still have some unease about it.  Firstly, I appreciate the need for some kind of collaborative environment which we can all input into.  However, the plan needs to have distinct 'releases' so that we have consistent benchmarks upon which we can analyse the system. E.g. this sets of releases of this set of software passed against these tests a, b, c described in version X of the test plan.  There's a danger of things becoming vague if we grow a document organically and don't have a concept of distinct versions.

The test plan is an expression of what we expect the system to be able to do so it reflects the overall requirements.  I think it's important we do this really well.  

Just as a general observation I think it can be hard with this kind of large collaborative project to document what we do consistently and that contrasts with the rigour I've seen applied to the documentation process in other software development projects.  That said, I think there's more that could be done and we would get a much better outcome for the system we deliver.

Cheers,
Phil

> -----Original Message-----
> From: esg-gateway-dev-bounces at mailman.earthsystemgrid.org [mailto:esg-
> gateway-dev-bounces at mailman.earthsystemgrid.org] On Behalf Of Eric
> Nienhouse
> Sent: 18 November 2010 16:44
> To: Estanislao Gonzalez
> Cc: go-essp-tech at ucar.edu; esg-gateway-dev at earthsystemgrid.org
> Subject: Re: [esg-gateway-dev] [ESG-CET] CMIP5 Federation Testing
> Outline
> 
> Hi Estani, Phil, Luca,
> 
> Thanks for all these thoughts and ideas (and Estani's action to place
> the doc on the ESGF wiki.)
> 
> I agree that version control on the document is important.  Is the
> version history tracked on the wiki?
> 
> If not, can we place a revision history block at the top of the
> document
> and agree to utilize it?
> 
> Phil, thanks for noting the security document and attaching it for all.
> I hesitated adding a link to that as it is not broadly accessible.
> Indeed, much of the content there is very pertinent to this pre-release
> testing work.
> 
> However, given the small amount of time we have to perform federation
> testing I'd like to scope our collaborative work to what we feel is of
> the highest importance (Eg: critical or blocking tests.)  I think the
> document should represent this as we work on it (tho there may be some
> differing opinions on priority and I'll try to focus on resolving them
> if so.)
> 
> I've sent a request to Gavin for editor acess on the wiki.  Let's try
> out collaborating on this on the ESGF wiki.  Who else needs access?
> 
> Note: I've dropped esg-cet from the cc: list to lower the volume on
> this
> thread.
> 
> Thanks,
> 
> -Eric
> 
> Estanislao Gonzalez wrote:
> > Hi,
> >
> > I've already took the liberty to move it into esgf. The page can be
> > found here: http://esgf.org/wiki/Cmip5Status/Tests (Phil I haven't
> > gone through your doc yet)
> >
> > I also modified some things, please check this is ok. (GridFTP from
> > federation node is not possible, is it? Maybe something else was
> > meant, please check.)
> >
> > I think it's better to keep everything on the same "page". We could
> > link every test to a page only viewable by Editors or any registered
> > user, as you like. We could use Jira or ESG's bugzilla (if it's
> ready)
> > for tracking specific problems, but I do think that storing the
> > results on tickets is not the best option, as with time they get
> > obfuscated by new tickets... A table with the results, or at least
> > with the positive results, should be IMO better. What what really
> > matters is to gather all links at the sameplace and I think esgf wiki
> > does the job.
> >
> > Until we have a plugin for managing users in place I need to know
> your
> > names to add you to the editor group. Please send them two
> > me/Gavin/Luca... or any other Admin(http://esgf.org/wiki/AdminGroup).
> >
> > Thanks,
> > Estani
> >
> >
> > On 11/18/2010 05:05 PM, philip.kershaw at stfc.ac.uk wrote:
> >> Hi Eric,
> >>
> >> There's actually also the document we started at the ANL security
> meeting in Sept. :) ...
> >>
> >> https://docs.google.com/document/d/17PcZqDs0zONyUa4A3Z835Rt-
> _ePsEBKHWWP8XkawEow/edit?hl=en#
> >>
> >> This is not visible to all so I've attached a PDF also.  We could
> lift what we have in there to whatever format doc we want to use to
> share with: wiki etc.
> >>
> >> My only reservation about a wiki approach is that you don't lose the
> concept of some kind of change control.  I think it's important to keep
> track of versions with a document like this.
> >>
> >> I think we could also prepare some test tools for system wide tests.
> I've attempted to kick this off by preparing a Python test script to
> probe the SSL endpoints in the federation and check for correct
> configuration:
> >>
> >> http://proj.badc.rl.ac.uk/ndg/browser/TI12-
> security/trunk/esg_system_tests/esg/security/test/system
> >>
> >> Cheers,
> >> Phil
> >>
> >>
> >>> -----Original Message-----
> >>> From: esg-gateway-dev-bounces at mailman.earthsystemgrid.org
> [mailto:esg-
> >>> gateway-dev-bounces at mailman.earthsystemgrid.org] On Behalf Of Eric
> >>> Nienhouse
> >>> Sent: 18 November 2010 15:52
> >>> To: Cinquini, Luca (3880)
> >>> Cc: ESG-CET Mail List; go-essp-tech at ucar.edu; esg-gateway-
> >>> dev at earthsystemgrid.org
> >>> Subject: Re: [esg-gateway-dev] [ESG-CET] CMIP5 Federation Testing
> >>> Outline
> >>>
> >>> Hi Luca, All,
> >>>
> >>> Thanks for this input.  What I would propose are two "documents" to
> >>> support this testing work:
> >>>
> >>> 1)  An outline of the tests with some detail about how to perform
> them,
> >>> and what constitutes and acceptable outcome.
> >>> 2)  A tracking document to capture the tests themselves with
> comments,
> >>> dates and test related details as you note below.
> >>>
> >>> I think a wiki is ideal for the outline (1) above.  I agree
> emailing a
> >>> document around is too cumbersome.
> >>>
> >>> I think an issue tracker is ideal for (2) above.  I propose we use
> the
> >>> existing NCAR Jira system for this.  Once we have a good outline
> >>> organized we can then create the necessary Jira tasks to track
> progress
> >>> and issues.
> >>>
> >>> Does this sound reasonable?
> >>>
> >>> Thanks,
> >>>
> >>> -Eric
> >>>
> >>> Cinquini, Luca (3880) wrote:
> >>>
> >>>> Hi Eric,
> >>>> 	thanks for starting this document, the list of tests to perform
> >>>>
> >>> looks like a good start - we might want expand it in a little more
> >>> detail, like for example distinguishing between:
> >>>
> >>>> 1) http file download via browser started at the gateway
> >>>> 2) http file download via browser started at the TDS page
> >>>> 3) http multi-files download via wget script (directly to the TDS)
> >>>>
> >>>> for both home data node and federated data node.
> >>>>
> >>>> We might also want to have a matrix table where we list the result
> of
> >>>>
> >>> conducting each test between institutions (A,B), with an attached
> date
> >>> when the test was last performed.
> >>>
> >>>> As for where the document should live, I personally favor the esgf
> >>>>
> >>> wiki option, although I could also go with 3) and 4). Exchanging
> the
> >>> document by email is kind of cumbersome IMO.
> >>>
> >>>> thanks, Luca
> >>>>
> >>>> On Nov 18, 2010, at 7:25 AM, Eric Nienhouse wrote:
> >>>>
> >>>>
> >>>>
> >>>>> Dear ESG Gateway Collaborators,
> >>>>>
> >>>>> We discussed the need for federation testing in our recent GO-
> ESSP
> >>>>> call.  I would like to continue our test planning discussion and
> >>>>> collaboratively develop a document to support this effort.
> >>>>>
> >>>>> I welcome your input and insights, in particular for testing
> >>>>>
> >>> scenarios
> >>>
> >>>>> related to security and access control of CMIP5 datasets.  I plan
> to
> >>>>> serve to coordinate our testing efforts, however, I need help
> from
> >>>>> others to develop and document them.
> >>>>>
> >>>>> I have an draft outline of these key federation tests at the link
> >>>>>
> >>> below
> >>>
> >>>>> and I would like your input:
> >>>>>
> >>>>>
> >>>>>
> >>>
> https://wiki.ucar.edu/display/esgcet/CMIP5+Federation+Testing+Task+Matr
> >>> ix
> >>>
> >>>>> Several options exist to enable collaborative access to this
> >>>>>
> >>> document.
> >>>
> >>>>> I'd like suggestions from others on the best approach.  Some
> options
> >>>>>
> >>> follow:
> >>>
> >>>>> 1)  Exchange this document by email and merge edits (we've done
> this
> >>>>>
> >>> in
> >>>
> >>>>> the past)
> >>>>> 2)  Move the document to the ESGF wiki (Stephen and Estani have
> >>>>> suggested this already.)
> >>>>> 3)  Keep it on wiki.ucar.edu and provide logins to all
> >>>>>
> >>> collaborators.
> >>>
> >>>>> 4)  Move it to Google Docs.
> >>>>>
> >>>>> There is momentum to use the ESGF wiki as at least one supporting
> >>>>> document may already be there (CMIP5 Status of Gateways and
> >>>>>
> >>> Datanodes.)
> >>>
> >>>>> What do others think would be best?
> >>>>>
> >>>>> Thanks for your input on this.
> >>>>>
> >>>>> Kind regards,
> >>>>>
> >>>>> -Eric
> >>>>>
> >>>>> Eric Nienhouse wrote:
> >>>>>
> >>>>>
> >>>>>> Dear ESG Gateway Collaborators,
> >>>>>>
> >>>>>> We're quickly approaching the 1.2.0 ESG Gateway software
> release.
> >>>>>>
> >>> This
> >>>
> >>>>>> version will provide the basis for the upcoming CMIP5 activity
> data
> >>>>>> management effort.
> >>>>>>
> >>>>>> The Gateway 1.2.0 Release Candidate 1 (RC1) is now available for
> >>>>>> inter-gateway federation testing.  We encourage all who can to
> >>>>>>
> >>> upgrade
> >>>
> >>>>>> to this version as soon as is reasonably possible. (In
> particular,
> >>>>>>
> >>> if
> >>>
> >>>>>> your gateway is on the list of "Key Participating Gateways"
> below.)
> >>>>>> For installation and update details, see the included email
> below.
> >>>>>>
> >>>>>> Please note that the Gateway 1.2.0 RC1 requires a corresponding
> >>>>>>
> >>> Data
> >>>
> >>>>>> Node version 1.0.4.0 or greater.  This Gateway upgrade may
> require
> >>>>>>
> >>> an
> >>>
> >>>>>> update to your Data Node components as well.
> >>>>>>
> >>>>>> We're in need of significant inter-gateway (eg: federation)
> >>>>>>
> >>> testing.
> >>>
> >>>>>> In order to begin this process, it is important that all
> >>>>>>
> >>> participating
> >>>
> >>>>>> gateways are running a consistent pre-release 1.2.0 version.
> The
> >>>>>> 1.2.0 RC1 is the recommended version for federation related
> >>>>>>
> >>> testing.
> >>>
> >>>>>> Following are the Key Participating Gateways.  Several have
> already
> >>>>>> upgraded to the 1.2.0 RC1 version.  If you have not yet upgraded
> to
> >>>>>> the 1.2.0 RC1 version, please advise as to when this may be
> >>>>>>
> >>> reasonably
> >>>
> >>>>>> possible.  (Please send any details or questions to:
> >>>>>> esg-gateway-dev at earthsystemgrid.org)
> >>>>>>
> >>>>>>  ANU
> >>>>>>  BADC
> >>>>>>  DKRZ
> >>>>>>  JPL
> >>>>>>  LBNL
> >>>>>>  NCAR
> >>>>>>  ORNL
> >>>>>>  PCMDI
> >>>>>>
> >>>>>> Please note the NCAR testing site
> (http://esg.prototype.ucar.edu)
> >>>>>>
> >>> is
> >>>
> >>>>>> currently running the 1.2.0 RC1 version for federation testing
> >>>>>>
> >>> purposes.
> >>>
> >>>>>> I would like to begin discussing the top priority federation
> tests
> >>>>>>
> >>> and
> >>>
> >>>>>> would like to hear your input.  In particular, testing scenarios
> >>>>>> related to security and access control of CMIP5 datasets are of
> key
> >>>>>> importance.  I will send a follow up email regarding this
> testing
> >>>>>>
> >>> effort.
> >>>
> >>>>>> Thank you for your involvement in this effort!
> >>>>>>
> >>>>>> Kind Regards,
> >>>>>>
> >>>>>> -Eric
> >>>>>>
> >>>>>> On Sun, 14 Nov 2010 20:15:48 -0700
> >>>>>> Nathan Wilhelmi <wilhelmi at ucar.edu> wrote:
> >>>>>>
> >>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> We have made a new release of the gateway available, 1.2.0-RC1.
> >>>>>>>
> >>> The
> >>>
> >>>>>>> release resolves a number of issues that were identified in the
> >>>>>>>
> >>> BETA3
> >>>
> >>>>>>> release.
> >>>>>>>
> >>>>>>> The new BETA release can be downloaded directly from here:
> >>>>>>>
> >>>>>>>
> >>>
> https://vets.development.ucar.edu/nexus/content/repositories/releases/s
> >>> gf/gateway/1.2.0-RC1/
> >>>
> >>>>>>> For this release the the new installation and upgrade from
> 1.1.0
> >>>>>>>
> >>> guides
> >>>
> >>>>>>> remain the same.
> >>>>>>>
> >>>>>>> If you already have 1.2.0-BETAx installed upgrade instructions
> can
> >>>>>>>
> >>> be
> >>>
> >>>>>>> found here:
> >>>>>>>
> >>>>>>>
> >>>
> https://wiki.ucar.edu/display/esgcet/Gateway+Upgrade+Instructions+%28ve
> >>> rsion+1.2.0-BETA-1.2.0-BETA2+to+version+1.2.0-BETA3%29
> >>>
> >>>>>>>
> >>>>>>> Please provide feedback (good or bad!) to:
> >>>>>>> esg-gateway-dev at earthsystemgrid.org
> >>>>>>>
> >>>>>>> -or-
> >>>>>>>
> >>>>>>> If you have Jira Issue Tracking access, please enter a trouble
> >>>>>>>
> >>> ticket
> >>>
> >>>>>>> against version "1.2.0-RC1" under the "Affects Version/s" drop-
> >>>>>>>
> >>> down.
> >>>
> >>>>>>> Thanks!!!
> >>>>>>>
> >>>>>>> -Nate
> >>>>>>>
> >>>>>>>
> >>>>> _______________________________________________
> >>>>> ESG-CET mailing list
> >>>>> ESG-CET at earthsystemgrid.org
> >>>>> http://mailman.ucar.edu/mailman/listinfo/esg-cet
> >>>>>
> >>>>>
> >>>>
> >>> _______________________________________________
> >>> esg-gateway-dev mailing list
> >>> esg-gateway-dev at mailman.earthsystemgrid.org
> >>> http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev
> >>>
> >>
> >>
> >> _______________________________________________
> >> esg-gateway-dev mailing list
> >> esg-gateway-dev at mailman.earthsystemgrid.org
> >> http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev
> >
> >
> > --
> > Estanislao Gonzalez
> >
> > Max-Planck-Institut für Meteorologie (MPI-M)
> > Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
> > Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
> >
> > Phone:   +49 (40) 46 00 94-126
> > E-Mail:  estanislao.gonzalez at zmaw.de
> >
> 
> _______________________________________________
> esg-gateway-dev mailing list
> esg-gateway-dev at mailman.earthsystemgrid.org
> http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev
-- 
Scanned by iCritical.

More information about the GO-ESSP-TECH mailing list