[Go-essp-tech] Plan

Gavin M. Bell gavin at llnl.gov
Sat Jan 21 01:59:38 MST 2012 

Hi Cecilia et. al.

On 1/20/12 1:56 PM, Cecelia DeLuca wrote:
> Thanks Luca.
>
> Here is a plan.  I recognize that it is stupid and probably
> counterproductive to continue making suggestions.  Unfortunately
> that does not seem to stop me.  I think these things should
> not be put off and would not take very long.
No no no... not at all... certainly it is never stupid and not
counterproductive (especially coming from you).
Your plan is clearly cogent and sound - the devil is in the details. I
/tried/ to not mince too finely on points in the interleaved response
below.  Many of the items in this plan are well underway and others are
burgeoning.  It is always good to have a plan. :-)
> 1. Do the ESGF demo next Tues from 8-9.
nod
> 2. Have the metrics discussion from 9-10.
What kind of metrics? (pardon if this is a silly question - just want
clarification "metrics" means many things to many people)
Or, do you mean matrix?
> 3. Start putting together a list of users whose input
> you think would be useful.  This is a good idea even if they
> only ever end up reviewing the p2p system.
Here at PCMDI we have had a handful of key scientists looking at, using
and critiquing the P2P system we are participating in building.  The
feedback they provided has been driving most of the changes that have
been made and the overall direction of the development, as they should. 
They continue to review and give feedback that we triage and address.
> 4. Update the PCMDI gateway so that it is at least to 1.3.4
> and is in the best shape it can be given the circumstances.
I think this has been done.
> 5. Update the wiki on the ESGF site so that the
> status of all gateways and nodes is current.
Yes, we should have a way to view the overall federation status, but I
am not convinced that the manual task of updating a wiki is quite the
best use of resources and time.  In terms of the P2P system, the
mechanics of the system itself would obviate this kind of manual
reporting.  Not to mince things too finely... yes, indeed having this
information would be useful.
> 6. Identify people and a schedule for periodically testing the
> status of the federation and keeping that status on the
> wiki up to date.  Create a minimum set of federation-level
> tests and post them.  This can be really minimal but it
> is important to start and create a routine.  Useful both
> now and later.
As mentioned, there is a small group of testers, both programmers and
scientists - certainly we can and *will* expand the group of test
(vanguard) users.  Luca has put together documentation on the minimal
set of federation tests as you mentioned.  Also, there will be a series
of automated tests that should provide a baseline for the system
mechanics.  John has been emphasizing testing and is establishing a
testing working group to focus on addressing this issue.

The idea is that everyone should use the system and post bugs when they
find them.  And even better yet fix them ;-).  IMHO - First things
first... it is key to have a core from which to grow.  This core is what
has been established and to be demonstrated (the ESGF P2P Node Stack).
> 7. Create a website where known bugs of the production system
> that affect users are listed, in terms that users
> can understand.  Make users aware of it.
nod
Currently we are using our bug tracking system
(http://esgf.org/bugzilla/) and mailing lists (esgf-node-dev,
esgf-node-user) to get feedback from users.
We will also be using the website, our blog, twitter page (I know... I
know...), and RSS feed notification as conduits for disseminating
information (news, updates, etc.) to the community.
> Bonus points:
> 8. Get together a management body with key stakeholders so
> that decisions are backed up and can be communicated more
> easily.
>
Nod, the governance structure will be formalized and posted.... among
other things.
> Maybe some of this is already done.  (Going to hide now.)
>
nod
> Cecelia
>
> ps.  I looked up polemicize and tried not to.
>
:-)
I personally don't eschew polemics, such engagements can lead to quite
fruitful ends - though they do require a thick skin.

As I mentioned... first things first... and that is to meet on Tuesday
and introduce the P2P system to the go-essp community (again).... then
we go from there. IMHO
> On 1/20/2012 11:37 AM, Cinquini, Luca (3880) wrote:
>> Hi Cecilia,
>>       certainly, as we discussed many times, feedback from users if of the outmost important, and being responsive to the that feedback is even more important:)
>>
>> This can take the form of scientists that use the system and offer their suggestions freely, or of a formal users review board for ESGF. We probably need both. And off course developers
>> need to estimate what is possible and what's not, and in which time frame. So we should devote some of the future GO-ESSP calls to put something like this in place. Looks like a busy few months ahead..
>>
>> thanks, Luca
>>
>> On Jan 20, 2012, at 11:26 AM, Cecelia DeLuca wrote:
>>
>>> Luca, maybe I'm wrong but this list seems overwhelmingly populated by
>>> developers,
>>> not by scientist users.
>>>
>>> It would be good to get an "external" scientist user group together.
>>> Jennifer and
>>> Gary are a start, but it also seems important to include some scientists
>>> who are not as
>>> closely tied to infrastructure as the folks who would normally show up here.
>>>
>>> I do think that, as tough as it might be, Stephen's list also needs to
>>> be reviewed by
>>> developers.  That discussion must be concerned with not just the current
>>> state of the system, which
>>> is what users see, but also the future of the system, and developers are
>>> best positioned
>>> to comment on that.
>>>
>>> -- Cecelia
>>>
>>>
>>> On 1/20/2012 10:12 AM, Cinquini, Luca (3880) wrote:
>>>> Hi Gary,
>>>>     I think everybody on this list is invited - just like it happened for the previous one. I hope you can be there :).
>>>> thanks, Luca
>>>>
>>>> On Jan 20, 2012, at 9:58 AM, Gary Strand wrote:
>>>>
>>>>> How many users of the overall system, outside of the developers, have been invited to these demos?
>>>>>
>>>>> On Fri Jan 20, 2012, at 9:57 AM, Sylvia Murphy wrote:
>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> It seems to me that this decsion is not going to be made through demos, but by a careful analysis of the criteria for CMIP5 and how each software stack stands up to that criteria.
>>>>>>
>>>>>> We need an evaluation and testing strategy, preferably involving key stakeholders who can safely examine both packages.
>>>>>>
>>>>>> Deciding on how to do this, the timelines etc seems to me to be our highest priority.
>>>>>>
>>>>>> 2 cents
>>>>>>
>>>>>>
>>>>>> Sylvia
>>>>>> On Fri, Jan 20, 2012 at 9:49 AM, Cinquini, Luca (3880)<Luca.Cinquini at jpl.nasa.gov>   wrote:
>>>>>> Hi Stephen,
>>>>>>        I think it will be extremely important to give P2P and Gateway 2.0 the exact same amount of time to showcase their capabilities, otherwise how is CMIP5 going to make an informed decision ?
>>>>>> Yes, P2P has been demonstrated one time before, but the gateway has been demoed many more times. So I beg to disagree with you... off course, ultimately the decision of what the agenda should be lies with the GO-ESSP PIs.
>>>>>> thanks, Luca
>>>>>>
>>>>>> On Jan 20, 2012, at 9:41 AM,<stephen.pascoe at stfc.ac.uk>   wrote:
>>>>>>
>>>>>>> Hi Luca,
>>>>>>>
>>>>>>> I really don't think this is a good use of time.  I know P2P is improving all the time but I've seen a number of demos now.  It would be more effective to go through the matrix and you describe how P2P meets the requirements or how you plan to meet them.
>>>>>>>
>>>>>>> Depending on who's on the call, we may be in a position to discuss the GW2/P2P deployment strategy for CMIP5 in more detail.  Therefore, I wouldn't want to schedule something that is likely to take up most of the hour.
>>>>>>>
>>>>>>> Stephen.
>>>>>>>
>>>>>>> ---
>>>>>>> Stephen Pascoe  +44 (0)1235 445980
>>>>>>> Centre of Environmental Data Archival
>>>>>>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Cinquini, Luca (3880) [mailto:Luca.Cinquini at jpl.nasa.gov]
>>>>>>> Sent: 20 January 2012 14:29
>>>>>>> To: Eric Nienhouse
>>>>>>> Cc: Pascoe, Stephen (STFC,RAL,RALSP); go-essp-tech at ucar.edu
>>>>>>> Subject: Re: [Go-essp-tech] CMIP5 Web interface requirements matrix
>>>>>>>
>>>>>>> Hi Stephen,
>>>>>>>       thanks for the matrix...
>>>>>>> I would like to suggest that we start with a full demo of the P2P system on Tuesday. The gateway 2.0 demo took a full hour - we will try to make the P2P demo shorter, but if it ends up taking a full hour, we can always discuss the matrix at the next call.
>>>>>>>
>>>>>>> thanks, Luca
>>>>>>>
>>>>>>> On Jan 20, 2012, at 7:23 AM, Eric Nienhouse wrote:
>>>>>>>
>>>>>>>> Hi Stephen,
>>>>>>>>
>>>>>>>> Thank you for this latest version of the matrix.  It will be good to
>>>>>>>> discuss it further on Tuesday.  I would like to understand the ranking
>>>>>>>> numbers and planned evaluation process and hope this is something we can
>>>>>>>> address on the call.
>>>>>>>>
>>>>>>>> Attached is an update to the previous version I was just about to send
>>>>>>>> as your recent email arrived.  Please note 4 highlighted rows.  (2 fall
>>>>>>>> in the non-functional category.)  I trust they can be incorporated and
>>>>>>>> discussed.
>>>>>>>>
>>>>>>>> Indeed the CMIP5 system(s) are under continued development and
>>>>>>>> enhancement and the matrix inherently has some relation to the
>>>>>>>> architectural choices up to this point.  Sticking with the current
>>>>>>>> replication section for the time being seems appropriate.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> -Eric
>>>>>>>>
>>>>>>>> stephen.pascoe at stfc.ac.uk wrote:
>>>>>>>>> Dear all,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I have merged Luca's suggestions into the CMIP5 GUI requirements
>>>>>>>>> matrix and reorganised it substantially.  Apart from those items that
>>>>>>>>> are architecture-specific I include most of them, sometimes under a
>>>>>>>>> different section.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm aware there are some architectural assumptions embedded in this
>>>>>>>>> sheet.  For instance, there has been a recent discussion about whether
>>>>>>>>> master/replica is the right model for depicting replicas.  It's a pity
>>>>>>>>> we're having this discussion now!  For the moment I've kept the
>>>>>>>>> replication section as-is.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> All scores are my opinion based on variable information.  Sometimes
>>>>>>>>> I've assumed something works or is planned in P2P without verifying
>>>>>>>>> it, sometimes I've assumed it doesn't after a quick test.  Where there
>>>>>>>>> is a "?" anywhere it would be really useful for the developers to
>>>>>>>>> contribute.  To this end I'd like to go through the sheet on Tuesday,
>>>>>>>>> or make a start at least, so that I can clarify what some items mean.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I know a lot of water has flowed under the bridge since I started this
>>>>>>>>> process.  No matter what strategic decisions the CMIP5 cores centres
>>>>>>>>> take on our future deployments I hope this matrix will be useful in
>>>>>>>>> focusing on the core CMIP5 requirements.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Stephen.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Stephen Pascoe  +44 (0)1235 445980
>>>>>>>>>
>>>>>>>>> Centre of Environmental Data Archival
>>>>>>>>>
>>>>>>>>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *From:* Pascoe, Stephen (STFC,RAL,RALSP)
>>>>>>>>> *Sent:* 10 January 2012 14:32
>>>>>>>>> *To:* go-essp-tech at ucar.edu
>>>>>>>>> *Subject:* CMIP5 Web interface requirements matrix
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Dear all,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> To help the CMIP5 centres plan our upgrade to the next major release
>>>>>>>>> of the CMIP5 archive system BADC has been collating a
>>>>>>>>>
>>>>>>>>> spreadsheet of web interface requirements with input from PCMDI, DKRZ
>>>>>>>>> and MOHC (see attached).  We hope to use this sheet as a tool to plan
>>>>>>>>> migration of the CMIP5 web interface at the main CMIP5 centres: PCMDI,
>>>>>>>>> BADC and DKRZ.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The sheet is still work in progress and there is lots to discuss and
>>>>>>>>> clarify.  I have begun to suggest scores for the three systems being
>>>>>>>>> considered, Gateway 1.3.4, Gateway 2.0 and P2P, but these are
>>>>>>>>> speculative at this stage, particularly for the P2P system with which
>>>>>>>>> I have the least experience.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> If there is time I'd like to briefly introduce the sheet at the
>>>>>>>>> GO-ESSP telco today and schedule a time for a more thorough discussion.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Stephen.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Stephen Pascoe  +44 (0)1235 445980
>>>>>>>>>
>>>>>>>>> Centre of Environmental Data Archival
>>>>>>>>>
>>>>>>>>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Scanned by iCritical.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> GO-ESSP-TECH mailing list
>>>>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>>> <CMIP5_GUI_Requirements_v20120110_ejn.xlsx>_______________________________________________
>>>>>>>> GO-ESSP-TECH mailing list
>>>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>> --
>>>>>>> Scanned by iCritical.
>>>>>> _______________________________________________
>>>>>> GO-ESSP-TECH mailing list
>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ******************************************
>>>>>> Sylvia Murphy
>>>>>> NESII/CIRES/NOAA Earth System Research Laboratory
>>>>>> 325 Broadway, Boulder CO 80305
>>>>>> Time Zone: U.S. Mountain
>>>>>> Web: http://www.esrl.noaa.gov/nesii/
>>>>>> Email: sylvia.murphy at noaa.gov
>>>>>> Phone: 303-497-7753
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> GO-ESSP-TECH mailing list
>>>>>> GO-ESSP-TECH at ucar.edu
>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>> Gary Strand
>>>>> strandwg at ucar.edu
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> GO-ESSP-TECH mailing list
>>>>> GO-ESSP-TECH at ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

-- 
Gavin M. Bell
--

 "Never mistake a clear view for a short distance."
       	       -Paul Saffo


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120121/268355a0/attachment-0001.html

More information about the GO-ESSP-TECH mailing list