[Go-essp-tech] Plan
Gavin M. Bell
gavin at llnl.gov
Sat Jan 21 04:24:29 MST 2012
Stephen, et. al.
First things first.
And the first of the first things is introducing the system on Tuesday.
We can than peruse the matrix.
We move forward by outlining the next steps.
We get the system in front of more eyes and collect feedback, etc...
Begin utilizing the channels of communication.
i.e. Posting information, etc...
...
Grow the system.
Make CMIP5 a huge success!!!!!!!! :-)
The matrix you, Stephen, have come up with is a good starting point for
a discussion, though it is not the whole story nor the sole metric to be
used as we move forward, I am sure this was the intention. No system is
perfect but a good foundation is key.
(rest of response below)
On 1/21/12 5:31 AM, stephen.pascoe at stfc.ac.uk wrote:
>
> Briefly getting back to the Agenda for Tuesday. I can make an
> extended 2 hour telco but the order we do things in will depend on who
> can attend 9-10 PTD. That is 6-7pm CET so people from DKRZ, IPSL,
> SMI etc. may not be able to make it. As well as the requirements
> matrix I'd like to cover setting up a scientific feedback panel as
> suggested by Cecelia and supported by Balaji.
>
>
>
> Inline comments follow, reordered for relevance.
>
>
>
> > 2. Have the metrics discussion from 9-10.
>
> > What kind of metrics?
>
> I think Cecelia means the requirements matrix aka spreadsheet I circulated
>
>
>
oh... ok
> > Bonus points:
> > 8. Get together a management body with key stakeholders so that decisions are backed up and can be
> > communicated more easily.
> >
>
> > Nod, the governance structure will be formalized and posted....
> among other things
>
> Gavin, who are the stakeholders going to be? This decision should be
> the result of a dialog under the auspices of GO-ESSP not just "posted".
>
Dean is the GO-ESSP P.I. he is where that process starts.
And ultimately... yes, at this stage, things get initially, "posted".
Refinement is another story.
(I am waving my hands on the details since Dean is running point on that)
more later....
>
>
>
>
>
> > 3. Start putting together a list of users whose input you think would be useful. This is a good idea even
> > if they only ever end up reviewing the p2p system.
>
> > Here at PCMDI we have had a handful of key scientists looking at,
> using and critiquing the P2P system we are participating in
>
> > building. The feedback they provided has been driving most of the
> changes that have been made and the overall direction of the
>
> > development, as they should. They continue to review and give
> feedback that we triage and address.
> Well, clearly some "key" CMIP5 scientists are feeling left out of this
> process. I think Cecelia's idea is excellent and we should act on
> it. The requirements matrix has had input from the UK Met Office and
> I'm confident they would like to be involved in something like a
> feedback panel.
>
Everyone can play... the ultimate goal is to get it in front of
*everybody*. The resources I mentioned at PCMDI are just an example as
they are my "local" resources, but certainly it is not limited to just them.
>
>
>
> We should give credit to the P2P team that they are putting together
> plenty of documentation on the ESGF wiki and the bug tracker is
> working well for developers -- I doubt many users know about it
> though! Gavin is right that keeping the federation status up to date
> manually is difficult and arduous. The P2P status system will make a
> big difference.
>
>
>
There indeed will be an education campaign to let folks know about the
system. But first we must birth this system that we have been pregnant
with for the last 18 months! :-).
>
> However the content on esgf.org is developer focussed. Scientists and
> data managers at the CMIP5 modelling centres are having to struggle
> with the current system and feel the ground shifting under their feet
> without any say in what's going on. If PCMDI want to deploy the P2P
> system within weeks then the wider CMIP5 users, not just those at
> PCMDI, need to be involved. Their work is critically dependent on
> what system we put in place.
>
>
>
Yes, esgf.org is developer centric as it is a site about the ESGF P2P
System, nothing more. There are other sites for CMIP directly that will
address scientists' concerns more directly (Ex: http://www-pcmdi.llnl.gov/).
> > 4. Update the PCMDI gateway so that it is at least to 1.3.4
> > and is in the best shape it can be given the circumstances.
>
> > I think this has been done.
> I thought it had too but it says "1.3.2" on pcmdi3 right now.
>
>
>
Hmmm... I don't know the details on that effort...
>
>
>
> > We will also be using the website, our blog, twitter page (I
> know... I know...), and RSS feed notification as conduits for
>
> > disseminating information (news, updates, etc.) to the community.
> What Gavin is hinting at here is that the ESGF blog has been a shell
> since the beginning. It's been there some 2 years but never had more
> than a couple of posts on it. I know where he's coming from here: for
> developers it's a lot more fun to setup a blog than keep it current
> and Gavin has a lot on his shoulders. I think this illustrates that
> organising ESGF should involve a broader range of people than the core
> developers.
>
>
>
Certainly.... The blog and these other conduits have had a dearth of
information (except the wiki) because we have not yet been introduced to
the larger community, so there really isn't any audience for them yet.
It is the small community on this list, etc that make up the initial
group this project is born from. Indeed the entire idea is for this
project to grow flourish. I don't intend to be the only content
provider for this project. Clearly, as we have seen with the wiki, many
people contribute and the idea is for many more to do so. The more
folks join and *participate* the more time I or others would have to add
to a blog, etc....
All things in due time. First things, first.
P.S. Setting up a blog or any of this support infrastructure is not what
I call "fun", at times it is a down right drag, but it has gotta get
done. There is a bit more infrastructure 'stuff' that will be put in
place shortly to round out all the necessary apparatus to support
development and collaboration and growth.
>
>
>
> Thanks for your attention,
>
> Stephen.
>
>
>
> ---
>
> Stephen Pascoe +44 (0)1235 445980
>
> Centre of Environmental Data Archival
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
>
> *From:*go-essp-tech-bounces at ucar.edu
> [mailto:go-essp-tech-bounces at ucar.edu] *On Behalf Of *Gavin M. Bell
> *Sent:* 21 January 2012 09:00
> *To:* Cecelia DeLuca
> *Cc:* Cinquini, Luca (3880); go-essp-tech at ucar.edu
> *Subject:* Re: [Go-essp-tech] Plan
>
>
>
> Hi Cecilia et. al.
>
> On 1/20/12 1:56 PM, Cecelia DeLuca wrote:
>
> Thanks Luca.
>
> Here is a plan. I recognize that it is stupid and probably
> counterproductive to continue making suggestions. Unfortunately
> that does not seem to stop me. I think these things should
> not be put off and would not take very long.
>
> No no no... not at all... certainly it is never stupid and not
> counterproductive (especially coming from you).
> Your plan is clearly cogent and sound - the devil is in the details. I
> /tried/ to not mince too finely on points in the interleaved response
> below. Many of the items in this plan are well underway and others
> are burgeoning. It is always good to have a plan. :-)
>
>
> 1. Do the ESGF demo next Tues from 8-9.
>
> nod
>
> 2. Have the metrics discussion from 9-10.
>
> What kind of metrics? (pardon if this is a silly question - just want
> clarification "metrics" means many things to many people)
> Or, do you mean matrix?
>
> 3. Start putting together a list of users whose input
> you think would be useful. This is a good idea even if they
> only ever end up reviewing the p2p system.
>
> Here at PCMDI we have had a handful of key scientists looking at,
> using and critiquing the P2P system we are participating in building.
> The feedback they provided has been driving most of the changes that
> have been made and the overall direction of the development, as they
> should. They continue to review and give feedback that we triage and
> address.
>
> 4. Update the PCMDI gateway so that it is at least to 1.3.4
> and is in the best shape it can be given the circumstances.
>
> I think this has been done.
>
> 5. Update the wiki on the ESGF site so that the
> status of all gateways and nodes is current.
>
> Yes, we should have a way to view the overall federation status, but I
> am not convinced that the manual task of updating a wiki is quite the
> best use of resources and time. In terms of the P2P system, the
> mechanics of the system itself would obviate this kind of manual
> reporting. Not to mince things too finely... yes, indeed having this
> information would be useful.
>
> 6. Identify people and a schedule for periodically testing the
> status of the federation and keeping that status on the
> wiki up to date. Create a minimum set of federation-level
> tests and post them. This can be really minimal but it
> is important to start and create a routine. Useful both
> now and later.
>
> As mentioned, there is a small group of testers, both programmers and
> scientists - certainly we can and *will* expand the group of test
> (vanguard) users. Luca has put together documentation on the minimal
> set of federation tests as you mentioned. Also, there will be a
> series of automated tests that should provide a baseline for the
> system mechanics. John has been emphasizing testing and is
> establishing a testing working group to focus on addressing this issue.
>
> The idea is that everyone should use the system and post bugs when
> they find them. And even better yet fix them ;-). IMHO - First
> things first... it is key to have a core from which to grow. This
> core is what has been established and to be demonstrated (the ESGF P2P
> Node Stack).
>
> 7. Create a website where known bugs of the production system
> that affect users are listed, in terms that users
> can understand. Make users aware of it.
>
> nod
> Currently we are using our bug tracking system
> (http://esgf.org/bugzilla/) and mailing lists (esgf-node-dev,
> esgf-node-user) to get feedback from users.
> We will also be using the website, our blog, twitter page (I know... I
> know...), and RSS feed notification as conduits for disseminating
> information (news, updates, etc.) to the community.
>
>
> Bonus points:
> 8. Get together a management body with key stakeholders so
> that decisions are backed up and can be communicated more
> easily.
>
>
> Nod, the governance structure will be formalized and posted.... among
> other things.
>
> Maybe some of this is already done. (Going to hide now.)
>
>
> nod
>
> Cecelia
>
> ps. I looked up polemicize and tried not to.
>
>
> :-)
> I personally don't eschew polemics, such engagements can lead to quite
> fruitful ends - though they do require a thick skin.
>
> As I mentioned... first things first... and that is to meet on Tuesday
> and introduce the P2P system to the go-essp community (again).... then
> we go from there. IMHO
>
> On 1/20/2012 11:37 AM, Cinquini, Luca (3880) wrote:
>
> Hi Cecilia,
>
> certainly, as we discussed many times, feedback from users if of the outmost important, and being responsive to the that feedback is even more important:)
>
>
>
> This can take the form of scientists that use the system and offer their suggestions freely, or of a formal users review board for ESGF. We probably need both. And off course developers
>
> need to estimate what is possible and what's not, and in which time frame. So we should devote some of the future GO-ESSP calls to put something like this in place. Looks like a busy few months ahead..
>
>
>
> thanks, Luca
>
>
>
> On Jan 20, 2012, at 11:26 AM, Cecelia DeLuca wrote:
>
>
>
> Luca, maybe I'm wrong but this list seems overwhelmingly populated by
>
> developers,
>
> not by scientist users.
>
>
>
> It would be good to get an "external" scientist user group together.
>
> Jennifer and
>
> Gary are a start, but it also seems important to include some scientists
>
> who are not as
>
> closely tied to infrastructure as the folks who would normally show up here.
>
>
>
> I do think that, as tough as it might be, Stephen's list also needs to
>
> be reviewed by
>
> developers. That discussion must be concerned with not just the current
>
> state of the system, which
>
> is what users see, but also the future of the system, and developers are
>
> best positioned
>
> to comment on that.
>
>
>
> -- Cecelia
>
>
>
>
>
> On 1/20/2012 10:12 AM, Cinquini, Luca (3880) wrote:
>
> Hi Gary,
>
> I think everybody on this list is invited - just like it happened for the previous one. I hope you can be there :).
>
> thanks, Luca
>
>
>
> On Jan 20, 2012, at 9:58 AM, Gary Strand wrote:
>
>
>
> How many users of the overall system, outside of the developers, have been invited to these demos?
>
>
>
> On Fri Jan 20, 2012, at 9:57 AM, Sylvia Murphy wrote:
>
>
>
> Hi Everyone,
>
>
>
> It seems to me that this decsion is not going to be made through demos, but by a careful analysis of the criteria for CMIP5 and how each software stack stands up to that criteria.
>
>
>
> We need an evaluation and testing strategy, preferably involving key stakeholders who can safely examine both packages.
>
>
>
> Deciding on how to do this, the timelines etc seems to me to be our highest priority.
>
>
>
> 2 cents
>
>
>
>
>
> Sylvia
>
> On Fri, Jan 20, 2012 at 9:49 AM, Cinquini, Luca (3880)<Luca.Cinquini at jpl.nasa.gov> <mailto:Luca.Cinquini at jpl.nasa.gov> wrote:
>
> Hi Stephen,
>
> I think it will be extremely important to give P2P and Gateway 2.0 the exact same amount of time to showcase their capabilities, otherwise how is CMIP5 going to make an informed decision ?
>
> Yes, P2P has been demonstrated one time before, but the gateway has been demoed many more times. So I beg to disagree with you... off course, ultimately the decision of what the agenda should be lies with the GO-ESSP PIs.
>
> thanks, Luca
>
>
>
> On Jan 20, 2012, at 9:41 AM,<stephen.pascoe at stfc.ac.uk> <mailto:stephen.pascoe at stfc.ac.uk> wrote:
>
>
>
> Hi Luca,
>
>
>
> I really don't think this is a good use of time. I know P2P is improving all the time but I've seen a number of demos now. It would be more effective to go through the matrix and you describe how P2P meets the requirements or how you plan to meet them.
>
>
>
> Depending on who's on the call, we may be in a position to discuss the GW2/P2P deployment strategy for CMIP5 in more detail. Therefore, I wouldn't want to schedule something that is likely to take up most of the hour.
>
>
>
> Stephen.
>
>
>
> ---
>
> Stephen Pascoe +44 (0)1235 445980
>
> Centre of Environmental Data Archival
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
>
>
>
> -----Original Message-----
>
> From: Cinquini, Luca (3880) [mailto:Luca.Cinquini at jpl.nasa.gov]
>
> Sent: 20 January 2012 14:29
>
> To: Eric Nienhouse
>
> Cc: Pascoe, Stephen (STFC,RAL,RALSP); go-essp-tech at ucar.edu <mailto:go-essp-tech at ucar.edu>
>
> Subject: Re: [Go-essp-tech] CMIP5 Web interface requirements matrix
>
>
>
> Hi Stephen,
>
> thanks for the matrix...
>
> I would like to suggest that we start with a full demo of the P2P system on Tuesday. The gateway 2.0 demo took a full hour - we will try to make the P2P demo shorter, but if it ends up taking a full hour, we can always discuss the matrix at the next call.
>
>
>
> thanks, Luca
>
>
>
> On Jan 20, 2012, at 7:23 AM, Eric Nienhouse wrote:
>
>
>
> Hi Stephen,
>
>
>
> Thank you for this latest version of the matrix. It will be good to
>
> discuss it further on Tuesday. I would like to understand the ranking
>
> numbers and planned evaluation process and hope this is something we can
>
> address on the call.
>
>
>
> Attached is an update to the previous version I was just about to send
>
> as your recent email arrived. Please note 4 highlighted rows. (2 fall
>
> in the non-functional category.) I trust they can be incorporated and
>
> discussed.
>
>
>
> Indeed the CMIP5 system(s) are under continued development and
>
> enhancement and the matrix inherently has some relation to the
>
> architectural choices up to this point. Sticking with the current
>
> replication section for the time being seems appropriate.
>
>
>
> Thanks,
>
>
>
> -Eric
>
>
>
> stephen.pascoe at stfc.ac.uk <mailto:stephen.pascoe at stfc.ac.uk> wrote:
>
> Dear all,
>
>
>
>
>
>
>
> I have merged Luca's suggestions into the CMIP5 GUI requirements
>
> matrix and reorganised it substantially. Apart from those items that
>
> are architecture-specific I include most of them, sometimes under a
>
> different section.
>
>
>
>
>
>
>
> I'm aware there are some architectural assumptions embedded in this
>
> sheet. For instance, there has been a recent discussion about whether
>
> master/replica is the right model for depicting replicas. It's a pity
>
> we're having this discussion now! For the moment I've kept the
>
> replication section as-is.
>
>
>
>
>
>
>
> All scores are my opinion based on variable information. Sometimes
>
> I've assumed something works or is planned in P2P without verifying
>
> it, sometimes I've assumed it doesn't after a quick test. Where there
>
> is a "?" anywhere it would be really useful for the developers to
>
> contribute. To this end I'd like to go through the sheet on Tuesday,
>
> or make a start at least, so that I can clarify what some items mean.
>
>
>
>
>
>
>
> I know a lot of water has flowed under the bridge since I started this
>
> process. No matter what strategic decisions the CMIP5 cores centres
>
> take on our future deployments I hope this matrix will be useful in
>
> focusing on the core CMIP5 requirements.
>
>
>
>
>
>
>
> Thanks,
>
>
>
> Stephen.
>
>
>
>
>
>
>
> ---
>
>
>
> Stephen Pascoe +44 (0)1235 445980
>
>
>
> Centre of Environmental Data Archival
>
>
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
>
>
>
>
>
> *From:* Pascoe, Stephen (STFC,RAL,RALSP)
>
> *Sent:* 10 January 2012 14:32
>
> *To:* go-essp-tech at ucar.edu <mailto:go-essp-tech at ucar.edu>
>
> *Subject:* CMIP5 Web interface requirements matrix
>
>
>
>
>
>
>
> Dear all,
>
>
>
>
>
>
>
> To help the CMIP5 centres plan our upgrade to the next major release
>
> of the CMIP5 archive system BADC has been collating a
>
>
>
> spreadsheet of web interface requirements with input from PCMDI, DKRZ
>
> and MOHC (see attached). We hope to use this sheet as a tool to plan
>
> migration of the CMIP5 web interface at the main CMIP5 centres: PCMDI,
>
> BADC and DKRZ.
>
>
>
>
>
>
>
> The sheet is still work in progress and there is lots to discuss and
>
> clarify. I have begun to suggest scores for the three systems being
>
> considered, Gateway 1.3.4, Gateway 2.0 and P2P, but these are
>
> speculative at this stage, particularly for the P2P system with which
>
> I have the least experience.
>
>
>
>
>
>
>
> If there is time I'd like to briefly introduce the sheet at the
>
> GO-ESSP telco today and schedule a time for a more thorough discussion.
>
>
>
>
>
>
>
> Cheers,
>
>
>
> Stephen.
>
>
>
>
>
>
>
> ---
>
>
>
> Stephen Pascoe +44 (0)1235 445980
>
>
>
> Centre of Environmental Data Archival
>
>
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
>
>
>
>
>
>
>
> --
>
> Scanned by iCritical.
>
>
>
>
>
> ------------------------------------------------------------------------
>
>
>
> _______________________________________________
>
> GO-ESSP-TECH mailing list
>
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
> <CMIP5_GUI_Requirements_v20120110_ejn.xlsx>_______________________________________________
>
> GO-ESSP-TECH mailing list
>
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
> --
>
> Scanned by iCritical.
>
> _______________________________________________
>
> GO-ESSP-TECH mailing list
>
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
>
>
>
>
>
> --
>
> ******************************************
>
> Sylvia Murphy
>
> NESII/CIRES/NOAA Earth System Research Laboratory
>
> 325 Broadway, Boulder CO 80305
>
> Time Zone: U.S. Mountain
>
> Web: http://www.esrl.noaa.gov/nesii/
>
> Email: sylvia.murphy at noaa.gov <mailto:sylvia.murphy at noaa.gov>
>
> Phone: 303-497-7753
>
>
>
>
>
> _______________________________________________
>
> GO-ESSP-TECH mailing list
>
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
> Gary Strand
>
> strandwg at ucar.edu <mailto:strandwg at ucar.edu>
>
>
>
>
>
>
>
> _______________________________________________
>
> GO-ESSP-TECH mailing list
>
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
> _______________________________________________
>
> GO-ESSP-TECH mailing list
>
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu <mailto:GO-ESSP-TECH at ucar.edu>
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
>
>
> --
> Gavin M. Bell
> --
>
> "Never mistake a clear view for a short distance."
> -Paul Saffo
>
>
> --
> Scanned by iCritical.
>
>
--
Gavin M. Bell
--
"Never mistake a clear view for a short distance."
-Paul Saffo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120121/048edce5/attachment-0001.html
More information about the GO-ESSP-TECH
mailing list