[Go-essp-tech] Upcoming gateway/datanode coordination points

[Bryan Lawrence]

Hi Gavin

I'll provide a proper reply next week ... 

Meanwhile,  the thing about parterships, especially distributed 
partnerships, is that even when things are "in progress" you have to 
tell folks what you are doing and why. Sometimes knowing that there 
*might be* something to report on is just as important as the eventual 
report itself & you have to remember that not everyone is on the call 
(any call).

I know all this communication is an overhead, and a pain, but it's one 
of the necessary pieces of collaboration :-)

Have a good weeknd!

Cheers
Bryan

> Hi Bryan,
> 
> Pardon my misunderstanding... I am a bit daft when it comes to
> nuance, at least that is what my wife tells me :-).
> 
> On 3/18/11 12:45 PM, Bryan Lawrence wrote:
> > Hi Gavin
> > 
> > I think you are misunderstanding the thrust of my question, which
> > is possibily fair enough, since I was being a wee bit provocative.
> :
> :-| (see above) :-)
> :
> > I don't much care what the SLA level is, I just care that it is
> > jointly understood, documented, and everyone knows what it is. It
> > clearly isn't. At this stage I cam concerend that most of us have
> > no visibility of what you are doing, what the dependencies are for
> > the rest of us on what you are doing, what the interfaces  are,
> > and what the timetables are.
> 
> What I am doing is putting in code to generate the XML "registration"
> document that describes a node and the services that it is running
> and additional details that may be needed (this is the iterative
> part that requires the initial coordination with those more
> intimately involved with the actual component data needs) so that we
> can get a clean XSD.  I have a tweaked the initial version that
> Neil, Rachana, Nate and myself came up with, hence we should touch
> base again.  Each node will stamp out this registration document and
> put it in an agreed upon
> web-reachable location.  These files will be able to be fetched and
> aggregated and thus also made available to all who ask - i.e. a
> registry service.  This is what we are very close to having.  The
> interface? - Simple the XSD and wget/curl... et.al.  This is what
> was discussed at the last meetings and in a few meetings prior with
> a smaller group going through iterations on the xsd.  The question
> of ingestion of this data is something that is up to the actor
> pulling this data.
> 
> > Given your response: I am particularly interested in where you have
> > written detials of how the new architecture of the node will
> > "ameliorate"  if not eliminate the issue of availabity of
> > federation metadata for the security.
> 
> The current plan is to have a central registry located at PCMDI...
> essentially, the gateway functionality that Nate mentioned, on the
> Tuesday call.  The gateway now knows how to ingest and aggregate this
> information.  The last part is for the gateway to know how to fetch
> this information, something that was not detailed in Nate's
> description of the gateway but be imagined without much effort. The
> gateway in question that all will look to for this information is
> PCMDI3.
> 
> Iterating on this... it is clear, as you alluded to in your original
> query (SLAs and such), that having a central anything in a system is
> a focal point and a single point of failure.  As the "node people" I
> am interested in addressing the issue of single point of failure by
> distributing this registry responsibility.
> 
> To the finer point you make about visibility. The question is not so
> much of visibility but of things being in a state of flux.  Nothing
> is in secret, they are just *in progress* :-).  So until there is
> something to report, there's nothing to really report.  It would be
> great if everything were just done, but life and other things get
> 'in the way'... so modulo that, we are all making progress everyday
> and pushing to the goal of having a debut of the new and improved
> node.  However, this is not in the critical path.
> 
> I hope you have a clearer idea of what is going on, and what is not
> going on, and the collective path forward.  Please let us know if
> there is anything that was unclear.
> 
> Thanks again. I really enjoy this collaboration and am hoping to
> cousin some help from Phil, Stephen and your team on some of this
> next iteration work.
> 
> Oh, timelines...! This is always sticky... Let's just say the goal is
> to have all things in play in the next couple weeks or so, modulo
> Murphy's Law.... the nature of the beast.
> 
> > Thanks
> > Bryan
> > 
> >> Hi Bryan,
> >> 
> >> As the other team members mentioned in this thread, availability
> >> is not under such a stringent 'SLA' such that it will impede the
> >> functioning of the federation.  In addition the architecture of
> >> the node will greatly ameliorate, if not eliminate the issue of
> >> availability.
> >> 
> >> Indeed Phil brought up a good point with the Attribute Service.  I
> >> would like to fix a time for us to all meet and discuss the issues
> >> surrounding that service and availability.
> >> 
> >> On 3/18/11 2:15 AM, Bryan Lawrence wrote:
> >>> Hi Gavin
> >>> 
> >>> What will the API be to this registry?
> >>> What service level with PCMDI provide for it (since it's going to
> >>> have to work 24x7 globally)?
> >>> 
> >>> Thanks
> >>> Bryan
> >>> 
> >>>> We will have a single registry, it should be served from PCMDI
> >>>> on pcmdi3.  The esgf nodes will all be coded to set pcmdi3 as
> >>>> the default location for this registry.  Indeed coordination
> >>>> with Nate and Neill and Rachana will have to take place.  I
> >>>> suspect that early next week will be that time.  Stay tuned,
> >>>> we'll make this happen. ;-)
> >>>> 
> >>>> On 3/17/11 8:32 AM, philip.kershaw at stfc.ac.uk wrote:
> >>>>> So long as we are co-ordinated on this.  Will we have a single
> >>>>> registry for the federation or many?  I'm opening that question
> >>>>> more widely to all …
> >>> 
> >>> --
> >>> Bryan Lawrence
> >>> Director of Environmental Archival and Associated Research
> >>> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
> >>> STFC, Rutherford Appleton Laboratory
> >>> Phone +44 1235 445012; Fax ... 5848;
> >>> Web: home.badc.rl.ac.uk/lawrence
> > 
> > --
> > Bryan Lawrence
> > Director of Environmental Archival and Associated Research
> > (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
> > STFC, Rutherford Appleton Laboratory
> > Phone +44 1235 445012; Fax ... 5848;
> > Web: home.badc.rl.ac.uk/lawrence

--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848; 
Web: home.badc.rl.ac.uk/lawrence