[Go-essp-tech] Upcoming gateway/datanode coordination points
philip.kershaw at stfc.ac.uk
philip.kershaw at stfc.ac.uk
Mon Apr 4 04:19:11 MDT 2011
Hi Gavin,
Do you have any more news on this?
I thought that the schema was already finalised and ready to go. I'm concerned that we were targeting this to be ready last week and time is moving on.
The registry is a key piece for everyone. For example, without the whitelisting configuration it makes it difficult to protect services and consequently users' personal information.
Cheers,
Phil
From: "Gavin M. Bell" <gavin at llnl.gov<mailto:gavin at llnl.gov>>
Date: Fri, 1 Apr 2011 08:38:58 -0700
To: Philip Kershaw <philip.kershaw at stfc.ac.uk<mailto:philip.kershaw at stfc.ac.uk>>
Cc: "go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>" <go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>>, "ejn at ucar.edu<mailto:ejn at ucar.edu>" <ejn at ucar.edu<mailto:ejn at ucar.edu>>, Dean Williams <williams13 at llnl.gov<mailto:williams13 at llnl.gov>>
Subject: Re: [Go-essp-tech] Upcoming gateway/datanode coordination points
Almost done with it.
On 4/1/11 7:02 AM, philip.kershaw at stfc.ac.uk<mailto:philip.kershaw at stfc.ac.uk> wrote:
Hi Gavin,
Any news on a finalised schema?
Cheers,
Phil
On 29/03/2011 21:09, "Eric Nienhouse" <ejn at ucar.edu><mailto:ejn at ucar.edu> wrote:
Hi Gavin and the Data Node Team,
Good to catch up on the GO-ESSP call today and thanks for the discussion
on the Federation Registry. I feel we need to further clarify the "who,
what and when" of the centralized federation registry service. Based on
the email discussions and today's call I understand the following:
* A centralized federation registry service is critical to
simplifying ESG federation management (fed gw, nodes, truststore,
etc.)
* Work is underway to develop an XML schema to describe the
federation gateways and data nodes.
* There is desire to keep the registry service simple and to support
production use as soon as possible (ideally for v. 1.3.0)
* PCMDI has proposed a federation registry solution based on the
data node manager and work is underway to develop a test service.
* The production federation registry service is planned to reside at
PCMDI.
* A temporary federation registry XML document is available for
1.3.0 federation testing at NCAR.
Please let me know if I've misrepresented any of the above. I realize
we're all busy and moving quickly on this.
As we've all discussed in email and on the call this morning, we as a
collaboration need more information about the federation registry
service under development.
* What is the scope of the registry service under development?
* When will it be available for testing?
* What is the general architecture of the federation registry
system?
* The registry service will be Hessian initially (REST proposed
for future.) Please share the interface specification of the test
service.
* You noted tweaks to the federation registry XML schema - has it
changed? Please share the current form of the registry schema.
* What are the plans for deploying a production federation
registry service on pcmdi3 (time-line, basic service level)?
This information is particularly useful for coordinating ESG federation
software releases which have been difficult to align in the past. As
Phil, Rachana and others have noted, getting a simple solution in place
first, then later refining it, has benefited us in the past. We're
simply looking to learn more about the work underway, see how we can
help out and best collaborate on this federation piece. We greatly
appreciate you taking on this integration effort in support of the ESG
federation.
Regards,
-Eric
Bryan Lawrence wrote:
Hi Gavin
I'll provide a proper reply next week ...
Meanwhile, the thing about parterships, especially distributed
partnerships, is that even when things are "in progress" you have to
tell folks what you are doing and why. Sometimes knowing that there
*might be* something to report on is just as important as the eventual
report itself & you have to remember that not everyone is on the call
(any call).
I know all this communication is an overhead, and a pain, but it's one
of the necessary pieces of collaboration :-)
Have a good weeknd!
Cheers
Bryan
Hi Bryan,
Pardon my misunderstanding... I am a bit daft when it comes to
nuance, at least that is what my wife tells me :-).
On 3/18/11 12:45 PM, Bryan Lawrence wrote:
Hi Gavin
I think you are misunderstanding the thrust of my question, which
is possibily fair enough, since I was being a wee bit provocative.
:
:-| (see above) :-)
:
I don't much care what the SLA level is, I just care that it is
jointly understood, documented, and everyone knows what it is. It
clearly isn't. At this stage I cam concerend that most of us have
no visibility of what you are doing, what the dependencies are for
the rest of us on what you are doing, what the interfaces are,
and what the timetables are.
What I am doing is putting in code to generate the XML "registration"
document that describes a node and the services that it is running
and additional details that may be needed (this is the iterative
part that requires the initial coordination with those more
intimately involved with the actual component data needs) so that we
can get a clean XSD. I have a tweaked the initial version that
Neil, Rachana, Nate and myself came up with, hence we should touch
base again. Each node will stamp out this registration document and
put it in an agreed upon
web-reachable location. These files will be able to be fetched and
aggregated and thus also made available to all who ask - i.e. a
registry service. This is what we are very close to having. The
interface? - Simple the XSD and wget/curl... et.al. This is what
was discussed at the last meetings and in a few meetings prior with
a smaller group going through iterations on the xsd. The question
of ingestion of this data is something that is up to the actor
pulling this data.
Given your response: I am particularly interested in where you have
written detials of how the new architecture of the node will
"ameliorate" if not eliminate the issue of availabity of
federation metadata for the security.
The current plan is to have a central registry located at PCMDI...
essentially, the gateway functionality that Nate mentioned, on the
Tuesday call. The gateway now knows how to ingest and aggregate this
information. The last part is for the gateway to know how to fetch
this information, something that was not detailed in Nate's
description of the gateway but be imagined without much effort. The
gateway in question that all will look to for this information is
PCMDI3.
Iterating on this... it is clear, as you alluded to in your original
query (SLAs and such), that having a central anything in a system is
a focal point and a single point of failure. As the "node people" I
am interested in addressing the issue of single point of failure by
distributing this registry responsibility.
To the finer point you make about visibility. The question is not so
much of visibility but of things being in a state of flux. Nothing
is in secret, they are just *in progress* :-). So until there is
something to report, there's nothing to really report. It would be
great if everything were just done, but life and other things get
'in the way'... so modulo that, we are all making progress everyday
and pushing to the goal of having a debut of the new and improved
node. However, this is not in the critical path.
I hope you have a clearer idea of what is going on, and what is not
going on, and the collective path forward. Please let us know if
there is anything that was unclear.
Thanks again. I really enjoy this collaboration and am hoping to
cousin some help from Phil, Stephen and your team on some of this
next iteration work.
Oh, timelines...! This is always sticky... Let's just say the goal is
to have all things in play in the next couple weeks or so, modulo
Murphy's Law.... the nature of the beast.
Thanks
Bryan
Hi Bryan,
As the other team members mentioned in this thread, availability
is not under such a stringent 'SLA' such that it will impede the
functioning of the federation. In addition the architecture of
the node will greatly ameliorate, if not eliminate the issue of
availability.
Indeed Phil brought up a good point with the Attribute Service. I
would like to fix a time for us to all meet and discuss the issues
surrounding that service and availability.
On 3/18/11 2:15 AM, Bryan Lawrence wrote:
Hi Gavin
What will the API be to this registry?
What service level with PCMDI provide for it (since it's going to
have to work 24x7 globally)?
Thanks
Bryan
We will have a single registry, it should be served from PCMDI
on pcmdi3. The esgf nodes will all be coded to set pcmdi3 as
the default location for this registry. Indeed coordination
with Nate and Neill and Rachana will have to take place. I
suspect that early next week will be that time. Stay tuned,
we'll make this happen. ;-)
On 3/17/11 8:32 AM, philip.kershaw at stfc.ac.uk<mailto:philip.kershaw at stfc.ac.uk> wrote:
So long as we are co-ordinated on this. Will we have a single
registry for the federation or many? I'm opening that question
more widely to all Š
--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848;
Web: home.badc.rl.ac.uk/lawrence
--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848;
Web: home.badc.rl.ac.uk/lawrence
--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848;
Web: home.badc.rl.ac.uk/lawrence
_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
--
Gavin M. Bell
--
"Never mistake a clear view for a short distance."
-Paul Saffo
--
Scanned by iCritical.
More information about the GO-ESSP-TECH
mailing list