[Go-essp-tech] Upcoming gateway/datanode coordination points
philip.kershaw at stfc.ac.uk
philip.kershaw at stfc.ac.uk
Fri Apr 1 08:02:59 MDT 2011
Hi Gavin,
Any news on a finalised schema?
Cheers,
Phil
On 29/03/2011 21:09, "Eric Nienhouse" <ejn at ucar.edu> wrote:
>Hi Gavin and the Data Node Team,
>
>Good to catch up on the GO-ESSP call today and thanks for the discussion
>on the Federation Registry. I feel we need to further clarify the "who,
>what and when" of the centralized federation registry service. Based on
>the email discussions and today's call I understand the following:
>
> * A centralized federation registry service is critical to
> simplifying ESG federation management (fed gw, nodes, truststore,
> etc.)
> * Work is underway to develop an XML schema to describe the
> federation gateways and data nodes.
> * There is desire to keep the registry service simple and to support
> production use as soon as possible (ideally for v. 1.3.0)
> * PCMDI has proposed a federation registry solution based on the
> data node manager and work is underway to develop a test service.
> * The production federation registry service is planned to reside at
> PCMDI.
> * A temporary federation registry XML document is available for
> 1.3.0 federation testing at NCAR.
>
>Please let me know if I've misrepresented any of the above. I realize
>we're all busy and moving quickly on this.
>
>As we've all discussed in email and on the call this morning, we as a
>collaboration need more information about the federation registry
>service under development.
>
> * What is the scope of the registry service under development?
> * When will it be available for testing?
> * What is the general architecture of the federation registry
>system?
> * The registry service will be Hessian initially (REST proposed
> for future.) Please share the interface specification of the test
> service.
> * You noted tweaks to the federation registry XML schema - has it
> changed? Please share the current form of the registry schema.
> * What are the plans for deploying a production federation
> registry service on pcmdi3 (time-line, basic service level)?
>
>This information is particularly useful for coordinating ESG federation
>software releases which have been difficult to align in the past. As
>Phil, Rachana and others have noted, getting a simple solution in place
>first, then later refining it, has benefited us in the past. We're
>simply looking to learn more about the work underway, see how we can
>help out and best collaborate on this federation piece. We greatly
>appreciate you taking on this integration effort in support of the ESG
>federation.
>
>Regards,
>
>-Eric
>
>Bryan Lawrence wrote:
>> Hi Gavin
>>
>> I'll provide a proper reply next week ...
>>
>> Meanwhile, the thing about parterships, especially distributed
>> partnerships, is that even when things are "in progress" you have to
>> tell folks what you are doing and why. Sometimes knowing that there
>> *might be* something to report on is just as important as the eventual
>> report itself & you have to remember that not everyone is on the call
>> (any call).
>>
>> I know all this communication is an overhead, and a pain, but it's one
>> of the necessary pieces of collaboration :-)
>>
>> Have a good weeknd!
>>
>> Cheers
>> Bryan
>>
>>
>>> Hi Bryan,
>>>
>>> Pardon my misunderstanding... I am a bit daft when it comes to
>>> nuance, at least that is what my wife tells me :-).
>>>
>>> On 3/18/11 12:45 PM, Bryan Lawrence wrote:
>>>
>>>> Hi Gavin
>>>>
>>>> I think you are misunderstanding the thrust of my question, which
>>>> is possibily fair enough, since I was being a wee bit provocative.
>>>>
>>> :
>>> :-| (see above) :-)
>>> :
>>>
>>>> I don't much care what the SLA level is, I just care that it is
>>>> jointly understood, documented, and everyone knows what it is. It
>>>> clearly isn't. At this stage I cam concerend that most of us have
>>>> no visibility of what you are doing, what the dependencies are for
>>>> the rest of us on what you are doing, what the interfaces are,
>>>> and what the timetables are.
>>>>
>>> What I am doing is putting in code to generate the XML "registration"
>>> document that describes a node and the services that it is running
>>> and additional details that may be needed (this is the iterative
>>> part that requires the initial coordination with those more
>>> intimately involved with the actual component data needs) so that we
>>> can get a clean XSD. I have a tweaked the initial version that
>>> Neil, Rachana, Nate and myself came up with, hence we should touch
>>> base again. Each node will stamp out this registration document and
>>> put it in an agreed upon
>>> web-reachable location. These files will be able to be fetched and
>>> aggregated and thus also made available to all who ask - i.e. a
>>> registry service. This is what we are very close to having. The
>>> interface? - Simple the XSD and wget/curl... et.al. This is what
>>> was discussed at the last meetings and in a few meetings prior with
>>> a smaller group going through iterations on the xsd. The question
>>> of ingestion of this data is something that is up to the actor
>>> pulling this data.
>>>
>>>
>>>> Given your response: I am particularly interested in where you have
>>>> written detials of how the new architecture of the node will
>>>> "ameliorate" if not eliminate the issue of availabity of
>>>> federation metadata for the security.
>>>>
>>> The current plan is to have a central registry located at PCMDI...
>>> essentially, the gateway functionality that Nate mentioned, on the
>>> Tuesday call. The gateway now knows how to ingest and aggregate this
>>> information. The last part is for the gateway to know how to fetch
>>> this information, something that was not detailed in Nate's
>>> description of the gateway but be imagined without much effort. The
>>> gateway in question that all will look to for this information is
>>> PCMDI3.
>>>
>>> Iterating on this... it is clear, as you alluded to in your original
>>> query (SLAs and such), that having a central anything in a system is
>>> a focal point and a single point of failure. As the "node people" I
>>> am interested in addressing the issue of single point of failure by
>>> distributing this registry responsibility.
>>>
>>> To the finer point you make about visibility. The question is not so
>>> much of visibility but of things being in a state of flux. Nothing
>>> is in secret, they are just *in progress* :-). So until there is
>>> something to report, there's nothing to really report. It would be
>>> great if everything were just done, but life and other things get
>>> 'in the way'... so modulo that, we are all making progress everyday
>>> and pushing to the goal of having a debut of the new and improved
>>> node. However, this is not in the critical path.
>>>
>>> I hope you have a clearer idea of what is going on, and what is not
>>> going on, and the collective path forward. Please let us know if
>>> there is anything that was unclear.
>>>
>>> Thanks again. I really enjoy this collaboration and am hoping to
>>> cousin some help from Phil, Stephen and your team on some of this
>>> next iteration work.
>>>
>>> Oh, timelines...! This is always sticky... Let's just say the goal is
>>> to have all things in play in the next couple weeks or so, modulo
>>> Murphy's Law.... the nature of the beast.
>>>
>>>
>>>> Thanks
>>>> Bryan
>>>>
>>>>
>>>>> Hi Bryan,
>>>>>
>>>>> As the other team members mentioned in this thread, availability
>>>>> is not under such a stringent 'SLA' such that it will impede the
>>>>> functioning of the federation. In addition the architecture of
>>>>> the node will greatly ameliorate, if not eliminate the issue of
>>>>> availability.
>>>>>
>>>>> Indeed Phil brought up a good point with the Attribute Service. I
>>>>> would like to fix a time for us to all meet and discuss the issues
>>>>> surrounding that service and availability.
>>>>>
>>>>> On 3/18/11 2:15 AM, Bryan Lawrence wrote:
>>>>>
>>>>>> Hi Gavin
>>>>>>
>>>>>> What will the API be to this registry?
>>>>>> What service level with PCMDI provide for it (since it's going to
>>>>>> have to work 24x7 globally)?
>>>>>>
>>>>>> Thanks
>>>>>> Bryan
>>>>>>
>>>>>>
>>>>>>> We will have a single registry, it should be served from PCMDI
>>>>>>> on pcmdi3. The esgf nodes will all be coded to set pcmdi3 as
>>>>>>> the default location for this registry. Indeed coordination
>>>>>>> with Nate and Neill and Rachana will have to take place. I
>>>>>>> suspect that early next week will be that time. Stay tuned,
>>>>>>> we'll make this happen. ;-)
>>>>>>>
>>>>>>> On 3/17/11 8:32 AM, philip.kershaw at stfc.ac.uk wrote:
>>>>>>>
>>>>>>>> So long as we are co-ordinated on this. Will we have a single
>>>>>>>> registry for the federation or many? I'm opening that question
>>>>>>>> more widely to all Š
>>>>>>>>
>>>>>> --
>>>>>> Bryan Lawrence
>>>>>> Director of Environmental Archival and Associated Research
>>>>>> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
>>>>>> STFC, Rutherford Appleton Laboratory
>>>>>> Phone +44 1235 445012; Fax ... 5848;
>>>>>> Web: home.badc.rl.ac.uk/lawrence
>>>>>>
>>>> --
>>>> Bryan Lawrence
>>>> Director of Environmental Archival and Associated Research
>>>> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
>>>> STFC, Rutherford Appleton Laboratory
>>>> Phone +44 1235 445012; Fax ... 5848;
>>>> Web: home.badc.rl.ac.uk/lawrence
>>>>
>>
>> --
>> Bryan Lawrence
>> Director of Environmental Archival and Associated Research
>> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
>> STFC, Rutherford Appleton Laboratory
>> Phone +44 1235 445012; Fax ... 5848;
>> Web: home.badc.rl.ac.uk/lawrence
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>
>
>_______________________________________________
>GO-ESSP-TECH mailing list
>GO-ESSP-TECH at ucar.edu
>http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
--
Scanned by iCritical.
More information about the GO-ESSP-TECH
mailing list