<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffcc">
Hello Jamie, <br>
<br>
Allow me to solely indulge your tangent for a moment... :-)<br>
<br>
The issue of knowing who is where etc. is solved by using a
sufficiently recent version of the ESGF "data" Node (v0.5.1+).<br>
The node-manager's registry component will automatically generate a
continuously updating descriptive (xml) document of nodes currently
present in the federation at a given time. This would have
ameliorated your task considerably.<br>
<br>
If you look at the sites you have collected; go to the
esgf-node-manager page and look at the bottom left corner for the
version.<br>
They are all earlier than v0.5.1 and hence do not have the automatic
federation feature in place.<br>
<br>
Ex:<br>
<a class="moz-txt-link-freetext" href="http://esgnode1.nci.org.au/esgf-node-manager/">http://esgnode1.nci.org.au/esgf-node-manager/</a> (v0.5.0)<br>
<a class="moz-txt-link-freetext" href="http://vesg.ipsl.fr/esgf-node-manager/">http://vesg.ipsl.fr/esgf-node-manager/</a> (v0.4.0)<br>
<a class="moz-txt-link-freetext" href="http://esg.cnrm-game-meteo.fr/esgf-node-manager/">http://esg.cnrm-game-meteo.fr/esgf-node-manager/</a> (v0.4.0)<br>
<a class="moz-txt-link-freetext" href="http://dap.cccma.uvic.ca/esgf-node-manager/">http://dap.cccma.uvic.ca/esgf-node-manager/</a> (v0.5.0)<br>
<a class="moz-txt-link-freetext" href="http://cmip-dn.badc.rl.ac.uk/esgf-node-manager/">http://cmip-dn.badc.rl.ac.uk/esgf-node-manager/</a> (v0.4.0)<br>
<br>
(NASA-GISS are not running a node manager at all)<br>
<br>
If you look at more recent node installations (version 0.5.1+) you
will see that there is a registration.xml document that is served
under esgf-node-manager. It is an active document that is
automatically updated by the node manager's registry service to
always reflect the current state of the federation.<br>
This is a feature of the new ESGF Node. Gateways are not running
node managers so they are not present in the registration.xml
document. However, you can find out about gateways indirectly by
looking at the ESGF Node's registration entry and looking at the
attribute "adminPeer" this indicates that node's target IDP service,
which in older ESG parlance indicates a "gateway". The new ESGF
Nodes are built based on a modular component architecture such that
sets of components embody functionality, these are what we call ESGF
Node "types". There are 4 node types. The node type that is
currently being installed is the well known "data" type a.k.a the
"data node", the other types are not mutually exclusive and extend
the ESGF Nodes functionality to include familiar features such as:<br>
- User credential management and single sign on support<br>
- Attribute management<br>
- Enhanced Federation-wide searching (with new search front-end)<br>
<br>
As well as recent features since v0.5.1 and pending features coming
on line such as:<br>
- Automatic fail-over and fault tolerance<br>
- New administrative front ends<br>
- Computation / Visualization tools<br>
- and more...<br>
<br>
I would suggest upgrading :-).<br>
<br>
The installation/upgrading process has been streamlined to make
things more straight forward - and the team and I are always glad to
help if needed. There are further enhancements in the queue that
will further streamline the process to make installation/upgrading
as turn-key as possible. There are also enhancements to the
federation protocol and new features as well, that will soon be
available in an upcoming v0.5.3 release that is currently in test.<br>
<br>
FYI:<br>
The current installer installs the ESGF Node at v0.5.1.<br>
In staging is v0.5.2<br>
In test is v0.5.3.<br>
<br>
Note: The list above are versions of the node manager component. As
it is a component of the ESGF Node, the node itself has a version
currently ESG Node v1.0.4+ (Stuyvesant release).<br>
<br>
The new ESGF Node augments the data node and is a complete solution
in and of itself while being compatible with the current Gateway.
It should be considered a useful tool to help the climate community
and adding to the ESG ecosystem of utilities :-).<br>
<br>
Whew... (that was a long email)<br>
I hope this was somewhat useful information in the context of your
tangent. :-)<br>
<br>
<br>
On 7/1/11 6:49 AM, Kettleborough, Jamie wrote:
<blockquote
cite="mid:E51EDFEBF10BE44BB4BDAF5FC2F024B90250C94D@EXXMAIL02.desktop.frd.metoffice.com"
type="cite">
<pre wrap="">I created this table by: looking at each gateway, figuring out which
modelling institutes contributed to the CMIP5 project, selecting a
sample data-set, creating a wget script, and then inspecting the url in
the script. (I couldn't get to any NCC data as I didn't have access).
I only sampled one dataset.
This feels a bit long winded - what is the expected way to do this?
Although today I was just gathering information on what data nodes are
out there I can imagine this as a part of a real life use case (a very
common use case). If I want to gather a diagnostic, such as monthly
mean surface temperature from as many models as I can, I think I'd have
to do this sort of trawling. OK I maybe only have to do the initial
mapping of institute to data node once, but I think there is still a
trawl needed between gateways to get the data. I may be missing
something - and I took some unnecessary steps. Please let me know if
this is the case. Estani, Martin, Sebastien - sounds like you have
already started to do this sort of thing?
I also note that not all gateways know about all institutes - I think
this is a known problem. For instance PCMDI doesn't know about IPSL,
and only NCI seems to know about CSIRO. Any ideas when this might be
resolved?
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Gavin M. Bell
Lawrence Livermore National Labs
--
"Never mistake a clear view for a short distance."
-Paul Saffo
(GPG Key - <a class="moz-txt-link-freetext" href="http://rainbow.llnl.gov/dist/keys/gavin.asc">http://rainbow.llnl.gov/dist/keys/gavin.asc</a>)
A796 CE39 9C31 68A4 52A7 1F6B 66B7 B250 21D5 6D3E
</pre>
</body>
</html>