<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffcc" text="#000000">
Hi Stephen, <br>
<br>
I am glad you liked the demo. We did indeed make a point to *<b>listen</b>*
to our scientists and colleagues, like you, to address their issues.<br>
<br>
(rest interleaved below)<br>
<br>
On 1/24/12 9:54 AM, <a class="moz-txt-link-abbreviated" href="mailto:stephen.pascoe@stfc.ac.uk">stephen.pascoe@stfc.ac.uk</a> wrote:
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Firstly, a great demo of the P2P system
today and I'm glad to see virtually all of the issues I've
been nagging you about are being addressed ;-). Whilst they
are on my mind I have some, hopefully constructive, comments.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
of course ;-).<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal">Metadata: we really need the CIM metadata
in there. A lot of effort has gone into filling out the CIM
metadata and the modelling centres won't be happy if it isn't
visible along with the data.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
This will be taken care of in a couple weeks or so. We would
probably schedule a short demo of in on one of the next, if not the
next go-essp call.<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal">Versions & Replicas: As you know these
features are critical for the European CMIP5 centres who are
committed to maintain a snapshot for IPCC. What Luca showed
on the JPL node is very exciting and I agree there are lots of
UI work that need to happen to make it more intuitive.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
Yes, the interface will be evolving. Also the esgf-sh shell will be
another interface into the system that command line folks may like
better. Certainly the primary point was to show that the capability
exists and viable. Point taken.<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal">Data integrity (checksums and
tracking_ids): I think search by checksum or tracking_id is
supposed to work but I can't make it happen on the JPL node.
Selecting a BADC dataset that JPL knows about and entering a
checksum or tracking_id doesn't produce any hits. I also hope
we can display the checksums in the interface so that users
can check them by hand.</p>
</div>
</blockquote>
This was working rather well in the rehearsal before the demo. It
is a small matter that we will fix. As for the display, again the
front end is evolving and will provide a more intuitive interface.
We will give the users what they want. Also there will be the
esgf-sh that will provide a shell interface with options to view the
results in many many different ways.<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Having played with pcmdi9 myself, I
actually find the sticky session feature rather confusing. If
I return to it after a few days I don't expect to see my
previous search criteria still active. If I enter text in the
"quick search" box I expect it to start a new search, not
append it to my current search. Luca showed how flexible the
Results/DataCart system is but it too can be confusing (maybe
I just have to get used to it). I've managed to show odd
things like an empty Results tab when there shouldn't be.
Some debugging is still needed.</p>
</div>
</blockquote>
This can be easily fixed... but some folks I have talked to liked
the stickiness as it provides continuity to the experience and they
don't have to rehash everything up again when they go away. We will
find the right tone for this. Also there are fixes that have been
done that are not yet deployed on pcmdi9.<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I noticed that the responsiveness of the
pcmdi9 interface was a lot better on the screencast than what
I see from my desk in the UK. It might be that the AJAX
techniques used are more susceptible to latency problems than
the traditional load-all-at-once model. I think it will be
important to test the software in a federation that includes
index nodes on other continents to make sure we don't recreate
slow search responses. I've noticed that viewing the data
cart can take a while with a few datasets selected.</p>
</div>
</blockquote>
As for speed... there are a few techniques we can do to address
that. Also we will engage in this trans-Atlantic / trans-Pacific
testing and see what's what. Much of this will be made moot as we
move data around under a more resource aware replication strategy.<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I stand by my concern about distributed
search. It would be best to give direct feedback to the user
when nodes are offline making clear that this will affect
search results. Users really notice when the number of hits
from a search result change and they want to know why.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</blockquote>
So, in my go-essp presentation about elastic networks... this p2p
network will prune nodes that are not on line to provide consistent
search results at any given point in time. When index nodes come up
they are automatically detected and sewn into the esgf p2p
dataspace. Okay, more directly addressing your concern... users can
use the dashboard to view the nodes that are in the federation when
they issue a query. As you can see from the demo, you will be able
to *see* the nodes and their state. There is also a historical
component to the dashboard monitoring that will also be present to
allow you to step through time to see the state of the federation
through time. So for the curious, the <b>dashboard</b> will be how
to exactly answer your question. <br>
<br>
There will be work moving forward to couple the RSS and the wget
procured files such that updating can be done easily. So if you
pulled down files today and an index node was down ... and there
were no replicas... (both of which make it more improbable that you
get an incomplete record), then you can run your script at some
other time when that index node is back up and/or replicas are on
line and update your downloaded files.<br>
<br>
Distributed search is good... it offers inherent consistency that
harvesting can't - by definition.<br>
<br>
All of this is just the tip of the iceberg. There is work to be
done making sure what we have it hardened. There are lots of
exciting features and capabilities on the horizon that we will be
getting done, that you are welcome to help with, that we will be
bringing to the community! :-)<br>
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B5211C999AB5@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<div class="WordSection1">
<p class="MsoNormal">Cheers,<o:p></o:p></p>
<p class="MsoNormal">Stephen.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size: 10.5pt;
font-family: Consolas;">---<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 10.5pt;
font-family: Consolas;">Stephen Pascoe +44 (0)1235 445980<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 10.5pt;
font-family: Consolas;">Centre of Environmental Data
Archival<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 10.5pt;
font-family: Consolas;">STFC Rutherford Appleton Laboratory,
Harwell Oxford, Didcot OX11 0QX, UK<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<br>
<p>-- <br>
Scanned by iCritical.
</p>
<br>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Gavin M. Bell
Lawrence Livermore National Labs
--
"Never mistake a clear view for a short distance."
-Paul Saffo
(GPG Key - <a class="moz-txt-link-freetext" href="http://rainbow.llnl.gov/dist/keys/gavin.asc">http://rainbow.llnl.gov/dist/keys/gavin.asc</a>)
A796 CE39 9C31 68A4 52A7 1F6B 66B7 B250 21D5 6D3E
</pre>
</body>
</html>