<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.19120"></HEAD>
<BODY bgColor=#ffffcc text=#000000>
<DIV dir=ltr align=left><SPAN class=887154408-20092011><FONT color=#0000ff
size=2 face=Arial>Hello Gavin,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=887154408-20092011><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=887154408-20092011><FONT color=#0000ff
size=2 face=Arial>thanks for this reply. Can you (or anyone) be more
specific on what catalogue entry a data user is supposed to use to understand
what files have changed? That would be really useful to know - it is
already hurting some users as they don't know how to get this information out
and so are downloading multiple copies of the same data.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=887154408-20092011><FONT color=#0000ff
size=2 face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=887154408-20092011><FONT color=#0000ff
size=2 face=Arial>Thanks,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=887154408-20092011></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=887154408-20092011><FONT color=#0000ff
size=2 face=Arial>Jamie</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="BORDER-LEFT: #0000ff 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px"
dir=ltr>
<DIV dir=ltr lang=en-us class=OutlookMessageHeader align=left>
<HR tabIndex=-1>
<FONT size=2 face=Tahoma><B>From:</B> Gavin M. Bell [mailto:gavin@llnl.gov]
<BR><B>Sent:</B> 19 September 2011 20:24<BR><B>To:</B> Kettleborough,
Jamie<BR><B>Cc:</B> Taylor, Karl Taylor; go-essp-tech@ucar.edu;
esg-node-dev@lists.llnl.gov<BR><B>Subject:</B> Re: [Go-essp-tech] Reasoning
for the use of symbolic links in drslib<BR></FONT><BR></DIV>
<DIV></DIV>Gentle-people, <BR><BR>This is what the catalogs are for.<BR>There
is too much filesystem talk. The filesystem can only do so much for
us.<BR>There is semantic knowledge that is trying to be captured in a
relatively rigid tree structure.<BR>We must out 'outside of the box', I feel
we are abusing the filesystem and trying to do gymnastics with it that we
don't need to do. The answer is not with the filesystem. The
answer is the catalogs and using them appropriately. We need to
stop hedging our bets against the system we are building and actually use it
to our benefit (the reason why we made it). The filesystem is a red
herring.<BR><BR>just my $0.02<BR><BR>I should pull out the email I wrote in
November of 2009 describing the catalog as the needed lingua fraca of this
endeavor.<BR><BR>-Cassandra.<BR><BR>On 9/19/11 9:00 AM, Kettleborough, Jamie
wrote:
<BLOCKQUOTE
cite=mid:E51EDFEBF10BE44BB4BDAF5FC2F024B90FB7F6F5@EXXMAIL02.desktop.frd.metoffice.com
type="cite">
<META name=GENERATOR content="MSHTML 8.00.6001.19120">
<DIV dir=ltr align=left><SPAN class=822044715-19092011><FONT color=#0000ff
size=2 face=Arial>Hello Karl, everyone,</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="BORDER-LEFT: rgb(0,0,255) 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px"
dir=ltr>
<DIV><FONT face="Times New Roman">For replicating the latest version, I
agree that your alternate structure poses difficulties (but it seems like
there must be a way to smartly determine whether the file you already have
a file and simply need to move it, rather than bring it over
again). </FONT><FONT color=#0000ff size=2 face=Arial><SPAN
class=822044715-19092011> </SPAN></FONT></DIV>
<DIV><FONT color=#0000ff size=2 face=Arial><SPAN
class=822044715-19092011></SPAN></FONT> </DIV></BLOCKQUOTE>
<DIV dir=ltr><SPAN class=822044715-19092011><FONT face=Arial><FONT
color=#0000ff size=2>Doesn't every user (not just the replication system)
have this problem: they want to know what files<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--> have changed (or not changed) at a new
publication version. No one wants to be using band width or storage
space to fetch and store files they already have. How is a user
expected to know what has really changed? Estani mentions check sums -
OK, but I don't think all nodes expose them (is this right?). You may
try to infer from modification dates (not sure, I haven't look at them that
closely). You may try to infer from the TRACKING_ID - but I'm not sure
how reliable this is (I can imagine scenarios where different files share
the same TRACKING_ID - e.g. if they have been modified with an nco
tool).</FONT></FONT></SPAN></DIV>
<DIV dir=ltr><SPAN class=822044715-19092011></SPAN> </DIV>
<DIV dir=ltr><SPAN class=822044715-19092011><FONT color=#0000ff size=2
face=Arial>Is there a recommended method for users to understand what
*files* have actually changed when a new publication version
appears?</FONT></SPAN></DIV>
<DIV dir=ltr><SPAN class=822044715-19092011></SPAN> </DIV>
<DIV dir=ltr><SPAN class=822044715-19092011><FONT color=#0000ff size=2
face=Arial>Thanks,</FONT></SPAN></DIV>
<DIV dir=ltr><SPAN class=822044715-19092011></SPAN> </DIV>
<DIV dir=ltr><SPAN class=822044715-19092011><FONT color=#0000ff size=2
face=Arial>Jamie</FONT></SPAN></DIV></BLOCKQUOTE><BR><PRE class=moz-signature cols="72">--
Gavin M. Bell
Lawrence Livermore National Labs
--
"Never mistake a clear view for a short distance."
-Paul Saffo
(GPG Key - <A class=moz-txt-link-freetext href="http://rainbow.llnl.gov/dist/keys/gavin.asc">http://rainbow.llnl.gov/dist/keys/gavin.asc</A>)
A796 CE39 9C31 68A4 52A7 1F6B 66B7 B250 21D5 6D3E
</PRE></BLOCKQUOTE></BODY></HTML>