<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.19154"></HEAD>
<BODY bgColor=#ffffff text=#000000>
<BLOCKQUOTE
style="BORDER-LEFT: #0000ff 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px"
dir=ltr>
<DIV>So, the way I picture this is:<BR>1) get the list of files to be
downloaded (in the wget script or by any other means)<BR>2) filter that to
remove what is not required<BR><SPAN
class=315445615-14122011> </SPAN></DIV></BLOCKQUOTE><SPAN
class=315445615-14122011>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN
class=315445615-14122011>This is basically what we do MO - we create list
of files to download, then compare it with our local file system, and we filter
out any we already have. I think the replication system would have to do
this too wouldn't it? For what its worth I think *every* user
has their own version of the replication problem - just the set of files they
are trying to replicate is different and they might be using a different
protocol to fetch the data.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN
class=315445615-14122011></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT color=#0000ff size=2 face=Arial><SPAN
class=315445615-14122011>If you accept this way of working as
valid/acceptable/encouraged then does it have implications for the (scriptable)
interfaces to either P2P and or gateway 2? I think it means there
'should' be an interface that returns a list of files (not wrapped in a wget
script) and then maybe a service (either client side or server side) that will
take a list of urls and generate the wget scripts. If you only have an
interface that returns wget scripts then users will have to parse these to
enable them to filter out the files they already have copies
of.</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff size=2 face=Arial></FONT> </DIV>
<DIV><SPAN class=315445615-14122011><FONT color=#0000ff size=2
face=Arial>Jamie</FONT></SPAN></DIV>
<DIV><SPAN class=315445615-14122011><FONT color=#0000ff size=2
face=Arial></FONT></SPAN> </DIV>
<DIV><SPAN class=315445615-14122011><FONT color=#0000ff size=2
face=Arial>(Sebastien - I'm aware this sort of touches on a set
of unanswered questions you asked a while ago related to what we
do at the MO... I've not forgotten I want to answer this is more
detail, apologies for being so rubbish at answering so
far). </FONT></SPAN></DIV></SPAN></BODY></HTML>