[Go-essp-tech] Reasoning for the use of symbolic links in drslib

[Karl Taylor]

Hi Stephen,

For replicating the latest version, I agree that your alternate 
structure poses difficulties (but it seems like there must be a way to 
smartly determine whether the file you already have a file and simply 
need to move it, rather than bring it over again).

I wanted to bring up another use case in which your alternative offers 
some advantages (after a slight modification):  a user using gridftp 
wants to retrieve the latest version of datasets distributed around 
several sites (ignore replicated versions for simplicity).  If the user 
wants to write a script to retrieve the data without having to look up 
what the latest version number is for each dataset, wouldn't it be best 
to put the actual files in "latest" subdirectory, with pointers going 
from the version identified directory to "latest"?   All of these 
complications seem to be due to gridftp's inability to follow links.

Are there other ways to enable users to do the above without putting the 
actual files in a directory named "latest"?

regards,
Karl

On 9/17/11 1:16 PM, stephen.pascoe at stfc.ac.uk wrote:
>
> Hi All,
>
> I'm aware that many people are reluctant to use drslib because of it's 
> use of symbolic links when constructing the DRS directory structure.  
> I completely understand caution when using symbolic links but I wanted 
> to make the case for why I believe it is necessary to meet the goals 
> of a consistent distributed and versioned archive.  Therefore I've 
> prepared a wiki page that goes into the technical details:
>
> http://esgf.org/wiki/DrsVersionLinking
>
> Please read it carefully if you are currently considering how to 
> implement the DRS directory structure.  I propose we discuss this at 
> the next ESGF telco.
>
> Thanks,
>
> Stephen.
>
> ---
>
> Stephen Pascoe  +44 (0)1235 445980
>
> Centre of Environmental Data Archival
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
> -- 
> Scanned by iCritical.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110918/19970dec/attachment.html