[Go-essp-tech] Reasoning for the use of symbolic links in drslib
Karl Taylor
taylor13 at llnl.gov
Sun Sep 18 12:25:00 MDT 2011
Hi Stephen,
For replicating the latest version, I agree that your alternate
structure poses difficulties (but it seems like there must be a way to
smartly determine whether the file you already have a file and simply
need to move it, rather than bring it over again).
I wanted to bring up another use case in which your alternative offers
some advantages (after a slight modification): a user using gridftp
wants to retrieve the latest version of datasets distributed around
several sites (ignore replicated versions for simplicity). If the user
wants to write a script to retrieve the data without having to look up
what the latest version number is for each dataset, wouldn't it be best
to put the actual files in "latest" subdirectory, with pointers going
from the version identified directory to "latest"? All of these
complications seem to be due to gridftp's inability to follow links.
Are there other ways to enable users to do the above without putting the
actual files in a directory named "latest"?
regards,
Karl
On 9/17/11 1:16 PM, stephen.pascoe at stfc.ac.uk wrote:
>
> Hi All,
>
> I'm aware that many people are reluctant to use drslib because of it's
> use of symbolic links when constructing the DRS directory structure.
> I completely understand caution when using symbolic links but I wanted
> to make the case for why I believe it is necessary to meet the goals
> of a consistent distributed and versioned archive. Therefore I've
> prepared a wiki page that goes into the technical details:
>
> http://esgf.org/wiki/DrsVersionLinking
>
> Please read it carefully if you are currently considering how to
> implement the DRS directory structure. I propose we discuss this at
> the next ESGF telco.
>
> Thanks,
>
> Stephen.
>
> ---
>
> Stephen Pascoe +44 (0)1235 445980
>
> Centre of Environmental Data Archival
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
>
> --
> Scanned by iCritical.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110918/19970dec/attachment.html
More information about the GO-ESSP-TECH
mailing list