[Go-essp-tech] [is-enes-sa2-jra4] Implementing the DRS on a disk farm
Bryan Lawrence
bryan.lawrence at stfc.ac.uk
Wed Jul 7 08:30:48 MDT 2010
Hi Folks
> I agree that we need to clarify the definition of replicated section
> -- but we can discuss this under the other thread (though I will say
> that it is absolutely certain that it won't be done at the file
> granularity).
:-) It's all about the definition of "it" ...
Clearly we'll be replicating a subset of the files, but we ought not be
subsetting into files, for the replication.
The definition of what would be replicated will effectively be a DRS
string, and I guess what we all need now is a list of those strings.
(In the case of replicated datasets which are temporal subsets I believe
we ought to be using the temporal modifier in the DRS to indicate what
should be replicated).
(I suspect a really useful tool for folks will be something that given a
DRS string, and a temporal modifier, can parse the files found in the
directory, and output a list of the files and optionally move it
elsewhere ... would feed very nicely into the replication use case, and
obviate the necessity for having two copies of the data locally anywhere
except where we are handling "original" data and "replicated" data
together ... and even there you could use it to set up the links if you
didn't want two copies of the data.)
Cheers
Bryan
--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848;
Web: home.badc.rl.ac.uk/lawrence
More information about the GO-ESSP-TECH
mailing list