[Go-essp-tech] [is-enes-sa2-jra4] Implementing the DRS on a disk farm

[Bryan Lawrence]

Hi Folks

> I agree that we need to clarify the definition of replicated section
>  -- but we can discuss this under the other thread (though I will say
>  that it is absolutely certain that it won't be done at the file
>  granularity).

:-) It's all about the definition of "it" ...

Clearly we'll be replicating a subset of the files, but we ought not be 
subsetting into files, for the replication.

The definition of what would be replicated will effectively be a DRS 
string, and I guess what we all need now is a list of those strings.
(In the case of replicated datasets which are temporal subsets I believe 
we ought to be using the temporal modifier in the DRS to indicate what 
should be replicated).

(I suspect a really useful tool for folks will be something that given a 
DRS string, and a temporal modifier, can parse the files found in the 
directory, and output a list of the files and optionally move it 
elsewhere ... would feed very nicely into the replication use case, and 
obviate the necessity for having two copies of the data locally anywhere 
except where we are handling "original" data and "replicated" data 
together ... and even there you could use it to set up the links if you 
didn't want two copies of the data.)

Cheers
Bryan
-- 
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848; 
Web: home.badc.rl.ac.uk/lawrence