[Go-essp-tech] Bulk data moving and the UNIDATA LDM

[Bryan Lawrence]

My understanding is that it doesn't compare on a file by file basis: 
GridFTP is clearly optomised to move big files fast. However, if we are 
moving more than dozens of files (as we are), then as I understand it, 
LDM would open multiple file transfer streams, so GridFTP's advantage 
will boil down to the (not inconsiderable) negotiated window size.

Someone else ought to be able to give much better information than that 
:-)

Cheers
Bryan


On Friday 04 Jun 2010 15:58:33 Alex Sim wrote:
> Can you tell us  about the wide-area transfer performance with
>  Unidata LDM compared to GridFTP server based transfers?
> 
> 
> -- Alex
> 
> On 6/4/10 3:39 AM, martin.juckes at stfc.ac.uk wrote:
> 
> Hello all,
> 
> 
> 
> There may be a simple answer to this, but is there a reason why we
>  shouldn't use the Unidata Local Data Manager (LDM) for bulk data
>  movement within the CMIP5 distributed archive? It appears to have a
>  lot of good features built in to verify success of data transfer
>  between sites, and runs successfully at many operational sites. This
>  would simplify the work flow, since all the complexity of the site
>  to site transfers would be dealt with by a tried and tested system,
> 
> 
> 
> Cheers,
> 
> Martin
> 

-- 
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848; 
Web: home.badc.rl.ac.uk/lawrence