[Go-essp-tech] On use of wget over http

[martin.juckes at stfc.ac.uk]

Hello,

Yesterday I ran a few tests transferring a 2Gb file from CSIRO to a server at Reading in the UK using wget over http. I ran the wget command 4 times, and each time got a file of the correct size and incorrect checksum. Wget was using multiple automatic retries. I then throttled back the transfer rate to 400Kbytes/s and got the file transferred in one go, and with the correct checksum. It just took a little longer.

My tentative  conclusions are that users cannot access the data reliably if we do not provide checksums, and that download scripts which do not verify checksums are not good enough for an archive of this size,

Cheers,
Martin

-- 
Scanned by iCritical.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20111027/e503bfbb/attachment.html