[Go-essp-tech] On use of wget over http
martin.juckes at stfc.ac.uk
martin.juckes at stfc.ac.uk
Thu Oct 27 04:13:54 MDT 2011
Hello,
Yesterday I ran a few tests transferring a 2Gb file from CSIRO to a server at Reading in the UK using wget over http. I ran the wget command 4 times, and each time got a file of the correct size and incorrect checksum. Wget was using multiple automatic retries. I then throttled back the transfer rate to 400Kbytes/s and got the file transferred in one go, and with the correct checksum. It just took a little longer.
My tentative conclusions are that users cannot access the data reliably if we do not provide checksums, and that download scripts which do not verify checksums are not good enough for an archive of this size,
Cheers,
Martin
--
Scanned by iCritical.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20111027/e503bfbb/attachment.html
More information about the GO-ESSP-TECH
mailing list