[Go-essp-tech] wget script reports 302 Moved Temporarily ... for days! [SEC=UNCLASSIFIED]

Fri Feb 17 08:58:45 MST 2012

Hi Hans,
obviously you must have implemented the ESGF search API, beautiful!

The same query can be performed versus an ESGF node:

http://esg-datanode.jpl.nasa.gov/esg-search/search?experiment=amip&experiment=amipFuture&experiment=amip4xCO2&experiment=amip4K&time_frequency=mon&institute=IPSL

or using the "experiment_family" parameter:

http://esg-datanode.jpl.nasa.gov/esg-search/search?experiment_family=Atmos-only&time_frequency=mon&institute=IPSL&facets=experiment

where Atmos-only includes the following experiments:

<lst name="experiment">
<int name="amip">46</int>
<int name="amip4K">12</int>
<int name="amip4xCO2">9</int>
<int name="amipFuture">8</int>
<int name="aqua4K">10</int>
<int name="aqua4xCO2">4</int>
<int name="aquaControl">10</int>
<int name="sstClim">12</int>
<int name="sstClim4xCO2">12</int>
<int name="sstClimSulfate">12</int>
</lst>

The only difference I see is that you are using the parameter "frequency" while we are using "time_frequency" - this clearly indicates we need to do a better job on control vocabulary.

Anyway, great job!

thanks, Luca

On Feb 17, 2012, at 7:45 AM, Hans Ramthun wrote:

Hallo Sébastien,

In the IS-ENES verc-portal there exists a rest webservice which is able to give you all the actual url of downloadable data like:

https://verc.enes.org/myapp/cmip5/ws/rest/search?experiment=amip&experiment=amipFuture&experiment=amip4xCO2&experiment=amip4K&frequency=mon&institute=IPSL

This example searches for all amipXX experiments and monthly data and institute IPSL inside the cmip5 data information harvested from all cmip5 thredds server.
As search criteria you can use all facets which are in the drs.

Maybe this helps to discover which data versions are available.

Cheers
Hans

Am 16.02.2012 22:01, schrieb Sébastien Denvil:
Hi,

I would say we have to check with NCAR people. They produced a version v20120203 just after the v20120202.

We have been able to download the v20120202, but never the v20120203 ; just like you. It seems to me like a mistake. The v20120202 is still available.

Anyone on the list could check directly with them this version story?

Regards.
Sébastien

Le 16/02/2012 21:44, Lawson Hanson a écrit :
Hi,

Can anyone explain why an "wget-download.sh" script
     obtained from the ESG should produce the following
     errors, one like this for each file listed in the script?

   --2012-02-17 06:35:35--  http://tds.ucar.edu/thredds/fileServer/datazone/cmip5_data/cmip5/output1/NCAR/CCSM4/rcp85/mon/atmos/Amon/r1i1p1/v20120203/hfss/hfss_Amon_CCSM4_rcp85_r1i1p1_200601-210012.nc
   Resolving tds.ucar.edu<http://tds.ucar.edu>... 128.117.12.72
   Connecting to tds.ucar.edu|128.117.12.72|:80... connected.
   HTTP request sent, awaiting response... 302 Moved Temporarily
   Location: https://tds.ucar.edu/OpenidRelyingParty/home.htm?redirect=http%3A%2F%2Ftds.ucar.edu%2Fthredds%2FfileServer%2Fdatazone%2Fcmip5_data%2Fcmip5%2Foutput1%2FNCAR%2FCCSM4%2Frcp85%2Fmon%2Fatmos%2FAmon%2Fr1i1p1%2Fv20120203%2Fhfss%2Fhfss_Amon_CCSM4_rcp85_r1i1p1_200601-210012.nc [following]
   --2012-02-17 06:35:36--  https://tds.ucar.edu/OpenidRelyingParty/home.htm?redirect=http%3A%2F%2Ftds.ucar.edu%2Fthredds%2FfileServer%2Fdatazone%2Fcmip5_data%2Fcmip5%2Foutput1%2FNCAR%2FCCSM4%2Frcp85%2Fmon%2Fatmos%2FAmon%2Fr1i1p1%2Fv20120203%2Fhfss%2Fhfss_Amon_CCSM4_rcp85_r1i1p1_200601-210012.nc
   Connecting to tds.ucar.edu|128.117.12.72|:443... connected.
   HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://tds.ucar.edu/thredds/fileServer/datazone/cmip5_data/cmip5/output1/NCAR/CCSM4/rcp85/mon/atmos/Amon/r1i1p1/v20120203/hfss/hfss_Amon_CCSM4_rcp85_r1i1p1_200601-210012.nc [following]
   --2012-02-17 06:35:38--  http://tds.ucar.edu/thredds/fileServer/datazone/cmip5_data/cmip5/output1/NCAR/CCSM4/rcp85/mon/atmos/Amon/r1i1p1/v20120203/hfss/hfss_Amon_CCSM4_rcp85_r1i1p1_200601-210012.nc
   Connecting to tds.ucar.edu|128.117.12.72|:80... connected.
   HTTP request sent, awaiting response... 403 Forbidden
   2012-02-17 06:35:45 ERROR 403: Forbidden.

What does the "302 Moved Temporarily" imply?
     And if the file has been moved, then why isn't there
     a link to it so that the "wget-download.sh" script
     just works?

I have tried several times over the past couple of days
     (thinking that perhaps there was a server fault, etc.)
     to run this "wget-download.sh" script, but each time
     with the same result.

Your resolution for this problem will be most appreciated by
     a number of CAWCR/BoM/CSIRO scientists in Australia,
     because they all want to include the "CCSM4" model
     in their analyses.

Best regards,

Lawson Hanson
-------------
     CAWCR, Climate Variability and Change group,
     Climate Change Science team,
     Bureau of Meteorology, 700 Collins Street,
     Melbourne Docklands, VIC 3008
     Telephone: +61 3 9669 4763

_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>
http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>
http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

--
Hans Ramthun
Tel.: +49 (0)40 460 094 - 112
Deutsches Klimarechenzentrum - DKRZ
Abteilung Datenmanagement     http://www.dkrz.de/
Bundesstr. 45a
D-20146 Hamburg              Germany

„Only he who knows his destination finds the way.“ (Laozi)

_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>
http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120217/68befe7c/attachment.html 