[Go-essp-tech] [esg-node-dev] Re: CMIP5 Datanode : data access services.

Tue May 3 01:55:39 MDT 2011

Hi Sebastien,

The latest release 4.1.2 has the security extensions.  There were problems with the initial beta release (not related to security ;) ) but AFAIK these were straightened out with the beta2 and 4.1.2 official release.

Cheers,
Phil

From: Sébastien Denvil <sebastien.denvil at ipsl.jussieu.fr<mailto:sebastien.denvil at ipsl.jussieu.fr>>
Date: Mon, 2 May 2011 22:49:04 +0200
To: <go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>>, "esg-node-dev at lists.llnl.gov<mailto:esg-node-dev at lists.llnl.gov>" <esg-node-dev at lists.llnl.gov<mailto:esg-node-dev at lists.llnl.gov>>
Subject: [esg-node-dev] Re: [Go-essp-tech] CMIP5 Datanode : data access services.

Hi,

very useful link Roland. Reading it I was not able to figure out if the bug comes from the client or the server?

This wiki points netcdf-4.1.2-beta2. The 4.1.2 has been released: http://www.unidata.ucar.edu/software/netcdf/release-notes-4.1.2.html
Do you know if the released library contains the C API Security Extension for Earth System Grid?

On the data node side : our production data node is using thredds Version 4.2.20100908.2246. The 4.2.5 thredds version released 2011-03-02 includes opendap ESG access control : ftp://ftp.unidata.ucar.edu/pub/thredds/4.2/CHANGES.txt
Do you know if this is  a required version for the data node.

Gavin : what is the installer version you suggest us to use. v1.0.3-14-g485f969 or a later one?

Thanks.
Sébastien

On 02/05/2011 17:51, Roland Schweitzer wrote:
Hi,

Here are some detailed notes on client access.  The netCDF library target that contains the security infrastructure has still not been officially release and in our most recent testing still contains some bugs.

I'll update this page after we get our builds finalized with the netCDF library release.

http://esgf.org/wiki/ClientAccessToESGFOPeNDAPServers

(I had  to login into the Wiki to see it.  Maybe Gavin can link to this page and make it more public.  I don't know how to manage the access in this Wiki.)

Roland

On 05/02/2011 10:14 AM, Estanislao Gonzalez wrote:
Hi Sébastien,

there is a requirement about the NetCDF library version that work with ESGF. I don't recall it now (Phil, which version was it?), but it means that not every nco can connect to the data node. The OpenDAP connector must provide the required certificate and follow redirects for it to work.

Thanks,
Estani

Am 02.05.2011 16:09, schrieb Sébastien Denvil:
 Thanks Luca.

On 02/05/2011 15:38, Cinquini, Luca (3880) wrote:
Hi Sebastian,
    the ESGF security infrastructure can be used for opendap access - infact, this is already working at JPL. Once opendap
access is configured with security, users can use either a browser or an esg-enabled client to access datasets.

Great. At least with opendap they can access from the browser a region if they know array indices that cover it. If they have nco operator with opendap client enable they should in theory be able to extract their area using physical lat/lon. Not everyone has made this nco compilation.

The good thing with netCDF subsets is that it works with longitude and latitude from the browser.... Lot of browser addict out there.

I will try to write some documentation on the esgf.org wiki either this evening or tomorrow, and then let you try it out...

thanks for raising the issue, I meant to start an email conversation on this but never quite got to it...

That's already big fun to setup model / run them / produce & distribute those data. We tried hardly in the past and was discourage by the difficulty and the fact that we did not know how to find out precisely what was supposed to work and what was not (given the day we install the stack). In that case it's not easy to investigate and to find a way to contribute without being in the development main stream.

Glad to hear it should be smooth install now. We will read your notes and try it out with confidence.

Cheers.
Sébastien

ps : any further comments about gridFTP or netcdf subsets? If you say it should work we will try and report, but we need this green light to try.

Luca

On May 2, 2011, at 7:32 AM, Sébastien Denvil wrote:

  Dear all,

we received increasing demand asking for regional model boundary
conditions from CMIP5 simulations. Those datasets are heavy and
therefore "wget" is not my favourite answer. I would like this answer to
be base on a ESG-F solution and not a home made one.

In order to build a meaningful response I would like to know what are
the plan regarding the datanode to have :
- openDAP enable with CMIP5 filters.
- netCDF subsets enable with CMIP5 filters.
     * openDAP and/or netCDF subsets will be very useful for those
focusing on a specific region (Africa OR Asia for example). They could
then extract only the area of interest.

- gridFTP enable with CMIP5 filters.
     *gridFTP will be very useful for those focusing on many regions
and/or doing the CORDEX exercise but without a limited area model. It's
very likely that they will download the whole datasets.

My understanding is that gridFTP is not fully functional at this point
of time and that openDAP and netCDF subsets are not yet secure by the
ESG-F security layer.

A corollary coming from those questions : Is there a documentation we
could refer to if we want to update a production data node?

Please refine my understanding.

Thanks.
Sébastien

--
Sébastien Denvil
IPSL, Pôle de modélisation du climat
UPMC, Case 101, 4 place Jussieu,
75252 Paris Cedex 5

Tour 45-55 2ème étage Bureau 209
Tel: 33 1 44 27 21 10
Fax: 33 1 44 27 39 02

<smime.p7s>_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>
http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

--
Estanislao Gonzalez

Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany

Phone:   +49 (40) 46 00 94-126
E-Mail:  gonzalez at dkrz.de<mailto:gonzalez at dkrz.de>

_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

--
Sébastien Denvil
IPSL, Pôle de modélisation du climat
UPMC, Case 101, 4 place Jussieu,
75252 Paris Cedex 5

Tour 45-55 2ème étage Bureau 209
Tel: 33 1 44 27 21 10
Fax: 33 1 44 27 39 02

-- 
Scanned by iCritical.