[Go-essp-tech] Aggregations and LAS in THREDDS catalogs

[Estanislao Gonzalez]

Hi Stephen,

I've never tried it but the TDS offers apparently WMS services just as 
OpenDAP AFAICT (that info is in the web.xml)
Have you tried it?

Thanks,
Estani

Am 20.03.2012 12:47, schrieb stephen.pascoe at stfc.ac.uk:
>
> Hi Roland,
>
> Thanks, that's very helpful.  Separating aggregations into contiguous 
> portions is completely sensible.  I'll roll this info into the THREDDS 
> spec.
>
> Incidentally, Marten and I recently prototyped bringing a CMIP5 
> dataset into KNMI's WMS visualisation portal that we are using in our 
> IS-ENES project.  This picks up the OPeNDAP aggregation endpoints and 
> res-serves them as WMS.  We managed to get this working through the 
> ESGF security layer.
>
> The repercussions of this are:
>
>  1) down the road there will be visualisation options that don't rely 
> on explicit declarations in THREDDS ( the visualisation will read the 
> THREDDS XML and know what to do)
>
> 2) Marten needs to know how the aggregations are laid-out in THREDDS 
> and what the subsets are for.
>
> Cheers,
>
> Stephen.
>
> ---
>
> Stephen Pascoe  +44 (0)1235 445980
>
> Centre of Environmental Data Archival
>
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
> *From:*Roland Schweitzer [mailto:roland.schweitzer at noaa.gov]
> *Sent:* 19 March 2012 14:40
> *To:* Pascoe, Stephen (STFC,RAL,RALSP)
> *Cc:* go-essp-tech at ucar.edu; drach1 at llnl.gov; plieger at knmi.nl
> *Subject:* Re: Aggregations and LAS in THREDDS catalogs
>
> Stephan,
>
> Sorry for the delay.  I've been away from the office.  I'll do my best 
> here to fill you in.
>
> On Fri, Mar 9, 2012 at 3:14 AM, <stephen.pascoe at stfc.ac.uk 
> <mailto:stephen.pascoe at stfc.ac.uk>> wrote:
>
> Hi,
>
> I'd like to understand better why aggregations are described as they 
> are in ESGF THREDDS catalogs and what LAS expects from them.
>
> Modern datanodes have 2 levels of nested dataset elements describing 
> aggregations.  E.g. [1].  In some cases there are multiple leaf 
> datasets "subset 1" "subset 2" etc.  What's the reason for this?  Is 
> it to split the field into manageable chunks for TDS or LAS?
>
> I believe the reasoning is as follows.  Some data sets have large 
> gaps. For example, a run might write out data for a period of 100 
> years, skip 100 years and then write additional data for 100 years. 
>  This makes for challenging user interface issues in LAS since it 
> expects data to be more or less regular and contiguous.  So, I 
> suggested that the publisher create separate aggregations where there 
> are these know large gaps in the data. This will allow LAS to think of 
> these things as two separate data sets in the UI and thus guarantee 
> that the UI is presenting only time ranges for which data exists.
>
>
>     There are LAS access elements for the dataset at 3 levels of the
>     dataset hierarchy: top-level dataset, first-level aggregation and
>     "subset" aggregation.  Are all three of these used by LAS?
>
> In the end, no they are not.
>
> Originally LAS would only load the single variable when that URL of 
> the "subset" was selected and would load all the variables in the data 
> set when the top-level URL was selected.  I subsequently changed how 
> the LAS used in the ESGF organized the data sets internally, so that 
> the "top-level" url is the only one that matters and it loads all of 
> the variables in the data set into the interface.
>
> Luca and I were working on this recently and concluded that the 
> publisher should only insert the "top-level" LAS URLs.  We could in a 
> subsequent release create URLs at the "subset" level that would load 
> all of the variable into the interface and then select that variable 
> as the active variable.  The "first-level" URL seems less useful since 
> we can't determine which subset the user wants.  However, neither of 
> these last two options seems pressing or necessary since the search UI 
> does not expose the links to individual variables.
>
> Roland
>
>     Thanks,
>     Stephen.
>
>     [1] -----------
>
>     <dataset
>     name="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation"
>     ID="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation"
>     restrictAccess="esg-user">
>     <property name="aggregation_id"
>     value="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation"/>
>     <!-- ... -->
>     <access
>     urlPath="?catid=D9C519D5A310E197819B7197215FD574_ns_cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation"
>     serviceName="LASatPCMDI9" dataFormat="NetCDF"/>
>     <dataset
>     name="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation
>     - Subset 1"
>     ID="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation.1"
>     urlPath="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation.1"
>     restrictAccess="esg-user">
>     <serviceName>gridded</serviceName>
>     <property name="aggregation_id"
>     value="cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation.1"/>
>     <property name="time_delta" value="1 month"/>
>     <property name="calendar" value="365_day"/>
>     <property name="start" value="2090-1-1 0:0:0.0"/>
>     <property name="time_length" value="1680"/>
>     <access
>     urlPath="?catid=D9C519D5A310E197819B7197215FD574_ns_cmip5.output1.INM.inmcm4.1pctCO2.mon.landIce.LImon.r1i1p1.sbl.20110323.aggregation.1"
>     serviceName="LASatPCMDI9" dataFormat="NetCDF"/>
>     </dataset>
>     </dataset>--
>     Scanned by iCritical.
>
>
> -- 
> Scanned by iCritical.
>
>
>
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech


-- 
Estanislao Gonzalez

Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany

Phone:   +49 (40) 46 00 94-126
E-Mail:  gonzalez at dkrz.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120320/6d6924fe/attachment.html