[Go-essp-tech] esg publisher, thredds and netcdf attributes

[bryan.lawrence at stfc.ac.uk]

Hi Bob

Thanks for this.

Unfortunately outlook web access gobbled up my first attempt to reply, so this is brief (maybe a good thing :-)

Anyway,  I think provenance is pretty important, knowing "who did what" will matter. But as you imply, it'll be complicated, especially with services. We'll parse the xsd and think some more about it.

Cheers
Bryan

-----Original Message-----
From: Bob Drach [mailto:drach at llnl.gov]
Sent: Wed 09/06/2010 19:12
To: Lawrence, Bryan (STFC,RAL,SSTD)
Cc: go-essp-tech at ucar.edu
Subject: Re: esg publisher, thredds and netcdf attributes
 
Hi Bryan,

As you point out the dataset properties don't carry along information  
about their origin, although I imagine they could if necessary. In the  
context of CMIP5 we could probably stipulate in advance which is  
which, but more generally the source of a property could be from a  
data file / database, directory structure, interactively from the data  
curator / publisher, or from a data handler. For example, if a CMIP5  
datafile was not generated through CMOR, and did not have the required  
properties defined, it could still be published by specifying those  
properties as command line arguments.

If this is an important issue it might be possible to group property  
elements within 'metadata' tags, based on their origin. See

http://www.unidata.ucar.edu/projects/THREDDS/tech/catalog/v1.0.2/InvCatalogSpec.html#metadataElement

for the gory schema details. Also be aware that in THREDDS there is a  
distinction between the configuration catalog - which the publisher  
generates - and the catalog as served by the TDS. For property  
elements they should be the same, but for some elements they will  
differ.

Best regards,

Bob

On Jun 9, 2010, at 2:13 AM, Bryan Lawrence wrote:

> Hi Bob
>
> As you know, we're in the process of building a tool to parse the
> thredds catalogs produced by esg-publisher.
>
> We're sitting in a meeting looking at the Thredds XML.
>
> At the moment there are dataset properties which reflect attributes  
> from
> the files, and some which come from esg-publisher. From the point of  
> view
> of automatically consuming these catalogs, we can't tell the  
> difference
> between these attributes.
>
> For example, the checksum information we presume comes from esg-
> publisher, and the tracking-id from internal attributes file.
>
> Is it possible in Thredds to distinguish between these things? It's
> pretty important for unpicking the origin of the information which
> appears in the catalog.
>
> Thanks
> Bryan
>
> -- 
> Bryan Lawrence
> Director of Environmental Archival and Associated Research
> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
> STFC, Rutherford Appleton Laboratory
> Phone +44 1235 445012; Fax ... 5848;
> Web: home.badc.rl.ac.uk/lawrence


-- 
Scanned by iCritical.