[Go-essp-tech] DRS structure

[Martina Stockhause]

Dear all,

we really need to fix the DRS structure and the reflectance of the DRS
syntax in the TDS catalogue.

During the QC, which runs in the file system with DRS syntax, I need to
have a connection to the TDS to check the consistency of data against
metadata after the automated checks. Since I don't want to touch each
dataset again, I take the TDS metadata as reference for data content.

Up to now it was possible to take the dataset_id as DRS name out of the
TDS in the atomic dataset (TDS aggregation = QC result level) and the
netcdf file level.

Now the <variable> part of the DRS is missing in the dataset_id of the
netcdf file, so that I am about to take the urlPath instead.
Is that ok?

Why can't we use the DRS syntax as IDs in the TDS and in metafor? That
would make things much easier.

The DRS syntax is my connection from the QC checked files to the TDS and
to metafor. Therefore the DRS syntax should be fixed soon and documented
in the DRS document. So, that we can start to adapt our examples and
scripts.

Best wishes,
Martina


Estanislao Gonzalez wrote:
> Hi all,
>
> I've realized we've been moving things from one place to another 
> regarding the DRS components, and the DRS Reference Syntax document 
> (from 7/4/2010) does not reflect this changes.
>
> There are two major difference here:
> 1) versioning: the drslib tool is creating a structure which is, for the 
> time being, not drs conform. I totally agree with the new 
> version-component placement, but should that not be reflected in the DRS 
> syntax document?
> 2) in CMIP5 Best Practices for Data Publication stays that the 
> dataset_id should be: 
> cmip5.<product>.<institute>.<model>.<experiment>.<time_frequency>.<realm>.<ensemble>
> I know the dataset_id is not required to necessary match any drs 
> structure. But I personally think we should avoid drs-similar 
> identifiers, as IMHO it increases confusion.
> I think this solution helps solving some publishing problems, but 
> defines a new dataset level, the "ensemble dataset". And the 
> realm-dataset is not being used anywhere else (or am I missing something?)
>
> I'm not aware of the reasons behind the definition of the DRS structure 
> as it currently is. But I think, we should avoid drifting away from that 
> document. In any case the document should be updated first.
>
> If I try to join all changes and proposals I've heard of, AFAIC the DRS 
> structure we are going to appears to look something like:
> cmip5.<product>.<institute>.<model>.<experiment>.<time_frequency>.<realm>.<ensemble>.<version>.<variable>
>
> Which is different from the original:
> cmip5.<product>.<institute>.<model>.<experiment>.<time_frequency>.<realm>.<variable>.<ensemble>.<version>
>
> Can anyone with more knowledge on the subject comment on this?
>
> Thanks,
> Estani
>
>   

-- 
----------- DKRZ / Data Management -----------

Martina Stockhause
Deutsches Klimarechenzentrum
Bundesstr. 45a
D-20146 Hamburg
Germany

phone:	+49-40-460094-122
FAX:	+49-40-460094-106
e-mail:	martina.stockhause at zmaw.de

----------------------------------------------