[Go-essp-tech] [metafor] the CMIP5 file uri structure

Tue Feb 17 03:53:49 MST 2009

Hello,

If I've understood, the problem is that CMIP5 will have models running the same scenario, configuration etc, but different start times. I'd suggest adding a controlled vocabulary component of the form "syyyymm[dd]", with the day "dd" optional (or perhaps "frtyyyymm[dd]", for forecast_reference_time, if we want to hint at the CF equivalent -- but I think "s" for start will be clearer to most users).

There appears to be some confusion between the definition of at atomic dataset and the URL. The intention was that the atomic dataset would refer to all the data delivered matching the institute, model, scenario etc of the definition. It is not a URL and there is no assumption about temporal continuity in the definition. The rational for having an atomic dataset definition which is distinct from the URL definition is:

  (1) We don't know what time periods modelling groups will provide data for (some may only provide the core request, others are expecting to provide more);
  (2) URLs may want to point to subsets of the data provided if, for example, the entire data volume in an atomic dataset is too large for convenient transfer as a single file.
  (3) It will me useful from an archive management perspective, to have some way of refering to collections of data which is not dependent of dataset specific information.

If defining a URL syntax for user access is now (and likely to remain) the main requirement of the document, then it should probably be rewritten with this focus,

cheers,
Martin

PS. Steve: I don't think I can post to go-essp-tech: can you forward it?
-----Original Message-----
From: go-essp-tech-bounces at ucar.edu on behalf of Bryan Lawrence
Sent: Tue 17/02/2009 08:10
To: Francisco J. Doblas-Reyes
Cc: Juckes, MN (Martin); go-essp-tech at ucar.edu; Reinhard Budich
Subject: Re: [Go-essp-tech] [metafor]  the CMIP5 file uri structure

Hi Paco

Thanks for this.

I think the two times (time instant and period) do correspond
with CF as you suggest.

You've got me worried about a couple of things on the time front though,
I had thought the decadal runs would essentially be different experiments
but you're right, the delineation might not be so clean. We'll make sure
we explicity put an example or two in the doc for the decadal runs so that we're sure
we have it right.

Cheers
Bryan

On Tuesday 17 February 2009 07:33:18 Francisco J. Doblas-Reyes wrote:
> Dear Bryan,
> 
> After having a look at the message below that you sent to the Metafor 
> community, I was wondering whether the DRS should apply to both types of 
> CMIP5 simulations: climate-change projections and decadal predictions. 
> If it applies, are the components "time instant" and "period" enough to 
> describe decadal forecasts that overlap in time? Are they equivalent to 
> the CF coordinates "forecast_reference_time" and "forecast_period"? For 
> the decadal forecasts in ENSEMBLES, we needed two axes to describe the 
> decadal forecasts in NetCDF. In GRIB we didn't have a problem because 
> ECMWF extensions already have to axes.
> 
> Regards,
> Paco
> 
> 
> Bryan Lawrence wrote:
> > hi Folks
> > 
> > See attached email which has the current state of play, and the attached doc.
> > It's not obvious to me who makes the decision that this is *THE* CMIP structure.
> > It yet needs linking to the proposed netcdf attributes.
> > 
> > Cheers,
> > Bryan
> > 
> > 
> > ------------------------------------------------------------------------
> > 
> > Subject:
> > Re: uniform URL syntax to support CMIP5/AR5
> > From:
> > Bryan Lawrence <bryan.lawrence at stfc.ac.uk>
> > Date:
> > Tue, 10 Feb 2009 15:28:27 +0000
> > To:
> > Steve Hankin <Steven.C.Hankin at noaa.gov>
> > 
> > To:
> > Steve Hankin <Steven.C.Hankin at noaa.gov>
> > CC:
> > Bryan Lawrence <b.n.lawrence at rl.ac.uk>, Michael Lautenschlager 
> > <lautenschlager at dkrz.de>, "V. Balaji" <V.Balaji at noaa.gov>, Don Middleton 
> > <don at ucar.edu>, Cinquini Luca <luca at ucar.edu>, "Dean N. Williams" 
> > <williams13 at llnl.gov>, "Karl E.Taylor" <taylor13 at llnl.gov>, 
> > charlotte.pascoe at rl.ac.uk
> > 
> > 
> > Hi Guys
> > 
> > I've made a couple of minor changes and comments.
> > 
> > 1): this is primarily about CMIP5 ... not AR5 ... so the nomenclature should reflect that. (I've fixed that throughout I think).
> > 
> > 2): Karl has already been working on some aspects of internal netcdf file vocabs, model short names etc, which should
> > match these ...  we should make sure these are all congruent with the file convention. Someone needs to take responsibility for this. I suggest that Charlotte do that ... (which consists mainly of downloading Karl's input into this doc).
> > 
> > 3): I think the hostname part of the path should be out of scope. We would expect multiple copies of atomic
> > datsets to be held within the federation. 
> > 
> > 4) The definition of an atomic dataset doesn't allow for the different periods within the same run being different atomic datasets (although the syntax does). I didn't fix that.
> > 
> > Cheers
> > Bryan
> > 
> > 
> > 
> > On Friday 06 February 2009 16:41:51 Steve Hankin wrote:
> >> Gents,
> >>
> >> In October 2007, when we met at GFDL for AR5/CMIP5 data management 
> >> planning, we agreed on the need for a shared URL syntax.  The idea was 
> >> that the URL would uniformly reference data holdings, regardless of 
> >> where they were located.  A draft of that document has been circulated 
> >> several times -- most recently at the GO-ESSP meeting in Seattle this 
> >> past September.  Based upon recent discussions Martin and I have  
> >> expanded the document (just a little) to encompass the encoding of time 
> >> periods as part of the syntax.  I have attached what I hope is the 
> >> semi-final (?) version of that document (v0.10) here.
> >>
> >> I suspect we need to take two steps to proceed from this point:
> >>
> >>    1. To suggest final changes (if any) and then to mutually endorse (if
> >>       still agreed) the use of this URL as a means to reference data
> >>       through *whatever* access mechanisms (OPeNDAP, WCS, FTP, ...)
> >>    2. To make implementation plans for specific systems (and for DNS
> >>       mapping as we see appropriate)
> >>
> >> Open for discussion ...
> >>
> >>     - Steve
> >>
> > 
> > 
> > 
> > 
> > ------------------------------------------------------------------------
> > 
> > _______________________________________________
> > metafor mailing list
> > metafor at lists.enes.org
> > https://lists.enes.org/mailman/listinfo/metafor
> 

-- 
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848; 
Web: home.badc.rl.ac.uk/lawrence
_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu
http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

-- 
Scanned by iCritical.