[Go-essp-tech] DRS corrections and extensions [SEC=UNCLASSIFIED]
Karl Taylor
taylor13 at llnl.gov
Wed Jun 6 18:32:11 MDT 2012
Dear Lawson,
The problem is that the gridspec filenames have already been written
into perhaps 100's of thousands of CMIP5 files, since it should appear
in the "associated_files" global attribute. Below I've copied the
output requirement defining this attribute.
I understand that searching for all "fx" fields by parsing the file
names will miss the gridspec files (unless you build in a bit more
logic), but why not simply do a couple more searches to find all the
files starting with "gridspec" to supplement what you found looking for
"fx"? I'm sure you've already thought of other work arounds that are
probably better than this.
By the way, although these hooks are in place, they don't yet lead you
to the gridspec files. We haven't gotten to that yet.
best regards,
Karl
? associated_files = a string listing the base URL for CMIP5 and the
location of the model’s gridspec file, followed, as appropriate, by the
name of the file containing the grid cell areas and/or grid cell
volumes.For CMIP5 this string
is:"baseURL:http://cmip-pcmdi.llnl.gov/CMIP5/dataLocationgridspecFile:<gridspec
file name> [areacella:<atmos. cell area file name>] [areacello:<ocean
cell area file name] [volcello:<ocean cell volume file name>]”, where
cell area and cell volume are only sometimes required.Here is an example:
? associated_files=
? “baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation
? gridspecFile: gridspec_ocean_fx_IPSL-CM5_historical_r0i0p0.nc
? areacello: areacello_fx_IPSL-CM5_historical_r0i0p0.nc
? volcello: volcello_fx_IPSL-CM5_historical_r0i0p0.nc”
? Note that the “cell_measures” column (U) of the CMIP5 Requested Output
<http://pcmdi-cmip.llnl.gov/cmip5/docs/standard_output.pdf> tables
indicates whether or not areacella, areacello, and/or volcello should be
included as associated_files.For each model version and experiment, only
one area field is requested by CMIP5 for the atmosphere and only one
area and one volume field are requested for the ocean.These cell areas
should be defined such that exact global integrals of energy fluxes at
the surface and “top of the atmosphere” can be computed.It is understood
that for some staggered grids, the meshes for horizontal velocities
might be offset from the radiation points, so in these cases exact
global integrals of momentum would require areas not requested by
CMIP5.In the associated_files attribute, include references to
areacella, areacello, and volcello only for variables carried on the
same mesh as the areas and volumes (i.e., only when it is appropriate to
do so).
On 6/6/12 2:46 PM, Lawson Hanson wrote:
>
> Dear Karl,
>
> In section 3.3 CMIP filename encoding, (page 10 of your DRS
> "v1-3-1-marked" document)
>
> there is mention of the single exception for "gridspec" files:
>
> In CMIP5 there is a single exception to use of the above template.
>
> For so-called gridspec files, which describe the grids used in a
> model,
>
> the filename should be constructed as follows:
>
> gridspec filename = gridspec_<modeling
> realm>_fx_<model>_<experiment>_r0i0p0.nc
>
> where <modeling realm> is now included and the variable name
>
> is replaced by “grid_spec”.
>
> Note also that this is a time-independent field, so the CMIP5 table
>
> is “fx” and the ensemble member is set to “r0i0p0”.
>
> Example:
>
> • gridspec_atmos_fx_IPSL-CM5_historical_r0i0p0.nc
>
> My Question:
>
> Is there any way that this could be re-defined so that the
> <modeling realm>
>
> element is moved across to be after the <ensemble member> element?
>
> This just seems to make more sense to me, and keeps more of the
>
> file name template (from left to right) consistent with the 99.99%
>
> of other CMIP5 file names. If this was adopted, the new template
>
> for a gridspec file would become:
>
> gridspec filename =
> gridspec_fx_<model>_<experiment>_r0i0p0_<modeling realm>.nc
>
> and the example would be:
>
> • gridspec_fx_IPSL-CM5_historical_r0i0p0_atmos.nc
>
> The combination of the unique 'r0i0p0' (<ensemble member>) element,
>
> followed by the <modeling realm> element in the filename template
>
> makes it far easier to parse a list of thousands of file names
>
> to find all of those where the second field (<mip table>) is 'fx',
>
> or 'Amon', or 'OImon', or 'day', etc.
>
> Thank you for your consideration of this request.
>
> Best regards,
>
> Lawson Hanson
>
> -------------
>
> CAWCR, Climate Variability and Change group,
>
> Climate Change Science team,
>
> Bureau of Meteorology, 700 Collins Street,
>
> Melbourne Docklands, VIC 3008, Australia
>
> *From:*go-essp-tech-bounces at ucar.edu
> [mailto:go-essp-tech-bounces at ucar.edu] *On Behalf Of *Karl Taylor
> *Sent:* Thursday, 7 June 2012 6:58 AM
> *To:* Kettleborough, Jamie; V. Balaji; Steve Hankin; Martin Juckes;
> Bryan Lawrence; Stephen Pascoe; go-essp-tech at ucar.edu
> *Subject:* Re: [Go-essp-tech] DRS corrections and extensions
>
> Dear all,
>
> In February I asked for comments on my proposal to extend the DRS to
> include information about spatio-temporal subsets or means. I heard
> from Jamie, but no one else. I respond to Jamie below, but I also
> would like your input specifically about:
>
> 1. Is this method of describing spatio-temporal subsets acceptable?
> 2. Is it worth taking this step if we don't say anything about other
> "processed" output? For example how to describe "regridded" data or
> multi-model means.
>
> I've attached the proposed version of the DRS, which differs from the
> one I sent in January only in a couple mods made in response to Jamie.
>
> Best regards,
> Karl
>
> On 2/13/12 6:47 AM, Kettleborough, Jamie wrote:
>
> Hello Karl,
>
> this will be terse as I have time to review, but not to necessarily
> get the words right - hope I don't say anything too bad because of this.
>
> 1. section 2.3, Not sure 'output' should be mentioned under
> 'product'. I don't think 'output' ever makes it to publication level,
> so does not need to appear in a publication level id. I know cmor
> produces it, but I think that's kind of historical isn't it, rather
> than necessary? Maybe its too late for details like this?
>
> It's true that in the end the CMIP5 output should not remain as
> "output", but be assigned to "output1" or "output2". Nevertheless, I
> don't think there is any harm in keeping it in the DRS.
>
> 2. section 2.3 version number: to be consistent with what we really
> have in CMIP5 I think you need to note that v1, v2 are also present,
> though any *new* versions should use vYYYYMMDD.
>
> I have modified the text to indicate that software cannot rely on the
> version number reflecting a date.
>
> 3. section 2.3 version: I wonder if you need to say more (maybe not
> here, but if not where?) about what triggers a new version. I think its
>
> a. anything that changes the content of a file already published and
>
> b. the addition or deletion of files from any publication data set.
>
> Pure 'data management' meta data changes (addition of checksums,
> move to new URL's) need not trigger a new version.
>
> Do you also need to say there is no guarantee that old versions will
> be kept (unless they have a DOI).
>
> I've added some of this information now to the document.
>
> 4. section 2.4 Temporal Subsets or means: I don't understand the 'avg'
> example, or if I do I don't know if its right (but the point is
> relatively minor). I think the example you quote as one 6 month mean
> field in it. This is based on 1 day means. I think its a little
> anomalous to keep the frequency as 'day' in this case. That's not
> quite consistent with the definition (and I think all other uses) of
> frequency. Strictly speaking frequency should be 6mon no? (I may
> have misunderstood).
>
> I think you're right. I'm not sure why I thought this was the right
> way to do it. I've changed the example,
>
> 5. section 3.5. Does this need clarifying? I think the current
> wording is potentially confusing, I think it should say something like:
>
> 'URLs referencing the data files will have a site dependent prefix
> (that may change due to site-specific data management tasks) followed
> by the directory structure. This directory structure should (but may
> not) follow the recommendations of section 3.3'
>
> I've modified the text as suggested.
>
> 6. I've noticed that the thredds catalogs also expose a thing called
> the file_id, e.g
>
> <property name="file_id"
> value="cmip5.output1.CNRM-CERFACS.CNRM-CM5.rcp45.mon.ocean.Omon.r1i1p1.vo_Omon_CNRM-CM5_rcp45_r1i1p1_203601-204512.nc"/>
>
> I don't know if they need a mention as being anything important (we
> don't use them as they don't give any version info).
>
> We've already given 5 use cases, which I think is enough. The DRS is
> used in a number of other ways.
>
> Hope this is useful,
>
> Yes thanks very much!
> Karl
>
> Jamie
>
> ------------------------------------------------------------------------
>
> *From:*go-essp-tech-bounces at ucar.edu
> <mailto:go-essp-tech-bounces at ucar.edu>
> [mailto:go-essp-tech-bounces at ucar.edu] *On Behalf Of *Karl Taylor
> *Sent:* 10 February 2012 01:32
> *To:* V. Balaji; Steve Hankin; Martin Juckes; Bryan Lawrence;
> Stephen Pascoe; go-essp-tech at ucar.edu <mailto:go-essp-tech at ucar.edu>
> *Subject:* [Go-essp-tech] DRS corrections and extensions
>
> Dear all,
>
> Attached is my attempt to make the DRS consistent with CMIP5 (in
> describing the precision of "time instants"), but primarily to
> extend it to a more complete treatment of spatio-temporal subsets
> or means. I've also corrected a few typos.
>
> Comments most welcome. In particular could someone recheck
> sections 3.3-3.5 (which haven't been changed by me) to see if they
> remain consistent with CMIP5?
>
> thanks and best regards,
> Karl
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120606/4e29c186/attachment-0001.html
More information about the GO-ESSP-TECH
mailing list