[Go-essp-tech] CSIRO Mk3.6 historicalMisc experiments renamed

Tue Apr 17 21:37:27 MDT 2012

Hi,

The CSIRO Mk3.6 CMIP5 datasets included five additional historicalMisc experiments which were named historicalAnt, historicalNoOz, historicalNoAA, historicalAA, historicalAntNoAA. We have since renamed those experiments to historicalMisc for consistency with other groups. The historicalXXX experiments have been renamed as follows:

historicalMisc r#i1p1 was previously historicalAnt     (anthropogenic-only)
historicalMisc r#i1p2 was previously historicalNoOz    (all forcings except ozone)
historicalMisc r#i1p3 was previously historicalNoAA    (all forcings except anthropogenic aerosols)
historicalMisc r#i1p4 was previously historicalAA      (anthropogenic aerosols only)
historicalMisc r#i1p5 was previously historicalAntNoAA (all forcings except anthropogenic aerosols outside the Asian region)

The data within the files have not been changed, but the metadata has changed. Our ESG is currently being rescanned to publish the data under the new names - this usually takes several days.

cheers

Stephen Jeffrey 

Queensland Climate Change Centre of Excellence 

Department of Environment and Heritage Protection

GPO Box 2454

Brisbane  Queensland  4001

Telephone +61 (0)7 3170 5519  Fax +61 (0)7 3170 5802

Email:  stephen.jeffrey at qld.gov.au

www.climatechange.qld.gov.au

			-------- Original Message -------- 
Subject: 	Re: [Go-essp-tech] Forcings on Historical	
Date: 	Sat, 31 Mar 2012 10:09:47 -0700	
From: 	Karl Taylor <taylor13 at llnl.gov> <mailto:taylor13 at llnl.gov> 	
To: 	Muhammad Atif <muhammad.atif at anu.edu.au> <mailto:muhammad.atif at anu.edu.au> 	
CC: 	Bryan Lawrence <bryan.lawrence at ncas.ac.uk> <mailto:bryan.lawrence at ncas.ac.uk> , "go-essp-tech at ucar.edu" <mailto:go-essp-tech at ucar.edu>  <go-essp-tech at ucar.edu> <mailto:go-essp-tech at ucar.edu> , Ben Evans <Ben.Evans at anu.edu.au> <mailto:Ben.Evans at anu.edu.au> 	

			Dear Muhammad,

			Thanks very much for volunteering to help clean this up.  As Bryan says, this will be a huge help to users and will save countless hours of effort for folks involved with ESGF development and those processing data.   There will be benefits for years to come.

			I hope others copied will read this and make sure I haven't forgotten anything.

			There are various degrees of conformance you might attempt, but I think renaming the files and republishing are both essential.  In the filenames, please

			1. Replace "historicalXXXX" with "historicalMisc"  (for the expts. listed in a previous email.)
			2. Replace "p1" with "pX", where the integer X should take on a different value for each of the different forcing runs (see 3 below for more info.)

			You should send Gavin Schmidt a table indicating which "p" value corresponds to which forcing, as was requested in an email about a week ago.  (If you didn't get that email, please let me know.)

			It would be desirable (and perhaps essential -- others might chip in here) also to correct the global attributes in the netCDF files themselves:

			1. Set experiment_id = 'historicalMisc'
			2. Set experiment = 'other historical forcing'
			3. Replace physics_version with "p" value consistent with the filename.
			physics_version = an integer (≥1) referring to the physics version used by the model If there is only one physics version of the model, then this argument should be normally given the value 1. Note that model versions that are substantially different should be given a different "model_id"; assigning a different "physics_version" should be reserved for closely-related model versions (e.g., as in a "perturbed physics" ensemble) or for the same model, but with different forcing or feedbacks active. In CMIP5, one would distinguish, for example, among runs forced by different combinations of "forcing" agents (as called for under the "historicalMisc" experiment - experiment 7.3) by assigning different values to physics_version. For fields appearing in table "fx" in the CMIP5 Requested Output, set physics_version=0 (violating the general rule that it should be a positive definite integer). Note that the physics_version is used in constructing the "ensemble member" called for by the DRS document; it is the value of L in r<N>i<M>p<L>.
			4. Generate a new tracking_id which is "a string that is almost certainly unique to this file and must be generated using the OSSP utility which supports a number of different DCE 1.1 variant UUID options. For CMIP5 version 4 (random number based) is required. Download the software from http://www.ossp.org/pkg/lib/uuid/. The tracking_id might look something like: 02d9e6d5-9467-382e-8f9b-9300a64ac3cd."
			5. Also check that the "forcing" attribute has the correct forcing identifier(s) (see http://cmip-pcmdi.llnl.gov/cmip5/docs/cmip5_data_reference_Appendix1-2.doc )

			I think this could be fairly easily scripted, but what sounds easy often is not.  If you are unfamiliar with netCDF Operators (NCO), they can be used to rewrite attributes in netCDF files.

			After correcting the files and filenames, I think you will have to republish.  I hope others will provide guidance on this.

			Please include anyone you think is interested in these emails.

			Thanks again for helping clean things up.  It is much appreciated.  
			And don't hesitate to write with any questions.

			Best regards,
			Karl

			On 3/30/12 10:58 PM, Muhammad Atif wrote: 

				Just to update; I had an email conversation with Estani when he first realised of the problem. We are prepared to do whatever might be required at the publishing end. 
				He suggested to wait for a nice solution.

				btw, should I bring the modellers (CSIRO-QCCCE team) into the email loop as well?

				Regards

				On 31/03/2012, at 4:46 PM, Bryan Lawrence wrote:

					I think it'd be better to fix this at source, otherwise these problems will lurk for ever not only in ESGF, but on the hard drives of everyone who downloads the data - causing a potential problem for the actual science because everyone will have to have their own hacks to deal with it.

					I realise this may be a bit unfair on folks who wrote their data in advance, but the flip-side is that not doing so is unfair on everyone who uses that data ...

					Cheers
					Bryan

					Hi Estani,

					Yes, the official name for all of these is historicalMisc, and the 
					different runs should have been identified by a different "p" value in 
					the "rip", which distinguishes among members of an ensemble.  The DRS 
					document describes this in more detail.

					I *think* these datasets may have been generated prior to the DRS 
					controlled vocabulary being fully settled, so I haven't raised a stink 
					about this.  I note that they show up in the old gateway1.3.4 search, 
					but not in the new p2p search.

					Is there some way that the non-conforming experiment names could be 
					changed, so that users can find these experiments under historicalMisc?

					Best regards,
					Karl

					On 3/30/12 3:19 AM, Estanislao Gonzalez wrote:

					Hi,

					I'm trying to replicate data from CSIRO-QCCCE and I see the following
					experiments names:
					historicalNoOz
					historicalNoAA
					historicalAntNoAA
					historicalAnt
					historicalAA

					These are used as the dataset id, e.g.
					cmip5.output1.CSIRO-QCCCE.CSIRO-Mk3-6-0.historicalAA.fx.atmos.fx.r0i0p0

					Shouldn't those be historicalMisc and the forcings detailed within the
					global attributes?

					By the way, we are also missing the contact data for CSIRO data node in
					the CMIP5 Status Page.

					Thanks,
					Estani

					--
					Bryan Lawrence
					University of Reading:  Professor of Weather and Climate Computing.
					National Centre for Atmospheric Science: Director of Models and Data. 
					STFC: Director of the Centre for Environmental Data Archival.
					Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence
					_______________________________________________
					GO-ESSP-TECH mailing list
					GO-ESSP-TECH at ucar.edu
					http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

				_______________________________________________
				GO-ESSP-TECH mailing list
				GO-ESSP-TECH at ucar.edu
				http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

		------------------------------

		The information in this email together with any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. There is no waiver of any confidentiality/privilege by your inadvertent receipt of this material. 

		Any form of review, disclosure, modification, distribution and/or publication of this email message is prohibited, unless as a necessary part of Departmental business.

		If you have received this message in error, you are asked to inform the sender as quickly as possible and delete this message and any copies of this message from your computer and/or your computer system network.

		------------------------------

	-- 
	Muhammad Atif
	ANU Supercomputer Facility      NCI National Facility
	Leonard Huxley, Mills Road      Room 318, Bldg # 56
	Australian National University  Ph:  +61 2 6125 5031
	Canberra, ACT 0200, Australia   M:   0430 393863
	CRICOS Provider #00120C         http://anusf.anu.edu.au

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120418/a88d9730/attachment.html 