[Go-essp-tech] Forcings on Historical

Sat Mar 31 11:09:47 MDT 2012

Dear Muhammad,

Thanks very much for volunteering to help clean this up.  As Bryan says, 
this will be a huge help to users and will save countless hours of 
effort for folks involved with ESGF development and those processing 
data.   There will be benefits for years to come.

I hope others copied will read this and make sure I haven't forgotten 
anything.

There are various degrees of conformance you might attempt, but I think 
renaming the files and republishing are both essential.  In the 
filenames, please

1. Replace "historicalXXXX" with "historicalMisc"  (for the expts. 
listed in a previous email.)
2. Replace "p1" with "pX", where the integer X should take on a 
different value for each of the different forcing runs (see 3 below for 
more info.)

You should send Gavin Schmidt a table indicating which "p" value 
corresponds to which forcing, as was requested in an email about a week 
ago.  (If you didn't get that email, please let me know.)

It would be desirable (and perhaps essential -- others might chip in 
here) also to correct the global attributes in the netCDF files themselves:

1. Set experiment_id = 'historicalMisc'
2. Set experiment = 'other historical forcing'
3. Replace physics_version with "p" value consistent with the filename.
physics_version = an integer (?1) referring to the physics version used 
by the model If there is only one physics version of the model, then 
this argument should be normally given the value 1. Note that model 
versions that are substantially different should be given a different 
"model_id"; assigning a different "physics_version" should be reserved 
for closely-related model versions (e.g., as in a "perturbed physics" 
ensemble) or for the same model, but with different forcing or feedbacks 
active. In CMIP5, one would distinguish, for example, among runs forced 
by different combinations of "forcing" agents (as called for under the 
"historicalMisc" experiment -- experiment 7.3) by assigning different 
values to physics_version. For fields appearing in table "fx" in the 
CMIP5 Requested Output, set physics_version=0 (violating the general 
rule that it should be a positive definite integer). Note that the 
physics_version is used in constructing the "ensemble member" called for 
by the DRS document; it is the value of L in r<N>i<M>p<L>.
4. Generate a new tracking_id which is "a string that is almost 
certainly unique to this file and must be generated using the OSSP 
utility which supports a number of different DCE 1.1 variant UUID 
options. For CMIP5 version 4 (random number based) is required. Download 
the software from http://www.ossp.org/pkg/lib/uuid/. The tracking_id 
might look something like: 02d9e6d5-9467-382e-8f9b-9300a64ac3cd."
5. Also check that the "forcing" attribute has the correct forcing 
identifier(s) (see 
http://cmip-pcmdi.llnl.gov/cmip5/docs/cmip5_data_reference_Appendix1-2.doc )

I think this could be fairly easily scripted, but what sounds easy often 
is not.  If you are unfamiliar with netCDF Operators (NCO), they can be 
used to rewrite attributes in netCDF files.

After correcting the files and filenames, I think you will have to 
republish.  I hope others will provide guidance on this.

Please include anyone you think is interested in these emails.

Thanks again for helping clean things up.  It is much appreciated.
And don't hesitate to write with any questions.

Best regards,
Karl

On 3/30/12 10:58 PM, Muhammad Atif wrote:
> Just to update; I had an email conversation with Estani when he first realised of the problem. We are prepared to do whatever might be required at the publishing end.
> He suggested to wait for a nice solution.
>
> btw, should I bring the modellers (CSIRO-QCCCE team) into the email loop as well?
>
>
> Regards
>
>
> On 31/03/2012, at 4:46 PM, Bryan Lawrence wrote:
>
>> I think it'd be better to fix this at source, otherwise these problems will lurk for ever not only in ESGF, but on the hard drives of everyone who downloads the data - causing a potential problem for the actual science because everyone will have to have their own hacks to deal with it.
>>
>> I realise this may be a bit unfair on folks who wrote their data in advance, but the flip-side is that not doing so is unfair on everyone who uses that data ...
>>
>> Cheers
>> Bryan
>>
>>
>>> Hi Estani,
>>>
>>> Yes, the official name for all of these is historicalMisc, and the
>>> different runs should have been identified by a different "p" value in
>>> the "rip", which distinguishes among members of an ensemble.  The DRS
>>> document describes this in more detail.
>>>
>>> I *think* these datasets may have been generated prior to the DRS
>>> controlled vocabulary being fully settled, so I haven't raised a stink
>>> about this.  I note that they show up in the old gateway1.3.4 search,
>>> but not in the new p2p search.
>>>
>>> Is there some way that the non-conforming experiment names could be
>>> changed, so that users can find these experiments under historicalMisc?
>>>
>>> Best regards,
>>> Karl
>>>
>>>
>>> On 3/30/12 3:19 AM, Estanislao Gonzalez wrote:
>>>> Hi,
>>>>
>>>> I'm trying to replicate data from CSIRO-QCCCE and I see the following
>>>> experiments names:
>>>> historicalNoOz
>>>> historicalNoAA
>>>> historicalAntNoAA
>>>> historicalAnt
>>>> historicalAA
>>>>
>>>> These are used as the dataset id, e.g.
>>>> cmip5.output1.CSIRO-QCCCE.CSIRO-Mk3-6-0.historicalAA.fx.atmos.fx.r0i0p0
>>>>
>>>> Shouldn't those be historicalMisc and the forcings detailed within the
>>>> global attributes?
>>>>
>>>> By the way, we are also missing the contact data for CSIRO data node in
>>>> the CMIP5 Status Page.
>>>>
>>>> Thanks,
>>>> Estani
>>>>
>>>>
>> --
>> Bryan Lawrence
>> University of Reading:  Professor of Weather and Climate Computing.
>> National Centre for Atmospheric Science: Director of Models and Data.
>> STFC: Director of the Centre for Environmental Data Archival.
>> Ph: +44 118 3786507 or 1235 445012; Web:home.badc.rl.ac.uk/lawrence
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20120331/8722cb7a/attachment.html 