<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi all,<div><span class="Apple-tab-span" style="white-space:pre"> </span>in preparation for our upcoming conference call, I compiled a list of questions that could guide our discussion. This is just a possible way to tackle the problem, if anybody thinks we should follow a different course we can certainly do that.</div><div><br></div><div>thanks, Luca</div><div><br></div><div><!--StartFragment--><p class="MsoNoteLevel1CxSpFirst" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">AGENDA</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">KEY QUESTIONS</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">1) What are the fields that unequivocally identify an
observational dataset, for the purpose of comparing to CMIP5 models ? Are the
fields the same for different types of observations ?</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in"><span class="Apple-tab-span" style="white-space:pre"> </span>- Fields that have the same values as for models:
variable, sampling frequency, realm, time period, version</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in"><span class="Apple-tab-span" style="white-space:pre"> </span>- Fields that have model correspondent, but not
necessarily the same values: activity, product</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in"><span class="Apple-tab-span" style="white-space:pre"> </span>- Fields that are specific to observations:
instrument, mission ?, agency ?, resolution ? level and/or processing algorithms, others ?</p><p class="MsoNoteLevel1CxSpFirst" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">2) Should observations be organized according to the same
directory structure as for models ? If not, should the hierarchy be the same
for all observations, or be different for different kind of observations ?</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">3) What is the convention naming observational files ? (informed
from the convention for the directory hierarchy)</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">4) Should CMOR be mandated for processing observations for CMIP5,
or should we rely on CF and CMOR checkers ?</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in"><span class="Apple-tab-span" style="white-space:pre"> </span>- If CMOR is mandated, how much work (if any) is
involved, and where should the funding come from ?</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">5) Should the Controlled Vocabulary for observations be encoded
in one CMOR table, or more than one ?</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">6) How can the Controlled Vocabulary be developed as a community
?</p><div><br></div><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">OTHER DETAILS</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">Should data be organized on disk as from the CMOR output, or
as the DRS specification ?</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">Which global attributes to be included in netCDF files ?</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">Which characters can be included in fields that are not
controlled vocabularies, for example the <processing level and product
version> ?</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">How to encode ascending versus descending satellite
measurements ?</p><p class="MsoNoteLevel1CxSpMiddle" style="margin-left:0in;mso-add-space:auto;
text-indent:0in">Should the names in the controlled vocabulary be case
sensitive ?</p><p class="MsoNoteLevel1CxSpLast" style="margin-left:0in;mso-add-space:auto;
text-indent:0in"><o:p> </o:p></p>
<!--EndFragment-->
</div><div><br></div><div><br><div><div>On Feb 9, 2011, at 10:07 PM, Karl Taylor wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<div bgcolor="#ffffff" text="#000000">
<font face="Times New Roman">Hi all,<br>
<br>
I'm at a meeting on the east coast and will be tied up from 11-12
this morning. Here are some brief comments concerning the draft
proposal that I hope will be of some use in my absence:<br>
<br>
1. The directory structure and filenames (and the underlying DRS)
are all meant to make it easier for users to navigate to the data
they want and to unambiguously identify the data, so that is much
more important than making it look like CMIP5. My sense is that
folks looking for observational data will want to be able to
easily see (through the DRS categories) <br>
a) the variable<br>
b) the sampling frequency<br>
c) perhaps the "realm"<br>
d) the time-period over which the variable was measured<br>
<br>
Users will want to be able to distinguish among the various
observational products available for his (her) purposes. So,<br>
a) something about how the measurement was made and processed
(maybe instrument is sufficient; perhaps institute, mission,
agency are not all needed)<br>
b) version of observational product<br>
<br>
2. Note that the default directory structure generated by CMOR2
differs from the ESGF directory structure, as described in
sections 3.1 and 3.3 of <br>
<br>
<a class="moz-txt-link-freetext" href="http://cmip-pcmdi.llnl.gov/cmip5/output_req.html?submenuheader=2#req_format">http://cmip-pcmdi.llnl.gov/cmip5/output_req.html?submenuheader=2#req_format</a></font>
<style>@font-face {
font-family: "Arial";
}@font-face {
font-family: "Arial";
}p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 6pt 0in; font-size: 12pt; font-family: "Times New Roman"; }p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle { margin: 6pt 0in; font-size: 14pt; font-family: Arial; font-style: italic; }span.SubtitleChar { font-family: Arial; font-style: italic; }.MsoChpDefault { font-size: 10pt; }div.WordSection1 { page: WordSection1; }</style><p class="MsoSubtitle" style="text-align: justify;"><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">In that document
"It is
recommended that ESGF data nodes should layout datasets on disk
mapping DRS components
to directories as:</span></p><p class="MsoSubtitle" style="margin-left: 1in; text-indent:
-0.5in;"><span style="font-size: 12pt; font-family: "Times
New Roman"; font-style: normal;" lang="X-NONE"><</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">activity</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">product</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">institute</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">model</span><span style="font-size:
12pt; font-family: "Times New Roman"; font-style:
normal;" lang="X-NONE">>/<</span><span style="font-size:
12pt; font-family: "Times New Roman";" lang="X-NONE">experiment</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">frequency</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">modeling realm</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">MIP table</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">ensemble
member</span><span style="font-size: 12pt; font-family:
"Times New Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">version number</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>/<</span><span style="font-size: 12pt; font-family: "Times New
Roman";" lang="X-NONE">variable
name</span><span style="font-size: 12pt; font-family:
"Times New Roman"; font-style: normal;" lang="X-NONE">>/
<</span><span style="font-size: 12pt; font-family:
"Times New Roman";" lang="X-NONE">CMOR filename</span><span style="font-size: 12pt; font-family: "Times New
Roman"; font-style: normal;" lang="X-NONE">>.nc</span></p><p class="MsoSubtitle"><span style="font-size: 12pt; font-family:
"Times New Roman"; font-style: normal;" lang="X-NONE">Example:</span></p><p class="MsoSubtitle" style="margin-left: 1in; text-indent:
-0.5in;"><span style="font-size: 12pt; font-family: "Times
New Roman"; font-style: normal;" lang="X-NONE">/CMIP5/output1/UKMO/HadCM3/decadal1990/day/atmos/day/r3i2p1/v20100105/tas/
tas_day_HADCM3_
decadal1990_r3i2p1_199001-199012.nc<br>
</span></p>
The observations don't need to follow this template (and probably
shouldn't), but the current observations draft document incorrectly
describes the CMIP5 structure.<br>
<font face="Times New Roman"><br>
3. I would recommend that observational products be written using
CMOR2. I do not think it is a good use of resources to generalize
and "harden" the CMOR checker to enforce anything. It wasn't
meant for this purpose and this would be a big job.<br>
<br>
4. I would advise that all variables that appear in a single CMOR
table at least<br>
a) share the same sampling frequency<br>
b) share the same realm (although you might want to include 2
closely-related realms)<br>
<br>
4. Recall Charles Doutriaux's note that we do not yet have
program support for some of what will be needed. <br>
<br>
5. also a reminder: observations should not be under the "CMIP5"
activity. I can ask the WGCM if something like "obs4CMIP5" would
be o.k. (I rather like this.)<br>
</font><br>
In preparing the above comments, I've mostly thought about gridded
global datasets.<br>
<br>
Best regards,<br>
Karl<br>
<br>
On 2/9/11 9:25 AM, Cinquini, Luca (3880) wrote:
<blockquote cite="mid:92D3F86B-72A0-45C9-8A8D-B9415A31EC09@jpl.nasa.gov" type="cite">
<pre wrap="">Hi all,
Dean Williams has kindly made the following number available for tomorrow's conference call:
(925) 424-8105
access code 305757#
The call is scheduled for 8am PST / 9am MST / 10am CST / 11am EST / 16pm GMT / 17pm France/Germany. We will discuss the adoption of community-wide metadata conventions for observational datasets that are going to be made part of the CMIP5 archive.
Thanks in advance to all for participating,
Luca
On Feb 9, 2011, at 8:35 AM, Christensen, Sigurd W. wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Luca,
Several of us think that a call Thursday would be good, but we probably won't finish then.
We think that not only a different CMOR table, but also a different directory structure/filename structure may be appropriate for three or more of the categories mentioned in the link:
-Decide on whether to have one single CMOR table for observations (currently "obsSites"), or more than one depending on types of observational data:
*remote sensed (grids and swaths)
*in-situ stations (time series and profiles)
*trajectory-based observations
*in-situ gridded products
The discussion thus far emphasizes fields and order for naming conventions for satellite-based data. Perhaps those can be finalized Thursday. But point-oriented surface and/or profile time-series data (such as ARM, AmeriFlux, etc.), and trajectory-based observations, will likely need more consideration. Karl, on January 31, indicated that variable name, modeling realm, and frequency should be carried to the DRS (Data Reference System), but the rest could in essence be tailored to the needs of observational data.
Thanks,
Giri and Sig
-----Original Message-----
From: Cinquini, Luca (3880) [<a class="moz-txt-link-freetext" href="mailto:Luca.Cinquini@jpl.nasa.gov">mailto:Luca.Cinquini@jpl.nasa.gov</a>]
Sent: Tuesday, February 08, 2011 15:58
To: Lynnes, Christopher S. (GSFC-6102)
Cc: Huffman, George J. (GSFC-613.1)[SCIENCE SYSTEMS APPLICATIONS]; Karl Taylor; Steve Hankin; Bryan Lawrence; <a class="moz-txt-link-abbreviated" href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a>; Sébastien Denvil; climate-obs; McCoy, Renata
Subject: Re: [Go-essp-tech] Extending the DRS syntax to observations
Hi all,
I would like to propose to have a conference call to discuss and hopefully resolve any remaining issues concerning metadata conventions for CMIP5 observations. Would anybody object if we had this call in only two days, next Thursday February 10, at 8am PST/11am EST - which I think is is 4pm in the UK and 5pm in France and Germany ? If this is too soon, we could postpone till next week.
As a remainder, this is the URL of the current proposal:
<a class="moz-txt-link-freetext" href="http://oodt.jpl.nasa.gov/wiki/display/CLIMATE/Data+and+Metadata+Requirements+for+CMIP5+Observational+Datasets">http://oodt.jpl.nasa.gov/wiki/display/CLIMATE/Data+and+Metadata+Requirements+for+CMIP5+Observational+Datasets</a>
which at the very beginning contains a summary of the issues still open. Please reply if you can't make the meeting and you really would like to attend, or if you think there are other issues to discuss.
Best regards,
thanks, Luca
P.S.: if the conference is a go, we'll setup a phone line....
On Feb 2, 2011, at 3:17 PM, Lynnes, Christopher S. (GSFC-6102) wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Feb 2, 2011, at 5:08 PM, Cinquini, Luca (3880) wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Chris and George,
thanks for your input... I guess the question is wether you would be opposed to re-arranging the fields according to an order that is commonly agreed upon (and that possibly resembles the DRS structure for models), provided that all the relevant information is included ?
</pre>
</blockquote>
<pre wrap="">Since my philosophy is to tailor for the expected user community, I defer to you and your colleagues regarding the order, since you know the community. My main interest is just ensuring the inclusion of the relevant information.
</pre>
<blockquote type="cite">
<pre wrap="">I think at this point we might be able to make faster progress by organizing a conference call to discuss these issues...
thanks, Luca
On Feb 2, 2011, at 2:42 PM, Lynnes, Christopher S. (GSFC-6102) wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Feb 2, 2011, at 4:26 PM, George J. Huffman wrote:
</pre>
<blockquote type="cite">
<pre wrap="">There are other variables that could go in the last position since the
original datasets contain multiple variables as "fields". I should say
that the Goddard DISC puts Level before Instrument, and you might want
to consider why they did that. [This is mostly an issue if you're
trying to build a syntax that is generally useful, not just focused on
gridded data.]
</pre>
</blockquote>
<pre wrap="">We (at Goddard DISC) put Level before Instrument because we anticipate that the user community for Level 3 gridded data is somewhat distinct than for Level 2 or Level 1 swath data, which require considerably more sophisticated and customized tools to work with than Level 3. I don't know if that is as relevant in the CMIP5 context as in our more generalized search interface (as George implies.)
--
Dr. Christopher Lynnes NASA/GSFC, Code 610.2 phone: 301-614-5185
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">--
Dr. Christopher Lynnes NASA/GSFC, Code 610.2 phone: 301-614-5185
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap=""></pre>
</blockquote>
</div>
</blockquote></div><br></div></body></html>