<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<font face="Times New Roman">Hello Laura,<br>
<br>
I'll insert some responses below:<br>
<br>
</font>
<div class="moz-cite-prefix">On 3/27/13 1:28 PM, Laura Carriere
wrote:<br>
</div>
<blockquote cite="mid:51535650.2040702@nasa.gov" type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div class="moz-cite-prefix"><br>
Karl,<br>
I have gone through your document and mapped the variables to
both the ones proposed in last week's telecon and to the NEX
variables. Your proposed attributes work for NEX without any
trouble, with one possible exception. <br>
<br>
I do have the following feedback/questions:<br>
<br>
1. Last weeks' proposal had included a "predictor-version"
attribute in order to be able to track which version of the
original model data was used. Your proposal includes a
"driving_data_tracking_ids" variable. Would I be correct in
assuming that this serves the same purpose? Also, we've just
realized that the data provider has not provided this
information to us and it may not be retrievable (we'll ask). If
it isn't, do you have a default or backup recommendation for
this attribute?</div>
</blockquote>
In CMIP5 version numbers are assigned by the person publishing the
data, and it seems that these numbers have not always been assigned
consistently. In CMIP5 a tracking_id is assigned automatically to
each file and is stored as a global attribute. As a first choice, I
was suggesting collecting the tracking_id's from all the files used
in "driving" the downscaling. As a second choice you could store
some other version identifier there (presumably, the CMIP5 dataset
version number). So, yes this was meant to replace your
"predictor_version" attribute. <br>
<blockquote cite="mid:51535650.2040702@nasa.gov" type="cite">
<div class="moz-cite-prefix"> <br>
2. You recommended that the resolution_id should contain the
resolution rounded off to the nearest km. The NEX resolution is
800 meters. While we could round up to 1, could we also not use
.8? Without arguing the current value of data downscaled to 800
meters, resolutions are moving in that direction.<br>
</div>
</blockquote>
I had assumed no downscaling method for climate data would be
accurate enough to warrant providing information on scales smaller
than a km. I guess some folks disagree, so to allow for the case
of 800 meters, I wouldn't recommend using a decimal point (because
that character appearing in filenames, for example, usually means
something special). For other purposes, we've substituted a "p" for
".", so perhaps that would be acceptable here. For 800 meters you
would write horizontal_id=hnp8 (or hip8, if this is an interpolated
product). Another option would be to place a "m" after the number
(e.g., hn800m). If this were done, we might consider adding "km" to
resolutions expressed in kilometers, but I'm not sure about that.
Comments welcome.<br>
<blockquote cite="mid:51535650.2040702@nasa.gov" type="cite">
<div class="moz-cite-prefix"> <br>
3. I'm still looking for a place to include the doi for the
dataset. The optional references attribute seems like the best
place for it but it could also go in the contact attribute? Any
thoughts? <br>
</div>
</blockquote>
Yes, I would certainly include it in the "source" attribute, rather
than the contact attribute, but it would make sense to me to also
include a new global attribute called "doi" and store in it the full
doi string. The source attribute can be understood by humans, but
the "doi" attribute could more readily be interpreted by software.<br>
<br>
best regards,<br>
Karl<br>
<blockquote cite="mid:51535650.2040702@nasa.gov" type="cite">
<div class="moz-cite-prefix"> <br>
<br>
Thanks.<br>
<br>
Laura.<br>
<br>
On 3/25/2013 5:49 PM, Karl Taylor wrote:<br>
</div>
<blockquote cite="mid:5150C675.8010309@llnl.gov" type="cite">
<meta name="Title" content="">
<p class="MsoNormal">Dear all,<o:p></o:p></p>
<p class="MsoNormal">I have spent considerable time reviewing
the following four documents:<o:p></o:p></p>
<p class="MsoNormal">A. The email (copied below) sent by <span
style="mso-fareast-font-family: "Times New
Roman";mso-bidi-font-family:"Times New
Roman"">Galia and Aparna, which </span>proposed
attributes, filenames, and directory structures for downscaled
data.<o:p></o:p></p>
<p class="MsoNormal">B. <span style="mso-spacerun:yes"> </span><a
moz-do-not-send="true"
href="http://cmip-pcmdi.llnl.gov/cmip5/docs/cmip5_data_reference_syntax.pdf">http://cmip-pcmdi.llnl.gov/cmip5/docs/cmip5_data_reference_syntax.pdf</a>
which describes the corresponding CMIP5 metadata.<o:p></o:p></p>
<p class="MsoNormal">C. <span
style="mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman""><a
moz-do-not-send="true"
href="http://cordex.dmi.dk/joomla/images/CORDEX/cordex_archive_specifications_121022.pdf"><span
style="font-size:11.5pt;font-family:Arial;color:#1155CC">http://cordex.dmi.dk/joomla/images/CORDEX/cordex_archive_specifications.pdf</span></a>
which describes the corresponding CORDEX metadata.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman"">D.<span
style="mso-spacerun:yes"> </span><a
moz-do-not-send="true"
href="http://cmip-pcmdi.llnl.gov/cmip5/docs/CMIP5_output_metadata_requirements.pdf">http://cmip-pcmdi.llnl.gov/cmip5/docs/CMIP5_output_metadata_requirements.pdf</a>
which specifies all the CMIP5 metadata requirements.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman"">I hope
that document A above could be made compatible with the
others and in general could provide a sound basis for
establishing more uniformity moving forward.<span
style="mso-spacerun:yes"> </span>Toward that end, I have
prepared the attached document describing for downscaled
data a minimal set of global attributes needed to augment
those used in CMIP5 and also the extensions needed to the
DRS document to accommodate downscaled data.</span></p>
<p class="MsoNormal"><span
style="mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman"">I hope at
least a few of you will take the time to study this document
and provide feedback. <br>
</span></p>
<p class="MsoNormal"><span
style="mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman"">Best
regards,<br>
Karl<br>
</span></p>
<p class="MsoNormal"><span
style="mso-fareast-font-family:"Times New Roman";
mso-bidi-font-family:"Times New Roman""><br>
</span>Mail sent by Galia Guentchev 3/12/13<br>
<br>
**********************************************************************<br>
Details of each element of the proposed directory structure:<br>
<br>
Proposed elements -<br>
/projectID/<font color="#3333ff">sub-project</font>/product/institution/<b>predictorModel/experimentID/frequency/realm/MIPtable/Pred</b><b><br>
</b><b>ictor_experiment_rip/predictorversion</b>/<i>downscalingMethod/predictand
(variableName)/region</i>/<i>DownscaledDataversion</i>/file_name.nc<br>
<br>
Example:<br>
<br>
/ncpp2013/<font color="#3333ff">perfectModel</font>/downscaled/NOAA-GFDL/<b>GFDL-HIRAM-C360-coarsened/amip/day/atmos/day/r1i1p1/v20121024</b>/<i>GFDL-ARRMv1/tasmax/US48/v20120227</i>/tasmax_day_amip_r1i1p1_downscaled_US48_GFDLARRMv1_19790101-19831231.nc<br>
<br>
The new element <font color="#3333ff">sub-project </font>(in
blue above) gives the opportunity to indicate to users that in
the one case the method was trained on observations (standard
setting), and in the other on model that was considered to be
the truth (perfect model setting);<br>
The options there could be: PerfectModel or Standard - where
possibly there could be a different name instead of 'standard'
for the standard downscaling setting.<br>
<br>
</p>
<p>For NASA datasets some of the directories could be: <br>
</p>
project = NEX<br>
product = downscaled<br>
institution = NASA-Ames<br>
predictorModel - original model value<br>
experimentID = historical<br>
frequency = mon<br>
realm = atmos<br>
Predictor_experiment_rip - original model value<br>
variable = precipitation or temperature<br>
region = CONUS<br>
<br>
DownscalingMethod will also be included as a directory to allow
for search on method.<br>
<br>
**********************<br>
There are a set of sub-directories that refer to the <u>PredictorModel</u>
- presented in bold - <b>/predictorModel/experimentID/frequency/realm/MIPtable/Pred</b><b><br>
</b><b>ictor_experiment_rip/predictorversion</b><br>
<br>
Where: <br>
<ul>
<li>predictor model - is the specific GCM which is the source
of the predictor data set - GFDL-HIRAM-C360-coarsened - in
the above example</li>
<li>experimentID - the specific experiment - amip in this case</li>
<li>frequency - refers to the temporal scale of the predictor
fields - daily</li>
<li>realm - the realm of the predictors - in this case
atmos(phere)</li>
<li>MIPtable - name of the model intercomparison table - daily
in this example, could be amon - for atm monthly data;</li>
<li>Predictor-Experiment-rip - follows the standard notation
from CMIP5</li>
<li>version - the version date of the global model that
provided the predictor dataset</li>
</ul>
<p>The elements above follow quite closely the structure for
CMIP5 model output directory elements.<br>
</p>
There is a set of sub-directories that refer to the Downscaling
method - presented in italics - <br>
<i><i>downscalingMethod/predictand (variableName)/region</i>/<i>DownscaledDataversion<br>
</i> <br>
</i>Where:<br>
<ul>
<li>downscalingMethod - is the downscaling method abbreviation
- in this case GFDL-ARRMv1 - the GFDL in the name indicates
that this is a setting applied by GFDL where there were two
sets of predictors, based on the ARRM method of K.Hayhoe;
also v.1 indicates which version of the ARRM method was used
(the original version) - more details about the method are
given in the global attributes of the file;</li>
<li>Predictand (variableName) - the specific predictand
variable that was downscaled; tasmax in this case;<br>
</li>
<li>region - indicates that the method was applied to the US48</li>
<li>DownscaledDataversion - the version of the downscaled
dataset<br>
</li>
</ul>
<p><b>For the purposes of standardization there are two
directions to consider:</b> <br>
</p>
<p>1) One is to have<b> one standard directory</b> structure
that will be used by all - for example, following the example
of GFDL to have the details of the predictor model first and
then the downscaling method details:</p>
<ul>
<li>ProjectID - sub-project - product - Institution -
Predictor dataset details - Downscaling method details -
Filename</li>
</ul>
<p> Having a standardized approach would help any automated
service/web service to detect the directory path for a
particular dataset. </p>
<p>2) During our last teleconference there was a proposal to
follow the downscaling practice and describe the downscaling
method first and then the predictor model. This leads to <b>two
paths</b>:</p>
<p> • ProjectID - <u><font color="#3333ff">Standard or
Perfect Model sub-project facet</font></u><font
color="#3333ff"> </font>- product - Institution - then see
below:<br>
- (if Perfect model setting) Predictor dataset
details - Downscaling method details, <br>
- (if Standard setting) - Downscaling method
details - Predictor dataset details<br>
</p>
<p><br>
The NCPP Core team accepts that it may be reasonable to have a
directory structure - where the method description is first;
and another directory structure - where the predictor
description is first and then the methods that are applied are
described; <b>NCPP will support either approach</b> (one
overall directory structure, or two separate pathways) and if
the second approach is chosen (with two different
sub-directory sequences) - we would like to promote and to
support the standardization of these different directory
pathways - meaning - we will support two standardized
directory structures to accommodate two common practices.<br>
</p>
<p><br>
</p>
******************<br>
<font color="#009900">Additional details<font color="#000000">:
</font></font><br>
<br>
<b>Variable level attributes-</b><br>
The published dataset should also conform to CF-standards. <br>
eg-<br>
<br>
tasmax:long_name = "Downscaled Daily Maximum
Near-Surface Air Temperature"
;
<br>
tasmax:units = "K"
;
<br>
tasmax:missing_value = 1.e+20f
;
<br>
tasmax:_FillValue = 1.e+20f
;
<br>
tasmax:standard_name = "air_temperature"
;
<br>
tasmax:original_units = "K"
;
<br>
<b> tasmax:downscaling_method: GFDL-ARRMv1</b><br>
<br>
<b>Global attributes- </b>listing a few here, several
CMIP-style attributes will be inherited. <br>
<br>
"predictorModel" will replace "model_id"<br>
For the 'downscaling model', as agreed with Luca on the call
it would be 'downscalingMethod' <br>
<br>
:Conventions = "CF-1.4" ;<br>
:references = "info about model, training
datasets etc will be provided here"<br>
:info = "additional info about the downscaling
method" <br>
:creation_date = "2011-08-19T21:57:06Z" ;<br>
:institution = "NOAA GFDL(201 Forrestal Rd,
Princeton, NJ, 08540)" ;<br>
:history = "info on file processing. Eg"
processed by toolX." ;<br>
:projectID = ncpp2013<br>
:subprojectID = perfectModel<br>
:product = downscaled<br>
:institution = NOAA-GFDL<br>
:predictorModel = GFDL-HIRAM-C360-coarsened<br>
:experimentID = amip<br>
:frequency = day<br>
:modeling_realm = atmos<br>
:Predictor_experiment_rip = r1i1p1<br>
:region = US48<br>
:table_id = day<br>
:version = v20120227<br>
:downscalingMethod = GFDL-ARRMv1<br>
**************************************************<br>
<br>
Best regards,<br>
Galia and Aparna<br>
<br>
<pre class="moz-signature" cols="72">--
Galia Guentchev, PhD
Project Scientist
National CLimate
Predictions and
Projections
Platform (NCPP)
NCAR RAL CSAP
FL2 3103
3450 Mitchell Lane
Boulder, CO, 80301
phone: 303 497 2743 </pre>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Laura Carriere, CSC <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:laura.carriere@nasa.gov">laura.carriere@nasa.gov</a>
NCCS, Code 606.2 301 614-5064</pre>
</blockquote>
<br>
</body>
</html>