<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.16981" name=GENERATOR></HEAD>
<BODY
style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2>Hi Bob,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2>Thanks for promptly commenting on the document.
Clarifying that the publisher has these features is great news and I'm sorry
that, in trying to give everyone time to digest the document by Tuesday, I
didn't have time to confirm the facts with you. I'm hoping this way any
errors will come out in the wash.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2>The main thing I missed was the ability to create multiple
THREDDS catalogues for a dataset (or 1 catalogue per dataset version).
Omitting this feature felt like a funder mental difference in model to the
DRS. I need to work out how to do this now and I'll revise the version
directory structure document too. Phil Bentley has recommended a different
structure that has some advantages so the document will probably look very
different next time.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2>Incidentally, I'm increasingly impressed with the ESG
publisher and I'm really enjoying working with it. T</FONT></SPAN><SPAN
class=740515807-16042010><FONT face=Arial color=#0000ff size=2>he stuff you've
done with project handler plugins in the latest release strengthens my
impression that it is a tool we will be using for a long
time.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2>Cheers,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=740515807-16042010><FONT face=Arial
color=#0000ff size=2>Stephen.</FONT></SPAN></DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV align=left><FONT face=Arial size=2>---</FONT></DIV>
<DIV align=left><FONT face=Arial size=2>Stephen Pascoe +44 (0)1235
445980</FONT></DIV>
<DIV align=left><FONT face=Arial size=2>British Atmospheric Data
Centre</FONT></DIV>
<DIV align=left><FONT face=Arial size=2>Rutherford Appleton
Laboratory</FONT></DIV>
<DIV> </DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Bob Drach [mailto:drach1@llnl.gov]
<BR><B>Sent:</B> 16 April 2010 00:14<BR><B>To:</B> Pascoe, Stephen
(STFC,RAL,SSTD)<BR><B>Cc:</B> go-essp-tech@ucar.edu<BR><B>Subject:</B> Re:
[Go-essp-tech] Proposed version directory structure
document<BR></FONT><BR></DIV>
<DIV></DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Hi Stephen,</SPAN></FONT>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></DIV>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Let me clarify a few points in the description of ESG
Publisher:</SPAN></FONT></DIV>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></DIV>
<DIV>
<DIV>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px">The document states:
"ESG Publisher version system is built around mutable datasets. It does
not attempt to maintain references to previous data and the dataset version
number is not part of the dataset id unless the publisher is configured to
include it from the dataset metadata. This means that it is not straight
forward at this time to publish multiple versions of an atomic dataset unless
each version is published as a separate dataset. This approach would
effectively ignore ESG Publisher's version system and manage all versions
independently."</SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px"><FONT
class=Apple-style-span face=Helvetica>- As of Version 2 the unit of publication
is in fact a 'dataset version', terminology that came out of the December
meeting in Boulder. A dataset version is an immutable object which can represent
a 'DRS dataset including version number'. The published 'dataset version' itself
has an identifier which typically consists of dataset_id+version number; this
appears in the THREDDS catalog. As you stated in the document, whether or not
the published dataset corresponds to a DRS dataset is a matter of publisher
configuration, not an inherent property of the
publisher.</FONT></SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px"><FONT
class=Apple-style-span face=Helvetica>- The node database does in fact maintain
references to the composition of previous dataset versions. It is possible to
have multiple versions published simultaneously, to list all published versions
of a dataset, and for any given dataset version the files contained in that
version can be listed.</FONT></SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px"><FONT
class=Apple-style-span face=Helvetica>- The intention of the publisher design is
to automate versioning as much as possible. A 'dataset' is considered to be a
collection of dataset versions. Consequently, 'publishing a dataset' really
means 'publishing a dataset version where the version number is incremented
relative to the previous version.' Similarly, 'unpublishing' a dataset by
default unpublishes all versions of a dataset. The terminology dataset_id#n can
be used to refer to a specific version.</FONT></SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
face=Helvetica size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
face=Helvetica size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px">In
short, there is no fundamental mismatch between the DRS model and the ESG
publisher.</SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
face=Helvetica size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
face=Helvetica size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px">Best
regards,</SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
face=Helvetica size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
face=Helvetica size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Bob</SPAN></FONT></P>
<P style="MARGIN: 0px 0px 6px; FONT: 12px Times"><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></P>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></DIV>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></DIV>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">On Apr 15, 2010, at 3:24 AM, <<A
href="mailto:stephen.pascoe@stfc.ac.uk">stephen.pascoe@stfc.ac.uk</A>>
wrote:</SPAN></FONT></DIV><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px"><BR
class=Apple-interchange-newline></SPAN></FONT>
<BLOCKQUOTE type="cite">
<DIV>
<DIV><SPAN class=723111410-15042010><FONT face=Arial><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Hi everyone,</SPAN></FONT></FONT></SPAN></DIV>
<DIV><SPAN class=723111410-15042010><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px"><FONT
face=Arial></FONT></SPAN></FONT></SPAN><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"></SPAN></FONT> </DIV>
<DIV><SPAN class=723111410-15042010><FONT face=Arial><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Attached is my view on how we should structure the
archive to support multiple versions. It divides into 2 main sections,
the first is a fairly lengthy summary of why this problem isn't solved yet in
terms of the differences between the ESG datanode software and the DRS
document. The second section lays out the proposed structure and how we
would manage symbolic links and moving from one version to another. I
restrict myself to directories below the atomic dataset level.
</SPAN></FONT></FONT></SPAN></DIV>
<DIV><SPAN class=723111410-15042010><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px"><FONT
face=Arial></FONT></SPAN></FONT></SPAN><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"></SPAN></FONT> </DIV>
<DIV><SPAN class=723111410-15042010><FONT face=Arial><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Lots of issues are left to resolve, in particular how
we ESG publisher can make use of this structure. I'll try and draw
attention to these points in the agenda for Tuesday's telco which will follow
later today.</SPAN></FONT></FONT></SPAN></DIV>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><FONT face=Arial></FONT></SPAN></FONT> </DIV>
<DIV><SPAN class=723111410-15042010><FONT face=Arial><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Cheers,</SPAN></FONT></FONT></SPAN></DIV>
<DIV><SPAN class=723111410-15042010><FONT face=Arial><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">Stephen.</SPAN></FONT></FONT></SPAN></DIV>
<DIV><SPAN class=723111410-15042010><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px"><FONT
face=Arial></FONT></SPAN></FONT></SPAN><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"></SPAN></FONT> </DIV>
<DIV align=left><FONT face=Arial><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px">---</SPAN></FONT></FONT></DIV>
<DIV align=left><FONT face=Arial><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px">Stephen Pascoe +44
(0)1235 445980</SPAN></FONT></FONT></DIV>
<DIV align=left><FONT face=Arial><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px">British Atmospheric Data
Centre</SPAN></FONT></FONT></DIV>
<DIV align=left><FONT face=Arial><FONT class=Apple-style-span size=4><SPAN
class=Apple-style-span style="FONT-SIZE: 14px">Rutherford Appleton
Laboratory</SPAN></FONT></FONT></DIV>
<DIV><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"></SPAN></FONT> </DIV><FONT class=Apple-style-span
size=4><SPAN class=Apple-style-span style="FONT-SIZE: 14px"><BR></SPAN></FONT>
<P><FONT class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">-- <BR>Scanned by iCritical. </SPAN></FONT></P><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><BR></SPAN></FONT></DIV><SPAN><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px"><ESGF_version_structure.odt></SPAN></FONT></SPAN><FONT
class=Apple-style-span size=4><SPAN class=Apple-style-span
style="FONT-SIZE: 14px">_______________________________________________<BR>GO-ESSP-TECH
mailing list<BR><A
href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</A><BR>http://*mailman.ucar.edu/mailman/listinfo/go-essp-tech<BR></SPAN></FONT></BLOCKQUOTE></DIV><BR></DIV>
<br><p>--
<BR>Scanned by iCritical.
</p>
<br></BODY></HTML>