<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi,<br>
<br>
just to be more precise: indeed I think security and CIM data
(meta-data from experiments and so on) should be kept away from the
catalogs.<br>
<br>
Still, Luca's idea of merging QC level info with the files itself
might be a valid idea. The difference is that QC is pretty much like
a "semantic" checksum for a file. AFAIK you cannot "downgrade" the
QC without altering the file, i.e. if the file is QC L2 and while
performing QC L3 checks an error is found, the file will either be
QC L3 approved (the modeler defines the "oddity" as "expected") or
it doesn't, which implies a "QC L2 passed; QC L3 failed" flag or the
retraction (and maybe re-publication) of it altogether.<br>
<br>
Well, that's at least why I think the QC flag is a little different
and it's *closely* related to the file. The only difference with the
checksum is IMHO that it takes more time to be determined (as well
as require other files for it's computation) and thus it's performed
in an out-of-band fashion.<br>
<br>
We need that QC flag somewhere... and it's far more important than
the rest of the CIM meta-data (getting back to Gavin's point about
CIM issues and differentiating it to this QC flag: yes, you can
still get to the file and download it without CIM data... but
without the QC flag you'll have no clue if you *really* want to rely
on this data).<br>
<br>
To be honest I can't understand why people download this data if
they *know* it might get corrected. Would you start writing a paper
on something that might be altogether wrong?... I suspect they don't
realize this.<br>
<br>
Anyway, my 2c...<br>
<br>
Thanks,<br>
Estani<br>
<br>
Am 24.05.2011 03:06, schrieb Gavin M. Bell:
<blockquote cite="mid:4DDB0479.2040904@llnl.gov" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Hi Luca, <br>
<br>
I think that the separation of concerns trumps the apparent
"simplicity". Though it is apparently easy to republish (I am not
sure I fully agree with that, at least not from the anecdotal
information I hear from folks)... it is unnecessary to publish if
we keep concerns separated.<br>
<br>
As Estani said, the publisher publishes and does basic mechanical
sanity checks on data. That should be the full extent of its
operation. As far as what is easy... one could 'easily' set up an
index over the CIM info and "join" on datasetid. This also
provides loose coupling. If the CIM system has issues, that just
means that when you look at your search results you won't see CIM
info, but you will still see the dataset and be able to fetch and
manipulate it and everything else. Also if the CIM changes it
doesn't affect the pubblisher or publishing in any way. Catalogs
should be viewed as "files" in the system... they essentially are
logical files (containing pointers to physical files).<br>
<br>
I am still not convinced by your arguments that fusing and
coupling these two semantically different aspects of the system so
tightly is the right long term architectural solution. It may be
good now, but it not as flexible later. We should leave open the
avenue for other meta-metadata to be imbued onto our system
ex-post-facto without much ado.<br>
<br>
my $0.02<br>
<br>
On 5/23/11 2:08 AM, <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:stephen.pascoe@stfc.ac.uk">stephen.pascoe@stfc.ac.uk</a>
wrote:
<blockquote
cite="mid:4C353E6E4A08AE4792B350DAA392B521196329@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<pre wrap="">I'm with Estani on this. Authorisation decisions are best decoupled from the application where possible. Phil is on leave today but I'm sure he'd say the same thing and give much more detailed reasoning.
I think the catalogue already mixes slightly too much information together: location-independent file metadata and location-specific service information. If we add access control it becomes too tightly coupled.
Stephen.
---
Stephen Pascoe +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
-----Original Message-----
From: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:go-essp-tech-bounces@ucar.edu">go-essp-tech-bounces@ucar.edu</a> [<a moz-do-not-send="true" class="moz-txt-link-freetext" href="mailto:go-essp-tech-bounces@ucar.edu">mailto:go-essp-tech-bounces@ucar.edu</a>] On Behalf Of Estanislao Gonzalez
Sent: 21 May 2011 09:30
To: Cinquini, Luca (3880)
Cc: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a>
Subject: Re: [Go-essp-tech] resolution on securing opendap aggregations via ESGF
Hi,
In my opinion we shouldn't encode the access restriction in the catalog
for these reasons:
1) Changing the access would involved re-publishing the files. (this
will be done for instance when QC L2 is reached CMIP5 Research -> CMIP5
Commercial). And think about what would happen if we want to change the
access restriction in a couple of years... we should publish everything
again, and that would involve quite some effort to understand the
procedure again...
2) I'm not sure of this, but I fear TDS security cannot handle multiple
roles. Right now you can publish to as many roles as required, and read
and write access is kept separately. This would involve extending the
TDS capabilities.
3) There could be potential inconsistencies if the authorization service
is detached from datanode (like with the gateway right now) and the
publisher alters the role but forgets to cascade the changes to the
authorizing service (which would proceed according to the last harvested
info)
4) And last but not least, I'm not sure we want to mix administration
with publication. The publisher should only care about making data
available, the administrator should organize this and be responsible for
the security.
So basically I don't agree :-) Although I do think, if required, we
could change "esg-user" for "esgf-controlled" if it's more intuitive.
My 2c anyways,
Estani
Am 20.05.2011 19:17, schrieb Cinquini, Luca (3880):
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
a few points again on the issue of securing opendap aggregations served by the TDS with ESGF filters:
o There's a new release of the ESGF security filters (esg-orp 1.1.2) that maps the TDS request URI to the dataset ID, and should solve this problem. You can experiment with the JPL test TDS server:
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://test-datanode.jpl.nasa.gov/thredds/catalog.html">http://test-datanode.jpl.nasa.gov/thredds/catalog.html</a>
where the AIRS dataset (and aggregations) is secured, the MLS is not.
o Now the data node authorization filter will correctly identify the aggregation as secured, and call the configured authorization service. Currently, the p2p Node authorization service can be configured to allow authorization based on URL matching, so it will work. The gateway authorization service will have to implement its own logic to establish authorization.
o Finally, I am wondering if we shouldn't change the way we encode authorization in thredds catalogs. Right now, we use restrictAccess="esg-user" for ALL collections, but should we consider about encoding the proper required access control attribute instead, for example restrictAccess="CMIP5 Research" ? Something to think about - there are prons and cons about this - it's all a question on wether the access control belongs in the catalog (and can be harvested for searching...) or not.
thanks, Luca
_______________________________________________
GO-ESSP-TECH mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Gavin M. Bell
--
"Never mistake a clear view for a short distance."
-Paul Saffo
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Estanislao Gonzalez
Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
Phone: +49 (40) 46 00 94-126
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:gonzalez@dkrz.de">gonzalez@dkrz.de</a> </pre>
</body>
</html>