<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Hi</div><div><br></div>... a news cast capability triggered manually ...<div>How about ESG-Publisher posting to an ESG-F data twitter feed that can in turn be queried via the twitter.api (<a href="http://dev.twitter.com/doc">http://dev.twitter.com/doc</a>, <a href="http://dev.twitter.com/doc/get/search">http://dev.twitter.com/doc/get/search</a>) </div><div><br></div><div>Perhaps not a serious suggestion but abit of fun ahead of GO-ESSP.</div><div><div><br></div><div>Mark</div><div><br></div><div>PS - Note the 16 character limit on hash tags</div><div><br></div><div>PS - @Phil Kershaw : <a href="http://dev.twitter.com/pages/auth_overview">http://dev.twitter.com/pages/auth_overview</a></div><div><br></div><div><br></div><div><br></div><div><div><div>On 28 Apr 2011, at 14:55, Cinquini, Luca (3880) wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi,<br><span class="Apple-tab-span" style="white-space:pre"> </span>FYI we are indeed looking at a distributed search architecture for ESGF, and since each dataset that enters the system has a "last update" time stamp, it already comes with the possibility of querying for anything new... but again, at a level of granularity that is probably too detailed for what we want the user to have. Maybe a news cast capability triggered manually would be more like what is needed here. This would be a great topics of discussion at the workshop.<br>thanks, Luca<br><br>On Apr 28, 2011, at 5:50 AM, <<a href="mailto:philip.kershaw@stfc.ac.uk">philip.kershaw@stfc.ac.uk</a>> wrote:<br><br><blockquote type="cite">Another option would be to proxy tomcat through Apache as JPL have done then the Pylons app could run via mod_wsgi.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers,<br></blockquote><blockquote type="cite">Phil<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">From: <stephen.pascoe@stfc.ac.uk<<a href="mailto:stephen.pascoe@stfc.ac.uk">mailto:stephen.pascoe@stfc.ac.uk</a>>><br></blockquote><blockquote type="cite">Date: Thu, 28 Apr 2011 08:49:43 +0000<br></blockquote><blockquote type="cite">To: <momipsl@ipsl.jussieu.fr<<a href="mailto:momipsl@ipsl.jussieu.fr">mailto:momipsl@ipsl.jussieu.fr</a>>>, <drach1@llnl.gov<<a href="mailto:drach1@llnl.gov">mailto:drach1@llnl.gov</a>>><br></blockquote><blockquote type="cite">Cc: <esg-gateway-dev@earthsystemgrid.org<<a href="mailto:esg-gateway-dev@earthsystemgrid.org">mailto:esg-gateway-dev@earthsystemgrid.org</a>>>, <go-essp-tech@ucar.edu<<a href="mailto:go-essp-tech@ucar.edu">mailto:go-essp-tech@ucar.edu</a>>>, <sebastien.denvil@ipsl.jussieu.fr<<a href="mailto:sebastien.denvil@ipsl.jussieu.fr">mailto:sebastien.denvil@ipsl.jussieu.fr</a>>><br></blockquote><blockquote type="cite">Subject: Re: [esg-gateway-dev] [Go-essp-tech] Status of the CMIP5 Archive<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Hi All,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Similarly we have a web app which exposes publishing information in the datanode database for use between us and the MetOffice. It uses the same SQLAlchemy model as esgcet with a few extra tables. I don't think it would take me long to create a view that displays recently published datasets as an atom feed.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">However, there are complications. This is a Pylons app which we deploy on a separate machine to the datanode or gateway. If we were to put it on the datanode it would have to listen on a separate port to tomcat. Or we could implement something similar in Java and put it in the datanode's tomcat.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Also, Luca and Mark are right. To be useful we would need to aggregate the information from each datanode and provide less fine-grained info like "Which experiments have just been published". It begins to look a lot more like a query service with output as Atom/RSS.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">S.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">---<br></blockquote><blockquote type="cite">Stephen Pascoe +44 (0)1235 445980<br></blockquote><blockquote type="cite">Centre of Environmental Data Archival<br></blockquote><blockquote type="cite">STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">From: go-essp-tech-bounces@ucar.edu<<a href="mailto:go-essp-tech-bounces@ucar.edu">mailto:go-essp-tech-bounces@ucar.edu</a>> [mailto:go-essp-tech-bounces@ucar.edu] On Behalf Of Mark Morgan<br></blockquote><blockquote type="cite">Sent: 28 April 2011 11:11<br></blockquote><blockquote type="cite">To: Drach, Bob<br></blockquote><blockquote type="cite">Cc: Sébastien Denvil; go-essp-tech@ucar.edu<<a href="mailto:go-essp-tech@ucar.edu">mailto:go-essp-tech@ucar.edu</a>>; esg-gateway-dev@earthsystemgrid.org<<a href="mailto:esg-gateway-dev@earthsystemgrid.org">mailto:esg-gateway-dev@earthsystemgrid.org</a>><br></blockquote><blockquote type="cite">Subject: Re: [Go-essp-tech] Status of the CMIP5 Archive<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Bob / Bryan<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Ontop of the possibilty of the DataNode exposing ESG-Publisher AtomPub HTTP endpoints, may I also add to the mix the possibility of ESG-Publisher search web services. Such web services would in turn permit the development of a ESG distributed search broker, i.e. an aggregator of search results pulled from multiple data nodes.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">At IPSL we have developed a portal that launches an overnight batch job to harvest & aggregate meta-data derived from the THREDDS catalogs published at each of our data nodes. We could certainly use the ESG-Publisher AtomPub feeds to optimise the synchronisation of the aggregated meta-data.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">However ultimately all paths lead to distributed search as in a few years we will have several millions of files/variables to search against and there are limits to what can be acheived with a solution based upon aggregation. Hence the ESG-Publisher search web services and associated distributed search broker are of real interest.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Regards<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Mark<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On 27 Apr 2011, at 23:54, Drach, Bob wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Hi Brian,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Excellent suggestion. Anyone familiar with setting up an RSS feed?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--Bob<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On 4/27/11 12:58 PM, "Bryan Lawrence" <bryan.lawrence@stfc.ac.uk<<a href="mailto:bryan.lawrence@stfc.ac.uk">mailto:bryan.lawrence@stfc.ac.uk</a>>> wrote:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">hi Bob<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I wonder how hard it would be to produce a data node feed (or a TDS<br></blockquote><blockquote type="cite">feed) of datasets published/revised as part of the publication step?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">It'd then be relatively easy to parse that for a "new items" page ...<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers<br></blockquote><blockquote type="cite">Bryan<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I'm happy to post a list of publication events if that would be<br></blockquote><blockquote type="cite">useful. But like you I would see this as a temporary solution until<br></blockquote><blockquote type="cite">some sort of registry solution could be devised (famous last words<br></blockquote><blockquote type="cite">...). Also I can't really commit to keeping such a list up-to-date<br></blockquote><blockquote type="cite">when vacation etc. intervenes.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I'd be curious to hear if others think this is a good idea as well.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--Bob<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On 4/27/11 3:30 AM, "Estanislao Gonzalez"<br></blockquote><blockquote type="cite"><estanislao.gonzalez@zmaw.de<<a href="mailto:estanislao.gonzalez@zmaw.de">mailto:estanislao.gonzalez@zmaw.de</a>>><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">wrote:<br></blockquote><blockquote type="cite">Hi Sébastien,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">indeed this is a great idea, but changing this manually on every<br></blockquote><blockquote type="cite">gateway is not practical at all.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I've already proposed moving this to a central registry of some<br></blockquote><blockquote type="cite">kind, but considering that the current registry, which is<br></blockquote><blockquote type="cite">essential, is not ready I'd suggest a quick and dirty procedure:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">* Bob (I know you love this :-), could you put this info in a file<br></blockquote><blockquote type="cite">publicly accessible at pcmdi3? (plain txt file, no headers,<br></blockquote><blockquote type="cite">nothing) * Gateway team: could you give us a one line ajax command<br></blockquote><blockquote type="cite">(or anything similar, preferably from the client side) depending<br></blockquote><blockquote type="cite">on the current js libraries to insert this text where it should?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I think that'll do.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Thanks,<br></blockquote><blockquote type="cite">Estani<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Am 27.04.2011 11:53, schrieb Sébastien Denvil:<br></blockquote><blockquote type="cite">Hi Bob, Stephen, Estanislao<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I noticed that pcmdi gateway have a notice on the homepage listing<br></blockquote><blockquote type="cite">new available datasets.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Latest notice being : "BCC datasets will be available at the end<br></blockquote><blockquote type="cite">of April."<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Bob, could add this to your list:<br></blockquote><blockquote type="cite">"IPSL-CM5A-LR piControl and historical datasets available"<br></blockquote><blockquote type="cite">They are open to CMIP5-research role since 20th of April.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Stephen, Estanislao I think it could be a good idea to duplicate<br></blockquote><blockquote type="cite">this notice on the other gateway to help people identifying which<br></blockquote><blockquote type="cite">datasets are accessible.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Cheers.<br></blockquote><blockquote type="cite">Sébastien<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">GO-ESSP-TECH mailing list<br></blockquote><blockquote type="cite">GO-ESSP-TECH@ucar.edu<<a href="mailto:GO-ESSP-TECH@ucar.edu">mailto:GO-ESSP-TECH@ucar.edu</a>><br></blockquote><blockquote type="cite"><a href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite">Bryan Lawrence<br></blockquote><blockquote type="cite">Director of Environmental Archival and Associated Research<br></blockquote><blockquote type="cite">(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)<br></blockquote><blockquote type="cite">STFC, Rutherford Appleton Laboratory<br></blockquote><blockquote type="cite">Phone +44 1235 445012; Fax ... 5848;<br></blockquote><blockquote type="cite">Web: home.badc.rl.ac.uk/lawrence<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">GO-ESSP-TECH mailing list<br></blockquote><blockquote type="cite">GO-ESSP-TECH@ucar.edu<<a href="mailto:GO-ESSP-TECH@ucar.edu">mailto:GO-ESSP-TECH@ucar.edu</a>><br></blockquote><blockquote type="cite"><a href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">---------------------------------------------------<br></blockquote><blockquote type="cite">Mark Morgan<br></blockquote><blockquote type="cite">Software Architect / Engineer<br></blockquote><blockquote type="cite">Institut Pierre Simon Laplace (IPSL),<br></blockquote><blockquote type="cite">Université Pierre Marie Curie,<br></blockquote><blockquote type="cite">4 Place Jussieu,<br></blockquote><blockquote type="cite">Tour 45-55, Salle #207,<br></blockquote><blockquote type="cite">Paris 75005<br></blockquote><blockquote type="cite">France.<br></blockquote><blockquote type="cite">Tel : +33 (0) 1 44 27 49 10<br></blockquote><blockquote type="cite">Email: momipsl@ipsl.jussieu.fr<<a href="mailto:momipsl@ipsl.jussieu.fr">mailto:momipsl@ipsl.jussieu.fr</a>><br></blockquote><blockquote type="cite">---------------------------------------------------<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">--<br></blockquote><blockquote type="cite">Scanned by iCritical.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">_______________________________________________ esg-gateway-dev mailing list esg-gateway-dev@mailman.earthsystemgrid.org<<a href="mailto:esg-gateway-dev@mailman.earthsystemgrid.org">mailto:esg-gateway-dev@mailman.earthsystemgrid.org</a>> <a href="http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev">http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev</a><br></blockquote><blockquote type="cite">-- <br></blockquote><blockquote type="cite">Scanned by iCritical.<br></blockquote><blockquote type="cite">_______________________________________________<br></blockquote><blockquote type="cite">esg-gateway-dev mailing list<br></blockquote><blockquote type="cite"><a href="mailto:esg-gateway-dev@mailman.earthsystemgrid.org">esg-gateway-dev@mailman.earthsystemgrid.org</a><br></blockquote><blockquote type="cite"><a href="http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev">http://mailman.earthsystemgrid.org/mailman/listinfo/esg-gateway-dev</a><br></blockquote><br><br></div></blockquote></div><br><div>
<div>---------------------------------------------------</div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Mark Morgan</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Software Architect / Engineer</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Institut Pierre Simon Laplace (IPSL),</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Université Pierre Marie Curie,</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">4 Place Jussieu,</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Tour 45-55, Salle #207,</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Paris 75005</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">France.</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Tel : +33 (0) 1 44 27 49 10</span></div><div><span class="Apple-style-span" style="color: rgb(84, 0, 0); "></span><span class="Apple-style-span" style="color: rgb(84, 0, 0); ">Email: <a href="mailto:momipsl@ipsl.jussieu.fr">momipsl@ipsl.jussieu.fr</a></span></div><div>---------------------------------------------------</div><div><br></div><br class="Apple-interchange-newline">
</div>
<br></div></div></body></html>