[Go-essp-tech] mindmap from "ESG monday"; first telco
Bryan Lawrence
bryan.lawrence at stfc.ac.uk
Sun Oct 11 04:41:11 MDT 2009
Hi Folks
Attached is the mindmap I was developing last thing on Monday. I've slightly tidied it up, but it's otherwise unchanged, and it's clearly just a snapshot full of innaccuracies. Obviously over the next week we'll try and turn it into docs and wiki pages etc and merge it with Anne's notes and make sure our informaiton is accurate.
I believe we have committed many of us to a schedule of targeted telecons to fit into a slot at 4pm UT on Tuesdays (which will rattle round in our local times as we all go through daylight saving changes over the next few weeks). These telcos ought to be limited to only those who are directly interested in the "topic of the day", to one hour, and notes from each will be produced for those of us who wont be able to attend.
On Monday last we thought we might begin them this week, but my feeling is that many of us will have hardly been back in the office ... and we should probably start them on the 20th of October, which gives us a bit of time to get some skeleton documents ready for the first one which will be on the data node. It also gives us time to work up the schedule of the other meetings and get it on a wiki etc.
(European Metaforians note that 4pm UT will clash with the metafor meeting on the 20th of October, but not the week later when I think Europe will have gone off daylight saving. )
Cheers
Bryan
--
Bryan Lawrence
Director of Environmental Archival and Associated Research
(NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
STFC, Rutherford Appleton Laboratory
Phone +44 1235 445012; Fax ... 5848;
Web: home.badc.rl.ac.uk/lawrence
-------------- next part --------------
<map version="0.9.0">
<!-- To view this file, download free mind mapping software FreeMind from http://freemind.sourceforge.net -->
<node CREATED="1254749711295" ID="ID_1183817409" MODIFIED="1254749727993" TEXT="ESG Federation">
<node CREATED="1254749770560" ID="ID_1302380660" MODIFIED="1254749779867" POSITION="left" TEXT="Versioning"/>
<node CREATED="1254749788649" FOLDED="true" ID="ID_953353051" MODIFIED="1255256448635" POSITION="left" TEXT="Gateways">
<node CREATED="1254750264052" ID="ID_477219840" MODIFIED="1254750266598" TEXT="Working Now">
<node CREATED="1254750269477" ID="ID_1884319180" MODIFIED="1254750272748" TEXT="Registration Works"/>
<node CREATED="1254749794729" ID="ID_224540214" MODIFIED="1254750355712" TEXT="Access Control">
<node CREATED="1254750291101" ID="ID_387264923" MODIFIED="1254750298352" TEXT="Given a token to allow download"/>
<node CREATED="1254750361097" ID="ID_562969616" MODIFIED="1255255943189" TEXT="One time use tokens to pass to services">
<node CREATED="1254750762160" ID="ID_1157937222" MODIFIED="1254750774336" TEXT="can't use wget scripts from one site at another"/>
<node CREATED="1255255947991" ID="ID_1876398971" MODIFIED="1255255962569" TEXT="better to move to filter based system"/>
</node>
</node>
</node>
<node CREATED="1254751225123" ID="ID_448675078" MODIFIED="1254751227377" TEXT="Missing Stuff">
<node CREATED="1254751229667" ID="ID_1579979576" MODIFIED="1254751234031" TEXT="Harvesting the version contrl"/>
<node CREATED="1254751236323" ID="ID_1073584638" MODIFIED="1254751269300" TEXT="Using the versoin control in the UI"/>
<node CREATED="1254751289119" ID="ID_1466129981" MODIFIED="1255256205479" TEXT="Packaging and DOCs need lots of work"/>
<node CREATED="1254751339329" ID="ID_553089502" MODIFIED="1254751342653" TEXT="beta testing"/>
<node CREATED="1254751387908" ID="ID_732428658" MODIFIED="1254751391643" TEXT="Questionnaire Input"/>
<node CREATED="1254751364595" ID="ID_1055345318" MODIFIED="1254751369764" TEXT="Data Pipelining">
<node CREATED="1254751374981" ID="ID_345718200" MODIFIED="1254751410214" TEXT="How is the data and gridspec and CIM being ingested"/>
</node>
<node CREATED="1254751419974" ID="ID_173147649" MODIFIED="1254751426583" TEXT="OpenID Secure Whitelisting"/>
<node CREATED="1254751431214" ID="ID_891289876" MODIFIED="1254751434444" TEXT="Attribute Service">
<node CREATED="1254751438407" ID="ID_440864394" MODIFIED="1254751441470" TEXT="whitelisting"/>
</node>
</node>
<node CREATED="1254751250508" ID="ID_6111740" MODIFIED="1254751258222" TEXT="Deploying outside ESG-CET"/>
</node>
<node CREATED="1254751306983" FOLDED="true" ID="ID_917871349" MODIFIED="1255256511564" POSITION="right" TEXT="Issues">
<node CREATED="1254750023694" ID="ID_281494480" MODIFIED="1254750061622" TEXT="Operational System @ NCAR"/>
<node CREATED="1255256474614" ID="ID_824089467" MODIFIED="1255256487437" TEXT="When is ESG "running"?"/>
<node CREATED="1255256492764" ID="ID_1134363370" MODIFIED="1255256509420" TEXT="What does data "availability" mean? To whom and when?"/>
</node>
<node CREATED="1254751558438" FOLDED="true" ID="ID_1310693633" MODIFIED="1255256449737" POSITION="left" TEXT="Controlled Vocabs">
<node CREATED="1254751564502" ID="ID_479284039" MODIFIED="1254751567413" TEXT="What is core?"/>
<node CREATED="1254751582615" ID="ID_1346713731" MODIFIED="1254751589411" TEXT="Controlled Vocabs for all terms in DRS"/>
</node>
<node CREATED="1254752193218" FOLDED="true" ID="ID_582622959" MODIFIED="1255256454034" POSITION="right" TEXT="Current ESG Cycle">
<node CREATED="1254752563599" ID="ID_980483037" MODIFIED="1254752568718" TEXT="Oct09">
<node CREATED="1254752608017" ID="ID_1145753583" MODIFIED="1254752611743" TEXT="V0.X">
<node CREATED="1254752571588" ID="ID_536576591" MODIFIED="1254752577274" TEXT="Data Node versioning API added"/>
<node CREATED="1254752598009" ID="ID_1785603710" MODIFIED="1254752604827" TEXT=""NCAR" publishing requirements"/>
</node>
</node>
<node CREATED="1254752259839" ID="ID_1018930559" MODIFIED="1254752264304" TEXT="Nov-Jan">
<node CREATED="1254752267214" ID="ID_349779812" MODIFIED="1254752481787" TEXT="V1.0 gateway beta running at NCAR&PCDMI">
<node CREATED="1254752282191" ID="ID_1585165754" MODIFIED="1254752294141" TEXT="publishing new data into the V1.0 gateway"/>
<node CREATED="1254752376708" ID="ID_585301869" MODIFIED="1254752380128" TEXT="all current data visible">
<node CREATED="1254752485570" ID="ID_112466031" MODIFIED="1254752487354" TEXT="CMIP3"/>
</node>
<node CREATED="1254752386765" ID="ID_65456060" MODIFIED="1254752391585" TEXT="run the old system in parallel"/>
</node>
<node CREATED="1254752297864" ID="ID_1382077858" MODIFIED="1254752305188" TEXT="remove the old system at the end of that period"/>
<node CREATED="1254752354283" ID="ID_1345194184" MODIFIED="1254752364186" TEXT="Developing V1.1 including version control in parallel"/>
</node>
<node CREATED="1254752330922" ID="ID_1578871493" MODIFIED="1254752336275" TEXT="Feb ">
<node CREATED="1254752338923" ID="ID_191575740" MODIFIED="1254752346551" TEXT="V1.1 including Version control deployed"/>
</node>
<node CREATED="1254752690646" FOLDED="true" ID="ID_164959474" MODIFIED="1255256221492" TEXT="Replication Dependencies">
<node CREATED="1254752699262" ID="ID_661962515" MODIFIED="1254752728156" TEXT="Bulk Data Movement Client available as a command line tool end of October">
<node CREATED="1254752731802" ID="ID_111666924" MODIFIED="1254752737683" TEXT="Need a client for this, plus two months"/>
</node>
<node CREATED="1254752803486" ID="ID_1806382857" MODIFIED="1254752810232" TEXT="Need an authorisation token service ">
<node CREATED="1254752814806" ID="ID_821862668" MODIFIED="1254752825367" TEXT="need to deploy at PCMDI"/>
<node CREATED="1254752832258" ID="ID_1400756882" MODIFIED="1254752840012" TEXT="provides download tokens"/>
</node>
<node CREATED="1254752847034" ID="ID_568242753" MODIFIED="1254752860138" TEXT="need to flag replicas in esg publication">
<node CREATED="1254753077359" ID="ID_1825846388" MODIFIED="1254753091065" TEXT="depends on DRS mods to support filename version"/>
<node CREATED="1254753135345" ID="ID_821562075" MODIFIED="1254753179182" TEXT="We've got file level replication, what's the equivalent of the DRS for non-CMIP5 (i.e. minimum defn of atomic dataset)"/>
</node>
<node CREATED="1254752864337" ID="ID_990283318" MODIFIED="1254752874464" TEXT="need to use those flags in the gateway catalog and UI"/>
<node CREATED="1254753239967" ID="ID_472806754" MODIFIED="1254753248283" TEXT="Dean has phyiscal disks planned for dec/jan">
<node CREATED="1254753278492" ID="ID_744765834" MODIFIED="1254753280450" TEXT="USB2"/>
</node>
</node>
<node CREATED="1254753369381" ID="ID_1088017386" MODIFIED="1254753375816" TEXT="Means the ESG federation needs to be working"/>
<node CREATED="1254754105175" ID="ID_1209365001" MODIFIED="1254754117728" TEXT="Draft Document for Data Node will be out in October"/>
</node>
<node CREATED="1254749783385" FOLDED="true" ID="ID_14095732" MODIFIED="1255256450700" POSITION="left" TEXT="Data Nodes">
<node CREATED="1254749801337" ID="ID_1901490868" MODIFIED="1254749804007" TEXT="Access Control"/>
<node CREATED="1254749806218" ID="ID_816142713" MODIFIED="1254749819213" TEXT="ESG Publisher">
<node CREATED="1254749827163" ID="ID_679944639" MODIFIED="1254750563146" TEXT="Tredds "/>
<node CREATED="1254749832083" ID="ID_1761797585" MODIFIED="1255256150144" TEXT="Versioning Support underway right now"/>
<node CREATED="1255256151882" ID="ID_1700478148" MODIFIED="1255256163432" TEXT="Needs checkpointing and inventories for replication">
<node CREATED="1255256169978" ID="ID_409049079" MODIFIED="1255256179738" TEXT="much of that might be in there, we need to understand it"/>
</node>
</node>
<node CREATED="1254749944626" ID="ID_421415680" MODIFIED="1254749950509" TEXT="Metrics Services Deployed"/>
<node CREATED="1254750130149" ID="ID_1428602094" MODIFIED="1255255971783" TEXT="Configurable Options">
<node CREATED="1254749963947" ID="ID_1736728349" MODIFIED="1254749969024" TEXT="Working Now">
<node CREATED="1254749895479" ID="ID_958596531" MODIFIED="1254749897905" TEXT="SRM Server"/>
<node CREATED="1254749884014" ID="ID_1546271158" MODIFIED="1254749889532" TEXT="GridFTP server">
<node CREATED="1254750828796" ID="ID_1605655997" MODIFIED="1254750843782" TEXT="instrumented to use their single-use tokens"/>
<node CREATED="1255255983623" ID="ID_706283638" MODIFIED="1255255992217" TEXT="Ideally we instrument to use filters"/>
</node>
</node>
<node CREATED="1254749822955" ID="ID_1998937798" MODIFIED="1254749861376" TEXT="LAS">
<node CREATED="1255255974303" ID="ID_1043053319" MODIFIED="1255255979355" TEXT="not yet in deployed version"/>
</node>
<node CREATED="1254750451903" ID="ID_1832503905" MODIFIED="1254750461990" TEXT="Wget">
<node CREATED="1254750466423" ID="ID_898051863" MODIFIED="1254750474985" TEXT="Using single-use tokens"/>
<node CREATED="1255255999873" ID="ID_476945441" MODIFIED="1255256019752" TEXT="Phil has instrumented to use ESG security directly provided server understands it"/>
</node>
<node CREATED="1254750878359" FOLDED="true" ID="ID_967036904" MODIFIED="1254750927529" TEXT="Data Mover LIte">
<node CREATED="1254750910710" ID="ID_1184774320" MODIFIED="1254750925650" TEXT="problematic ... with tokens you have to use DML V3"/>
</node>
<node CREATED="1254750869646" ID="ID_1291134474" MODIFIED="1254750874250" TEXT="Not going to be there">
<node CREATED="1254750853813" ID="ID_541399920" MODIFIED="1254750856554" TEXT="Opendap"/>
</node>
</node>
<node CREATED="1254751001078" ID="ID_1047274087" MODIFIED="1254751004344" TEXT="Key things Missing">
<node CREATED="1254751007103" ID="ID_1212050172" MODIFIED="1254751026186" TEXT="Metrics"/>
<node CREATED="1254751028802" ID="ID_137581790" MODIFIED="1254751030725" TEXT="Versioning"/>
<node CREATED="1254751032897" ID="ID_1165705608" MODIFIED="1254751073520" TEXT="Not all Applications are Secured"/>
<node CREATED="1254751093092" ID="ID_740697738" MODIFIED="1254751121856" TEXT="Notification: how does a data provider tell folk I've changed my data"/>
<node CREATED="1254751127534" ID="ID_1447097694" MODIFIED="1254751216443" TEXT="Notification: how to signal data available for replication"/>
<node CREATED="1254752973062" ID="ID_1262623664" MODIFIED="1254752984575" TEXT="Inventory for replication"/>
</node>
<node CREATED="1255256059236" ID="ID_779877860" MODIFIED="1255256060936" TEXT="TDS">
<node CREATED="1254750565565" ID="ID_304014718" MODIFIED="1254750595536" TEXT="Is the TDS which can use single-use token ready yet?"/>
<node CREATED="1255256066024" ID="ID_308433132" MODIFIED="1255256073826" TEXT="Can we filter the security instead?"/>
</node>
<node CREATED="1255256077718" ID="ID_824774661" MODIFIED="1255256080366" TEXT="Installation">
<node CREATED="1254750976884" ID="ID_1083505349" MODIFIED="1255256092686" TEXT="Gavin's on top of scripting"/>
<node CREATED="1255256095148" ID="ID_1716693853" MODIFIED="1255256121465" TEXT="Better yet the open virtual machine approach?"/>
</node>
</node>
<node CREATED="1254751319328" FOLDED="true" ID="ID_1110374253" MODIFIED="1255257549238" POSITION="right" TEXT="Operational ESG Federation">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
 What's needed to call this operational?
</p>
</body>
</html>
</richcontent>
<node CREATED="1254751448375" ID="ID_1372411706" MODIFIED="1254751457513" TEXT="seen data published and replicated">
<node CREATED="1254751546933" ID="ID_1480000615" MODIFIED="1254751550473" TEXT="replicated to one site"/>
<node CREATED="1254751471361" ID="ID_443632183" MODIFIED="1254751476222" TEXT="to all the "core" copy sites"/>
</node>
<node CREATED="1254751466304" ID="ID_1101751454" MODIFIED="1254751769739" TEXT="meta publishing ">
<node CREATED="1254751772155" ID="ID_1650044065" MODIFIED="1254751786006" TEXT="packaging up the metadata from mulitple sources and getting it to the gateways"/>
<node CREATED="1254751839534" ID="ID_1926592550" MODIFIED="1254751852856" TEXT="data can be published by nodes, but not by core until the questionnaire is filled in."/>
</node>
</node>
<node CREATED="1254749730951" ID="ID_923590180" MODIFIED="1254749734587" POSITION="left" TEXT="Replication">
<node CREATED="1254749737430" ID="ID_591668535" MODIFIED="1254749740066" TEXT="Software"/>
<node CREATED="1254749743550" ID="ID_941746121" MODIFIED="1254749753698" TEXT="Operational To PCMDI"/>
<node CREATED="1254749756527" ID="ID_862076370" MODIFIED="1254749762261" TEXT="Operational to Core Sites"/>
</node>
<node CREATED="1254749875799" ID="ID_897561871" MODIFIED="1254749881399" POSITION="left" TEXT="Monitoring, Metrics"/>
<node CREATED="1254755503063" FOLDED="true" ID="ID_1986115285" MODIFIED="1255257582242" POSITION="right" TEXT="Next steps">
<node CREATED="1255256232013" FOLDED="true" ID="ID_1384558789" MODIFIED="1255257581399" TEXT="Documents Needed ASAP">
<node CREATED="1255256239862" ID="ID_1705047885" MODIFIED="1255256243943" TEXT="Replication">
<node CREATED="1255256246502" ID="ID_1062039536" MODIFIED="1255256250807" TEXT="ESG Procedural"/>
<node CREATED="1255256253702" ID="ID_1029621080" MODIFIED="1255256257885" TEXT="ESG Software Description"/>
</node>
<node CREATED="1255256260407" ID="ID_1421666097" MODIFIED="1255256262600" TEXT="Data Node">
<node CREATED="1255256265199" ID="ID_19809593" MODIFIED="1255256271567" TEXT="Installation "/>
<node CREATED="1255256282528" ID="ID_1129503462" MODIFIED="1255256289168" TEXT="ESG Procedural">
<node CREATED="1255256300649" ID="ID_1617583164" MODIFIED="1255256300649" TEXT=""/>
</node>
</node>
<node CREATED="1255256294074" ID="ID_464732007" MODIFIED="1255256296807" TEXT="Timelines"/>
<node CREATED="1255256306993" ID="ID_1196423988" MODIFIED="1255256411040" TEXT="Resolution of conflict between immediate availability and satisfactory metadata"/>
<node CREATED="1255256417726" ID="ID_1684768847" MODIFIED="1255256425107" TEXT="DOI plans"/>
</node>
<node CREATED="1254755507839" FOLDED="true" ID="ID_1813879125" MODIFIED="1255257577375" TEXT="targeted telcons, focus groups">
<node CREATED="1254755528480" ID="ID_714795590" MODIFIED="1254755530632" TEXT="data node">
<node CREATED="1254755789191" ID="ID_1476593395" MODIFIED="1254755796409" TEXT="we need the "doc""/>
<node CREATED="1254755799272" ID="ID_1272950121" MODIFIED="1254755803205" TEXT="need a meeting soon">
<node CREATED="1254755808025" ID="ID_1935713769" MODIFIED="1254755812500" TEXT="next week"/>
</node>
</node>
<node CREATED="1254755533689" ID="ID_1846833526" MODIFIED="1254755540192" TEXT="gateway installatoin">
<node CREATED="1254755825049" ID="ID_1149537626" MODIFIED="1254755831517" TEXT="first or second week of november"/>
</node>
<node CREATED="1254755542913" ID="ID_1847603799" MODIFIED="1254755546910" TEXT="operational procedures">
<node CREATED="1254755836458" ID="ID_34096886" MODIFIED="1254755846287" TEXT="third or fourth week of november"/>
</node>
<node CREATED="1254755549481" ID="ID_65171201" MODIFIED="1254755553396" TEXT="replication">
<node CREATED="1254755853307" ID="ID_1484492717" MODIFIED="1254755866412" TEXT="early november"/>
</node>
<node CREATED="1254755555986" ID="ID_1238487329" MODIFIED="1254755557872" TEXT="versioning">
<node CREATED="1254755900661" ID="ID_357808423" MODIFIED="1254755903692" TEXT="early november"/>
</node>
<node CREATED="1254755618997" ID="ID_484538009" MODIFIED="1254755621293" TEXT="overarching">
<node CREATED="1254755625814" ID="ID_789870553" MODIFIED="1254755631344" TEXT="federation level">
<node CREATED="1254755906830" ID="ID_1569009418" MODIFIED="1254755918136" TEXT="december 1"/>
</node>
</node>
<node CREATED="1254755651831" ID="ID_1494556054" MODIFIED="1254755656645" TEXT="quality control">
<node CREATED="1254755952272" ID="ID_1656873278" MODIFIED="1254755959582" TEXT="what do we think this means">
<node CREATED="1254755962313" ID="ID_1804296796" MODIFIED="1254755967326" TEXT="Don, Michael, Dean, Bryan"/>
</node>
<node CREATED="1254755973546" ID="ID_1645304500" MODIFIED="1254755989918" TEXT="report back to federation meeting december 1"/>
</node>
<node CREATED="1254755667616" ID="ID_1270015689" MODIFIED="1254755687771" TEXT="milestones">
<node CREATED="1254755677557" ID="ID_1564774889" MODIFIED="1254755681635" TEXT="need a clear schedule"/>
<node CREATED="1254755718379" ID="ID_1884495089" MODIFIED="1254755719837" TEXT="timeline"/>
<node CREATED="1254755722819" ID="ID_1312126356" MODIFIED="1254755742156" TEXT="get a strawman (Dean, Don, bryan,michael)"/>
<node CREATED="1254755732960" ID="ID_356511051" MODIFIED="1254755735880" TEXT="regular reschedule"/>
<node CREATED="1254755770598" ID="ID_1398162920" MODIFIED="1254755779167" TEXT="priority meeting (2-3 weeks)"/>
</node>
<node CREATED="1254756043310" ID="ID_160985570" MODIFIED="1254756051829" TEXT="pipelining, metadata into the gateway discussions">
<node CREATED="1254756067521" ID="ID_197434882" MODIFIED="1254756070878" TEXT="december"/>
<node CREATED="1254756088026" ID="ID_933041518" MODIFIED="1254756095608" TEXT="curator/meta4 to handle before then"/>
</node>
<node CREATED="1254756224248" ID="ID_634551549" MODIFIED="1254756233554" TEXT="4pm tuesdays if we can"/>
</node>
<node CREATED="1254752560423" FOLDED="true" ID="ID_1244737561" MODIFIED="1255257579292" TEXT="Contact Methods">
<node CREATED="1254754500151" ID="ID_897569612" MODIFIED="1254754504783" TEXT="Gateway Mailing List">
<node CREATED="1254755145866" ID="ID_1682885419" MODIFIED="1254755152164" TEXT="to be used from Nov"/>
<node CREATED="1254755004914" ID="ID_552880977" MODIFIED="1254755014143" TEXT="Gateway installation playground in November"/>
<node CREATED="1254754659983" ID="ID_130820507" MODIFIED="1254754668910" TEXT="New People In Charge of The Gateway">
<node CREATED="1254754674223" ID="ID_209920883" MODIFIED="1254754679427" TEXT="need to wait til they're up to speed"/>
<node CREATED="1254754726250" ID="ID_1910370429" MODIFIED="1254754738034" TEXT="How do you add a new data node in the federation"/>
<node CREATED="1254754740643" ID="ID_749453954" MODIFIED="1254754747461" TEXT="How do you bring up a new gateway in the federation"/>
</node>
</node>
<node CREATED="1254754507518" ID="ID_40635640" MODIFIED="1254754519810" TEXT="Data Node Mailing List">
<node CREATED="1254754718482" ID="ID_1013764776" MODIFIED="1254754723187" TEXT="How do you add a new data node"/>
</node>
<node CREATED="1254754522631" ID="ID_412274488" MODIFIED="1254754528014" TEXT="Wiki Pages">
<node CREATED="1254754530455" ID="ID_90959193" MODIFIED="1254754532029" TEXT="Gateway"/>
</node>
<node CREATED="1254755230327" ID="ID_266815503" MODIFIED="1254755249665" TEXT="go-essp-tech list for cmip5"/>
</node>
</node>
</node>
</map>
More information about the GO-ESSP-TECH
mailing list