[Go-essp-tech] GO-ESSP call Jan 10, 2012
Eric Nienhouse
ejn at ucar.edu
Mon Jan 9 16:53:38 MST 2012
Hi All,
We plan to hold a GO-ESSP tech call tomorrow (Jan 10, 2012.) Agenda
follows:
* Overview and demo of Gateway 2.0.
* Near and mid term technology roadmap for CMIP5 operations (open
discussion.)
We'll be using ReadyTalk web sharing for the Gateway 2.0 demo. There is
a brief registration page to join in. The link follows:
https://cc.readytalk.com/r/6icl9p8fqbey
Please access the telco number for the audio portion:
+01 (925) 424-8105 access code 305757#
*Telco details:
*16:00 BST, 17:00 CEST, 8:00 PDT, 9:00 MDT, 11:00 EDT.
Apologies - the previous email subject noted the telco is in "2011". It
is, of course, scheduled for Jan 10, 2012. Happy New Year :-)
Best regards,
-Eric
Eric Nienhouse wrote:
> Hi All,
>
> We plan to hold a GO-ESSP tech call tomorrow. Following is our
> tentative agenda:
>
> * Overview and demo of Gateway 2.0.
> * Near and mid term technology roadmap for CMIP5 operations (open
> discussion.)
>
> We'll be using ReadyTalk web sharing for the Gateway 2.0 demo. I'll
> send a follow-up email with connection details.
>
> Best regards,
>
> -Eric
>
> Eric Nienhouse wrote:
>
>> Hi Micheal,
>>
>> Thank you. I agree discussing the future path of the running CMIP5
>> archive and the ESGF efforts openly is extremely important as we all
>> plan and work to meet the needs of our user community. We should
>> devote the time necessary to discuss this topic (or at least begin to)
>> in our upcoming call on Jan 10th as you suggest.
>>
>> Best regards,
>>
>> -Eric
>>
>> Michael Lautenschlager wrote:
>>
>>> Dear Eric,
>>>
>>> I support your suggestion to postpone the gateway discussion to
>>> January 10th. Then most of us should be back from vacation between
>>> the years. But I suggest that we should plan for more than 30 minutes
>>> discussion on this topic because I think we should discuss the short
>>> term development for the running CMIP5 archive and should also start
>>> discussion on the mid term development within ESGF. Both aspects are
>>> important for us at DKRZ because we would like to contribute to ESGF,
>>> share work between ESGF partners and avoid double work.
>>>
>>> Best wishes, Michael
>>>
>>> ---------------
>>> Dr. Michael Lautenschlager
>>> Head of DKRZ Department Data Management
>>> Director World Data Center Climate
>>>
>>> German Climate Computing Centre (DKRZ)
>>> ADDRESS: Bundesstrasse 45a, D-20146 Hamburg, Germany
>>> PHONE: +4940-460094-118
>>> E-Mail: lautenschlager at dkrz.de
>>>
>>> URL: http://www.dkrz.de/
>>> http://www.wdc-climate.de/
>>>
>>>
>>> Geschäftsführer: Prof. Dr. Thomas Ludwig
>>> Sitz der Gesellschaft: Hamburg
>>> Amtsgericht Hamburg HRB 39784
>>>
>>>
>>> Am 20.12.2011 16:44, schrieb Eric Nienhouse:
>>>
>>>> Hi Dean, All,
>>>>
>>>> We'd like to offer a walk-through of the Gateway 2.0 once our GO-ESSP
>>>> call schedule resumes after the new year.
>>>>
>>>> I propose Tuesday Jan 10th for this topic.
>>>>
>>>> We could likely fit this into 30 minutes and have time to address other
>>>> topics if needed.
>>>>
>>>> How does that sound?
>>>>
>>>> Best regards,
>>>>
>>>> -Eric
>>>>
>>>> Williams, Dean N. wrote:
>>>>
>>>>> Hi Don,
>>>>>
>>>>> We have something already in the works for this coming week. With the
>>>>> holiday's coming in the EU and the US, perhaps it might not be best to
>>>>> have this the first week in January. Looking forward to seeing it.
>>>>>
>>>>> Best regards,
>>>>> Dean
>>>>>
>>>>> From: Don Middleton<don at ucar.edu<mailto:don at ucar.edu>>
>>>>> Date: Tue, 13 Dec 2011 07:54:57 -0800
>>>>> To: "Taylor, Karl Taylor"<taylor13 at llnl.gov
>>>>> <mailto:taylor13 at llnl.gov>>, Adams Jennifer<jma at cola.iges.org
>>>>> <mailto:jma at cola.iges.org>>
>>>>> Cc: "go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>"
>>>>> <go-essp-tech at ucar.edu<mailto:go-essp-tech at ucar.edu>>
>>>>> Subject: Re: [Go-essp-tech] Status of Gateway 2.0
>>>>>
>>>>> Karl, thanks for following up on this. Jennifer, Eric spoke correctly
>>>>> last week: at NCAR we are in the process of moving 2.0 into local
>>>>> production as a priority next step. I think this will be a big
>>>>> improvement for the scientific community, and I hope we can deploy it
>>>>> more broadly for the same reason.
>>>>>
>>>>> There was discussion at the end of last week on having a PI level
>>>>> meeting but I haven't heard any more on that so far. It would helpful
>>>>> to have some more open, community discussion of how we move forward;
>>>>> across the federation there are a lot of investments, stakeholders,
>>>>> and resource issues to consider. We would also like to offer a
>>>>> walk-through of 2.0. There's not a lot of time left before the
>>>>> holidays, but we do have next Tuesday's GO-ESSP slot. If you're
>>>>> available and interested, please let us know, and Eric& Nate will put
>>>>> together a tour.
>>>>>
>>>>> cheers - don
>>>>>
>>>>>
>>>>> On Dec 7, 2011, at 1:01 PM, Karl Taylor wrote:
>>>>>
>>>>>
>>>>>> Dear Jennifer and Jean-Yves,
>>>>>>
>>>>>> I apologize that there has been some confusion perpetrated concerning
>>>>>> the planned paths for ESG improvement. For CMIP5 we expect to clear
>>>>>> this up by the end of next week when we will post a clear statement
>>>>>> outlining our plans.
>>>>>>
>>>>>> Thanks again for providing feedback that will improve the system.
>>>>>>
>>>>>> Best regards,
>>>>>> Karl
>>>>>>
>>>>>> On 12/7/11 9:10 AM, Jennifer Adams wrote:
>>>>>>
>>>>>>> Thank you, Karl, for a very helpful straight answer. One thing I
>>>>>>> don't understand: Eric Neinhouse's statement that 'getting Gateway
>>>>>>> 2.0 in production at NCAR is a top priority' seems to be in direct
>>>>>>> conflict with what you wrote. Will Gateway 2.0 be for non-CMIP5 ESG
>>>>>>> datasets, or are NCAR and PCMDI taking different approaches?
>>>>>>> --Jennifer
>>>>>>>
>>>>>>> On Dec 7, 2011, at 11:29 AM, Karl Taylor wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Dear Jennifer,
>>>>>>>>
>>>>>>>> Thank you for your valuable input on how to improve the search and
>>>>>>>> scripting capabilities of both the ESG Gateway 2.0 and the ESGF P2P
>>>>>>>> system.
>>>>>>>>
>>>>>>>> Before you spend too much time attempting to optimize your download
>>>>>>>> of CMIP5 data, you should know that we are very unlikely to
>>>>>>>> actually make Gateway 2.0 operational for CMIP5 model output.
>>>>>>>> Instead we will move directly to P2P. We believe that with
>>>>>>>> currently available resources PCMDI and the ESG federation cannot
>>>>>>>> support two new systems. Instead we will keep our current
>>>>>>>> operational system running and bring on line the P2P. The testing
>>>>>>>> phase of this should happen by the end of the month. At some point
>>>>>>>> the currently operational system will be turned off and the P2P
>>>>>>>> will become operational. Therefore, if as a CMIP5 user you plan to
>>>>>>>> further explore the new systems, I suggest your time might be best
>>>>>>>> spent focusing on P2P.
>>>>>>>>
>>>>>>>> P2P and Gateway 2.0 may have similar capabilities at this time, and
>>>>>>>> both are substantially improved over previous versions of the
>>>>>>>> Gateway, but we're convinced that P2P provides a more robust and
>>>>>>>> flexible framework for adding future improvements. By going
>>>>>>>> directly to P2P, we'll also minimize the inconvenience for users
>>>>>>>> already on the system.
>>>>>>>>
>>>>>>>> Please let me know if you have any concerns about our plans. We
>>>>>>>> realize the current system continues to be more difficult to use
>>>>>>>> than it should be, so your feedback and suggestions are
>>>>>>>> invaluable. Our focus is on getting the improvements offered by
>>>>>>>> P2P in place as soon as possible. I will let you know if our
>>>>>>>> planned path to a better ESGF delivery system for CMIP5 data
>>>>>>>> changes. Please feel free to write me directly at
>>>>>>>> taylor13 at llnl.gov
>>>>>>>>
>>>>>>>> best regards,
>>>>>>>> Karl
>>>>>>>>
>>>>>>>> On 12/7/11 6:14 AM, Jennifer Adams wrote:
>>>>>>>>
>>>>>>>>> Dear Colleagues,
>>>>>>>>> Here is more feedback on the prototype of Gateway 2.0. My current
>>>>>>>>> use case is searching with a fixed experiment, realm, frequency,
>>>>>>>>> and a selected subset of variables. I'm delighted that you can
>>>>>>>>> select more than one variable and it will keep adding them to the
>>>>>>>>> list of selected results. THANK YOU for listing the variables
>>>>>>>>> alphabetically by short name ("Variable Name"); the listing by
>>>>>>>>> long name ("CF Variable") is not useful to me, it is not
>>>>>>>>> alphabetized, and some names are so long they wrap so it looks
>>>>>>>>> like there are three variables called "Flux in the Atmosphere",
>>>>>>>>> "Due to Natural", and "Sources". Oops, my bad, it's actually
>>>>>>>>> "Surface Carbon Mass Flux into the Atmosphere Due to Natural
>>>>>>>>> Sources". Heh. QED.
>>>>>>>>>
>>>>>>>>> When I'm done clicking on all my desired constraints, I get a URL
>>>>>>>>> that looks like this :
>>>>>>>>> http://search-esg.prototype.ucar.edu/search?Type=Dataset&Experiment=decadal1960&Frequency=mon&Realm=atmos&Variable+Name=clt&Variable+Name=prw
>>>>>>>>>
>>>>>>>>> <http://search-esg.prototype.ucar.edu/search?Type=Dataset&Experiment=decadal1960&Frequency=mon&Realm=atmos&Variable+Name=clt&Variable+Name=prw>
>>>>>>>>>
>>>>>>>>> That is great, because it means I can take a short cut from all
>>>>>>>>> the mouse clicking by building a URL that will create my desired
>>>>>>>>> collection.
>>>>>>>>>
>>>>>>>>> I am happy with the search results, so I click on the "Download
>>>>>>>>> Selected" button, and get a wget script called "script.wget".
>>>>>>>>>
>>>>>>>>> 1. It would be great if you could give the script a name that
>>>>>>>>> reflects the search constraints used to build that script --
>>>>>>>>> something like
>>>>>>>>> wget.cmip5.decadal1960.mon.atmos.clt+prw.sh
>>>>>>>>> Every time I download a new wget script, I have to rename it to
>>>>>>>>> something meaningful (such as the above) because the very next
>>>>>>>>> search result will overwrite script.wget, or give me
>>>>>>>>> "script(1).wget" instead, and I'll have no simple way to know what
>>>>>>>>> data set that wget script is configured to bring back. Can you
>>>>>>>>> PLEASE give the wget script a relevant name for me so I can skip
>>>>>>>>> this time-consuming step?
>>>>>>>>>
>>>>>>>>> 2. Is there a way to use our favorite tool (wget) or curl to
>>>>>>>>> download the wget script so I can bypass the Gateway 2.0 browser
>>>>>>>>> interface altogether? My guess is the answer is no, because of the
>>>>>>>>> elaborate login process with OpenID and username/password, but
>>>>>>>>> it's worth asking…
>>>>>>>>>
>>>>>>>>> -----------
>>>>>>>>>
>>>>>>>>> As an alternative, I have looked carefully at the wiki page (and
>>>>>>>>> its embedded links) on automated generation of wget scripts,
>>>>>>>>> http://www.esgf.org/wiki/ESGF_scripting. This is very promising,
>>>>>>>>> and Luca assures me that the P2P service will be working in a
>>>>>>>>> matter of weeks.
>>>>>>>>>
>>>>>>>>> I think the syntax of the URL for the same search as my example
>>>>>>>>> above would be:
>>>>>>>>> http://esg-datanode.jpl.nasa.gov/esg-search/wget?experiment=decadal1960&realm=atmos&time_frequency=month&variable=clt,prw
>>>>>>>>>
>>>>>>>>> <http://esg-datanode.jpl.nasa.gov/esg-search/wget?experiment=decadal1960&realm=atmos&time_frequency=month&variable=clt,prw>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 1. Is the comma-delimited list of desired variables legitimate
>>>>>>>>> syntax?
>>>>>>>>> 2. Will it work the way the Gateway 2.0 does to deliver my subset
>>>>>>>>> of variables?
>>>>>>>>> 3. Will "http://esg-datanode.jpl.nasa.gov
>>>>>>>>> <http://esg-datanode.jpl.nasa.gov/>" be the only location
>>>>>>>>> providing this service?
>>>>>>>>> 4. Will the delivered wget script use certificates and checksums?
>>>>>>>>>
>>>>>>>>> Since this isn't fully operational at the moment, I can't say for
>>>>>>>>> sure whether this will be the better route for reaching my goal of
>>>>>>>>> acquiring CMIP5 data using only scripts and no browser.
>>>>>>>>>
>>>>>>>>> Respectfully submitted,
>>>>>>>>> Jennifer
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Jennifer M. Adams
>>>>>>>>> IGES/COLA
>>>>>>>>> 4041 Powder Mill Road, Suite 302
>>>>>>>>> Calverton, MD 20705
>>>>>>>>> jma at cola.iges.org<mailto:jma at cola.iges.org>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> --
>>>>>>> Jennifer M. Adams
>>>>>>> IGES/COLA
>>>>>>> 4041 Powder Mill Road, Suite 302
>>>>>>> Calverton, MD 20705
>>>>>>> jma at cola.iges.org<mailto:jma at cola.iges.org>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> GO-ESSP-TECH mailing list
>>>>>> GO-ESSP-TECH at ucar.edu<mailto:GO-ESSP-TECH at ucar.edu>
>>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> GO-ESSP-TECH mailing list
>>>>> GO-ESSP-TECH at ucar.edu
>>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>>
>>>>>
>>>> _______________________________________________
>>>> GO-ESSP-TECH mailing list
>>>> GO-ESSP-TECH at ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>>>>
>>>>
>>
>
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
More information about the GO-ESSP-TECH
mailing list