[ncl-talk] Important announcement regarding the future of NCL

Wed Feb 6 21:24:38 MST 2019

Barry et al.,

I don’t think those programs have to be rewritten. My understanding is the NCL version on your system will keep working, it just won’t get future updates from the NCL team, and maybe less user support. Like many, I have dozens of NCL scripts, but they will keep working. For the future though, NCL users might be smart to transitioning to R or Python. FOR R and R Studio there are NCL packages, basically add-ons that allow you to process netCDF data in R. The basics of Python you can learn for example through Software Carpentry <https://software-carpentry.org/>.

I hope that helps.

Toni

Toni Klemm, Ph.D.
Postdoctoral Research Associate
Department of Ecosystem Science and Management
College of Agriculture and Life Sciences
Texas A&M University, College Station, TX
Contributor to the Early Career Climate Forum <https://www.eccforum.org/>
www.toni-klemm.de <http://www.toni-klemm.de/> | @toniklemm <https://twitter.com/ToniKlemm>

> On Feb 6, 2019, at 10:06 PM, Barry Lynn <barry.h.lynn at gmail.com> wrote:
> 
> Hi Mary:
> 
> Thanks.
> 
> A most pertinent question: how hard will it be for someone who has worked hard to "know" NCL to transition to Python.
> 
> Also, keep in mind that I (and others) have written 10s of programs in NC, and these would need to be rewritten.
> 
> Barry
> 
> On Thu, Feb 7, 2019 at 12:59 AM Mary Haley <haley at ucar.edu <mailto:haley at ucar.edu>> wrote:
> Hi Barry,
> 
> I encourage folks to read the report, as it covers in detail why the decision to transition to Python, and what Python brings to the table:
> 
> http://www.ncl.ucar.edu/Document/Pivot_to_Python/NCL_Pivot_to_Python_Report_and_Roadmap.pdf <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ncl.ucar.edu_Document_Pivot-5Fto-5FPython_NCL-5FPivot-5Fto-5FPython-5FReport-5Fand-5FRoadmap.pdf&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=yz-Fa4nJPdDM4mRUn0H5NfqdFWgASciTiMBeSdNjHSg&e=>
> 
> There's a section "Why Python" (starts on page 5) that explains some of the reasoning behind this decision.
> 
> Here's the pertinent part of that section:
> 
> Why Python?
> Python has gained widespread acceptance by universities and research organizations around the world and is being adopted as the programming language of choice for scientific computing. This is evidenced by several factors: 1) the availability of quality scientific Python modules via the SciPy ecosystem, 2) the continued and growing popularity of the annual SciPy conference, now in its 17th year, 3) the availability of books on Python for scientists, and 4) the increasing number of scientific graduate students who are learning Python in college as an open source alternative to other non-free software like IDL and MATLAB. In September 2018—for the first time in history—Python entered the TIOBE index top 3 (www.tiobe.com <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.tiobe.com&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=NPg_3kgLJfrAEc8Q3bF6ZufzwtozXd-5FCPtL3LKB9g&e=>), a measure of popularity of programming languages based on search engine results.
> 
> Python has picked up rapid steam in the geoscientific community as well. For the last eight years the American Meteorological Society Annual Meeting has hosted a popular and well-attended symposium on the “Advances in Modeling and Analysis Using Python”. NCAR is a major partner in the Pangeo (pangeo.io <https://urldefense.proofpoint.com/v2/url?u=http-3A__pangeo.io&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=_BI6R6g4EqWUuigwl0IZHXnwHXFSOz47FSLhJSZdYMI&e=>) community, an NSF EarthCube funded effort that provides an “open source scientific Python ecosystem for ocean / atmosphere / land / climate science” and is focused on providing tools and support for handling petabyte-scale datasets on HPC and cloud platforms. There are hundreds of scientific Python modules that provide domain-specific functionality for reading/writing data, computational analyses, and visualization. The benefit of these individual packages is that they are usually specialized for a specific domain or class of problems, thus filling a critical need that a more general-purpose language cannot.
> 
> The Python language itself provides rich language features that NCL does not have, including optional arguments, a robust interactive interface, generators, exception handling, and built-in debugging and testing. The Python community has a rapidly growing base of scientific software developers that are able to address the growing needs of the geoscientific community much faster than we can in the areas of scalability, interfaces to other languages like R for statistical calculations, and support for a wider range of complex data formats. By replacing the NCL language with the Python language, the NCL user base will instantly gain access to these features, and we will be able to benefit from the already vibrant and active open development Python community. Python itself has been open developed since October 2000.
> Last but not least, it is becoming harder to hire developers who want to work on a programming language with a narrow focus, versus a highly visible and mainstream language like Python.
> 
> I hope this addresses your questions.
> 
> --Mary
> 
> 
> On Wed, Feb 6, 2019 at 7:29 AM Barry Lynn <barry.h.lynn at gmail.com <mailto:barry.h.lynn at gmail.com>> wrote:
> Hello Mary:
> 
> Could you please help us understand what critical features are missing from NCL but present in python so that we better understand why we should switch.
> 
> Thank you,
> 
> Barry
> 
> On Wed, Feb 6, 2019 at 3:23 PM Mary Haley <haley at ucar.edu <mailto:haley at ucar.edu>> wrote:
> Dear NCL Users,
> 
> This letter is in regard to the future of NCL, following NCAR's decision to move to Python as the scripting language of choice for future visualization and analysis software development. Note that this decision targets new development, leaving existing NCL functionality intact.
> 
> NCAR is committed to supporting data analysis software for atmospheric, oceanic, and climate science research. However, decreases in budgets and staff, coupled with the enormous functionality that Python brings to the earth sciences, has made it difficult to justify continuing new development on NCL. Python has seen rapid adoption by the earth science community and duplicates much of NCL's functionality, while adding critical features that NCL doesn't offer.
> 
> Based on recommendations from NSF, CISL and NCL advisory panels, the results of the NCL survey, and months of evaluating different strategies for the future development and support of NCL, NCAR has arrived at these major decisions, effective immediately:
> Python will be adopted as the scripting language platform for future visualization and analysis development.
> NCL's core language and file I/O will be placed into maintenance mode.
> NCL's graphics will have continued development through PyNGL <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pyngl.ucar.edu&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=XjUKpVjTosG1QXg8tPanvc1VWz8lgMBhRmHlr54D4ww&e=>*.
> NCL's unique and critical computational routines will be ported to an as-yet-to-be-named Python package.
> PyNIO <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pyngl.ucar.edu_Nio.shtml&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=eYYQhhwmjnYoMGlTw_DrlkxnLRKaZf9qt4qZ7oIl4b0&e=>* will be placed into maintenance mode.
> Development will continue on WRF-Python <https://urldefense.proofpoint.com/v2/url?u=https-3A__wrf-2Dpython.readthedocs.io_en_latest_&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=SRZaRV6rql3QcJHAIFeRtPaJNotCWR_YFuywSlAxDAk&e=>*.
> All software, including NCL and PyNIO, will be moved to a more open development software platform to allow for continued community development.
> * PyNIO, PyNGL, and WRF-Python are Python modules built on top of NCL libraries, and are developed and supported by the NCL team.
> 
> NCAR recognizes the significance of these changes. It will take time for NCL users to transition to Python, and some users may not want to make the switch at all. As such, we want to stress that NCL is not going away. NCL users will be able to download NCL and execute their scripts for the foreseeable future.
> 
> To help users who want to begin transitioning their graphical NCL scripts to PyNGL right away, Karin Meier-Fleischer of DKRZ has written a first draft of an "NCL-to-Python Transition Guide <https://urldefense.proofpoint.com/v2/url?u=http-3A___www.ncl.ucar.edu_Document_Manuals_NCL-5Fto-5FPython_Transition-5FGuide-5FNCL-5FPyNGL.pdf&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=a3sboqqRFVThkjcuPudM_0tLy_Bo0eDIIlgK5OASFwo&e=>" accompanied by a suite of NCL and Python examples <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ncl.ucar.edu_Applications_NCL-5Fto-5FPython_&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=LEBpgpbmvF5EKsuZVq-Oik-e3WTT26MmT6Zy0FCeb40&e=>.  Additionally, we will soon begin converting a subset of the NCL application examples to Python, using PyNGL <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.pyngl.ucar.edu_&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=VZDbL9EjlrCHncOZb_tT6Q8ifLAfXDt5XR1_thUOj9M&e=> and matplotlib <https://urldefense.proofpoint.com/v2/url?u=https-3A__matplotlib.org&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=houOTRZi2bLqejxNKntfvEpoJj3nsFfjc_keFU-o4NE&e=>, and will continue to answer questions on the ncl-talk <https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.ucar.edu_mailman_listinfo_ncl-2Dtalk&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=TEqpggwoOULSGTXA0ktYGBrVl79eAsBtpqWNPVyiGe4&e=> email list, but scaling back in order to start helping with Python questions.
> 
> For a detailed report and roadmap on the "pivot to Python" decision and transition plan, please read the "NCL and the Pivot to Python: Discussion and Roadmap <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ncl.ucar.edu_Document_Pivot-5Fto-5FPython_NCL-5FPivot-5Fto-5FPython-5FReport-5Fand-5FRoadmap.pdf&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=yz-Fa4nJPdDM4mRUn0H5NfqdFWgASciTiMBeSdNjHSg&e=>" report, which can be found on a special page we created containing other supporting documents <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ncl.ucar.edu_Document_Pivot-5Fto-5FPython_&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=SRVKQ5isbDnyHvL4wzQeQEpJthMV6M3e0w01qJF9S50&e=>.
> 
> The NCL team welcomes your input on this decision. We also want to know if there are other ways we can help ease the transition to Python and encourage users to become more active contributors through open development. Please use this GitHub issue <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_NCAR_ncl_issues_64&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=2dmElBi9C2oq4us6lvx7AJ7C9I8qbH8wiV1PzKs9u3k&e=> to submit questions or comments so we can keep the discussion public.
> 
> NCL Team:
> John Clyne (acting group head)
> Rick Brownrigg
> Mary Haley
> Kevin Hallock
> Bill Ladwig
> 
> 
> 
> 
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu <mailto:ncl-talk at ucar.edu>
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk <https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.ucar.edu_mailman_listinfo_ncl-2Dtalk&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=TEqpggwoOULSGTXA0ktYGBrVl79eAsBtpqWNPVyiGe4&e=>
> 
> 
> -- 
> Barry H. Lynn, Ph.D
> Senior Associate Scientist, Lecturer,
> The Institute of the Earth Science, 
> The Hebrew University of Jerusalem, 
> Givat Ram, Jerusalem 91904, Israel 
> Tel: 972 547 231 170
> Fax: (972)-25662581
> 
> C.E.O, Weather It Is, LTD
> Weather and Climate Focus
> http://weather-it-is.com <https://urldefense.proofpoint.com/v2/url?u=http-3A__weather-2Dit-2Dis.com&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=Uv0CFIV_YJOKuyMuPo3lRqFEZpZiEHfHOPwrZRcPADU&e=>
> Jerusalem, Israel
> Local: 02 930 9525
> Cell: 054 7 231 170
> Int-IS: x972 2 930 9525
> 
> 
> 
> -- 
> Barry H. Lynn, Ph.D
> Senior Associate Scientist, Lecturer,
> The Institute of the Earth Science, 
> The Hebrew University of Jerusalem, 
> Givat Ram, Jerusalem 91904, Israel 
> Tel: 972 547 231 170
> Fax: (972)-25662581
> 
> C.E.O, Weather It Is, LTD
> Weather and Climate Focus
> http://weather-it-is.com <https://urldefense.proofpoint.com/v2/url?u=http-3A__weather-2Dit-2Dis.com&d=DwMFaQ&c=ODFT-G5SujMiGrKuoJJjVg&r=WsLJ5ywbW3XnTzqo_ETV62H6s1a5bcGgnrpl7xPRfic&m=iOo39tnO8aeORpG5noJIhwsDxjOcD1HnwHFp0MTvde4&s=Uv0CFIV_YJOKuyMuPo3lRqFEZpZiEHfHOPwrZRcPADU&e=>
> Jerusalem, Israel
> Local: 02 930 9525
> Cell: 054 7 231 170
> Int-IS: x972 2 930 9525
> 
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu <mailto:ncl-talk at ucar.edu>
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk <http://mailman.ucar.edu/mailman/listinfo/ncl-talk>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190206/858d4b4c/attachment.html>