[ncl-talk] Important announcement regarding the future of NCL

Wed Feb 6 15:59:01 MST 2019

Hi Barry,

I encourage folks to read the report, as it covers in detail why the
decision to transition to Python, and what Python brings to the table:

http://www.ncl.ucar.edu/Document/Pivot_to_Python/NCL_Pivot_to_Python_Report_and_Roadmap.pdf

There's a section "Why Python" (starts on page 5) that explains some of the
reasoning behind this decision.

Here's the pertinent part of that section:

*Why Python?*
Python has gained widespread acceptance by universities and research
organizations around the world and is being adopted as the programming
language of choice for scientific computing. This is evidenced by several
factors: 1) the availability of quality scientific Python modules via the
SciPy ecosystem, 2) the continued and growing popularity of the annual
SciPy conference, now in its 17th year, 3) the availability of books on
Python for scientists, and 4) the increasing number of scientific graduate
students who are learning Python in college as an open source alternative
to other non-free software like IDL and MATLAB. In September 2018—for the
first time in history—Python entered the TIOBE index top 3 (www.tiobe.com),
a measure of popularity of programming languages based on search engine
results.

Python has picked up rapid steam in the geoscientific community as well.
For the last eight years the American Meteorological Society Annual Meeting
has hosted a popular and well-attended symposium on the “Advances in
Modeling and Analysis Using Python”. NCAR is a major partner in the Pangeo (
pangeo.io) community, an NSF EarthCube funded effort that provides an “open
source scientific Python ecosystem for ocean / atmosphere / land / climate
science” and is focused on providing tools and support for handling
petabyte-scale datasets on HPC and cloud platforms. There are hundreds of
scientific Python modules that provide domain-specific functionality for
reading/writing data, computational analyses, and visualization. The
benefit of these individual packages is that they are usually specialized
for a specific domain or class of problems, thus filling a critical need
that a more general-purpose language cannot.

The Python language itself provides rich language features that NCL does
not have, including optional arguments, a robust interactive interface,
generators, exception handling, and built-in debugging and testing. The
Python community has a rapidly growing base of scientific software
developers that are able to address the growing needs of the geoscientific
community much faster than we can in the areas of scalability, interfaces
to other languages like R for statistical calculations, and support for a
wider range of complex data formats. By replacing the NCL language with the
Python language, the NCL user base will instantly gain access to these
features, and we will be able to benefit from the already vibrant and
active open development Python community. Python itself has been open
developed since October 2000.
Last but not least, it is becoming harder to hire developers who want to
work on a programming language with a narrow focus, versus a highly visible
and mainstream language like Python.

I hope this addresses your questions.

--Mary

On Wed, Feb 6, 2019 at 7:29 AM Barry Lynn <barry.h.lynn at gmail.com> wrote:

> Hello Mary:
>
> Could you please help us understand what critical features are missing
> from NCL but present in python so that we better understand why we should
> switch.
>
> Thank you,
>
> Barry
>
> On Wed, Feb 6, 2019 at 3:23 PM Mary Haley <haley at ucar.edu> wrote:
>
>> Dear NCL Users,
>>
>> This letter is in regard to the future of NCL, following NCAR's decision
>> to move to Python as the scripting language of choice for future
>> visualization and analysis software development. Note that this decision
>> targets new development, leaving existing NCL functionality intact.
>>
>> NCAR is committed to supporting data analysis software for atmospheric,
>> oceanic, and climate science research. However, decreases in budgets and
>> staff, coupled with the enormous functionality that Python brings to the
>> earth sciences, has made it difficult to justify continuing new development
>> on NCL. Python has seen rapid adoption by the earth science community and
>> duplicates much of NCL's functionality, while adding critical features that
>> NCL doesn't offer.
>>
>> Based on recommendations from NSF, CISL and NCL advisory panels, the
>> results of the NCL survey, and months of evaluating different strategies
>> for the future development and support of NCL, NCAR has arrived at these
>> major decisions, effective immediately:
>>
>>    - Python will be adopted as the scripting language platform for
>>    future visualization and analysis development.
>>    - NCL's core language and file I/O will be placed into maintenance
>>    mode.
>>    - NCL's graphics will have continued development through PyNGL
>>    <http://www.pyngl.ucar.edu>***.
>>    - NCL's unique and critical computational routines will be ported to
>>    an as-yet-to-be-named Python package.
>>    - PyNIO <http://www.pyngl.ucar.edu/Nio.shtml>*** will be placed into
>>    maintenance mode.
>>    - Development will continue on WRF-Python
>>    <https://wrf-python.readthedocs.io/en/latest/>***.
>>    - All software, including NCL and PyNIO, will be moved to a more open
>>    development software platform to allow for continued community development.
>>
>> *** PyNIO, PyNGL, and WRF-Python are Python modules built on top of NCL
>> libraries, and are developed and supported by the NCL team.
>>
>>
>> NCAR recognizes the significance of these changes. It will take time for
>> NCL users to transition to Python, and some users may not want to make the
>> switch at all. As such, we want to stress that NCL is not going away. NCL
>> users will be able to download NCL and execute their scripts for the
>> foreseeable future.
>>
>> To help users who want to begin transitioning their graphical NCL scripts
>> to PyNGL right away, Karin Meier-Fleischer of DKRZ has written a first
>> draft of an "NCL-to-Python Transition Guide
>> <http:///www.ncl.ucar.edu/Document/Manuals/NCL_to_Python/Transition_Guide_NCL_PyNGL.pdf>"
>> accompanied by a suite of NCL and Python examples
>> <http://www.ncl.ucar.edu/Applications/NCL_to_Python/>.  Additionally, we
>> will soon begin converting a subset of the NCL application examples to
>> Python, using PyNGL <http://www.pyngl.ucar.edu/> and matplotlib
>> <https://matplotlib.org>, and will continue to answer questions on the
>> ncl-talk <http://mailman.ucar.edu/mailman/listinfo/ncl-talk> email list,
>> but scaling back in order to start helping with Python questions.
>>
>> For a detailed report and roadmap on the "pivot to Python" decision and
>> transition plan, please read the "NCL and the Pivot to Python:
>> Discussion and Roadmap
>> <http://www.ncl.ucar.edu/Document/Pivot_to_Python/NCL_Pivot_to_Python_Report_and_Roadmap.pdf>"
>> report, which can be found on a special page we created containing other
>> supporting documents <http://www.ncl.ucar.edu/Document/Pivot_to_Python/>.
>>
>> The NCL team welcomes your input on this decision. We also want to know
>> if there are other ways we can help ease the transition to Python and
>> encourage users to become more active contributors through open
>> development. Please use this GitHub issue
>> <https://github.com/NCAR/ncl/issues/64> to submit questions or comments
>> so we can keep the discussion public.
>>
>> *NCL Team:*
>> John Clyne (acting group head)
>> Rick Brownrigg
>> Mary Haley
>> Kevin Hallock
>> Bill Ladwig
>>
>>
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
>
> --
> Barry H. Lynn, Ph.D
> Senior Associate Scientist, Lecturer,
> The Institute of the Earth Science,
> The Hebrew University of Jerusalem,
> Givat Ram, Jerusalem 91904, Israel
> Tel: 972 547 231 170
> Fax: (972)-25662581
>
> C.E.O, Weather It Is, LTD
> Weather and Climate Focus
> http://weather-it-is.com
> Jerusalem, Israel
> Local: 02 930 9525
> Cell: 054 7 231 170
> Int-IS: x972 2 930 9525
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190206/3c689ce8/attachment.html>