[ncl-talk] Important announcement regarding the future of NCL

Thu Feb 7 19:04:50 MST 2019

Like Carl, I have assumed this inevitability, although I have also
scientifically "grown up" with NCL over the past decade+ and have many
hundreds (probably thousands!) of hours invested in using NCL to answer a
vast array of scientific questions. While I think the community's march
towards Python has some underappreciated warts (I spent a couple hours
updating/rolling back/softlinking mismatched libraries on my personal
machine after simply running conda update last week), there is no denying
that its growth across, not only Earth, but the broader data sciences and
rising stature in university curricula makes it likely the preeminent tool
of the next generation of researchers. For better or for worse, adoption
and growth of languages are driven by the size/engagement of their userbase
instead of optimal practicality for each individual.

A couple thoughts after reading the pivot roadmap...

A.) Based on my "toe-in-the-water geoscience Python scripting" the last few
years, I actually think one of the weaker points in (right now) is that I
have yet to find a workflow for weather/climate data I/O where opening,
massaging, and writing netCDF datasets (and their corresponding variables,
dimensions, etc.) comes with the same ease as in NCL. Some of that is
naturally due to tremendous work that has been undertaken in the past to
support multiple formats behind the curtain, but this also includes NCL's
amazingly useful calendar routines, coordinate subscripting, simple
handling of attributes, ability to trivially output a new *.nc file in 3
lines, etc. As the roadmap notes on pg. 13-14, currently a host of somewhat
fractured Python packages that can lack clean, easy-to-understand syntax
are used by the community and (to me) this is a large gap that has remained
unfilled even with the proliferation of Python support over the last few
years.

Therefore, I would argue additional examples* regarding side by side
snippets of "writing Python for NCLers" with a specific emphasis on I/O
would be incredibly helpful moving forward, particularly with respect to
functionality NCL has historically "wrapped" to make life easier (for me,
trading off ease-of-use for a little abstraction/processing inefficiency is
OK, especially if it means I have time to grab a cup of coffee).

B.) Hopefully I am interpreting the goal of NCOMP correctly, but I am also
glad to see there is effort to port NCL's functions that are
highly-specific to the atmospheric sciences (ex: aforementioned calendar
routines, climatological analysis, vertical coordinate interpolation,
etc.). This functionality is the workhorse when, for example, I can take
data straight from a native model grid and have clean, interpolated,
post-processed variables "on my desk" in a couple minutes with ~10 lines of
code.

-Colin

*the first draft/examples on the NCL site from Karin Meier-Fleischer are
already incredibly useful!

On Thu, Feb 7, 2019 at 4:25 PM Adam Phillips <asphilli at ucar.edu> wrote:

> Hi all,
> Not everyone got my original post so I am resending this to the cgd-etal
> list. My apologies for the duplicate email.
>
> See Mary Haley's email below for information about the future of NCL.
> Quick summary: NCL is *not* going away, NCL graphics will continue to be
> developed, and the support structure remains intact. If you have any
> questions, feel free to contact myself, Nan Rosenbloom, or Laura Landrum,
> as all of us serve on the NCL Advisory Board.
> Best,
> Adam
>
> ---------- Forwarded message ---------
> From: Mary Haley <haley at ucar.edu>
> Date: Wed, Feb 6, 2019 at 6:24 AM
> Subject: [ncl-talk] Important announcement regarding the future of NCL
> To: Ncl-talk <ncl-talk at ucar.edu>
>
>
> Dear NCL Users,
>
> This letter is in regard to the future of NCL, following NCAR's decision
> to move to Python as the scripting language of choice for future
> visualization and analysis software development. Note that this decision
> targets new development, leaving existing NCL functionality intact.
>
> NCAR is committed to supporting data analysis software for atmospheric,
> oceanic, and climate science research. However, decreases in budgets and
> staff, coupled with the enormous functionality that Python brings to the
> earth sciences, has made it difficult to justify continuing new development
> on NCL. Python has seen rapid adoption by the earth science community and
> duplicates much of NCL's functionality, while adding critical features that
> NCL doesn't offer.
>
> Based on recommendations from NSF, CISL and NCL advisory panels, the
> results of the NCL survey, and months of evaluating different strategies
> for the future development and support of NCL, NCAR has arrived at these
> major decisions, effective immediately:
>
>    - Python will be adopted as the scripting language platform for future
>    visualization and analysis development.
>    - NCL's core language and file I/O will be placed into maintenance
>    mode.
>    - NCL's graphics will have continued development through PyNGL
>    <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pyngl.ucar.edu&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487407101&sdata=M92NtzKIYOk6B31Me1UB%2BKRGqVPlskQqj2Rz6hBqswo%3D&reserved=0>
>    ***.
>    - NCL's unique and critical computational routines will be ported to
>    an as-yet-to-be-named Python package.
>    - PyNIO
>    <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pyngl.ucar.edu%2FNio.shtml&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487407101&sdata=PBIVL%2BUZ4iXvEiOpZSi%2B3fg5%2F%2FZEyU2ZF%2FZzX2oaPPQ%3D&reserved=0>
>    *** will be placed into maintenance mode.
>    - Development will continue on WRF-Python
>    <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwrf-python.readthedocs.io%2Fen%2Flatest%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487417106&sdata=4AUCC128c90a5ZjLQdg0ttaqhQjiikW84wmwibCXW0s%3D&reserved=0>
>    ***.
>    - All software, including NCL and PyNIO, will be moved to a more open
>    development software platform to allow for continued community development.
>
> *** PyNIO, PyNGL, and WRF-Python are Python modules built on top of NCL
> libraries, and are developed and supported by the NCL team.
>
>
> NCAR recognizes the significance of these changes. It will take time for
> NCL users to transition to Python, and some users may not want to make the
> switch at all. As such, we want to stress that NCL is not going away. NCL
> users will be able to download NCL and execute their scripts for the
> foreseeable future.
>
> To help users who want to begin transitioning their graphical NCL scripts
> to PyNGL right away, Karin Meier-Fleischer of DKRZ has written a first
> draft of an "NCL-to-Python Transition Guide
> <http:///www.ncl.ucar.edu/Document/Manuals/NCL_to_Python/Transition_Guide_NCL_PyNGL.pdf>"
> accompanied by a suite of NCL and Python examples
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FApplications%2FNCL_to_Python%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487417106&sdata=OUUTrGkALC3USzYa8GP3gSCkTFm72p0ZK7Yz4luKqYA%3D&reserved=0>.
> Additionally, we will soon begin converting a subset of the NCL application
> examples to Python, using PyNGL
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pyngl.ucar.edu%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487427111&sdata=hr3gpDV65YDMYJzWPYEuu3bHnpz6cBa2hL2%2Fr80%2BYXo%3D&reserved=0>
> and matplotlib
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmatplotlib.org&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487427111&sdata=RauXYSJZZsxQh7cDSPtss0Woz2PHcMgVBxMwE7qAmi4%3D&reserved=0>,
> and will continue to answer questions on the ncl-talk
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.ucar.edu%2Fmailman%2Flistinfo%2Fncl-talk&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487437120&sdata=5NS69uehgGN8K8agtBo9KyXgrFw2fl1RoysYMTuF4SE%3D&reserved=0>
> email list, but scaling back in order to start helping with Python
> questions.
>
> For a detailed report and roadmap on the "pivot to Python" decision and
> transition plan, please read the "NCL and the Pivot to Python: Discussion
> and Roadmap
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FDocument%2FPivot_to_Python%2FNCL_Pivot_to_Python_Report_and_Roadmap.pdf&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487437120&sdata=PlSPR%2BHTiPLXaRtuZq60KHzZ%2FqdZD0Ux89Zh76GN5ps%3D&reserved=0>"
> report, which can be found on a special page we created containing other
> supporting documents
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FDocument%2FPivot_to_Python%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487447130&sdata=46CI9mvV%2FLNWvIGGD492vddz0PItL4trdPUXk15lR8E%3D&reserved=0>
> .
>
> The NCL team welcomes your input on this decision. We also want to know if
> there are other ways we can help ease the transition to Python and
> encourage users to become more active contributors through open
> development. Please use this GitHub issue
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNCAR%2Fncl%2Fissues%2F64&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487447130&sdata=f6slOq7pKScfUdv2hiW5u5BTXTIDTMQ8e6xluqmTeXA%3D&reserved=0>
> to submit questions or comments so we can keep the discussion public.
>
> *NCL Team:*
> John Clyne (acting group head)
> Rick Brownrigg
> Mary Haley
> Kevin Hallock
> Bill Ladwig
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.ucar.edu%2Fmailman%2Flistinfo%2Fncl-talk&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487457135&sdata=x3LkBTRWrJRHPb0NChym0No6QWBE2gQ5jims0UyUCRA%3D&reserved=0>
>
>
> --
> Adam Phillips
> Associate Scientist,  Climate and Global Dynamics Laboratory, NCAR
> www.cgd.ucar.edu/staff/asphilli/
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487457135&sdata=1nLuGaRf5AggSZyznMgb9QlSGyntziTbTw1v%2Bt8MEV8%3D&reserved=0>
> 303-497-1726
>
>
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487457135&sdata=kCdxy%2FQRpsQirNd4%2Fwt9RTXdtYgyYpr51HllJPA3cw4%3D&reserved=0>
>
>
> --
> Adam Phillips
> Associate Scientist,  Climate and Global Dynamics Laboratory, NCAR
> www.cgd.ucar.edu/staff/asphilli/
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487467140&sdata=rOw%2F9v%2Ba1l7c6AseFcLB3DIHEH%2BcNchZdD4UKCaVa6o%3D&reserved=0>
> 303-497-1726
>
>
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487467140&sdata=77bg13k%2BE1VGDTHk26IBVqIXQTju1hawg3pDFYJg2xs%3D&reserved=0>
>

-- 
Colin M. Zarzycki, Ph.D.
Assistant Professor of Meteorology and Climate Dynamics
Department of Meteorology and Atmospheric Science
Pennsylvania State University
Web: http://www.colinzarzycki.com | Office: +1 (814) 863-6888
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190207/2b8c0a77/attachment.html>