[ncl-talk] NCL/GeoCat (Pivot to Python) update

[John Clyne]

Dear All,

It’s been several months since the NCL “Pivot to Python” was announced, and many exciting things have happened since that time. Apologies in advance for this lengthy email, but I wanted to provide you with some important updates. 

Staff changes:

As some of you know we’ve had some staff changes over the past few months. Mary Haley retired after decades of outstanding leadership and support of the NCL package. Bill Ladwig and Rick Brownrigg both moved on to greener pastures. It’s taken us a while to recover, but I’m excited to announce that the group is now fully staffed back up. We have three new full time software engineers with strong backgrounds in both software development and the earth sciences. 

Geoscience Community Analysis Toolkit (GeoCAT):

If you haven’t heard, we’ve formally named the effort to migrate NCL functionality into Python; the project is called the Geosciences Community Analysis Toolkit (GeoCAT). Though still a work-in-progress a public website for GeoCAT is now available: https://geocat.ucar.edu/. The GeoCAT project will serve as an umbrella for all of our Python tools: PyNGL, PyNIO, wrf-python, and GeoCAT-Comp. The latter is the newest and our current area of focus. GeoCAT-Comp will encompass our computational routines. We are anticipating a first release later this year, which will provide a collection of climate focused computational capabilities. The list of functions to be included in the first release can be viewed as a dynamic GitHub “kanban” roadmap here: https://github.com/NCAR/geocat-comp/projects. This public roadmap will be continuously updated as the development team makes progress. 

Regarding releases: as part of our migration to Open Development we’re moving toward a hybrid of continuous and formal releases: as new functions are added to GeoCAT-Comp they will become immediately available for easy download and installation, providing opportunities for early adopters to experiment with and provide feedback on the code, the documentation, and the examples. Formal (stable) releases will still occur with a frequency of once or twice per year. 

At the time of this writing at least one family of NCL computational functions, linint2, has been ported to Python. I encourage you to look at the new documentation format, example code, and even consider downloading and running the new function from the GeoCAT web site (https://geocat.ucar.edu). This is your opportunity to help ensure that the community’s needs are being met. We strongly encourage and welcome your input!

Open Development:

One of our goals with both NCL and GeoCAT is to foster an environment that supports Open Development, i.e. we want to both  enable and encourage the community to contribute in numerous ways such as responding to user support questions, requesting features, evaluating new capabilities, writing documentation or examples, and even developing code. NCL has already been successful in getting community contributions to support questions via the ncl-talk mailing list, and we are incredibly grateful for the numerous individuals who help out. Moving forward we want to further expand community involvement. Toward this end we have begun developing “contributor’s guides” for each of the projects that spell out the necessary steps for contributing in various areas. The wrf-python contributor’s guide is complete and available here: https://wrf-python.readthedocs.io/en/latest/contrib.html.  We encourage you to peruse it and provide feedback as we prepare guides for NCL and GeoCAT-Comp.

Training:

The GeoCAT team has been partnering with NCAR’s Pangeo group and Climate and Global Dynamics (CGD) division to offer tutorials on using Python for geoscience data analysis at the NCAR Mesa Lab. The material for these tutorials is being re-factored into a public on-line, self-guided introduction to scientific python. We intend to announce the availability of this resource shortly. GeoCAT specific training material will be added to the site as it becomes available, hopefully later this year.

NCL:

To summarize the status of NCL, the project is “feature frozen”. CISL does not have plans at this time to add new features to NCL. We will, however, prepare maintenance releases containing critical bug fixes and user-contributed code on an infrequent basis. Thus, there is no reason for members of the community with substantial investments in NCL to consider porting their NCL code to Python (unless you want to take advantage of the improved scalability afforded by our use of Dask).

We do not have a firm schedule for the next maintenance release, but would anticipate this happening sometime in early CY20. To facilitate user contributions, particularly bug fixes, we are preparing a contributor’s guide as noted above, and the source code for NCL is now available from its GitHub repository: https://github.com/NCAR/ncl. We will soon be in a position to accept “pull requests” for community contributions. 

As always we welcome and value your comments and questions. 

The GeoCAT Group




John Clyne
National Center for Atmospheric Research
303.497.1236 (w), 303.809.1922 (c)
clyne at ucar.edu