[ncl-talk] Important announcement regarding the future of NCL

Thu Feb 7 20:56:44 MST 2019

Hello

Now that we read about the obvious advantages (logic and ease of i/o) of
NCL, is it possible for someone to list why someone would want to switch to
python?  What does python have for the scientist that NCL does not?

Right now, I read netcdf files, ascii files, csv files, and I made graphs
and maps, as well as write data using Fortran based programs (WRAPIT).  I
find this all relatively easy to do (I do it, so it can't be too hard, it's
just a lot to know!).

Why haven't I switched to Python already.  It sounds like the wave of the
future!  What's in the wave?

Barry

On Fri, Feb 8, 2019 at 4:05 AM Colin Zarzycki <czarzycki at psu.edu> wrote:

> Like Carl, I have assumed this inevitability, although I have also
> scientifically "grown up" with NCL over the past decade+ and have many
> hundreds (probably thousands!) of hours invested in using NCL to answer a
> vast array of scientific questions. While I think the community's march
> towards Python has some underappreciated warts (I spent a couple hours
> updating/rolling back/softlinking mismatched libraries on my personal
> machine after simply running conda update last week), there is no denying
> that its growth across, not only Earth, but the broader data sciences and
> rising stature in university curricula makes it likely the preeminent tool
> of the next generation of researchers. For better or for worse, adoption
> and growth of languages are driven by the size/engagement of their userbase
> instead of optimal practicality for each individual.
>
> A couple thoughts after reading the pivot roadmap...
>
> A.) Based on my "toe-in-the-water geoscience Python scripting" the last
> few years, I actually think one of the weaker points in (right now) is that
> I have yet to find a workflow for weather/climate data I/O where opening,
> massaging, and writing netCDF datasets (and their corresponding variables,
> dimensions, etc.) comes with the same ease as in NCL. Some of that is
> naturally due to tremendous work that has been undertaken in the past to
> support multiple formats behind the curtain, but this also includes NCL's
> amazingly useful calendar routines, coordinate subscripting, simple
> handling of attributes, ability to trivially output a new *.nc file in 3
> lines, etc. As the roadmap notes on pg. 13-14, currently a host of somewhat
> fractured Python packages that can lack clean, easy-to-understand syntax
> are used by the community and (to me) this is a large gap that has remained
> unfilled even with the proliferation of Python support over the last few
> years.
>
> Therefore, I would argue additional examples* regarding side by side
> snippets of "writing Python for NCLers" with a specific emphasis on I/O
> would be incredibly helpful moving forward, particularly with respect to
> functionality NCL has historically "wrapped" to make life easier (for me,
> trading off ease-of-use for a little abstraction/processing inefficiency is
> OK, especially if it means I have time to grab a cup of coffee).
>
> B.) Hopefully I am interpreting the goal of NCOMP correctly, but I am also
> glad to see there is effort to port NCL's functions that are
> highly-specific to the atmospheric sciences (ex: aforementioned calendar
> routines, climatological analysis, vertical coordinate interpolation,
> etc.). This functionality is the workhorse when, for example, I can take
> data straight from a native model grid and have clean, interpolated,
> post-processed variables "on my desk" in a couple minutes with ~10 lines of
> code.
>
> -Colin
>
> *the first draft/examples on the NCL site from Karin Meier-Fleischer are
> already incredibly useful!
>
> On Thu, Feb 7, 2019 at 4:25 PM Adam Phillips <asphilli at ucar.edu> wrote:
>
>> Hi all,
>> Not everyone got my original post so I am resending this to the cgd-etal
>> list. My apologies for the duplicate email.
>>
>> See Mary Haley's email below for information about the future of NCL.
>> Quick summary: NCL is *not* going away, NCL graphics will continue to be
>> developed, and the support structure remains intact. If you have any
>> questions, feel free to contact myself, Nan Rosenbloom, or Laura Landrum,
>> as all of us serve on the NCL Advisory Board.
>> Best,
>> Adam
>>
>> ---------- Forwarded message ---------
>> From: Mary Haley <haley at ucar.edu>
>> Date: Wed, Feb 6, 2019 at 6:24 AM
>> Subject: [ncl-talk] Important announcement regarding the future of NCL
>> To: Ncl-talk <ncl-talk at ucar.edu>
>>
>>
>> Dear NCL Users,
>>
>> This letter is in regard to the future of NCL, following NCAR's decision
>> to move to Python as the scripting language of choice for future
>> visualization and analysis software development. Note that this decision
>> targets new development, leaving existing NCL functionality intact.
>>
>> NCAR is committed to supporting data analysis software for atmospheric,
>> oceanic, and climate science research. However, decreases in budgets and
>> staff, coupled with the enormous functionality that Python brings to the
>> earth sciences, has made it difficult to justify continuing new development
>> on NCL. Python has seen rapid adoption by the earth science community and
>> duplicates much of NCL's functionality, while adding critical features that
>> NCL doesn't offer.
>>
>> Based on recommendations from NSF, CISL and NCL advisory panels, the
>> results of the NCL survey, and months of evaluating different strategies
>> for the future development and support of NCL, NCAR has arrived at these
>> major decisions, effective immediately:
>>
>>    - Python will be adopted as the scripting language platform for
>>    future visualization and analysis development.
>>    - NCL's core language and file I/O will be placed into maintenance
>>    mode.
>>    - NCL's graphics will have continued development through PyNGL
>>    <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pyngl.ucar.edu&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487407101&sdata=M92NtzKIYOk6B31Me1UB%2BKRGqVPlskQqj2Rz6hBqswo%3D&reserved=0>
>>    ***.
>>    - NCL's unique and critical computational routines will be ported to
>>    an as-yet-to-be-named Python package.
>>    - PyNIO
>>    <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pyngl.ucar.edu%2FNio.shtml&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487407101&sdata=PBIVL%2BUZ4iXvEiOpZSi%2B3fg5%2F%2FZEyU2ZF%2FZzX2oaPPQ%3D&reserved=0>
>>    *** will be placed into maintenance mode.
>>    - Development will continue on WRF-Python
>>    <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwrf-python.readthedocs.io%2Fen%2Flatest%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487417106&sdata=4AUCC128c90a5ZjLQdg0ttaqhQjiikW84wmwibCXW0s%3D&reserved=0>
>>    ***.
>>    - All software, including NCL and PyNIO, will be moved to a more open
>>    development software platform to allow for continued community development.
>>
>> *** PyNIO, PyNGL, and WRF-Python are Python modules built on top of NCL
>> libraries, and are developed and supported by the NCL team.
>>
>>
>> NCAR recognizes the significance of these changes. It will take time for
>> NCL users to transition to Python, and some users may not want to make the
>> switch at all. As such, we want to stress that NCL is not going away. NCL
>> users will be able to download NCL and execute their scripts for the
>> foreseeable future.
>>
>> To help users who want to begin transitioning their graphical NCL scripts
>> to PyNGL right away, Karin Meier-Fleischer of DKRZ has written a first
>> draft of an "NCL-to-Python Transition Guide
>> <http:///www.ncl.ucar.edu/Document/Manuals/NCL_to_Python/Transition_Guide_NCL_PyNGL.pdf>"
>> accompanied by a suite of NCL and Python examples
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FApplications%2FNCL_to_Python%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487417106&sdata=OUUTrGkALC3USzYa8GP3gSCkTFm72p0ZK7Yz4luKqYA%3D&reserved=0>.
>> Additionally, we will soon begin converting a subset of the NCL application
>> examples to Python, using PyNGL
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.pyngl.ucar.edu%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487427111&sdata=hr3gpDV65YDMYJzWPYEuu3bHnpz6cBa2hL2%2Fr80%2BYXo%3D&reserved=0>
>> and matplotlib
>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmatplotlib.org&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487427111&sdata=RauXYSJZZsxQh7cDSPtss0Woz2PHcMgVBxMwE7qAmi4%3D&reserved=0>,
>> and will continue to answer questions on the ncl-talk
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.ucar.edu%2Fmailman%2Flistinfo%2Fncl-talk&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487437120&sdata=5NS69uehgGN8K8agtBo9KyXgrFw2fl1RoysYMTuF4SE%3D&reserved=0>
>> email list, but scaling back in order to start helping with Python
>> questions.
>>
>> For a detailed report and roadmap on the "pivot to Python" decision and
>> transition plan, please read the "NCL and the Pivot to Python:
>> Discussion and Roadmap
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FDocument%2FPivot_to_Python%2FNCL_Pivot_to_Python_Report_and_Roadmap.pdf&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487437120&sdata=PlSPR%2BHTiPLXaRtuZq60KHzZ%2FqdZD0Ux89Zh76GN5ps%3D&reserved=0>"
>> report, which can be found on a special page we created containing other
>> supporting documents
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ncl.ucar.edu%2FDocument%2FPivot_to_Python%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487447130&sdata=46CI9mvV%2FLNWvIGGD492vddz0PItL4trdPUXk15lR8E%3D&reserved=0>
>> .
>>
>> The NCL team welcomes your input on this decision. We also want to know
>> if there are other ways we can help ease the transition to Python and
>> encourage users to become more active contributors through open
>> development. Please use this GitHub issue
>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNCAR%2Fncl%2Fissues%2F64&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487447130&sdata=f6slOq7pKScfUdv2hiW5u5BTXTIDTMQ8e6xluqmTeXA%3D&reserved=0>
>> to submit questions or comments so we can keep the discussion public.
>>
>> *NCL Team:*
>> John Clyne (acting group head)
>> Rick Brownrigg
>> Mary Haley
>> Kevin Hallock
>> Bill Ladwig
>>
>>
>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmailman.ucar.edu%2Fmailman%2Flistinfo%2Fncl-talk&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487457135&sdata=x3LkBTRWrJRHPb0NChym0No6QWBE2gQ5jims0UyUCRA%3D&reserved=0>
>>
>>
>> --
>> Adam Phillips
>> Associate Scientist,  Climate and Global Dynamics Laboratory, NCAR
>> www.cgd.ucar.edu/staff/asphilli/
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487457135&sdata=1nLuGaRf5AggSZyznMgb9QlSGyntziTbTw1v%2Bt8MEV8%3D&reserved=0>
>> 303-497-1726
>>
>>
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487457135&sdata=kCdxy%2FQRpsQirNd4%2Fwt9RTXdtYgyYpr51HllJPA3cw4%3D&reserved=0>
>>
>>
>> --
>> Adam Phillips
>> Associate Scientist,  Climate and Global Dynamics Laboratory, NCAR
>> www.cgd.ucar.edu/staff/asphilli/
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli%2F&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487467140&sdata=rOw%2F9v%2Ba1l7c6AseFcLB3DIHEH%2BcNchZdD4UKCaVa6o%3D&reserved=0>
>> 303-497-1726
>>
>>
>> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cgd.ucar.edu%2Fstaff%2Fasphilli&data=02%7C01%7Ccmz5202%40psu.edu%7C475b847b8a1a4321b2a808d68d42d347%7C7cf48d453ddb4389a9c1c115526eb52e%7C0%7C0%7C636851715487467140&sdata=77bg13k%2BE1VGDTHk26IBVqIXQTju1hawg3pDFYJg2xs%3D&reserved=0>
>>
>
>
> --
> Colin M. Zarzycki, Ph.D.
> Assistant Professor of Meteorology and Climate Dynamics
> Department of Meteorology and Atmospheric Science
> Pennsylvania State University
> Web: http://www.colinzarzycki.com | Office: +1 (814) 863-6888
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>

-- 
Barry H. Lynn, Ph.D
Senior Associate Scientist, Lecturer,
The Institute of the Earth Science,
The Hebrew University of Jerusalem,
Givat Ram, Jerusalem 91904, Israel
Tel: 972 547 231 170
Fax: (972)-25662581

C.E.O, Weather It Is, LTD
Weather and Climate Focus
http://weather-it-is.com
Jerusalem, Israel
Local: 02 930 9525
Cell: 054 7 231 170
Int-IS: x972 2 930 9525
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20190208/992064d8/attachment.html>