[pyngl-talk] delegation, division of labor, packaging, was: netcdf4-python vs PyNIO

[Tom Roche]

Tom Roche Thu Jun 25 13:47:45 MDT 2015[1]
>> a more sensible policy--ceteris paribus[2]--for the allocation of scarce resources would be layering+[division of labor (DoL)]:

>> 1. PyNIO should delegate basic netCDF4 support to netcdf4-python.

>> 2. PyNIO should focus on supporting additional/non-netCDF formats.

>> 3. PyNIO should adopt netcdf4-python as a dependency.

David Brown Mon, 29 Jun 2015 19:02:20 -0600[3]
> the new [PyNIO] development is not just for NetCDF4 but also to expose the HDF5 capabilities of the NIO library.

So have you maybe discussed merging your HDF5 code into netcdf4-python[4] with the Unidata folks[5]?

> if we knew that the NetCDF library would be providing more complete coverage of HDF5 in the near future, it still might make sense to use netcdf4-python. I am not sure why certain things seem to be missing.

So have you maybe discussed *that* with the Unidata folks? I dunno if you/CISL and they/Unidata have facetime opportunities (are they "walk-down-the-hall-able"?) but certainly they have a web interface for this[6].

Just to be clear: I'm not trying to valorize or deprecate anyone's coding abilities. I *am* trying to emphasize scarcity of coding resources (notably, person-time) available for this task and important, related others (such as testing, maintenance, packaging, documentation) that are all-too-often neglected. If my empirical claim

>> Current economic and political "realities" imply that budgets for "the sort of stuff" listizens in general do, and NCAR/UCAR specifically, are unlikely to improve significantly near-term.

is plausible, then ISTM (and you seem to agree) layering+DoL is probably the most rational response to that scarcity, in the sense that it delivers more-useful code more sustainably over the longterm. Layering+DoL is almost never the most *emotionally-satisfying* response (which all-too-often is to say "I'll just suck it up and code this *myself*!" accompanied by crotch-tugging :-) since adopting dependencies is never fun and is real work. (Lemme emphasize that, having worked for a three-letter acronym that develops very-large-scale software systems, I *know* dependency management is *not* fun.)

HTH, Tom Roche <Tom_Roche at pobox.com>

[1]: http://mailman.ucar.edu/pipermail/pyngl-talk/2015-June/000052.html
[2]: https://en.wiktionary.org/wiki/ceteris_paribus
[3]: http://mailman.ucar.edu/pipermail/pyngl-talk/2015-June/000058.html
[4]: http://unidata.github.io/netcdf4-python/
[5]: https://github.com/orgs/Unidata/people
[6]: https://github.com/Unidata/netcdf4-python/issues