<div dir="ltr">hi,<div><br></div><div>so AFAICS there are 3 issues:</div><div><br></div><div>1) suggestion to go open source</div><div>2) suggestion to track code and issues on GitHub or SourceForge or similar</div><div>3) pip-ify/streamline build/install process</div><div><br></div><div>these are in principle independent. 2) requires 1) if the hosting is supposed</div><div>to be public.</div><div><br></div><div>i also think that 1) and 2) will simplify 3) because then external developers</div><div>with expertise in the build mechanism can join the effort.</div><div><br></div><div>in order to perhaps help in the discussion/decision about 2), how are the sources</div><div>and issues tracked/managed as of now? how is the code currently managed?</div><div><br></div><div>i don't want to start a flame war on Git vs. Subversion and GitHub vs. other sites</div><div>but a number of projects that i have been involved in have transitioned</div><div>in small steps via Subversion etc. but are eventually</div><div>converging to GitHub and if there is interest</div><div>i can argue why. at least for larger teams every transition is possibly disruptive</div><div>so i am not convinced that a detour via Subversion is helpful because the workflow</div><div>is very different from Git.</div><div><br></div><div>best wishes,</div><div> radovan</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 30, 2014 at 9:14 PM, Root, Benjamin <span dir="ltr"><<a href="mailto:broot@aer.com" target="_blank">broot@aer.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Oleksandr,<br>
<br>
I wouldn't go so far as to start compiling other projects as part of<br>
your setup.py, but I see what you are getting at. I would avoid mixing<br>
makefiles and setup.py, though. If you want to go the makefile route<br>
for python, there is cmake.<br>
<br>
Mary,<br>
<br>
Don't let perfect be the enemy of "good enough". Take small steps.<br>
Anything would be an improvement over what is happening now. If you are<br>
not ready for the move to git, I would recommend using Sourceforge,<br>
which has greatly improved since the "mass exodus" of projects to<br>
github four years ago. It supports both git and subversion. This can<br>
make the transition to git easier in the future when you are ready to<br>
do so. The important thing is to get these projects opened up and<br>
receiving patches from the community, and to track bug reports.<br>
<br>
Once these projects are opened up, we can then work on getting the<br>
distribution side of things fixed through contributions from those with<br>
experience. For example, I am one of the core developers for<br>
matplotlib. The matplotlib project has to support Linux, Windows, and<br>
Macs, on py2.6, 2.7, 3.3 and up. We also have an (overly-)complicated<br>
build system and dependencies on both python and non-python packages.<br>
If we get the code up onto sourceforge, it would be much easier for<br>
someone like me to be able to dive in and start applying my knowledge<br>
to your project.<br>
<br>
As for pip. pip will only build a package from source if a binary wheel<br>
(or the older style eggs) are not available for the system making the<br>
pip request (or Linux, for complicated reasons). So, you can upload<br>
binary builds of the packages to PyPi as well as the source eggs to<br>
maximize the chances of a successful install. binstar is a different<br>
thing and i would worry about one thing at a time. Even if you are not<br>
ready to release binary builds of the projects to pypi, it would not<br>
hurt one bit to get the source eggs up on there. At the very least,<br>
snag the relevant project names before someone else does! This happened<br>
to the iris project, and causes a lot of confusion for newcomers.<br>
<br>
You don't know how happy it makes me that this idea is even being<br>
entertained! I really hope that the team follows through with this,<br>
even if it is in baby steps. Do let me know if you have questions about<br>
the transition process. It can be very scary, but quite liberating once<br>
it is done.<br>
<br>
Cheers!<br>
Ben Root<br>
<span class=""><br>
<br>
On Sat 27 Dec 2014 12:44:46 PM EST, Oleksandr Huziy wrote:<br>
> Hi Mary:<br>
><br>
> I am not a pip expert but what I know is:<br>
><br>
> pip is just executing your setup.py, what I do for one of my projects<br>
> is: add a call to make and package the<br>
> Makefile...(<a href="https://github.com/guziy/pylibrmn/blob/master/setup.py" target="_blank">https://github.com/guziy/pylibrmn/blob/master/setup.py</a>)<br>
><br>
> There is a package netcdf4-python on github, it depends on hdf5 and<br>
> netcdf and just checks if those 2 are installed either by looking at<br>
> env variables HDF5_DIR, NETCDF4_DIR, or using config utilities<br>
> nc-config .... <a href="https://github.com/Unidata/netcdf4-python" target="_blank">https://github.com/Unidata/netcdf4-python</a><br>
><br>
> What I would do is just list those dependencies, and the way they are<br>
> supposed to be built, compiler versions and flags, and install only<br>
> wrappers and python codes using pip...<br>
><br>
> Cheers<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> 2014-12-27 11:44 GMT-05:00 Mary Haley <<a href="mailto:haley@ucar.edu">haley@ucar.edu</a><br>
</span>> <mailto:<a href="mailto:haley@ucar.edu">haley@ucar.edu</a>>>:<br>
<div><div class="h5">><br>
> Hi all,<br>
><br>
> All of the NCL, PyNGL, and PyNIO code is currently under<br>
> subversion. We have talked about moving to other source code<br>
> control systems, but would need to do all three packages<br>
> simultaneously, and just haven't had the time yet.<br>
><br>
> These three software packages depend on about 20 external software<br>
> packages, several of which have different build systems, and we've<br>
> been trying to see what it would take to get these all under some<br>
> common build system. Many of these packages have to be built a<br>
> specific way with particular options turned off, so they don't<br>
> clash with other packages. This means we can't use generically<br>
> built packages that are provided by other vendors.<br>
><br>
> The main issue for me is my admitted lack of knowledge on how pip<br>
> works. I assuming that all the external packages that PyNIO and<br>
> PyNGL depend on would also have to be either prebuilt on many<br>
> systems, or would have to be readily buildable via source code.<br>
> Several of these packages are not under github either.<br>
><br>
> The packages that are required include:<br>
><br>
> cairo<br>
> curl<br>
> esmf<br>
> expat<br>
> freetype<br>
> g2clib<br>
> gdal<br>
> hdf<br>
> hdf5<br>
> hdfeos<br>
> hdfeos5<br>
> jasper<br>
> jpeg<br>
> libpng<br>
> libxml2<br>
> netcdf<br>
> pixman<br>
> pkg-config<br>
> proj<br>
> szip<br>
> udunits<br>
> zlib<br>
><br>
><br>
> As a side: I used conda to install different versions of Python<br>
> and NumPy, and see that it has a free service called Binstar for<br>
> hosting software for package managers. I had to give up on conda,<br>
> because it was using the old, default compiler on my Mac, and I<br>
> couldn't build my other packages with this compiler. I realize<br>
> there's probably a way to customize conda so that it uses a newer<br>
> compiler, but I haven't tried this yet.<br>
><br>
> Given my lack of experience with python installers, I would love<br>
> to hear from the python community about how we could best proceed<br>
> with getting all of these packages to work with a python installer<br>
> like pip. It seems there's more than one way to do this, but it's<br>
> not clear to me what is the best way.<br>
><br>
> Thanks for the discussion!<br>
><br>
> --Mary<br>
><br>
><br>
> On Thu, Dec 18, 2014 at 7:42 AM, Root, Benjamin <<a href="mailto:broot@aer.com">broot@aer.com</a><br>
</div></div><span class="">> <mailto:<a href="mailto:broot@aer.com">broot@aer.com</a>>> wrote:<br>
><br>
> +1 for posting to github or any one of the other code<br>
> collaboration<br>
> sites like gitlab, sourceforge (which supports both svn and<br>
> git), etc.<br>
><br>
> Same thing, if possible, for PyNIO, please? One of the biggest<br>
> barriers<br>
> for me recommending PyNIO to others has been the lack of the<br>
> ability to<br>
> do "pip install pynio".<br>
><br>
> Cheers!<br>
> Ben Root<br>
><br>
> On Wed 17 Dec 2014 05:15:41 PM EST, Radovan Bast wrote:<br>
> > +1<br>
> > i wanted to ask the exact same question almost the exact same moment.<br>
> > i think PyNGL is a great code and i believe it would greatly benefit<br>
> > to be placed under GitHub, then everybody could submit patches<br>
> > and it would be less work for Mary et al.<br>
> > best greetings,<br>
> > radovan<br>
> ><br>
> > On Wed, Dec 17, 2014 at 10:26 PM, Oleksandr Huziy<br>
> > <<a href="mailto:guziy.sasha@gmail.com">guziy.sasha@gmail.com</a> <mailto:<a href="mailto:guziy.sasha@gmail.com">guziy.sasha@gmail.com</a>><br>
</span>> <mailto:<a href="mailto:guziy.sasha@gmail.com">guziy.sasha@gmail.com</a> <mailto:<a href="mailto:guziy.sasha@gmail.com">guziy.sasha@gmail.com</a>>>><br>
<span class="">> wrote:<br>
> ><br>
> > Is there a reason why you are not putting it on github?<br>
> ><br>
> > Cheers<br>
> ><br>
> > 2014-12-17 15:59 GMT-05:00 Mary Haley <<a href="mailto:haley@ucar.edu">haley@ucar.edu</a> <mailto:<a href="mailto:haley@ucar.edu">haley@ucar.edu</a>><br>
</span>> > <mailto:<a href="mailto:haley@ucar.edu">haley@ucar.edu</a> <mailto:<a href="mailto:haley@ucar.edu">haley@ucar.edu</a>>>>:<br>
<span class="">> ><br>
> > Thanks Orion, I'll take a look at the code and implement these<br>
> > changes.<br>
> ><br>
> > --Mary<br>
> ><br>
> ><br>
> > On Tue, Dec 16, 2014 at 3:32 PM, Orion Poplawski<br>
> > <<a href="mailto:orion@cora.nwra.com">orion@cora.nwra.com</a> <mailto:<a href="mailto:orion@cora.nwra.com">orion@cora.nwra.com</a>><br>
</span><span class="">> <mailto:<a href="mailto:orion@cora.nwra.com">orion@cora.nwra.com</a> <mailto:<a href="mailto:orion@cora.nwra.com">orion@cora.nwra.com</a>>>> wrote:<br>
> ><br>
> > On 12/16/2014 10:50 AM, Mary Haley wrote:<br>
> > > Hi Orion,<br>
> > ><br>
> > > Sure thing, I just put it on the web:<br>
> > ><br>
> > ><a href="http://www.pyngl.ucar.edu/Download/files/PyNGL-1.5.0-beta.tar.gz" target="_blank">http://www.pyngl.ucar.edu/Download/files/PyNGL-1.5.0-beta.tar.gz</a><br>
> ><br>
> > One other nit - you should exit with a non-zero value on<br>
> > error in setup.py:<br>
> ><br>
> > e.g.:<br>
> ><br>
> > except ImportError:<br>
> > print "Error: Cannot import NumPy. Can't continue."<br>
> > sys.exit(1)<br>
> ><br>
> > except:<br>
> > print "NCARG_ROOT is not set; can't continue!"<br>
> > sys.exit(1)<br>
> ><br>
> ><br>
> > --<br>
> > Orion Poplawski<br>
> > Technical Manager <a href="tel:303-415-9701%20x222" value="+13034159701">303-415-9701 x222</a><br>
</span>> <tel:303-415-9701%20x222> <tel:303-415-9701%20x222><br>
<span class="">> > NWRA, Boulder/CoRA Office FAX:<a href="tel:303-415-9702" value="+13034159702">303-415-9702</a> <tel:<a href="tel:303-415-9702" value="+13034159702">303-415-9702</a>><br>
</span>> > <tel:<a href="tel:303-415-9702" value="+13034159702">303-415-9702</a> <tel:<a href="tel:303-415-9702" value="+13034159702">303-415-9702</a>>><br>
> > 3380 Mitchell Lane <a href="mailto:orion@nwra.com">orion@nwra.com</a><br>
> <mailto:<a href="mailto:orion@nwra.com">orion@nwra.com</a>> <mailto:<a href="mailto:orion@nwra.com">orion@nwra.com</a><br>
> <mailto:<a href="mailto:orion@nwra.com">orion@nwra.com</a>>><br>
> > Boulder, CO 80301<a href="http://www.nwra.com" target="_blank">http://www.nwra.com</a><br>
<div class="HOEnZb"><div class="h5">> ><br>
> ><br>
> > _______________________________________________<br>
> > pyngl-talk mailing list<br>
> > List instructions, subscriber options, unsubscribe:<br>
> ><a href="http://mailman.ucar.edu/mailman/listinfo/pyngl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/pyngl-talk</a><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > Sasha<br>
> ><br>
> > _______________________________________________<br>
> > pyngl-talk mailing list<br>
> > List instructions, subscriber options, unsubscribe:<br>
> ><a href="http://mailman.ucar.edu/mailman/listinfo/pyngl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/pyngl-talk</a><br>
> ><br>
><br>
> ________________________________<br>
><br>
> This email is intended solely for the recipient. It may<br>
> contain privileged, proprietary or confidential information or<br>
> material. If you are not the intended recipient, please delete<br>
> this email and any attachments and notify the sender of the error.<br>
><br>
> _______________________________________________<br>
> pyngl-talk mailing list<br>
> List instructions, subscriber options, unsubscribe:<br>
> <a href="http://mailman.ucar.edu/mailman/listinfo/pyngl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/pyngl-talk</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> pyngl-talk mailing list<br>
> List instructions, subscriber options, unsubscribe:<br>
> <a href="http://mailman.ucar.edu/mailman/listinfo/pyngl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/pyngl-talk</a><br>
><br>
><br>
><br>
><br>
> --<br>
> Sasha<br>
<br>
________________________________<br>
<br>
This email is intended solely for the recipient. It may contain privileged, proprietary or confidential information or material. If you are not the intended recipient, please delete this email and any attachments and notify the sender of the error.<br>
<br>
_______________________________________________<br>
pyngl-talk mailing list<br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/pyngl-talk" target="_blank">http://mailman.ucar.edu/mailman/listinfo/pyngl-talk</a><br>
</div></div></blockquote></div><br></div>