[pyngl-talk] pyngl 1.5.0 source
Radovan Bast
radovan.bast at gmail.com
Mon Jan 5 09:03:21 MST 2015
hi,
so AFAICS there are 3 issues:
1) suggestion to go open source
2) suggestion to track code and issues on GitHub or SourceForge or similar
3) pip-ify/streamline build/install process
these are in principle independent. 2) requires 1) if the hosting is
supposed
to be public.
i also think that 1) and 2) will simplify 3) because then external
developers
with expertise in the build mechanism can join the effort.
in order to perhaps help in the discussion/decision about 2), how are the
sources
and issues tracked/managed as of now? how is the code currently managed?
i don't want to start a flame war on Git vs. Subversion and GitHub vs.
other sites
but a number of projects that i have been involved in have transitioned
in small steps via Subversion etc. but are eventually
converging to GitHub and if there is interest
i can argue why. at least for larger teams every transition is possibly
disruptive
so i am not convinced that a detour via Subversion is helpful because the
workflow
is very different from Git.
best wishes,
radovan
On Tue, Dec 30, 2014 at 9:14 PM, Root, Benjamin <broot at aer.com> wrote:
> Oleksandr,
>
> I wouldn't go so far as to start compiling other projects as part of
> your setup.py, but I see what you are getting at. I would avoid mixing
> makefiles and setup.py, though. If you want to go the makefile route
> for python, there is cmake.
>
> Mary,
>
> Don't let perfect be the enemy of "good enough". Take small steps.
> Anything would be an improvement over what is happening now. If you are
> not ready for the move to git, I would recommend using Sourceforge,
> which has greatly improved since the "mass exodus" of projects to
> github four years ago. It supports both git and subversion. This can
> make the transition to git easier in the future when you are ready to
> do so. The important thing is to get these projects opened up and
> receiving patches from the community, and to track bug reports.
>
> Once these projects are opened up, we can then work on getting the
> distribution side of things fixed through contributions from those with
> experience. For example, I am one of the core developers for
> matplotlib. The matplotlib project has to support Linux, Windows, and
> Macs, on py2.6, 2.7, 3.3 and up. We also have an (overly-)complicated
> build system and dependencies on both python and non-python packages.
> If we get the code up onto sourceforge, it would be much easier for
> someone like me to be able to dive in and start applying my knowledge
> to your project.
>
> As for pip. pip will only build a package from source if a binary wheel
> (or the older style eggs) are not available for the system making the
> pip request (or Linux, for complicated reasons). So, you can upload
> binary builds of the packages to PyPi as well as the source eggs to
> maximize the chances of a successful install. binstar is a different
> thing and i would worry about one thing at a time. Even if you are not
> ready to release binary builds of the projects to pypi, it would not
> hurt one bit to get the source eggs up on there. At the very least,
> snag the relevant project names before someone else does! This happened
> to the iris project, and causes a lot of confusion for newcomers.
>
> You don't know how happy it makes me that this idea is even being
> entertained! I really hope that the team follows through with this,
> even if it is in baby steps. Do let me know if you have questions about
> the transition process. It can be very scary, but quite liberating once
> it is done.
>
> Cheers!
> Ben Root
>
>
> On Sat 27 Dec 2014 12:44:46 PM EST, Oleksandr Huziy wrote:
> > Hi Mary:
> >
> > I am not a pip expert but what I know is:
> >
> > pip is just executing your setup.py, what I do for one of my projects
> > is: add a call to make and package the
> > Makefile...(https://github.com/guziy/pylibrmn/blob/master/setup.py)
> >
> > There is a package netcdf4-python on github, it depends on hdf5 and
> > netcdf and just checks if those 2 are installed either by looking at
> > env variables HDF5_DIR, NETCDF4_DIR, or using config utilities
> > nc-config .... https://github.com/Unidata/netcdf4-python
> >
> > What I would do is just list those dependencies, and the way they are
> > supposed to be built, compiler versions and flags, and install only
> > wrappers and python codes using pip...
> >
> > Cheers
> >
> >
> >
> >
> >
> >
> >
> > 2014-12-27 11:44 GMT-05:00 Mary Haley <haley at ucar.edu
> > <mailto:haley at ucar.edu>>:
> >
> > Hi all,
> >
> > All of the NCL, PyNGL, and PyNIO code is currently under
> > subversion. We have talked about moving to other source code
> > control systems, but would need to do all three packages
> > simultaneously, and just haven't had the time yet.
> >
> > These three software packages depend on about 20 external software
> > packages, several of which have different build systems, and we've
> > been trying to see what it would take to get these all under some
> > common build system. Many of these packages have to be built a
> > specific way with particular options turned off, so they don't
> > clash with other packages. This means we can't use generically
> > built packages that are provided by other vendors.
> >
> > The main issue for me is my admitted lack of knowledge on how pip
> > works. I assuming that all the external packages that PyNIO and
> > PyNGL depend on would also have to be either prebuilt on many
> > systems, or would have to be readily buildable via source code.
> > Several of these packages are not under github either.
> >
> > The packages that are required include:
> >
> > cairo
> > curl
> > esmf
> > expat
> > freetype
> > g2clib
> > gdal
> > hdf
> > hdf5
> > hdfeos
> > hdfeos5
> > jasper
> > jpeg
> > libpng
> > libxml2
> > netcdf
> > pixman
> > pkg-config
> > proj
> > szip
> > udunits
> > zlib
> >
> >
> > As a side: I used conda to install different versions of Python
> > and NumPy, and see that it has a free service called Binstar for
> > hosting software for package managers. I had to give up on conda,
> > because it was using the old, default compiler on my Mac, and I
> > couldn't build my other packages with this compiler. I realize
> > there's probably a way to customize conda so that it uses a newer
> > compiler, but I haven't tried this yet.
> >
> > Given my lack of experience with python installers, I would love
> > to hear from the python community about how we could best proceed
> > with getting all of these packages to work with a python installer
> > like pip. It seems there's more than one way to do this, but it's
> > not clear to me what is the best way.
> >
> > Thanks for the discussion!
> >
> > --Mary
> >
> >
> > On Thu, Dec 18, 2014 at 7:42 AM, Root, Benjamin <broot at aer.com
> > <mailto:broot at aer.com>> wrote:
> >
> > +1 for posting to github or any one of the other code
> > collaboration
> > sites like gitlab, sourceforge (which supports both svn and
> > git), etc.
> >
> > Same thing, if possible, for PyNIO, please? One of the biggest
> > barriers
> > for me recommending PyNIO to others has been the lack of the
> > ability to
> > do "pip install pynio".
> >
> > Cheers!
> > Ben Root
> >
> > On Wed 17 Dec 2014 05:15:41 PM EST, Radovan Bast wrote:
> > > +1
> > > i wanted to ask the exact same question almost the exact same
> moment.
> > > i think PyNGL is a great code and i believe it would greatly
> benefit
> > > to be placed under GitHub, then everybody could submit patches
> > > and it would be less work for Mary et al.
> > > best greetings,
> > > radovan
> > >
> > > On Wed, Dec 17, 2014 at 10:26 PM, Oleksandr Huziy
> > > <guziy.sasha at gmail.com <mailto:guziy.sasha at gmail.com>
> > <mailto:guziy.sasha at gmail.com <mailto:guziy.sasha at gmail.com>>>
> > wrote:
> > >
> > > Is there a reason why you are not putting it on github?
> > >
> > > Cheers
> > >
> > > 2014-12-17 15:59 GMT-05:00 Mary Haley <haley at ucar.edu
> <mailto:haley at ucar.edu>
> > > <mailto:haley at ucar.edu <mailto:haley at ucar.edu>>>:
> > >
> > > Thanks Orion, I'll take a look at the code and
> implement these
> > > changes.
> > >
> > > --Mary
> > >
> > >
> > > On Tue, Dec 16, 2014 at 3:32 PM, Orion Poplawski
> > > <orion at cora.nwra.com <mailto:orion at cora.nwra.com>
> > <mailto:orion at cora.nwra.com <mailto:orion at cora.nwra.com>>>
> wrote:
> > >
> > > On 12/16/2014 10:50 AM, Mary Haley wrote:
> > > > Hi Orion,
> > > >
> > > > Sure thing, I just put it on the web:
> > > >
> > > >
> http://www.pyngl.ucar.edu/Download/files/PyNGL-1.5.0-beta.tar.gz
> > >
> > > One other nit - you should exit with a non-zero
> value on
> > > error in setup.py:
> > >
> > > e.g.:
> > >
> > > except ImportError:
> > > print "Error: Cannot import NumPy. Can't
> continue."
> > > sys.exit(1)
> > >
> > > except:
> > > print "NCARG_ROOT is not set; can't continue!"
> > > sys.exit(1)
> > >
> > >
> > > --
> > > Orion Poplawski
> > > Technical Manager 303-415-9701 x222
> > <tel:303-415-9701%20x222> <tel:303-415-9701%20x222>
> > > NWRA, Boulder/CoRA Office FAX:
> 303-415-9702 <tel:303-415-9702>
> > > <tel:303-415-9702 <tel:303-415-9702>>
> > > 3380 Mitchell Lane orion at nwra.com
> > <mailto:orion at nwra.com> <mailto:orion at nwra.com
> > <mailto:orion at nwra.com>>
> > > Boulder, CO 80301http://www.nwra.com
> > >
> > >
> > > _______________________________________________
> > > pyngl-talk mailing list
> > > List instructions, subscriber options, unsubscribe:
> > >http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
> > >
> > >
> > >
> > > --
> > > Sasha
> > >
> > > _______________________________________________
> > > pyngl-talk mailing list
> > > List instructions, subscriber options, unsubscribe:
> > >http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
> > >
> >
> > ________________________________
> >
> > This email is intended solely for the recipient. It may
> > contain privileged, proprietary or confidential information or
> > material. If you are not the intended recipient, please delete
> > this email and any attachments and notify the sender of the
> error.
> >
> > _______________________________________________
> > pyngl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
> >
> >
> >
> > _______________________________________________
> > pyngl-talk mailing list
> > List instructions, subscriber options, unsubscribe:
> > http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
> >
> >
> >
> >
> > --
> > Sasha
>
> ________________________________
>
> This email is intended solely for the recipient. It may contain
> privileged, proprietary or confidential information or material. If you are
> not the intended recipient, please delete this email and any attachments
> and notify the sender of the error.
>
> _______________________________________________
> pyngl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/pyngl-talk/attachments/20150105/1df67fdf/attachment-0001.html
More information about the pyngl-talk
mailing list