[SciPy-user] Running SciPy SVN without installing Was: Matlab IO -- can others with matlab test this bug?
Jonathan Hunt
Thanks for the advice. But all of these seem like hacks. Is there any
clean way to use SVN SciPy without installing it. This is particularly
important since I'm hoping to modify SciPy (I want to add a mio
wrapper for MATLAB 7/HDF5 files) and obviously I would like to avoid
having the build/install everytime I make a modification to the
library that I want to test.
Thanks,
Jonny
On Tue, Jul 29, 2008 at 10:37 PM, David Cournapeau
<da...@ar.media.kyoto-u.ac.jp> wrote:
> Zachary Pincus wrote:
>>
>> Perhaps some others can chime in with more "official" methods, but I
>> typically do one of several things to test SVN versions:
>>
>
> On Unix, a nice and easy way to easily switch between different numpy
> installs without touching at all the environment is stow. Basically what
> you do once stow is installed is:
>
> python setup.py install --prefix=SOMEPATH/stow/numpy1
> python setup.py install --prefix=SOMEPATH/stow/numpy2
>
> And then stow makes link between the 'real' numpy and the one you want:
>
> stow numpy1 -> numpy1 is used
> stow -R numpy1 -> numpy1 is disabled
> stow numpy2 -> numpy2 is enabled
>
> I switch between many numpy configurations (one which only uses released
> versions for 'real' work, many other with various compilers combination
> to test the build system). It also enables true uninstallation, since
> any installed version is self-contained in one directory (you can rm -rf
> the directory without any chance of removing something unrelated).
>
> cheers,
>
> David
--
Jonathan J Hunt <j...@42quarks.com>
Homepage: http://www.42quarks.net.nz/wiki/JJH
(Further contact details there)
"Physics isn't the most important thing. Love is." Richard Feynman
_______________________________________________
SciPy-user mailing list
SciPy...@scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-user
David Cournapeau
> Hi,
>
> Thanks for the advice. But all of these seem like hacks.
stow is certainly not an hack, particularly compared to not installing
scipy. There is the possibility to use the develop mode of setuptools,
or varisous "sandboxed" python, but they are likely to be more work, and
I don't know anything about them, so I can't really comment on them (I
know other scipy developers do, though).
> Is there any
> clean way to use SVN SciPy without installing it. This is particularly
> important since I'm hoping to modify SciPy (I want to add a mio
> wrapper for MATLAB 7/HDF5 files) and obviously I would like to avoid
> having the build/install everytime I make a modification to the
> library that I want to test.
>
This is not obvious. For most softwares, you cannot test them without
'installing' them, why should it be different for python packages ? Any
method enabling the test of the software without installing it (besides
sandbox) is an hack by definition.
cheers,
David
Emanuele Olivetti
(in bash, continuing the Zachary's example on numpy1)
export PYTHONPATH="SOMEPATH/stow/numpy1"
then run python/emacs/ipython/whatever and use numpy1.
Emanuele
_______________________________________________
David Cournapeau
> Insted of stow why not using just PYTHONPATH?
> (in bash, continuing the Zachary's example on numpy1)
> export PYTHONPATH="SOMEPATH/stow/numpy1"
> then run python/emacs/ipython/whatever and use numpy1.
>
it breaks (or more exactly becomes complicated) once you depend on
several interdependent packages (say numpy and scipy, for example).
Granted, it may not be a problem for the problem encountered by John,
but I am always surprised by the number of people who use - buggy -
hacks to do something for which stow was invented.
Jonathan Hunt
Firstly David, hack was not intended as an insult. Thanks for pointing
me to stow I haven't heard of it before.
However. Could someone tell me how most SciPy developers work on the
library. Because stow still doesn't seem to get around the need to do
a build/install everytime one makes a modification to the library. I
look at online documentation for developers but I can't find any
clues. What's the best way to modify/test SciPy SVN?
Thanks,
Jonny
--
Jonathan J Hunt <j...@42quarks.com>
Homepage: http://www.42quarks.net.nz/wiki/JJH
(Further contact details there)
"Physics isn't the most important thing. Love is." Richard Feynman
Pierre GM
Hopefully you won't need to modify a lot of files all over Scipy, but just a
few of them ? Then you may want to install the SVN version and create
symbolic links pointing to the files/directories you'll be actually
modifying.
That works well, may come to bite you if you don't delete an old installation
of Scipy before installing a new one, and is a bit problematic from times to
times, but (goto 1)...
David Cournapeau
> Hi,
>
> Firstly David, hack was not intended as an insult.
I did not take it as an insult, don't worry. I just pointed out that any
supposedly non hackish thing is more hackish than stow.
>
> However. Could someone tell me how most SciPy developers work on the
> library.
Personally, that's how I develop on scipy and numpy.
> Because stow still doesn't seem to get around the need to do
> a build/install everytime one makes a modification to the library.
I think that getting around this need brings more problems than it
solves. I always install and test from the installed version; in most
cases, I even clean the installed version first because this often leads
to subtle problems. Installing numpy/scipy like that takes a few
seconds, and is easy to automate by make, rake files if I need
portability on Windows, etc...
Jonathan Hunt
Thanks for all the comments. If anyone uses the develop mode of
setuptools or other methods I'd be interested to hear.
Also, might I suggest (apologies if they do exist somewhere and I
missed them) that the (summarized) contents of this thread might be
useful on the SciPy developers wiki to lower the barrier for others
getting started hacking SciPy.
Thanks for all the help so far,
Jonny
--
Jonathan J Hunt <j...@42quarks.com>
Homepage: http://www.42quarks.net.nz/wiki/JJH
(Further contact details there)
"Physics isn't the most important thing. Love is." Richard Feynman
David Cournapeau
> Jonathan,
>
> Hopefully you won't need to modify a lot of files all over Scipy, but just a
> few of them ? Then you may want to install the SVN version and create
> symbolic links pointing to the files/directories you'll be actually
> modifying.
>
That's precisely how stow works ...
> That works well, may come to bite you if you don't delete an old installation
> of Scipy before installing a new one, and is a bit problematic from times to
> times, but (goto 1)...
>
... except that it solves this problem too :) Really, stow is nice. The
only problem is that it does not work on windows.
cheers,
David
Pierre GM
> Pierre GM wrote:
> > Then you may want to install the SVN version and
> > create symbolic links pointing to the files/directories you'll be
> > actually modifying.
>
> That's precisely how stow works ...
OK, I just found about stow. I gonna have to try. Thanks for the info.
Gael Varoquaux
> However. Could someone tell me how most SciPy developers work on the
> library.
I don't work on scipy, but on other project I use a combination of
virtualenv and setuptools develop feature, giving it the right prefix. It
adds some overheard indeed.
I rerun python setup.py develop when there are any changes to the C code.
I do this because I need to be able to run different branches of the of
the same project in the same time to check for differences.
Prabhu has explained the way he uses virtualenv here:
http://prabhuramachandran.blogspot.com/2008/03/using-virtualenv-under-linux.html
I do things a bit different. One of the big difference is the instead of
python setup.py install, I use python setup.py develop.
All these approach tend to have there own quirks so you have to get to
know them.
Gaël
Gael Varoquaux
> Really, stow is nice. The
> only problem is that it does not work on windows.
Can I have two shells, one running one environment, and the other
another, say to compare results with stow?
Gaël
Gael Varoquaux
> Thanks for all the comments. If anyone uses the develop mode of
> setuptools or other methods I'd be interested to hear.
I do. I am not sure what to tell you about it. Just be aware that when
you use setuptools, your import path gets messed up (print sys.path to
reaize that), and as a result import priorities get scrambled. If you
know that and keep watch on what is important, when you have several
version of different packages installed, you can usual understand what is
going on.
Gaël
Jonathan Hunt
SciPy just uses distutils it doesn't have develop mode. Is there an
easy way to make it use setuptools?
Thanks,
Jonny
--
Jonathan J Hunt <j...@42quarks.com>
Homepage: http://www.42quarks.net.nz/wiki/JJH
(Further contact details there)
"Physics isn't the most important thing. Love is." Richard Feynman
Gael Varoquaux
> Thanks for the help. I tried using develop mode in SciPy but since
> SciPy just uses distutils it doesn't have develop mode. Is there an
> easy way to make it use setuptools?
Use setupegg.py instead of setup.py.
David Cournapeau
> On Tue, Jul 29, 2008 at 11:08:57PM +0900, David Cournapeau wrote:
>
>> Really, stow is nice. The
>> only problem is that it does not work on windows.
>>
>
> Can I have two shells, one running one environment, and the other
> another, say to compare results with stow?
>
Sure, just get one stow environment per shell, and set PYTHONPATH
differently for each one (for example by using an alias
pythonfoo='PYTHONPATH=foo python'). The great advantage of stow is its
simplicity. If you screw up something, you just remove the directory,
there is no side effect at all. In a way, stow fixes one of the things
which I really hate with unix, the FHS.
cheers,
David