statsmodels install with python 3.3

[VincentAB]

Hi, sorry to bother with this, but I can't seem to get this to work on a osx machine. Any help would be much appreciated!

1) clone repo

2) cd to local repo

3) python3 setup.py install

4) python3

5) import statsmodels.api as sm

6) Error: ("testing" directory is a virtualenv, but I get the same error when installing globally)

Python 3.3.0 (default, Jan 25 2013, 19:59:47) 

[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import statsmodels.api as sm

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/Users/username/Downloads/testing/lib/python3.3/site-packages/statsmodels-0.5.0-py3.3-macosx-10.8-x86_64.egg/statsmodels/api.py", line 3, in <module>

    from . import regression

ImportError: cannot import name regression

[josef...@gmail.com]

statsmodels is not yet compatible or verified to be compatible with python 3.3.

The problem when I checked this after python 3.3 was released was that
numpy was not compatible.
I didn't get far enough to see if scipy would work on python 3.3

I might have to wait until both numpy and scipy have python 3.3
compatible releases, because I don't have a fortran compiler that is
compatible with scipy.

On Sat, Jan 26, 2013 at 8:13 AM, VincentAB <vincen...@gmail.com> wrote:
> Hi, sorry to bother with this, but I can't seem to get this to work on a osx
> machine. Any help would be much appreciated!
>
> 1) clone repo
> 2) cd to local repo
> 3) python3 setup.py install

after this you have to cd out of the local repo

> 4) python3
> 5) import statsmodels.api as sm
> 6) Error: ("testing" directory is a virtualenv, but I get the same error
> when installing globally)
>
> Python 3.3.0 (default, Jan 25 2013, 19:59:47)
> [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on
> darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import statsmodels.api as sm
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File
> "/Users/username/Downloads/testing/lib/python3.3/site-packages/statsmodels-0.5.0-py3.3-macosx-10.8-x86_64.egg/statsmodels/api.py",
> line 3, in <module>
> from . import regression
> ImportError: cannot import name regression

This is an error message that we also get when we are in the source
tree, and may not python 3.3 specific

Josef

[Vincent Arel]

> --
>
>

I had cd-ed out of the source tree, so this is not the problem. Numpy
and Scipy were both installed from github master. While I haven't
tested them extensively, they seemed to install fine and I was able to
create arrays of random numbers and replicate a couple of scipy's
cookbook recipes from their website.

I understand if you want to wait until there's a 3.3 version out on
pip. It's just that 3.3. is the python that's distributed with
homebrew, so it would be convenient to get it to work there.
Unfortunately, I don't have a good sense of what's needed here.

Vincent

[josef...@gmail.com]

I would prefer not to wait, but I don't have binaries of scipy for
python 3.3. numpy has a 1.7 beta out, the first scipy betas should be
out in the next month.

I don't remember all the details of how I did my last compatibility work.

Essentially it's checking that 2to3 doesn't report any errors, and
then to track down where the conversion is incorrect.

What you could try to do is to import individual sub-packages or
modules instead of statsmodels.api.
statsmodels.api imports a lot and the import error could hide anywhere.

One possibility is also to run nosetests on individual parts of statsmodels.

either
import statsmodels.discrete as smd
smd.test()

or on the command line
> nosetests statsmodels.discrete
> nosetests <path_to_test_module>

My guess is that there are not many conversion errors to hunt down.

Josef


>
> Vincent
>
> --
>
>

[josef...@gmail.com]

If you find specific errors:
The way I usually proceeded was to get it to run on python 3 by making
any required changes in the python 3 virtualenv, and then "backport"
the changes, adjusting until 2to3 produces a correct result.

You can also open a ticket and copy any problems there. In most cases
I had a good guess what needed to be changed.

Josef

[josef...@gmail.com]

I didn't check http://www.lfd.uci.edu/~gohlke/pythonlibs/ in a while.
He has python 3.3 binaries for Windows including statsmodels 0.4.3

So, I can look into python 3.3 compatibility next week.
(I just checked: I still have only Python 3.3.0b1 installed from when
I wanted to check compatibility.)

If you want to give it a try first, then that would be very welcome.



Josef

[Vincent Arel]

> --
>
>

I'll see what I can do but these days are very busy with work & family stuff

Vincent

[josef...@gmail.com]

Unfortunately I'm also limited in my time, and my first attempt to
install things for python 3.3 ended in a segfault in the scipy test
suite.

Maybe full python 3.3 compatibility has still to wait a bit.

Josef

>
> Vincent

[VincentAB]

Arrgh! I wish I could delete this thread. After installing patsy and nose, statsmodels now works on 3.3.0. 

Sorry for the false alert!

Vincent 

[Ralf Gommers]

Scipy 0.11 indeed won't work; the segfault has only been fixed yesterday.

Ralf
 

[josef...@gmail.com]

Very good, Gohlke has 12.0 dev that I used but of course not nightly.

Josef

>
> Ralf
>
>
>
>>
>> Maybe full python 3.3 compatibility has still to wait a bit.
>>
>> Josef
>>
>> >
>> > Vincent
>
>

> --
>
>

[josef...@gmail.com]

That's very good news. I love it when upgrades are backwards
compatible and don't require extra work.

Did you run the statsmodels test suite?

>
> Sorry for the false alert!

Maybe false alert, but I'm glad when we get a confirmation that it
works on different versions.

Thanks,

Josef

>
> Vincent
>
> --
>
>

[Vincent Arel]

I did:

Ran 1821 tests in 313.629s

OK (SKIP=35)
Out[3]: <nose.result.TextTestResult run=1821 errors=0 failures=0>

Vincent

[josef...@gmail.com]

good.
One question I just remembered
Did it install the extensions?
you should have a tsa\kalmanf\kalman_loglike.pyd and a
nonparametric\fast_linbin.pyd

Thanks for testing

Josef

>
> Vincent

[Vincent Arel]

I'm still trying to figure out how paths work on the mac, so I'm not
100% certain of anything, but it looks like the 2 files you mention
are *not* there.

[josef...@gmail.com]

Not good. I think there are more troubles ahead.
If they are missing, then statsmodels uses the pure python fallbacks,
and we might have to investigate in more detail later.

On Windows I have problems with pandas and now with statsmodels.
Likely because of the missing microsoft c runtime, where I don't know
where I'm supposed to put it, neither pip nor easy_install work
currently for statsmodels.

(I should have waited when I have more quite time next week. It's not
a 10 minute update job.)

Josef

[Skipper Seabold]

Do you have compilers from Xcode installed with the command line tools?

Skipper

[josef...@gmail.com]

setup.py install
worked for me, including the pyd extension files

I get one precision error

======================================================================
FAIL: statsmodels.emplike.tests.test_regression.TestRegressionNM.test_ci_beta2
----------------------------------------------------------------------

Traceback (most recent call last):

File "C:\Programs\Python33\lib\site-packages\nose\case.py", line
198, in runTest
self.test(*self.arg)
File "C:\Programs\Python33\lib\site-packages\statsmodels-0.5.0-py3.3-win32.egg\statsmodels\emplike\tests\test_regression.py",
line 166, in test_ci_beta2
assert_almost_equal(ul_pval, .050000, 4)
File "C:\Programs\Python33\lib\site-packages\numpy\testing\utils.py",
line 468, in assert_almost_equal
raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 4 decimals
ACTUAL: 0.049948922342124891
DESIRED: 0.05

----------------------------------------------------------------------
Ran 1821 tests in 551.801s

FAILED (SKIP=35, failures=1)
<nose.result.TextTestResult run=1821 errors=0 failures=1>

Josef

[josef...@gmail.com]

On Sat, Jan 26, 2013 at 3:51 PM, Skipper Seabold <jsse...@gmail.com> wrote:
> On Sat, Jan 26, 2013 at 3:25 PM, Vincent Arel <vincen...@gmail.com>
> wrote:
>>
>> On Sat, Jan 26, 2013 at 2:11 PM, <josef...@gmail.com> wrote:
>> >
>> > One question I just remembered
>> > Did it install the extensions?
>> > you should have a tsa\kalmanf\kalman_loglike.pyd and a
>> > nonparametric\fast_linbin.pyd
>> >
>> > Thanks for testing
>> >
>> > Josef
>> >
>> >>
>> >> Vincent
>>
>> I'm still trying to figure out how paths work on the mac, so I'm not
>> 100% certain of anything, but it looks like the 2 files you mention
>> are *not* there.

I tried to check the presence of the extension by loading them, but I get

>>> import statsmodels.nonparametric.fast_linbin as tmp

Traceback (most recent call last):
File "<stdin>", line 1, in <module>

File "fast_linbin.pyx", line 9, in init
statsmodels.nonparametric.fast_linbin
(statsmodels\nonparametric\fast_linbin.c:3591)
ValueError: level must be >= 0
>>> import statsmodels.tsa.kalmanf.kalman_loglike as tmp

Traceback (most recent call last):
File "<stdin>", line 1, in <module>

File "kalman_loglike.pyx", line 3, in init
statsmodels.tsa.kalmanf.kalman_loglike
(statsmodels\tsa\kalmanf\kalman_loglike.c:6325)
ValueError: level must be >= 0
>>> from statsmodels.nonparametric.fast_linbin import linbin

Traceback (most recent call last):
File "<stdin>", line 1, in <module>

File "fast_linbin.pyx", line 9, in init
statsmodels.nonparametric.fast_linbin
(statsmodels\nonparametric\fast_linbin.c:3591)
ValueError: level must be >= 0

No idea what they mean, I have the right path
>>> import statsmodels.tsa.kalmanf as tmp
>>> tmp.__file__
'C:\\Programs\\Python33\\lib\\site-packages\\statsmodels-0.5.0-py3.3-win32.egg\\statsmodels\\tsa\\kalmanf\\__init__.py'

Josef

>
>
> Do you have compilers from Xcode installed with the command line tools?
>
> Skipper
>

> --
>
>

[Skipper Seabold]

On Sat, Jan 26, 2013 at 4:08 PM, <josef...@gmail.com> wrote:

On Sat, Jan 26, 2013 at 3:51 PM, Skipper Seabold <jsse...@gmail.com> wrote:
> On Sat, Jan 26, 2013 at 3:25 PM, Vincent Arel <vincen...@gmail.com>
> wrote:
>>
>> On Sat, Jan 26, 2013 at 2:11 PM,  <josef...@gmail.com> wrote:
>> >
>> > One question I just remembered
>> > Did it install the extensions?
>> > you should have a tsa\kalmanf\kalman_loglike.pyd and a
>> > nonparametric\fast_linbin.pyd
>> >
>> > Thanks for testing
>> >
>> > Josef
>> >
>> >>
>> >> Vincent
>>
>> I'm still trying to figure out how paths work on the mac, so I'm not
>> 100% certain of anything, but it looks like the 2 files you mention
>> are *not* there.

I tried to check the presence of the extension by loading them, but I get

I don't understand what you did to install. What did you use to build the extensions. Did you use mingw32 or windows sdk?

Skipper

[Matthew Brett]

Hi,

Sorry to leap in without understanding, but I've seen that error from
Cython extensions compiled with Cython < 0.17 and run in Python 3.3.
Fixing that for scipy was just recompiling the pyx files with Cython
0.17.

Cheers,

Matthew

[josef...@gmail.com]

I did just the basic
setup.py install
in the source dir with mingw32 as compiler

I didn't clean the source dir, so there is still a chance that it got
something wrong.

I just checked the compiled extension is not used. the try ... except
must be catching the exception.

Josef

>
> Skipper
>
> --
>
>

[josef...@gmail.com]

Very good, thanks

my cython is quite a bit older than 0.17

Josef
>
> Cheers,
>
> Matthew

[josef...@gmail.com]

That was it

One problem I had remaining was that 2to3 didn't add the . in the
import, and we still have a caught import error out of the box:
from fast_linbin import linbin
instead of
from .fast_linbin import linbin

>>> import statsmodels.nonparametric.kde as tmp

Traceback (most recent call last):
File "<stdin>", line 1, in <module>

File "C:\Programs\Python33\lib\site-packages\statsmodels-0.5.0-py3.3-win32.egg\statsmodels\nonparametric\kde.py",
line 24, in <module>
from fast_linbin import linbin
ImportError: No module named 'fast_linbin'


>>> import statsmodels.nonparametric.kde as tmp
>>> tmp.linbin
<built-in function linbin>
>>> tmp.linbin.__module__
'statsmodels.nonparametric.fast_linbin'

So I have a python 3.3 statsmodels, for 32bit Windows only so far.

Josef

>
> Josef
>>
>> Cheers,
>>
>> Matthew

[Vincent Arel]

> --
>
>

Yes, I believe that both of these things are properly installed, but I
have never worked with a mac before, so we should probably confirm
with a more experienced user before concluding there's a problem here.