pip install broken for python 3.3

[Matthew Brett]

Hi,

I think pip on Python 3.3 is picking up the wrong version of Sympy:

http://nipy.bic.berkeley.edu/builders/nipy-py3.3/builds/4/steps/shell_2/logs/stdio

Exploring, this fails with the above error:

pip install sympy (for python 3.3)

These succeed:

pip install sympy==0.7.2 (for python 3.3)
easy_install sympy (for python 3.3)
pip install sympy (for python 3.2)

When pip fails, it starts off like this:

Downloading/unpacking sympy
Downloading sympy-0.7.3.tar.gz (6.4MB): 6.4MB downloaded

When it succeeds (e.g pip install sympy on Python 3.2):

Downloading/unpacking sympy
Downloading sympy-0.7.3-py3.2.tar.gz (6.4MB): 6.4MB downloaded

or

Searching for sympy
Reading http://pypi.python.org/simple/sympy/
Best match: sympy 0.7.3-py3.3

I'm afraid I don't understand the algorithm that pip / easy_install
uses to chose between sympy-0.7.3.tar.gz and sympy-0.7.3-py3.3...

Cheers,

Matthew

[Aaron Meurer]

Ah yes, I knew this would happen. It's because PyPI wouldn't let me
upload the same file twice, and I never regenerated a
sympy-0.7.3-py33.tar.gz.

I guess I probably should fix that...

easy_install works because it still crawls webpages (bad).

Aaron Meurer

Sent from my iPad

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Ondřej Čertík]

On Sun, Jul 28, 2013 at 8:54 AM, Aaron Meurer <asme...@gmail.com> wrote:
> Ah yes, I knew this would happen. It's because PyPI wouldn't let me
> upload the same file twice, and I never regenerated a
> sympy-0.7.3-py33.tar.gz.
>
> I guess I probably should fix that...
>
> easy_install works because it still crawls webpages (bad).

Aaron, let's just merge this:

https://github.com/sympy/sympy/pull/2318

and then we have the option to only release one tarball, and I think
it should fix this pip issue as well.

Ondrej

[Matthew Brett]

Hi,

That would be great - but it would be good to have a fix for this
release too, especially if it is relatively easy,

Cheers,

Matthew

[Aaron Meurer]

On Sun, Jul 28, 2013 at 1:18 PM, Ondřej Čertík <ondrej...@gmail.com> wrote:

Yeah at the next release...

Aaron Meurer

[Aaron Meurer]

OK, I've uploaded a new file, which should fix it. I just untarred the
existing tarball and retarred it, which changed the md5.

Let me know if it works.

Aaron Meurer

[Matthew Brett]

Hi,

On Sun, Jul 28, 2013 at 2:04 PM, Aaron Meurer <asme...@gmail.com> wrote:
> OK, I've uploaded a new file, which should fix it. I just untarred the
> existing tarball and retarred it, which changed the md5.
>
> Let me know if it works.

Not yet:

http://nipy.bic.berkeley.edu/builders/nipy-py3.3/builds/6/steps/shell_2/logs/stdio

Is there some wait time before pypi registers the presence of the new file?

Cheers,

Matthew

[Aaron Meurer]

PyPI shows the file. Maybe pip is caching it?

Aaron Meurer

[Matthew Brett]

Hi,

On Sun, Jul 28, 2013 at 12:29 PM, Aaron Meurer <asme...@gmail.com> wrote:
> PyPI shows the file. Maybe pip is caching it?

Not the archive at least:

(sympy-test2)[mb312@angela ~]$ pip install sympy
Downloading/unpacking sympy
Downloading sympy-0.7.3.tar.gz (6.4MB): 10% 659kB

Thanks,

Matthew

[Aaron Meurer]

Well let me know if it doesn't work by tomorrow or something. Or try
in a fresh environment. It is showing up at
https://pypi.python.org/pypi/sympy for me, and also at
https://pypi.python.org/simple/sympy/, which is what pip searches.

Aaron Meurer

[Matthew Brett]

Hi,

On Sun, Jul 28, 2013 at 1:40 PM, Aaron Meurer <asme...@gmail.com> wrote:
> Well let me know if it doesn't work by tomorrow or something. Or try
> in a fresh environment. It is showing up at
> https://pypi.python.org/pypi/sympy for me, and also at
> https://pypi.python.org/simple/sympy/, which is what pip searches.

I'm installing into fresh virtualenvs for testing.

Here's some output of a pip.log just now:

Skipping link
https://pypi.python.org/packages/any/s/sympy/sympy-0.7.3.win32.exe#md5=b6ec7471286127aa6f103e5147e62c6e
(from https://pypi.python.org/simple/sympy/); unknown archive format:
.exe
Skipping https://pypi.python.org/packages/source/s/sympy/sympy-0.7.3-py3.2.tar.gz#md5=c6343a8eb1ab9a9741b3a3bba02a0eeb
(from https://pypi.python.org/simple/sympy/) because Python version is
incorrect
Found link https://pypi.python.org/packages/source/s/sympy/sympy-0.7.3.tar.gz#md5=91e3dc58c0af063bc64b17674296cb35
(from https://pypi.python.org/simple/sympy/), version: 0.7.3
Found link https://pypi.python.org/packages/source/s/sympy/sympy-0.7.3-py3.3.tar.gz#md5=46132548644c97034899c1a739d06127
(from https://pypi.python.org/simple/sympy/), version: 0.7.3
Found link https://github.com/sympy/sympy/releases/download/sympy-0.7.3/sympy-0.7.3-py3.3.tar.gz#md5=%20c6343a8eb1ab9a9741b3a3bba02a0eeb
(from https://pypi.python.org/simple/sympy/), version: 0.7.3
Using version 0.7.3 (newest of versions: 0.7.3, 0.7.3, 0.7.3, 0.7.2,
0.7.2, 0.7.1, 0.7.0, 0.6.6, 0.6.5, 0.6.4, 0.6.3, 0.6.2, 0.6.1, 0.6.0,
0.5.15, 0.5.14)
Downloading from URL
https://pypi.python.org/packages/source/s/sympy/sympy-0.7.3.tar.gz#md5=91e3dc58c0af063bc64b17674296cb35
(from https://pypi.python.org/simple/sympy/)

So it seems to be finding the new py3.3 tar archive, but for some
reason believes that the sympy-0.7.3.tar.gz archive is newer?

Would it make sense to remove sympy-0.7.3.tar.gz in favor of
sympy-0.7.3-oy2.5.tar.gz etc?

Thanks,

Matthew

[Aaron Meurer]

Hmm, can you figure out what the difference is between this version
and 0.7.2? Because it works perfectly in 0.7.2, at least for pip.

Aaron Meurer

[Matthew Brett]

Hi,

On Sun, Jul 28, 2013 at 3:15 PM, Aaron Meurer <asme...@gmail.com> wrote:
> Hmm, can you figure out what the difference is between this version
> and 0.7.2? Because it works perfectly in 0.7.2, at least for pip.

Looking at index.py around 188, it looks like pip is reading and
selecting links off this page:

https://pypi.python.org/simple/sympy/

then reverse sorting (by version string) to find the preferred
package. In our case, there is more than one matching package with the
same required version, and the sort by version does not affect the
order of the packages with the same version.

In the case of 0.7.2 the relevant links are in this order:

sympy-0.7.2-py3.3.tar.gz
sympy-0.7.2.tar.gz

so pip selects both, then prefers the first.

In the case of 0.7.3 the links are in order:

sympy-0.7.3.tar.gz
sympy-0.7.3-py3.3.tar.gz
https://github.com/sympy/sympy/releases/download/sympy-0.7.3/sympy-0.7.3-py3.3.tar.gz#md5=
c6343a8eb1ab9a9741b3a3bba02a0eeb

and it finds sympy-0.7.3.tar.gz first and prefers that.

Can you reorder the links on the page? I suspect it will fix it,

Cheers,

Matthew

[Matthew Brett]

Hi,

I did some more investigation as to how pypi orders the links on the
'simple' page; it looks like (in a current version of the code) it
pulls them out of a database, I couldn't at a first pass work out what
dictated the order that they come out from the database:

http://wiki.python.org/moin/CheeseShopDev
https://bitbucket.org/pypa/pypi/src
https://bitbucket.org/pypa/pypi/src/f970cf92b100886e03954f651986dd4c2ba46091/webui.py?at=default#cl-775

Did you already try removing the sympy-0.7.3.tar.gz from the pypi
files using the pypi files interface, then adding it again?

Best,

Matthew

[Aaron Meurer]

I did try removing the link to the GitHub releases page. Did that help?

Aaron Meurer

[Matthew Brett]

Hi,

On Tue, Jul 30, 2013 at 12:10 PM, Aaron Meurer <asme...@gmail.com> wrote:
> I did try removing the link to the GitHub releases page. Did that help?

No - same as before, I think for the same reason - the order of links
on the pypi page:

sympy-0.7.3.tar.gz
sympy-0.7.3-py3.3.tar.gz

Does pip install work for you in a python 3.3 virtualenv?

Cheers,

Matthew

[Aaron Meurer]

OK, I reuploaded the sympy-0.7.3.tar.gz file, and it seems to work now.

Aaron Meurer

[Matthew Brett]

Hi,

On Tue, Jul 30, 2013 at 1:10 PM, Aaron Meurer <asme...@gmail.com> wrote:
> OK, I reuploaded the sympy-0.7.3.tar.gz file, and it seems to work now.

Excellent thanks - it looks like that solved it,

Cheers,

Matthew

[Matthew Brett]

Hi,

Oh - dear - broken again, for Python 3.2 and Python 3.3:

http://nipy.bic.berkeley.edu/builders/nipy-py3.2/builds/10/steps/shell_2/logs/stdio
http://nipy.bic.berkeley.edu/builders/nipy-py3.3/builds/14/steps/shell_2/logs/stdio

It looks like the links here:

https://pypi.python.org/simple/sympy/

got rearranged somehow. Maybe someone refreshed the database on the
server or something?

Would you mind re-uploading the bare archives again? Or can you think
of a better solution?

Cheers,

Matthew

[Aaron Meurer]

I reuploaded the file, but that doesn't seem to have fixed it.

The easiest solution is probably to finish up
https://github.com/sympy/sympy/pull/2333 and do another release.

Aaron Meurer

[Matthew Brett]

Hi,

On Wed, Aug 21, 2013 at 6:12 PM, Aaron Meurer <asme...@gmail.com> wrote:
> I reuploaded the file, but that doesn't seem to have fixed it.
>
> The easiest solution is probably to finish up
> https://github.com/sympy/sympy/pull/2333 and do another release.

That's a good long term solution - but it still means that anyone
trying to install sympy==0.7.3 or sympy==0.7.2 is going to run into
confusing trouble.

Did you delete the archive from pypi before uploading again?

Cheers,

Matthew

[Aaron Meurer]

Aaron Meurer

Sent from my iPad

Yes. I don't think it would even let me do it otherwise.

Aaron Meurer

[Matthew Brett]

Hum,

On Wed, Aug 21, 2013 at 6:21 PM, Aaron Meurer <asme...@gmail.com> wrote:
> Aaron Meurer
>
> Sent from my iPad
>
> On Aug 21, 2013, at 7:15 PM, Matthew Brett <matthe...@gmail.com> wrote:
>
>> Hi,
>>
>> On Wed, Aug 21, 2013 at 6:12 PM, Aaron Meurer <asme...@gmail.com> wrote:
>>> I reuploaded the file, but that doesn't seem to have fixed it.
>>>
>>> The easiest solution is probably to finish up
>>> https://github.com/sympy/sympy/pull/2333 and do another release.
>>
>> That's a good long term solution - but it still means that anyone
>> trying to install sympy==0.7.3 or sympy==0.7.2 is going to run into
>> confusing trouble.
>>
>> Did you delete the archive from pypi before uploading again?
>
> Yes. I don't think it would even let me do it otherwise.

So you deleted sympy-0.7.3.tar.gz, reuploaded, but this didn't change the order?

And just to be clear - this is what you did last time, and it did
change the order?

Ah pypi...

I guess it would be worth considering putting up sympy-0.7.3-py27,
sympy-0.7.3-py26, sympy-0.7.3-py25, and removing the bare tarball.

Cheers,

Matthew

[Aaron Meurer]

On Wed, Aug 21, 2013 at 7:26 PM, Matthew Brett <matthe...@gmail.com> wrote:
> Hum,
>
> On Wed, Aug 21, 2013 at 6:21 PM, Aaron Meurer <asme...@gmail.com> wrote:
>> Aaron Meurer
>>
>> Sent from my iPad
>>
>> On Aug 21, 2013, at 7:15 PM, Matthew Brett <matthe...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> On Wed, Aug 21, 2013 at 6:12 PM, Aaron Meurer <asme...@gmail.com> wrote:
>>>> I reuploaded the file, but that doesn't seem to have fixed it.
>>>>
>>>> The easiest solution is probably to finish up
>>>> https://github.com/sympy/sympy/pull/2333 and do another release.
>>>
>>> That's a good long term solution - but it still means that anyone
>>> trying to install sympy==0.7.3 or sympy==0.7.2 is going to run into
>>> confusing trouble.
>>>
>>> Did you delete the archive from pypi before uploading again?
>>
>> Yes. I don't think it would even let me do it otherwise.
>
> So you deleted sympy-0.7.3.tar.gz, reuploaded, but this didn't change the order?
>
> And just to be clear - this is what you did last time, and it did
> change the order?

I think last time there wasn't the one tarball. I don't remember, though.

>
> Ah pypi...
>
> I guess it would be worth considering putting up sympy-0.7.3-py27,
> sympy-0.7.3-py26, sympy-0.7.3-py25, and removing the bare tarball.

I guess so. It's annoying because I have to recompress it each time so
that it has a different md5. And we'll lose all download counts.

[Aaron Meurer]

Sigh.

OK, I've done this. Can you check that it works in Python 2 and Python 3?

Aaron Meurer

[Matthew Brett]

Hi,

Thanks for doing that. Tested on pythons 2.6 through 3.3:

http://nipy.bic.berkeley.edu/builders/nipy-py2.6/builds/277/steps/shell_2/logs/stdio
http://nipy.bic.berkeley.edu/builders/nipy-py2.7-osx-10.8/builds/19/steps/shell_2/logs/stdio
http://nipy.bic.berkeley.edu/builders/nipy-py3.2/builds/11/steps/shell_2/logs/stdio
http://nipy.bic.berkeley.edu/builders/nipy-py3.3/builds/15/steps/shell_2/logs/stdio

Cheers,

Matthew

[Angus Griffith]

Is there a workaround for this?

I have a python2 application that depends on Sympy and I'm trying to upgrade the dependency to 0.7.3.

I've tried a few things but nothing successful yet. (Either the package isn't found or it tries (and consequently) fails to install the python3 version.)

sympy==0.7.3, sympy-py2, sympy==0.7.3-py2

No local packages or download links found for ...

sympy>=0.7.3, sympy

Best match: sympy 0.7.3-py3.3

...

ImportError: You appear to be using the Python 3 version of SymPy in Python 2. Use Python 3 or get the Python 2 source code from http://sympy.org.

On Sunday, 28 July 2013 17:33:41 UTC+10, Matthew Brett wrote:

Hi,

I think pip on Python 3.3 is picking up the wrong version of Sympy:

http://nipy.bic.berkeley.edu/builders/nipy-py3.3/builds/4/steps/shell_2/logs/stdio

Exploring, this fails with the above error:

pip install sympy (for python 3.3)

These succeed:

pip install sympy==0.7.2 (for python 3.3)
easy_install sympy (for python 3.3)
pip install sympy (for python 3.2)

When pip fails, it starts off like this:

Downloading/unpacking sympy

[Angus Griffith]

I found a workaround: install Sympy from the git repo. For example:

pip install git+https://github.com/sympy/symp...@sympy-0.7.3

[Ondřej Čertík]

On Wed, Sep 11, 2013 at 4:04 AM, Angus Griffith <16s...@gmail.com> wrote:
> Is there a workaround for this?
>
> I have a python2 application that depends on Sympy and I'm trying to upgrade
> the dependency to 0.7.3.
>
> I've tried a few things but nothing successful yet. (Either the package
> isn't found or it tries (and consequently) fails to install the python3
> version.)
>
> sympy==0.7.3, sympy-py2, sympy==0.7.3-py2
>
> No local packages or download links found for ...

That's weird. Here is what I do:

https://gist.github.com/certik/6525973

and it seem to work. I also tried

pip install sympy==0.7.3

and it worked too.

Ondrej

[Angus Griffith]

You're right. For some reason (lack of sleep?) I assumed setuptools used pip (but actually it uses easy_install). 

The actual problem is with setuptools (maybe this is a known problem?).

I repeated the steps in the gist (virtualenv etc) with a basic setup.py and it still happens: http://pastebin.com/9xDjp4yK

[Aaron Meurer]

Yes, unfortunately, it doesn't seem to be possible to make separate
Python 2/3 tarballs work with easy_install. This will be fixed in the
next version because we have moved to a single codebase for the two,
so there will no longer be separate tarballs.

Aaron Meurer