Are the Sage binaries for OS X are crap?

[William Stein]

Hi,

A random student with OS X 10.6.4 with a 2.4 GHz Intel Core 2 Duo came
into my laptop, and when I tried to install the sage-4.8-64bit-10.6
app binary [1], it fails with "Symbol not found:
_PyUnicodeUCS4_AsUTF8String". This is I think the same error message
I saw on numerous laptops when I was at the Joint Math Meetings.
I've pasted the message below. The binary I had made [2] myself on
bsd.math.washington.edu works fine.

[1] http://boxen.math.washington.edu/sage/osx/intel/sage-4.8-OSX-64bit-10.6-x86_64-Darwin-app.dmg

[2] http://wstein.org/home/wstein/tmp/sage-4.7.3.alpha1-x86_64-Darwin.dmg

-- William

---------- Forwarded message ----------
Date: Thu, Feb 2, 2012 at 1:12 PM
Subject: error message
To: William Stein <wst...@gmail.com>

Last login: Wed Feb  1 19:43:38 on console
'/Applications/Sage-4.8-OSX-64bit-10.6.app/Contents/Resources/sage'/sage
--notebook
d-173-250-203-199:~ MikeKyllo$
'/Applications/Sage-4.8-OSX-64bit-10.6.app/Contents/Resources/sage'/sage
--notebook
----------------------------------------------------------------------
| Sage Version 4.8, Release Date: 2012-01-20                         |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------

Please wait while the Sage Notebook server starts...
Traceback (most recent call last):
 File "/Applications/Sage-4.8-OSX-64bit-10.6.app/Contents/Resources/sage/local/bin/sage-notebook",
line 9, in <module>
   from sage.server.notebook.all import notebook
 File "/Users/buildbot/build/sage/bsd-1/bsd_64_binary/build/sage-4.8/local/lib/python2.6/site-packages/sage/server/notebook/all.py",
line 22, in <module>
 File "/Users/buildbot/build/sage/bsd-1/bsd_64_binary/build/sage-4.8/devel/sagenb/sagenb/notebook/all.py",
line 16, in <module>
 File "/Users/buildbot/build/sage/bsd-1/bsd_64_binary/build/sage-4.8/devel/sagenb/sagenb/notebook/notebook_object.py",
line 17, in <module>
 File "/Users/buildbot/build/sage/bsd-1/bsd_64_binary/build/sage-4.8/devel/sagenb/sagenb/notebook/notebook.py",
line 30, in <module>
ImportError: dlopen(/Applications/Sage-4.8-OSX-64bit-10.6.app/Contents/Resources/sage/local/lib/python2.6/lib-dynload/cPickle.so,
2): Symbol not found: _PyUnicodeUCS4_AsUTF8String
 Referenced from:
/Applications/Sage-4.8-OSX-64bit-10.6.app/Contents/Resources/sage/local/lib/python2.6/lib-dynload/cPickle.so
 Expected in: flat namespace
 in /Applications/Sage-4.8-OSX-64bit-10.6.app/Contents/Resources/sage/local/lib/python2.6/lib-dynload/cPickle.so
d-173-250-203-199:~ MikeKyllo$

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[François Bissey]

On Thu, 02 Feb 2012 13:28:22 William Stein wrote:
> Hi,
>
> A random student with OS X 10.6.4 with a 2.4 GHz Intel Core 2 Duo came
> into my laptop, and when I tried to install the sage-4.8-64bit-10.6
> app binary [1], it fails with "Symbol not found:
> _PyUnicodeUCS4_AsUTF8String". This is I think the same error message
> I saw on numerous laptops when I was at the Joint Math Meetings.
> I've pasted the message below. The binary I had made [2] myself on
> bsd.math.washington.edu works fine.
>
> [1]
> http://boxen.math.washington.edu/sage/osx/intel/sage-4.8-OSX-64bit-10.6-x86
> _64-Darwin-app.dmg
>
> [2] http://wstein.org/home/wstein/tmp/sage-4.7.3.alpha1-x86_64-Darwin.dmg
>
> -- William
>

Actually someone made a post last week I think because he couldn't build
sage on OS X. The build was stopping because this symbol was missing in
python.
If I remember right that meant that setuptools didn't work. I could see the
symbol being reported missing several time in their python build.

Francois

[Georg S. Weber]

Hi,

interesting.
I have the faint hope that the fix at http://trac.sagemath.org/sage_trac/ticket/12161
(which I hope to review this weekend --- it requires building twice a
Sage app on OS X 10.6, and then test these on OS X 10.7) will also
cure this. Even though this new report is on OS X 10.6!

Reasoning:
The underlying problem fixed by the patch at #12161 was, that a system
python binary (/usr/bin/python) was called, instead of the python
binary that comes with Sage. According to some web quick searches,
"_PyUnicodeUCS4_AsUTF8String" is encountered when trying to import
Python extensions into Python binaries which are slightly incompatible
with each other (the difference being about the idea of how to
represent Unicode, as UCS2 or UCS4) ...

The #12161 problem was so difficult to track because it hurts only on
systems that *never* had a Sage installed on it, more precisily that
are missing the ".sage" subdirectory under their user's home
directory.

I remember that in the past, we had problems because nobody had tried
to test whether relocatability still works. I'm not quite sure how
many regular build/regression tests we currently run (except for mere
building and doctesting), and if some relocatability test is among
these (this just can't be "easily doctested" for obvious reasons).
Maybe it would be a good idea to also test Sage rc releases in
environments, where ".sage" has been removed (for each major platform)
--- there may be many other issues, with matplotlib, gap, and any of
the other components that write (or only look) under ".sage". (Testing
this also is pretty hard to do by the doctesing framework, I presume).


Cheers,
Georg

[William Stein]

Very interesting!! In fact, every single case of this bug that I've
seen (and there have been at least three) was on a system in which
Sage had never successfully been run before, ever. This is, of
course, exactly the sort of user that we do not want to alienate with
our binaries. For all I know, that are hundreds of such people now
who have hit this (I sure hope not, but I am not optimistic).

-- William

>
>
> Cheers,
> Georg
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

[Mike OS]

Just corroborating.

I encountered the same problem this week installing sage for the first
time on a laptop running 10.6.8.
I had no problem installing on a imac running 10.6.8 that already had
an older copy of sage.

[Ivan Andrus]

FWIW you should be able to work around the problem by opening Terminal.app before starting sage. At least if it's the manifestation of the problem that I think it is.

-Ivan

[William Stein]

Hi,

I just got yet another (almost daily now) bug report about our OS X
binaries being complete crap. This time from a Fields Medalist! So
why don't we deal with this?

The traceback he sent me is exactly the same as the one that started
this thread. This is really stupid.

-- William

--

[Ivan Andrus]

I'm fairly certain #11881 (which still needs review) will solve the problem. At least the most common problem.

Has anyone tried the work around I mentioned below?

-Ivan

[Dima Pasechnik]

On Wednesday, February 8, 2012 3:34:38 AM UTC+8, William wrote:

Hi,

I just got yet another (almost daily now) bug report about our OS X
binaries being complete crap.  This time from a Fields Medalist!  So
why don't we deal with this?

The traceback he sent me is exactly the same as the one that started
this thread.    This is really stupid.

as a courtesy to the new users, one at least should put a big warning on the download page of this buggy download (*mirrorsname*/sage/osx/intel/index.html)

And tell them to use the Terminal (i.e. non '-app' file) version for the time being.

Otherwise it indeed alienates a lot of people :-(

Dima

 

 -- William

[Jason Grout]

On 2/7/12 4:28 PM, Ivan Andrus wrote:
> I'm fairly certain #11881 (which still needs review) will solve the problem. At least the most common problem.

Do you, by chance, mean http://trac.sagemath.org/sage_trac/ticket/12161

#11881 is a huge ticket with lots of patches.

Thanks,

Jason

[kcrisman]

On Feb 7, 2:34 pm, William Stein <wst...@gmail.com> wrote:
> Hi,
>
> I just got yet another (almost daily now) bug report about our OS X
> binaries being complete crap.  This time from a Fields Medalist!  So
> why don't we deal with this?

William, on a related note, have all the people who used your
4.8.alpha3 or whatever "custom" binary successfully people who would
now have a .sage directory (after having unsuccessfully used the
"regular" binary)?

And I assume this is using the app bundle that caused the problem,
correct?

If there is something wrong with the buildbot one, we could just
create a custom one (I wouldn't, because I don't have libtiff or
something so those files in plot/plot3d/base.pyx fail tests) on a
machine we know does work.

Georg makes it sound like one could test #12161 as being the problem
by removing/renaming the .sage directory and then starting up the app
bundle and seeing if this fixes the problem. Am I reading this right?

If I find a little time I may yet try this tonight, since ...

> The traceback he sent me is exactly the same as the one that started
> this thread.    This is really stupid.

... you are right about that. Maybe Harald could remove/hide the app
bundles for the time being, though that's a drastic move.

I wonder if one of our Python updates (certainly not the 2.7 one, of
course) was to blame...

[kcrisman]

>
> If I find a little time I may yet try this tonight, since ...
>

I can't reproduce this even if I do

mv .sage/ .oldsage

with the "sketchy" binary in question. It must be highly dependent on
a lot of stuff :( sorry.

[William Stein]

Yes, I've never replicated this on any of my own computers either. It
just happens on important people around me at the worst moment.
Really annoying.

[Ivan Andrus]

On Feb 8, 2012, at 1:18 AM, Jason Grout wrote:

> On 2/7/12 4:28 PM, Ivan Andrus wrote:
>> I'm fairly certain #11881 (which still needs review) will solve the problem. At least the most common problem.
>
> Do you, by chance, mean http://trac.sagemath.org/sage_trac/ticket/12161
>
> #11881 is a huge ticket with lots of patches.
>
> Thanks,
>
> Jason

Err, yes of course. Sorry for the confusion.

-Ivan

[Harald Schilly]

On Wednesday, February 8, 2012 1:12:22 AM UTC+1, Dima Pasechnik wrote:

as a courtesy to the new users, one at least should put a big warning on the download page of this buggy download (*mirrorsname*/sage/osx/intel/index.html)

And tell them to use the Terminal (i.e. non '-app' file) version for the time being.

Hi, thx for CCing me.  Should I remove the -app files from osx/intel or put a warning note there? If yes, what warning?

greetings Harald

[Dima Pasechnik]

Hi Harald,

It has come to our attention recently that on MacOSX computers, where
Sage has never been installed before, the `-app` version of Sage might
fail to start. We apologize for the inconvenience.
While we are working on a fix for this problem, please use the
non-`-app` version of Sage.

Something like this...
Dima

> greetings Harald

[Harald Schilly]

On Wednesday, February 8, 2012 1:18:35 PM UTC+1, Dima Pasechnik wrote:

Something like this...

thx. done! 

[Ivan Andrus]

Excellent. That's the first confirmation I've had from someone with the problem that it really was #12161. I'm cc-ing sage-devel to let them know it worked.

-Ivan

On Feb 10, 2012, at 1:17 AM, Michael E O'Sullivan wrote:

> Brilliant!
>
> I restarted terminal and
> MAC043:~ instructor$ /Applications/sage/sage

> ----------------------------------------------------------------------
> | Sage Version 4.8, Release Date: 2012-01-20 |
> | Type notebook() for the GUI, and license() for information. |
> ----------------------------------------------------------------------

> The Sage installation tree may have moved
> (from /Users/buildbot/build/sage/bsd-1/bsd_64_binary/build/sage-4.8 to /Applications/sage).
> Changing various hardcoded paths...
> (Please wait at most a few minutes.)
> DO NOT INTERRUPT THIS.
> Done resetting paths.
> Setting permissions of DOT_SAGE directory so only you can read and write it.
> sage: F.<a> = GF(16)
> sage: a^5
> a^2 + a
>
> Thank you!
>
> Mike
> Michael E O'Sullivan
> Professor of Mathematics
> San Diego State University
> mosu...@math.sdsu.edu
> 619 594-6697
>
>

[Georg S. Weber]

Hi all,

I finally could verify that the patch at trac 12161 is good, i.e.
heals not only the original problem for OS X 10.7, but the
"_PyUnicodeUCS4_AsUTF8String" incarnation, too (at least on one of my
own OS X 10.6 installations --- try starting a Sage "app" after
(closing all Sage and all terminal sessions, and) nuking/
removing .sage, *and* do so on a system that boots up in 64bit mode,
i.e. by pressing "6" and "4" during startup --- but maybe the computer
just has to be "young enough" so that the OS X 10.6/10.7 installs
something different than on "older" ones, which would be 64bit
capable, but Apple does boot them only in 32bit mode; see
http://support.apple.com/kb/HT3773 and http://support.apple.com/kb/HT3770).

Anyway, would it make sense now to upload some official update of the
OS X 10.6 app (with only the trac 12161 patch as delta to the original
one currently in the download area)?


Cheers,
Georg

[kcrisman]

On Feb 21, 6:20 pm, "Georg S. Weber" <georgswe...@googlemail.com>
wrote:

> Hi all,
>
> I finally could verify that the patch at trac 12161 is good, i.e.
> heals not only the original problem for OS X 10.7, but the
> "_PyUnicodeUCS4_AsUTF8String" incarnation, too (at least on one of my
> own OS X 10.6 installations --- try starting a Sage "app" after
> (closing all Sage and all terminal sessions, and) nuking/
> removing .sage, *and* do so on a system that boots up in 64bit mode,
> i.e. by pressing "6" and "4" during startup --- but maybe the computer
> just has to be "young enough" so that the OS X 10.6/10.7 installs
> something different than on "older" ones, which would be 64bit

> capable, but Apple does boot them only in 32bit mode; seehttp://support.apple.com/kb/HT3773and  http://support.apple.com/kb/HT3770).

>
> Anyway, would it make sense now to upload some official update of the
> OS X 10.6 app (with only the trac 12161 patch as delta to the original
> one currently in the download area)?

Unless 5.0 is coming within a week (and given the status of
http://trac.sagemath.org/sage_trac/ticket/12024 and
http://trac.sagemath.org/sage_trac/ticket/11881, that seems highly
unlikely), I think this is a VERY good idea. As I've said before, my
own laptop has something weird with a missing libtiff or something, so
I am reluctant to do it, but hopefully anyone else could? Or the
buildbot?

And if someone can make Sage 4.8 binaries for Intel 10.4 and 10.5,
that would be even better :)