SAGE64 + linux, SAGE64 in general

7 views
Skip to first unread message

John H Palmieri

unread,
Nov 20, 2010, 5:36:53 PM11/20/10
to sage-devel
Is SAGE64 supposed to have an effect on linux machines, or indeed on
anything except OS X and Solaris? In various spkgs, I think there are
lines to the effect

if [ -z $CFLAG64 ] ; then
CFLAG64=-m64
fi

if [ "x$SAGE64" = xyes ]; then
echo "64 bit build of cddlib"
CFLAGS="$CFLAGS $CFLAG64"; export CFLAGS
fi

which looks like it should have an effect on any machine. However, in
sage-env and elsewhere, there are comments like this:

# In case SAGE64 has been set to yes before re-inject it into the
environment
# This is only done on OSX and Solaris since those are the only real
multi lib
# arches we support. Eventually Linux PPC on the PS3 might need to be
added here

And then in sage-check-64, on OS X and Solaris, a file SAGE_LOCAL/lib/
sage-64.txt is created if SAGE64 is set, but this file isn't created
on other platforms. The same script checks whether this file exists
to see if this looks like a 64-bit build. Is the file sage-64.txt used
anywhere except in sage-check-64?

As far as I can tell, we actually don't need to execute sage-check-64
except before running sage-spkg or possibly sage-build. Can anyone
comment on the accuracy of this statement? (I'd like to remove the
execution of sage-check-64 out of sage-env and into the places that
actually need it.)

Summarizing, my questions are:

- is SAGE64 supposed to have an effect on platforms other than OS X
and Solaris? (I think so.)

- is the file SAGE_LOCAL/lib/sage-64.txt used anywhere except sage-
check-64?

- is sage-check-64 and/or the variable SAGE64 used anywhere besides
sage-spkg (and the spkg-install files for the various spkgs) and
possibly sage-build?

--
John

William Stein

unread,
Nov 20, 2010, 6:47:34 PM11/20/10
to sage-...@googlegroups.com

This is not really an answer, but a remark. SAGE64 only got
introduced by mabshoff when he was trying to port Sage to OS X 64-bit
last year. So for 4-5 year of building Sage on Linux, SAGE64 didn't
even exist.

>
>  - is the file SAGE_LOCAL/lib/sage-64.txt used anywhere except sage-
> check-64?
>
>  - is sage-check-64 and/or the variable SAGE64 used anywhere besides
> sage-spkg (and the spkg-install files for the various spkgs) and
> possibly sage-build?
>
> --
> John
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

Dr. David Kirkby

unread,
Nov 20, 2010, 8:32:48 PM11/20/10
to sage-...@googlegroups.com
On 11/20/10 10:36 PM, John H Palmieri wrote:

> Summarizing, my questions are:
>
> - is SAGE64 supposed to have an effect on platforms other than OS X
> and Solaris? (I think so.)

I would say yes.

In practice it is only currently used on OS X and Solaris, but looking forward,
it could be used on AIX, HP-UX and perhaps even Linux systems on mobile phones.

So if you come across anything that would stop it working on any platform at
all, I would remove them.

Micheal making it work only on OS X was a stupid thing to do, considering he was
actually employed to get Sage working on Solaris, not OS X. But hindsight is a
wonderful thing.

> - is the file SAGE_LOCAL/lib/sage-64.txt used anywhere except sage-
> check-64?

Not to my knowledge. I just did a recursive grep of sage-64.txt and don't see it
anywhere else.

grep -R sage-64.txt *

just showed it in install.log and sage-check-64.

> - is sage-check-64 and/or the variable SAGE64 used anywhere besides
> sage-spkg (and the spkg-install files for the various spkgs) and
> possibly sage-build?

Again, recursive grep helps.

local/bin/sage-env
spkg/base/prereq-0.7-install
(plus other 'prereq' related files such as spkg/build/prereq-0.7/Makefile)

Apart from documentation, I can't see it anywhere else.

It's important that the check is made regularly, as someone could start building
with SAGE64=yes, then upgrade, or what I've done before, open another terminal
and not set SAGE64=yes. If the check did not exist, this would lead to a mix of
32-bit and 64-bit objects.

It would actually be good if the file was always created but had "yes" or "no"
in it. Then it could prevent

1) Initial 64-bit builds getting corrupted with 32-bit components.

2) Initial 32-bit builds getting corrupted with 64-bit components

Currently the file only protects against (1).

I've managed to mess up 32-bit builds by having SAGE64 in the environment at the
wrong time. Currently (2) is not prevented, but it could be if the file
contained yes or no, rather than just being an empty file. But it is not a high
priority task.

Dave

John H Palmieri

unread,
Nov 20, 2010, 10:09:00 PM11/20/10
to sage-devel
On Nov 20, 5:32 pm, "Dr. David Kirkby" <david.kir...@onetel.net>
wrote:
> On 11/20/10 10:36 PM, John H Palmieri wrote:
>
> > Summarizing, my questions are:
>
> >   - is SAGE64 supposed to have an effect on platforms other than OS X
> > and Solaris?  (I think so.)
>
> I would say yes.
>
> In practice it is only currently used on OS X and Solaris, but looking forward,
> it could be used on AIX, HP-UX and perhaps even Linux systems on mobile phones.

Is that the compiler flag "-m64" has no effect on other systems? Some
spkg-install files just check whether SAGE64 is set, not the platform,
and then they add -m64 to the flags. So if that does anything on
linux, then SAGE64 can be used on linux.

> It's important that the check is made regularly, as someone could start building
> with SAGE64=yes, then upgrade, or what I've done before, open another terminal
> and not set SAGE64=yes. If the check did not exist, this would lead to a mix of
> 32-bit and 64-bit objects.

Right.

> It would actually be good if the file was always created but had "yes" or "no"
> in it. Then it could prevent
>
> 1) Initial 64-bit builds getting corrupted with 32-bit components.
>
> 2) Initial 32-bit builds getting corrupted with 64-bit components
>
> Currently the file only protects against (1).

On the other hand, if I have a 32-bit install and I want to test
building an spkg in both 32-bit mode and 64-bit mode, I would like to
be able to do

$ sage -f new.spkg
...
$ do some testing
$ export SAGE64='yes'
$ sage -f new.spkg
...

for example, just to see if the compiler flags are set correctly. I
shouldn't need separate 32-bit and 64-bit builds just for a simple
check like that. And if the spkg is just supposed to produce some
binary executables which don't rely on any libraries in SAGE_LOCAL,
then I should be able to completely test the spkg this way. But once I
set SAGE64=yes on OS X or Solaris, then unless I manually delete the
file sage-64.txt, it will forever think it's a 64-bit build. I think
I should be able to do

$ export SAGE64='no'

and then Sage should *delete* the file sage-64.txt if it's present.

Another option to prevent (2) could be done in sage-check-64: if
SAGE64='yes' but the sage-64.txt does not exist (along with some other
check to make sure the Sage build is not just starting), or if
SAGE64='no' and the sage-64.txt does exist, ask if you really want to
continue. That is, the presence of absence of the file would be
equivalent to your idea of having the file contain "yes" or "no".
Having a query would allow my desired behavior also.

--
John

David Kirkby

unread,
Nov 20, 2010, 10:43:36 PM11/20/10
to sage-...@googlegroups.com

> John

I agree, sometimes for the purpose of testing, it is useful to be able
to mix objects, if you only want to check if a package builds, and
don't care that Sage will function 100%.

But if I'm running such tests, I don't have a problem with typing

echo yes > local/lib/sage-64.txt

I believe however the default behavior should be to protect the
installation, and not let someone mix object files.

I was not the one that added this facility, but someone obviously felt
it important enough to ensure that 64-bit builds could not be
corrupted with 32-bit objects. But the converse case was overlooked.

Dave

Dave

Peter Jeremy

unread,
Nov 22, 2010, 2:55:51 AM11/22/10
to sage-...@googlegroups.com
On 2010-Nov-20 19:09:00 -0800, John H Palmieri <jhpalm...@gmail.com> wrote:
>Is that the compiler flag "-m64" has no effect on other systems? Some
>spkg-install files just check whether SAGE64 is set, not the platform,
>and then they add -m64 to the flags. So if that does anything on
>linux, then SAGE64 can be used on linux.

'-m64' is recognized by gcc and the SunPRO compilers. It's not recognized
by the HP Tru64 C compiler (which is natively 64-bit). I can't comment
on any other compilers.

Generically adding '-m64' when SAGE64 is set is definitely wrong because
not all compilers will support that flag. In addition, gcc treats x86
and x86_64 as different variants of the one architecture - so gcc on a
32-bit platform can compile x86_64 code. On 32-bit x86 Linux and *BSD,
using '-m64' will cause gcc to build x86_64 code - which the kernel won't
be able to execute - so this is highly undesirable.

Ideally, all skpg files should inherit {C,CPP,CXX,FC,LD}FLAGS from the
environment (adding spkg-specific options if required). This would
allow SAGE64 to be processed in one spot fairly early on in the build
- adding '-m64' or equivalent to {C,FCC,FC,LD}FLAGS, which is then
inherited by all succeeding spkg builds. This would remove a lot of
special-casing from the build.

--
Peter Jeremy

Dr. David Kirkby

unread,
Nov 22, 2010, 12:05:16 PM11/22/10
to sage-...@googlegroups.com
On 11/22/10 07:55 AM, Peter Jeremy wrote:
> On 2010-Nov-20 19:09:00 -0800, John H Palmieri<jhpalm...@gmail.com> wrote:
>> Is that the compiler flag "-m64" has no effect on other systems? Some
>> spkg-install files just check whether SAGE64 is set, not the platform,
>> and then they add -m64 to the flags. So if that does anything on
>> linux, then SAGE64 can be used on linux.
>
> '-m64' is recognized by gcc and the SunPRO compilers. It's not recognized
> by the HP Tru64 C compiler (which is natively 64-bit). I can't comment
> on any other compilers.

-m64 is not reconised by the HP compiler on HP-UX (where one needs +DA2.OW or
+DD64 depending on the CPU type).

Neither is -m64 reconised by IBM's compiler on AIX, where one needs -q64.

So there are a few cases where -m64 is not appropriate. At some point in the
future, who knows what other systems or compilers will not want -m64?

> Generically adding '-m64' when SAGE64 is set is definitely wrong because
> not all compilers will support that flag. In addition, gcc treats x86
> and x86_64 as different variants of the one architecture - so gcc on a
> 32-bit platform can compile x86_64 code. On 32-bit x86 Linux and *BSD,
> using '-m64' will cause gcc to build x86_64 code - which the kernel won't
> be able to execute - so this is highly undesirable.

But if SAGE64 is set to "yes", then adding an appropriate flag should not be a
problem.

If SAGE64 is set to anything other than "yes", and a compiler flag like -m64 was
added, then clearly that would be stupid.

> Ideally, all skpg files should inherit {C,CPP,CXX,FC,LD}FLAGS from the
> environment (adding spkg-specific options if required). This would
> allow SAGE64 to be processed in one spot fairly early on in the build
> - adding '-m64' or equivalent to {C,FCC,FC,LD}FLAGS, which is then
> inherited by all succeeding spkg builds. This would remove a lot of
> special-casing from the build.

Agreed.

I tried that before

http://trac.sagemath.org/sage_trac/ticket/7818

so every package would get a common set of flags, unless they were set
otherwise. Those flags were -g -Wall for gcc, but changed for non-GNU compilers
and 64-bit builds.

But doing this removes the -fno-strict-aliasing option from the gcc options that
Cython uses, which broke things all over the place.

IMHO, it is a fault of Cython. But that ticket caused so much hassle, I have
given up trying to fix it.

The CFLAG64 option is not an ideal solution but came about by the following set
of circumstances.

1) Micheal made the variable add -m64 *only* on OS X.

2) I change some instance of it so it worked on both Solaris and OS X, since I
knew both the Sun and GNU compilers accepted this option. At this point, I gave
no consideration to other platforms.

3) Later I since realised that (2) was a bit dumb, as potentially one could use
another operating system, and potentially the flag might not be -m64. AIX and
HP-UX are examples of this, but there may be others now or in the future.

So more recent changes made by me should allow any user-specificed flag(s) to be
set for 64-bit builds, but the will default to -m64 if SAGE64=yes.


Dave

Reply all
Reply to author
Forward
0 new messages