First impressions on Sage 9.4 source build

[Randall]

Hi all:

I am building sage version 9.4 from source on my OpenSuse Linux Leap 15.2 OS and like the install document stated, it has taken hours (using a HP Z420 workstation with 6 Xeon processors) 

I have noticed some odd things occurring which caught my attention as at one time in my life I was in charge of 1+million line code handoff to another company and our code had to be very high quality.

Here's what caught my eye:

1. dozens of warnings during Cythonizing of comparing signed variables to unsigned. This warning needs to be heeded and the code fixed

2. dozens of warnings of compares using uninitialized variables. It goes without saying that using uninitialized variables is only asking for trouble later on. These need to be fixed also.

3. hundreds of warnings during package compilation, the source code does need improvement. I did notice that some C programmers were attempting to write C++ code, but this is asking for trouble too. C++ is not advanced C, despite the popular (mis) belief in this.

I am attempting to install the optional packages just to see what happens, for the build process, but it looks like bundling several packages (say x) on the ./compile line is only asking for repeats of the whole compile run x-times. It should be possible to install all on only one run, not x-runs.

I have some questions here

1. Will "make all" build all the optional packages and avoid the experimental?

(I did NOT try make all, because just doing make itself chewed up about 9 hours time)

2. I noticed that sage recommended several OpenSuse system packages on the zypper install line, but I did run this and believe it or not, nothing was available. Is there some consideration to creating a public mathematical software repository where these OpenSuse packages are available? (This probably needs to be extended to most of the public Linux systems)

3. Can the dochtml installer be decoupled from the regular install, and it run optionally? It  takes up a lot of time for it to discover that everything has already been installed.

Thanks for reading this post.

- Randall

[William Stein]

On Fri, Dec 17, 2021 at 2:33 PM Randall <randall...@gmail.com> wrote:
>
> Hi all:
>
> I am building sage version 9.4 from source on my OpenSuse Linux Leap 15.2 OS and like the install document stated, it has taken hours (using a HP Z420 workstation with 6 Xeon processors)
>
> I have noticed some odd things occurring which caught my attention as at one time in my life I was in charge of 1+million line code handoff to another company and our code had to be very high quality.
>
> Here's what caught my eye:
>
> 1. dozens of warnings during Cythonizing of comparing signed variables to unsigned. This warning needs to be heeded and the code fixed

Can you please clarify what you mean by "This warning needs to be
heeded and the code fixed"? In particular, which of the following is
what you are saying?

- [ ] If these issues don't get fixed, I won't use Sage.

- [ ] If some random college professors doing Sage development in
their spare time for free don't drop what they are doing and fix these
bugs, then you won't be able to sell Sage to a defense contractor,
like the 1 million lines of code my company sold once.

- [ ] I am asking whether the Sage development community would be
amenable to accepting patches that fix these issues? If so, I may
have some followup questions.

- [ ] You guys must allocate some of the fulltime paid Sage developers
to fixing these issues, because ...

- [ ] None of the above.

-- William

(I of course know you personally, and know that you are a very serious
skilled professional software engineer and computational number theory
researcher. However, the tone of your message is a bit hard to parse
in email.)


--
William (http://wstein.org)

[Matthias Koeppe]

On Friday, December 17, 2021 at 2:33:55 PM UTC-8 Randall wrote:

I am building sage version 9.4 from source on my OpenSuse Linux Leap 15.2 OS and like the install document stated, it has taken hours (using a HP Z420 workstation with 6 Xeon processors) 

[...]

2. I noticed that sage recommended several OpenSuse system packages on the zypper install line, but I did run this and believe it or not, nothing was available.

Your distro is too old. Consider updating to Leap 15.3 or use OpenSuse Tumbleweed.

 

[Randall]

William:

Personal email is on the way to you.

Randall

[Randall]

I think you are right, I need to move up to Leap 15.3 as you have suggested. My past experience with the Tumbleweed releases is that they are too unstable. Once I had a mix of graphics drivers and got sudden complete desktop lockups forcing a manual reset by hitting the power switch, unfortunately hammering the drives and inserting disk cluster errors or inode problems into the disk files and inodes.

[John H Palmieri]

On Friday, December 17, 2021 at 2:33:55 PM UTC-8 Randall wrote:

Hi all:

I am building sage version 9.4 from source on my OpenSuse Linux Leap 15.2 OS and like the install document stated, it has taken hours (using a HP Z420 workstation with 6 Xeon processors) 

I have noticed some odd things occurring which caught my attention as at one time in my life I was in charge of 1+million line code handoff to another company and our code had to be very high quality.

Here's what caught my eye:

1. dozens of warnings during Cythonizing of comparing signed variables to unsigned. This warning needs to be heeded and the code fixed

2. dozens of warnings of compares using uninitialized variables. It goes without saying that using uninitialized variables is only asking for trouble later on. These need to be fixed also.

3. hundreds of warnings during package compilation, the source code does need improvement. I did notice that some C programmers were attempting to write C++ code, but this is asking for trouble too. C++ is not advanced C, despite the popular (mis) belief in this.

I am attempting to install the optional packages just to see what happens, for the build process, but it looks like bundling several packages (say x) on the ./compile line is only asking for repeats of the whole compile run x-times. It should be possible to install all on only one run, not x-runs.

Do you mean the "./configure" line? If you do "./configure --enable-qhull --enable-libsemigroups" at the start, then "make" should just increase the build time by whatever it takes for those two packages — it shouldn't repeat anything.

I have some questions here

1. Will "make all" build all the optional packages and avoid the experimental?

(I did NOT try make all, because just doing make itself chewed up about 9 hours time)

No, "make all" just builds the standard packages and the documentation.

9 hours seems way too long. Are you building in parallel (step 7 in https://doc.sagemath.org/html/en/installation/source.html#step-by-step-installation-procedure)? That should speed things up. For instance it takes under an hour on my 2017 iMac, not a super-fast machine, although it does use some system packages to avoid building time-consuming packages like gcc/gfortran.

 

2. I noticed that sage recommended several OpenSuse system packages on the zypper install line, but I did run this and believe it or not, nothing was available. Is there some consideration to creating a public mathematical software repository where these OpenSuse packages are available? (This probably needs to be extended to most of the public Linux systems)

We rely on package managers for the different linux systems to produce packages and then we use what's available. (Some Sage developers help with those packages.) I don't know how receptive OpenSuse would be to a repository of packages that they hadn't vetted; would they welcome this or view it suspiciously.

For the way we use packages now, see https://doc.sagemath.org/html/en/developer/packaging.html#allowing-for-the-use-of-system-packages and for example build/pkgs/arb/distros/opensuse.txt.

 

3. Can the dochtml installer be decoupled from the regular install, and it run optionally? It  takes up a lot of time for it to discover that everything has already been installed.

Running "make build" will build the standard packages and not build the documentation.

 

--

John