Fixing build with GCC 10

[Peter Shkenev]

Hello everyone,

Currently, Minix can be built with GCC 8. But on my working system I use GCC 10, so I try to fix the build.

Issues which happen when porting on newer GCC versions are described there:

https://gcc.gnu.org/gcc-9/porting_to.html

https://gcc.gnu.org/gcc-10/porting_to.html

So far, I have encountered these issues:

- Make fails to build because of global variable file_debug in make.h . GCC 10 does not tolerate globals in headers declared without extern (-fno-common). This can be worked around by adding -fcommon flag, but this is an error nevertheless. Also, we can try current make from netbsd version.

- The build system uses binutils-2.23.2 and patches it. But this version of binutils cannot be built because of C++ header changes in GCC 10. There are two options: patch this version of binutils, or upgrade the package. I would prefer the latter, but we need then to upgrade patches in the minix 3. But then we need somebody who understand what these patches actually do.

I would like to hear your opinions on these issues.

Best regards,

Peter

[Vishal Oza]

I would try building Minux 3 with the current version of clang first. If the same errors occur then we need to patch Minux 3. If it does not then we should do nothing because this fix is for a version of gcc that conflicts with its licence.

--
You received this message because you are subscribed to the Google Groups "minix3" group.
To unsubscribe from this group and stop receiving emails from it, send an email to minix3+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/79bc9d2e-8862-420c-a3bc-031284e04d86n%40googlegroups.com.

[Peter Shkenev]

Hello,

Thank you for answering.

Clang 11 fails to build the system because of very same issues. I think Clang 10 will do the job, but we still have a problem. Now, the thread is about fixing build with GCC 10 and Clang 11.

One have to do something about binutils.

Best regards,

Peter

пятница, 13 ноября 2020 г. в 16:30:50 UTC+3, vic...@gmail.com:

[Vishal Oza]

This implies that the issue is that the binutils tool set is too old and Minux 3 may not build on newer compiler systems. I think that we will have to replace binutils because patching the system could violate the Minux 3 licensing. If we patch binutils we might  push the binutils version to GPL-version 3 that might also make Minux 3 also fall under GPL-v3

To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/677601fc-2f6a-4c8e-8409-438b3fd0f77cn%40googlegroups.com.

[heitai]

On Fri, 2020-11-13 at 11:49 -0600, Vishal Oza wrote:

This implies that the issue is that the binutils tool set is too old and Minux 3 may not build on newer compiler systems. I think that we will have to replace binutils because patching the system could violate the Minux 3 licensing. If we patch binutils we might  push the binutils version to GPL-version 3 that might also make Minux 3 also fall under GPL-v3

hi,

can you explain me better the licence problem ?

i have an github for up minix to C11 norm, and i will was use BSC tree closes licence.

sorry my english is litle and i not undestand all but go see my github may be a way to solution...

take care

[Peter Shkenev]

Hello,

We use binutils covered by GPLv3 to build Minix. But there are some patches located at external/gpl3/binutils/patches . These patches, being a derivative work of binutils, must be covered by GPLv3, but Minix distributed under BSD License, and this is a possible GPLv3 violation.

However, it is better to ask lawyer or FSF if this is an actual violation. If it is, we should replace binutils with LLVM.

Best regards,

Peter

понедельник, 16 ноября 2020 г. в 15:45:48 UTC+3, hei...@minix-c11.org:

[Vishal Oza]

I believe that this is the point where the issue becomes shaky. I believe the best solution going forward is the replace the binutil with LLVM tools when possible

To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/12aed806-6151-4dbe-85f4-cec57d3c9ef3n%40googlegroups.com.

[Peter Shkenev]

I agree with you, but I just want to sort things out.

понедельник, 16 ноября 2020 г. в 19:03:52 UTC+3, vic...@gmail.com:

[stux...@gmail.com]

Hi everyone,

I've linked this thread to a minix build issue I created:

• https://github.com/Stichting-MINIX-Research-Foundation/minix/issues/316
  

The reason we have the binutils patch is that at the time there was no Clang equivalent to binutils.  Has that changed? Is lld its replacement? I would also agree that switch to a clang-only solution is the way forward.  Now that I see this thread this minix-dev thread makes more sense (please correct me if i'm mistaken):

• https://groups.google.com/g/minix-dev/c/kcSATH-cQow

If there is a pull request ready please submit one so it can be pushed in the future:

• https://github.com/Stichting-MINIX-Research-Foundation/minix/pulls

Thanks everyone for your help!

-stux

[Peter Shkenev]

Hello,

Thanks for creating issue. I've pointed out issues I've met so far.

Best regards,

Peter

среда, 24 марта 2021 г. в 13:59:21 UTC+3, stux...@gmail.com:

[stux...@gmail.com]

Thank you very much for the update Peter!
One thing I should mention is that cross compiling on Linux with GCC performs two steps:

• A minix clang cross-compiler is built using GCC
• Minix is built using this Minix cross compiler

This is part of the reason the compilation takes so long: clang is built twice (cross-compiler clang and Minix native clang).  Both are an older version of clang so I'm hesitant to "fix" this to run on native GCC by editing the source code. If possible, I would like to separate the built even further: could we build clang using clang using tinycc? In other words:

1. Build TinyCC using gcc. (Actually, I just noticed TinyCC is LGPL, so we'd need to find a BSD equivalent)
2. Build the core cross-compiler clang using tinyCC
3. Build the full-cross compiler clang using the core clang
   
4. Build Minix using the full cross-compiler clang

I'm guessing that tinyCC wouldn't be complete enough to build all of clang (or any of it, since clang is partially or fully build in C++ now that I think about it). The goal is to have a much smaller compilation core that depends on an external compiler. If clang can't be built using c in step 2, what about building an older version of gcc instead?

In any case, I beleive such a scheme would increase compilation portability and decrease cross-compilation headaches.  What do you think?

-stux

[Peter Shkenev]

Hello Stux,

I see your point, but I doubt that this would help rather than hurt. It may increase time and complexity of build process. Still, it is worth to try out.

In general, I think the build system can be reworked. Current build.sh approach is quite opaque and undocumented. Also, considering this: https://groups.google.com/g/minix-dev/c/tqpH9W1Sm1U , the current scheme may blow up.

It should be possible to control which parts of Minix are built: cross-tools, userland, kernel, drivers, etc. While this would not speed up build of whole Minix, it would help in development process.

Best regards,

Peter

четверг, 25 марта 2021 г. в 14:12:57 UTC+3, stux...@gmail.com:

[stux...@gmail.com]

Hi Peter,
Indeed, the TinyCC proposal indeed seems to be more work than I originally thought.  Reading on other attempts there may be multiple stages of building the system (tinycc->gcc4->???->clang).  There are other alternatives to tinycc (such as pcc and ack) but they all have to similarly build an early gcc first. Your analysis on the breaking changes also reveals that one of the problems isn't too bad, and the other problem (binutils) should be addressed regardless.  This means that my proposal should be moved to my wishlist instead! ^_^;

As you've mentioned before, the -fno-common issue is easy to fix, so we'll have to focus on replacing binutils with LLVM tools. Looking at the development topic that you linked to, it sounds like you're trying to compile a newer version of Clang/LLVM.  I don't remember what version of clang minix currently has, but my questions about these are:

• What tools do we need to replace binutils? Are lld and llvm-objcopy all that we need? What about an assembler?
• Are these tools available for the currently included version of clang within Minix?
  

I would like to avoid upgrading clang within the current master branch of Minix if possible, but I would still like to replace binutils with the appropriate clang-supported tools.  I hope this helps steer us in the right direction!

Thanks,
-stux

[Peter Shkenev]

I hope to see your results on multi-stage build experiment!

I didn't work with LLVM tools earlier, so I don't know how to replace binutils, if it is actually possible. Also, these patches do not disappear, we need to know what they do to find out if we need to do something about vanilla LLVM/Clang.

Also, I've talked with FSF Compliance lab on the subject of GPLed code in repository. I will post their answer soon. In short, it is legal to store those patches, so we don't have to hesitate with this transition.

As far as I remember, in-tree version of Clang is something like 3.x, so it is really obsolete. Thus, we have to upgrade it sometime in order to get statical analysis and other nice things for kernel and userland.

I can add yet another issue with current build system: it looks like build.sh can rebuild the whole world only.

Best regards,
Peter

четверг, 25 марта 2021 г. в 16:42:40 UTC+3, stux...@gmail.com:

[heitai seishin]

Sorry I haven't time to try on my Ubuntu

Le jeu. 25 mars 2021 à 12:13, stux...@gmail.com <stux...@gmail.com> a écrit :

Thank you very much for the update Peter!
One thing I should mention is that cross compiling on Linux with GCC performs two steps:

• A minix clang cross-compiler is built using GCC
• Minix is built using this Minix cross compiler

This is part of the reason the compilation takes so long: clang is built twice (cross-compiler clang and Minix native clang).  Both are an older version of clang so I'm hesitant to "fix" this to run on native GCC by editing the source code. If possible, I would like to separate the built even further: could we build clang using clang using tinycc? In other words:

1. Build TinyCC using gcc. (Actually, I just noticed TinyCC is LGPL, so we'd need to find a BSD equivalent)
2. Build the core cross-compiler clang using tinyCC
3. Build the full-cross compiler clang using the core clang
   
4. Build Minix using the full cross-compiler clang

I'm guessing that tinyCC wouldn't be complete enough to build all of clang (or any of it, since clang is partially or fully build in C++ now that I think about it). The goal is to have a much smaller compilation core that depends on an external compiler. If clang can't be built using c in step 2, what about building an older version of gcc instead?

No other way that tinyCC ?

 In any case, I beleive such a scheme would increase compilation portability and decrease cross-compilation headaches.  What do you think?

I think compilation portability is better, maybe I test on FreeBSD too.

-stux

On Wednesday, March 24, 2021 at 7:34:32 AM UTC-4 Peter Shkenev wrote:

Hello,

Thanks for creating issue. I've pointed out issues I've met so far.

Best regards,

Peter

среда, 24 марта 2021 г. в 13:59:21 UTC+3, stux...@gmail.com:

Hi everyone,

I've linked this thread to a minix build issue I created:

• https://github.com/Stichting-MINIX-Research-Foundation/minix/issues/316
  

The reason we have the binutils patch is that at the time there was no Clang equivalent to binutils.  Has that changed? Is lld its replacement? I would also agree that switch to a clang-only solution is the way forward.  Now that I see this thread this minix-dev thread makes more sense (please correct me if i'm mistaken):

• https://groups.google.com/g/minix-dev/c/kcSATH-cQow

If there is a pull request ready please submit one so it can be pushed in the future:

• https://github.com/Stichting-MINIX-Research-Foundation/minix/pulls

Thanks everyone for your help!

-stux

--

You received this message because you are subscribed to the Google Groups "minix3" group.
To unsubscribe from this group and stop receiving emails from it, send an email to minix3+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/ce4c4606-316e-4345-9177-86a9d0c6ef2bn%40googlegroups.com.

[Ssessaazi Phrunsys]

Hi everyone I'm so glad talking to you all..... Name is Francis ... I picked interest in ACK compiler and currently from it I've developed a new programming language "scpel" pronounced - spell .... And I have been working on both the micro and mono kernels of Minix ... I have created an artificial intelligence system "Maya" 

I have a lot say and explain but will be glad if you guys (from wherever you are) choose to contact me.... Let's revolutionaries humanity.... I want C and Assembly (Intel) programmers and whoever got good algorithm development skills.... 

To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/CALwZvUgB4uCbysgMniDF70%2BsTrT6wuEt1u%2B1xqxUQxojCixMjA%40mail.gmail.com.

[stux...@gmail.com]

Hi Heitai,

No worries. No rush.  Take your time.

The goal of the tinycc suggestion was to reduce the requirements of needing a specific version of gcc or clang to cross-compile the source.  As mentioned above this would be a completely separate project.

-stux

[stux...@gmail.com]

Thanks! Me too!  Though that is definitely a ways down the road... (the multi-stage build).

I saw the post on the FSF, thank you for checking!  I've bookmarked that link for future reference. (It's in my personal notes.)

Oh, wow, 3.x? I didn't realize it was that old!  Or I guess it's more like, time flies! Okay, I guess we first need to check the exact version of clang used and see if ld tools even existed back then (or if they can even be patched in).

I'm guessing the build.sh issue is that you can't use it to build individual portions of the code? If so, that would be a lower priority for the time being.  The first goal is to reach functional parity with behavior from a few years ago. Also, cross-compilation uses a different set of scripts (I forgot their names) and it just occurred to me that any changes to the build system must not break compiling from within minix either!  Lots of things to tackle...

-stux

[marcelo...@yahoo.com.br]

LLVM is 3.6.1

Why can't we sync the build.sh system with current NetBSD source?

[marcelo...@yahoo.com.br]

I could get i586-elf32-minix-clang compiled with a patched binutils-2.34 after updating and patching some tools (tools/make, usr.bin/xinstall, bin/pax. I don't think they are required, but it made build.sh happy to continue).

Now I'm stuck compiling the actual binutils binaries for Minix (see 1). It seems to expect msync from libc, which is not available/couldn't find.

Or it's just me not understanding how it works. It's here, in case someone wants to see: https://github.com/marceloalencar/minix/commits/make-netbsdcurrent

(1) Output:

dependall ===> external/gpl3/binutils/usr.bin/ld

#      link  ld/ld

/home/alencar/minix/src/../obj.i386/tooldir.Linux-5.11.10-arch1-1-x86_64/bin/i586-elf32-minix-clang    --sysroot=/home/alencar/minix/src/../obj.i386/destdir.i386 -L /home/alencar/minix/src/../obj.i386/destdir.i386/usr/lib   -static  -o ld  ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o plugin.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o eelf_i386.o eelf_iamcu.o ldelf.o ldelfgen.o ldbuildid.o  -L/home/alencar/minix/src/../obj.i386/external/gpl3/binutils/lib/libgnuctf -lgnuctf -L/home/alencar/minix/src/../obj.i386/external/gpl3/binutils/lib/libbfd -lbfd -L/home/alencar/minix/src/../obj.i386/external/gpl3/binutils/lib/libiberty -liberty -lintl -lz    

/home/alencar/minix/src/../obj.i386/tooldir.Linux-5.11.10-arch1-1-x86_64/bin/i586-elf32-minix-ld: /home/alencar/minix/src/../obj.i386/external/gpl3/binutils/lib/libgnuctf/libgnuctf.a(ctf-archive.o): in function `ctf_arc_write_fd':

/home/alencar/minix/src/external/gpl3/binutils/lib/libgnuctf/../../dist/libctf/ctf-archive.c:(.text+0x4e6): undefined reference to `__msync13'

i586-elf32-minix-clang: error: linker command failed with exit code 1 (use -v to see invocation)

*** [ld] Error code 1

[Peter Shkenev]

Hello Marcelo,

Thank you for joining our effort!

Looks like Minix does not have __msync13. I have run readelf -s /usr/lib/libc.so | grep msync in Minix and it returned nothing. Meanwhile, libc in NetBSD current image contains such symbol.

You can pull latest libc from NetBSD tree, but it might not compile if Minix kernel lacks features required by libc. So it's better to disable this binutils build.

Best regards,

Peter

четверг, 1 апреля 2021 г. в 07:23:18 UTC+3, marcelo...@yahoo.com.br:

[marcelo...@yahoo.com.br]

It seems undefining HAVE_MMAP solved this problem (while mmap is available as a system call, msync isn't).

Everything compiles, and I can run the generated image using qemu.

Changes come from NetBSD-current as of 30/03/2021 16:00 (GMT -3).

Now, what can we do to update this ancient clang version? I see it retains the same license until version 8. From 9 onwards it is licensed as Apache 2.

[Peter Shkenev]

Hello Marcelo,

Congratulations! But did binutils preserve its full functionality?

As for Clang, I don't think Apache license would be an issue. Main thing now is that modern LLVM doesn't support Makefile build. We need to reintroduce Makefiles or enable our build sysem to use CMake.

Best regards,

Peter

суббота, 3 апреля 2021 г. в 19:27:29 UTC+3, marcelo...@yahoo.com.br:

[marcelo...@yahoo.com.br]

It seems to be working. Can you test https://github.com/Stichting-MINIX-Research-Foundation/minix/pull/319?

I'll check what NetBSD is doing about CMake for clang. They still use the build.sh system.

On Monday, April 5, 2021 at 3:19:23 PM UTC-3 Peter Shkenev wrote:

Hello Marcelo,

[Peter Shkenev]

Hello,

Test suites in /usr/tests/minix-posix have nothing to do with binutils. You can try to compile binutils within Minix and run its tests.

Best regards,

Peter

понедельник, 5 апреля 2021 г. в 21:35:45 UTC+3, marcelo...@yahoo.com.br:

[stux...@gmail.com]

Just to clarify, all tests need to pass before a binutils update can be pushed (afaik there was one that wasn't earlier but it was fixed in the latest update to 319).

I also wanted to bring up a topic that was discussed in the IRC channel earlier: it's possible that updating clang will break a lot of stuff in the current minix.  This means that in order to update clang, a full sync with the current NetBSD 9.0 user space should be performed. I think it would be best to leave that task for a 3.5 release of minix.  Once Minix can compile on modern gcc with the minimum updates, we should be able to focus on the last items needed for a 3.4 release (which might involve fixing pull request 195).

-stux

[heitai seishin]

Can you synthesis this exchange please, for better understanding.

Thanks

--

You received this message because you are subscribed to the Google Groups "minix3" group.
To unsubscribe from this group and stop receiving emails from it, send an email to minix3+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/44321c9f-ceb3-4b15-b86b-29484a233298n%40googlegroups.com.

[Peter Shkenev]

Stux,

1) Are there any records of IRC?

2) I agree that we should wait with upgrading userspace, but still: can we update userspace without modifying kernel and its userspace services?

Best regards,

Peter

вторник, 6 апреля 2021 г. в 16:27:57 UTC+3, stux...@gmail.com: