Minix 4 ? eeMinix ? GNU/Minix ?

[Patrick]

Hi Everyone

I re-read the mail archives recently, I have been with this project
since 2012. I get frustrated and walk away but I just keep coming back.
I like the people and the project. I can't always get what I want done
but I learn from each failure.

This past week I looked at the code in some detail for the first time.
Specifically 3.1.0 and 3.4 R6.

I have some concerns and concerns about the direction the project is
heading in.

Please correct my mistakes but from what I can piece together, AT&T sued
BSD and hampered that project for some time. In this time period Andy
started Minix. His students did most of the coding and he oversaw the
project.

Thomas Cort added ARM support and as a fellow Canadian, it is unlikely
that he attended a university in Amsterdam. However, aside from Thomas,
is there much of any code that was committed to the project by a person
who has never spoken to AST or spoken to any of his students? I am in
this category.

My best guess right now is that there was ample student labour to code
in the past but with AST retiring and not involved with the project(last
post June 2017) and with the the Minix group at the university
disbanded, there is now a major labour shortage.

There is little developer documentation and little in the way of
changelogs or other documentation for those who want to continue.

Thomas was a university "outsider" that contributed but aside from him,
it looks like the work was largely done by "insiders" who left very
little documentation.

Minix has weird macros in it like _PROTOTYPE. Looking this up via ctags,
it looks like this was done to work around bad compilers. When was this
needed? back in 1990 ? Now it just looks wacky.

If I have a project called foo and I want you to install it on your OS,
I have to test for library support but if the project /*is*/ an OS, why
do we need a lot of macros? there shouldn't really be external
dependencies and these days if there is a wonky compiler, tell people to
use one that works.

Again, I love the project, please correct my mistakes especially the
negative comments. However if most of this is true, where are we going now?

What is the project's reason to exist? In terms of huge embedded there
is Linux and BSD. In terms of small there are many RTOS such as
FreeRTOS, Contiki, Chiba etc.

Minix is no longer suitable for teaching as it has grown far too large
yet it is not a refined micro-kernel version of NetBSD either and it's
unlikely to make it to this goal considering the manpower shortage now.

There are also design decisions that to me would seem to indict that
today's' situation was not planned for. If Andy had to implement
libraries from scratch because of the AT&T lawsuit and the fact that
Linux did not exist, that would seem reasonable but it seems that they
were always intended to be tied to Minix.

Take for instance the curses library in 3.1.0. It is peppered with
_PROTOTYPE macros. Version 3.4 R6 has curses from NetBSD. This is better
but even this might not be such a good idea. ncurses is well maintained
by Thomas Dickey and it's an X11 license piece of software, not GPL. Why
not just run ./configure to have the most recent version rather then
depending on source code from NetBSD which is several years old.

This isn't about getting other people to do the work, it's about making
wise use of my time and me doing it... What about tearing Minix out of
3.4 R6 and creating a tiny OS that is suitable for teaching again? or
starting over with 3.1.0 and adding in changes from the other versions
if they don't enlarge it too much?

What about reworking it to reflect today's design needs such as
depending on other projects to do the heavy lifting whenever possible.

I could put together a bunch of libraries and commands that come from
NetBSD or FreeBSD or if clearly sectioned off, why not Linux too? The
core could still be pure BSD license with optional GPL code.

What about reworking the code until it can compile under Linux with
chroot? Sure we have build.sh but it's huge. What about a weeny little
tarbar of code well under 100K lines? It's Minix batteries not included
but rock solid and ready to build upon?

Would it be good to call a new effort Minix 4? Or how about
eeMinix(Embedding & Educational Minix. Or what about even Gnu/Minix and
have a BSD "kernel-land" and a surrounding GPL userland?

I have very little free time, please guide me so I can use it well-Patrick

[u-l...@aetey.se]

Hello Patrick,

On Mon, Mar 25, 2019 at 05:57:16PM -0400, Patrick wrote:
> Minix has weird macros in it like _PROTOTYPE. Looking this up via ctags, it
> looks like this was done to work around bad compilers. When was this needed?
> back in 1990 ? Now it just looks wacky.

Some changes are not undertaken, just because the additional cost of
analysis and implementation may exceed the gain and starve other more
useful changes.

> need a lot of macros? there shouldn't really be external dependencies and
> these days if there is a wonky compiler, tell people to use one that works.

It is seldom popular to just tell people to do something.
They may have their own reasons and different criteria.

Who will volunteer to do the necessary changes and testing (both are
quite tedious and do not lead to any new functionality)?

> What is the project's reason to exist? In terms of huge embedded there is

Different people doing contributions (for example your contribution by
doing your research and summarizing your findings) have their own varying
motivations. A community project does not have any other entitlement
than the people's desire to participate in it.

That's why it can be hard to figure out what is "needed" or "best"
for a project.

> Minix is no longer suitable for teaching as it has grown far too large yet

This is stated like an opinion of a teaching professional (?).
Do we know that the majority of teaching professionals share this view?

> it is not a refined micro-kernel version of NetBSD either and it's unlikely
> to make it to this goal considering the manpower shortage now.

Why does it have to be "refined" and how much?

According to Wikipedia

"The main goal of the project is for the system to be fault-tolerant".

May be not too exciting as a community project, but looks fair enough.

With different goals in mind other project(s) may be possibly needed,
whether reusing bits from Minix or not.

Compare this to Minix-vmd, a fork created to meet the developers' needs
and preferences.

> Take for instance the curses library in 3.1.0. It is peppered with
> _PROTOTYPE macros. Version 3.4 R6 has curses from NetBSD. This is better but
> even this might not be such a good idea. ncurses is well maintained by
> Thomas Dickey and it's an X11 license piece of software, not GPL. Why not
> just run ./configure to have the most recent version rather then depending
> on source code from NetBSD which is several years old.

If you are sure that this is important, let you go ahead and package
a Minix distro with the software of your choice. This is separate from
the kernel and this is a lot of work.

As far as I can see, the reuse of a complete existing solution from NetBSD
was a very reasonable practical choice.

> This isn't about getting other people to do the work, it's about making wise
> use of my time and me doing it...

...

> I have very little free time, please guide me so I can use it well-Patrick

It is appreciated that you wish to donate your time in the most efficient way,
but it is the guidance which is a precious resource.

To make optimal choices one needs specific competence in multiple areas,
from system design to leadership, and of course the time to be dedicated
to the analysis.

These things are scarce.

Regards,
Rune

[stux...@gmail.com]

Hi Patrick,

Your message actually touches up on a few things that I wanted to bring up in a new email thread, but I'll probably leave the rest of those items for the time I'm ready to publish said thread.

So I wanted to respond to a few items and hopefully clarify some things from your message:

On Monday, March 25, 2019 at 5:57:22 PM UTC-4, Patrick McC^very wrote:

Hi Everyone

I re-read the mail archives recently, I have been with this project
since 2012. I get frustrated and walk away but I just keep coming back.
I like the people and the project. I can't always get what I want done
but I learn from each failure.

I'm glad that you keep coming back!  Don't give up on the project.  If it survives long enough, maybe some day it will gather enough traction! (Maybe)
 

This past week I looked at the code in some detail for the first time.
Specifically 3.1.0 and 3.4 R6.

I have some concerns and concerns about the direction the project is
heading in.

Overall, there isn't much that can be said about 'direction' with regards to the project.  With no full-time hands at the helm, progress is slow, almost at a standstill.  Fortunately, there are industrious individual developers that have contributed awesome individual fixes to the project, but it seems they're not available for full-time contribution either.  So keep this perspective in mind when thinking about the project (and some of my responses below).
 

Please correct my mistakes but from what I can piece together, AT&T sued
BSD and hampered that project for some time. In this time period Andy
started Minix. His students did most of the coding and he oversaw the
project.

This is old, old, old news.  How old? The lawsuit in question is from 1992 [source].  Minix was created in 1987 in part as a response to "Lions' Commentary on UNIX" no longer being made available for use in the classrom, to ensure that the code had no AT&T intellectual property and to make it minimal [source 1][source 2][source 3].  (I'd encourage reading through these sources.)

So that was the onset of Minix 1.  I would imagine that the very first version of minix was written entirely by Andy himself.  Subsequent versions were improved and developed by him and his students. Now, it's important to distinguish the major versions of minix: 

• Minix 1, 1.5 and 2.0 were written in the 90s and have a very similar structure. They work very well as educational platforms, but have certain limitations in what they can do.
• Minix 3 (released in 2005) is the first attempt at having an OS that serves both educational and real-world needs.  As I understand it, there was a grant from the EU that helped to develop minix into a system that was more fault-tolerant and useful for embedded systems. With these goals in mind there were several changes from previous versions such as BSD licensing and moving drivers out of the kernel and into userspace (to name a few).
• Minix 3.1.0 is the same as the book version.  This is the version that is closest to the previous versions of minix (1,2) in terms of style and inherited code.  That means that older coding conventions are present, possibly even from the 90s. 
  
• Since then, over the last decade or so, Minix has significantly evolved primarily for use as a modern OS (sometimes sacrificing its educational benefits).  A lot of the old coding conventions were deprecated in favor of modern conventions, the original compiler (ack) and userspace tools were abandoned in favor or modern compilers (gcc/clang) and userspace tools (netbsd userspace compatibility).  And minix services were better restructured and abstracted likely to improve compatibility with Netbsd and to add missing features (3.1.0 has fewer, less complex services than 3.4.0rc6).
• Overall, there has been a decade of work to update Minix into a modern operating system useful for modern applications, and so there's a lot of difference in the code structure between 3.1.0 and 3.4.0rc6.
  

 

Thomas Cort added ARM support and as a fellow Canadian, it is unlikely
that he attended a university in Amsterdam. However, aside from Thomas,
is there much of any code that was committed to the project by a person
who has never spoken to AST or spoken to any of his students? I am in
this category.

Most major changes made to Minix 3 were likely made by VU students, but not all.  There have been a few GSOC projects that made contributions and there are small contributions from programmers all over the world. 
Aside from Thomas, I'm sure there were other non-VU volunteer developers that contributed large chunks of time to the project as well.  However, I cannot give you a list, as I was not actively involved.  That said, I don't think that "speaking to AST" or being a VU student currently has much bearing in the future development of minix.
 

My best guess right now is that there was ample student labour to code
in the past but with AST retiring and not involved with the project(last
post June 2017) and with the the Minix group at the university
disbanded, there is now a major labour shortage.

True. There was also money behind that development.  The grant I mentioned earlier paid for a lot of the early development of minix 3, afaik.  Switching to NetBSD userspace is actually a labor-saving cost because it means that Minix developers no longer have to worry about maintaining aging userspace utilities.  This may have started as funding was beginning to run out, but don't quote me on that. I'll defer to someone who was in the trenches.
 

There is little developer documentation and little in the way of
changelogs or other documentation for those who want to continue.

Thomas was a university "outsider" that contributed but aside from him,
it looks like the work was largely done by "insiders" who left very
little documentation.

Have you checked the wiki?  Sadly, it's true that there is no "centralized" document that explains all the nuances of the kernel and the services.  Some of that information is likely buried in the wiki, some of it could be gleaned from the OS book, some of that from various papers and theses that backed many of the major projects that changed Minix 3 into the OS that it is today.  Again, this boils down to a manpower issue: we need manpower to consolidate all the necessary documentation into an easy-to-access format, and if possible, offload missing knowledge from former developers. Not an easy task.

 

Minix has weird macros in it like _PROTOTYPE. Looking this up via ctags,
it looks like this was done to work around bad compilers. When was this
needed? back in 1990 ? Now it just looks wacky.

This is one of those things where you have to come from the right perspective.  I don't have the lines of code where you're seeing this (linking to the source on github would help) but any code from 3.1.0 is old code. There is no benefit in criticizing it, especially if it's not used.  Now if you're trying to understand the original 3.1.0 minix code, that's a different story.  Kindly ask in the group what specific parts of the code do (linking to the exact line number) and most likely a helpful soul will be able to explain.
 

If I have a project called foo and I want you to install it on your OS,
I have to test for library support but if the project /*is*/ an OS, why
do we need a lot of macros? there shouldn't really be external
dependencies and these days if there is a wonky compiler, tell people to
use one that works.

Uh, if you're criticizing C macros... well, I can't help you there.  The C macro system will start a religious war all on its own, but it's there and it's been proven.  It has a lot of downsides (it can make automated code analysis difficult i'd guess and you have to hunt down its definition sometimes) but it also has its benefits (such as allowing for de-duplication of code without hampering performance and resource usage). 
 

Again, I love the project, please correct my mistakes especially the
negative comments. However if most of this is true, where are we going now?

What is the project's reason to exist? In terms of huge embedded there
is Linux and BSD. In terms of small there are many RTOS such as
FreeRTOS, Contiki, Chiba etc.

This is one of the things I wanted to bring up in my future thread: these days there isn't much of a difference between embedded, server and desktop OSes.  And the original grant money has run out, so I think there's little need in emphasizing the embedded aspect of Minix development.  That said, we still have our fundamental problem: lack of full-time volunteers for the project.  That's the first thing that needs to be addressed.

 

Minix is no longer suitable for teaching as it has grown far too large
yet it is not a refined micro-kernel version of NetBSD either and it's
unlikely to make it to this goal considering the manpower shortage now.

Minix 3.1.0 is still suitable for teaching.  And I'd argue that Minix 3.4.0 is also suitable for teaching more advanced concepts.  I don't think it's appropriate to say that it's "too large".  Just focus on the core aspects of the kernel and services and leave everything else as addeundums.  Nevertheless, a concerted effort would be needed to create up-to-date teaching material that centers on the current version of minix.  (This was another point i was going to bring up).  But like you said, manpower shortage is the elephant in the room.
 

There are also design decisions that to me would seem to indict that
today's' situation was not planned for. If Andy had to implement
libraries from scratch because of the AT&T lawsuit and the fact that
Linux did not exist, that would seem reasonable but it seems that they
were always intended to be tied to Minix.

I think you're confusing several historical inaccuracies here. Refer to my discussion above. Also note that Linux 0.1 was originally written on Minix!  At the time Minix wasn't what we today call "open source".  The license allowed for educational use and tinkering, but it wasn't as permissive as the GPL and BSD licenses are.  GNU was trying to build its own GPL OS that would use its userspace tools (these are the origins of HURD [source]) but in the end Torvalds was more successful in building the Linux monolithic kernel.  Eventually, Linux was re-licensed as GPL and compared to Minix 1's restrictive license, allowed it to take off into the OS "juggernaut" it is today [source].  Also note that the GNU project likes to refer to Linux as GNU/Linux: that is, it uses the GNU user space utilities and the Linux kernel.  Their ideal alternative would be: GNU/Hurd, but you could imagine other combinations such as GNU/Minix, NetBSD Users space/Linux, and NetBSD User space/Minix.  Actually, that last one is what modern Minix is really aiming to be.  However, mixing GPL code with BSD makes everything GPL licensed.  Minix's core developers wanted to keep everything BSD based, so a GNU/Minix is unlikely to happen unless a 3rd party creates a fork of minix with the GNU userspace, all licensed under GPL.
 

Take for instance the curses library in 3.1.0. It is peppered with
_PROTOTYPE macros. Version 3.4 R6 has curses from NetBSD. This is better
but even this might not be such a good idea. ncurses is well maintained
by Thomas Dickey and it's an X11 license piece of software, not GPL. Why
not just run ./configure to have the most recent version rather then
depending on source code from NetBSD which is several years old.

I had to re-read this paragraph a couple of times to understand what you were saying.  As before: ignore the 3.1.0 legacy code when it comes to 3.4 discussions.  Now, as mentioned above, the reason we stick to netbsd code is to minimize on user space labor.  That means pilfering... I mean, copying as much NetBSD code as possible, and letting NetBSD developers worry about it.  However, when that is done, Minix needs to "Freeze" NetBSD code so that it has a 'stable' API and predictable behavior known to work with other parts of netbsd.  There are obvious security downsides to that but the savings in labor tremendously outweigh this (especially since minix is not considered by any means to be production ready).  If we had the manpower, we'd be able to quickly re-sync with the latest netbsd and maybe even upstream changes.  Outside of that is pkgsrc: netbsd's userland package repository that lets you get the latest and greatest version of a package that you need.  But the 'core' out-of-the-box ISO needs to be consistent and so it won't have the 'latest and greatest' (ideally you'd offload as much as you can into pkgsrc).  There are other problems with pkgsrc though: eventually the packages will expect newer OS functionality, so that again, requires active developers to tackle that.

 

This isn't about getting other people to do the work, it's about making
wise use of my time and me doing it... What about tearing Minix out of
3.4 R6 and creating a tiny OS that is suitable for teaching again? or
starting over with 3.1.0 and adding in changes from the other versions
if they don't enlarge it too much?

I think starting with 3.1.0 code is too-old of a code base.  You can, actually, have a minimal minix installation that only has servers and nothing else (a large part of the source is netbsd's user space!).  You could, theoretically, just add busybox and voila! you have a minimal minix installation.  Though busybox is GPL iirc, so... licensing.  But as a side project for learning I think it would be perfect.  In my opinion, ripping the latest changes out of minix to make it "educational" would deter those interested in using minix as their 'primary desktop' OS and set that progress back.  I think having people work on documentation would be a better long term solution towards making minix more approachable to the educational market.

If you're interested in Minix development look at the developers guide:

• https://wiki.minix3.org/doku.php?id=developersguide:start
• Also helpful is the OS dev wiki: http://wiki.osdev.org/Main_Page
• I thought there was a guide on how to build a "minimal" minix but I'll have to see if I can find it in my notes. 
  

 

What about reworking it to reflect today's design needs such as
depending on other projects to do the heavy lifting whenever possible.

I could put together a bunch of libraries and commands that come from
NetBSD or FreeBSD or if clearly sectioned off, why not Linux too? The
core could still be pure BSD license with optional GPL code.

 

What about reworking the code until it can compile under Linux with
chroot? Sure we have build.sh but it's huge. What about a weeny little
tarbar of code well under 100K lines? It's Minix batteries not included
but rock solid and ready to build upon?

Would it be good to call a new effort Minix 4? Or how about
eeMinix(Embedding & Educational Minix. Or what about even Gnu/Minix and
have a BSD "kernel-land" and a surrounding GPL userland?

I have very little free time, please guide me so I can use it well-Patrick

You could do all this! But you'll basically have to make and maintain your own fork.  Such as project is unlikely to be forked back into Minix mainline and may suffer from licensing issues (which may mean not even being able to upstream changes back to minix!)  So keep this in mind when choosing how you want to contribute to the project.  If you have specific ideas that you'd like to follow feel free to post them to the group!  I'm sure you'll get some feed back (I can try to reply within my knowledge).

Anyway, I hope this clarified a lot of things and gives you a better picture of the history and (possible) future of the minix project.  I'll try to post my thread as soon as I can in order to discuss the future more in detail.

Thanks again for your time and interest!  (Also, thanks Rune for responding, sorry I couldn't respond to your post directly.)

-stux


 

 

[David van Moolenbroek]

Hi Patrick,

On Monday, March 25, 2019 at 10:57:22 PM UTC+1, Patrick McC^very wrote:

Thomas Cort added ARM support

Given the excellent replies so far, I hope you'll forgive me for limiting this post to a factual correction, so as to give credit where credit is due. Thomas Cort has done substantial work on the ARM port, especially driver-related work; however, it was Arun Thomas who ported MINIX3 to the ARM in the first place. This does in fact further support the argument you make there: Arun was one of the scientific programmers on the VU minix team at the time.

Regards,

David

[Thomas Cort]

On Monday, March 25, 2019 at 5:57:22 PM UTC-4, Patrick McC^very wrote:

Please correct my mistakes but from what I can piece together

...

Thomas Cort added ARM support and as a fellow Canadian, it is unlikely
that he attended a university in Amsterdam. However, aside from Thomas,
is there much of any code that was committed to the project by a person
who has never spoken to AST or spoken to any of his students?

I didn't add ARM support. That was done by others. My Minix on ARM contributions were mainly just a handful of drivers for some chips on the BeagleBone Black, BeagleBone White, BeagleBoard-xM, and expansion boards; other people did the leg work to port Minix to ARM and get those boards booting. I'm not a Canadian and didn't attend university in Amsterdam. Lots of non-VU people have made great contributions over the years, but as you note, the major contributors were/are affiliated with the VU.

- Thomas

[Patrick]

Thanks to Rune, David and Thomas

and a special thank you to Stux for a fantastic post.

Thanks very much for correcting my mistakes and adding useful
information. I look forward to Stux's future post he spoke of. At
present, I still don't really know what I can do to help or if I am even
capable of helping at all. I have perhaps 1-2 hours per week to
contribute and I service scientific instruments for a living and have no
formal background in OS design. If the goal is to create an everyday OS,
I will likely not be able to cope with the size and complexity.

Again, I think the people here are great and it's a very interesting
project. I will jump at a chance to help, if I can.

Have a great day-Patrick

[u-l...@aetey.se]

On Wed, Mar 27, 2019 at 08:23:40AM -0400, Patrick wrote:
> I have perhaps 1-2 hours per week to contribute and I service scientific
> instruments for a living and have no formal background in OS design. If the
> goal is to create an everyday OS, I will likely not be able to cope with the
> size and complexity.

Please do not give up. Remember that "everyday OS" means very different
things, depending on who you ask. Still, may be it is the desired link?

There are probably many people who wish an efficient (light on resources)
and safe/robust (only possible with a constrained amount of code to audit -
the more understandable the better) OS for everyday use.[*]

This means the opposite to "intractably complex".

Many of those people likely also have relevant competence and would be
interested to contribute in various ways, given compelling guidance.

This might work, if this community would be coordinated to act for a common
purpose.

If true, this makes your question about the stated purpose of the project
highly relevant.

"A Posix-like system with the roots in teaching; the testing ground for
a research effort around robustness; embeddable"

This is what Minix looks like now, to me.

This _might_ bring together very differently motivated people but there
would be hardly someone who cares about all those virtues simultaneously.

Given that separately each of these goals is achievable by alternative
means, nobody comes here.

I join your call for a better manifesto! :)

Rune

[*] This is hardly compatible with browsing commercial web as "the"
everyday use but even such use would be possible, given sandboxing and
an additional ABI compatibility layer to some "majority OS"; otherwise
the developers of browsers for the commercial web are not motivated to
port their products to anything than biggest OS environments, due to
their inherent (and apparently intractable) complexity

[Patrick]

Thanks Rune !

I won't give up, I am just going to go into a "holding pattern" for a
while until I know what to do with Minix next.

Have a great day-Patrick

[Tom Chandler]

I have been a long time follower of Minix.  It was fun to work on.  I worked more on the application

side than the system side.  I was able to port Asterisk, php, and mysql client to Minix.  Then

the support for Gcc got dropped and CLANG has issues trying to compile the above.  So I guess

what I am saying is I would love to work on the application side of Minix, but need GCC back supported.

Thank You

Tom Chandler

--
You received this message because you are subscribed to the Google Groups "minix3" group.
To unsubscribe from this group and stop receiving emails from it, send an email to minix3+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Patrick]

Hi Tom

I am trying to build a linux64-minix32 cross compiler this weekend. But
usually every "weekend" project takes a month. I will report back if it
works, maybe it could help you ?

Thanks for responding to my post-Pat

[stux...@gmail.com]

Hi Patrick,

I'm glad to hear that you found my post informative!  I wanted to clarify as much of Minix's history as I could.  I hope I can make my "future" post sooner rather than later.

So, you still expressed interest in contributing to the project.  I wanted to ask a few questions and post some suggestions that I hope can help.

* First and foremost, what is your level of programming knowledge and experience?  Are you familiar with C? If so, how comfortable are you programming with it?   The Minix kernel and servers are 99% written in C (the other 1% being assembly). 
* In my opinion, the best way to get a foot in the door when it comes to understanding Minix (and Netbsd) code is to post NetBSD userland programs to Minix.
** What this means is that userland utilities (such as bash, whois, cron/at, ls, ps, etc.) are ported from the NetBSD distribution (not pkgsrc) into Minix so that they can become part of the Minix distribution.
** Sometimes changes to the source code need to be made so that the utility remains compatible with minix, but as more of the NetBSD function calls are emulated, the process becomes more straightforward.
** The following wiki page has more information on the process as well and current progress (notice I've been working on porting cron/at and are mostly done except I have issues getting at to work for non-root users): https://wiki.minix3.org/doku.php?id=developersguide:portingnetbsduserland
** Also helpful is a step-by-step guide for this process if you're cross-compiling in Linux: https://wiki.minix3.org/doku.php?id=developersguide:portingnetbsdstepbystep
* Once  familiar with Minix and NetBSD, there's a few projects of varying difficulty that could help bring Minix:
** Minix has no kernel threads. Most modern applications expect kernel threads.  As a workaround, programs can be compiled with user space threads with some success.  It would be helpful if pkgsrc automatically compiled all its programs and libraries with user threads.  There is supposed to be a way to turn this on in pkgsrc.  While not a perfect solution it would allow for a lot more utilities to be compiled.
** Pull request 195 changes ARM's compiler from gcc to clang. This pull request isn't ready to merge because all tests need to pass.  Fixing things so that the tests pass would be ideal.  However, this is a much more complicated fix that will require a bit of knowledge and familiarity with the systems in question.
** How can pf be ported to minix?  Ever since since lwip was ported to Minix, the project has moved a step closer towards allowing a minix-based router.  Several individuals have expressed interest in being able to use Minix as a router, but without a stateful firewall and packet filter, Minix cannot do this.  This is also a difficult problem that requires some experience and creative thought.  Afaik, LwIP does not have facilities to make routing possible but perhaps minor changes and creative encapsulation might produce solutions.
** Porting RUMP: this is another big project.  If successful, NetBSD drivers could potentially be ported directly to Minix. More information here.
** Implementing Job Control: https://wiki.minix3.org/doku.php?id=implementingjobcontrol
** Helping me finish/figure out the issues with cron/at:  I can provide the work that I have.  I have some test scripts where everything works except when firing "at" from a non-root user.  We need to use ptrace to figure out what the OS is doing (or not doing) that prevents at from working properly.  I've had nearly zero time to work on this :(.
** Although kind of out-of-date, the wiki also has a roadmap and wishlist that can also be examined.

So, in summary, there are plenty of projects that can be worked on provided we can find people with the right amount of available time and experience for each.   Even then, for those who have time, there are introductory tasks that can help reach that level of knowledge.

Feel free to ask if you have any questions!

[stux...@gmail.com]

Hi Tom.

That's pretty cool about porting applications to minix!  I assume that by Asterix you're referring to the PBX?  That's pretty cool to see minix work as a PBX!   Now, as for compiling the utilities themselves: as mentioned in my previous post, the manner in which applications and utilities are ported to minix has changed. Instead of porting the utilities directly (which can still be done but won't make it into mainline, unfortunately), they need to be able to be compiled via pkgsrc.  This means that either missing NetBSD functions need to be implemented or Minix's copy of the pkgsrc repository needs to be updated, or both! (There has been work on merging that repository into mainline pkgsrc but I'm not sure where that's at.)  Of significant note is Minix's lack of kernel threads.  My money is on the theory that 90% of these utilities not compiling is because of the user threads library not being automatically compiled with these packages.  This is reflected in the most recent pkgsrc build log where the "packages causing most breakage" like libgcrypt, python and ruby, fail due to there being no threading functions available.  I wouldn't be surprised if pkgsrc could be configured to automatically compile with user threads, even packages like php and mysql could compile!

Overall, I doubt that switching to clang is the primary cause behind these compilation failures. As I understand it, clang was designed to be "mostly" gcc compatible, with notable exceptions.  These should be well understood and accounted for in most modern application packages.  Instead, it is more likely that Minix's own limitations in implementing NetBSD's OS interface are to blame.  These should be worked on alongside NetBSD's package manager in order for these applications to work again in Minix! 

Finally, I'd like to mention the switch from gcc to clang: as mentioned in previous emails, Minix aims to be a BSD-licensed distribution.  That means eliminating GPL components as much as possible (or completely) including gcc.  However, gcc is only removed from the base distribution and it is still available via pkgsrc.  Although the default distribution won't use gcc to build its applications and libraries, you're still free to use gcc for your own testing and experimentation.

I hope this helps and may encourage you to consider contributing to the project once again!  If you have any questions, feel free to ask!

-stux

To unsubscribe from this group and stop receiving emails from it, send an email to min...@googlegroups.com.

[stux...@gmail.com]

Errata:

There's a couple of things I've noticed and found out since my last post that I'd like to correct and/or clarify:

• My link to the pkgsrc repository in Tom's email was incorrect. It should've linked to: https://github.com/Stichting-MINIX-Research-Foundation/pkgsrc-ng
• It turns out NetBSD has its own firewall (separate from OpenBSD's) called NPF. There was even a GSOC that had NPF integration as a goal.  Since LwIP has already been integrated, firewall integration may actually be a lot closer than I originally thought.  Nevertheless it will still be a big project.
• Other interesting tidbits I just found:
  
• NPF may be used as a library: http://rmind.github.io/npf/intro.html
• Article on RUMP (includes perspective on LwIP): https://github.com/rumpkernel/wiki/wiki/Info:-Comparison-of-rump-kernels-with-similar-technologies
• Minix's gsoc project repository (not up to date): https://github.com/Stichting-MINIX-Research-Foundation/gsoc
  

[Patrick]

Hi Stux

Thanks very much for your post! I can program in lots of languages
including C and very basic assembly. However and this is a big one, I
have trouble programming with autotools(as a programmer writing it) and
I am just learning bmake. I have spent lots of time fooling around with
programming languages but not enough time fooling around with build systems.

I have been programming since 2004 but it's all part time. I am an
electronics/optics person by day.

Thanks for all the great links. I have mixed feelings about some of
these. After your post, I gave build.sh/x86_cdimage another try.

It works but it's huge. It's NetBSD but not exactly Minix. There are
features that appear to be available for Minix but are not applicable
such as usb images and such. I can still use this and the work flow
would be to add to it and run it again and again while booting ISO
images under qemu and Linux but it's huge. It's not user friendly
either. running it with "--help" gives limited results and no
explanation of how to call "make clean" etc.

The size of the current snapshot is also an "everyday use" Minix, not
particularly suited to education or embedding, it's almost 8M lines of
code ! scary.

I will try to use this but I would like to create a smaller branch ASAP.
I want to use Minix to teach my kids about operating systems in the
years to come. The original draw to Minix was it's size and the fact
that it could be learned in a semester. This is good for the non-student
too, why do embedded work and not understand the low level aspects of
the target. Again, I am dying for a manifesto of some sort.

I will try to port the NetBSD package TinyCC. GCC is a bloated pig but
clang isn't far behind. TCC rocks, about 7-8 times faster. There is a
helpful community and the limited lines of source mean it is potentially
maintainable if the community collapses. It generates 32 ELF, so it
should just be a matter of supplying it with libc which is in turn built
with the right headers.

About kernel threads... could you tell me how to specify user threads
instead?

Thanks again-Pat

[stux...@gmail.com]

Hi Patrick,

Thanks for responding to my questions! To be honest, I'm quite lost when it comes to autotools and bmake as well!  When I contribute i simply figure x86_cdimage and let it roll!  I wait a couple of hours and either an error will pop out or a shiny new ISO is generated! This is slow and cumbersome but it still allows me to get some stuff done and contribute, especially since it's for the simpler task of migrating netbsd utilities.  Obviously, if I had more serious work to do (somewhere in the servers and kernel), this would not be as convenient.

If you want "smaller" minix look no further than the minix subfolder.  It contains the kernel, servers, drivers (drivers/fs/net), core libraries (lib/libc) and a handful of user space utilities.  Most of what used to be in this folder has been moved outside.  Everything you need to build a bare-bones minix is in there. Your "fork" only needs that folder (and maybe a few things from other folders). 

Regarding threads: I went through my notes and unfortunately, I don't have the instructions on how to do this.  The closest thing are these notes for pkgsrc: https://pastebin.com/HVGbBGCz .  Basically, the program needs to be compiled and linked with the appropriate library (and the right flags set).  The information in that link might help you get an idea of what to do, or ask in a separate google group thread.

Good luck!

-stux

[Patrick]

Hi Stux

I am just running out the door to pick up my son but I just wanted to
thank you. Your point about the Minix folder being an option for a
free-standing compile is the holy grail I have been seeking !!!

Now I don't care if it's part of a larger everyday OS, if I can rip it
out and use it like one would a RTOS(not necessary real time) then I
have exactly what I need and an everyday OS in the making as a bonus :)

I feel like my concerns about the project are resolved now and I am
ready to roll :)

-Pat

[M@T]

Hello everyone,

I thought it would be good to jump in and let you all know that i'm also

interested contributing to MINIX.

Unfortunately I don't have all that experience that Patrick has,

and probably this is going to be a bit more painful for me,

but its OK I'm here to learn too.

I'm currently a computer science/math  student with academic knowledge

of software engineering and programming languages.

My interests are primarily coding, algorithms, OS, reverse eng, IT Sec

I have passed experience as a sysadmin guy, and always build scripts

to make my work easier and automated, but that's the furthest I can go in terms

of professional experience in coding.

I have a list of OSs I would like to gain intimate knowledge of:

MINIX, FreeRTOS, Barrelfish, Linux

From the academic point of view MINIX seems the most documented so it

should be easier for a noob like me to understand it as a whole.

Hopefully it's gonna be a win win for both of us, for MINIX project to grow

and for me to learn new skills and sell them one day to my future employer.

Time of course is a scarse resource for all of us, and with the exams

I wont be able to dedicate full time to this project, but I promise I will make

the effort to be constant and start with small things, like porting BSD userland apps

Like Patrick said, in this journey of ours it would be nice to find our Virgilio

willing to guide us.

Enough talking, lets get down to business:

1 - Is it still worth reading and learning the 3rd edition of Tanenbaum's book

along with the MINIX 3.1 and then work my way up to current stable? 

I've read the previous discussion but I'm still

conflicted on this topic. What I want to accomplish here is learn the full

minix source code and understand it all like Linus did back in the days.

Or should I just start with the 3.3 and read only what I find on the wiki

(projects, thesis, assignments from other students...)?

2 - What BSD apps you want ported? Does it matter the order?

Do you have any preferences?

3 - Should I get familiar with NetBSD (is it necessary in order to port apps to minix)?

4 - This I like:

How can pf be ported to minix?  Ever since since lwip was ported to Minix, the project has moved a step closer towards allowing a minix-based router.  Several individuals have expressed interest in being able to use Minix as a router, but without a stateful firewall and packet filter, Minix cannot do this.  This is also a difficult problem that requires some experience and creative thought.  Afaik, LwIP does not have facilities to make routing possible but perhaps minor changes and creative encapsulation might produce solutions.

Im also interested in networking, maybe this is something for the next year for my graduation project.

Patrick maybe we could teamup and start porting BSD apps together, 2 heads are better than 1

Let me know what you all think, Im open to any suggestion, like I said some initial guidance would be needed.

Cheers,

M@T

[Patrick]

Hi Mat

Thanks for your post! I am sorry to say, that I won't be much help. My kids needs have intensified and I have a lot of home schooling work to do with them on top of my regular work.

I think it's nice that there is a Minix folder in what is now mostly a BSD distro but I do think that the 3.1 version was better in some ways.

I think it's worth reading the book and playing with 3.1. It was still small enough to be study friendly.

The later releases of Minix are pretty large and there is no book to accompany them. Andy retired several years ago and hasn't participated in the mailing list so I don't think a new book is coming.

If you want to port over apps, I would start with what interests you. I personally had way more luck with GNU Autotools and I would port this over first. If you build packages with this, GNU Autotools does not have to be installed later and I found it much easier to build with it then whatever autotools ships with it.

You could play with NetBSD but maybe don't. I would read the book and play with 3.1

One thing I like to think about when I set out to do something is what I can learn even if I fail. Reading the book and playing with 3.1 will help even if you give up on Minix later. Struggling to understand one part of larger recent Minix releases/NetBSD might not give you this.

Hope this helps a little-Pat

--
You received this message because you are subscribed to the Google Groups "minix3" group.

To unsubscribe from this group and stop receiving emails from it, send an email to minix3+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/minix3/6482ab31-f71f-48b7-a700-764cd22305ea%40googlegroups.com.