Can anybody use an extra hand?

[Mohamed Katri]

I am new to Minix and although I've used open source software for years, I am also new to the idea of contributing to one of the open source projects.
I am an undergrad CS student and I've always been interested in operating systems, 

I read Andy's Minix book, other books on the topic, and I completed couple of undergrad and postgrad OS courses from MIT's open courseware.
You may call me too ambitious, but I'm particularly interested in one of the projects on the wish list: "giving Minix kernel threads", and I intend to do it.
I understand however that I need to come closer with Minix's code base first.

So, my question is:
Does anybody here need an extra hand? May be a testing or an experimental job that you don't have time for but also don't mind to risk assigning it to a noob?

What about the GSoC lwip integration project? What's the status of that? Where there any tasks hanging after the deadline?

[Tomas Hruby]

The project went pretty well, i will start ustreaming it roughly in october. Yes, there are pending tasks, come back in october and we can discuss possible (small?) projects.

Thanks for you interest
>
> --
> You received this message because you are subscribed to the Google Groups "minix3" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/minix3/-/QYgp4z1_5rYJ.
> To post to this group, send email to min...@googlegroups.com.
> To unsubscribe from this group, send email to minix3+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/minix3?hl=en.

[Ben Gras]

Great offer!

I am collecting a small list of such tasks under 'isolated improvement tasks' at http://wiki.minix3.org/en/Wishlist. Any of those (although some are claimed, see '*') would be a welcome contribution!

[katri]

Great to hear the project went well
I guess i'll find myself something to get me busy till October :)

[katri]

Thanks Ben !
I am inspecting this list, already interested in some of the items.
Think I will pick one of the beginner tasks for starter and will post to the group when I do.

[katri]

Hi Ben,

Could you please confirm i understand this task correctly??

improve mkfs.mfs so that we don't need to statically size the ramdisk (number of inodes and blocks in ramdisk/proto), and there would be no wasted space.

I understand the proto file guides mkfs.mfs into creating the ramdisk image

the required improvement is for mkfs.mfs to create a file system that fits exactly the data provided by the proto file instead of creating a statically over-sized fs.

on my system, mkfs.mfs creates an approximately 130MB mfs of which a few MB only are used.

is that correct?

On Sunday, September 2, 2012 4:28:55 PM UTC+2, Ben Gras wrote:

[Ben Gras]

Hey,

You are mostly right :-).

 - Yes, the suggested improvement is to create a file system that fits the proto file perfectly, and do away with the static sizing. (Both to optimize for space and to avoid the annoyance of having to adjust it when necessary.)
 - 130MB? Not sure what you are thinking of - I am thinking of the ramdisk created in src/drivers/ramdisk. The image is currently 1200 blocks * 4kB blocks = 4800kB, of which currently 836kB are unused.

Also, if mkfs.mfs could create filesystems where directories had indirect blocks, we could save a bit more space by creating 1kB-block filesystems instead of 4kB.

A better solution might be to use a FUSE filesystem that reads a .tar.bz2 directly, solving both problems even better. Either of these, if cleanly implemented and acceptably licensed, would be a welcome improvement.

[Antoine LECA]

Ben Gras wrote:
> Also, if mkfs.mfs could create filesystems where directories had indirect
> blocks, we could save a bit more space by creating 1kB-block filesystems
> instead of 4kB.

Is it really a big winner?
I mean, beside the (modest) space gain, you will drop the default value
used for blocks (and their alignments), which happens to be the correct
value to maximize the usage of the level 1 cache in MFS.

OTOH, I guess having the feature available would allow for testing,
something we cannot currently afford...


Antoine

[katri]

yup, sorry, I mistakenly took the free space reported by Linux when I mounted the ramdisk image to examine it as reference.

I am looking into the .tar.bz2 compressed image possibility

[katri]

Well, I mostly agree with you Antoine, I don't think the mkfs.mfs improvement is worth it.

If we however, as suggested by Ben, find a way to include the ramdisk image as a compressed .tar archive and a filesystem that reads it directly,

the elimination of the cryptic proto files will be very convenient, don't you think?

[pikpik]

Hi katri,

On Wednesday, September 5, 2012 8:43:45 PM UTC-4, katri wrote:

 

If we however, as suggested by Ben, find a way to include the ramdisk image as a compressed .tar archive and a filesystem that reads it directly,

the elimination of the cryptic proto files will be very convenient, don't you think?

I like this idea.

I've been trying to port the FUSE-based Archivemount filesystem (in /usr/pkgsrc/filesystems/fuse-archivemount) for this. If you'd like, you can see where I am with it: http://wiki.minix3.org/en/pikpik/now#Porting_Archivemount_for_FUSE

I can put my changes online, if you'd like. I would be glad if you worked on this instead of me.

Unfortunately, Archivemount uses the GPLv2 license. As far as I know, this means it's incompatible with MINIX's license. Since MINIX uses a BSD license, having copyleft software in a critical part of the operating system would seem like a problem.

If you know of a Tar FS implementation that's permissively-licensed (using an MIT, BSD, or compatible license), then I think that would be awesome for MINIX.

Thanks for being interested!
pikpik

[Antoine LECA]

katri wrote:
> Well, I mostly agree with you Antoine, I don't think the mkfs.mfs
> improvement is worth it.

Sorry, but I do not feel I wrote that; I just challenged the claimed
benefits of the *also* clause.
There is no doubt for me that enhancing prototized mkfs.mfs to make it
handle correctly large directories would be a welcome addition,
regardless of its use for the ramdisk.

And obviously, just improving a small bit mkfs.mfs to prevent itself to
allocate all the size, rather just the used blocks, does not seem to
have the same magnitude order as the other ideas which were spreaded:
that way the count of blocks (which drives the sizes of the bitmaps)
could be set at <32767 (i.e. the minimum when it comes to the bitmnap
when considering 4K blocks) and mkfs.mfs would mark the real count of
blocks in the final superblock; I do not see any actual difference with
a real perfect fit here;
Things are a bit more complex when it comes to the number of inodes: the
obvious way is to count the number of lines in proto, rounding up to
next 64 and done; but it needs a full read of the proto file: clearly it
is above the freetime I can dedicate to it :-(.

> If we however, as suggested by Ben, find a way to include the ramdisk image
> as a compressed .tar archive and a filesystem that reads it directly,
> the elimination of the cryptic proto files will be very convenient, don't
> you think?

I do not find proto to be any more cryptic than dd, sed, DNSzones etc.,
but it is probably because I am somewhat used to Unix. Sorry for that.

Using a tar.anyZ as ramdisk source would then probably lead to using a
different file server (perhaps like NBSD's ustarfs) for the initial
ramdisk, I guess; the obvious advantage is that it would be lighter, so
easier to start; it could also makes mfs cleaner, definitively something
to be welcomed... The bad news I believe is that it still has to handle
the raw bread/bwrite for the unmounted devices...

Also there is an important argument against image ramdisks (and always
has been: see the history of the MINIX code here; or the theme of
initrd): unlike all the other boot-time modules, such an image would NOT
be of an executable format (unless you surround it within an ELF box:
which, albeit an incredubly common workaround, does make the start-up
code more complex.)


Antoine

[Antoine LECA]

pikpik wrote:
> If you know of a Tar FS implementation that's permissively-licensed (using
> an MIT, BSD, or compatible license), then I think that would be awesome for
> MINIX.

[cvs.NetBSD.org] / src / sys / lib / libsa / ustarfs.[hc] ?

Unless you prefer the code already there in MINIX: have a look then to
src/sys/lib/libsa/ustar.[hc]... ;-)


Antoine

[katri]

I have no idea how FUSE is implemented in MINIX, is it even available at boot time?

[katri]

Now I feel like I should've picked another task of that list :D

[katri]

Is the mfs handling of the bread/bwrite documented anywhere?

[Antoine LECA]

katri asked:

> Is the mfs handling of the bread/bwrite documented anywhere?

http://wiki.minix3.org/en/DevelopersGuide/VfsFsProtocol#Device_drivers
(and next section), between the lines...


HTH
Antoine

[katri]

Thanks!

[katri]

pikpik, I can test your ntfs-3g fix for you

[pikpik]

Hi katri,

On Saturday, September 8, 2012 8:19:56 AM UTC-4, katri wrote:

 

pikpik, I can test your ntfs-3g fix for you

Sure! I'll make sure my changes are ready first. 

[pikpik]

Hi Antoine,

Ah, thanks! I guess katri or I will have try that. :) 

[pikpik]

Hi katri,

On Thursday, September 6, 2012 4:22:17 PM UTC-4, katri wrote:

I have no idea how FUSE is implemented in MINIX, is it even available at boot time?

If I understand MINIX's FUSE implementation correctly, only one or more libraries are needed to be statically-linked with FUSE-based filesystems to translate messages going to and from the VFS server. So, FUSE support should be available at boot time.

[katri]

If I understand MINIX's FUSE implementation correctly, only one or more libraries are needed to be statically-linked with FUSE-based filesystems to translate messages going to and from the VFS server. So, FUSE support should be available at boot time.

yup, I figured that out right after I asked the question :D 

[pikpik]

On Saturday, September 8, 2012 4:38:19 PM UTC-4, katri wrote:

If I understand MINIX's FUSE implementation correctly, only one or more libraries are needed to be statically-linked with FUSE-based filesystems to translate messages going to and from the VFS server. So, FUSE support should be available at boot time.

 

yup, I figured that out right after I asked the question :D 

Oh, awesome. :D 

[pikpik]

On Saturday, September 8, 2012 8:19:56 AM UTC-4, katri wrote:

pikpik, I can test your ntfs-3g fix for you

Ok. My changes should still work with the latest changes in pkgsrc.

In case you haven't already started using pkgsrc, here's MINIX's guide to it: Installing Source Packages.

Thanks, and please let us know if you need anything. :)

[pikpik]

Keep in mind that it might eat your hard drive. :)

I'd only test this on data that has been completely backed-up or has no value.

[Ben Gras]

I recently implemented general-purpose munmap(), making it possible to free the boot-time-ramdisk image memory (from a 'random' place within the memory driver). This makes it less urgent / important to trim the ramdisk size, and really only the auto-sizing aspect remains imho.

[katri]

I've already implemented the auto-sizing functionality in mkfs, but then I got lazy and didn't test it yet. Will test it this weekend and will tell.

[Ben Gras]

Interesting! Testing whether your new mkfs also works when cross-compiling is rather important and would be good to test too if you have a suitable machine.

I'm curious to see this auto-sizing mkfs!

[katri]

well, i finally had the time to test it, it generated a 949 blocks ramdisk for the current proto file and i was able to use it for booting. 

I also tried cross-compiling it and it worked just as fine.

you can review my changes https://github.com/mkatri/minix/commit/38ef8b0ace1e348493e059abcf61f5c8f33a1779

[Ben Gras]

Dear all,

I've tested the change both natively and while cross-compiling and it seems fine. Those ramdisk numbers had to change quite a lot in the past so I'm glad that bothersome dependency is gone :-)

I just pushed it to master.

Thanks a lot for your contribution!