loadable kernel modules patches
Peter MacDonald
---------
This is to announce patches against linux 99p14f that convert nearly all
Linux devices and facilities into loadable module. There is also
a bootstrap loader to bundle modules up with a kernel image.
There is initial support for /proc/dev. This is a testing release.
Updated patches will be made in a week or so. The patches can be found
on tsx-11.mit.edu. Both the patches and the pre-patched kernel source
are in:
tsx-11.mit.edu:pub/linux/packages/SLS/test/
The relevant files are modutils14f.tgz and linux99p14f-mod.tgz.
LOADABLE DEVICES:
-----------------
As well as regular devices, such as CD's, parallel ports and mice,
all of file systems are now loadable. Ditto for networking code. Even
the unix domain sockets are loadable (perhaps only useful for development).
So also is the sound code, the elf emulator and the math emulator code.
The ramdisk is also loadable and unloadable, and so can have it's
size set at runtime.
The SCSI devices have also been made loadable, but since SCSI has
both low and high level drivers, the concept of composite modules is
introduced. Composite modules simply permit a module to add symbols
to the kernel symbols list that is exported for module linking. In the case
of SCSI, first the generic SCSI code is loaded. Then the low-level (board/
controller) drivers are loaded. Lastly are the high-level (tape/disk/CD)
drivers. The last high-level driver loaded is passed an argument telling it
that it is last, and so to initialize the scsi subsystem. Failure to detect
a board, causes all scsi modules to be deleted.
BOOTTIME MODULE LOADING:
------------------------
The other major piece is a bootstrap loader for linux modules. It allows
for modules to be bundled with the boot image. Essentially, the bootstrap
loader piggybacks a collection of modules onto the end of the kernel
image before it is compressed. At boot time, the modules are initialized
and autodetected as usual. In all cases, a module is unloaded
at boottime if requisite hardware is not present, freeing it's memory.
For math.o for example, this means that detection of a coprocessor
automatically unloads the math emulation software.
POSTBOOT MODULE LOADING:
------------------------
Some drivers, such as disk controller code, need to be loaded at
boottime, but others, such as Network code can be loaded after bootup.
To have the modules loaded automatically for you, you can just
move or copy the desired modules to /install/module/ and add:
if [ -d /install/modules -a -d /proc/dev ]; then
(cd /install/modules; for i in *.o; do insmod $i; done )
fi
to the top of /etc/rc.local. Prime candiates for this are network
and file system code.
Interestingly, object files that are loadable modules are no different
than ordinary .o's in that they can still be statically linked. There
are now three options: include, exclude and loadable. You can identify
loadable module in "make config" by a "*" in the first column.
Selecting 'l' instead of 'y' or 'n' makes the module loadable.
Implementation-wise, loadable modules use the same .o files that the
kernel links in. There is no need to recompile the module if you decide
to statically link it, rather than load it.
PROC/DEV:
---------
An other thing that these patches add is initial support for /proc/dev.
The purpose of /proc/dev is to allow the kernel to report which
devices it currently considers valid. Currently, the device reporting
is not yet very accurate, but eventually this feature
could be used by configuration software, in setting up the system,
and in particular, the /dev directory as AIX does.
MODULE ARGUMENTS:
-----------------
The insmod.c code has also been modified to allow passing arguments to modules
using the -a option. This is used by such modules as ramdisk.o, to allow setting
the ramdisk size. The size can be changed without rebooting by unloading and
reloading with a different value (4Meg is the current max).
There is also a -f option, to force loading even if the version of
the module does not match the kernel version (probably not a good idea).
Arguments passed to modules should at some point be integrated with
the xxx_setup() scheme used by lilo currently.
BOOTSTRAP MODULES:
------------------
It was also necessary to modify the compressed bootup code, because
there was no longer room in the < 640K area for the compressed kernel
and the gzip code to run. So now the kernel is moved to just below
the 2Meg mark before decompression, and the uncompressing data
chases the compressed data boundary.
CONFIGURATION VIA LILO:
-----------------------
A few simple options are currently supported with lilo.
no=module1,module2,...
causes the named modules to be *not* loaded or autodetected.
They are instead disabled and discarded.
yes=module1,module2,...,moduleN
Does the opposite. Only the modules listed are autodetected.
The others are discarded. The thinking here is that future
installation programs (like in the upcoming SLS release) can
do the autodetection only upon installation. After which,
bootup should be as fast as a custom compiled kernel. The intent
is clearly to reduce, and perhaps someday eliminate, the need
for most people to compile the kernel.
dma=N
Reserves N blocks of ram for modules at boottime. This
is released by the time init is called. Currently
the default value is 256. It is called dma because
this type of memory allocation is the only kind that can
be used for dma (memory_start math). kmalloc will not
work.
USING IT:
---------
Apply the patches to 99p14f as usual. Then make config.
Just hit enter at each question (except perhaps for the
sound module which if you say yes, choose 230, instead
of the default 220: well, you'll know what I mean).
You will need to compile and install the accompanying module
utilities. To control exactly which modules are loaded into the
bootstrap, you will need to edit zBoot/Makefile. Then type
"make mImage". This will make "mImage" which you can boot exactly
as zImage. After bootings, do "lsmod" to see the loaded modules.
NOTABLE TODO's:
---------------
The (inet) network code, although loadable, needs to be modified to
allow loading individual low-level (board) drivers. Since it is
changing so much, it was not done. The same is true for the
sound modules. Additionally, sound can really only be loaded
at boottime because it needs 1:1 mapped memory for dma. I guess
we need a true dma_malloc().
The console, keyboard and serial code (and ipc?) should also be made
loadable. Better detection of when modules are busy, and hence
unloadable, should also be added. Dependencies (such as in the scsi
composite modules) should be enforce to prevent accidental unloading of
dependant modules.
A (much) better configuration control mechanism should be put
in place.
Better memory management could be used, say removing
the restriction of aligning modules on page boundaries.
In a 20 module installation, this could save about 40K
(average 1/2 page per module). There is currently a hard limit
on the kernel size of about 1Meg. A little creative memory
management in the loader could push this easily to about 1.4Meg.
The math module adds an extra (unnecessary) function call.
Rather than add linker logic code to the kernel, the fixup has been done
at compile time (using a modified insmod.c from the module package).
Perhaps in later implementations this will change. This might
be necessary to save room in the boot image if the kernel
continues to grow.
Future implementations might include auto loading/unloading
of devices on demand (ya right).
I believe that it is a great testament to Linux that this conversion
to modularization was so relatively easy. Great work Linus and company.
Peter MacDonald
pmac...@sanjuan.uvic.ca
--
Mail submissions for comp.os.linux.announce to: linux-a...@tc.cornell.edu
PLEASE remember Keywords: and a short description of the software.