custom kernels and heterogeneous hardware

Mark Komarinski mkomarinski at wayga.org
Wed Jun 21 12:24:00 EDT 2006


Paul Lussier wrote:
> Hi all,
>
> Please excuse yet another on-topic post, I'll try not to do it too
> frequently ;)
>
> I'm wondering how others handle needing to deploy a custom kernel to a
> wide variety of hardware.  We maintain a "single" kernel for all of
> our hardware, both for in-house systems and for the platforms we ship
> on.  At one time we used to just compile everything into a monolithic
> kernel, but the variety of hardware is getting big enough to make that
> impractical a this point.
>
> We're considering configuring our kernels such that everything is
> compiled as a module, however, that means you need to know what to
> load at boot time.  On Debian, you place what you want loaded at boot
> time in /etc/modules and it gets loaded.  We could just add an entry
> for everything in there, but that might make for a rather noisy boot
> with all the non-existant devices it tries to load drivers for.  On
> the other hand side of the coin, we don't want to load a bunch of
> modules for stuff which won't be used.
We've got a cluster with 4 different hardware types plus two 
interactive/web servers plus another fileserver all running off the same 
kernel.

We're doing exactly what you mentioned - put everything we want in 
/etc/modules.  In most cases, if the hardware doesn't exist, the driver 
doesn't load, so it doesn't take up any additional memory.  It makes the 
boot process more noisy, but we don't reboot all that often and it's 
only a few extra lines in a few hundred of the regular boot process.

The other important thing to remember about using modules is that you'll 
need to make an initrd.img to load in modules that are required to boot 
(like your SCSI or RAID controllers).  We make an initrd.img that has 
'most' drivers loaded and then add in ones we specifically know we need 
to boot all our machine types (piix, ips, mptscsih, etc.).  Same deal - 
drivers for hardware that doesn't exist on that machine type don't get 
loaded.  With the initrd.img created, that gets you enough to mount /, 
and then put in the non-critical drivers like Ethernet into /etc/modules.

CAVEAT:  The main cluster is still running a 2.4 kernel.  Our test 
cluster has a few 2.6 systems on it and so far it seems to hold to what 
I said above, but YMMV.

-Mark



More information about the gnhlug-discuss mailing list