nu-boot :-) (was Re: using kboot for booting)

Werner Almesberger werner at openmoko.org
Thu Apr 24 09:50:34 CEST 2008


Wolfgang Spraul wrote:
> If anybody would want to look into this problem, what would be your  
> approach?

Here we go:

Introduction
------------

First of all, kboot is 90% concept and 10% implementation. Also, it's
only in a proof-of-concept stage, although some quite impressive work
as been built on top of it.

The fundamental idea is to use a Linux system for the part of the boot
process that does something "intelligent", such as interact with the
user or access non-trivial devices.

So the boot process with kboot is as follows:

- firmware initializes the basic hardware
- first-stage boot loader loads the kboot Linux system (kernel with
  initramfs)
- kboot decides what to do and retrieves the files necessary
  (kernel, initrd, etc.)
- kboot uses kexec to boot the real system

I will first describe a basic setup with these items in a bit more
detail, and then discuss some improvements.


First-stage boot loader
-----------------------

In our case (embedded), firmware and first-stage boot loader are the
same thing. The basic hardware initialization is to configure memory
timing, bring up the PLLs, etc., and also to bring up all the
peripherals we need to talk to in order to load the kernel, and to
resolve any conflicts that may exist with the default settings of the
hardware.

Such conflicts could be subsystems that are activated or driven into
a confused state when GPIO pins are in their post-reset state (Z), any
PMU settings we don't like, etc.

If loading the kboot system from microSD, also the card interface has
to be powered up.

Most of the fun stuff for bringing up the CPU can be found in
u-boot/cpu/arm920t/start.S
u-boot/board/neo1973/common/lowlevel_init.S


Stand-alone kernel
------------------

Currently, the kernel expects that u-boot has already done fairly
comprehensive hardware initializations. Since the simple boot loader
outlined above only does the bare minimum necessary, we need to add code
to the kernel to enable it to do the remaining hardware initialization.

This eliminates some rather worrisome code duplication between between
boot loader and kernel.

I would also strongly recommend to develop tools that allow expressing
register settings and particularly GPIO configurations in a way that's
understandable to human beings. This could be in the form of macros or
some scripts that generate header files from user-friendly input. Macros
are probably better, since they don't add anything unusual to the kernel
build process. I'm talking about things like

        gpio->GPGCON = 0x013DFDFA;

This example from u-boot/board/neo1973/gta02/gta02.c sets the behaviour
of 16 GPIO pins. Good luck figuring out to what ;-)


NAND loader
-----------

I'd recommend to start with a NAND-only loader, since it's a bit simpler
to do than an SD loader that also reads FAT. (No rocket science in the
latter either, but it's nice to have quick success :)

The Samsung CPUs have a mechanism called SteppingStone that loads the
first 4kB of NAND into a RAM buffer inside the CPU. That code can then
initialize the CPU, set up the DRAM, and load more boot loader code, if
necessary, or the kernel directly.

I'd recommend storing the first-stage loader in the first NAND erase
page (128kB), and the kboot system in the following pages. That way,
the kboot system can be replaced without changing the first-stage loader,
and vice versa.

The NAND loader also has to check for bad blocks, and skip them. This is
basically the logic we have in

u-boot/cpu/arm920t/s3c24x0/nand_read.c

To tell the first-stage boot loader how much data it has to load for the
kboot system, one could just prefix the kernel-with-initramfs data with
its size in bytes. The uImage format we currenly use has a much more
elaborate header, see u-boot/tools/mkimage.c, but I don't think we want
all that.

An alternative would be to follow the kernel with a data pattern that
is very unlikely to occur inside it, e.g., an erased NAND page. That
way, no special header, trailer, or install tool is needed.

If we want to include a boot parameter line to pass to the kernel of the
kboot system, I'd suggest to simply put it into the second good page of
the NAND, and then to put the kernel into the following pages.


First milestone
---------------

Once the first-stage boot loader (for NAND) is done and the kernel has
been adapted to perform the initializations that have previously been
done by u-boot, i.e., GPIOs and the PMU, we can already boot any Linux
system that's stored in NAND.

Since GTA02 has NOR, u-boot is still around to help with the development.
I.e., one can just boot u-boot from NOR, and then put the new items into
NAND using DFU.

Note that the partitions u-boot assigns don't quite fit our needs, so the
NOR image of that device should either be adapted, or the partition layout
should be changed manually. I'd recommend adapting the NOR image, namely
the u-boot command script in
svn.openmoko.org/trunk/src/host/devirginator/mknor

So this would be a first milestone that shouldn't take too long to reach.


The kboot system
----------------

kboot is basically a small Linux distribution, centered around an
all-knowing Makefile that pulls in packages, configures and builds them,
and cherry-picks the files to include in the kboot file system.

It also contains the shell script that implements the kboot shell and a
few helper programs.

For quick results, one could just try to adapt the Makefile to build for
ARM. However, in the long run, we should look for a solution that scales
better and that does not duplicate work other people have already done
for us.

(The hole goal of kboot is to unabashedly reuse existing work and to give
anything that would lead to errecting some code monument a wide berth.
Survival of the laziest ;-)


The kboot "distribution"
------------------------

Creating yet another Linux distribution isn't such a great idea. I did
this for kboot, because it led to results more quickly and more
dependably than adapting any of the existing distributions available
at that time.

kboot currently builds its own cross-development environment for uClibc,
and then builds the packages busybox, udev, kexec tools, dropbear, and
kbd. (kbd is something we don't need in our case since we don't have
a keyboard.)

I looked into using OpenEmbedded, but it didn't even build reliably
on x86, which is the first platform I targetted with kboot, so I
postponed that exercise.

However, given that OE has matured in the past years and that we have
excellent OE support available, we should use OE as our main source of
packages. If we need more granularity for some items than OE provides,
we can either cherry-pick from ipkgs or break down packages into smaller
pieces, whatever works best.


The kboot shell
---------------

The kboot shell gives a command line from where one can invoke programs.
It doesn't implement any fancy shell features, such as programming
constructs, but it includes an automounter and it also understands
syntaxes like http://... and such.

More details about this shell can be found in the paper and talks
available at
http://kboot.sourceforge.net/

This shell is itself a Bourne shell script. This causes three problems:

- it's slow
- the code pushes what can be sensibly done with a Bourne shell
- it's not entirely bullet-proof when shell meta-characters are used
  in the user input or in file names

All this means that the kboot shell should be rewritten in C. I think
it's best if I do that part.

This rewrite can be done after we have a first prototype.


System integration
------------------

kboot originates from the PC environment, which means that there are a
few assumptions that are not true on our embedded platform. The ones
that I can readily think of are

- the first stage boot loader doesn't let the user pass boot parameters.
  kboot uses the syntax kboot=... to modify its behavior (e.g., to skip
  loading broken configuration files)

- the network is assumed to be on eth0, not usb0

- kboot expects to interact with a terminal or similar. I'll address
  this issue below.

- tools like nandwrite and such should be added to the kboot environment

So there are a few of these small issues to resolve. We should also
explore adding WLAN support (so eth0 comes back :-), etc.


Second milestone
----------------

Once the kboot environment is in shape, we have a boot solution for
NAND. From the kboot system, kernels can then be booted from any device
the kernel understands, and limited system repair can be done from
within the kboot environment, including a shell escape and mounting the
"real" rootfs and running programs that live there.

Note however, that we still need a serial console or SSH to interact
with the kboot system.


Fastpath
--------

Now, always booting another Linux system before we can boot the real
system is relatively painless on a PC, but on an embedded system, both
the system's performance limitations and the very short boot time users
tend to expect don't agree with this concept.

So by default, we should have a faster way to boot into the "normal"
system. This could be accomplished by storing the default boot parameter
line and the default kernel behind the kboot system at a fixed offset in
NAND, e.g., at 16MB, and make the NAND first-stage loader check for some
user indication.

If the user indication is present, we follow the sequence outlined so
far. If the user indication is absent, we start the same process, but
at offset 16MB.

The user indication could be the user pressing on the touch screen.
Or we could also load the default kernel first (which will take a bit
of time), then check if POWER is held, if yes, load and start the
kboot system, otherwise start the default system we've already loaded.

Note: since I mentioned POWER, in order to prevent users from
inadvertently activating their Neo, we currently sample POWER in u-boot
to switch the system off if POWER isn't held for a little while. We
need to think about what we want to do about this scenario.


kboot user interface
--------------------

Since we don't have a keyboard or similar, a menu should be provided.
This could be a text menu like we have it in u-boot in all our existing
products, or it could even be some simple GUI.

Early last year, I saw that the PS3 Linux project had developed a
lightweight but optically very pleasing GUI for kboot. It would be nice
if we could use something like this.

Since the environment kboot provides is essentially a normal Linux
system, it's of course easy to do even fancier things here. E.g., we
could also have the command prompt plus a virtual keyboard, perhaps
by combining the ideas from Raster's "smart" keyboard and the graphical
toolkit underneath the GUI or DM2.


100mA path
----------

In order to do proper battery management we need to boot the kernel.
If the battery is very weak, this may mean that we have to be able to
make it on 100mA USB power alone. We thus need to examine current
consumption along the way, and ensure that it never exceeds 100mA.

If this should turn out to be possible, we'd have to enable some
basic charging (enable charger, USB current limited to 100mA) in the
first-stage boot loader, so that the battery can be brought up to a
level where we can boot.


Third milestone
---------------

At this point, the boot solution provides at least the functionality
u-boot currently has for NAND booting, plus a lot more flexibility
and extensibility.

Some of the system update and recovery process is of course different,
and needs documenting.


SD first-stage loader
---------------------

Our goal for GTA04 is to be able to avoid using NAND. We thus need a
first-stage boot loader that can boot from a microSD card. There are
two functions in this:

- detecting the microSD card and reading data from it

- find and read files from VFAT

We may want to use separate files for kernel and boot parameters.

The SD first-stage loader will also allow us to efficiently bring up a
system without u-boot:

- insert an SD card with the system and tools we need

- load the SD boot loader into RAM with JTAG

- execute it in RAM and boot. Note: the boot loader needs additional
  code to handle starting from RAM, see u-boot/cpu/arm920t/start.S

Like the NAND loader, the SD loader has to have a fast path. In this
case, it's just a switch of file names.

Since GTA01, GTA02, and GTA04 have different MMC interfaces, we have
to decide which ones we want to support. GTA04 is mandatory, and I'd
consider supporting GTA01 inclusive-or GTA02 as highly desirable, also
to give our (soon hopefully incredibly vast ;-) user base a chance to
play with things and give us feedback.


Fourth milestone
----------------

We can now update the production process and then safely bury u-boot.


NOR boot
--------

The final touch of perfection (for GTA02) would be to also put kboot
into NOR.

The first stage boot loader could be completely eliminated, since we
can execute code from NOR in place. It's important to set up the MMU
quickly, so that the user releasing AUX (which removes the NOR from
address range 0x0...) won't crash the system.

The real challenge is of course to cram kernel and initramfs into only
2MB.

Note: instead of starting the implementation with NAND, once could
also start with NOR. NAND needs more code but it also avoids the
space constraints.


kboot improvements
------------------

There are some improvements that should be made to kboot in general,
which will also be particularly beneficial in our environment. E.g.,
the ability to switch root file systems, and have kboot update its
configuration data accordingly.


> Maybe you also throw in the ideas we had about using that opportunity  
> to simplify the partitions and more?

I'll handle this in a separate mail after a nap :)

- Werner 




More information about the openmoko-kernel mailing list