(GTA02) environment update problem in DM2, and solution

[Werner Almesberger]

Last week, we found a problem that caused some GTA02s in the production
line to suddenly lose their boot menu. This turned out to be caused by
a combination of a change to the usual way the environment is handled
and the operator selecting a menu option that reset the environment in
memory.

The fix was easy - we just removed that menu option, since it wasn't
used anyway.

Now, the new use of environment is worth some consideration. The
problem with the environment is that it contains the partition table,
which varies from device to device. The usual approach for changing the
environment is to preserve the partition table across changes, e.g.,

- use u-boot to edit the environment

- dfu-util -U it to the PC, edit it with envedit.pl, then send it back
  with dfu-util -D

- nanddump it, copy it to the PC, edit it with envedit.pl, then send it
  back and nandwrite it

The alternative approach is to ignore the partition table in the
environment, and to recreate it after the editing. This simplifies the
process considerably, because one can now just copy a pre-fabricated
binary environment to the device, and then do a "dynpart; saveenv" from
u-boot, and it has the correct partition table again.

So far so good. Unfortunately, a pre-fabricated environment was also
written in the "final" partition update, i.e., at the very end of the
production line. And there was no partition table update through u-boot
after this.

So all devices would leave the factory with the same partition table,
which then didn't quite reflect the bad block situation. Oops.


Now, there are a few possibilities how we can solve this problem:

- use a fixed partition layout, as I've briefly described at the end of
  http://lists.openmoko.org/pipermail/openmoko-kernel/2008-April/002238.html

  This would be a bold step in the direction of simplifiction, but it
  would also require changes at various places in the production
  process, and we don't have enough information about the distribution
  of bad blocks to verify whether the hypothesis that bad blocks are
  evenly distributed is really true. (I'll post a bit more about this
  topic later.)

  So this change would be risky, particularly given that we don't have
  much time to play.

- just boot u-boot and rebuild the partition table

  This would do the trick, but at this point, there is no convenient
  way to do this (i.e., a boot menu item), it would add another step
  to the production line, operators would have to be trained, etc.

  So this approach would have a high overhead.

- use a PC for the final update, like we did in HXD8

  If a PC is involved, it can just retrieve the environment, modify it
  with envedit.pl, and send it back, as described above. Unfortunately,
  involving a PC when there isn't one already isn't trivial, ...

  So again, too much overhead.

- Now, all we would need that PC for is to run envedit.pl. If we could
  run it on the GTA02, this would solve the problem.

  The reason we can't run it directly on the Neo is that we don't have
  Perl on our rootfs. Perl is a fairly large package, so I wouldn't
  feel very comfortable to ask to add it just for envedit.pl.

  Also, envedit.pl calculates a CRC in a fairly slow loop. This is slow
  on a modern PC, so it would be dreadful on the Neo. (In the GTA01
  days, the environment was 16x smaller, so the speed was not an
  issue.)

Of all these solutions, the last one is clearly the one that would
upset our current production process the least. Since the core
functionality of envedit.pl isn't extremely complex, I rewrote it in C,
and it can now be used on the PC as well as on the Neo.

There is one twist: envedit.pl also implements a cpp-like macro
processor. Doing this in C would have been more demanding work.
However, since the file with the environment changes does not depend on
the device or its partition table, we can just pre-process this file
with a Perl script, and then use the preprocessed version.

The new envedit is in svn.openmoko.org/trunk/src/host/envedit/
The preprocessor is in svn.openmoko.org/trunk/src/host/envedit/envcpp.pl

- Werner