[Stable] Unable to compile GTA02 kernel without CONFIG_MACH_NEO1973_GTA01 set

[Werner Almesberger]

Nelson Castillo wrote:
> I already thought about it and I will write a Python program really
> soon. I would like to start with something that is not that smart but
> that can be very useful.

Okay, the test harness that runs the builds is certainly important.
It'll also be good to use that one on "standard" configurations we
(Openmoko) or others recommend or at least provide. I would expect
that the number of such configurations would increase with time, as
people add "special interest" scenarios that may not be suitable for
general use.

By the way, another way of failing are new compiler warnings, which
may even herald functional defects.

> I think this is very simple and useful. Then I'd like to ask for some
> feedback and add the automatic tests since we have to think about some
> issues that might come out generating variations of existing
> configurations... I mean, I'm sure I'll think more about this problem
> while I build the basic program I proposed.

Yup. Just turning options on and off (or to some other value, if
they're not boolean) randomly is one way of exercising the
configuration system, but this may create a very large state space
that is time-consuming to explore.

Whether this is an issue or not depends on the structure of that
state space. E.g., if no option depended on another, you could test
all possible pairs of boolean options with about 99% probability in
20 random runs. But if, say, a number of options only become
available if ten other options each have a specific value, this
would take much longer.

It should be easy to test whether this is an issue or not: just take
an old revision from the repository, run the random configuration
test, and see how many of the configuration bugs fixed since then it
finds.

If it finds them all within reasonable time, then the simple
algorithm is sufficient. If it misses a lot, then this needs a bit
more thought. E.g., we could eliminate options our own code doesn't
use.

By the way, some of the Linux testing projects may also have done
work in this direction, so they may be useful as sources for ideas,
code, and maybe even testing services for things we put into
mainline.

- Werner