Openmoko build infrastructure questions

[Werner Almesberger]

Holger Freyther wrote:
> Regarding distribution tool:

Once upon a time, I used SLS, Slackware, then for a long time Red Hat,
a bit of Debian, finally for a long time Gentoo, and now more and more
Ubuntu. Except for Gentoo, I never worried much about the distribution's
build process.

Why should I ? Either the distribution provided a binary package for me,
or some 3rd party did, or, if all else failed, I compiled it from
sources, and did any fixes/hacks that were needed to run it on Linux
myself. In those cases where that looked like to momentous a task, I put
the darn thing aside for a few months and then looked again if someone
had gone through the effort of building a friendly package for me.

Guess what, that friendly package usually materialized before too long.
And nowadays, it's very rare that the nice package wasn't already there
long before I even thought I'd need it.

Why should Openmoko be any different ?


>From the software author's point of view, I learned how to write Red Hat
spec files, because Red Hat what I was using all the time back then, and
it wasn't too much effort for me to do so. However, if someone sent me
the respective meta-data for Debian or Gentoo, I checked that for obvious
errors, and if it looked reasonable, I included it in the distribution of
my package. When necessary, they sent me updates, or (less good but
popular) they fixed it in their distribution. (I hated it when they
fixed portability issues and didn't tell me, though.)

Again, Openmoko/OE does that. Since we have quite a number of people in
our team who understand OE very well, they also write their .bb files.
If I need one, and it's not absolutely trivial for me to write it myself
(i.e., beyond mere copy and paste), I just ask them to help me.


> OE as a development tool:

This is where I see the big problem area. Why should I need substantially
more than  make CC=arm-angstrom-linux-gnueabi-gcc  ? I don't expect this
to figure out dependencies for me. I know that make doesn't do such
things, but there's probably a README that will warn me about unusual
stuff.

I don't want it to build a cross-compiler for me. And I certainly don't
want it to install a secondary system environment for me in which I'm
supposed to live while trying to compile and perhaps change that package.

If the magic phrase is configure --target=mumble --prefix=blah, so be it.
Perhaps a little README.Openmoko or README.OE could guide me. That could
come with the original package or Openmoko/OE could provide it. The
height of luxury would be a shell script that contains the few lines of
wisdom I really need to care about here.

This doesn't have to cover all possible configurations and all that. Just
make the thing build. The fine-grained embedded optimizations, whatever
they are, will most likely just bore me. I can very well live with a
scenario where the thing that's my current focus takes up a few MB more
space on my Neo, or doesn't merge itself seamlessly into my setup.

I'll expect any package that's really worth keeping will eventually find
its way into the distribution, so all these issues will be solved for me
in the future when I wish meet the same package as a mere user, not as a
developer. If it ends up a permanent local addition with no support from
upstream, then I'll most likely abandon it as well, since it's just not
worth the effort.


So this last role, that of the development tool, is where I see OE fail
us. All I need is the small task of running the right build command,
and maybe set up a few files and fix a few portability issues in the
sources. Instead, I get asked to spend at least a day and significant
system resources to set up and build a monster machine that will let me
become my own distribution czar, and - almost as an afterthought - will
also let me build that package.

Chances are that I need this service for a few days, and then all the
infrastructure and the knowledge how to use it will rot for months, and
I'll have to throw it all away and start from scratch again the next
time I need it.


That's why I love John's toolchain. It does exactly what I need to
successfully build things with "make", and I can download and set it
up in a few minutes, using blatantly obvious operations. It doesn't
force me to join a new religion, it doesn't force me to learn some IDE
and adapt my workflow to its logic, it doesn't waste my time by
building its own microcosmos where it can be god. It just makes sure
that the same sane build process I know works for native builds
continues to work sanely also for cross-compilation.

Now, there are obviously limitations to this approach. E.g., if
something really doesn't want to be cross-compiled, I'll hit a problem.
Also, if it needs libraries John didn't think I'd need, there's another
problem.

Packages that are just plainly hostile to cross-compilation need
patching. So if I have to touch their source, I'd need some way to
find those patches. Ideally, they would have been merged upstream long
ago, but I understand that this sometimes may not have happened yet,
or that some developers are just unresponsive. That's the first problem
to solve.

>From this discussion, I gather that Poky is basically the natural
extension of the toolchain approach that solves the second problem,
namely the libraries something may depend on. Is this so ? If yes, can
we make OE and Poky work together, so that each is used where it is
strong, and lets the other take over where it isn't ? Or if they can't
work together, can OE at least steal the good ideas ?

- Werner