Request for stable, automated build process

[Lally Singh]

Just out of curiosity, would maven be completely out of the question?
Package up the dependent tarballs into maven projects, and list the OM
stack as depending on specific versions of them.  Maven can wrap up
the complexity of finding all the various packages for people who just
want to download & install.  If the process of converting a tarball
into a maven project can be automated (even if a few scripts are
necessary for different projects), then the real work need only be
done once per part.

It's also work that can get split up a bit amongst the community.

Would make continuous integration simple, and would act as a nice
wrapper around the current build system.  Hudson
<https://hudson.dev.java.net/> has tools for starting/stopping VMs,
etc as part of the test suite.

Also, being selfish for a moment, it'd be nice to have a build process
that'd port easily enough to Solaris & OS X :-)  It'd also enable some
IDEs to read the project for an easier dev cycle.

If anyone's interested in this line of consideration, I'd be willing
to do a little research into what's needed to set it up.

On Fri, May 2, 2008 at 6:12 PM, Holger Freyther <zecke at openmoko.org> wrote:
> On Thursday 01 May 2008 18:01:38 Bobby Martin wrote:
>  > I sent this before to openmoko-devel and was greeted with a deafening
>  > silence, so I'm resending to a broader audience.  As evidenced by a recent
>  > post to openmoko-devel by saurabh gupta, one of the GSoC selectees, this
>  > really is a problem.  The longer it's put off, the more potential benefits
>  > (in the form of community contributors who give up rather than improve OM)
>  > are lost.
>
>  Well, the reason I didn't reply to the first mail is because it is touching
>  way too many topics to answer all of them in depth and finding a time slot to
>  do this.
>
>  I'm working on the april software update. I have my branch and it has a high
>  probablity of always building. Everything we built for the
>  openmoko-qtopia-x11-image (just for testing) is set to a fixed
>  version/srcdate. I can rebuild the whole distribution without internet
>  (because I don't have internet at home) and I have written a task for
>  base.bbclass to help me to lock down the built to versions.
>
>
>  The obvious downside is: To upgrade certain components I have to spent close
>  to a day to build, and test the result.
>
>  The good thing: I can make sure to have an always building image, which allows
>  you to make phone calls, sms,...
>
>
>
>  > ---------------------------------------------------------------------------
>
> >-----
>  >
>  > I really think OM is shortchanging themselves with the current build
>  > process.  As far as I can tell:
>  >
>  >     * there is no way to run a build that has a high probability of working
>  > all the way through
>
>  Not true for my org.openmoko.zecke.april-update branch because everything is
>  locked down (I can build without internet if I have fetched all sources
>  before)
>
>
>
>  >     * there is no way to identify the latest code that builds together
>
>  You know the SRCDATE, SRCREVS, tarballs you are going to use. The information
>  for a whole system is massive though.
>
>
>
>  >     * there is no automated build process, nor even a stable script one can
>  > follow to do a complete build from source on one's local machine
>
>  Right, I have attached my Openembedded scripts and configuration I use to
>  build on my hardy system. You see it is really basic/simple.
>
>
>
>  >     * it looks to me as if the code lives in at least three different kinds
>  > of repositories: svn, GIT, and mtn
>
>  Yes, we integrate a lot of different projects. They use tarballs on http/ftp,
>  cvs, hg, p4, mtn, git, svn. There is hardly anything we can change about
>  that.
>
>
>  >     * I'm sure that there is a way to identify the code that went into a
>  > particular snapshot found on the web, but I don't know what it is
>
>  The ipk files should point to the recipe used to build the package and has the
>  SRCDATE/SRCREV in the same. So it is easy to find out.
>
>
>  >     * there is no way to identify which tasks definitely have someone
>  > reliable working on them, which are open issues & need attention they're
>  > not getting, and which have some casual developer looking at them
>
>
>  I agree, we don't have an bugmaster at Openmoko who could help on this. For
>  the april software update I mostly track the issues myself and pester the
>  people involved (oh already my 3rd task).
>
>
>  >
>  > I have done several very minor programs for my neo, which I would have much
>  > preferred to integrate into the real build process.  However, when I've
>  > tried the Mokomakefile process, it will run for hours and then die in some
>  > obscure way.  I spent a few hours over the course of several days debugging
>  > that process on my Ubuntu machine at home, before I realized there were
>  > version issues with the assembler that I didn't see any way to work around.
>
>  Three things:
>         - Finding the balance between always stable and cutting edge, specially while
>  in development is pretty hard. I have opted for locking everything down
>  because I lack internet at home which is helpful for what you want as well.
>
>         - You can ask some one of the distro team to add your software to the build
>  process, we are happy to do so.
>
>         - Most of us use debian/ubuntu systems so should be able to help you in
>  getting it built.
>
>
>
>  > I just don't have the energy to spend a dozen hours debugging a process
>  > that will enable me to spend a few dozen hours doing development.  I would
>  > think that many other potential contributors fall in the same boat.
>  >
>  > I think that a few procedures could dramatically increase the development
>  > community's involvement in OM, resulting in faster bugfixes, more features
>  > added sooner, and greater visibility into the project status:
>
>  Yes, I agree. Ading poky's SDK management will help to easily build your
>  software. If you use autotools in a proper way it should be easy to package
>  them. I think our distro team is interested in helping here. The alternative
>  would be to create a feed with user submitted packages but it has the
>  downside of not easily recreating the package (e.g. due changing ABI,
>  compilers, c libraries..., we have been through this with the Sharp Zaurus).
>
>
>
>
>  > There should also be a test harness for developers to easily insert tests
>  > for their code, and for which every bug (except those that require hardware
>  > to test) must have a test inserted for the bug to be considered fixed.  The
>  > test harness should likewise be runnable trivially, with one simple command
>  > (e.g. 'make test')
>
>  Yes, part of the upstream projects:
>         E.g., WebKit has a 300 MB Layout regression test suite, my openmoko rss
>  reader has a couple of tests as well. The thing is properly packaging the
>  tests and executing them on the device ("make test" will not work...packaging
>  the build dir will not work as well). The best thing I have seen so far is
>  bazaars selftest (which fails on ubuntu systems).
>
>
>  >
>  >
>  > *Continuous integration* (CI)
>  > Someone needs to set up a server that on a periodic basis (one or more
>  > times per day) pulls the latest build process script, labels the code
>
>  what is "the code"? Again we integrate hundreds of different projects. How do
>  you label a tarball? Feel free to join my multi-year project Utopia and work
>  on my whole system regression testing and data mining system ideas.
>
>
>
>  > If the process succeeds, then the code labeled CI_CANDIDATE would be
>  > relabeled something like GOOD_BUILD_2008_04_23 and the CI_CANDIDATE label
>  > removed.  Further, the test harness should be run, and if it passes a label
>  > like TESTS_PASS_2008_04_23 should be added.
>
>  Yes, we can use mtn certs to attach to a revision to say if that built or
>  didnt.
>
>
>
>  > *Task tracking*
>  > Maybe this is supposed to happen in bugzilla now, but it seems to me that
>  > we need the kind of bug tracking I was discussing above - a full list of
>  > tasks, priority (something as simple as High, Medium, Low would work fine),
>  > and a way to see if it is *really* being worked or not.  Having a project
>  > manager who is an OM employee for every task would be helpful, since we
>  > could then expect to get an answer if we email a query about the status.
>
>  Yup, we need a bug master. We need one inside Openmoko as well to help us to
>  focus on what needs to be done for a product. For the April software update I
>  kind of assume this position (4th role, but it is sadly mostly on paper and
>  poking the devs directly).
>
>
>
>  > *Documentation*
>  > If the above were implemented, that documentation could be as simple as
>  > telling people:
>  >     * the script location and how to configure what labels are used for
>  > each subsystem
>  >     * the location to download the VM or chroot tarball if necessary
>  >     * the meaning of the continuous integration labels, and which ones to
>  > look for
>  >     * perhaps the place to look for CI build statuses (most recent
>  > successful build, etc)
>  >     * the location of the task tracker and the important fields (status,
>  > priority, assignee(s), project manager)
>  >
>  > Some or possibly even all of this may exist, but if so I couldn't find the
>  > documentation.  In particular, CI and a test harness make a HUGE difference
>  > in the likelihood of success of a project, and the ease of contributing to
>  > the project.
>
>  At the beginning I ran a tinderbox with build results and build failures,
>  automatically filing bug reports. The reality is, no user looked at the build
>  status before starting his build. What I learned from a friend called Laibsch
>  is: There are so many cool things you could automate and write scripts for
>  but there are other things non-technical solutions that help so much more. :)
>
>
>
>  >
>  > I realize that this would cost someone several days of effort, but the
>  > payback wouldn't just be for the community, the developers at OM would
>  > benefit, and the managers and executives would have much greater visibility
>  > into the status of the project.
>
>
>  What are your issues?
>         a) Want to have an always building distribution.
>         b) Want to be able to add your package to the distribution process
>         c) Want to enable users to help fixing issues
>         d) What did I miss.
>
>
>  My take on these:
>
>  a)
>         -The qtopia-x11-image should always built. I think opkg was the last thing we
>  forget to set to a certain revision.
>         - You know which versions got built and can fetch the sources and rebuilt
>  without internet.
>
>  b)
>         - I'm against user submitted feeds as we have no control on the quality and
>  we lack the possibility to rebuild packages
>         - I think it is in the interest of the distro team to assist you in packaging
>  your software. Maybe write distro-devel for a request of packaging?
>         - With the qtopia-x11-image you should be able to easily build and package
>  autotools based projects
>
>  c)
>         - We need someone to summarize important issues, set the right priorities,
>  collect information.
>         - I think this can be done within the community.
>         - The next thing is to actually get the source and fix it. I work on
>  this with the merging of poky's SDK capabilities and also considering debian
>  source packages (work from the mamona guys at indt)
>
>  d)
>         - No idea
>
>
>  this is my take on this... I think we have most of what you want, on some
>  parts I agree with you (issue tracking), on the automation side I do not
>  fully agree, there are better things to do which require less effort and have
>  a bigger impact/success (e.g. a user telling which mtn revision of the OM
>  metadata works...).
>
>         z.
>
> _______________________________________________
>  Openmoko community mailing list
>  community at lists.openmoko.org
>  http://lists.openmoko.org/mailman/listinfo/community
>
>



-- 
H. Lally Singh
Ph.D. Candidate, Computer Science
Virginia Tech