[Insight-developers] Process breakdown

[Bill Lorensen]

Folks,

In the past 2 weeks we have seen at least three instances where merged
topics are not compiling on some platforms. When the dashboard turns
red on multiple systems because of compiler errors, we essentially
lose a day to fix prior problems and we fail to detect defects
introduced by other topic merges.

We already have two mechanisms in place to prevent this, but they are
not working.

1) Gerrit robot builds: The sole robot windows build cannot keep up
with the workload. Consequently, topics are getting merged without a
windows build.
2) Continuous builds: It seems we only have a Mac and Sun build. The
Sun is really fast. But we lack a windows continuous build.

As we approach the alpha and beta deadlines we will see a flurry of
merges. We cannot afford to have a broken process.

Until our automated processes become more robust, I suggest the following;

1) For gerrit, if you only have a linux (windows) system, do not merge
your topic until the robot for windows(linux) has a clean build/test
of one of the reviewers verifies the topic on their windows(linux)
system.
2) If you merge a topic, manually watch the ITKv4 Continuous
dashboard. If you're luck, the fast Sun build will help catch possible
windows error.

At this point I'm sorry to say that we are in worse shape than when we
had a cross section of continuous builds. Altough Gerrit is helping us
review code, it is not preventing dashboard disasters.

Bill