[Insight-developers] Testing patches on all platforms

[Matt McCormick]

Hi Dirk,

I agree that it would be excellent to make this happen, and I
empathasize with your pain, but there are some difficult technical,
resource, and people hurtles to overcome to make all the dashboard
builds report in Gerrit.

The CDash at Home system is a push based system where the CDash server
tells registered clients what and when to build.  All other
submissions are not like that -- the dashboard build maintainer
configures when and what type of a build occurs.  So, we do not have
sufficient control over the dashboard builds to trigger them.  The
dashboard maintainers usually want it this way so one and only one
Nightly build runs in the early morning hours, and they have access to
all the resources on their box when they are finished with their
coffee.

Thanks,
Matt

On Tue, Jul 2, 2013 at 2:47 PM, Padfield, Dirk R (GE Global Research)
<padfield at research.ge.com> wrote:
> Hi Matt,
>
> That makes sense.  Thanks for the clarification!
>
> Given that patches are often reviewed over the course of many days, would there be a way to include their testing in the nightly builds as well?  This would enable us to ensure that, once the patch is merged, the dashboard will still be green.  As it is, we don't know that the patch will cause problems on some architectures until it is merged and wreaks havoc, at which point there is a mad dash to try to fix the problem to make the dashboard green again.  It seems a bit backwards given that we could have access to the same information before the merge.
>
> Here is one example.  Let's say I have a filter that has a tolerance parameter that enables me to tune the required accuracy of the output result.  I don't want to set this tolerance too high because then errors will get through.  So I set it relatively low.  The continuous builds are fine with it, so the patch is merged.  Then I find that it doesn't pass on some other architecture.  So I loosen the tolerance a bit, submit a new patch, wait for it to get merged (after being reviewed by multiple people again), and then find that the tolerance is still too strict!  I go through this loop of change tolerance, submit patch, merge patch, cry when I see the dashboard multiple times until the dashboard is green.  It would be a lot easier to get this information before merging and then merge once.
>
> Thanks,
> Dirk
>
> ________________________________________
> From: Matt McCormick [matt.mccormick at kitware.com]
> Sent: Tuesday, July 02, 2013 10:35 AM
> To: Padfield, Dirk R (GE Global Research)
> Cc: <insight-developers at itk.org> Developers
> Subject: Re: [Insight-developers] Testing patches on all platforms
>
> Hi Dirk,
>
> The Gerrit builds are Continuous builds that register themselves with
> the CDash at Home system.  Most of the builds on the dashboard are
> Nightly builds that the owners configure to only submit once per day.
> So, checking the Nightly dashboard the next day is still a must-do
> part of the development process.
>
> Thanks,
> Matt
>
> On Wed, Jun 12, 2013 at 2:47 PM, Padfield, Dirk R (GE Global Research)
> <padfield at research.ge.com> wrote:
>> Hi All,
>>
>> This may be an obvious question, but is there any way to have a gerrit patch tested on all platforms before it is merged?  I have had a couple of instances where I submitted a patch, it passed the robot builds, but then when it was merged it failed on some other platform.  Being able to test on all machines before merging would reduce the incidence of de-greenifying the dashboard.
>>
>> Thanks,
>> Dirk
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Kitware offers ITK Training Courses, for more information visit:
>> http://kitware.com/products/protraining.php
>>
>> Please keep messages on-topic and check the ITK FAQ at:
>> http://www.itk.org/Wiki/ITK_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.itk.org/mailman/listinfo/insight-developers