[ITK-dev] [ITK] New Highly Parallel Build System, the POWER8

[Chuck Atkins]

(re-sent for the rest of the dev list)
Hi Bradley,

It's pretty fast. The interesting numbers are for 20, 40, 80, and 160.
That aligns with 1:1, 2:1, 4:1, and 8:1 threads to core ratio.  Starting
from the already configured ITKLinuxPOWER8 currently being built, I did a
ninja clean and then "time ninja -jN".  Watching the cpu load for 20, 40,
and 80 cores though, I see a fair amount of both process migration and
unbalanced thread distribution, i.e. for -j20 I'll often see 2 cores with 6
or 8 threads and the rest with only 1 or 2.  So in addition to the -jN
settings, I also ran 20, 40, and 80 threads using numactl with fixed
binding to physical CPU cores to evenly distribute the threads across cores
and prevent thread migration.  See timings below in seconds:

ThreadsRealUserSysTotal CPU Time201037.09719866.685429.79620296.481*(Numa
Bind) 20**915.910**16290.589**319.017**16609.606*40713.77226953.663556.960
27510.623(Numa Bind) 40641.92422442.685432.37922875.06480588.35740970.439
822.94441793.383*(Numa Bind) 80**538.801**35366.297**637.922**36004.219*160
572.49262542.9011289.86463832.765(Numa Bind) 160549.74261864.6661242.975
63107.641


So it seems like core binding gives us an approximate 10% performance
increase for all thread configurations.  And while clearly the core-locked
4:1 gave us the best time, looking at the total CPU time (user+sys) the 1:1
looks to be the most efficient for actual cycles used.

It's interesting to watch how the whole system gets used up for most of the
build but everything gets periodically gated on a handful of linker
processes.  And of course, it's always cool to see a screen cap of htop
with a whole boat load of cores at 100%


- Chuck

On Thu, Apr 23, 2015 at 10:01 AM, Bradley Lowekamp <blowekamp at mail.nih.gov>
wrote:

> Matt,
>
> I'd love to explore the build performance of this system.
>
> Any chance you could run clean builds of ITK on this system with
> 20,40,60,80,100,120,140 and 160 processes and record the timings?
>
> I am very curious how this unique systems scales with multiple heavy
> weight processes, as it's design appears to be uniquely suitable to lighter
> weight multi-threading.
>
> Thanks,
> Brad
>
> On Apr 22, 2015, at 11:51 PM, Matt McCormick <matt.mccormick at kitware.com>
> wrote:
>
> > Hi folks,
> >
> > With thanks to Chuck Atkins and FSF France, we have a new build on the
> > dashboard [1] for the IBM POWER8 [2] system.  This is a PowerPC64
> > system with 20 cores and 8 threads per core -- a great system where we
> > can test and improve ITK parallel computing performance!
> >
> >
> > To generate a test build on Gerrit, add
> >
> >  request build: power8
> >
> > in a review's comments.
> >
> >
> > There are currently some build warnings and test failures that should
> > be addressed before we will be able to use the system effectively. Any
> > help here is appreciated.
> >
> > Thanks,
> > Matt
> >
> >
> > [1]
> https://open.cdash.org/index.php?project=Insight&date=2015-04-22&filtercount=1&showfilters=1&field1=site/string&compare1=63&value1=gcc112
> >
> > [2] https://en.wikipedia.org/wiki/POWER8
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Kitware offers ITK Training Courses, for more information visit:
> > http://kitware.com/products/protraining.php
> >
> > Please keep messages on-topic and check the ITK FAQ at:
> > http://www.itk.org/Wiki/ITK_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://public.kitware.com/mailman/listinfo/insight-developers
> > _______________________________________________
> > Community mailing list
> > Community at itk.org
> > http://public.kitware.com/mailman/listinfo/community
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20150423/8d5d6d9a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gcc112-ninja160.png
Type: image/png
Size: 398119 bytes
Desc: not available
URL: <http://public.kitware.com/pipermail/insight-developers/attachments/20150423/8d5d6d9a/attachment-0001.png>