[Insight-users] Itk 4 - Multithreading over multiple nodes of linux high performance computer

[Matt McCormick]

Hi Bilal,

We have started to merge these patches, and we will continue to merge
them in the coming weeks.  Instruction for downloading the Git version
can be found here:

  http://www.itk.org/Wiki/ITK/Git/Download

To get involved in the patch review and development, it would be good
to join the code review hangouts announced on the ITK Bar Camp G+
Community:

 https://plus.google.com/communities/111375098792764998322/stream/a1cf9df1-06f5-4abd-9142-bd8149f6b191

Thanks,
Matt

On Thu, Oct 3, 2013 at 2:21 PM, Bilal Tahir
<bilalahmedtahir at googlemail.com> wrote:
> Dear Hans,
>
> Many thanks for your reply. I would like to try your patches for
> antsRegistration to see if it speeds up my pipelines using the SyN
> algorithm. I am new to the process of applying patches so any guidance on
> what steps are required would be greatly appreciated.
>
> Best wishes
>
> Bilal
>
> On 3 Oct 2013 14:58, "Johnson, Hans J" <hans-johnson at uiowa.edu> wrote:
>>
>> Bilal,
>>
>> Unfortunately the answer is that you can not take advantage of multiple
>> nodes or OpenMP.  The reality is that in many cases ITK does not efficiently
>> take advantage of the parallelism that is available in modern multi-core
>> computers.  My experience is that it often scales well to 4 processors, but
>> rarely takes full advantage of the resources for 16 processors.  This is an
>> ongoing area of improvement.
>>
>> Of particular interest is the antsRegistration example. I have been
>> working on improving the performance of this tool in these patch sets
>> http://review.source.kitware.com/#/q/status:open+project:ITK+branch:master+topic:RemoveFalseSharing,n,z
>>
>> In some cases I have been able to improve the perforce by 70x by
>> identifying the ways to better utilize the existing threads.  It is my
>> belief that you would benefit more from helping improve the utilization of
>> the existing threads than to try to add more threads across different
>> platforms.
>>
>> Regards,
>> Hans
>>
>> From: Bilal Tahir <bilalahmedtahir at googlemail.com>
>> Date: Thursday, October 3, 2013 8:50 AM
>> To: "insight-users at itk.org" <insight-users at itk.org>
>> Subject: [Insight-users] Itk 4 - Multithreading over multiple nodes of
>> linux high performance computer
>>
>> Dear ITK community,
>>
>> This is my first email on this mail base so please bear with me.
>>
>> My query is regarding multithreading over multiple cores. In our high
>> performance Linux Sun Grid Engine, we have 16 CPUs per node. Currently I use
>> image registration software based on ITK 4 (e.g. ants Registration,
>> Elastix)  and set the environment variable
>> ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS and openMP options in my bash scripts
>> to 16.
>>
>> However, some of our Linux administrators suggested to run parallel across
>> multiple nodes to increase the total number of CPUs. Is this possible with
>> ITK 4 and does it require something like the MPI framework? Without some
>> form of message passing I would imagine it will be difficult to envisage
>> taking advantage of multiple nodes from a single program. I heard there were
>> some discussions about MPI implementations of ITK 4 but nothing has
>> materialised yet.
>>
>> Could anyone shed any light as to whether it is feasible to run ITK 4 over
>> several nodes and how to go about doing this?
>>
>> Best wishes
>>
>> Bilal
>>
>>
>>
>> ________________________________
>> Notice: This UI Health Care e-mail (including attachments) is covered by
>> the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is
>> confidential and may be legally privileged.  If you are not the intended
>> recipient, you are hereby notified that any retention, dissemination,
>> distribution, or copying of this communication is strictly prohibited.
>> Please reply to the sender that you have received the message in error, then
>> delete it.  Thank you.
>> ________________________________
>
>
> _____________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Kitware offers ITK Training Courses, for more information visit:
> http://www.kitware.com/products/protraining.php
>
> Please keep messages on-topic and check the ITK FAQ at:
> http://www.itk.org/Wiki/ITK_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.itk.org/mailman/listinfo/insight-users
>