VTK/Multicore and Streaming

Multicore and Streaming

Background

This page is to act as a facilitator for current efforts in adding advanced multicore and streaming functionality into the VTK framework. We will be building on some past work by others, and this page is meant to facilitate discussion and collaboration of those contributing via a number of synergistic research efforts.

Discussions

Meeting - 9 June 2009

• Brian would like to have the pipelines on the wiki along with the discussion, design, use cases, etc.
• Berk would like to modify Huy's current changes (in git) to leverage the existing streaming classes in vtk. Huy indicated that the current streamers require extents calculation, which might not always be available. Berk suggested that we look at using pieces instead of extents.
• Need to subclass the composite data pipeline with support for temporal data.
• Huy is still working on Push model. It's implemented but not complete and accessible.
• Berk would like more of the functionality available in the executive. VTK should be changed rather than putting a band-aid on the current implementation.
• Tim will send Huy a simple text pipeline that he would like to have streamed (this includes tables.)
• Need to start moving text analysis filters into VTK proper.
• Huy will remain in the git branch to continue experimentation with the streaming, and we're going to start implementing a text streaming set of examples for testing the new functionality.
• Berk suggested that we need to develop a 'vision' statement (with use cases) for how we want the new multicore/streaming algorithms to work together.

Action items

• Huy will update the wiki to include the papers and a graphvis version of the pipeline on the website that can be used to facilitate discussion
• Tim/Jason will start moving TextAnalysis kit into VTK proper
• Tim/Huy will start developing examples demonstrating text streaming as a means of vetting the streaming/multicore functionality that Huy has already developed.
• Jason/Berk/Huy need to develop a vision statement and place it on the wiki.
• Jason/Berk/Huy need to develop a list of goals and use cases for uniting this work to current efforts and desired efforts in VTK.

Meeting - 8 July 2009

• Updates and Current efforts-
  • Leveraging the existing VTK streaming structures (see http://www.vtk.org/Wiki/VTK/ThreadedStreamingPipeline)
  • Pieces vs. extents
  • subclassing composite data pipeline to access temporal data
  • moving functionality to the executives
• Other efforts-
  • Development of a push model
  • Text analysis pipeline
• Multicore vision statement
  • vision statement
  • new proposals (CSRF10)
  • uniting this work to current and desired efforts
• Wiki updates

Action items

• Huy will work on updating the wiki with the latest technical information and discussions.
• Huy to continue implementation and completion of technical items from last meeting, as well as some incorporation of discussion regarding using extents correctly.
• Berk will start a draft vision statement for the multicore work. Claudio and Jason will follow up with edits.
• Jason will move the text analysis filters from vtkSNL to vtk to facilitate creation of a more comprehensive streaming example. We will meet in two weeks to discuss implementation details of this new example.
• Jason to call Claudio regarding potential new proposal.

GraphViz Test