VTK/Composite data changes: Difference between revisions

From KitwarePublic
< VTK
Jump to navigationJump to search
(Created page with "Primarily motivated by better support of AMR data structures, we have introduced changes to various classes related to vtkCompositeDataSet.")
 
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
Primarily motivated by better support of AMR data structures, we have introduced changes to various classes related to vtkCompositeDataSet.
Primarily motivated by better support of AMR data structures, various
APIs related to vtkCompositeDataSet have been changed.
 
=== The New vtkCompositeDataSet ===
 
In previous versions, vtkCompositeDataSet represents a tree structure that
stores the actual data sets at its leaves. Most usage of it, on the other
hand, treats it simply as a data set container that can be
traversed. So we extracted the set traversal APIs to form a fully
abstract class, again named vtkComposisteDataSet. The original
vtkCompositeDataSet is renamed vtkDataObjectTree.  The refactored
hierarchy is shown below. Note that in this hierarchy,
vtkUniformGridAMR switches to a new implementatation (See the third
section for details).
 
As a consequence of the change, the class vtkCompositeDataIterator is
refactored in a similary fashion: Those APIs that support simple
traverals form the new vtkCompositeDataIterator; the original class is
is renamed vtkDataObjectTreeIterator.
 
[[Image:new_composite.png]]
 
===  Demand Driven Composite Data Loading  ===
 
One strategy to visualize large data sets is to make the data loading
demand driven: A data source only produces the data blocks that
downstream filters ask for. We now provide the support for composite
data through two pipeline keys:
 
* vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA(), created by a data
source to provide meta data for downstream filters.
 
* vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES(), created by
downstream filters to request blocks from upstream data sources.
 
====Example====
 
We can look at how a vtkAMRBaseReader source works with a
vtkCompositeCutter filter in a demand driven pipeline. At the request
information pass, a vtkAMRBaseReader creates a meta data object that
has the same type as the output but no actual data blocks:
 
<source lang="cpp">
int vtkAMRBaseReader::RequestInformation(... ...)
  ... ...
  this->Metadata = vtkOverlappingAMR::New();
  this->FillMetaData( ); //where file format specific subclasses would fill the meta data
  vtkInformation* info = outputVector->GetInformationObject(0);
  info->Set( vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA(), this->Metadata );
  ... ...
</source>
 
At the RequestUpdateExtent pass, vtkCompositeCutter iterates over the
bounding boxes of all data blocks, determines which ones are needed
and sends their indices upstream.
 
<source lang="cpp">
int vtkCompositeCutter::RequestUpdateExtent(... ...)
  ... ...
  std::vector<int> intersected;
  vtkCompositeDataSet * meta= vtkCompositeDataSet::SafeDownCast(inInfo->Get(vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA()));
  vtkSmartPointer<vtkCompositeDataIterator> iter;
  iter.TakeReference(meta->NewIterator());
  iter->SetSkipEmptyNodes(false);
  for(iter->InitTraversal(); !iter->IsDoneWithTraversal(); iter->GoToNextItem())
    {
    double* bb = iter->GetCurrentMetaData()->Get(vtkDataObject::BOUNDING_BOX());
    for (int c=0; c < this->ContourValues->GetNumberOfContours(); c++)
      {
      if(IntersectBox(this->GetCutFunction(),bb,this->ContourValues->GetValue(c)))
        {
        intersected.push_back(iter->GetCurrentFlatIndex());
        break;
        }
      }
    }
  inInfo->Set(vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES(), &intersected[0], static_cast<int>(intersected.size()));
... ...
</source>
 
Note that vtkCompositeCutter does not down cast the meta data "all the
way" to vtkOverlappingAMR but only to vtkCompositeDataSet.  This is
designed so that this filter would work for any composite data set
that provides bounding box information through its iterator.
 
Finally, in the RequestData pass, the amr source would load the
exactly the requested blocks using the key
vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES().
 
=== New AMR Data Structures ===
 
The previous implementation of vtkUniformGridAMR uses data
structures provided by the parent class (the old vtkCompositeDataSet).
The implementation is inefficient when there are hundreds of thousands
of data blocks, which is unfortunately common for AMR. In the new
implementation, we use more compact data structures: The following two
members of vtkUniformGridAMR respectively stores the meta data and the
actual data:
<source lang="cpp">
vtkAMRInformation* AMRInfo;
vtkAMRDataInternals* AMRData;
</source>
 
There are two children classes of vtkUniformGridAMR: vtkOverlappingAMR
and vtkNonOverlappingAMR. vtkOverlappingAMR is designed for AMR file
types that provide meta data, such as Enzo or Flash, so it
provides access to both AMRInfo and AMRdata; vtkNonOverlappingAMR is
designed for AMR file types that do not contain meta data, such as
Chmobo, so it provides access to only AMRData.

Latest revision as of 00:28, 9 October 2012

Primarily motivated by better support of AMR data structures, various APIs related to vtkCompositeDataSet have been changed.

The New vtkCompositeDataSet

In previous versions, vtkCompositeDataSet represents a tree structure that stores the actual data sets at its leaves. Most usage of it, on the other hand, treats it simply as a data set container that can be traversed. So we extracted the set traversal APIs to form a fully abstract class, again named vtkComposisteDataSet. The original vtkCompositeDataSet is renamed vtkDataObjectTree. The refactored hierarchy is shown below. Note that in this hierarchy, vtkUniformGridAMR switches to a new implementatation (See the third section for details).

As a consequence of the change, the class vtkCompositeDataIterator is refactored in a similary fashion: Those APIs that support simple traverals form the new vtkCompositeDataIterator; the original class is is renamed vtkDataObjectTreeIterator.

New composite.png

Demand Driven Composite Data Loading

One strategy to visualize large data sets is to make the data loading demand driven: A data source only produces the data blocks that downstream filters ask for. We now provide the support for composite data through two pipeline keys:

  • vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA(), created by a data

source to provide meta data for downstream filters.

  • vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES(), created by

downstream filters to request blocks from upstream data sources.

Example

We can look at how a vtkAMRBaseReader source works with a vtkCompositeCutter filter in a demand driven pipeline. At the request information pass, a vtkAMRBaseReader creates a meta data object that has the same type as the output but no actual data blocks:

<source lang="cpp"> int vtkAMRBaseReader::RequestInformation(... ...)

 ... ...
 this->Metadata = vtkOverlappingAMR::New();
 this->FillMetaData( ); //where file format specific subclasses would fill the meta data
 vtkInformation* info = outputVector->GetInformationObject(0);
 info->Set( vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA(), this->Metadata );
 ... ...

</source>

At the RequestUpdateExtent pass, vtkCompositeCutter iterates over the bounding boxes of all data blocks, determines which ones are needed and sends their indices upstream.

<source lang="cpp"> int vtkCompositeCutter::RequestUpdateExtent(... ...)

 ... ...
 std::vector<int> intersected;
 vtkCompositeDataSet * meta= vtkCompositeDataSet::SafeDownCast(inInfo->Get(vtkCompositeDataPipeline::COMPOSITE_DATA_META_DATA()));
 vtkSmartPointer<vtkCompositeDataIterator> iter;
 iter.TakeReference(meta->NewIterator());
 iter->SetSkipEmptyNodes(false);
 for(iter->InitTraversal(); !iter->IsDoneWithTraversal(); iter->GoToNextItem())
   {
   double* bb = iter->GetCurrentMetaData()->Get(vtkDataObject::BOUNDING_BOX());
   for (int c=0; c < this->ContourValues->GetNumberOfContours(); c++)
     {
     if(IntersectBox(this->GetCutFunction(),bb,this->ContourValues->GetValue(c)))
       {
       intersected.push_back(iter->GetCurrentFlatIndex());
       break;
       }
      }
   }
 inInfo->Set(vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES(), &intersected[0], static_cast<int>(intersected.size()));

... ... </source>

Note that vtkCompositeCutter does not down cast the meta data "all the way" to vtkOverlappingAMR but only to vtkCompositeDataSet. This is designed so that this filter would work for any composite data set that provides bounding box information through its iterator.

Finally, in the RequestData pass, the amr source would load the exactly the requested blocks using the key vtkCompositeDataPipeline::UPDATE_COMPOSITE_INDICES().

New AMR Data Structures

The previous implementation of vtkUniformGridAMR uses data structures provided by the parent class (the old vtkCompositeDataSet). The implementation is inefficient when there are hundreds of thousands of data blocks, which is unfortunately common for AMR. In the new implementation, we use more compact data structures: The following two members of vtkUniformGridAMR respectively stores the meta data and the actual data: <source lang="cpp"> vtkAMRInformation* AMRInfo; vtkAMRDataInternals* AMRData; </source>

There are two children classes of vtkUniformGridAMR: vtkOverlappingAMR and vtkNonOverlappingAMR. vtkOverlappingAMR is designed for AMR file types that provide meta data, such as Enzo or Flash, so it provides access to both AMRInfo and AMRdata; vtkNonOverlappingAMR is designed for AMR file types that do not contain meta data, such as Chmobo, so it provides access to only AMRData.