VTK/MultiPass Rendering

Overview

Modifying the logic of the rendering pipeline of VTK is cumbersome: it requires to modify several existing classes, and often requires a full understanding of most of the rendering classes. Modifying the rendering logic is also really sensitive and error-prone. More importantly, it also means that it is impossible from an external project to modify the rendering pipeline.

The multi-pass rendering framework tries to address these issues by providing a hook in the vtkRenderer class where a developer can plug her own rendering algorithms (passes) and combine them with other algorithms at compile-time (statically) or a run-time (dynamically).

The approach is, even if the VTK source code is freely available (published under a (non-copyleft) free software license), to think of it as a closed-source package and see how it could be extended by relying only on its public API.

Note there is another way to attach a custom rendering algorithm but it doesn't not provide a framework: a developer can subclass vtkRendererDelegate and call vtkRenderer::SetDelegate() .

• Examples of passes
• 

  vtkDepthPeelingPass

• 

  vtkShadowMapPass

• 

  vtkGaussianBlurPass

• 

  vtkSobelGradientMagnitudePass

• 

  combination of post-processing image passes

Date of Implementation

• The mechanism is ready to use since 2009/06/19.
• Refactoring of the shadow mapping has been completed on 2010/07/13.
• Implemented by (any question to) François Bertel.

Fundamental classes

The following classes are the building blocks of the multi-pass rendering framework:

• vtkRenderPass
• vtkRenderState
• vtkSequencePass (allow run-time combinations of passes)

vtkRenderPass

The gut of vtkRenderPass is to perform some rendering based on some render state. vtkRenderPass is a deferred (abstract) class. The central method is

 virtual void Render(const vtkRenderState *s)=0

Any effective (concrete) subclass will implement this method.

vtkRenderState

vtkRenderState aggregates information necessary for the vtkRenderPass to perform rendering. The state consists of:

• a vtkRenderer (which gives access to the vtkCamera, to the vtkRenderWindow and to the vtkProps)
• a list of props to render (it is a subset of the props on the renderer, usually the visible props that remain after the frustum culling operation) A RenderPass may ignore this cached list because it is not relevant for it and may work again directly on all the props from the Renderer. The typical example is a shadow mapping pass, that will change the active camera to be one of the light, so culling from the original camera does not make sense and will produce wrong result.
• the current framebuffer in use, as some passes can render in vtkFrameBufferObject, not only on the framebuffer provided by the vtkRenderWindow.
• a list of required property keys that have to be present on the props. this is a flexible way to filter props to render. Keys on the props allow to add customized properties. For example, the vtkShadowMapPass defines two types of keys OCCLUDER and RECEIVER to identify props that will cast the shadows and the props that will receive the shadows (see #Property_keys).

Putting things together

Attach a vtkRenderPass to a vtkRenderer

To attach the main render pass to a vtkRenderer, use void vtkRenderer::SetPass(vtkRenderPass *p). At run-time, if a pass is set on the renderer, the standard rendering code is bypassed in the render pass is used instead.

Only one pass is actually attached. If there are several passes, the combination of them is up to the pass attached to the renderer. The following section explains how to combine render passes.

Combining passes

There are two ways to combine passes:

1. by composition, at run-time with vtkSequencePass
2. by derivation, at compile-time by writing a subclass of vtkRenderPass connected with some delegate passes

Of course, both ways can be used together.

vtkSequencePass

vtkSequencePass is a effective render pass which executes a list of render passes sequentially. The list of passes is contained in a vtkRenderPassCollection.

Example:

#include "vtkMyRenderPass1.h"
#include "vtkMyRenderPass2.h"
#include "vtkSequencePass.h"
#include "vtkRenderPassCollection.h"

vtkMyRenderPass1 *p1=vtkMyRenderPass1::New();
vtkMyRenderPass2 *p2=vtkMyRenderPass2::New();
vtkSequencePass *s=vtkSequencePass::New();
vtkRenderPassCollection *passes=vtkRenderPassCollection::New();

passes->AddItem(p1);
passes->AddItem(p2);
s->SetPasses(passes);

// a call to s->Render() will execute p1->Render() followed by p2->Render

As vtkSequencePass is a vtkRenderPass itself, it is possible to have a hierarchy of render passes built at runtime (Composite design pattern).

Example:

// (the creation of the objects is not reported for clarity)
vtkMyRenderPass1 *p1;
vtkMyRenderPass2 *p2;
vtkMyRenderPass2 *p3;
vtkMyRenderPass2 *p4;
vtkSequencePass *s1;
vtkRenderPassCollection *passes1;
vtkSequencePass *s2;
vtkRenderPassCollection *passes2;

passes1->AddItem(p1);
passes1->AddItem(p2);
s1->SetPasses(passes1);

passes2->AddItem(p3);
passes2->AddItem(s1);
passes2->AddItem(p4);
s2->SetPasses(passes2);

// a call to s2->Render() will execute p3->Render() followed by s1->Render(), followed by p4->Render(). The overall sequence will be p3, p1, p2, p4. 

Subclassing

A render pass may delegate some of its work to other passes (not only one). There is no commitment about when the delegates will be called and how many times.

Particularly, the render pass and its delegates don't have to form a sequence of passes. It is clear in the case of a depth peeling pass that the depth peeling pass and its delegate don't form a sequence, as its delegate is called multiple times (it is called each time a peel is rendered).

Property keys

vtkProp is now equipped with a set of property keys ( Get/SetPropertyKeys() ).

Before the introduction of this mechanism, a vtkProp had a fixed number of properties: Visibility, Pickable, Dragable. These properties actually work in conjunction with the VTK rendering pipeline. For example, visibility is used to compute the bounding box and to reset the camera in the right position, or to cull the geometry out.

Thinking again of the case of a render pass written in an external project, with the assumption that the VTK code cannot be changed, it makes sense to allow the designer of a render pass to introduce new type of properties specific to a render pass. This is the real purpose of property keys: a flexible mechanism to add custom properties at run-time.

For example, even if vtkShadowMapPass is part of VTK, it could have been written out of it with no change to VTK. vtkShadowMapPass defines two types of keys OCCLUDER and RECEIVER to identify props that will cast the shadows and the props that will receive the shadows.

A vtkRenderPass can query if a vtkProp has a given set of property keys with vtkProp::HasKeys(). It can also ask a vtkProp to render part of its geometry only if a set of property keys exist on it. For example vtkProp::RenderFilteredOpaqueGeometry(vtkViewport *v,vtkInformation *requiredKeys) will render the opaque geometry only if the required keys are all in the property keys of the prop.

Keep in mind that a vtkProp can be a composite prop, like vtkAssembly.

Elementary passes

Several effective classes have been implemented to break down the standard VTK rendering into elementary steps:

• vtkCameraPass
• vtkClearZPass
• vtkLightsPass
• vtkDefaultPass
  • vtkOpaquePass
  • vtkTranslucentPass
  • vtkVolumetricPass
  • vtkOverlayPass

vtkDefaultPass has convenient protected methods to iterate over vtkProps and render part of their geometry with or without a set of required key (for example, RenderOpaqueGeometry(const vtkRenderState *s) and RenderFilteredOpaqueGeometry (const vtkRenderState *s)). Its Render() method also defines a fixed sequence of passes: it renders the opaque geometry, the translucent geometry, the volumetric geometry and the overlay.

Existing passes

The following classes are provided with VTK. The key is they could have been written out of VTK. It demonstrates that the rendering VTK pipeline can be customized and extended in an external project without modifying VTK:

• vtkDepthPeelingPass (deal with translucent geometry, more details on depth peeling here)
• vtkShadowMapBakerPass/vtkShadowMapPass (add shadows with the shadow map technique)
• post-processing image passes (vtkImageProcessingPass):
  • vtkGaussianBlurPass (blur the rendered image),
  • vtkSobelGradientMagnitudePass (screen-space edge detection)

Parallel rendering

The following render passes allow support for parallel rendering:

• vtkCompositeZPass (perform compositing of depth buffers in a parallel environment)
• vtkCompositeRGBAPass (perform back-to-front RGBA image compositing in a parallel environment, used in conjunction with a kd-tree, for example a kd-tree coming from the vtkDistributedDataFilter "D3" )
• vtkClientServerCompositePass (to handle client-server image delivery, to be used in two-processes configurations)

Examples

Simulate standard VTK rendering with render passes

// The elementary passes.
vtkLightsPass *lights=vtkLightsPass::New();
vtkDefaultPass *default=vtkDefaultPass::New();

// Put them in a sequence.
vtkRenderPassCollection *passes=vtkRenderPassCollection::New();
  passes->AddItem(lights);
  passes->AddItem(default);

vtkSequencePass *seq=vtkSequencePass::New();
  seq->SetPasses(passes);

// Make the sequence the delegate of a camera pass.
vtkCameraPass *cameraP=vtkCameraPass::New();
  cameraP->SetDelegatePass(seq);
  
// The main pass is camera pass, attach it to the renderer

  vtkRenderer *renderer = vtkRenderer::New();
  renderer->SetPass(cameraP);

Post-processing pass

Starting from the previous example #Simulate_standard_VTK_rendering_with_render_passes, the pass attached to the renderer will be a blur pass. The blur pass will delegate the scene rendering to the camera pass (and perform blur on the resulting image)

  
// Create the post-processing pass and attach the camera pass as its delegate

  vtkGaussianBlurPass *blurP=vtkGaussianBlurPass::New();
  blurP->SetDelegatePass(cameraP);

// The main pass is the blur pass, attach it to the renderer

  vtkRenderer *renderer = vtkRenderer::New();
  renderer->SetPass(blurP);

Shadow mapping

In order to render well with IceT ( see multi-pass rendering framework with IceT ), the unique shadow pass (vtkShadowMapPass) has been broken down into two passes after VTK 5.6:

1. vtkShadowMapBakerPass, which builds the shadow maps
2. vtkShadowMapPass, which renders the scene by applying the shadow maps on the geometry.

VTK 5.6

In this example, property keys will be used to tag vtkProps as receiver and/or occluder of shadow maps.

A shadow map pass only deal with opaque geometry. Starting from the example #Simulate_standard_VTK_rendering_with_render_passes, we cannot use vtkDefaultPass as we have to break down the default sequence.

We first replace

vtkDefaultPass *default=vtkDefaultPass::New();

by

vtkOpaquePass *opaque=vtkOpaquePass::New();
vtkTranslucentPass *translucent=vtkTranslucentPass::New();
vtkVolumetricPass *volume=vtkVolumetricPass::New();
vtkOverlayPass *overlay=vtkOverlayPass::New();

We attach an opaque pass combination as the delegate of the shadow map pass:

vtkSequencePass *opaqueSequence=vtkSequencePass::New();
vtkRenderPassCollection *passes2=vtkRenderPassCollection::New();
passes2->AddItem(lights);
passes2->AddItem(opaque);
opaqueSequence->SetPasses(passes2);
vtkCameraPass *opaqueCameraPass=vtkCameraPass::New();
opaqueCameraPass->SetDelegatePass(opaqueSequence);
vtkShadowMapPass *shadows=vtkShadowMapPass::New();
shadows->SetOpaquePass(opaqueCameraPass);

We set parameters for the shadow map pass:

shadows->SetResolution(1024); // the shadow map will be 1024*1024

// To cancel self-shadowing. 
shadows->SetPolygonOffsetFactor(3.1f);
shadows->SetPolygonOffsetUnits(10.0f);

We add the shadow map to a sequence pass,

  vtkSequencePass *seq=vtkSequencePass::New();
  vtkRenderPassCollection *passes=vtkRenderPassCollection::New();
  passes->AddItem(shadows); // the opaque pass with shadow mapping
  passes->AddItem(volume);
  passes->AddItem(overlay);
  seq->SetPasses(passes);

The sequence becomes the delegates another camera pass.

  cameraP->SetDelegatePass(seq);

This camera pass becomes the attached pass to the renderer:

renderer->SetPass(cameraP);

All the passes are now initialized. We still need to attach property keys to the props of the 3D scene. To make a prop an occluder, we call

 vtkProp *p;
  vtkInformation *properties=vtkInformation::New();
  properties->Set(vtkShadowMapPass::OCCLUDER(),0); // 0 is a dummy value, some other keys in other passes can have relevant values
  p->SetPropertyKeys(properties);

To make a prop an receiver, we call

 vtkProp *p;
  vtkInformation *properties=vtkInformation::New();
  properties->Set(vtkShadowMapPass::RECEIVER(),0); // 0 is a dummy value
  p->SetPropertyKeys(properties);

To make a prop an occluder and a receiver, we call

 vtkProp *p;
  vtkInformation *properties=vtkInformation::New();
  properties->Set(vtkShadowMapPass::OCCLUDER(),0);
  properties->Set(vtkShadowMapPass::RECEIVER(),0);
  p->SetPropertyKeys(properties);

VTK > 5.6

With the 2 passes approach, we first bake the shadow maps using the opaque geometry:

vtkSequencePass *opaqueSequence=vtkSequencePass::New();
  
vtkRenderPassCollection *passes2=vtkRenderPassCollection::New();
passes2->AddItem(lights);
passes2->AddItem(opaque);
opaqueSequence->SetPasses(passes2);

vtkCameraPass *opaqueCameraPass=vtkCameraPass::New();
opaqueCameraPass->SetDelegatePass(opaqueSequence);

vtkShadowMapBakerPass *shadowsBaker=vtkShadowMapBakerPass::New();
shadowsBaker->SetOpaquePass(opaqueCameraPass);

We also have to pass the parameters to avoid self-shadowing as well as the resolution of the square shadow maps:

shadowsBaker->SetResolution(1024);
// To cancel self-shadowing.
shadowsBaker->SetPolygonOffsetFactor(3.1f);
shadowsBaker->SetPolygonOffsetUnits(10.0f);

We then connect the two shadow map passes together:

vtkShadowMapPass *shadows=vtkShadowMapPass::New();
shadows->SetShadowMapBakerPass(shadowsBaker);

The second shadow map pass needs also to have access to an opaque pass:

shadows->SetOpaquePass(opaqueSequence);

Finally, we can build the main sequence, starting by baking the shadow maps, followed by rendering the opaque geometry with the shadows

vtkSequencePass *seq=vtkSequencePass::New();
vtkRenderPassCollection *passes=vtkRenderPassCollection::New();
seq->SetPasses(passes);
passes->AddItem(shadowsBaker);
passes->AddItem(shadows);

If we want, we can add passes for translucent polygonal geometry, volumes and overlay:

passes->AddItem(lights);
passes->AddItem(peeling);
passes->AddItem(volume);
passes->AddItem(overlay);

The last step is to connect the camera pass to this sequence and make it the main delegate for the renderer:

cameraP->SetDelegatePass(seq);
renderer->SetPass(cameraP);

Other resources

You can also take a look at the VTK regression tests:

VTK/Rendering/Testing/Cxx:

• TestBlurAndSobelPasses.cxx
• TestGaussianBlurPass.cxx
• TestGenericVertexAttributesGLSLDepthPeelingPass.cxx
• TestShadowMapPass.cxx
• TestSobelGradientMagnitudePass.cxx
• TestTranslucentLUTDepthPeelingPass.cxx

VTK/Parallel/Testing/Cxx:

• DistributedDataRenderPass.cxx
• TestDistributedDataCompositeZPass.cxx
• TestDistributedDataShadowMapPass.cxx
• TestPCompositeZPass.cxx
• TestPShadowMapPass.cxx
• TransmitImageDataRenderPass.cxx