VTK/MultiPass Rendering

WORK IN PROGRESS

Overview

Modifying the logic of the rendering pipeline of VTK is cumbersome: it requires to modify several existing classes, and often requires a full understanding of most of the rendering classes. Modifying the rendering logic is also really sensitive and error-prone. More importantly, it also means that it is impossible from an external project to modify the rendering pipeline.

The multi-pass rendering framework try to address these issues by providing a hook in the vtkRenderer class where a developer can plug her own rendering algorithms (passes) and combine them with other algorithms at compile-time (statically) or a run-time (dynamically).

The approach is, even if the VTK source code is freely available (published under a (non-copyleft) free software license), to think of it as a closed-source package and see how it could be extended by relying only on its public API.

Note there is another way to attach a custom rendering algorithm but it doesn't not provide a framework: a developer can subclass vtkRendererDelegate and call vtkRenderer::SetDelegate() .

Fundamental classes

The following classes are the building blocks of the multi-pass rendering framework:

• vtkRenderPass
• vtkRenderState
• vtkSequencePass (allow run-time combinations of passes)

vtkRenderPass

The gut of vtkRenderPass is to perform some rendering based on some render state. vtkRenderPass is a deferred (abstract) class. The central method is

 virtual void Render(const vtkRenderState *s)=0

Any effective (concrete) subclass will implement this method.

vtkRenderState

vtkRenderState aggregates information necessary for the vtkRenderPass to perform rendering. The state consists of:

• a vtkRenderer (which gives access to the vtkCamera, to the vtkRenderWindow and to the vtkProps)
• a list of props to render (it is a subset of the props on the renderer, usually the visible props that remain after the frustum culling operation) A RenderPass may ignore this cached list because it is not relevant for it and may work again directly on all the props from the Renderer. The typical example is a shadow mapping pass, that will change the active camera to be one of the light, so culling from the original camera does not make sense and will produce wrong result.
• the current framebuffer in use, as some passes can render in vtkFrameBufferObject, not only on the framebuffer provided by the vtkRenderWindow.
• a list of required property keys that have to be present on the props. this is a flexible way to filter props to render. Keys on the props allow to add customized properties. For example, the vtkShadowMapPass defined two types of keys OCCLUDER and RECEIVER to identify props that will cast the shadows and the props that will receive the shadows (see #Property_keys).

Putting things together

Attach a vtkRenderPass to a vtkRenderer

To attach the main render pass to a vtkRenderer, use void vtkRenderer::SetPass(vtkRenderPass *p). At run-time, if a pass is set on the renderer, the standard rendering code is bypassed in the render pass is used instead.

Only one pass is actually attached. If there are several passes, the combination of them is up to the pass attached to the renderer. The following section explains how to combine render passes.

Combining passes

There are two ways to combine passes:

1. by composition, at run-time with vtkSequencePass
2. by derivation, at compile-time by writing a subclass of vtkRenderPass connected with some delegate passes

Of course, both ways can be used together.

vtkSequencePass

vtkSequencePass is a concrete render pass which executes a list of render passes sequentially. The list of passes is contained in a vtkRenderPassCollection.

Example:

#include "vtkMyRenderPass1.h"
#include "vtkMyRenderPass2.h"
#include "vtkSequencePass.h"
#include "vtkRenderPassCollection.h"

vtkMyRenderPass1 *p1=vtkMyRenderPass1::New();
vtkMyRenderPass2 *p2=vtkMyRenderPass2::New();
vtkSequencePass *s=vtkSequencePass::New();
vtkRenderPassCollection *passes=vtkRenderPassCollection::New();

passes->AddItem(p1);
passes->AddItem(p2);
s->SetPasses(passes);

// a call to s->Render() will execute p1->Render() followed by p2->Render

As vtkSequencePass is a vtkRenderPass itself, it is possible to have a hierarchy of render passes built at runtime (Composite design pattern).

Example:

// (the creation of the objects is not reported for clarity)
vtkMyRenderPass1 *p1;
vtkMyRenderPass2 *p2;
vtkMyRenderPass2 *p3;
vtkMyRenderPass2 *p4;
vtkSequencePass *s1;
vtkRenderPassCollection *passes1;
vtkSequencePass *s2;
vtkRenderPassCollection *passes2;

passes1->AddItem(p1);
passes1->AddItem(p2);
s1->SetPasses(passes1);

passes2->AddItem(p3);
passes2->AddItem(s1);
passes2->AddItem(p4);
s2->SetPasses(passes2);

// a call to s2->Render() will execute p3->Render() followed by s1->Render(), followed by p4->Render(). The overall sequence will be p3, p1, p2, p4. 

Subclassing

A render pass may delegate some of its work to other passes (not only one). There is no commitment about when the delegates will be called and how many times.

Particularly, the render pass and its delegates don't have to form a sequence of passes. It is clear in the case of a depth peeling pass that the depth peeling pass and its delegate don't form a sequence, as its delegate is called multiple times (it is called each time a peel is rendered).

Property keys

vtkProp is now equipped with a set of property keys ( Get/SetPropertyKeys() ).

Before the introduction of this mechanism, a vtkProp had a fixed number of properties: Visibility, Pickable, Dragable. These properties actually work in conjunction with the VTK rendering pipeline. For example, visibility is used to compute the bounding box and to reset the camera in the right position, or to cull the geometry out.

Thinking again of the case of a render pass written in an external project, with the assumption that the VTK code cannot be changed, it makes sense to allow the designer of a render pass to introduce new type of properties specific to a render pass. This is the real purpose of property keys: a flexible mechanism to add custom properties at run-time.

Even if vtkShadowMapPass is part of VTK, it could have been written out of it with no change to VTK. vtkShadowMapPass defined two types of keys OCCLUDER and RECEIVER to identify props that will cast the shadows and the props that will receive the shadows.

A vtkRenderPass can query if a vtkProp has a given set of property keys with vtkProp::HasKeys(). It can also ask a vtkProp to render part of its geometry only if a set of property keys exist on it. For example vtkProp::RenderFilteredOpaqueGeometry(vtkViewport *v,vtkInformation *requiredKeys) will render the opaque geometry only if the required keys are all in the property keys of the prop.

Keep in mind that a vtkProp can be a composite prop, like vtkAssembly.

Elementary passes

Several effective classes have been implemented to break down the standard VTK rendering into elementary steps:

• vtkCameraPass
• vtkClearZPass
• vtkLightsPass
• vtkDefaultPass
  • vtkOpaquePass
  • vtkTranslucentPass
  • vtkVolumetricPass
  • vtkOverlayPass

Existing passes

The following classes are provided with VTK. The key is they could have been written out of VTK. It demonstrates that you can customize and extent the rendering VTK pipeline in an external project without modifying VTK:

• vtkDepthPeelingPass
• vtkShadowMapPass
• post-processing image passes (vtkImageProcessingPass):
  • vtkGaussianBlurPass,
  • vtkSobelGradientMagnitudePass

Parallel rendering

• vtkCompositeZPass
• vtkCompositeRGBAPass

• Examples of passes
• 

  vtkDepthPeelingPass

• 

  vtkShadowMapPass

• 

  vtkGaussianBlurPass

• 

  vtkSobelGradientMagnitudePass

• 

  combination of post-processing image passes