VTK/Wrapper Update 2012

The VTK wrappers received a major overhaul in 2010 prior to the release of VTK 5.8. Work on improving the wrappers has continued (though at a slower pace) and this document aims to describe the wrapper goals for VTK 6.0.

Overview

The "Wrapper Overhaul 2010" project (Source article) had four primary goals, three of which were completed:

1. Cleaning up the wrapper code (always more to do, though...)
2. Wrapping vtkStdString (and vtkUnicodeString)
3. Wrapping vtkVariant (and other non-vtkObject types)
4. Eliminating BTX/ETX from the VTK header files

Item 3 was eventually extended to cover templated classes like vtkVector, which is a good demonstration of the power of the VTK wrapping tools. Items 3 and 4 necessitated the adoption of the vtkWrapHierarchy tool, which is only now (Aug 2012) being adopted into VTK-based projects like ParaView. As a result, Item 4 was not completed... it is possible to build VTK with all the BTX/ETX markers removed, but this will cause external wrappers like those in ParaView to fail.

The new goals, which are targeted for the VTK 6.0 release, are:

1. More wrapper code cleanup (always more to do here)
2. Integrate wrapping with the new modular VTK cmake scripts (Kitware has mostly finished this)
3. Make the wrapping tools themselves into a VTK module
4. Finish the work on removing BTX/ETX

Detailed Descriptions

Code Cleanup and Enhancement

Nearly all the parsing code for the wrappers was contained in one huge file, vtkParse.y. In 2010 the code in this file was cleaned up, but the file itself was still huge. Now, the parser code has been reorganized into several smaller "vtkParse..." files such as:

• vtkParseMain.c - argument parsing and other useful things for wrapper tool executables
• vtkParseHierarchy.c - allow wrapper tools to introspect the class hierarchy (created in 2010)
• vtkParseData.c - data structures for describing C++ classes, constants, functions, templates, and types
• vtkParseString.c - an efficient replacement for the dreaded "strdup()" that allows memory reclamation
• vtkParseMangle.c - provide symbol mangling utilities
• vtkParseExtras.c - a catch-all for utilities that don't fit elsewhere (created in 2010)

The tokenizer file, vtkParse.l, was much smaller than vtkParse.y but it has also been reorganized and supplemented:

• vtkParsePreprocess.c - preprocess C++ files, keep a table of macro definitions

The fact that the wrapper tools can expand macros and follow preprocessor directives makes them much more robust, though in order for the full power of the preprocessor to be realized the wrapper tools must be supplied with a list of include directories given on the command line as "-I" arguments.

Along with the code cleanup, the grammar rules in vtkParse.y have been rewritten. This started with trial-and-error modifications to enable the parser to wrap header files that contained ITK class declarations, and culminated in the renaming and rewriting the rules to approximate the grammar for the 1998 ISO C++ standard. With these changes, it is hoped that only truly pathological code will be able to break the parser without also breaking at least one of the supported VTK compilers.

Making a WrapperTools Module

In order to build vtkWrapCientServer, the ParaView cmake files must grab certain wrapper source files from the VTK source code and compile them. This means that any cleanup or reorganization of the VTK wrapper tool code necessitates manual adjustments to the build scripts for vtkWrapClientServer. If the VTK wrapper tools existed as a library module, then it would be much easier to create and maintain external wrapper tools like vtkWrapClientServer.

Remove BTX/ETX

Completing the removal of BTX/ETX from the VTK header files requires that external projects use vtkWrapHierarchy and that they pass "-I" include directory arguments to the wrapper tools so that all macro definitions can be found and all preprocessor directives can be followed. If external projects use the VTK module macros to build their own libraries and executables, then this will occur automatically.

Wish List

These items are unlikely to be completed for VTK 6.0, but sure would be nice to have eventually:

Eliminate Hierarchy Requirement

Now that the wrapper tools are provided with "-I" arguments so that they can search for included files, the vtkWrapHierarchy tool is not strictly necessary. However, further work has to be done before vtkWrapHierarchy (and the headaches that come with it) can be eliminated completely. This "further work" involves writing code for the parser that performs context-sensitive name lookups, which can be very complicated e.g. for templated code.

Support Doxygen-Style Comments

The Tcl and Python wrappers include full copies of the VTK header-file documentation. Many users rely on this documentation extensively. Currently, the wrapper tools require all documentation to be in the "//Description" format used by VTK. It would be very nice to also support doxygen-style comments in the header files, rather than to continue to require a format that is used only by VTK.

Python 3 Support

Python 3 introduced major API changes for strings and ints, and minor API changes elsewhere.

• It will be possible to support Python 2.3 though Python 3
• "Classic" classes are gone, PyVTKObject and PyVTKClass will need to be replaced
• PyIntObject and PyStringObject are absent in Python 3, but are used widely in VTK/Python
• Python 3 uses unicode for all strings (ramifications for VTK?)
• Language changes: some examples will have to be rewritten to work with python 3