ITK/Backward Compatibility Open Discussion

The topic of backward compatibility has generated animated discussions among users and developers.

We gather here multiple points of view from some of the influential users and developers.

The content of this page is not intended to be conclusive on any of the proposed topics. It simple intends to gather elements of discussion that will serve as food-for-thought.

Main Discussion Topics

Discussion Topics

1. Deprecation of classes / method. When to deprecate, and how to deprecate.
2. Drawing the line where fixing a bug will result in backward compatibility breaks.
3. Cathedral/Bazar: How much burden to put on new contributions to the toolkit.
   1. Is the Insight Journal + Code Review directory a process that is too burdensome.
4. When can we require additional effort from users to update their code for using new versions of ITK ?
   1. Never
   2. Once a year
   3. Once every five years
5. How far back in time should we maintain backward compatibility (relates to the previous topic)
   1. Today's version should be compatible with ITK version from N years ago
      1. 1 year ?
      2. 2 years ?
      3. 5 years ?

Some general issues:

• Who are the customers for ITK?
• What do they want from ITK?
• How much of what the customers want is the development community actually able to deliver?
• Once there's agreement on what ITK is actually committing to deliver, the backward compatibility policy can be implemented to clearly communicate how various components of the code are evolving.

Position Statement 2 (Bill Lorensen)

These comments are addressing source code backward compatibility.

One of the major criticisms of open-source software is that new revisions are not compatible with old revisions. Breaking compatibility impedes the acceptance and utility of open-source software. On the other hand, strict backward compatibility polices can impede innovation in software. The tension between these two viewpoints is not easily resolved.

As projects mature and the customer base grows, backward compatibility becomes more important. Commercial hardware and software products call this customer base, the installed base. Commercial products usually have a known customer base consisting of those who have purchased or licensed the software. Also, commercial systems seldom expose internal API's. Open source projects rarely know the identities of their customers. And, since the source is open, customers have access to all public and protected classes, methods and data in the code. For open source software, it is almost impossible to determine how the customer base is using the software.

When a project hits a certain point in its life cycle, the unpleasant issue of backward compatibility begins to rear its ugly head. All of a sudden the changes introduced in a new release of the software have a dark side to them; they hold hidden possibilities that will break something one of your users depends on. This is true in both open and closed source projects, but in the open source world it seems that the community has spent less time worrying about it than in the closed source world. From “Preserving Backward Compatibility, http://www.onlamp.com/lpt/a/5626, Garrett Rooney.

The Dark Side of Extreme Programming: The nightly test/build was so effective in empowering programmers to make changes, that API changes occurred too frequently without the necessary buy-in from the user community. From “Insight Insight”, http://www.itk.org/CourseWare/Training/InsideInsight.pdf, Bill Lorensen

Some argue that open source software should be used at your own risk. But even using open source software requires an investment in time, energy and funds. Also the reputation of the development community is at risk.

...consider your user base. If you have only a dozen highly technical users, jumping through hoops to maintain backward compatibility may be more trouble than it's worth. On the other hand, if you have hundreds or thousands of nontechnical users who cannot deal with manual upgrade steps, you need to spend a lot of time worrying about those kinds of issues. Otherwise, the first time you break compatibility you'll easily burn through all the goodwill you built up with your users by providing them with a useful program. It's remarkable how easily people forget the good things from a program as soon as they encounter the first real problem. From “Preserving Backward Compatibility, http://www.onlamp.com/lpt/a/5626, Garrett Rooney.

These investments are made by customers that include developers, users and sponsors.

Position Statement 3 (Steve Pieper)

This is primarily a policy discussion, and so the central issue is how to effectively communicate with users of the toolkit about what they get when they use a particular piece of code. The general policies could be summarized as:

• Different things should have different names.

• Similar things should have similar names.

• If two things have the same name, you can assume they will behave the same.

A way to interpret this is that if you come up with non-backwards compatible version of an algorithm, you should give it a new class name, like MyFilter2, rather than relying on the toolkit version number to indicate that it is different. Deprecation warnings at compile time can inform the developer that MyFilter is out of date. Dropping support for deprecated classes should happen when the toolkit itself gets a new name (like ITK4 instead of ITK3). Developers can choose when to migrate to a new class and/or a new version of the toolkit.

Position Statement 4 (Stephen Aylward)

Backward compatibility is paramount in a toolkit, particularly one that is used by researchers. Being used by researchers means that the toolkit is being used by people who are accustom to creating on their own and who are not patient with outside impediments to their work. If a toolkit continually requires them to re-develop/re-test code that had previously worked, the toolkit will become viewed as an impediment to their work. The researcher will rightly place a high cost on the time spent re-developing, re-testing and the perceived risk that their research is subject to the whim of others. Eventually, that cost will outweigh the benefits, and the user-pool will dwindle.

Backward compatibility applies to the API and the operation of a toolkit. One person's bug is another person's feature. An incorrectly spelled function name is not a bug once that function has been called by a user. Subsequently changing the function's name to the correct spelling does create a bug in the user's code. The same bug/feature dichotomy exists when the API of a set of functions is perceived to be inconsistent. The same bug/feature dichotomy may even exist when a function has side-effects that are perceived as unwanted or even when a function has outputs that are perceived as incorrect. The guiding philosophy should be: Once a function is released and it performs a particular operation, even if the operation it performs is not what was originally intended, its operation cannot be considered a bug. If you want to perform a different operation, you should create a new function, and perhaps begin to deprecate the other function. As it is a general philosophy, there will be times when it does not apply, however, in a strict environment, the sole exception may be when the function name or the function's design specification unambiguously defines the intended operation and yet the function does not provide that operation. Nevertheless, the reality of most development environments is that additional exceptions to the rule are needed. The cost to the user must still be considered.

Current developers and users must work to promote the next generation of developers and users. Telling a user that they should not upgrade is an expression of the judgment that the future benefit of the toolkit does not justify the cost of the upgrade. It is saying to the developers that their continued efforts are not likely to be useful to the user. It is saying to the user that they are not welcome as active members in the community. Freezing a toolkit at a particular version is a legitimate project management decision for a user. It should, however, not be the mantra of the toolkit's developers.

Admittedly, radical changes are occasionally needed in a toolkit to keep it current. When making those changes; however, it is important to apply an otherwise contrary adage: "if you are going to break a standard, then you should REALLY break the standard." That is, the changes to the toolkit should be planned and drastic. Planned means that the changes (1) should be announced and discussed well in-advance of their release; (2) should be well supported, with clear transition paths and documentation; and (3) should be driven by the needs of the community. Drastic means that the changes should be extensive. If the changes being introduced have a subtle appearance then it is likely that they could instead be done in a backward compatible way or as an extension to the existing framework. Breaking backward compatibility should only be allowed if the collective voice of the user community calls for a change that necessitates the complete overhaul of a framework or function to support the trends in research, hardware, or compilers.

Balancing the above issues is best handled by a systematic process for adopting new features, testing backward compatibility, and implementing alternative frameworks. The Insight Toolkit has a well established method for adopting new features, the Insight Journal. The Insight Journal has evolved to be and continues to improve as an effective method for receiving and reviewing candidate methods for inclusion in the Insight Toolkit. Testing backward compatibility is enabled by CTest and coverage counts, but disciplined application of those technologies is and will probably always be a challenge. An established mechanism for implementing, reviewing, and releasing alternative frameworks does not currently exist.

Establishing a mechanism (policy and software) for implementing, reviewing, and releasing alternative frameworks is critical to the continued success of the Insight Toolkit. If such a mechanism could be provided, then daily backward compatibility challenges would have a controlled outlet.

Background Links

• http://www.joelonsoftware.com/items/2008/03/17.html
• http://buytaert.net/backward-compatibility