Monthly Archives: May 2012

Why Change Management is Hard

Posted on by

This post is about software change management, the problem and procedures involved with maintaining progress and quality of a team-developed software solution over time.

As Alex puts it in his unusual DailyWTF post this is a discipline that is neither part of software development nor software engineering. And more so, the inherent problems are hardly recognized by most developers nor architects. For most IT people this is located somewhere in the Q-Space, something somebody has to take care of and usually only gets on your nerves. I am no exception to that. It took me years to realize that this is the spine of the software development process. Initially almost any approach will do, but when team and solutions grow, a weak spine will bend and finally break.

When asked about software change management most developers will think of SCM systems like Suversion, Git, or Perfore – the tools that we are faced with when generating change in the first place.

The frustrating part of change management starts when selectively moving changes from one “solution version”, say a branch, to another: Most everybody will have had these super frustrating moments when merging changes has turned into an endless, painful double maintenance experience.

Here is a simplified view of places that receive changes (typically a branch in some SCM, at the far right end in many cases a runtime):

The problem does not lie in re-merging of feature branches – illustrated on the left side of the flow diagram. Those flows are rather well addressed by standard SCM workflows (more true for Git than Subversion though).

Neither does the final push of changes to the execution environment pose a problem – typically a straightforward forward-merge.

The merge approach breaks down when managing the conflicting interest of feature integration and selective feature release into pre-production stabilization. I.e. when crossing the “divide” in the diagram:

  • Without seeing features work in integration, all talk about feature readiness is moot. This is particularly true for solution/application development. There is a need to have changes enter into integration as soon as possible. On the other hand:
  • You do not want to stall development. There is no pause. In order to qualify for production a subset of all change history must be isolated from receiving further changes to be tested (and usually fixed) to qualify for production.

The reason the second issue is hard lies in the nature of solution development. Technologists suggest to approach integration problems by reducing the size of components and attempts to define slowly changing component interfaces, so that you can version and release on disjoint parts of the code base. In application development however complexity has a tendency to spread out: End-user requirements typically do not map well to module boundaries so that there is constant overlap between completed developments and half-done features.

The selective push of features from integration to stabilization has a number of technical problems and some more interesting process problems. The decision to release a set of changes that make a feature should be with the ones being ultimately responsible to also fix problems later on: The development team, or preferably the individual developer who introduced the changes. Technically changes made by different developers may interfere with each other so that management of a change release queue is necessary.

There is other approaches out there. For example the Facebook style “release everything from trunk every week” (and hence avoiding selective integration at all, see here). But that applies “only” to the Facebook frontend where features can even be turned on and off selectively, and where the backend takes the burden of maintaining consistency (plus it’s functionally a rather simple application anyway). I doubt that that would work well for Facebook backend development.

So much for the problem description.

Coincidentally I know from my past experience that this is one of the problems that traditional SAP software logistics solves quite nicely. Unfortunately I was not able to identify a publicly available tool that addresses these problems.

How are we doing this today? We muddle through, try to identify complete snapshots, stall development…. it sucks.

It would be extremely cool to have a tool that

  1. allows developers to group changes into “feature change sets” and release them for integration into another code line
  2. manages a change set queue and implements workflows to resolve dependencies on the queue of change sets

Know something? Please let me know as well.

References