Here is yet another piece on my favorite subject: Keeping big and growing systems manageable. Or, conversely, why that is so hard and failing so often?
Why do large projects fail and why is productivity diminishing in large projects?
Admittedly, that is a big question. But here is some piece on humans in that picture.
Modularization – once more
Let’s concentrate on modularization as THE tool to scale software development successfully and the lack that drives projects into death march mode and eventually into failure.
In this write-up, modularization is all the organization of structure of software above the coding level and below the actual process design. All the organization of artefacts to obtain building blocks for the assembly of solutions that implement processes as desired. In many ways the conceptual or even practical interface between specified processes to implement and their actual implementation. So in short: This is not about any specific modularization approach into modules, packages, bundles, namespaces or whatever.
I hereby boldly declare that modularization is about
Isolation of declarations and artifacts from the visibility and harmful impact on other declarations, artifacts, and resource;
Sharing of declarations, artifacts, resources with other declarations, artifacts, and resources in a controlled way;
Leading the extensibility and further development by describing structure and interplay declarations, artifacts, and resources in an instructive way.
Depending on the specific toolset, using these mechanism we craft APIs and implementations and assemble systems from modular building blocks.
If this was only some well-defined engineering craft, we would be done. Obviously this is not the case as so many projects end up as some messy black hole that nobody wants to get near.
The Problem is looking back at you from the Mirror
The task of getting a complex software system into shape is a task that is performed by a group of people and is hence subject to all human flaws and failures we see elsewhere – sometimes leading to one of the greater examples of successful teamwork assembling something much greater than the sum of its pieces.
I found it appropriate to follow the lead of the deadly sins and divine virtues.
Lets start by talking about hubris: The lack of respect when confronted with the challenge of growing a system and overestimating of abilities to fix structural problems on the go. “That’s not rocket science” has been heart more than once before hitting the wall.
This is followed closely in rank and time by syndroms of greed. The unwillingness to invest into structural maintenance. Not so much when things start off, but very much further down the timeline when restructurings are required to preserve the ability to move on.
Different, but possibly more harmful, is astronaut-architecting, creating an abundance of abstraction layers and “too-many-steps-ahead” designs. The modularization gluttony.
Taking pride in designing for unrequested capabilities while avoiding early verticals, showing off platforms and frameworks where solutions and early verticals are expected is a sign of over-indulgence in modularization lust and built-up of vainglory from achievements that can be useful at best but are secondary for value creation.
Now sticking to a working structure and carefully evolving it for upcoming tasks and challenges requires an ongoing team effort and a practiced consensus. Little is as destructive as team members that work against a commonly established practice out of wrath, resentment, ignorance, or simply sloth.
Modularization efforts fail out of ignorance and incompetence
But it does not need to. If there are human sins increasing the likelihood of failure, virtues should work the opposite.
Every structure is only as good as it is adaptable. A certain blindness for personal taste in style and people may help implement justice towards future requirements and team talent and so improve development scalability. Offering an insulation of harmful influences, a modularized structure can limit the impact of changes that are still to be proven valuable.
At times it is necessary to restructure larger parts of the code base that are either not up to latest requirements or have been silently rotting due to unfitness for some time already. It can take enormous courage and discipline to pass through days or weeks of work for a benefit that is not immediate.
Courage is nothing without the prudence to guide it towards the right goal, including the correction of previous errors.
The wise thing however is to avoid getting driven too far by the momentum of gratifying design by subjecting yourself to a general mode of temperance and patience.
MAY YOUR PROJECTS SUCCEED!
(Pictures by Pieter Brueghel the older and others)
z the world 