Complex Adaptive Systems: Why most Information Systems don’t Measure Up.

Complex Adaptive Systems, or Complexity Science, is about a lot more than Information Systems, we human beings being a prime example of such complexity. Wikipedia and other sources provide all the details you want on this topic, but one view of it that has stuck with me over the years is the difference between ‘complicated’ and ‘complex’.

Roughly speaking, a complicated system is what it sounds like, a system that is not simple and not intuitively understandable; however, given time and effort, you could analyze and identify all its parts and pieces and describe the whole system. The key is that it is a fixed system.

In contrast, a complex system cannot be analyzed and described as above because it is changing over time; analyze it last week, and again next week, and you will get two different descriptions. For decades, we have been fairly good at creating complicated information systems, but complex was and is rare, if even considered. Instead, what we have in IT is the Change Request, the Backlog, the up-coming Maintenance Release.

The problem has been our creating of information systems based on the needs of an organization at the time we asked. We were guilty of hard-coding the business process and rules (while probably not recognizing them as such) and delivered that code in working order, to be told that the business wasn’t doing things that way anymore; or worse, the business would keep this to themselves and start creating the first of many ‘workarounds’ needed to get past the system limitations and get things done. In either case, the situation would deteriorate and then the change requests would start coming.

Now, hind-sight is wonderful, and decades of hard, well-intentioned IT work should not be scoffed at; this has been a difficult realization to come to, common in many disciplines where complexity science has tussled with older paradigms; and the realization did come together in pieces over time as parts of the problem were recognized and new technologies & tools were developed to help. The now ubiquitous Database Management System came from realizing that lots of flat or indexed files processed in various batch jobs resulted in lousy information, and certainly weren’t going to work in on-line transactions, and even then we had to work our way through competing designs like hierarchical systems before the relational systems we use today emerged.

However, when it comes to complexity, data needed by a business is comparatively stable; it is how the data is used that changes more. Take a typical business process of handling a loan application; today the big loans go to Fred to process, but next week they go to Anne. That is the kind of change you cannot make to a hard-coded system fast enough to support the business. Beyond that, consider the rules that drive a process, such as loans over $100,000 need to be approved by a VP; and starting next week, it is changing to loans over $75,000.

This kind of complexity has finally led to some changes and new tools to help the business. First out of the block has been Business Process Management tools (BPM), which originated when imaging systems changed offices from moving paper around to moving units of work. More recently, Business Rule Management Systems are being implemented that move the changing of rules from the realm of the programmer to that of the business expert.

Will these tools be enough to move us from complicated, maintenance-heavy systems to truly flexible complex systems? There is no denying it will help; they will certainly solve a lot of old problems, so that we can move on to solving newer, better problems and what more can you ask for.

Once a project is started, finish it.

No principle screams from the page to me than this one, yet rarely have I seen it implemented.

I have worked under various managerial styles, but it boils down to two approaches for me, integrated or divided. All enterprises of any size do need to be composed of many organizational units, but that is to divide up the people into manageable groups; it should not divide the enterprise into separate fiefdoms. Just think how many times you have seen or been part of a re-organization, and then think about how many times that has changed the actual work you or anyone else does; the latter adds up to just about nil.

Consider a complete enterprise, with all the usual line and staff functions ; ask anyone in that enterprise how it is structured to do its business and you will probably be shown an org chart. The usual hierarchy of boxes and lines, from CEO down to the base units, is the business graphic that is the oldest and most commonly used. However, why does it have to keep changing? That’s because people are not automons, they need to be grouped in ways that maximize their contribution to the enterprise, while hopefully having job satisfaction and a boss they don’t loathe; and since people come and go, and since people can change over time, any one instance of an organization (as represented by an org chart) will be become sub-optimal as time passes, so re-organization is needed, and it is a good thing if it re-energizes people and re-marshals the enterprise to be its most effective.

The problem is when management believes that the org chart at any one time also represents how the work gets done. Worse yet, they also use the org chart as the basis for changing/improving how the work gets done. This is the ‘divided’ managerial style I mentioned earlier, and it directly affects how the enterprise carries out projects.

At any one time, an enterprise has certain amount of capacity to implement change; the main numbers are how many people there are to do projects, and/or how much money is available to hire outside people to do projects. These numbers are a primary input to that common-place activity of annual budgeting/planning; these numbers are limited/finite, so managers need to decide how to best use them. In the divided managerial style, each major tree below the CEO on the org chart is allocated some portion of those numbers; if you have 10 people available for projects, the number used may be 120 person months. So, Sales is allocated 20 months, Operations gets 40, Finance gets 35, etc. Management may come up with what it believes are equitable/effective means of divvying up the number beyond just dividing it into equal units, but that is a sham; this is further reinforced by the sham idea that if each unit operates and changes successfully in of itself, then the total of all the units’ success will equal the overall success of the enterprise. Does your company have internal charges between departments, often referred to as ‘funny money’? Then your company is divided, because all that funny money means nothing to the bottom line of the enterprise.

A divided approach is a problem when projects are used to improve the business, because projects change the work, not the organization. Take an important and common business activity, order fulfillment. If you trace the work from when a customer first orders a thing or a service, to when its delivery is complete, many (many!) org units will be involved. So, if each unit independently attempts to change/improve that stream of work, they will either overlap with other units or duplicate what other units are doing without knowing it.

The main result for IT is that each org unit will want to use its allocated resources to get its own projects done. So, irrespective of their overall value to the enterprise, a number of projects will get initiated. This will result in overlap or duplication within the systems IT delivers, assuming the systems are delivered. IT management will be faced with conflicting demands on their limited resources, so only some sub-set of the projects will be proceeding while others wait for resources. This leads to projects stopping for a while when they get to a point where they need new types of resources, and then have to start up again when those resources eventually become available.

So, irrespective of the method used to divvy up project resources, the ‘divide’ approach produces an ineffective and wasteful projects environment. The solution to this problem is to move to an integrated managerial approach.

An ‘integrated approach’ recognizes that an enterprise is a team brought together to operate a business, not a collection of independent fiefdoms; a CEO that recognizes and leverages this approach also knows that projects to improve the business belong to the enterprise, and almost always impact multiple organization units. Given this recognition, how an enterprise organizes its resources to carry out projects then will (or should) be tailored to the mix of people involved, just like any other aspect of organization. This could be anything, from ad-hoc teams to a full Project Management Office. Any good structure will do, as long as the enterprise view is maintained.

And that’s all I have to say about that.

IT Project Cost-Benefit Analysis

Assuming you got through the rough stuff of defining costs and benefits, the analysis should be simple. What you want to know is if the benefits of the project will exceed the costs, and by how much. Since costs and benefits may be spread over long periods of time, Present Values of these amounts are also usually calculated to compare the amounts in “today’s” dollars.

When I did my first Cost-Benefit Analysis for a major project, I had to work with an spreadsheet expert to to do these calculations; now you can probably download something for free or a nominal fee that will do basic and advanced calculations. The key things these tools need is the two dollar values, cost and benefit, and the length of time of the analysis. The latter is usually defined by accounting standards at your company, and the most popular time periods used are three years and five years, often based around your company’s depreciation procedures/periods.

Given that, you can usually get calculations like:
• Break-Even Point, the point in time when the benefits realized exceed the project cost
• Various rate of return and yield values, like IRR.

These calculations may be used to determine if a project passes a funding hurdle; its not enough that a project makes money, but it has to make more than investing the equivalent dollars of the project cost in securities or other investments.

After all this is done, a project can now proceed into the gating process to see if it has enough expected value to warrant its being initiated and carried out. Of course, if your analysis has determined already that the project does not have positive return or does not surpass the hurdle rate, you can stop now and move onto the next project idea. Determining that a project is not good for the business is just as valuable as finding those projects that are good for the business. Resources should not be wasted.