Agile Organizations - Dealing with Expected Change

Agile is an attractive word. It means swiftness with discipline, with an emphasis on alertness to change in one’s external surroundings and quickly responding to change as needed.

The word I want to focus on from above is external. When the surroundings that generate change are actually controllable, then the need for agility is greatly reduced. Agility comes at a price, and maintaining agility is a waste of resources when change is controllable.

A prime example of the difference between internal (controlled) and external (un-controlled) is an operating enterprise: business company, non-profit group and the like. The interesting problem for most enterprises is how effectively they control their own operations. The boundary between internal and external can become nebulous if control is lacking in parts of an enterprise.

Control sounds like a bad word. It implies top-down definition of all aspects of enterprise operations. This can be true, but control can also include delegation of authority, with expectation of results without definition of methods used. Control is maintained, but authority is distributed. This is the sphere of business management practices, defined and changing and transforming from Henry Ford to Jack Welch. It is not easy, and many others have and will address it better than I, so I will reference them as need be.

The topic of interest here is the boundary between internal and external, controlled and uncontrolled, event and response… but not between knowable and unknowable. An enterprise needs to know what is happening around it in order to focus its operations on knowing what events are important, events that it needs to respond to. For these events, an enterprise defines processes/procedures to execute when triggered by events, to provide the desired results in response to the events; successful enterprises include those that recognize that the nature of events and corresponding processes change over time, sometimes very quickly. So, at any one point in time, an enterprise needs to be in control of its processes, but that control can’t stifle the need to change as fast as the world around it changes. Further, this rapid change needs to be accomplished without disrupting on-going operations.

What we are discussing here is Controlled Agility; the ability of an enterprise to respond swiftly to changes in its surroundings without losing its internal cohesion.

The basis for controlled agility in an enterprise includes understanding that:

•         an enterprise runs on its processes, not its org chart
•        processes are composed of distinct activities and decisions
•        processes run on information, gathered and stored and used
•       processes are guided by business rules, especially when decisions need to be made.

With this basis, an enterprise is well-positioned to be Agile without losing Control. It all comes down to understanding Change.

Change comes in many degrees of impact to an enterprise. Consider workflow, which is based on process with roles and authority levels defined. If all loan applications over $100,000- go to Fred to be underwritten, the workflow will change temporarily when Fred goes on vacation. Those loan applications have to go to someone else until Fred gets back. This kind of change can happen frequently, but more importantly, it is a kind of change the enterprise knows can happen.

Frequent known changes are the heart of controlled agility. This is the kind of change an enterprise must be able to do, without changing itself, as part of operations. It should not need a project, especially an IT project, to make the change.

Temporarily changing workflow might be considered a simple example, but it is actually an example of an overall type of change that can occur frequently, and that is business rule change. Business Rules are so embedded in how an enterprise works, it is actually a new and revealing approach to call them out separately. In a lender enterprise, it is likely a fact that “Customers may apply for Loans”, but “Customers may apply for Loans, only if they are 18 years old or older” is a Business Rule; it constrains an aspect of enterprise operations and, as said above, rules change (next year it could be 19 years). An enterprise that knows the facts and rules of their operations is well positioned to change rules as quickly as needed. However, rule changes still need to be controlled to avoid using incorrect rules to the detriment of the enterprise. Still, these changes should not require a project to change the structure of the operation and, again, not need an IT project.

A major use of rules is to support making decisions.  Loan underwriting across many financial lending/leasing enterprises is heavily rule-driven, the information about a customer and the loan product being used to decide to lend or not. These can be complex rules, or simply that the enterprise does not lend to anyone with a Credit Score below a stated value; and again, the rules will be changed over time as a lender tunes its underwriting, or is willing to take on more risk, etc..

Decisions also play a role in defining what processes/activities are used to respond to events. We all know of BPM diagrams with boxes for activities, diamonds for decision points, and arrowed lines connecting them all. A process may be composed of 20 possible activities, but less than 20 might be used in response to any one event because of decisions based on rules.

What can change for a process? The number of processes in an enterprise, once all are defined, is relatively stable over time; new processes usually mean the enterprise has expanded its products/services to areas which need their own processes. Within a process, however, the exact activities that are carried out may change, or be re-ordered, or even be retired. It is the ability to change processes as quickly as possible, again without a project, that marks an enterprise as Agile.

Changes in decisions and rules used in processes are the most frequent changes an Enterprise must do. Changes to the activities in a process change less frequently than decisions/rules, but often enough the agile enterprise needs to it as part of its operations.

Are there aspects of an enterprise that are more stable? And are types of changes to them not necessarily known or predictable? This is a loaded question, because the answer is information or specific data used by an enterprise. Once all the information needs are known for an enterprise, these are very stable over time. The need to change information needs is similar to the need to add more processes; it happens when (as above) the enterprise is expanding into new products or services different from current products/services. To be more precise, information needs may change as the enterprise expands, but they may not as well; data requirements have been shown to be the most stable aspect of an enterprise.

So we have two ends of the change pendulum: from frequent and known, to rare and unknown. As said earlier, frequent known changes are the heart of controlled agility. 

To Agile or Not To Agile

After a long period of trying to grapple with "agile" or "not agile" etc., I finally realized something; it really depends on what you are trying to develop. My thoughts now divide software grossly between "product" and "process".

If you are developing a software product, especially to actually sell to customers, then I think product development methods are the way to go, and Agile fits this approach well. You do start with a Product Owner's vision of functionality, and user stories are a great way to express that vision as a starting point for developing a prototype or first release.

If you are dealing with a business process to automate, then a 'vision' is not appropriate; you have to document what the process is, completely and clearly. High-level process maps supported by use cases are a great approach for this. You have to  realize that there can be no arbitrary first release for a process based on a sprint or other cut-off. If you are automating a process to, say, transfer money from one bank account to another, you can't first release the "from" account piece without the "to" account piece.

So, use the appropriate technique for the software you are developing; no one technique meets all needs.

You have to have an acceptable budget for a project before you can start managing the budget for a project.

Over the decades, I have seen many methods and tools for managing things you create, but don’t offer any help in actually creating them. In the BA/requirements world, the key example is Requirements Management Tools; useful once you have requirements, but defining clear and complete requirements is the difficult part. Anyone who has read my past posts knows that doing requirements definition/discovery is, well, what I do.

But there other such things, and some of which having good requirements can help. For example, do you need to budget for your projects before you start them? You have an annual or quarterly process about submitting budgets and getting approved, and then a separate process for changing the budget during the projects. However, do these processes and supporting tools actually help you come up with a budget number you are comfortable with? No, so you need something else to help with that and, no surprise, requirements can be that something.

What can clear  requirements help you get? Estimates, for design, development, and testing. But, you ask, are you saying to do all the requirements for your upcoming projects long before they start, to get budget amounts? Yes and no. Requirements can be defined at different levels of detail, you just have to balance the level of detail to the granularity of the estimates needed for budgets.

When you actually start to elicit requirements for a project, the first thing you have to do is scope the requirements analysis work. This scoping will take a half-day to a day to complete. What I am suggesting is to do this scoping work-only for all your planned projects. This scoping will provide the main business activities and information  to be addressed in the project. That is enough detail for t-shirt sizing of each project,  small/medium/large; align this sizing to previous projects  of similar sizes, get the costs of those projects (not always easy but it should be somewhere) and you have numbers that are based on something real. The percentage amount of variations might still be high, 20 to 40% for example, but the numbers are based on something real, not just SWAG or other “best guesses”.

Is a day per project for requirements scoping during budgeting too much to ask to get some real numbers? I think not.

Celebrating a 5 Year Anniversary

I am celebrating of those significant work anniversaries in 2013; I have now been a Requirements Consultant for five years with IAG Consulting. For some decades before 2008 I was an employee of several different organizations as (early on) a programmer and then a Business Analyst. As those years went by, I had begun to wonder if doing business analysis as a consultant was a viable career choice; finding IAG Consulting showed me it could be, and it would be tough to see myself resuming my earlier life as an employee. As a consultant, virtually every work day is meaningful and delivers value to a client; you frankly get the focus and involvement of people when you are a clear cost, more than (sadly) an employee Business Analyst usually gets.

On the other hand, I could probably not be the effective consultant I am today without at least many years of the experience I gained as an employee. One thing as an employee does have is usually the chance to attend conferences; I also participated in industry-level standards efforts at different times. In the end, though, the move to consulting was right for me.

Since then, I have worked on projects for dozens of companies, working with hundreds of good business people to help them effectively and completely define their requirements for processes and systems. I expect to be at it for many years to come, so if you organization is looking for this kind of help, come visit us at http://www.iag.biz/ to see what we (and I) can do for you.

In response to "#NoEstimates Part 1: Doing Scrum without estimates"

In response to "#NoEstimates Part 1: Doing Scrum without estimates" at http://neilkillick.com/2013/01/31/noestimates-part-1-doing-scrum-without-estimates/

Sure, estimates are, well, estimated... based on what the person estimating knows at a point in time, the choices are (1) I estimate it it will require "X" to do it, or (2) I can't estimate what is required to do it. I think this comes down to people not wanting to accept that an estimate could be wrong, or not right enough.

I get an estimate when I need car repairs, to make sure I can or want to afford it. My mechanic calls me when he finds something he did't know that will change the estimate, and I decide what to do. If I find over time that my mechanic always under-estimates the cost, eventually I will change mechanics. The average reality should be a bell curve around an estimated value, an estimate is almost never exact, but a statistically acceptable variation is what you should look for.

   Now, such analogies of physical things are problematic when used to describe an aspect of software development. Software is intangible for one thing, and we really haven't been creating it very long to have some good tools like other disciplines (like Engineering) for predicting outcomes; but like everything else, it takes time and money to create it, so the people investing that time and money usually want an idea of how much that investment will be to get what they need. Just saying an estimate is worthless isn't really going to go over well.

    The advantage of software over physical things is that you can indeed create small amounts of it that can be seen to work. Long before 'agile' got its name, a lot of software projects switched from estimates  to time-boxing: give me x amount of resources and the project will deliver what it can in that 'box'. Investor spends a little to see what he can get for that amount. If it looks good, run another time-box. If it doesn't look good, you can stop with only a small investment amount having been spent.

    Lastly I would comment on your statement on what does an estimate have to do with creating good software and a good return on investment. ROI is a funny thing, often very date-specific; delivering great software after a competitor has already done it and captured the market will kill your ROI. Some windows of opportunity are very short; miss them and the return is nil. In these cases, you have to have some way of knowing if you can make the window. Like it or not, an estimate is one way of trying to figure that out. It is these  kinds of situations that will make investors wary of new development and look for packages or services to meet the need. Nothing is perfect  but not being to estimate software development may just lead to no new software being developed.

Business Analysis - what is it, anyway?

Way back in college I had a an economics professor who was a bit radical. I took his Intro to Economics course, and the first thing he said is " The only thing I am not allowed to change is the name of the course, but  what I will be teaching is totally different than what the course description says." And that is what he did.

I say that because sometime between then and now I started this blog and called it "Business Analysis". At the time it seemed to be the best name for attracting readers for what I would write.

Since then, one of the ongoing discussions out here in the interweb is, just what is Business Analysis? What does someone called a Business Analyst do? My usual rejoinder is to re-use of some Bill Clinton's words: "It's the Requirements...dummy." OK, I leave out the "dummy" unless i really know who I am talking to, but that is the essence. Suffice it to say, my answer (repeated many times, which you could see across many linkedin BA groups) has not  settled the issue. And that's OK, one must not assume their opinion will matter to everyone, especially when they have not paid for it (like buying a book or attending a conference).

(An aside: I was in a tourist shop somewhere and it had a little laminated sign that said "Everyone is entitled to my opinion." I bought and put it on the wall in my cubicle... when I still had one.)

So, to get to a point here, I wish I could change the name of this blog to something more specific to Requirements, but so far I find that is not possible on this platform, and I don't want to start a new one and lose followers and such. That is a technical problem, but it illustrates that I want to be more specific about what I post about. I think that could be summarized with a title like "Requirements Discovery". Until I (bother to) sort out the tech thing, just think of that name when you come by this blog.

So, if you are reading this post because of it's title, I will admit that I did not specifically answer the question; I can only opine on what I think it is: that its core is Requirements Discovery, Description, and Documentation and, despite what else a Business Analyst is asked to do (project manage, test, etc), BAs are the only people/role that has this focus.I don't mean to restrict the scope of what anyone thinks Business Analysis is, but I say if you are not doing Requirements work, then you are probably doing something other than Business Analysis.

Follow-up to “So, you’re buying a package? Don’t forget your business requirements…” Don’t over-analyze!

Follow-up to “So, you’re buying a package? Don’tforget your business requirements…” Don’t over-analyze!

For those of you who do define requirements for their software development projects, but are new to buying packages, a cautionary warning; they are not the same thing. Consider the following “the system shall” requirement  statements.

1) The system shall determine if a person ordering pizza is a current customer.

2) The system shall determine if a person ordering pizza is a current customer, only if the person is phoning to place an order.

3) The system shall determine if a person ordering pizza by phone is a current customer using the person’s name and phone number.

Each requirement is at a certain level, increasing in detail from (1) to (2), and from (2) to (3).

- Requirement (1) is at high-level, stating that the system shall do something with certain information.

- Requirement (2) is more detailed, stating that the system shall do something with certain information, in a certain situation.

- Requirement (3) is fully detailed, stating that the system shall do something with certain information, in a certain situation, according to certain rules.

Requirement (3) is what you need to provide to designers/developers to use in new software development. However, this not what you need to provide to vendors when you send them an RFP; in fact, this level of detail can be over-kill analysis and can restrict the number of vendors who could meet your needs. By being very specific about rules and restrictions, you may eliminate a vendor who has a different and possibly better approach to meeting the business need, e.g. a better way to determine who is a current customer.

So what level of detail do you need? High level requirements like (1) can be useful at the start of a project, to describe scope and support planning; this level can also be used in a Request for Information (RFI) sent to vendors, just to get an initial view of what the marketplace looks like.

When you get to an RFP, you need more detail to differentiate vendors better, and Requirement (2) is an example of this level, which I call mid-level or standard-level of detail for requirements. At this level, you can see the difference between vendors in meeting your requirements, enough to evaluate their product and decide which one will serve you best.

So I now have two recommendations for organizations looking to buy a package:

- Don’t forget your business requirements…

- … but don’t over-analyze your needs,  so you can best evaluate those packages.