CHANGE: Planned & Unplanned

CHANGE: Planned & Unplanned

Volume 8 of the Quality Software Series

by

Gerald M. Weinberg

SMASHWORDS EDITION

PUBLISHED BY:

Gerald M. Weinberg on Smashwords

CHANGE: Planned & Unplanned

Copyright © 2011 by Gerald M. Weinberg

All rights reserved. Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form, or by any means (electronic, mechanical, photocopying, recording, or otherwise) without the prior written permission of both the copyright owner and the above publisher of this book.

Smashwords Edition License Notes

This ebook is licensed for your personal enjoyment only. This ebook may not be re-sold or given away to other people. If you would like to share this book with another person, please purchase an additional copy for each person you share it with. If you're reading this book and did not purchase it, or it was not purchased for your use only, then you should return to Smashwords.com and purchase your own copy. Thank you for respecting the author's work.

Part I. Planning for the Future Organization

Recognition of the distinction between a stable system and an unstable one is vital for management. The responsibility for improvement of a stable system rests totally on the management. A stable system is one whose performance is predictable. It is reached by removal, one by one, of special causes of trouble…

There are many ways to categorize software organizations. In this series, I've used the concept of software cultural patterns, based on the work of Philip Crosby. The focus of this volume is on moving toward a Pattern 4 (called Anticipating) culture, assuming that you already have a Pattern 3 (Steering) culture. Every culture changes, of course, but the Anticipating culture is a continuously changing culture that plans its changes to shape the organization it wants. Thus it is concerned with both the process of fostering change and the process of maintaining stability in the face of change.

Most writers on improving software engineering emphasize the essential changes, but neglect the subject of how these changes are to be brought about. These writers seem to assume that all that's required is telling people what to change—but this is the Pattern 2 (Routine) paradigm.

The Pattern 4 paradigm emphasizes process before product—and this emphasis carries over to the process of changing the organization itself. Until you know how to plan for change, you will continue to be frustrated—knowing which things to change, but never seeming able to accomplish them, even with the help of experienced change artists (the subject of Volume 7). If you hope to transform your software engineering organization to Pattern 4, you must master the art of planning for change, and that is the subject of Part I.

Chapter 1. Meta-Planning: Part I. Information

Climate is what you expect. Weather is what you get. - Anonymous

The winds and waves are always on the side of the ablest navigators. - Edward Gibbon

1.1 Start by Meta-Planning

Figure 1-1 gives an overview of strategic planning as practiced by many effective software engineering organizations.

Figure 1-1. The strategic planning process uses both internal and external information to generate visions of both product and process. These visions, in turn, drive the action planning process.

The plan addresses two broad questions for a time interval that is typically three to five years long:

• What products/services will we supply?

• What processes/resources/culture will we need in order to supply them?

These two visions are, of course, interrelated, so the strategic planning process will have to be able to estimate the feasibility of some of the visions, asking:

• Can we build these products with the processes/resources/culture we now have?

If the answer is yes, then the process vision is one of maintaining and perfecting the present process. If the answer is no, then the planners must decide whether to reduce their ambitions or to raise their process vision.

The reason we create these visions of product and process is for them to become inputs to a tactical planning process. Some organizations, however, lose sight of this purpose, and the actual strategic planning process more closely resembles Figure 1-2, which was drawn for me by one of my clients. In this so-called strategic process, planning becomes an end in itself, unconnected with the rest of the organization, and producing lots of filed paper but no action.

Figure 1-2 How strategic planning is actually practiced in some organizations.

Looked at cynically, the process of Figure 1-2 is what you get when some of the big shots are given the chance to play at being executives. A more realistic interpretation is that these are scared folks, just like you and me, who have been forced by their management to carry out a process for which they have no experience, aptitude, training, or skill.

Managers who do not know how to get useful feedback about business information feel out of control. The only things they have any sense of control over are time and money—hence the popularity of budget and deadline restrictions. No wonder so much drinking goes on. Perhaps the organization is fortunate the vision document hides in the files, never to be seen again.

When done well, however, the entire process of strategic planning for change can be regarded as having three major components carried on iteratively to produce successive versions of the plan:

• information gathering, including both observing and ignoring information from the entire organization and all parts of its environment

• problem solving, including systems thinking, negotiating, and translating into action-generating principles

• mechanics, including knowing when, where, and how the planning is done, as well as who is involved, and understanding the difference between tactics and strategy

In Variable (Pattern 1) and Routine (Pattern 2) organizations, strategic planning attempts are almost always a fruitless exercise. Until problems in all three components are cleared up, the essentials needed for a fruitful change strategy planning are simply not available. In these circumstances, you need to bootstrap the planning process. You must restrict the output of your first planning sessions to meta-plans—plans that need to be carried out for the planning process itself before you can do effective planning on the rest of the organization (Figure 1-3).

Figure 1-3. The effectiveness of meta-planning has a major impact on the quality and effectiveness of every stage of the organizational change process.

Some of these meta-plans will concern the quality and quantity of information, and that's mostly a matter of skill at communicating with people. For instance, one of the most powerful meta-plans is a blueprint for training to raise the communication skill level of the planners. Some of the meta-plans will deal with mechanics, such as steps leading to the use of a skilled facilitator. And some of the steps will deal with problem solving, such as learning to think about risks and trade-offs.

The sections of this and the next chapter consider each of these components from the point of view of how they can be managed or mismanaged. These chapters provide a checklist of dangers you need to watch for, and suggests meta-plan actions to initiate when you detect any of those dangers.

1.2 Information Gathering

The information for making strategic decisions needs to come from both inside and outside the organization. The quality of the planning will be supported by the quality of the information. Three types of problems plague the quality of the information used in strategic planning:

• Omission of some essential source, particularly those doing the work

• Dependence upon an unreliable source

• Wallowing in excessive detail, while ignoring relevant details

Let's study each of these problems, to discover their causes and to frame meta-plan actions that might overcome them.

1.2.1 Omission of information

As with any meeting, the success of a planning meeting is 90% determined by the events that take place before everybody gets into the meeting room. Nowhere is this principle more clear than in the case of missing information. Managers may be the smartest people in the organizations, but they're not psychic. Here are some typical problems of missing information, their causes, and suggested actions to remedy them:

Problem: Omitting customers.

Software engineering organizations often fail to consider customers' wishes in their planning process, arguing customers don't know what they want, or else they want everything. Naomi Karten gives an example that would be funny if it weren't tragic:

A company invited me to help them establish a service-level agreement. When I got out there, they explained that this effort was part of a comprehensive IS-wide strategic review and a revamping of their services, policies, etc. They explained that this effort had been given the name The Voice of the Customer, and it had been in progress for more than a year. Oh, I asked, and to what extent have customers been involved? Not at all, they told me. The Voice of the Customer?

Cause: Though it hides behind blaming, this problem is pure irrelevance. Managers hide out because they feel inadequate when actually forced to communicate and negotiate with customers. Some managers hide because they don't know how to tackle what appears to be a massive and unfamiliar undertaking in which they fear they'll have to give away the store.

Meta-Plan Action: In the strategic meta-plan, specify that customer satisfaction will be measured regularly and systematically. Also specify that customer desires will be updated regularly, using such varied techniques as brainstorming, focus groups, surveys, and questionnaires. Until you've executed this meta-plan, there's not much sense holding your first planning meeting.

The measurement of customer satisfaction need not be overly precise, nor imply that every customer's tiniest whim must be satisfied. It does have to be accurate, however. It also has to be made available to everybody in the development organization, so they can understand their customers and will have the information needed to make sensible decisions on all scales. As my colleague Lynne Nix points out, developers in many organizations don't even know how to use the product they're developing. How could such developers ever make sensible decisions about anything that might affect usability?

Sometimes hearing the Voice of the Customer involves removing management-imposed barriers. For example, Norm Kerth observes:

One of my clients won't let their software developers meet their customers because

(1) They would embarrass us.

(2) They would make it a boondoggle.

And Phil Fuhrer adds, from his experience,

(3) They might disclose product plans before they are ready.

(4) They might make promises that can't be kept.

Problem: Omitting potential customers.

Frequently, these organizations fail to consider potential customers: those people who might be using their products and services, but are not. One anonymous correspondent from a failing software company told me:

At our peak, we had about 1,200 customers for our mainframe package. When we were down to about 600 customers, I suggested we interview some of the customers we had lost to find out what we should do differently. I was told that we didn't want to hear from people who weren't loyal to our company. When we got down to 300 customers, I left. Now I'd be surprised if they still had 100.

Cause: Managers do not know, or are afraid to use, any processes to gather information on satisfaction of existing customers, let alone potential customers, so they deny that such information is important.

Meta-Plan Action: In the strategic meta-plan, specify that customer measurement be extended beyond current customers to include former customers and potential customers.

Problem: Omitting overall business strategy.

Even if they completely ignore their customers, managers, you'd logically assume, would not want to ignore their own executives. Thus, they would want the corporate business plans to be part of the planning information base. Yet, software engineering managers are often unable to demonstrate any relationship between their software engineering strategy and the firm's overall business strategy. Indeed, in one organization,

Someone said we should look at the corporate strategic plan. Nobody had one at the meeting, so we took a break while we went to look for one. After thirty minutes, nobody could find a copy, so we resumed our planning session without it. The next day, somebody brought one, but by that time, most of our work was done. Besides, everybody thought it was too thick to read.

Cause: Sometimes the planning process produces a report purporting to show how software engineering relates to the overall corporate strategy—but the report turns out to be a fraudulent assemblage of obfuscating verbiage. In one organization, a secretary is given the job of preparing this document, filling in arbitrary connections according to a formula. This process actually saves executive time since nobody will ever read the document anyway.

Meta-Plan Action: In the strategic meta-plan, specify that departmental plans be reviewed by upper management before they become plans, and that steps be taken in the planning process to ensure this review passes before the process proceeds.

Problem: Omitting any organizational information.

Amazingly, many strategic planning sessions use essentially no information of any kind from within their own organization. A technical lead told me this story:

I'd never been at one of the strategic off-sites, but my boss was sick so I had to take her place. The only thing they used was an org chart, and they jockeyed back and forth, trading people and projects like the National Football League. I was scared to say anything, so my boss lost 7 of 29 people who were working for her. She was really mad at me, but I guess it didn't matter, because I was one of the 7.

Cause: Quite possibly, no useful data have ever been collected. Instead, participants simply give their opinions of how things should be. They don't have any measures of the productive capacity of their organization, nor do they have a clue about the magnitude of the design maintenance debt of existing products they plan to modify.

Meta-Plan Action: In the strategic meta-plan, specify that staff along with the managers—not the managers alone—establish a system of measurement to provide the measures needed for future strategic planning.

Problem: Omitting any outside information.

When it comes to outside sources of information, the track record of these organizations is no better.

Cause: Frequently, they enter the planning process with lots of technology data, but this tends to be supplied by a favored vendor or two. Sometimes the entire executive crew is wafted off to a vendor site for a technology briefing as preparation for the planning process. One of the more enterprising vendors actually facilitated the strategic planning at the Country Club, with some of the major decision-making done on the golf course.

Meta-Plan Action: In the strategic meta-plan, specify that staff with managers establish a system of environmental search—scanning what exists in the world outside the organization—to provide the measures needed for future strategic planning. This search is not to be limited to technological information, for technology is only a small part of the total context in which a system lives. Moreover, the sources need to be reliable and unbiased.

Problem: Lack of organizational development knowledge.

As poorly informed about technology as the planners may be, when it comes to knowledge of organizational development possibilities, many software engineering managers are better informed about the performance characteristics of their BMWs. At least when it comes to the state of the economy and their industry, they read Business Week or Time. A fellow consultant told me:

I conducted a telephone survey of the nine managers, one of whom had invited me to facilitate a strategic planning meeting dedicated to upgrading their development organization. Six of them had never heard of the Software Engineering Institute. Two of them had heard of it, but couldn't tell me what it was, or what it did. The only one who knew something about it was the one who invited me. I suggested they do some reading before they had the meeting. They got themselves a different facilitator.

Cause: Many software engineering executives lack training in what structures and processes are possible in organizations. Moreover, since many have spent their careers in a single organization, their experience is limited.

Meta-Plan Action: In the strategic meta-plan, specify training in organizational possibilities for future planning participants. This training should include working in a staff position for a period of time to experience the workers' realities. Also specify participation in planning sessions by some organizational development specialists

A radical alternative solution is to abandon strategic planning for change altogether and adopt the SEI approach whole hog. At the very least, the planning team should be required to read and discuss the SEI's Capability Maturity Model (CMM). Although I disagree with the SEI on many points, and I believe the CMM doesn't cover half of what has to be done, this approach would save a lot of time, and produce a better plan than three-fourths of the strategic change planning efforts I've witnessed. If you can't do it better, why not reuse someone else's work—and be a model for your developers?

Problem: Running sham planning sessions.

Given a lack of valid information, any sort of planning would be a sham; yet it continues to be done in this way, at great cost and disruption to the organization. Such a sham demoralizes workers and discredits their management. As one wag expressed the principal advantage of strategic planning in his organization, At least it keeps them out of the office so we can get some work done.

Cause: When the planning is strategic, however, the impact is so far in the future that it's easy to hide the emperor's nakedness. When all the alleged planning is done at a luxurious executive retreat, the grunts at the office may suspect it's a sham but, like their managers, have no data to prove their theory.

Meta-Plan Action: In those organizations that do plan effectively, strategic planning sessions are not some sort of executive perquisite, but hard, intelligent work based on meaningful data. Specify that the planners do their homework and come to the sessions informed about the critical areas that will affect their plans. Hold the sessions more openly, and conduct them in slightly more Spartan surroundings than a five-star resort. Another good idea is to require a public display and review of a substantive work product produced by the planning.

1.2.2 Unreliable sources

Unfortunately, simply working hard to gather data is not sufficient, because most of the data that are readily available are of questionable quality.

Problem: Running on visible figures alone.

Internal reports are not reliable indicators of what's happening in the organization, or if they are reliable, they're irrelevant.

Cause: Deming's Fifth Deadly Disease is running a company on visible figures alone. He points out that there are some good things like ecstatic customers that cannot be reduced to figures in any regular way. But fear of customers and employees leads managers to hide behind their spreadsheets.

Meta-Plan Action: Use the technique of management by walking around prior to planning exercises to validate those information systems that are buried several levels down from the manager. As skilled managers know, a few sample conversations will quickly indicate whether or not the information in formal reports has the meaning it appears to have. Some managers hire consultants to do this, but walking around does the same and more if the managers know how to listen. If they don't, they create great, unregulated disturbances by walking around, and they're better off hiding in their offices.

Problem: Unstable current processes.

The processes being measured are so unstable that measurements have no meaning except to show the instability, as one developer illustrated:

Our test manager reported to the meeting that the bug file had increased from 11,392 to 12,514 since the last monthly meeting. This, he said, was an indicator of progress, because this was the first time since we started measuring that the monthly increase had been less than 10% (It was 9.8%).

Cause: The causes of instability are many. Systems thinking—the ability to reason about nonlinear feedback systems—is needed to root them out.

Meta-Plan Action: In