Innovation Thrives between the Lines of Chaos and Control

Innovation killer #4: Create an obstacle course for ideas. Guaranteed way to kill the innovative spirit? Model your processes on Kafka’s The Trial or your typical parking clerk’s office.

CIO Magazine, July 24, 2007

On the heels of the SpigitFusion release, I’ve had the opportunity to hear from a number of people on the topic of process in innovation. There has been plenty of positive feedback. For example, Gwen Ishmael of Decision Analyst Inc. tweeted: “Very nice – Evaluation + #Crowdsourcing = Faster #Innovation.”

Others have raised a very valid concern. Namely, that the introduction of a more formalized tracking process to evaluate ideas risks overengineering the innovation program. Put too much structure in place, and the fragile essence of a good idea will be snuffed out.

This is certainly a balancing act. Too little process, and there’s no systematic way to ensure good ideas get into the corporate ecosystem. Too much control, and bureaucracy drags down good ideas.

The easiest answer is: “it depends”. It depends on the organizational culture and existing ways of working. But there is more to it than that.

The better answer is: “it’s situational.” Even within the same organization.

The Chaos to Control Spectrum

In a 2006 issue of Emergence: Complexity & Organization, academics Ysanne Carlisle and Elizabeth McMillan wrote about the issue of chaos versus control in organizational innovation. Their article,  Innovation in Organizations from a Complex Adaptive Systems Perspective (pdf link), includes a framework for considering when to give free rein, when to tighten things up. First, understand the spectrum of control applied to innovation:

The authors lay out a spectrum of control inside organizations, ranging from absolute randomness to lock-to-down control. Characteristics that define each are provided. The larger context of the matrix is that organizations need to both:

Exploitation is appropriate in stable environments, and will typically involve more process and control. Exploration is needed in environments that are unstable, and leveraging existing knowledge and competencies may not provide the needed perspectives. Generally speaking, exploitation is more associated with the right side of the spectrum. Exploration is with the left side of the spectrum. The sweet spot? See that center column, Complex. That’s the place where companies that want to succeed in innovation will live.

Toggling between the Edges of Chaos and Control

There is a point of view that runs strong in innovation discussions, that says innovation cannot really be managed, or that process is not appropriate for achieving innovations. For example, this thought about organizational norms and innovation:

Most organizations have strong influences to normalize employee behavior. The objective often seems to limit behavior / activities to acceptable levels. This creates a behavior pattern that is (usually) an unspoken standard of what is / is not appropriate for the entire organization. The greatest threat to innovation is the response of an organization. When activities extend beyond the accepted pattern (an innovation spike), the organization enacts normalizing (can be subtle).

The counter to that observation is often that there should be zero process and oversight on innovation. Any type of organizational control is going to kill innovation. It is a fatalistic view of innovation ever occurring. Which is not the case. It also assumes a nearly anarchist approach to innovation.

Carlisle and McMillan offer a sophisticated framework which considers the internal and external environments when it comes to levels of control of the innovation process. Generally speaking, there are times a stronger degree of control is needed, and times when a bit of “chaos” is in order.

Organizations are best served if they occupy the Complex part of the innovation control spectrum above. For more discontinuous, radical types of innovations, organizations should operate at the border between Complex and Chaotic. For more sustaining types of innovation, organizations operate at the border between Complex and Hierarchical.

Below are two examples that bring this to life. On the left is a 2009 post by a Facebook engineer. On the right is a recent research study on consumer packaged goods companies (CPGs). Notice the contrasting views on the role of process and control in innovation:

Consider the environment in which Facebook operates. It is a land grab, with fluid business models and changing consumer desires. New entrants hit the market seemingly every day. Facebook operates in a relatively unstable environment, and treats process accordingly. There is a strong exploration orientation. Note that the engineer’s post is from 2009, it’s possible more process has been introduced since then as Facebook grows, establishes the default social network operating model, generates real revenue and profit, and eyes an IPO.

CPGs, on the other hand, have established market positions and business models. The modes of distribution (e.g. grocers’ shelves), brands’ awareness and consumer tastes are well-established. They operate in an environment of relative stability. Hence the orientation toward greater control in the innovation process. Exploitation is the primary mode of innovating.

CPGs, of course, need to be vigilant for changing distribution models and consumer tastes. Disruption can come from multiple places. In those situations, the CPGs would be well-advised to stop a rigid stage process and put more experimentation into the innovation mix. This is something my colleague James Gardner advocates. It’s also a time to use more adaptive evaluation processes, not preset criteria.

Pay Attention to the Operating Environment

As Carlisle and McMillan write:

We suggest that different parts of an organization need to operate in different parts of the spectrum to respond and adapt to internal and external change, adjusting their responses in line with valuable learning experiences.

This is sound advice for companies. Consider the operating environment in which a business operates. Apply different evaluation philosophies consistent with the level of stability or potential instability, as the case may be. Control and process are essential to making innovation a reality inside organizations. They have to be, because:

The key is allowing the level of these elements to vary, both philosophically and tactically. Just avoid creating a Rube Goldberg process of managing ideas.

(Cross-posted @ Spigit » Blog)