Blog

Defense contractor: IT must embrace radical transparency and culture change

Although innovation and being responsive to customers is the lifeblood of every technology organization, maintaining the spark of innovation inside a large company is hard.

To gain insight into how to drive change and innovation at a large company, I spoke with a senior executive at L3 Technologies, which is the eighth largest defense contractor in the United States. The company is doing something, to which facts attest:

  • Number 276 of the Fortune 500, with revenue over $11 billion and 38,000 employees
  • Stock price increasing since 1998
  • Top 50 employer on the Women Engineer list
  • Recognized as a top supporter of historically black college and university (HBCU) engineering programs
  • Shortlisted for HR Distinction award

Heidi Wood is L3 Technologies’ Senior Vice President for Strategy and Technology. She describes her role this way:

“My team pushes for growth, both organic as well as inorganic growth. I have the privilege of being the agent that pushes for L3 to grow going into the future. Technology is always changing, and we always want to be on the cutting edge, and so we need to peer over into the abyss of what’s possible and what is conceivable into what might be inconceivable now.”


Also


Her comments are interesting, so we created a video for CXOTalk, which features conversations on innovation and transformation with business and technology leaders.

Watch our entire conversation above and read the edited comments that follow.

What is your philosophy of innovation and transformation?

Heidi Wood: The one thing that’s permanent in life is change. Once you have a company culture that embraces it, challenges it, and says we can never rest on our laurels, we can never stick with status quo — we always have to be improving, challenge ourselves to be better, and also thinking about what’s the new thing that’s going to obsolete. Whatever our customers are relying on or that we think we’re good at, we have to be ahead of everybody.

You have to embrace innovation. You have to make that part of your corporate culture. You have to encourage risk-taking because that’s a necessary, and frequently not enough spoken about, an element of innovation, which is the willingness to take risks, the willingness to be bold, put yourself out there, and be courageous.

One of the things that I talk about from a strategy is, I tell people, “I want you to be unreasonable. Don’t give me anything reasonable. I’m totally bored with that. I want to see your daring, courageous, bold things that have never been heard of before.” Crazy: that’s what we want to hear because if you’re really on the edge, then it won’t look right in the near term but, three, five years from now, people will say, “Wow. That was prescient. Who imagined that this would be the case?”

How do you drive that approach based on data inside a large organization?

Heidi Wood: I’m a big proponent of radical transparency. To reach radical transparency, you need to see data. To get the entire group of people to move along with you, you need to move towards being a data-driven enterprise.

Featured stories

Again, what we’re talking about at L3 is being data-driven, being radically transparent, brave, [and] courageous. Again, when you do it collectively, then people can better see.

The way I like to describe it is, we took all of the different systems that we have, and we piped them together into a fused system. It helps us come back to better decisions. Together, we can move with speed because all of us are seeing it at the same time and it’s based on fact, not anecdotes.

One of the things I like to write on the board is, you take that Greek symbol for the sum. I put the sum and then, in parentheses, anecdotes, and then not equal data. In other words, sometimes when you say, “How is something working?” they say, “Fine.”

“How is this other thing?”

“Oh, good.”

I’m sitting there thinking, I don’t want an English word for the answer. I want the data.

Where do data and corporate culture intersect?

Heidi Wood: One of the things that I didn’t like when I joined, and it’s common in corporate culture but I wanted to change within L3 is, somebody would ask a perfectly reasonable question, and another person will say, “Well, we don’t know the answer. We’ll get back to you.”

Well, now the conversation is dead. But, what you want is an active, “Well, let’s see what the data says,” because it exists, right? That’s the thing that drove me crazy. Somebody has got the data. How come it all isn’t in a giant warehouse so that we can peal things back to the infinite level that we want to? Sort of like an onion. You peel back the layers and peel back the layers to get down to the answer to say, “The reason why this isn’t working or is working so well, is because of this.”

People must get used to having so much data being available. Again, it’s human nature to want to shade. You want to show your better parts. But, you kind of get to a stage where everybody gets comfortable with, look, this is the truth; this is where we’re really, really at. It enables more collective contributions because people can see the areas that are ailing and say “Well, I’ve got some guys that can help with this thing that you’re working on because now we can see that that area needs work.

Sunlight, if you go from dark to sunlight, there’s that moment where you go, “Oh, it’s blinding.” But then, as soon as your eyes adjust, everybody is going to say, “I’d rather be in the sunlight than in the dark cave,” right?

One of the exciting things about IT is changing the culture with what we’re doing with radical transparency. You have an angle where IT is helping change the culture of a company.

Disclosure: This video is part of a series in which SAP invited me to SAP Select, as a paid engagement, to conduct interviews with senior executives on topics related to Intelligent Enterprise.

(Cross-posted @ ZDNet | Beyond IT Failure Blog)

CIO playbook: Citizen development is your ‘most important’ tool

CIO playbook Citizen development is your 'most important' tool

A citizen developer in action. Photo by Michael Krigsman.

The topic of low-code platforms and citizen development has come up a lot lately during my discussions with Chief Information Officers. Given the mandate to innovate while spending less money, interest in this topic is not surprising.

The heart of citizen development is giving knowledgeable end-users the tools to create applications on their own, without IT involvement or intervention.

Citizen development offers several benefits:

  • Fit. No one knows what a business user needs better than that person herself. When users create apps, by definition, those apps fit the users’ business purposes.
  • Speed. Citizen developers don’t wait for IT to approve apps, specs, screens, data access, or anything else. They do the work and have the result right away.
  • Cost. IT saves money on developers when users in other departments build apps; it’s a simple cost equation.

During episode 283 of the CXOTalk series of conversations with the world’s top innovators, I asked three-time CIO and author of the book Driving Digital, Isaac Sacolick, for his thoughts on citizen development:

I think it’s probably the most important technology tool that CIOs need to look at. We do not have enough staff and our staff who are strong at AppDev really need to focus on the customer-facing pieces that are going to move the needle.

Building tools for dealing with knowledge bases, doing workflows, integrating enterprise data sources into a single utility that can be used for a one-time purpose — these are great use cases for low-code environments to get developers to be more productive and even citizen development programs.

Sacolick also addressed potential issues with citizen development, in areas such as governance, security, and data integration:

CIOs should stop using the word governance. They should implement governance in a way that people will understand it. Governance includes things like version control, locking down information but giving access to the people who need it. It means improving data quality and solving those problems when you give new technology to a business group.

We need to provide tools [that allow citizen developers] to do things in a safe way, in a controlled way, and a practiced way. That’s what governance is about. We must help end-users understand the practices they must implement.

Citizen development is a crucial topic for CIOs today. Nick Johnson of Salesforce interviewed me recently about citizen development, and the conversation sheds light on important issues.

Here is a somewhat edited and reworked version of that conversation:

SALESFORCE: In 2017, all U.S. computer science graduates would have filled less than 9% of all open developer positions. There are plenty of people out there who are suggesting there is a large and growing “skills gap” in the IT world. Do you agree with them and how did we get to this point?

MICHAEL KRIGSMAN: I agree that there’s an IT skills gap, but to answer your question, let’s look at the history of IT.

IT as a corporate function began with technology in the 1950’s. Computers were large, expensive, and complicated machines, so it made sense to create layers of protection between business users and those with the technical skills to operate that equipment. We all know the stereotype of people in white coats with punch cards.

From the dawn of corporate computing until recently, the focus of IT has been on protecting those complicated machines and providing an interface to make them available to business users.

1280px-Univac_1108_Census_Bureau

1280px-Univac_1108_Census_Bureau

UNIVAC computer, used for the 1960 U.S. census

We live in a different world now — a world of digital transformation — and two key shifts have had a major impact on the role, responsibilities, and purpose of the IT department.

First, equipment has become much less expensive and therefore pervasive. Second, we’re all computer experts now, and computing is an integral part of our daily lives.

Although these shifts have had profound implications for modern IT departments, some senior IT leaders still have the old mentality and act like the primary role of IT is protecting corporate assets and infrastructure. Although these attitudes are changing, they are still out there.

And yet, business partners and users today expect IT to support agility and speed rather than merely protect assets. Yes, protection and governance are essential, but users want IT to deliver clear business benefit – they want IT to supply technology that solves their problems fast and without hassle.

We need to distinguish between speed, efficiency, and innovation when it comes to IT.

The business wants IT to be fast (meaning agility) and efficient (meaning, get more stuff done with fewer resources), but also to support innovation (which means improvement, or doing things better).

The modern challenge for IT is changing the focus from efficiency to innovation. This the challenge number one for the CIO today.

It’s hard, because the business says, “We want you to do a lot more stuff, and we want you to innovate, but we also want to cut your budget.”

SALESFORCE: How does this history of IT relate to the skills gap today?

MICHAEL KRIGSMAN: We need to look at three main areas:

First, we know that the business expects IT to be a strategic partner. But, does the IT department have the skills needed to fulfill this demand? Can we realistically expect IT to engage with the business beyond technology solutions and infrastructure, to have expertise about marketing campaigns, financial programs, and the like? There’ a business skills gap. Actually, that’s the most important gap.

Second, we have the mindset issue. The demand that people in IT shift their thinking from efficiency as the prime directive to innovation as the essential IT mandate.

Third, the mandate of IT has changed, and people in business have strong technical skills, so they expect IT to supply resources of a certain caliber. Does IT have those resources on-board to allocate quickly to meet business demand?

Coming back to citizen developers, the tools now exist for non-technologists to take a certain level of development into their own hands. In this world, companies must decide where development and computing should take place.

With the proliferation of easy-to-configure SaaS applications, there’s a fine line between departments buying their own computing applications and doing their own development.

The ease of buying web-based applications has accelerated the rise of “shadow IT.” The concept of shadow IT means people in the business buying (and configuring or developing) applications without involving or asking IT.

The skills gap is about meeting business needs in a world where IT does not have unlimited resources, but the business has almost insatiable demand for IT support.

That’s your gap.

SALESFORCE: How can citizen development help bridge that skills gap?

MICHAEL KRIGSMAN: CIOs should provide the services, the infrastructure, get the right governance in place — and then let your employees have at it. It’s beneficial for IT and helpful for employees who want to become citizen developers.

Why not offload a lot of the work of traditional IT on to the people who need it and know what they want? If you give them the right tools and they have the right skills, and they can do it themselves. They don’t have to talk to IT; they just go do it.

It’s less work for IT and lets the CIO do more with less, as I described earlier. It’s faster for the business, which increases speed and makes them happy.

Citizen development lets the CIO focus on infrastructure, services, enterprise architecture, security, and other areas that must remain the exclusive province of corporate IT.

SALESFORCE: How should companies define the governance around this new cadre of citizen developers?

MICHAEL KRIGSMAN: Make sure that your systems don’t allow users to bypass core governance standards.

For example, don’t allow them to expose data through the firewall unless it’s governed in the usual ways that the corporation allows. Give citizen developers access to certain types of data only, whatever is within their permission role or profile. Things like that. Only expose the services that are, again, appropriate to the organizational role and profile of the citizen developer.

Build governance into the system and then let users do whatever they want. That’s the mark of an efficient and innovative IT organization!

(Cross-posted @ ZDNet | Beyond IT Failure Blog)

CIO playbook: Citizen development is your ‘most important’ tool

CIO playbook Citizen development is your 'most important' tool

A citizen developer in action. Photo by Michael Krigsman.

The topic of low-code platforms and citizen development has come up a lot lately during my discussions with Chief Information Officers. Given the mandate to innovate while spending less money, interest in this topic is not surprising.

The heart of citizen development is giving knowledgeable end-users the tools to create applications on their own, without IT involvement or intervention.

Citizen development offers several benefits:

  • Fit. No one knows what a business user needs better than that person herself. When users create apps, by definition, those apps fit the users’ business purposes.
  • Speed. Citizen developers don’t wait for IT to approve apps, specs, screens, data access, or anything else. They do the work and have the result right away.
  • Cost. IT saves money on developers when users in other departments build apps; it’s a simple cost equation.

During episode 283 of the CXOTalk series of conversations with the world’s top innovators, I asked three-time CIO and author of the book Driving Digital, Isaac Sacolick, for his thoughts on citizen development:

I think it’s probably the most important technology tool that CIOs need to look at. We do not have enough staff and our staff who are strong at AppDev really need to focus on the customer-facing pieces that are going to move the needle.

Building tools for dealing with knowledge bases, doing workflows, integrating enterprise data sources into a single utility that can be used for a one-time purpose — these are great use cases for low-code environments to get developers to be more productive and even citizen development programs.

Sacolick also addressed potential issues with citizen development, in areas such as governance, security, and data integration:

CIOs should stop using the word governance. They should implement governance in a way that people will understand it. Governance includes things like version control, locking down information but giving access to the people who need it. It means improving data quality and solving those problems when you give new technology to a business group.

We need to provide tools [that allow citizen developers] to do things in a safe way, in a controlled way, and a practiced way. That’s what governance is about. We must help end-users understand the practices they must implement.

Citizen development is a crucial topic for CIOs today. Nick Johnson of Salesforce interviewed me recently about citizen development, and the conversation sheds light on important issues.

Here is a somewhat edited and reworked version of that conversation:

SALESFORCE: In 2017, all U.S. computer science graduates would have filled less than 9% of all open developer positions. There are plenty of people out there who are suggesting there is a large and growing “skills gap” in the IT world. Do you agree with them and how did we get to this point?

MICHAEL KRIGSMAN: I agree that there’s an IT skills gap, but to answer your question, let’s look at the history of IT.

IT as a corporate function began with technology in the 1950’s. Computers were large, expensive, and complicated machines, so it made sense to create layers of protection between business users and those with the technical skills to operate that equipment. We all know the stereotype of people in white coats with punch cards.

From the dawn of corporate computing until recently, the focus of IT has been on protecting those complicated machines and providing an interface to make them available to business users.

1280px-Univac_1108_Census_Bureau

1280px-Univac_1108_Census_Bureau

UNIVAC computer, used for the 1960 U.S. census

We live in a different world now — a world of digital transformation — and two key shifts have had a major impact on the role, responsibilities, and purpose of the IT department.

First, equipment has become much less expensive and therefore pervasive. Second, we’re all computer experts now, and computing is an integral part of our daily lives.

Although these shifts have had profound implications for modern IT departments, some senior IT leaders still have the old mentality and act like the primary role of IT is protecting corporate assets and infrastructure. Although these attitudes are changing, they are still out there.

And yet, business partners and users today expect IT to support agility and speed rather than merely protect assets. Yes, protection and governance are essential, but users want IT to deliver clear business benefit – they want IT to supply technology that solves their problems fast and without hassle.

We need to distinguish between speed, efficiency, and innovation when it comes to IT.

The business wants IT to be fast (meaning agility) and efficient (meaning, get more stuff done with fewer resources), but also to support innovation (which means improvement, or doing things better).

The modern challenge for IT is changing the focus from efficiency to innovation. This the challenge number one for the CIO today.

It’s hard, because the business says, “We want you to do a lot more stuff, and we want you to innovate, but we also want to cut your budget.”

SALESFORCE: How does this history of IT relate to the skills gap today?

MICHAEL KRIGSMAN: We need to look at three main areas:

First, we know that the business expects IT to be a strategic partner. But, does the IT department have the skills needed to fulfill this demand? Can we realistically expect IT to engage with the business beyond technology solutions and infrastructure, to have expertise about marketing campaigns, financial programs, and the like? There’ a business skills gap. Actually, that’s the most important gap.

Second, we have the mindset issue. The demand that people in IT shift their thinking from efficiency as the prime directive to innovation as the essential IT mandate.

Third, the mandate of IT has changed, and people in business have strong technical skills, so they expect IT to supply resources of a certain caliber. Does IT have those resources on-board to allocate quickly to meet business demand?

Coming back to citizen developers, the tools now exist for non-technologists to take a certain level of development into their own hands. In this world, companies must decide where development and computing should take place.

With the proliferation of easy-to-configure SaaS applications, there’s a fine line between departments buying their own computing applications and doing their own development.

The ease of buying web-based applications has accelerated the rise of “shadow IT.” The concept of shadow IT means people in the business buying (and configuring or developing) applications without involving or asking IT.

The skills gap is about meeting business needs in a world where IT does not have unlimited resources, but the business has almost insatiable demand for IT support.

That’s your gap.

SALESFORCE: How can citizen development help bridge that skills gap?

MICHAEL KRIGSMAN: CIOs should provide the services, the infrastructure, get the right governance in place — and then let your employees have at it. It’s beneficial for IT and helpful for employees who want to become citizen developers.

Why not offload a lot of the work of traditional IT on to the people who need it and know what they want? If you give them the right tools and they have the right skills, and they can do it themselves. They don’t have to talk to IT; they just go do it.

It’s less work for IT and lets the CIO do more with less, as I described earlier. It’s faster for the business, which increases speed and makes them happy.

Citizen development lets the CIO focus on infrastructure, services, enterprise architecture, security, and other areas that must remain the exclusive province of corporate IT.

SALESFORCE: How should companies define the governance around this new cadre of citizen developers?

MICHAEL KRIGSMAN: Make sure that your systems don’t allow users to bypass core governance standards.

For example, don’t allow them to expose data through the firewall unless it’s governed in the usual ways that the corporation allows. Give citizen developers access to certain types of data only, whatever is within their permission role or profile. Things like that. Only expose the services that are, again, appropriate to the organizational role and profile of the citizen developer.

Build governance into the system and then let users do whatever they want. That’s the mark of an efficient and innovative IT organization!

(Cross-posted @ ZDNet | Beyond IT Failure Blog)

Comcast: How customer experience drives product development

Customer experience is one of those buzzwords that has come to mean anything, everything, and yet nothing at all. Although hype-mongers and tricksters have co-opted customer experience, in truth, the concept is profoundly important.

At its heart, customer experience means all touchpoints or interactions between an organization or brand and its customers. It’s a simple concept that is fraught with complexity.

Think about the touchpoints that a typical consumer or business buyer may have when interacting with a seller through the entire lifecycle or customer journey:

  • First, they research the product and company, seeking information directly from the brand but also from reviewers, friends, and other sources.
  • Having decided on a product or service, the consumer evaluates where to buy and may choose among product variants and configurations. Obviously, the nature of the purchase, whether consumer or enterprise, for example, dictates specifics of the actual transaction flow.
  • Following the purchase, the customer may need post-sales help with setup and initial use.
  • Eventually, that consumer may seek technical support or other forms of customer service.
  • At some point, the consumer may make another purchase, beginning the cycle again, or, hopefully, remain with the brand as a repeat buyer.

While the brand can control some of these steps directly — such as product features or technical support — other aspects of this broad customer journey are fully in the hands of consumers. No brand can control, for example, the discussions that consumers have among themselves outside of official company channels. Through its actions, the brand can influence, but not control, these conversations.

Customer experience is challenging because so many points of interaction must come together to create a positive impression in the customer’s mind. Product design, engineering, marketing, sales, support, and service all contribute to a buyer’s overall experience with a company or brand.

To dive into the complexities of customer experience, I asked the seven-time author of books on customer experience and digital transformation, Anurag Harsh, to share his thoughts. Harsh is the chief marketing officer at IPsoft [disclosure: a CXOTalk underwriter] that develops cognitive AI technology for customer service. He also co-founded the large publishing company, Ziff Davis, giving him a broad perspective on these issues:

Customer experience is not just a touchpoint or a series of interactions between a customer and a company, but a voyage. I call it a voyage because customer experience demands re-wiring a company’s systems, employees, and culture towards the sole benefit of the customer.

Companies must learn to view the world from the customer’s point of view and deploy resources around the customer’s needs, to build a company culture that screams “customer first.” This task is hard, especially in companies with large numbers of employees, because everyone must evolve together, speaking the same company language.

In customer support, for example, creating the right experience goes beyond conversations between a customer and support agent. You need a customer-first culture that performs well behind the scenes at the back-end. All this directly affects the quality of the agent’s response back to the customer.

Great experience happens when a customer-first culture meets the right back-end processes and technology, supported at every step by the organization’s leadership.

Because customer experience is profoundly important, I invited two of the world’s top practitioners to join me on episode 267 of the CXOTalk series of conversations with innovators.

Chris Satchell is executive vice president and chief product officer at Comcast, where is he responsible for the company’s product, design, and innovation teams. Comcast has annual revenue over $80 billion.

Brian Solis is one of the most well-known authors and analysts on digital transformation. He is a principal analyst at Altimeter Group, a Prophet company. Among his reports are the 2017 State of Digital Transformation and the Digital Change Agent’s Manifesto.

See also: Comcast: How AI, machine learning, DevOps, and a bit of hardware may make it a smart home platform

During this episode of CXOTalk, these experts discuss the subtle points of customer experience and present a framework for thinking about the problem. The conversation describes how Comcast uses customer experience as a reference point when developing new products and services.

Watch the entire conversation embedded at the top of this page and read a complete transcript at the CXOTalk site.

Here are edited highlights from the discussion:

What are the core issues around customer experience?

Brian Solis: If you look at the proper definition of customer experience, or employee experience for that matter, it is the sum of all engagements someone has with your organization throughout the entire journey and throughout the lifecycle. It’s not just about any one moment. It’s about how all those moments come together.

Chris Satchell: It’s about designing the entire journey. There’s a couple of things we focused on [when I worked] at Nike.

One was this idea of consumer brand business. Do what’s right for the customer first; then worry about the brand and the brand promise you make to the customer; then worry about the business. If you get the first two right, the third will come.

The second one was about thinking about the entire journey. Every touch point on that journey is an interaction with the customer. That can be positive or negative so that you could be building promoters all the way along or detractors all the way along.

You must think very broadly, and so you think way beyond when you’ve got a product installed, or you’ve got it in your home. You have to think, “How did I learn about it? How did I acquire it? How did I pay for it? How did it get to me? How did I install it?”

At Nike, we would think all the way to, “What is my interaction with an in-store athlete that was serving me?” because that is a great connection point with the company. Every point along a journey is your brand. You have to be authentic, and you have to serve the customer correctly there. That’s some of the things we brought here.

Then from my time at Xbox, again it’s a lot about delivering the very best experience, not settling, and never being content with what you’re providing, no matter how good it is, because you have to think in this consumer world about how good you think your experience is. There is somebody out there merrily raising the bar on you. It won’t have to be in your sector. It doesn’t have to be in your industry.

Brian Solis: I wrote a book a couple of years ago called X: The Experience When Business Meets Design. It explained how to think differently about innovation by taking a step back and thinking about design for a new generation of customers and employees that are not in alignment with today’s corporate policies, processes, and even just how we think and how we think about productization.

The consumer doesn’t care about all of the politics and BS that happen within the organization. They just want the experience to be personalized. They want it to be great. They want it to be intuitive, maybe transparent in many ways. Innovation is as much about products as it is about policies, processes, and how we even work as well. I think the biggest thing is just shifting mindsets.

I look at today’s — I call them — Generation C. They’re not millennials. They’re not Centennials. They’re not Generation X. They’re just anybody who lives a digital lifestyle. What they all share is this heightened bar for expectations and these new behaviors. They’re impatient. For example, we talk about the uberization or the consumerization of technology. When someone uses Uber, that becomes their standard of engagement. When someone uses an Apple product, that’s their standard — or Google Search.

This new level of experience is blurring the line regardless of products or services that they want that same sort of intuition, that same sort of clarity and cleanliness throughout the entire journey. Yet, organizations are built on these 50-, 60-year-old structures that have all those things apart.

Chris Satchell: If I’m a customer, my reaction is, “Your org structure is not my problem.” We used to talk about this back at Xbox about trying to paper over the cracks in our org structure so that the consumer didn’t see. You’ll see so many companies when you track that product portfolio, it matches the org structure, and you’ve got to fight so hard to take that mentality out, and you’ve got to find leaders who will be selfless and say, “Okay. Yes, I have this release vehicle, but I’m going to take functionality from somewhere else. I’m going to give up something in my release vehicle because it doesn’t make sense to the customer.”

How do you empower teams to create great customer experiences?

One of the things that we’ve done that’s helped empower the teams — and it’s going to sound boring, but it’s so important — is we have this quarterly planning process. It’s how we take our annual goals for our portfolio and break it into quarters. Every quarter what we do is, the products managers, they get with all their stakeholders, wherever they are, including user research and what they want to do, and they write. They say, “For my area, here is a one-page spec of what I want to do,” and it’s something that can be achieved in a quarter for their end of the product or their product.

It says, “Here’s all the teams’ help I need.” What we do is we have this process where we stack rank them. Then we plan, and we just plan from top to bottom, making sure that any higher priority thing, you know, it fills resources in first.

There are 2,000 people in my organization and another 8,000 people we work with. What you don’t want to have happen is you start off on something and then find out one of the constituent teams can’t deal with the capacity constraints for that quarter, and you can’t deliver anything.

We solved that problem, but importantly, it gives all your partners somewhere to go. When they say mid-quarter, “We’d like to go in this direction,” or, “We want to change what’s happening,” you say, “Great. Talk to your product manager. If they like the idea, they can bring it to the next planning.”

It’s a way that we have managed to bring quarterly agility to annual planning. What we do is we only schedule 50 percent of our capacity that way. We call it “directed.” We do 50 percent of what we call “trusted capacity” where we just say to the scrum teams, “Hey, work your backlog. Put on your backlog what you know that you need, what the customer needs. That’s your capacity to manage. Go manage it.” We work very hard to carve off part of their capacity that they could just use to do the right thing. It’s taken us a year and a half of constant effort to get that to work, but it has helped us take the 36 teams that we’re feeding into video and make them more agile and coordinate across them. It’s agile writ large at a very big scale.

How do you measure ROI?

Here’s a controversial statement. I think it is pointless measuring ROI below the portfolio level for a given line of business. I sometimes have some very spirited discussions with our finance team around this. The reason is, we’ve got all these projects. They’re feeding into the overall experience the consumer gets. Then the consumer, especially in our business, has got a subscription they’re holding because of that.

When somebody comes to me and says, “Well, we need to know exactly what it costs,” I go, “Why do you need to know what it costs? You know what the portfolio costs.”

They’re like, “Well, so we can plan ROI.” I’m like, “How on earth do you know what the return is? There is no way to untangle these variables. That is impossible. It’s mathematically impossible. We don’t have that precision.”

And so, I think one of the problems is when people start measuring ROI. Measure it at an appropriate level. The level I think is appropriate is: Here’s what we invest in a business, and here’s what that business returns. If you start looking at features, and you start looking at product extensions and all these other things, and saying, “Well, we need an ROI,”

I think you’re missing the point in the modern world. You need to look at total investment, total return. That clears up a lot of the mess if you can convince people of that. I find that a lot of organizations love, would much prefer, to be precisely incorrect than right, because it gives them a sense of, “Well, we must be on it because we got all these detailed numbers.”

Well, the detailed numbers are fiction. We don’t know how the customer will receive it. How many of us see ROI projections that pan out?

Now, large-cap scale investment and capital investment, that’s a different matter. You can plan that. But, when it comes to consumer-based products, I just don’t think, other than the line of business, you can plan it. The first one is, if you can, don’t get caught in the game of ROI for small things. Talk about portfolio ROI.

Then what we measure depends. You’ve got your vanity stats because you want to know your population and what your monthly actives are and your unique users. But beyond that, you must measure, one, what you think is important. If you’re in a moment of truth, you need to measure success across a moment of truth. Maybe you need to measure net promoter score on one side then the other.

That means you must run experiments, take people through a new experience and measure what their net promoter score was at the end versus the net promoter score of people on the old path.

We have this idea of relationship net promoter score, so RNPS, which is the long-term [of] how you feel. Then TNPS, which is, through a transaction, how did you feel? Then other than that it’s, you’ve got to come back to the product teams. It’s like any good data science. KPI is no different. What question do you need to answer? You have to think about the questions you need to answer and then plan for the data to answer those questions.

From a development perspective, it’s great to put the infrastructure in to be able to say, “I want real-time stats. I want batch stats. I’ve got these different things that I want to get back from my application to make it very easy for developers to instrument.” Whereas, product comes in and says, “Could you find these things out for me?” They’re like, “Yeah, that’s easy. I can just go and add that.”

Beyond that, it depends [on] what you’re trying to answer for that question. If you’ve got a funnel problem with, “Hey, how do I track from when somebody downloads an application, how many people go through, set up an account, and they watch that first video and go to the second video?” That’s very different than saying, “I want to understand the heat map of how somebody moves through our user experience.” We’d say in England, “Horses for courses,” but it is about understanding the question; design your data feeds and your data analysis for the answer.

How does data help customer experience?

Chris Satchell: We have huge amounts of data on everything, whether it’s our products. You can only vaguely imagine how much data our network produces. We use it in many ways. We use it operationally to keep the service running, to give customers a great service. We also use it, as I said, to answer product questions, to understand where we should go next in our products.

We have a very strong machine learning, artificial intelligence, and deep learning set of core teams here. We’re using that data to recognize new insights in our products and also to

create new product experiences that you can only do with those intelligent methods.

The same with operations, feeding data in and looking for that sort of pattern matching recognition and next action recommendation that you can only do by using very deep networks to recognize all this data coming in.

We’re starting to use data as a way to change how we operate and as a core of how we build and the functionality our products deliver. I think that’s going to become common to many companies. Data will become part of the product.

We’re finding that the algorithms that are available are becoming a commodity. You can get great data algorithms everywhere.

The actual technology frameworks — whether it’s MXNet, whether it’s TensorFlow — analysis and modeling frameworks are becoming a commodity. The real thing you have as a company is your data. The models you build with that data, that is your secret sauce. That is your gold.

We’re very focused on using our data effectively. It’s a question of capacity. We have infinite amounts of great questions and things we can do. It’s just sequencing them through product development, through product insights, through network operations, and customer experience to get the most valuable things done first.

I think we always talk about big data. Now we’re talking about AI and machine learning, but all of that — let’s just remember, they’re just tools. Without great people thinking great ideas, without being able to develop it, without being able to take the insights or the data and have the actuation loop to affect things, there’s no point collecting it.

I used to joke that what would happen in the big data world, you’d have a board of directors that says, “We need data.” Dutifully, the company would go off and gather huge amounts of data. Then they would say, “Well, nothing is happening,” and so they go, “Ah, we need more data!” So you get even bigger data.

Then you realize a little bit later, you’ve got no insights from it, so you start building the insight engine. You have this, like, huge first bit, and then it narrows to insights. Then still nothing happens.

Everybody is scratching their heads, and then you realize actuation. There was no pathway to take the results we had and change the world based on that. You want it to look more like a pipe where your insights match your analysis match your ability to actuate.

There is also a cultural element where you need to check your ego and say, “Wow. I’m surprised. I had an insight. My insight was wrong, but I’ve got a new insight. Let’s go drive that.” If you can get those to line up, you can start making a change in the org.

Brian Solis: [Chris just described] the biggest challenge I’ve seen with data. This is across the board. The challenges for any of this are human.

You’re working against a lengthy career of experiences that are behind every executive or decision-maker. They got to that role of where they are because they’ve made great decisions along the way. Those decisions have fortified their experiences and have validated their beliefs and their perspectives. When you’re trying to challenge convention, data only reinforces what you want to see or what you expect to see.

Being a data storyteller and having common language [means] getting data to tell the story of what is happening based on assumptions that will challenge convention. That’s the art.

Final thoughts on innovation?

Brian Solis: I often talk about the difference between iteration and innovation. Many companies think they’re innovative, but they’re actually being iterative, which I describe as doing the same thing, but better, whereas innovation is doing new things that create new value.

I look at the Comcast or the Xfinity remote as sort of this metaphor for the two. Buttons are iterative: backlit keys, dedicated buttons. Then the voice, the whole infrastructure for voice was innovative.

Chris Satchell: It’s a continuum, so I think small iteration is just micro innovation. You need innovation that’s small. You need innovation that’s medium where you’re expanding products. You need innovations doing completely new things, and you have people dedicated across that time continuum.

CXOTalk brings together the most world’s top business and government leaders for in-depth conversations on digital disruption, AI, innovation, and related topics. Be sure to watch our many episodes! Thumbnail image Creative Commons from Pixabay.

(Cross-posted @ ZDNet | Beyond IT Failure Blog)