Dealing with the Innovation Valley of Death (Part 2)

Developing the Thunderhorse field in the Gulf of Mexico was never going to be easy, it was a major technological challenge due to the depth of the find. Not only were the distances large, but the depths of the formations holding the hydrocarbons created pressures over 1,200 bars (120 MPa) and temperatures of 135 °C (275 °F). Prior to Thunder Horse, no field had ever been developed at such a depth. At the time the semi-submersible production facility was the largest in the world, the technical challenges were considerable.

Originally scheduled to produce oil in 2005 the field eventually started producing oil several years later in 2008. It suffered from various challenges, the most dramatic being an incident following Hurricane Dennis in 2005 after which the vessel was found severely listing. The company and industry held its breath whilst a recovery plan was implemented.

With this and other notable incidents at BP around the same time, its upstream projects leadership decided to introduce more robust project management processes across the business, which was still evolving following a series of mergers and acquisitions. The mix of company personnel and cultures had led to an assortment of approaches across the regions and so a lack of consistency. The intention of the new Major Projects Common Process was to develop a unified approach based on best practices to drive rigour and consistency of outcome.

A solution

When the Vice President for Engineering of BP’s major projects organisation came calling back in 2007 looking for help to develop the process, it was both flattering and a brilliant opportunity. I had just delivered BP’s Autonomous Underwater Vehicle programme and successfully carried out a series of trials leading to a full-scale offshore deployment that proved the technology was a commercial option. This was a notable achievement in an organisation which, not like any other large organisation, struggled to be nimble and exploit new technologies with success.

As well as creating the process and project governance, a series of supporting guidelines were to be introduced, one of which was in relation to Engineering and Quality Management. This document was structured to cover a series of topics and my initial brief was to develop guidance on the successful development and deployment of technology in BP’s upstream projects.

“Each project will therefore require a Technology Plan to ensure the true value of the chosen technology is realised” – Engineering and Quality Management Guideline 2007

I spent time looking at other industries, some years previously NASA had developed its own system for managing the introduction of technology based on Technology Readiness Levels (TRL). It turned out that automotive and aerospace sectors had adopted a similar approach and so it made absolute sense to do the same at BP. The definitions were tweaked slightly to suit, and the number of levels kept to 7, but it was essentially the same.

In addition, the maturity of the technology (or TRL) was also linked to the definition of the project. In simple terms, technologies were expected to be closer to maturity the later the project phase and closer to delivery. This was intended to force more rigour, earlier. Finally, with TRLs at the core of the new guidance, narrative was added around them to help with interpretation and use on projects. The new guidance was well received by the leadership of the Projects Organisation and we all felt confident that it would be helpful to the wider organisation.


The first engineering standards were created and used in the industrial revolution. The need for high precision machine tools and interchangeable parts were the catalyst. The British Standard Whitworth thread was adopted by many companies as the first (non-official) standard in 1841. Today, standards cover just about every topic imaginable.

Where they are used processes, standards and guides help drive consistency of outcome. That is their primary goal and reason for their creation. They help to make things more predictable, efficient (at least most of the time) and safer. So, when the time came to implement the new material in the business, I felt there was no reason to think that the BP’s new upstream Technology Management guidance would be any different. However, I completely under-estimated the capacity of people to work around, avoid or modify the process I’d worked so hard to make clear, effective and valued.

  • One senior engineering manager decided that the new company process on Technology was wrong and wrote an alternative version for his project (similar in scale to Thunderhorse).
  • Another ‘sub-contracted’ the application of the process to suppliers, so handing over accountability and creating a bizarre situation where suppliers were being to ask to assess and manage the introduction of their own technology on a BP led project.
  • The biggest surprise of all though was the was strict interpretation and application of the process, so stifling the uptake of innovation and technology.

Engineers are typically structured in their thinking, logical, rational and often conservative. All fine strengths in an industry like oil and gas where the stakes can be high and safety needs to be paramount. They are also particularly good at writing down processes and following them, often to the letter. Bearing this in mind, it became clear within a few months that new technologies, instead of being successfully introduced to and managed in projects, were being squeezed out.

This was usually on the grounds of immaturity and being at the ‘wrong TRL’ for the stage of the project. Instead of the guidance being used to help manage the introduction of innovation, it was becoming responsible for preventing its uptake in the first place. Of course, the challenges and failures BP had seen on previous projects largely disappeared as a result, so in one sense the guidance was a success! However, they disappeared because innovation and technology were being avoided altogether. Standard, ‘low risk’, off the shelf solutions were being pursued instead. Hardly setting the business up for future growth beyond the basics.


To survive every business must keep moving, to improve and embrace new ways of doing things, implement new technologies. There was no intent (or wording) in the guide to drive a user to eliminate use of new technology, only a desire to bring some structure and consistency to the way they were being selected and implemented. As it turned out, a process that was introduced to help successfully manage technology and benefit the business became a barrier instead.

“Only two things are infinite, the universe and human stupidity, and I’m not sure about the former” – Albert Einstein

It all came down to how the user chose to use the guidance. I say chose but, rather than considering the wider business goals, I suspect most people stopped thinking. They defaulted to inherent risk aversion and simply applied the guidance to the letter. There is no doubt that it hampered innovation in BP’s projects and pushed some technologies into the Innovation Valley of Death. It probably increased costs in the long run and certainly hindered the ability of projects to use beneficial technologies.

A small example from my local shop can illustrate what happens when a business doesn’t evolve. Greg and Jerry have been in business thirty years and until 2018 had only taken cash payments for their fruit, vegetables and grocery products. They are conservative, risk averse and liked dealing in with straight forward cash. It had worked well for a long time and they were resistant to moving to electronic transactions.

Eventually, the change in their suppliers’ banking habits forced Greg and Jerry to introduce card and electronic payments in their shop. To their delight and surprise, it increased turnover. Suddenly, established customers started spending a bit more per visit and by passers that called in without cash were able to purchase items. So, by introducing innovation in their business, there was a commercial benefit. It could have been realised much sooner though and ironically by trying to minimise risk (unknown and untrusted technology), the shop owners introduced more risk (reduced turnover).

Which brings me back to the point about processes, procedures, guidance and standards in the context of introducing technology and innovation. They must be used intelligently otherwise eventually they won’t work at all.

If we focus on the positive objective of ensuring that a project or business is adopting and managing beneficial technologies, then strong and relevant processes help. Failure to focus on this means we fall victim to ‘blind compliance’, things get stuck, dogma arises, nothing changes or improves, companies become less competitive. Innovation stops.

Next time

What would I do next time? How would I introduce a constructive process that would help the company and be applied in a positive, valuable way?
That’s easy, I’d spend more time on the people. You see once that original guidance had been written there was simply a series of ‘roadshows’ to introduce the new approach to projects and their personnel. The events were one-way ‘show and tells’. We ‘ticked a box’ but didn’t ensure understanding and appreciation. We threw people the keys and wished them good luck, we didn’t engage hearts and minds.

Aristotle’s Theory of Rhetoric is still valuable today. He concluded that to make a compelling case for change and to influence people, beyond introducing the theme (e.g. improve technology implementation in BP) there should be three components to an argument:

  1. Logos – A logical structure
  2. Ethos – Credibility and trust
  3. Pathos – Emotional connection

Looking back, we addressed the first two but not the third. If we had made the emotional connection, engaged the group in the business’s aspiration, the application of the new process would have been more rounded, considered and would have delivered what was intended; a consistent and successful introduction of innovation and technology to major projects.

Innovation lessons

What specific lessons can we take regarding innovation and avoiding the Valley of Death?

  1. Many industries use processes and standards to help manage the introduction of innovation and technology. Be aware that they can help or hinder innovation, the direction can be dictated by the person (or company) applying them.
  2. Processes, standards and guides can also introduce new risks because they can stop us from thinking. If we are clear about the innovation objectives and are committed to them, then any processes used should help us get there effectively if they are used at all.
  3. People are crucial in the successful introduction of innovation. An innovation or technology management process will not overcome innate technology or risk aversion, so remember to engage hearts and minds too.

Share this post