Editor’s Note: This blog post originally appeared as an issue of Dennis Lendrem’s LinkedIn newsletter, Apes in Lab Coats. The origin story can be found in the first issue. In each issue, Lendrem walks the reader through different situations he observed “in the wild” – from the perspective of both an expert in experimental design and a trained ethologist. His subject of choice: the homo sapiens in general, and in particular scientists (who are often spotted wearing lab coats).

Designed Advantage

I met with Bob, the President of the company, and thanked him for taking time out from his busy schedule. Bob confirmed he had a busy schedule and apologised that he’d only be able to stay for half an hour. I reassured him. Simply by pitching up and saying a few words, he was signalling to his staff: This is important.

Bob opened the meeting, stressed the importance of DOE, then retired to the back of the room with his colleague Rob and took a seat. Throughout the workshop, a steady flow of people slipped into the room for Bob to sign documents and then slipped away again.

Meanwhile, my colleague – the very brilliant Paul Nelson – launched into the first process simulation workshop: The Humbling Exercise.

And, as per usual, we watched as Bob’s brightest and best floundered to solve what should have been a straightforward problem.

In the debrief, we discovered that all five groups had come up with different ‘solutions’ to the problem. None of them correct. Worse still, none of the five groups could tell us with any confidence what effect changing each of the process variables would have on the outcome. They even disagreed about the direction of change.

Bob’s brightest and best had come up with the wrong answers, learned virtually nothing about the system they were exploring, and burned varying amounts of resource in the process.

Bob leaned over and whispered to his minder, Rob. Rob jumped up and disappeared out of the door. The steady flow of people at the back of the room stopped.

Bob had engaged.

Paul then stepped them through a designed approach. The teams went away re-ran the process, represented the results.

Their maps of the space were projected side-by-side on the screen.

For an up-front quantifiable amount of resource, all five groups had generated pretty-much the same maps and arrived at almost identical solutions complete with robust operating window for the process.

Ta-dah!

At the end of the debrief, I asked Bob for his take.

How did this sit with his experience of leading an R&D organization?

Bob hesitated, then admitted that this was a window into his world. He opened up. He shared the challenges of managing pharmaceutical R&D:

• The projects that drifted

• The solutions always just around the corner

• The issues that got ‘resolved’ only to resurface later

• The resources burned ‘solving’ the wrong problem

• The timelines full of uncertainty

• The prospects ultimately canned

Put yourself in Bob's shoes.

As President of an R&D organization what are you going to prefer?

• Scientific foraging which consumes an ill-defined amount of resource to sometimes yield a quick solution, but more often yields a product or process that will fall down later in the R&D process. Or,

• A designed approach, with a robust solution, charting the space, with predictable up-front quantifiable resources.

It's a no-brainer.

R&D leadership recognize Designed Advantage.

Why throw yourself to the mercy of the scientific foragers.

A design-led approach makes the entire R&D process more manageable.

The outcomes are more reliable.

And the resource, budgeting and staffing is more predictable.

An End to Scientific Foraging

In Lost in Space we observed the search paths of scientists moving through a 12-D space in the Virtual PCR Simulator. This captured movements of scientists in multidimensional space as they attempt to solve a ubiquitous laboratory task - DNA amplification through the Polymerase Chain Reaction. We noted that most scientists use their scientific insight to perform area-restricted searching around a likely region of the space. They then make short excursions, or Levy flights, from that point in the hope of finding better, more productive regions. Eventually they run out of time or other resources and end up running with their best results to date. And the Paradox of the Self-Fulfilling Prophecy means that these 'solutions' are often close to their original starting point reinforcing their scientific intuition.

However, unlike designed experiments that generate maps of these multidimensional spaces, scientific foraging and other low-dimensional search tools such as One-Factor-At-A-Time leave us with little Intelligence about the multidimensional space of interest. As a result, any foraging solution is often unstable and likely to fall over at the next step in research and development.

Scientists grossly underestimate the resources needed to arrive at the wrong answer.

In addition we learned that scientists often grossly underestimate the resources required to arrive at the wrong answer using traditional scientific foraging methods. Moreover, the amount of time or resource needed to solve a scientific issue is massively variable. From an R&D management point of view this is a nightmare.

Scientific foraging may alight on a reasonable solution quite quickly, but at other times a solution is never found even after running many experiments. More often than not, the R&D team simply runs out of time and is forced to ship the existing product or process knowing that it represents a highly unstable solution. Those downstream in the process pick up the pieces - troubleshooting and fire-fighting glitchy products and processes at the next translational step.

It's time to call an end to Scientific Foraging.

The Momentum Trap

There is a defining tension in industrial R&D between the pressure to show progress - to get through the next gate and move on to the next phase - versus the need for robustness - something that will survive in the wild.

When R&D mistakes movement for momentum, the illusion of progress becomes the enemy of real progress. Progress for progress’s sake. A trap set by pressure, ego, or poor metrics. In The Momentum Trap, velocity replaces validation - and truth does not survive the impact.

When this tension isn't managed properly, we get fragile solutions that collapse shortly after the gate review. Boxes ticked, but bombs planted.

Consciously, or unconsciously, most R&D middle managers wrestle with this tension.

On the one hand they might have a deadline to meet. They want to push a project out of the door. On the other, they don't want to see it coming back on them. Once it's gone, it's gone.

Sorry sunshine, not my problem.

DO NOT try to sell DOE tools to this group on the basis of optimization.

They are not interested in optimizing assays.

They are not interested in optimizing products and processes.

They are not there to optimize anything.

Optimization is a dirty word.

Fortunately, DOE is there to manage the development process - to "get stuff out the door." By leveraging expert and domain knowledge, designed experiments allow us, for a quantifiable amount of resource, to screen a brainstormed list of potential factors, and identify settings for that product or process that are relatively robust to variation in those critical factors. Bingo!

We can get it out the damn door.

Don’t let your scientific intuition lead you astray.
Watch Dennis Lendrem’s on-demand webinar—no registration required—to discover how to move beyond intuition and drive lasting change with the right tools.

Watch now

About the author