Article: How to break free from the infinite loop of pilotitis

22 July 2020

Matt Capelli, July 2020 

The concept of a pilot test (i.e. pilots) shows up everywhere in the development sector. Pilots are a powerful way to design new solutions by testing hypotheses and validating what is actually working. In fact, running pilots is so popular that there is even a term, pilotitis, for piloting too much without ever actually implementing a long-term solution. 

I’m an unwavering advocate of pilots, and I believe they are an essential component in the process of implementing technology. Pilots are goldmines of innovation when structured correctly. But if I’m being honest, I’ve often found myself gravitating toward a state of pilotitis, either driven by a tendency for perfection or because pilots are just more fun. They are also low risk. Put bluntly, pilots usually let us “check a box” that shows a stakeholder that we tried, without requiring us to successfully navigate the difficult journey of implementing long-term, sustainable solutions.

To understand how to escape pilotitis, let’s first define what a pilot is within the context of technology. A pilot can be thought of as an experiment you conduct to determine whether a new technology solution will work. Like any proper experiment, you want to test solutions in a low risk environment that replicates your day-to-day operations. For agribusinesses, this usually means you are trying new technologies in specific geographical areas that limit the number of smallholders affected, or other experimental areas of operations, such as demo plots.

Pilot tests should be set up so they are fast, cheap, and easy to manage. The best way to do this is through the build-measure-learn loop. Each cycle through this loop will include a small change to the technology or the way the technology is used (the build phase) followed by an observation of if this change made a difference (the measure and learn phases). But not just any difference! We want our pilots to be centered around producing outcomes which are tied directly back to the business. In short, we aren’t testing things for the love of science or for the sake of learning. We are testing technology so we can race toward making our business better as fast as possible.

To make sure we are not just spinning around learning for the sake of learning, thoughtfully structured pilots rely on key metrics. Key metrics are simple and objective numbers that provide you with insight into whether the changes to the technology solution are solving your key challenges and improving your business. As you learn what works and what does not work, you will quickly improve the way in which you are using and implementing technologies, ultimately resulting in scalable tech that is perfectly aligned with your goals. 

Hold on! I’ve never seen it work like that!

If you are saying this to yourself, you are correct. I’ve never seen it work like that either. Implementing the build-measure-learn loop in practice is daunting because learnings and their associated benefits are not always clear-cut. Testing technology is about diving into the unknown - it’s far from a perfect process and rarely will produce a perfect solution. If we fixate on finding the perfect solution and removing all unknowns, we’ll be left paralyzed and unable to move forward, perpetually stuck in a state of pilotitis.

How do you know when something is working “good enough” and it’s time to push forward? The secret lies not in the technology itself, but rather in two key components we’ve missed up until this point: 1) what makes sense to our users and 2) our business’s bottom line. If we add these two items into our build-measure-learn loop, we’ll know when to break free. 

You will know it’s time to scale when three things happen:

  1. Your end users are using the technology without being forced to (it is providing your users enough of a benefit that they like it)
  2. You can clearly articulate and document the required solution
  3. The solution has proven to be beneficial and valuable for the company, providing you with a positive ROI. 

If you have achieved the above, break free! The tech might not be perfect at this point. There may still be unresolved challenges in your business and your operations. You might have to follow the 80/20 rule. But whether it’s perfect or not, you will at this point have in your hands a technology solution that is proven to be in the best interest of your business. It’s time to move on and up!

Originally posted at Simple Data Tools.

AgDevCo’s Smallholder Development Unit is supported by the Mastercard Foundation and UK aid.