Lean Experimentation

If you are hoping to use lean product management practices, you will need to understand how to manage lean experimentation. Experimentation is an iterative process which allows product teams to test variations of a new feature based on customer feedback.

Having a constant feedback loop with your customers is important – after all, you can’t expect to be right from the get go. By testing a variation of a new feature, your product team can see if it is improving the overall customer experience.

If it is, it can then support a hypothesis for a larger more impactful change. If it isn’t, you have the opportunity to learn and go back to the drawing board, perhaps coming up with a different experiment.

Experimentation feeds nicely into continuous discovery as a process – you experiment on something small, gather some feedback, learn from it, and improve the opportunity.

By experimenting on variations of a feature, it doesn’t mean you need to rely on your development team to code something up for you. You can start a variation of that experiment by finding a low-risk, low-effort way of running the experiment, like for example testing with forms and emails before you implement something custom within your product. This will help support the hypothesis as to whether or not involving your development team in something more in-depth is still worth looking at. 

Build, Measure, Learn

At the core of Lean experimentation is the Build –> Measure –> Learn concept, and out of this comes the experiments that are set up and tested in order to try to hit outcomes. Build –> Measure –> Learn is a bit of a misnomer though, as it doesn’t really start with Build. It’s a cycle, and it really starts with measuring and learning, before anything is built, and follows with more measuring and learning. 

Experiments are like bets

The term bets works really well in this scenario, as it takes the pressure off individuals to be right, and provides the freedom to have lots of different bets. Every day, we all have our own insights and opinions on what’s going to work and what’s not. As an experimentation-driven team, it’s possible to see them through to a conclusion to decide which bets landed and which ones didn’t. 

Some bets are going to be wrong. And that’s okay! It wasn’t a person that was wrong. It was only a bet, and now you know what doesn’t work, and you can move on to the next thing. You might even have other bets out on the same initiative, and that’s cool too. You win some, and you lose some. 

What’s really happening behind the scenes is the team is getting smarter and smarter with each pass or fail, each lesson learned, and the product and service provided is getting better and better. Double down on the bets that win and revert the ones that didn’t.

Experiments can be many things

When talking about ideation, it’s common to think of new feature ideas, and the roadmap quickly becomes a list of new features. A roadmap that’s a list of features will only ever deliver you a bunch of new features, at best, and everyone knows that a bunch of features doesn’t mean it’s a better product. 

Instead, it’s better to think of ideas as lean experiments. Sometimes, the experiments on the roadmap will be about features. For example, adding a feature could be a great way to solve a problem (in ProdPad we define these as Roadmap Initiatives) and hit an Objective. Sometimes the experiment might be to improve a feature or even remove a feature. These are great ways to solve certain problems, particularly around tech debt or usability. 

But experiments can be so much more than just feature-related. They can be things like pricing experiments, or changes to positioning or packaging. These are all things that can be tested in order to try to solve problems and hit the team’s goals.  

Sometimes experiments are huge, though many are going to be tiny. That’s good – little increments are healthy and exactly what are needed in many cases. Updating a bit of UI or adding some copy to a page are both valid and trackable experiments that can be learned from.

Experiments are scientific

Think back to high school science classes. You probably remember some of the stuff you had to do before you set your bunsen burner alight. Start with a hypothesis. What do you think will happen as a result of this experiment? Write down how you plan on checking if this experiment was a success or a failure. 

Once the smoke has cleared, what was the outcome? Did your experiment set on fire in a good way or a bad way? What were the actions that came out of it? 

Lean product management is just the same concept! The way that this scientific process has been adopted by companies has been transformative. After all, science is progress. The scientific method bakes in continuous learning, even when experiments fail.

And experiments will fail. That’s totally expected and okay. See it as a lesson learned. You just learned 5 ways of not creating gold from alchemy, and that’s great. Now we know more as a team and we can focus on the things that likely will work. More important than success is simply seeing experiments through to the point that there is learning as a result.