Faster, Smarter Product Development – The Power of MVP Testing

What is MVP testing?

A Google search on MVPs and how to test them yields A LOT of disinformation. In this post, we’re going to head back to first principles and cut out all the fat.

Our Lean Startup Co. Founder Eric Ries defines an MVP as “a version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort”.

Validated learning is insight into what creates value for customers that’s based on experimental evidence instead of guessing or just asking customers what they want. If your team is currently uncertain about who your customers are and what they want, then validated learning is your team’s essential unit of progress.

The kind of MVP your team should choose to test is therefore a function of what your team needs to learn next about what creates value for customers. For this reason, MVPs range in complexity from extremely simple (napkin sketches, brochures, explainer videos, etc.) to more advanced (prototypes, early working versions of the product, pilots, etc.).

That’s right – contrary to popular belief, an MVP need not be a working version of the product.

To test an MVP, it’s delivered to customers along with a specific Call To Action (CTA) that serves as the mechanism for achieving validated learning.

Here’s an example of a complete napkin sketch MVP test:

In case it’s not obvious, the MVP above isn’t a working version of the product.

Why is MVP testing important?

Has your company ever spent millions of dollars and years of effort to build a product that customers don’t want? I’m going to guess your answer is “yes”, and not because your company is unusual. In our work at Lean Startup Co. with companies of all shapes and sizes, we’ve yet to meet a single company that doesn’t have at least one horror story about wasted time and effort.

As just one example, we’re currently supporting several clients who are developing AI-powered products. Prior to our work with one of them, one of their AI product teams spent ten months building a product, only to find after launching it that customers weren’t even willing to try it.

If the learning goal was to understand if customers were interested enough to try the product, the right MVP to build would have been something like a simple landing page with an explainer video. This would have allowed the team to measure demand for the product in less than a week instead of 10 months, eliminating the wasted time and money.

As Lean practitioners, we should always be asking ourselves which activities create value and which are a form of waste. When teams are uncertain about their answers to key questions such as “who are our customers?”, “what problem should we solve for them?”, and “what offering creates value for them?”, any activity not related to learning the answers to these questions is a waste of time.

How do you run an MVP test?

Here’s a simple 5-step process for running an MVP test.

1. Identify what your team needs to learn next. Be specific and avoid the temptation to try to learn everything at once. In general, prioritize learning about who your customers are, what problem(s) are impactful to solve for them, and if your solution creates value for them. In the AI product team example, what the team should have focused on next was determining if their product’s basic value proposition was of interest to customers.

2. Pick a Call to Action (CTA) and set the bar. What CTA would convince your team and your investors (potentially corporate executives for you intrapreneurs out there) that your learning has indeed been validated? There are an infinite number of CTAs, and picking one is a blend of art and science. Your choice should always be directly related to where you are in the development phase and how much prior evidence you have. That being said, common CTAs we see when testing MVPs include “download”, “buy now”, “reserve now”, “sign up for product updates”, “attend a meeting to learn more”, and “sign up to be an alpha user”.

Once you’ve picked your CTA, don’t stop there! Next, set the bar. Of the total number of customers that will interact with your MVP, specify precisely how many would need to take the CTA in order for your learning to be validated. In the AI product team example, the team could have set the CTA as “clicking the Try Now button” and set the bar at “40 out of 100 customers will click on the Try Now button”.

3. Pick an MVP. With your CTA chosen and the bar set, now it’s time to pick an MVP that will generate your CTA. Do this by asking yourself what type of MVP would be best-suited for generating your CTA with the least amount of effort. If your CTA is “schedule another call with us to learn more” then even a napkin sketch of your solution might be the best MVP. If your CTA is “give us $1000 to reserve a spot in our pilot program”, then you’ll need a more sophisticated MVP. Note that “pick an MVP” is step three here, not step one. Since the point of an MVP is to collect validated learning about customers with the least effort, it’s not possible to pick an MVP if you don’t first define what you seek to learn or without having a specific CTA defined.

In the AI product team example, the team could have picked an explainer video as their MVP, placing it on a landing page with the Try Now button next to it. Alternatively, it could have sent the explainer video out to customers via email and included a Try Now link.

4. Run the MVP test. Don’t forget this part! It’s easy to get nervous and find reasons to delay testing your MVP. After all, what happens if your test leads to unexpected results? We have news for you: your MVP tests will almost ALWAYS lead to unexpected results, especially early on. But that’s the whole point. Coming back to Lean principles, when would you rather find out that your assumptions are wrong? Now, or 10 months from now after you’ve wasted your team’s time, money, and engagement building something customers don’t want?

In the AI product team example, the team would have discovered that zero out of 100 customers took their “try now” call to action, well below the bar of 40 they set for themselves in advance. And that would have changed their next steps.

5. Repeat. Take what you learn from your MVP test and go back to step 1 above. What do you need to learn next?

In the AI product team example, the team would need to learn why nobody chose to try their product.

“The goal of your first MVP test is to begin the process of learning, not end it.”

Remember: you should repeat the MVP cycle as many times as you need to until you’re confident in the validated learning you have unlocked about what creates value for customers. Lean Startup is a process, not a one time effort. Go forth and build!

Bonus material:
Let’s put your skills to the test. How might you pick an MVP for a 30-ton power generator? If you answered “it depends on what the team needs to learn next”, you are correct. Check out this post to find out how one company did it: There’s an MVP for That.

Interested in testing MVPs at your company? Learn more about Lean Startup Co.’s resources and services here.