We recently hosted a webcast conversation between Beth and Adam focused on how to best use a testing dashboard, including overcoming common challenges. Don’t have time for the full webcast now? Below you can catch the highlights and tips from their talk in our companion blog.

Using Dashboards to Get the Most From Lean Startup Experiments

As your team iterates and innovates, testing dashboards are the tools that keep your work on track. They’re great for unblocking workflows, maintaining momentum, and focusing on what  Beth Sordi, Group Product Manager at Adobe, calls “the right tasks at the right time.” Or, if you want to think of them another way, testing dashboards are a bit like “an experiment board stand-up,” says Lean Startup Co. Faculty member Adam Berk.

Setting your testing dashboard up for success

Basically, these dashboards are a way to keep track of a team’s progress, says Beth, whether you’re using software such as Trello or physical cards. Cards represent the tests your team wants to run. “Hopefully, they’ve only got one or two tests that they’re working on at a time,” she adds. “You wanna really limit that work, or test, in progress.”

Adam adds that dashboards make the Lean Startup process more visual, malleable, and compartmentalized.

Teams prioritize their tests in order and create columns for various statuses of the work being done. “There’s a planned column, a blocked column, a build column, and then an in-flight column,” says Beth. And then finally there’s a learning column—which, Beth says, “after the test is run, there’s a step where the team needs to go back and confirm and validate what the assumption was what originally started off the test.” As a team, you want to move quickly through these tests so you can iterate upon what you’re learning.

Another bonus of using dashboards, says Adam, is that organizations are forced to have clear objectives. “If the board is set up correctly, it demands that you map it to one of the existing strategic priorities,” he says. “You can’t just leave that blank, so by definition, an experiment wouldn’t get run if it doesn’t at least have [strategic] elements.

“Another bonus of using dashboards, says @adamberk, is that organizations are forced to have clear objectives.” Click To Tweet

The goal is learning, not necessarily validation

There can be an expectation among teams that these cycles should lead to validation of assumptions if they’re working correctly. Adam describes that sort of thinking as having an optimism bias, instead of a learning bias. By shifting your bias toward learning, you can glean insights just from designing the experiment, he says, or from talking to customers, understanding who the stakeholders, seeing where you’re getting blocked, and learning how you get unblocked.

Start with cataloging the jobs to be done

There are a number of areas you want to learn from by using a dashboard, says Beth, including what to test next, identifying blockers, and understanding the why behind whatever product or project you’re working on. You’ll want to repeat your system of tests multiple times to enable as much insight as possible.

The work goes beyond the tests, though, she adds, and into making Lean Startup part of the cultural DNA of the organization. This means “staying organized and communicating to stakeholders, keeping our team accountable, [and] making our team’s progress visible.”

“There are a number of areas you want to learn from by using a dashboard, says Beth, including what to test next, identifying blockers, and understanding the “why” behind whatever project you’re working on.” Click To Tweet

Document everything

Getting the dashboard together is a big step, but there are other issues to consider as you move through the Lean Startup process. Adam lists off some of the common challenges he’s witnessed with the teams he’s helped over the years, such as a reluctance to commit to a date for a test or focusing on singular outcomes. Adam stresses the importance of getting teams to write things down—dates, assumptions— even if it means checking back in six months to find people were wrong about something. “We do want to give teams room to learn, evolve, and pivot, and not just create a robotic outputs,” he says.

Beth adds that when you don’t document your work as you go along, you can easily forget about a test you ran, say, six months ago. “All of the sudden you’ve got your brainstorming ideas and everyone looks around at each other like, ‘Oh didn’t we do that before?,’ and they have no reference,” she says. “It’s really good to be able to go back to the documentation or go back to the dashboard, and be like, ‘Yes, we did run this experiment, and here were the findings,’ and make an informed decision about whether we want to rerun the test or if we wanna do something different.”

“We want to give teams room to learn, evolve, and pivot, and not just create a robotic outputs.” Click To Tweet

Work with blocks instead of working around around them

Beth says she’s worked with teams that try to jump past blocks and work on different tests instead of doing the work to break through these important learning moments. This is where the cards can aid teams. By documenting everything, people are able to learn insights firsthand, broken down by experiment, team, and/or stakeholder.

As an example, Adam says, if one team hits a block, what if a different team runs the same test with the same stakeholder? “What did that team learn? That should never be used as the ultimate evidence, but it can be used to form hypotheses,” he says.

He adds that if a project is older than six months, it merits a reinvestigation. “Both from your own personal point of view, but also since environments shift,” he says.

The key is holding teams and individuals accountable. So you ask yourself, Adam explains, “‘Did this shift? Yes, great. Did it not shift? Why?’ Not getting anyone in trouble for that, but just really understanding why didn’t it shift and then making a secondary decision on do we unblock that? Do we give it an extra week or two? Will that make a difference? Or do we run something else, or do we test it a different way?”

Look for patterns

Dashboards can help teams discern patterns of blockers over time—the idea is to keep your eye out for them. “These patterns are permission based, resource based, they’re [from teams] waiting for a specific event to come…some conference that’s six months in the future,” says Adam. “Identifying the patterns can inform either how you go about unblocking them, or how you re-prioritize how you work in the first place.” He adds that the patterns are just as important as the individual blockers.

Patterns of blockers are just as important as the individual blockers and dashboards can help you discover and then watch for them. Click To Tweet

Testing dashboards are long term investments in the health of an organization

Over time, Adam says, there should be fewer and fewer days per year without an experiment in flight. This approach will “increase all of the other metrics that people really want to see,” he adds, such as new revenue from innovation, number of new projects and markets, and stock prices.

Repetition also helps insert the processes and learnings into a team’s DNA, so both what’s on the cards and who the stakeholders are become more clearly defined. “You’re not only going from problem to solution to scale,” Adam says. Teams also become more efficient with the experiments, leading to increased clarity and integrity in the learnings.

You just need to make sure you’re always approaching tests like a treasure hunter, Adam says. “You don’t know how everything is gonna match up,” he says, “so document everything… Don’t be afraid to sort of start from the beginning.”

When it comes to dashboards, repetition helps insert the processes and learnings into a team’s DNA, so both what’s on the cards and who the stakeholders are become more clearly defined. Click To Tweet

Thanks to Jennifer Maerz for contributing this piece. If you seek to bring the entrepreneurial spirit to your organization, Lean Startup Company can help.