If you want to get a competitive edge, building a culture of experimentation is critical. It can result in cost savings, higher revenue, and improved user experience.
People have been building scientific and technological innovations through testing and experimentation for centuries.
Eratosthenes measured the circumference of Earth 2,000 years ago.
Newton's prism experiment proved color is an inherent property of light itself.
Marie Curie, using an instrument called electrometer, found that radioactivity is an inherent property of individual atoms that emanates from their internal structure.
These experiments have paved the way for modern medicine, geography, energy, communication, physics, etc.
In today's technological world, experimentation holds the same power when it comes to innovation.
Some of the biggest companies, like Amazon, Netflix, and Google, use this scientific method to challenge the status quo. In return, they innovate, grow, and improve.
Amazon CEO, Jeff Bezos, has said: “Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day.”
Mark Zuckerberg has reported that the company conducts thousands of experiments at any given time.
As far back as 2008, Google was already leveraging its search engine users to simultaneously conduct 50–200 experiments.
The end goal of this "test everything" mentality? To roll out new products, get past the competition, improve customer experience, and try new business models.
Why an Experimentation Culture is Valuable
Only a decade ago, Yahoo was on a par with Google. The two tech giants were competing fiercely to conquer the territory of the World Wide Web. But Google's ferocious experimentation led the company on a path of continuous improvement. Today, Google has a market share of 92.17%, Bing has 2.78%, and Yahoo has 1.6%.
But not all experiments yield successful innovations.
At Google and Bing, about 10% to 20% of controlled experiments have positive results. One-third of experiments at Microsoft prove effective, one-third have neutral results, and one-third have negative results.
But that capability to test out everything to see what works and what doesn't has given these companies a competitive edge.
Experimentation has helped Bing make dozens of monthly improvements, which collectively have boosted revenue per search by 10% to 25% a year.
When Facebook tested a secondary "Explore" Feed to its primary News Feed in six countries that let users discover content that they haven't explored yet, it resulted in a 60-80% engagement rate.
To sum up: although experimentation may scare you with its tiny success rate, the benefits can help a company iterate quickly, fail fast, and pivot.
How to Build an Experimentation Culture
Successful innovation requires building a culture of experimentation at a company-level. A culture of experimentation involves nurturing employees' curiosity, listening to data, allowing everyone to conduct or commission a test, and managers accepting a new model of leadership.
A Culture of Experimentation Starts With Accepting Failure
"Failure and invention are inseparable twins." It's what Jeff Bezos says. And he's a smart man.
Building a culture of experimentation starts with the engineering manager accepting that failure will happen, but that shouldn't be an obstacle to continue experimenting.
Instead, failure should be viewed as an inherent and integral part of the learning process.
When leaders accept it as such, it will send a message to the entire company that failure isn't something to be dreaded or scared of. It means the company is learning, growing, and with each experiment, it's closer to achieving its goals.
Everyone involved will feel more comfortable giving feedback, sharing ideas, and brainstorming. Creative minds at the company will be unleashed to try big, new, unproven ideas, and at the same time, business executives will get guaranteed results fast.
Small Changes Can Generate Huge Benefits
We'll use one of Microsoft's real-life stories as an example.
In 2008, an employee suggested having a new tab automatically open when a user clicks on the Hotmail link, instead of opening in the same tab.
A seemingly small change, but when it was tested on 900,000 users in the UK, the engagement of users who opened Hotmail increased by a whopping 8.9%. In 2010, the company introduced the change to 2.7 million users in the US, and it yielded the same results.
Amazon did a similar thing when it moved its credit card offers from the home page to the shopping cart page. The result? Profits increased by tens of millions of dollars annually.
The lesson in these stories is that there's value in introducing a small change quickly, to a small set of users, in order for your engineers to learn whether you're building the right thing.
Starting small is a better option than polishing a large feature, introducing it to a large user base, only to hear a few weeks or months later that the business goals have not been achieved or that the key metric hasn't changed.
More Experiments Are Better Than Few
Many companies today run tens of thousands of online experiments every year. Out of those tens of thousands of experiments, only a few will be successful.
Although it sounds expensive, it's actually riskier to run only a few experiments per year than a thousand.
When you conduct a large number of experiments, a few will be successful. The success of those few experiments will make the financial costs of a string of failed experiments irrelevant.
On the other hand, if you run only a few experiments annually and none of them prove successful, you'll be left dealing with a substantial financial loss.
Failing is useful because it allows software developers to eliminate unfavorable ideas and focus their attention on more-promising solutions.
Rely on Data, Not on Strong Opinions
Every company has people working there, and people have strong opinions. When results from an experiment clash with strong opinions by a team or employee, make sure the empirical results of your online experiments prevail.
The tale of Ron Johnson when he became the CEO of JC Penney is another great story that carries a lesson.
When Johnson joined the company in 2011, he had a bold new plan in mind: he would use technology to eliminate the need for cashiers, get rid of coupons and clearance racks, and fill the stores with branded pieces.
Almost 17 months later, sales had decreased, significant losses were experienced, and ultimately, Johnson had lost his job.
What happened?
Unfortunately, many managers in the business realm lack sufficient data or relevant experience to lead their business decisions. Many managers also rely more on their intuition than on numbers.
If the experiment tells you that you should remove the green button, you remove the green button.
This doesn't mean that managers shouldn't be listened to on all occasions. When it comes to making strategic decisions that can't be tested, managers' input holds value.
But all things that can be tested in a company should be tested. Experiments can guide managers' decisions and fuel healthy debates. Even if those debates end up in overruling the data, at least the company is conscious of the experiments' results and is making a move that's in the company's best interest.
Keep the Feedback Coming
Giving feedback to your software developers can inspire them to do future tasks even better and with higher motivation. It can engage an underperforming individual to improve his/her performance to a higher level.
Feedback is also the only way to ensure an employee doesn't continue making the same mistakes.
The majority of employees are interested in hearing feedback from their managers; however, what they crave for is specific feedback.
Specific feedback is telling your engineer what they're doing right and where there's room for improvement.
For example, instead of saying to your JavaScript developer, "You could be more proactive," say, "You said you would come up with some ideas for overcoming the issue with technical debt, but I still haven’t received anything. Is there some problem?"
One great way of giving feedback is the SBI (Situational, Behavior, Impact) Framework.
The SBI framework outlines a simple structure that you can use to give feedback:
- Situation: define the where and the when of the situation you're referring to.
- Behavior: describe the specific behavior of the employee in the situation you're referring to.
- Impact: say how the person's behavior has impacted you or others.
In the long run, this can lead to higher performance, unlike less specific feedback that just informs employees of whether they are performing well.
Finally, Celebrate Your Wins and Learn From Your Failures
When an experiment yields a successful innovation, celebrate! Celebrating your successes and giving credit where it is due will motivate others to become a part of the culture of experimentation at your company.
And when you fail, learn from it! Even if it's a failed experiment, failures can steer you in the right direction when you lack answers or people have conflicting opinions about an idea.
