Businesses have to juggle a variety of competing priorities when planning IT projects. Guaranteeing robust levels of quality and performance, achieving a final product that is attractive and intuitive to use and completing development and launch with minimum disruption to current operations all rank highly.
There is also a strong incentive for completing projects as quickly as possible, or at least getting new applications, platforms and systems up and running without delay. In the past, this objective was often sacrificed to the demands of quality, usability and minimal disruption. Development would take place in the background, away from the day-to-day workings of the business so there would be no interference, with the final product only rolled out on completion, potentially after many months of work.
The frustration for businesses was that this approach often proved expensive, with considerable resources diverted away from front line operations for extended lengths of time, and also delayed any value benefits the organisation could get from the new product.
This is one of the core issues which Lean, one of the Agile family of development methodologies, attempts to resolve. According to Lean, quality and usability can be combined with much faster roll out, giving a business the benefits of a new product sooner, without compromising current operations.
The key to this approach is the concept of the Minimal Viable Product (MVP), the most basic iteration of the final desired product that is still capable of delivering the core intended functions. An MVP can be launched in real-life environments in a fraction of the time it takes to complete a full development. The idea is that the company can be reaping some of the value from their new product in a matter of weeks, and can then refine and add to it in production, ultimately leading to a final version that is more carefully honed to actual use and operations.
This progression from MVP to final product, through a series of iterative cycles typical of all Agile methodologies, relies on the close integration of testing with development. Testing is used to assess the impact of added and refined features in parallel to a live system, with the best options able to be deployed very quickly to ensure highly efficient progress. There are numerous types of testing methodology used in Lean development. In this article, we will explore one of them – A/B testing.
Lean Start-ups and MVP
First, a little more background on the concept of MVP, where it comes from, its aims and its benefits. The origins of MVP can be traced back to Eric Ries’s influential book The Lean Startup, which sought to apply the principles of lean manufacturing, with its focus on eliminating all waste and inefficiency from production cycles, to product development for business start-ups.
The idea is that entrepreneurs looking to make waves in a new market face certain structural disadvantages compared to incumbents, which are often larger, well-established businesses. Working with minimal resources, startups need to be as sure as possible that their product will attract interest, and they need to make a virtue of their smaller size by showing greater agility in how they respond to market demands.
Ries proposed a model for how startups could achieve these objectives known as the Build-Measure-Learn loop. And it was at the ‘Build’ stage that Ries outlined the concept of a minimal viable product. He argued that startups were better off getting a version of their product to market as soon as possible, giving them a chance to grab market share and earn, rather than spending months in the development lab. The key then was to measure, or test. If your product proved popular, great – now you could assess ways to make it even better and add value. If it wasn’t having the impact you hoped for, it was time to pivot – to make whatever changes were necessary. All changes would be tested in the market, informing the learning stage which would then lead on to the next iterative build.
This simple model proved so effective that it quickly spread beyond the world of entrepreneurial start-ups. According to one survey, 82% of R&D and product development executives at blue chip US companies say they have implemented at least some aspects of Lean into their strategies. The same study also listed the top five benefits of Lean as:
- Focusing decision-making on evidence and data.
- Faster development cycles and quicker realisation of ideas.
- Better-quality feedback from customers and stakeholders, based on what they buy rather than what they say in a focus group.
- Speaking to and observing real customers and stakeholders.
- Greater flexibility about making changes to ideas.
The Lean approach has been particularly successful in software development. Google famously launched its first search engine as a basic HTML page to get a feel for how web users would react to it. Successful tech startups like Dropbox and Pebble started out with MVP releases that were attracting pre-orders while still very much in the beta stage. We will return to some of these success stories later, but now let’s look at the process by which lean developers evaluate and refine their MVP into the final product – testing.
What is A/B Testing? What are the Benefits?
One way to understand the role of MVP in Lean development is to see it as the kickstarter for the testing and improvement cycle in a live environment. It gives developers their first tangible test, their first opportunity to assess performance and gather feedback which will inform the ‘learn’ process ahead of the next iteration.
But in order to learn anything meaningful from the launch of an MVP and all subsequent updates, developers must take a structured, quantitative approach to testing which provides measurable data they can base next step decisions on. There are a range of different methods used to test performance of products in the market and to inform decision-making for future iterations. But one of the methodologies advocated by Eric Ries himself in The Lean Startup is A/B Testing.
Ries describes A/B testing as “an experiment in which different versions of a product are offered to customers at the same time.” Digging deeper into how an A/B test works, the idea is to offer near-identical versions of the same product, but with one feature or function altered to offer two alternatives – A and B. By releasing both simultaneously, the development team can monitor user behaviour and preferences to establish which is preferred.
A/B testing can be done by giving users two options and seeing which proves the most popular, e.g. two different layouts on a web page (see diagram below). Equally, it can be done by splitting user groups into different ‘buckets’, allowing them to experience either option A or option B, and then gathering feedback to compare.
The main point is to only change one metric (i.e. a feature or function) at a time, so that you understand clearly the impact that one variable has on user preferences and behavior. This has two clear benefits over building complete versions of a product before launch. One, it makes it much more likely that your final version will be something your users respond well to, as you have chosen features and functions piece by piece according to evidence of real-world preferences. Second, it means that your product is out there ‘in the wild’ winning over early adopters, building the presence of your brand and generating revenue even as you refine it. With cycles of A/B testing, you are methodically improving usability and quality step by step, making it much more likely that you end up with a product people want to use and pay for.
A/B Testing and MVP in Action
A/B testing has perhaps become best known as a tool used by digital marketers and web developers who use comparisons between ‘live’ alternatives to tweak UX to follow the path of identified user preferences. For example, A/B testing is recognised as an effective means of improving rates of conversion from website hits to desired actions like making a purchase or registering an email address.
The principles of A/B testing can be applied to any type of product. But what makes it particularly well suited to digital assets like websites and cloud apps is the very rapid feedback and results developers can get, thanks to very short paths to market. That is why some of the best examples of A/B testing in action come from software development and digital start-ups.
Eric Ries describes one as providing him with inspiration for The Lean Startup. He describes the journey of Votizen, a US-based online voter organisation platform first developed in 2007. The product had a slow start in life, before David Binetti started A/B testing product features and design. Once he started testing and iterating, Binetti saw registration numbers climb, activation/participation rates rocket and, by 2010, he was able to secure $1.5 million in funding.
An excellent example of A/B testing in action is the story of mobile app Bounce, a handy personal organising tool that combines your calendar and your location to tell you when you need to leave for an appointment. Bounce’s developers effectively ‘crowdtested’ an optimum price point for the app by A/B testing different prices through crowd funding platform Self Starter – people weren’t just signing up to back the project, they were signing up to pay a specific price for it. They found, for example, that a higher proportion of backers would pay $10 compared to $5, telling them they could afford to set a higher price.
Finally, for an example of how the principles of MVP and A/B testing can be taken right to the heart of a company’s business strategy, we can single out Atlassian, the development firm behind enterprise collaboration apps like Trello and Jira. As a young startup offering its products on a SaaS ‘self-service’ model, Atlassian found that the key to cementing and developing its share of market was focusing acutely on user and customer experience, and being prepared to experiment continuously to find the best options available. The company’s methodology was to formulate hypotheses about what might work, and then simply test them by giving real customers A/B alternatives, thus gathering data to back up the original suggestion.
Find out more
At Aspire, our mission is to help clients develop high quality software solutions that answer real world business challenges and meet the needs of their customers and users, quickly, efficiently and with value in mind. We apply A/B testing and other aspects of Lean methodology across web application development, mobile application development, application migration and integration, software product development and more. To find out more about how we work and how we can help you, contact us today.
