18 posts tagged with "AB testing"

Suppose you have two variations of a software product, but you're not sure which one to deploy. That's the problem A/B testing solves. And in mobile development, running a clean A/B test is harder than it sounds because you can't just toggle a server-side flag and move on. You're dealing with bundle updates, app store review times, and users who haven't updated in months.

This guide shows how to run a proper A/B test in a React Native app using ConfigCat feature flags for the rollout and Amplitude for analytics. By the end, you'll have a working implementation, a real results-reading framework, and a clear picture of what to do when the test is over.

Let's say your team has developed a new feature update and is planning to release it to the public. There can be some uncertainty and risk because it is hard to predict how users will react to the change. Will the new update have a negative impact and drive users away from the app? The best way to know for sure is to adopt an A/B testing approach by releasing it to a subset of users to measure its impact prior to making a full deployment. This gives you enough room to uncover bugs and refine the feature without disrupting the experience for everyone.

Knowing up front whether a new feature will improve or worsen user experience can be a challenging decision. If we don't adopt the proper mechanisms and processes to test new features, we stand a high risk of introducing new bugs. By incorporating A/B testing into our feature release workflow, we can minimize these situations by testing the new feature with a small sample of users before deploying it.

In this post, we'll put this into practice by A/B testing a pricing feature on a demo Angular website. We'll track the performance of two variations to see which one attracts more clicks.

The ability to make good decisions is often the leading factor in the success of a business. Yet, it is becoming increasingly difficult for companies to decide on what ideas to develop and content to optimize for users with certainty that it will perform as predicted.

Feature flagging is a vital technique that enables businesses to perform controlled A/B test experiments to gauge and analyze the impact of their decisions. A/B testing can effectively improve a business's overall performance and boost conversion rates by comparing and contrasting multiple implementations based on their performance with real users.

Scenario: You’ve thought up a small change for your app. You write and test the code, and everything looks good. As you’re about to push it into production, you stop and ask yourself, “Will the users like this?”

You start having doubts, that maybe the idea isn’t as good as you previously thought. Still, you continue to have a strong feeling that it’ll make your app better.

One solution to this dilemma is to gradually introduce the change to a portion of users and track its impact on them. This is called A/B testing, and it’s a simple, low-risk way of letting your users pick which variant yields better results.

Product managers are responsible for deciding what products to develop that align with the company's goals and also satisfy the needs of the customers. To be a successful product manager, you have to start by defining what a winning product looks like and iterate over it until it's right. To do so, you have to understand that at its core, a product is broken down into its features and quality of user experience.

However, when developing these features, even after thoroughly testing them in your development environment for potential bugs and issues that may negatively impact user experience, it may not always be sufficient to ensure a successful release to users.

Most companies believe they understand the customer, only to be shocked when their customers behave differently than what they expected, either intentionally or unintentionally. That's where A/B testing comes in to kick all these doubts and prevent the shock.

We’ll play around to see how A/B testing works with ConfigCat’s feature flag management service to take your experiments to the next level by giving you the ability to remotely control and configure your features without going back to the code.

This is a step-by-step guide on implementing an A/B testing scenario using ConfigCat and visualizing the results in a funnel with Amplitude. There is a working sample application on GitHub if you want to follow along.