3 A/B tests eMeals ran to make it easier for families to plan healthy, affordable meals

Posted by Page Grossman on Oct 30, 2017

How often do you find yourself hauling a disgruntled toddler through the grocery store while trying to figure out just what you’ll cook this week? Once you finally figure out what to buy and throw in a few packaged items to un-gruntle the toddler, you get to the cashier line only to wait for five people to check out ahead of you. By the time you actually make it to swiping your credit card, you’ve checked all your phone notifications and tossed in a pack of gum and a totally unnecessary candy bar which you promised yourself you wouldn’t do.

Does any of that sound familiar? We’re guessing it does and that it’s not something you enjoy.

That’s why eMeals, one of our awesome Outbound customers, created an app that helps busy families plan healthy meals, manage their shopping list, and even have fresh ingredients delivered to their doorstep.

As any startup knows, helping customers through the entire lifecycle is vital to getting new shoppers in the door and turning them into repeat buyers. To guide users through their customer lifecycle, eMeals turned to Outbound to help welcome new customers to their app and educate them on all the benefits of their service.

“One of the key reasons we went with Outbound was that it would have taken too long for us to build and implement a targeted messaging tool like Outbound,” said eMeals’ Chief Technology Officer Aaron Kenny. “Before Outbound, our engineers had to help customize email templates and trigger event-based messages. Additionally, we didn’t do any email testing because we simply didn’t have the framework to execute experiments and measure the results.”

But as Kenny pointed out, Outbound is more than just a messaging tool; it also allows them to get to know their customers better and understand exactly what drives them to buy. “We know that if we get people into the app quickly and encourage them to add meals to their shopping list, they’re more likely to see the value of our app and convert into a paying customer,” said Kenny.

To help customers experience an ideal journey, eMeals is using Outbound to run and test various campaigns at different points of the customer lifecycle including the following:

1) New Customer Welcome Email

One of the first campaigns eMeals implemented with Outbound was to A/B test a new welcome email when a new customer signs up. Kenny noted that the eMeals’ welcome email hadn’t been changed in a long time, so he wanted to experiment with the messaging to find out if tweaks to the content would drive more customers to download the app and take advantage of all the features.

After running two versions of the welcome email as a test for a couple weeks, Kenny uncovered the winner. The winning variant was a long-form email that contained more details about how to make the most of the eMeals service.

With the new welcome email, eMeals saw more customers installing their app, adding meals to their shopping list, and placing grocery orders.

2) Inactive Customer Re-engagement SMS

Often times, when customers first sign up to try a service, they need a little nudge to get started. So when eMeals sees a trial customer with no app activity, they follow up with a text message to check-in. Instead of guessing what message would work best as an SMS, Kenny and his team decided to run an A/B test in Outbound when setting up this campaign.

The first message they tested focused on the pain of going to the grocery store and how eMeals can help shoppers skip the line. The second message focused more on the benefit of customizing your meal plan. So which text won? The second message was the winner with almost twice the response rate.

“One of the things that’s been enlightening is the difference that messaging content can make,” said Kenny. “You can sit in a room and have five smart people come up with what they think is the best message. Then someone else can come up with an alternate in five minutes, and the alternate might play out better! We’re a very data driven company and you never know when little changes in the messaging will meaningfully influence customer behavior.”

3) Referral Program Improvement

With all the successes eMeals has seen with A/B tests, they continued their experiments to find out how they could increase customer referrals from the program they’d been running for some time. After a customer selects a meal and shops for groceries online, they receive an email powered by Outbound to refer their friends. However, eMeals wasn’t getting as many referrals as they wanted. What was the problem?

Reviewing Outbound’s analytics, they came up with a hypothesis: Customers were not completing the referral form because they had to login in order to do so.

To test their hypothesis, eMeals quickly spun up another A/B test with Outbound to validate their theory. Half of the customers would continue to have the same referral form experience, while the other half would be logged in automatically. The result? Removing the login page allowed eMeals to double their referrals. Without insights and the ability to test in Outbound, they might have never noticed this problem, let alone been able to run an experiment to prove their hypothesis and solve the problem.

“With Outbound, we’ve been able to extend our data-driven approach to messaging,” Kenny said. “Running campaigns and A/B tests is second nature now. Being able to think of an idea, quickly test it, and then measure the metrics has been a big step forward and has helped us improve the overall experience of our customers.”

Learn more about how you too can use Outbound to A/B test the messages you send to your customers.