Case study

Streamlining AB Testing Capabilities

Media, Telecommunications

Forecast delivered statistical capabilities to allow for a streamline of the AB test procedures within the customer experience department.

Client

Our client is a provider of subscription television, internet, and mobile services. They have an international reach totalling over 22 million customers, and a history of developing innovative products and entertainment content.

Purpose

Forecast was asked to provide strategic guidance and operational support in streamlining the AB testing procedure within the area of the business charged with improving the customer experience (Content Engagement). The in-house resources were focused on day-to-day reactive analysis and would benefit from the oversight needed to optimise computational procedures that would free up capacity to address more strategic priorities.

Approach

Forecast worked alongside the Content Engagement team and provided both strategic consulting guidelines as well as operational support for day-to-day business. The areas in which Forecast provided added capabilities in this project were:

  • Consulting in the creation of a dynamic cohort creation Python-based tool, which led to reducing the time required to provide end-to-end hypothesis testing by over 50%.
  • Ongoing statistical support to refine AB test analysis robustness by introducing new ways of working and introducing new software packages.
  • Developing a standardised approach to translating and presenting the test findings to stakeholders through actionable insights and recommendations.

Outcome

Forecast delivered statistical capabilities to allow for a streamline of the AB test procedures within the customer experience department. Some of the key outcomes included:

  • 50% reduction in effort across the whole AB tests procedure by capitalising on a python-based tool and robust statistical analysis.
  • Increased visibility and standards of the AB test key insights and recommendations.

Like this article? You’re one step closer to making data-driven decisions.

Reach out to us to start a conversation – we welcome an opportunity to understand your data needs, challenges and aspirations. We utilise an expert toolkit of data capabilities and services to help take guesswork out of your decision making.

More like this

Case study

Media, Telecommunications

Data Migration & Engineering

Case study

Media, Telecommunications

Covid-19 Behavioural Machine Learning Model

See more Case Studies Contact us