Our client is a provider of subscription television, internet, and mobile services. They have an international reach totalling over 22 million customers, and a history of developing innovative products and entertainment content.
Forecast was asked to provide strategic guidance and operational support in streamlining the AB testing procedure within the area of the business charged with improving the customer experience (Content Engagement). The in-house resources were focused on day-to-day reactive analysis and would benefit from the oversight needed to optimise computational procedures that would free up capacity to address more strategic priorities.
Forecast worked alongside the Content Engagement team and provided both strategic consulting guidelines as well as operational support for day-to-day business. The areas in which Forecast provided added capabilities in this project were:
- Consulting in the creation of a dynamic cohort creation Python-based tool, which led to reducing the time required to provide end-to-end hypothesis testing by over 50%.
- Ongoing statistical support to refine AB test analysis robustness by introducing new ways of working and introducing new software packages.
- Developing a standardised approach to translating and presenting the test findings to stakeholders through actionable insights and recommendations.
Forecast delivered statistical capabilities to allow for a streamline of the AB test procedures within the customer experience department. Some of the key outcomes included:
- 50% reduction in effort across the whole AB tests procedure by capitalising on a python-based tool and robust statistical analysis.
- Increased visibility and standards of the AB test key insights and recommendations.