Validating Valpak’s Machine Learning Investment with a PoC
Valpak, one of North America’s leading direct marketing companies, recognized an opportunity to boost customer retention using insights from historical customer behavior data. However, the Valpak team was unsure if their data could predict which customers are likely to leave and why, and they lacked the data science expertise necessary to extract those insights. To determine if they could predict customer churn, AgileThought conducted Predictive Analytics Discovery—a solution for analyzing data, statistically evaluating data quality, and delivering a proof of concept (PoC) that evaluates the viability of a specific business use case. Now, the Valpak team has a deeper data science understanding and uses the PoC codebase to run their own predictive models.
- Determining if Valpak’s predictive analytics use case— reducing customer churn—was feasible based on historical customer data
- Narrowing down which specific data points would produce the most valuable customer churn insights
- Navigating various data science tools and techniques, and understanding how to use those tools to determine the best approach to predict churn
- Developing a low-risk proof of concept to justify a machine learning solution before investing in it
Defining the Data Science Approach
AgileThought partnered with Valpak to reframe its business problem— reducing its customer churn rate—as a data science objective: Predicting when customers will leave, and why. By analyzing Valpak’s historical data and researching similar business scenarios, AgileThought quantified the business problem and identified which features the inputs that are most meaningful for analysis—would be ideal for the PoC.
Formulating, Shaping & Standardizing Data
After helping Valpak define its data science objective, AgileThought’s data scientists helped formulate, shape and standardize Valpak’s data to ensure it was in the right format for the predictive model. Using Python, AgileThought’s data scientists performed exploratory analysis to summarize trends and identify which customer data could predict customer churn. With these insights, AgileThought then performed feature engineering to enhance the data and determine which features—like the frequency of touchpoints between Valpak’s account executives and customers—would help improve the model’s performance.
Training and Fine-Tuning the Predictive Model
Once AgileThought’s data scientists prepared the data, they trained models using two algorithms—logistic regression and gradient boosted trees—to determine which model most accurately predicts churn in a given timeframe. If either model performed poorly, the data scientists would take different actions to enhance the model’s performance:
- Tuning the Model—Changing the model itself by adjusting the algorithm
- New features—Adding new features, like how often a customer clicks a digital coupon
- Problem adjustments—For example, tweaking the problem to predict the likelihood of customer churn in three months instead of one month
Interpreting the Results
At the end of the engagement, AgileThought delivered the code base for the model and prepared a final report including the model’s performance, additional feature recommendations, and a high-level roadmap to implement the solution at scale. AgileThought also taught the Valpak team how to interpret the model’s results to determine what behaviors—such as infrequent contacts with the customers—often indicate churn, so they could experiment with different customer retention initiatives to decrease the probability of churn.
Chris Cate, Chief Operating Officer and CIO, Valpak
Predicting Churn with a PoC
Now, with insights from the PoC, Valpak can quantify the value of each customer and determine whether or not customer retention tactics are necessary—a capability they previously lacked.
Running Predictive Models Autonomously
After building the PoC, AgileThought delivered the code base so Valpak could run the predictive models autonomously. AgileThought also taught Valpak how to continue using the PoC—including how to re-train and tune the model over time—and explained how to move forward with scaling the completed PoC.
To keep teams aligned throughout the process, AgileThought’s data scientists helped the Valpak team establish metrics, measure progress and interpret the results of the model. By transferring this knowledge—and showing the Valpak team how to use the machine learning tools—AgileThought has helped Valpak broaden its data science capabilities.