Churn Prediction under Feature Drift in Subscription Analytics Platforms: Benchmark Study
Keywords:
Feature Drift, Churn Prediction, Subscription Analytics, Machine Learning, Predictive ModelingAbstract
The rapid proliferation of the subscription economy has fundamentally transformed consumer behavior and corporate revenue models across global markets. In this context, predicting customer churn has emerged as a critical operational imperative for subscription analytics platforms. However, the predictive performance of machine learning models deployed in these environments is often severely compromised by feature drift, a phenomenon where the statistical properties of input variables change over time due to shifting consumer preferences, economic conditions, or platform updates. This paper presents a comprehensive benchmark study that systematically evaluates the impact of feature drift on churn prediction algorithms within subscription analytics. By designing a robust evaluation framework, we analyze the degradation of predictive accuracy across various traditional and contemporary machine learning architectures when exposed to synthetic and naturally occurring feature drift. Our experimental methodology incorporates diverse drift profiles, including gradual, sudden, and recurring shifts, to simulate real-world volatility. The findings indicate that while complex ensemble models and deep learning architectures exhibit superior baseline performance, they are frequently more susceptible to feature drift compared to regularized linear models unless specifically coupled with adaptive learning mechanisms. This benchmark provides a foundational reference for practitioners and researchers to design more resilient churn prediction systems, ultimately enhancing customer retention strategies in dynamic subscription environments.References
1. Soybilgen, B.; Yazgan, E. Nowcasting US GDP using tree-based ensemble models and dynamic factors. Comput. Econ. 2021, 57, 387–417.
2. Qu, L.; Lu, H.; Tian, Z.; Schoorl, J.M.; Huang, B.; Liang, Y.; Qiu, D.; Liang, Y. Spatial prediction of soil sand content at various sampling density based on geostatistical and machine learning algorithms in plain areas. CATENA 2024, 234, 107572.
3. Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed.; Springer: New York, NY, USA, 2009.
4. Kakhani, N.; Taghizadeh-Mehrjardi, R.; Omarzadeh, D.; Ryo, M.; Heiden, U.; Scholten, T. Towards Explainable AI: Interpreting Soil Organic Carbon Prediction Models Using a Learning-Based Explanation Method. Eur. J. Soil Sci. 2025, 76, e70071.
5. Iglewicz, B. Summarizing Data with Boxplots. In International Encyclopedia of Statistical Science; Springer: Berlin/Heidelberg, Germany, 2011; pp. 1572–1575.
6. Niyongabo Rubungo, A.; Li, K.; Hattrick-Simpers, J.; Bousso Dieng, A. LLM4Mat-Bench: Benchmarking Large Language Models for Materials Property Prediction. Mach. Learn. Sci. Technol. 2025, 6, 020501.
7. Lv, C.; Zhou, X.; Zhong, L.; Yan, C.; Srinivasan, M.; Seh, Z.W.; Liu, C.; Pan, H.; Li, S.; Wen, Y.; et al. Machine Learning: An Advanced Platform for Materials Development and State Prediction in Lithium-Ion Batteries. Adv. Mater. 2022, 34, e2101474.
8. U.S. Department of Energy. Climate Calculations: Engineering Reference—EnergyPlus 9.2; U.S. Department of Energy: Washington, DC, USA, 2019.
9. Beyhum, J.; Striaukas, J. Factor-Augmented Sparse MIDAS Regressions with an Application to Nowcasting; Faculty of Economics and Business Discussion Paper Series; DPS 24.14; KU Leuven: Leuven, Belgium, 2024; 65p.
10. Ganfure, G.O.; Wu, C.F.; Chang, Y.H.; Shih, W.K. DeepPrefetcher: A Deep Learning Framework for Data Prefetching in Flash Storage Devices. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 2020, 39, 3311–3322.
11. Ferrara, L.; Simoni, A. When Are Google Data Useful to Nowcast GDP? An Approach via Preselection and Shrinkage. J. Bus. Econ. Stat. 2023, 41, 1188–1202.
12. Eichenauer, V.Z.; Indergand, R.; Martínez, I.Z.; Sax, C. Obtaining consistent time series from Google Trends. Econ. Inq. 2022, 60, 694–705.
13. Duveiller, G.; Hooker, J.; Cescatti, A. The Mark of Vegetation Change on Earth’s Surface Energy Balance. Nat. Commun. 2018, 9, 679.
14. Sezer, N.; Yoonus, H.; Zhan, D.; Wang, L.L.; Hassan, I.G.; Rahman, M.A. Urban Microclimate and Building Energy Models: A Review of the Latest Progress in Coupling Strategies. Renew. Sustain. Energy Rev. 2023, 184, 113577.
15. LaDochy, S. Los Angeles’ Urban Heat Island Continues to Grow: Urbanization, Land Use Change Influences. J. Urban Environ. Eng. 2021.
16. Bera, R.; Kanellopoulos, K.; Nori, A.; Shahroodi, T.; Subramoney, S.; Mutlu, O. Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning. In Proceedings of the 54th Annual IEEE/ACM International Symposium on Microarchitecture, Virtual Event, 18–22 October 2021; pp. 1121–1137.
17. Coulombe, P.G.; Leroux, M.; Stevanovic, D.; Surprenant, S. Macroeconomic data transformations matter. Int. J. Forecast. 2021, 37, 1338–1354.
18. Kibrete, F.; Trzepieciński, T.; Gebremedhen, H.S.; Woldemichael, D.E. Artificial Intelligence in Predicting Mechanical Properties of Composite Materials. J. Compos. Sci. 2023, 7, 364.
19. Lian, Z.; Ma, Y.; Li, M.; Lu, W.; Zhou, W. Discovery Precision: An Effective Metric for Evaluating Performance of Machine Learning Model for Explorative Materials Discovery. Comput. Mater. Sci. 2024, 233, 112738.
20. Subedi, P.; Davis, P.E.; Parashar, M. Leveraging Machine Learning for Anticipatory Data Delivery in Extreme Scale In-situ Workflows. In Proceedings of the 2019 IEEE International Conference on Cluster Computing (CLUSTER), Albuquerque, NM, USA, 23–26 September 2019; pp. 1–11.
21. Heuvelink, G.B.M.; Angelini, M.E.; Poggio, L.; Bai, Z.; Batjes, N.H.; van den Bosch, R.; Bossio, D.; Estella, S.; Lehmann, J.; Olmedo, G.F.; et al. Machine learning in space and time for modelling soil organic carbon change. Eur. J. Soil Sci. 2020, 72, 1607–1623.
22. Namazi, H.; Pradhan, A.K.; Frischer, R.; Socha, L. Machine Learning-Guided Electrolyte–Interface Co-Design for Li-Based Anode-Free Batteries: Toward a Predictive Framework for Stable Metal Deposition. J. Power Sources 2026, 690, 240826.
23. Lee, J.; Kim, H.; Vuduc, R. When Prefetching Works, When It Doesn’t, and Why. ACM Trans. Archit. Code Optim. 2012, 9, 1–29.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Authors

This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles are distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0), unless otherwise stated.