Production
https://prod.org.br/article/doi/10.1590/0103-6513.20220035
Production
Thematic Section - Resilient and innovative operations management

Interpreting direct sales’ demand forecasts using SHAP values

Mariana Arboleda-Florez; Carlos Castro-Zuluaga

Downloads: 1
Views: 357

Abstract

Paper aims: Several concerns regarding the lack of interpretability of machine learning models obstruct the implementation of machine learning projects as part of the demand forecasting process. This paper presents a methodology to support the introduction of machine learning into the forecasting process of a traditional direct sales company by providing explanations for the otherwise obscure results. We also suggest incorporating human knowledge inside the machine learning pipeline as an essential part of capturing the business logic and integrating machine learning into the existing processes.

Originality: Using explainable machine learning methods on real-life company data demonstrates that machine learning techniques are functional beyond the academy and can be introduced to everyday companies' production.

Research method: The project used real-world data from a company and followed a traditional machine learning pipeline to collect, preprocess, select and train a machine learning model, to conclude with the explanation of the model results through the implementation of SHAP

Main findings: The results provided insights regarding the contribution of the features to the forecast. We analyzed individual predictions to understand the behavior of different variables, proving helpful when interpreting complex machine learning models.

Implications for theory and practice: This study contributes to a discussion about adopting new technology and implementing machine learning models for demand forecasting. The methodology presented in this paper can be used to implement similar projects on interested companies.

Keywords

Explainable Artificial Intelligence, Machine learning, Sales forecasting

References

Abdulhai, B., & Kattan, L. (2003). Reinforcement learning: introduction to theory and potential for transport applications. Canadian Journal of Civil Engineering, 30(6), 981-991. http://dx.doi.org/10.1139/l03-014.

Adadi, A., & Berrada, M. (2018). Peeking inside the black box: a survey on explainable artificial intelligence (XAI). In Proceedings of the IEEE Access: Practical Innovations, Open Solutions (Vol. 6, pp. 52138-52160). USA: IEEE. http://dx.doi.org/10.1109/ACCESS.2018.2870052.

Bandeira, S., Alcalá, S., Vita, R., & Barbosa, T. (2020). Comparison of selection and combination strategies for demand forecasting methods. Production, 30, e20200009. http://dx.doi.org/10.1590/0103-6513.20200009.

Barredo Arrieta, A., Díaz-Rodríguez, N., del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities, and challenges toward responsible AI. Information Fusion, 58, 82-115. http://dx.doi.org/10.1016/j.inffus.2019.12.012.

Bertrand, J. W. M., & Fransoo, J. C. (2002). Operations management research methodologies using quantitative modeling. International Journal of Operations & Production Management, 22(2), 241-264.

Bisong, E. (2019). What is machine learning? In E. Bisong. Building machine learning and deep learning models on google cloud platform (pp. 169-170). Berkeley: Apress. http://dx.doi.org/10.1007/978-1-4842-4470-8_13.

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32. http://dx.doi.org/10.1023/A:1010933404324.

Brockwell, P. J., & Davis, R. A. (1987). Time series: theory and methods. New York: Springer. http://dx.doi.org/10.1007/978-1-4899-0004-3

Brynjolfsson, E., & Mitchell, T. (2017). What can machine learning do? Workforce implications. Science, 358(6370), 1530-1534. http://dx.doi.org/10.1126/science.aap8062. PMid:29269459.

Bugaj, M., Wrobel, K., & Iwaniec, J. (2021, May 12-16). Model explainability using SHAP values for LightGBM predictions. In International Conference on Perspective Technologies and Methods in MEMS Design. USA: IEEE. http://dx.doi.org/10.1109/MEMSTECH53091.2021.9468078.

Carbonneau, R., Laframboise, K., & Vahidov, R. (2008). Application of machine learning techniques for supply chain demand forecasting. European Journal of Operational Research, 184(3), 1140-1154. http://dx.doi.org/10.1016/j.ejor.2006.12.004.

Castro-Zuluaga, C. A., & Arboleda, M. (2019). Sales forecasting difficulties' analysis on colombian direct sales companies. In M. T. Castañeda Galvis, J. Nuñez Rodriguez, M. C. Pérez Ordoñez & M. Villa Marulanda (Eds.), Proceedings of the International Congress of Industrial Engineering (ICIE2019). ICIE 2019. Lecture Notes on Multidisciplinary Industrial Engineering (pp. 112-118). Cham: Springer. https://doi.org/10.1007/978-3-030-49370-7_12.

Castro-Zuluaga, C., & Arboleda-Florez, M. (2021). Introduction. In M. Hemmati & M. S. Sajadieh (Eds.), Influencing customer demand: an operations management approach (pp. 1-16). Boca Raton: CRC Press. http://dx.doi.org/10.1201/9781003107446-1.

Chatfield, C. (2000). Time-series forecasting. London: Chapman and Hall/CRC. https://doi.org/10.1201/9781420036206.

Chen, I. F., & Lu, C. J. (2021). Demand forecasting for multichannel fashion retailers by integrating clustering and machine learning algorithms. Processes, 9(9), 1578. https://doi.org/10.3390/PR9091578.

Clinciu, M. A., & Hastie, H. (2019). A survey of explainable AI terminology. In Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence (NL4XAI 2019) (pp. 8-13). http://dx.doi.org/10.18653/v1/W19-8403.

Crum, C., & Palmatier, G. E. (2003). Demand management best practices: process, principles, and collaboration. Boca Raton: J. Ross Publishing.

Dairu, X., & Shilong, Z. (2021). Machine learning model for sales forecasting by using XGBoost. In 2021 IEEE International Conference on Consumer Electronics and Computer Engineering, ICCECE 2021 (pp. 480-483). USA: IEEE. http://dx.doi.org/10.1109/ICCECE51280.2021.9342304.

Deloitte. (2017). Bullish on the business value of cognitive Leaders in cognitive and AI weigh in on what's working and what's next. New York: Deloitte.

Dong, G., & Liu, H. (Eds.). (2018).Feature engineering for machine learning and data analytics. Boca Raton: CRC Press.

Friedman, J., & Popescu, B. E. (2003). Gradient directed regularization for linear regression and classification. Technical Report, Statistics Department, Stanford University.

Gartner Inc. (2017). Analysts Answer What Are Top Client Concerns About Ai. Retrieved January 24, 2022, from https://www.gartner.com/smarterwithgartner/analysts-answer-what-are-top-client-concerns-about-ai

Gilbert, F. (2019). Introducing SHAP Decision Plots. Visualize the inner workings of machine learning models with greater detail and flexibility. Towards Data Science. Retrieved January 24, 2022, from https://towardsdatascience.com/introducing-shap-decision-plots-52ed3b4a1cba

Gramegna, A., & Giudici, P. (2021). SHAP and LIME: an evaluation of discriminative power in credit risk. Frontiers in Artificial Intelligence, 4, 752558. http://dx.doi.org/10.3389/frai.2021.752558. PMid:34604738.

Gumani, M., Korke, Y., Shah, P., Udmale, S., Sambhe, V., & Bhirud, S. (2017). Forecasting of sales by using fusion of machine learning techniques. In 2017 International Conference on Data Management, Analytics and Innovation, ICDMAI 2017, (pp. 93-101). USA: IEEE. https://doi.org/10.1109/ICDMAI.2017.8073492.

Hiziroglu, A. (2013). Soft computing applications in customer segmentation: state-of-art review and critique. Expert Systems with Applications, 40(16), 6491-6507. http://dx.doi.org/10.1016/j.eswa.2013.05.052.

Ishikawa, F., & Yoshioka, N. (2019). How Do Engineers Perceive Difficulties in Engineering of Machine-Learning Systems? - Questionnaire Survey. In Proceedings of the 2019 IEEE/ACM Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Engineering Research and Industrial Practice (SER&IP) (pp. 2-9). USA: IEEE. https://doi.org/10.1109/CESSER-IP.2019.00009.

Jeon, Y., & Seong, S. (2021). Robust recurrent network model for intermittent time-series forecasting. International Journal of Forecasting, 38(4), 1415-1425. http://dx.doi.org/10.1016/j.ijforecast.2021.07.004.

Kormushev, P., Calinon, S., & Caldwell, D. G. (2013). Reinforcement learning in robotics: applications and real-world challenges. Robotics, 2(3), 122-148. http://dx.doi.org/10.3390/robotics2030122.

Krishna, A., Akhilesh, V., Aich, A., & Hegde, C. (2018). Sales-forecasting of Retail Stores using Machine Learning Techniques. In Proceedings of the 2018 3rd International Conference on Computational Systems and Information Technology for Sustainable Solutions, CSITSS 2018, 160–166. http://dx.doi.org/10.1109/CSITSS.2018.8768765

Ktenioudaki, A., O’Donnell, C. P., Emond, J. P., & do Nascimento Nunes, M. C. (2021). Blueberry supply chain: critical steps impacting fruit quality and application of a boosted regression tree model to predict weight loss. Postharvest Biology and Technology, 179, 111590. http://dx.doi.org/10.1016/j.postharvbio.2021.111590.

Kumar, V., & Boulanger, D. (2020). Explainable automated essay scoring: deep learning really has pedagogical value. Frontiers in Education, 5, 572367. http://dx.doi.org/10.3389/feduc.2020.572367.

Lorente-Leyva, L. L., Alemany, M. M. E., Peluffo-Ordóñez, D. H., & Araujo, R. A. (2021). Demand forecasting for textile products using statistical analysis and machine learning algorithms. In N. T. Nguyen, S. Chittayasothorn, D. Niyato & B. Trawiński (Eds.), Intelligent Information and Database Systems. ACIIDS 2021. Lecture Notes in Computer Science (vol. 12672, pp. 181-194). Cham: Springer. http://dx.doi.org/10.1007/978-3-030-73280-6_15.

Lundberg, S. (2019). GitHub - slundberg/shap: a game theoretic approach to explain the output of any machine learning model. Retrieved January 24, 2022, from https://github.com/slundberg/shap

Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2019). Consistent individualized feature attribution for tree ensembles. arXiv, 1802.03888v3, 1-9. https://doi.org/10.48550/arXiv.1802.03888.

Lundberg, S., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. arXiv, 1705.07874v2, 1-10. https://doi.org/10.48550/arXiv.1705.07874.

Makridakis, S. (1988). Metaforecasting. International Journal of Forecasting, 4(3), 467-491. http://dx.doi.org/10.1016/0169-2070(88)90112-4.

Marcilio, W. E., & Eler, D. M. (2020). From explanations to feature selection: assessing SHAP values as feature selection mechanism. In Proceedings of the 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 340-347). USA: IEEE. https://doi.org/10.1109/SIBGRAPI51738.2020.00053.

Meng, Y., Yang, N., Qian, Z., & Zhang, G. (2020). What makes an online review more helpful: an interpretation framework using XGBoost and shap values. Journal of Theoretical and Applied Electronic Commerce Research, 16(3), 466-490. http://dx.doi.org/10.3390/jtaer16030029.

Mitchell, T. M. (1997). Does machine learning really work? Retrieved January 24, 2022, from https://ojs.aaai.org/index.php/aimagazine/article/view/1303

Mitchell, T. M. (2006). The discipline of machine learning. Pittsburgh: Carnegie Mellon University.

Moore, J. D., & Swartout, W. R. (1988). Explanation in expert systems: a survey. University of Southern California Marina del Rey Information Sciences Inst.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: machine Learning in Python. Journal of Machine Learning Research, 12, 2825-2830.

Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2019). CatBoost: unbiased boosting with categorical features. Retrieved March 2, 2022, from https://github.com/catboost/catboost

Raschka, S., & Mirjalili, V. (2015). Python_machine_learning. Birmingham, Reino Unido: Packt Publishing Ltd.

Ren, S., Chan, H. L., & Siqin, T. (2020). Demand forecasting in retail operations for fashionable products: methods, practices, and real case study. Annals of Operations Research, 291(1-2), 761-777. http://dx.doi.org/10.1007/s10479-019-03148-8.

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Model-agnostic interpretability of machine learning. arXiv, 1606.05386v1, 91-95. https://doi.org/10.48550/arXiv.1606.05386.

Samek, W., & Müller, K.-R. (2019). Towards explainable artificial intelligence. In W. Samek, G. Montavon, A. Vedaldi, L. Hansen & K. R. Müller (Eds.), Explainable AI: Interpreting, explaining and visualizing deep learning. Lecture notes in computer science (Vol. 11700, pp. 5-22). Cham: Springer. http://dx.doi.org/10.1007/978-3-030-28954-6_1.

Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., & Young, M. (2014). Machine learning: the high-interest credit card of technical Debt. In Proceedings of the SE4ML: Software Engineering for Machine Learning (NIPS 2014 Workshop) (pp. 1-9). Google, Inc.

Seaman, B., & Bowman, J. (2021). Applicability of the M5 to Forecasting at Walmart. International Journal of Forecasting, 38(4), 1468-1472. http://dx.doi.org/10.1016/j.ijforecast.2021.06.002.

Seeling, M. X., Scavarda, L. F., & Thomé, A. M. T. (2019). A sales and operations planning application in the Brazilian subsidiary of a multinational chemical company. Brazilian Journal of Operations & Production Management, 16(3), 424-435. http://dx.doi.org/10.14488/BJOPM.2019.v16.n3.a6.

Shams Amiri, S., Mottahedi, S., Lee, E. R., & Hoque, S. (2021). Peeking inside the black-box: Explainable machine learning applied to household transportation energy consumption. Computers, Environment and Urban Systems, 88, 101647. http://dx.doi.org/10.1016/j.compenvurbsys.2021.101647.

Sheshasaayee, A., & Logeshwari, L. (2018). Implementation of clustering technique based RFM analysis for customer behaviour in online transactions. In Proceedings of the 2018 2nd International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 1166-1170). USA: IEEE. https://doi.org/10.1109/ICOEI.2018.8553873.

Silver, E. A., Pyke, D. F., & Thomas, D. J. (2016). Inventory and production management in supply chains. Boca Raton: CRC Press. https://doi.org/10.1201/9781315374406.

Strumbelj, E., & Kononenko, I. (2010). An efficient explanation of individual classifications using game theory. Journal of Machine Learning Research, 11, 1-18.

Su, X., Yan, X., & Tsai, C. L. (2012). Linear regression. Wiley Interdisciplinary Reviews: Computational Statistics, 4(3), 275-294. http://dx.doi.org/10.1002/wics.1198.

Sun, Z. L., Choi, T. M., Au, K. F., & Yu, Y. (2008). Sales forecasting using extreme learning machine with applications in fashion retailing. Decision Support Systems, 46(1), 411-419. http://dx.doi.org/10.1016/j.dss.2008.07.009.

Sutton, R., & Barto, A. G. (2018). Reinforcement learning: an introduction (2nd ed.). London: MIT Press.

Tarallo, E., Akabane, G. K., Shimabukuro, C. I., Mello, J., & Amancio, D. (2019). Machine learning in predicting demand for fast-moving consumer goods: an exploratory research. IFAC-PapersOnLine, 52(13), 737-742. http://dx.doi.org/10.1016/j.ifacol.2019.11.203.

Tirkolaee, E. B., Sadeghi, S., Mooseloo, F. M., Vandchali, H. R., & Aeini, S. (2021). Application of machine learning in supply chain management: a comprehensive overview of the main areas. Mathematical Problems in Engineering, 2021, 1-14. http://dx.doi.org/10.1155/2021/1476043.

Vafeiadis, T., Diamantaras, K. I., Sarigiannidis, G., & Chatzisavvas, K. Ch. (2015). A comparison of machine learning techniques for customer churn prediction. Simulation Modelling Practice and Theory, 55, 1-9. http://dx.doi.org/10.1016/j.simpat.2015.03.003.

Vogel, W., & Lasch, R. (2016). Complexity drivers in manufacturing companies: a literature review. Logistics Research, 9(1), 25. http://dx.doi.org/10.1007/s12159-016-0152-9.

Wenzel, H., Smit, D., & Sardesai, S. (2019). A literature review on machine learning in supply chain management. In W. Kersten, T. Blecker & C. M. Ringle (Eds.), Proceedings of the Hamburg International Conference of Logistics (HICL) (pp. 413-441). Artificial Intelligence and Digital Transformation in Supply Chain Management. https://doi.org/10.15480/882.2478.

Yao, Q., Wang, M., Chen, Y., Dai, W., Li, Y.-F., Tu, W.-W., Yang, Q., & Yu, Y. (2018). Taking human out of learning applications: a survey on automated machine learning. arXiv, 1810.13306v4, 1-20. https://doi.org/10.48550/arXiv.1810.13306.

Zhang, B., & Ma, D. (2020). Flight delay prediciton at an airport using maching learning. In Proceedings - 2020 5th International Conference on Electromechanical Control Technology and Transportation, ICECTT 2020 (pp. 557-560). USA: IEEE. http://dx.doi.org/10.1109/ICECTT50890.2020.00128.
 


Submitted date:
03/04/2022

Accepted date:
11/29/2022

63b2e043a953950c665305c4 production Articles
Links & Downloads

Production

Share this page
Page Sections