Research Article

Assessing the service quality in Software-as-a-Service from the customers’ perspective: a methodological approach and case of use

Freitas, André Luís Policani; Freitas Neto, Michelle Maria

Downloads: 1
Views: 585


Abstract: Despite the advances in the evaluation of the quality of Software-as-a-Service (SaaS), new studies seem to be necessary, since the existing criticisms concerning the SERVQUAL scale and the lack of studies concerning the use of SERVPERF for this purpose. This work aims to fulfil this gap by proposing a methodological approach to assess the SaaS service quality by measuring SaaS customers’ satisfaction. Factor analysis is used to summarize the information contained in the original items into a smaller set of new dimensions and Quartile analysis is suggested to determine the most critical items. By conducting a study, the factors that most influence customer’s satisfaction are customer service, customer assistance and the reliability of SaaS. Most of the critical items are associated with the transparency and accuracy in correcting errors, the company's interest in solving customer problems, SaaS application's ability to meet business requirements, implemented updates and regularity of service performance.


SaaS service quality, Customer satisfaction, Service quality, Software as a service


Babakus, E., & Mangold, W. G. (1992). Adapting the SERVQUAL scale to hospital services: an empirical investigation. Health Services Research, 26(6), 767-786. PMid:1737708.

Bansal, H. S., & Taylor, S. F. (1999). The Service Provider Switching Model (SPSM): a model of consumer switching behavior in the services industry. Journal of Service Research, 2(2), 200-218.

Barnes, S. J., & Vidgen, R. T. (2002). An integrative approach to the assessment of e-commerce quality. Journal of Electronic Commerce Research, 3, 114-127.

Bauer, H. H., Falk, T., & Hammerschmidt, M. (2006). eTransQual: a transaction process-based approach for capturing service quality in online shopping. Journal of Business Research, 59(7), 866-875.

Benlian, A., Koufaris, M., & Hess, T. (2012). Service quality in Software-as-a-Service: developing the SaaS-Qual measure and examining its role in usage continuance. Journal of Management Information Systems, 28(3), 85-126.

Bitner, M. J. (1990). Evaluating service encounters: the effects of physical surroundings and employee responses. Journal of Marketing, 54(2), 69-82.

Bolton, R. N., & Drew, J. H. (1991). A multistage model of customers’ assessments of service quality and value. The Journal of Consumer Research, 17(4), 375-384.

Bolton, R. N., & Drew, J. H. (1994). Linking customer satisfaction to service operations and outcomes. In R. T. Rust & R. L. Oliver (Eds.), Service quality: new directions in theory and practice (pp. 173-200). Newbury Park, CA: Sage Publications Inc.

Brown, T. J., Churchill Junior, G. A., & Peter, J. P. (1993). Improving the measurement of service quality. Journal of Retailing, 69(1), 127-139.

Buttle, F. (1996). SERVQUAL: review, critique, research agenda. European Journal of Marketing, 30(1), 8-32.

Cancian, M. H. (2009). Uma proposta de guia de referência para provedores de software como um serviço (dissertação de mestrado). Universidade Federal de Santa Catarina, Santa Catarina.

Carman, J. M. (1990). Consumer perceptions of service quality: an assessment of the SERVQUAL dimensions. Journal of Retailing, 66(1), 33-55.

Chou, S. W., & Chiang, C. H. (2013). Understanding the formation of software-as-a-service (SaaS) satisfaction from the perspective of service quality. Decision Support Systems, 56(1), 148-155.

Cisco. (2014). Cisco global cloud index: forecast and methodology, 2013-2018 (White Paper). Retrieved in 2017, March 27, from

Costa, H. G., Mansur, A. F. U., Freitas, A. L. P., & Carvalho, R. A. (2007). ELECTRE TRI aplicado a avaliação da satisfação de consumidores. Produção, 17(2), 230-245.

Cristobal, E., Flavián, C., & Guinalíu, M. (2007). Perceived e-service quality (PeSQ): measurement validation and effects on consumer satisfaction and web site loyalty. Managing Service Quality, 17(3), 317-340.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.

Cronin, J. J., & Taylor, S. (1992). Measuring service quality: a reexamination and extension. Journal of Marketing, 56(3), 55.

Dabholkar, P., Shepherd, C. D., & Thorpe, D. I. (2000). A comprehensive framework for service quality: an investigation of critical conceptual and measurement issues through a longitudinal study. Journal of Retailing, 76(2), 139-173.

Ding, D. X., Hu, P. J.-H., & Sheng, O. R. L. (2011). e-SELFQUAL: a scale for measuring online self-service quality. Journal of Business Research, 64(5), 508-515.

Domínguez-Mayo, F. J., García-García, J. A., Escalona, M. J., Mejías, M., Urbieta, M., & Rossi, G. (2015). A framework and tool to manage Cloud Computing service quality. Software Quality Journal, 23(4), 595-625.

Duarte Filho, N. F., Padua, C. I. P. S, Bermejo, P. H. S., Zambalde, A. L., & Barros, U. S. (2013). Saasquality - a method for quality evaluation of software as a service (Saas). International Journal of Computer Science and Information Technology, 5(3), 101-117.

Freitas, A. L. P., & Albernaz, C. M. R. M. (2012). A multicriteria approach for assessing the quality of information technology support services. International Journal of Business and Social Science, 3(16), 59-72. Retrieved in 2017, March 27, from

Freitas, A. L. P., Manhães, N. R. C., & Cozendey, M. I. (2006, October 9-11). Using SERVQUAL to evaluate the quality of information technology services: an experimental analysis. In Proceedings of the XII International Conference on Industrial Engineering and Operations Management (pp. 1-8). Fortaleza: ICIEOM.

Ganesan, R., Sarkar, S., & Tewari, N. (2012, June 25-28). An independent verification of errors and vulnerabilities in SaaS cloud. In Proceedings of the IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN 2012) (pp. 1-6). Boston, MA, USA: IEEE.

Gao, J., Pattabhiraman, P., Bai, X., & Tsai, W. T. (2011, December 12-14). SaaS performance and scalability evaluation in clouds. In Proceedings of the 2011 IEEE 6th International Symposium on Service Oriented System (SOSE) (pp. 61-71). Irvine, CA: IEEE.

Godse, M., & Mulik, S. (2009, September 21-25). An approach for selecting Software-as-a-Service (SaaS) Product. In Proceedings of the IEEE International Conference on Cloud Computing (pp. 155-158). Washington: IEEE Computer Society.

Goode, S., Lin, C., Tsai, J. C., & Jiang, J. J. (2015). Rethinking the role of security in client satisfaction with Software-as-a-Service (SaaS) providers. Decision Support Systems, 70, 73-85.

Goscinski, A., & Brock, M. (2010). Toward dynamic and attribute based publication, discovery and selection for cloud computing. Future Generation Computer Systems, 26(7), 947-970.

Gupta, P., Seetharaman, A., & Raj, J. R. (2013). The usage and adoption of cloud computing by small and medium businesses. International Journal of Information Management, 33(5), 861-874.

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th ed.). New Jersey: Pearson Prentice Hall.

Iacobucci, D., Ostrom, A., & Grayson, K. (1995). Distinguishing service quality and customer satisfaction : the voice of the consumer. Journal of Consumer Psychology, 4(3), 277-303.

Ju, J., Wang, Y., Fu, J., Wu, J., & Lin, Z. (2010, June 22-23). Research on Key Technology in SaaS. In Proceedings of the 2010 International Conference on Intelligent Computing and Cognitive Informatics (pp. 384-387). Kuala Lumpur, Malaysia: IEE.

Kang, S., Myung, J., Yeon, J., Ha, S. W., Cho, T., Chung, J. M., & Lee, S. G. (2010). A general maturity model and reference architecture for SaaS service. In H. Kitagawa, Y. Ishikawa, Q. Li, & C. Watanabe (Eds.), DASFAA 2010: Database Systems for Advanced Applications (Lecture Notes in Computer Science, vol 5982, pp. 337-346). Heidelberg: Springer.

Karadsheh, L. (2012). Applying security policies and service level agreement to IaaS service model to enhance security and transition. Computers & Security, 31(3), 315-326.

Kim, D. (2010). An integrated framework of HoQ and AHP for the QOE improvement of network-based ASP services. Annales des Télécommunications, 65(1-2), 19-29.

Kuo, Y.-F., Wu, C.-M., & Deng, W.-J. (2009). The relationships among service quality, perceived value, customer satisfaction, and post-purchase intention in mobile value-added services. Computers in Human Behavior, 25(4), 887-896.

Lee, H., Lee, Y., & Yoo, D. (2000). The determinants of perceived service quality and its relationship with satisfaction. Journal of Services Marketing, 14(3), 217-231.

Lee, J. Y., Lee, J. W., Cheun, D. W., & Kim, S. D. (2009, December 2-4). A quality model for evaluating Software-as-a-service in cloud computing. In Proceedings of the 2009 Seventh ACIS International Conference on Software Engineering Research, Management and Applications (pp. 261-266). Haikou, China: IEE.

Lian, J. W., Yen, D. C., & Wang, Y. T. (2014). An exploratory study to understand the critical factors affecting the decision to adopt cloud computing in Taiwan hospital. International Journal of Information Management, 34(1), 28-36.

Loiacono, E. T., Watson, R. T., & Goodhue, D. L. (2002). WEBQUAL: a measure of website quality. In K. R. Evans & L. K. Scheer (Eds.), Proceedings of the American Marketing Association: Winter Marketing Educators’ Conference (pp. 432-438). Chicago: American Marketing Association.

Loukis, E., & Charalabidis, Y. (2011). Why do eGovernment projects fail? Risk factors of large information systems projects in the greek public sector. International Journal of Electronic Government Research, 7(2), 59-77.

Low, C., Chen, Y., & Wu, M. (2011). Understanding the determinants of cloud computing adoption. Industrial Management & Data Systems, 111(7), 1006-1023.

Mahesh, S., Landry, B. J. L., Sridhar, T., & Walsh, K. R. (2011). A decision table for the cloud computing decision in small business. Information Resources Management Journal, 24(3), 9-25.

Malhotra, N. K. (2007). Marketing research: an applied orientation (5th ed.). New Jersey: Pearson Prentice Hall.

Mesquida, A. L., Mas, A., Amengual, E., & Calvo-Manzano, J. A. (2012). IT service management process improvement based on ISO/IEC 15504: a systematic review. Information and Software Technology, 54(3), 239-247.

Parasuraman, A., Grewal, D., & Krishnan, R. (2004). Marketing research. Boston: Houghton Mifflin Company.

Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1985). A conceptual model of service quality and its implications for future research. Journal of Marketing, 49(4), 41-50.

Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing, 64(1), 12-40.

Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1994). Reassessment of expectations as a comparison standard in measuring service quality: implications for further research. Journal of Marketing, 58(1), 111-124.

Parasuraman, A., Zeithaml, V. A., & Malhotra, A. (2005). E-S-QUAL: a multiple-item scale for assessing electronic service quality. Journal of Service Research, 7(Feb), 1-21.

Peppard, J. (2003). Managing IT as a portfolio of services. European Management Journal, 21(4), 467-483.

Raciti, M. M., Ward, T., & Dagger, T. S. (2013). The effect of relationship desire on consumer‐to‐business relationships. European Journal of Marketing, 47(3/4), 615-634.

Rimal, B. P., Choi, E., & Lumb, I. (2009, August 25-27). A taxonomy and survey of cloud computing systems. In Proceedings of the Fifth International Joint Conference on INC, IMS and IDC (pp. 44-51). Washington: IEEE Computer Society.

Roses, L. K., Hoppen, N., & Henrique, J. L. (2009). Management of perceptions of information technology service quality. Journal of Business Research, 62(9), 876-882.

Sahinoglu, M., & Cueva-Parra, L. (2011). CLOUD computing. Wiley Interdisciplinary Reviews: Computational Statistics, 3(1), 47-68.

Sureshchandar, G. S., Rajendran, C., & Anantharaman, R. N. (2002). The relationship between service quality and customer satisfaction – a factor specific approach. Journal of Services Marketing, 16(4), 363-379.

Teas, R. K. (1993). Expectations, performance evaluation, and consumers’ perceptions of quality. Journal of Marketing, 57(4), 18.

Ueltschy, L. C., Laroche, M., Eggert, A., & Bindl, U. (2007). Service quality and satisfaction: an international comparison of professional services perceptions. Journal of Services Marketing, 21(6), 410-423.

Walz, J., & Grier, D. A. (2010). Time to push the cloud. IT Professional, 12(5), 14-16.

Wei, Z. W. Z. (2010, November 4-6). An initial review of cloud computing services research development. In Proceedings of the 2010 International Conference on Multimedia Information Networking and Security. Nanjing, Jiangsu: IEEE.

Wen, P. X., & Dong, L. (2013, September 9-11). Quality model for evaluating SaaS service. In Proceedings of the 4th International Conference on Emerging Intelligent Data and Web Technologies (pp. 83-87). Xi'an, China: EIDWT.

Wolfinbarger, M., & Gilly, M. C. (2003). eTailQ: dimensionalizing, measuring and predicting etail quality. Journal of Retailing, 79(3), 183-198.

Yang, Z., Sun, J., Zhang, Y., & Wang, Y. (2015). Computers in human behavior understanding SaaS adoption from the perspective of organizational users: a tripod readiness model. Computers in Human Behavior, 45, 254-264.

Yee, R. W. Y., Yeung, A. C. L., & Edwin Cheng, T. C. (2010). An empirical study of employee loyalty, service quality and firm performance in the service industry. International Journal of Production Economics, 124(1), 109-120.

Zeithaml, V. A., Bitner, M. J., & Gremler, D. D. (2006). Services marketing: integrating customer focus across the firm (4th ed.). Boston: McGraw-Hill/Irwin.

Zhang, Q., Cheng, L., & Boutaba, R. (2010). Cloud computing: state-of-the-art and research challenges. Journal of Internet Services and Applications, 1(1), 7-18.

Zhou, X., Yi, L., & Liu, Y. (2011, July 10-12). A collaborative requirement elicitation technique for SaaS applications. In Proceedings of the 2011 IEEE International Conference on Service Operations, Logistics and Informatics (pp. 83-88). Beijing, China: IEEE.

5b86d9a70e8825d912e4c89d production Articles
Links & Downloads


Share this page
Page Sections