Thematic Section - Production Engineering leading the Digital Transformation

Development of a measurement instrument to evaluate integrated management systems and differences in perception: an approach to item response theory and the quality management process

Rafael da Silva Fernandes; Tamyres Rodrigues da Rocha; Jaynne Mendes Coelho; Dalton Francisco de Andrade

Downloads: 0
Views: 416


Paper aims: The first aim is methodological, by developing a conceptual model to describe the internal relationship environment (IRE), the critical factors that impact this environment, the characterization of the parties involved, and their relationships. The second is practical and instrumentalizes the model to measure the effect of differences perceived by internal customers.

Originality: Distinct works focus on the formulation of management systems, successful implementation, or external and market environmental factors, although there is a lack of studies that relate organizational performance to differences in perceived quality between the parties.

Research method: The methodology followed a flow of collection/analysis, of the informational data of the company, sketch of the model and flow of information, exploratory focus group, thematic analysis of content, and confirmatory focus group. Then, the procedure of operationalization of the model.

Main findings: The conceptual model and its instrumentalization describe the apparent relationships between the support team and the operations teams, the underlying relationships of the ERI with the company's management model, and organizational performance.

Implications for theory and practice: In practice, the proposed measurement instrument allows evaluation of the effects of differences in the perceived quality of internal customers.


Assessment requirements, Business management models, Dimensions of quality, Item response theory, Organizational excellence


Andrade, D. F., Tavares, H. R., & da Cunha Valle, R. (2000). Teoria da Resposta ao Item: conceitos e aplicações. Sao Paulo: ABE.

Asif, M., Searcy, C., Zutshi, A., & Ahmad, N. (2011). An integrated management systems approach to corporate sustainability. European Business Review, 23(4), 353-367.

Banta, T. W., Pike, G. R., & Hansen, M. J. (2009). The use of engagement data in accreditation, planning, and assessment. New Directions for Institutional Research, 2009(141), 21-34.

Barratt, T., & Ellem, B. (2019). Temporality and the evolution of GPNs: remaking BHP’s Pilbara iron ore network. Regional Studies, 53(11), 1555-1564.

Basso, V., Andrade, B., Jacovine, L., Silva, E., Alves, R., & Nardelli, A. (2020). Forest management certification in the Americas: difficulties in complying with the requirements of the FSC system. International Forestry Review, 22(2), 169-188.

Battilana, J., Obloj, T., Pache, A.-C., & Sengul, M. (2020). Beyond shareholder value maximization: accounting for financial/social trade-offs in dual-purpose companies. Academy of Management Review, 47(2), 237-258.

Belotti Pedroso, C., Tate, W. L., Lago da Silva, A., & Ribeiro Carpinetti, L. C. (2021). Supplier development adoption: a conceptual model for triple bottom line (TBL) outcomes. Journal of Cleaner Production, 314, 127886.

Bernardo, S. M., Rampasso, I. S., Quelhas, O. L., Leal Filho, W., & Anholon, R. (2022). Method to integrate management tools aiming for organizational excellence. Production, 32, e20210101.

Bhadani, K., Asbjörnsson, G., Hulthén, E., & Evertsson, M. (2020). Development and implementation of key performance indicators for aggregate production using dynamic simulation. Minerals Engineering, 145, 106065.

Bokrantz, J., Skoogh, A., Berlin, C., Wuest, T., & Stahre, J. (2020). Smart Maintenance: a research agenda for industrial maintenance management. International Journal of Production Economics, 224, 107547.

Bouslah, B., Gharbi, A., & Pellerin, R. (2016). Integrated production, sampling quality control, and maintenance of deteriorating production systems with AOQL constraints. Omega, 61, 110-126.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.

Bulkan, J. (2020). Smallholder forestry in the FSC system: a review. Revue Gouvernance/Governance Review, 17(2), 7-29.

Chatterjee, S., Narayanan, V., & Malek, W. (2016). How strategy execution maps guided Cisco System’s Sales Incentive Compensation plan. Strategy and Leadership, 44(6), 25-34.

Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015). Construção de instrumentos de medida na área da saúde. Ciencia & Saude Coletiva, 20(3), 925-936. PMid:25760132.

Cronin Junior, J. J., & Taylor, S. A. (1994). Server versus servqual: reconciling performance-based and perceptions-minus-expectations measurement of service quality. Journal of Marketing, 58(1), 125-131.

De Ayala, R. J. (2013). The theory and practice of item response theory. New York: Guilford Publications.

Drummond, P., Araujo, F., & Borges, R. (2017). Meeting halfway: assessing the differences between the perceptions of ERP implementers and end-users. Business Process Management Journal, 23(5), 936-956.

Elassy, N. (2015). The concepts of quality, quality assurance, and quality enhancement. Quality Assurance in Education, 23(3), 250-261.

Fernandes, R. S., Biffe, B. G., Louzada, M. J. Q., Bornia, A. C., & Andrade, D. F. (2022a). Standardized measure for performance assessment of athletes in the crossfit open: theoretical structuring and item response theory. Research Square, 1-15.

Fernandes, R. S., Luz, R. M. N., Reis, D. C., Luz, M. A. L., & Guimarães, G. V. (2022b). Elaboration of quality perception instrument of remote teaching amidst COVID-19 pandemics in a University of Northern Brazil. Research Square, 1-20.

Fernandes, R. S., Sousa, L. R. C., & Santos, T. L. (2021). Análise, investigação e avaliação da gestão da qualidade no processo de mineração. Revista Produção Online, 21(3), 770-793.

Gackowiec, P., Podobińska-Staniec, M., Brzychczy, E., Kühlbach, C., & Özver, T. (2020). Review of key performance indicators for process monitoring in the mining industry. Energies, 13(19), 5169.

Green, D. (1994). What Is quality in higher education? London: ERIC.

Hadidi, L. A., Turki, U. M. A., & Rahim, A. (2012). Integrated models in production planning and scheduling, maintenance and quality: a review. International Journal of Industrial and Systems Engineering, 10(1), 21-50.

Hezri, A. A., & Dovers, S. R. (2006). Sustainability indicators, policy, and governance: Issues for ecological economics. Ecological Economics, 60(1), 86-99.

Houaiss, A. (2001). Dicionário eletrônico Houaiss da língua portuguesa. Rio de Janeiro: Ed. Objetiva.

International Organization for Standardization – ISO. (2014a). ISO 22400-1. Automation systems and integration — Key performance indicators (KPIs) for manufacturing operations management — Part 1: Overview, concepts, and terminology (pp. 19). Geneva, Switzerland: ISO.

International Organization for Standardization – ISO. (2014b). ISO 22400-2. Automation Systems and Integration - Key Performance Indicators (KPIs) for Manufacturing Operations Management, Part 2: Definitions and Descriptions. Geneva, Switzerland: ISO.

International Organization for Standardization – ISO. (2015). ISO 9001. Quality management systems - Fundamentals and vocabulary (pp. 29). Geneva, Switzerland: ISO.

International Organization for Standardization – ISO. (2017). ISO 22400-2/AMD 1. Automation systems and integration — Key performance indicators (KPIs) for manufacturing operations management — Part 2: Definitions and descriptions — Amendment 1: Key performance indicators for energy management (pp. 10). Geneva, Switzerland: ISO.

International Organization for Standardization – ISO. (2018). ISO/TR 22400-10. Automation systems and integration — Key performance indicators (KPIs) for manufacturing operations management — Part 10: Operational sequence description of data acquisition (pp. 34). Geneva, Switzerland: ISO.

International Organization for Standardization – ISO. (2020). ISO 9000. Quality management systems - Fundamentals and vocabulary (pp. 23). Geneva, Switzerland: ISO.

International Organization for Standardization – ISO. (2021). ISO 10013. Quality management systems. In Guidance for documented information (pp. 14). Geneva, Switzerland: ISO.

Juran, J. M. (1992). Juran on quality by design: the new steps for planning quality into goods and services. New York: Simon and Schuster.

Kang, N., Zhao, C., Li, J., & Horst, J. A. (2016). A Hierarchical structure of key performance indicators for operation management and continuous improvement in production systems. International Journal of Production Research, 54(21), 6333-6350. PMid:29398722.

King, L. O. (2016). Functional sustainability indicators. Ecological Indicators, 66, 121-131.

Lee, R. G., & Dale, B. G. (1998). Business process management: a review and evaluation. Business Process Management Journal, 4(3), 214-225.

Maheswari, C., Priyanka, E., Thangavel, S., Vignesh, S. R., & Poongodi, C. (2020). Multiple regression analysis for the prediction of extraction efficiency in the mining industry with industrial IoT. Production Engineering, 14(4), 457-471.

Mehrabian, A., & Russell, J. A. (1974). An approach to environmental psychology. Cambridge: The MIT Press.

Miles, M. P., & Covin, J. G. (2000). Environmental marketing: a source of reputational, competitive, and financial advantage. Journal of Business Ethics, 23(3), 299-311.

Mohammadi, M., Rai, P., & Gupta, S. (2017). Performance evaluation of bucket-based excavating, loading, and transport (BELT) equipment–an OEE approach. Archives of Mining Sciences, 62(1), 105-120.

Mohsin, M., Zhu, Q., Naseem, S., Sarfraz, M., & Ivascu, L. (2021). Mining industry impact on environmental sustainability, economic growth, social interaction, and public health: an application of semi-quantitative mathematical approach. Processes (Basel, Switzerland), 9(6), 972.

Nairn, R. W., LaBar, J. A., Oxenford, L. R., Shepherd, N. L., Holzbauer-Schweitzer, B. K., Arango, J. G., Tang, Z., Dorman, D. M., Folz, C. A., & McCann, J. I. (2020). Toward sustainability of passive treatment in legacy mining watersheds: operational performance and system maintenance. In Proceedings of the IMWA 2020 “Mine Water Solutions (pp. 123-128). Christchurch, New Zealand: IMWA.

Newton, P. E. (2017). There is more to educational measurement than measuring: the importance of embracing purpose pluralism. Educational Measurement: Issues and Practice, 36(2), 5-15.

Nicholls, J. A. (2020). Integrating financial, social, and environmental accounting. Sustainability Accounting, Management and Policy Journal, 11(4), 745-769.

Nickel, E. M., Ferreira, M. G. G., Forcellini, F. A., Santos, C. T., & Silva, R. A. Á. (2010). Modelo multicritério para referência na fase de Projeto Informacional do Processo de Desenvolvimento de Produtos. Gestão & Produção, 17(4), 707-720.

Nunes, R. A., Delboni Junior, H., Tomi, G., Infante, C. B., & Allan, B. (2019). A decision-making method to assess the benefits of a semi-mobile in-pit crushing and conveying alternatives during the early stages of a mining project. REM - International Engineering Journal, 72(2), 285-291.

Ory, J. C. (1992). Meta-assessment: evaluating assessment activities. Research in Higher Education, 33(4), 467-481.

Otto, T., & Musingwini, C. (2020). A compliance driver tree (CDT) based approach for improving the alignment of spatial and intertemporal execution with mine planning at open-pit mines. Resources Policy, 69, 101826.

Pan, X., Sinha, P., & Chen, X. (2021). Corporate social responsibility and eco-innovation: the triple bottom line perspective. Corporate Social Responsibility and Environmental Management, 28(1), 214-228.

Parmenter, D. (2015). Key performance indicators: developing, implementing, and using winning KPIs. Hoboken: John Wiley & Sons.

Pascual, R., Madariaga, R., Santelices, G., Godoy, D., & Droguett, E. L. (2016). A structured methodology to optimize the throughput of production lines. International Journal of Mining, Reclamation and Environment, 30(1), 25-36.

Pasquali, L. (2017a). Psicometria: teoria dos testes na psicologia e na educação. São Paulo: Editora Vozes Limitada. Retrieved in 2022, June 1, from

Pasquali, L. (2017b). Validade dos testes. Examen: Política, Gestão e Avaliação da Educação, 1(1), 14-48. Retrieved in 2022, June 1, from

Pasquali, L. (2020). TRI–Teoria de resposta ao item: teoria, procedimentos e aplicações. Curitiba: Editora Appris.

Peral, J., Maté, A., & Marco, M. (2017). Application of data mining techniques to identify relevant key performance indicators. Computer Standards & Interfaces, 54, 76-85.

Pereira, L., & Nunes, N. (2018). Performance evaluation in non‐intrusive load monitoring: datasets, metrics, and tools—A review. Wiley Interdisciplinary Reviews. Data Mining and Knowledge Discovery, 8(6), e1265.

Petrick, J. F. (2002). Development of a multi-dimensional scale for measuring the perceived value of a service. Journal of Leisure Research, 34(2), 119-134.

PMBOK Guide. (2021). A Guide to the Project Management Body of Knowledge (PMBOK guide) (pp. 763). Pennsylvania: Project Management Institute/Newtown Square.

Qiao, G., Schlenoff, C., & Weiss, B. A. (2017). Quick positional health assessment for industrial robot prognostics and health management (PHM). In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA). USA: IEEE.

Quigley, M., & Dimitrakopoulos, R. (2020). Incorporating geological and equipment performance uncertainty while optimizing short-term mine production schedules. International Journal of Mining, Reclamation and Environment, 34(5), 362-383.

Rocha, C. S., Cardoso, E. L. S. S., Fernandes, R. S., Branco, N. C. N. M., & Luz, R. M. N. (2021). Quality assessment of divergences caused in weighing copper ore at a mining company in the amazon region. International Journal of Developmental Research, 11(8), 49633-49639.

Robertson, J. (1996). Review of: “ Product Design Fundamentals and Methods ” by N. F. M. ROOZENBURG and J. EEKELS, Wiley (1995), pp, 397, £27.50, ISBN 0-471-95465-9. Ergonomics, 39(7), 992-993.

Sane, R. (2018). Beneficiation and agglomeration of manganese ore fines (an area so important and yet so ignored). IOP Conference Series: Materials Science and Engineering, 285, 012033.

Santos, M. S., Pinto, T. V., Júnior, Ê. L., Cota, L. P., Souza, M. J., & Euzébio, T. A. (2020). Simheuristic-based decision support system for efficiency improvement of an iron ore crusher circuit. Engineering Applications of Artificial Intelligence, 94, 103789.

Shen, X., Chen, L., Xia, S., Xie, Z., & Qin, X. (2018). Burdening proportion and new energy-saving technologies analysis and optimization for iron and steel production system. Journal of Cleaner Production, 172, 2153-2166.

Silva, B. L. F., Santos Neto, I., Fernandes, R. S., Branco, N. C., & Guimarães, G. V. (2021). Dimensionamento e viabilidade econômica de um sistema fotovoltaico: um estudo de caso na UFRA/Parauapebas. Revista Produção Online, 21(3), 863-890.

Singh, R. K., Murty, H. R., Gupta, S. K., & Dikshit, A. K. (2009). An overview of sustainability assessment methodologies. Ecological Indicators, 9(2), 189-212.

Sisodia, R., & Villegas Forero, D. (2019). Quality 4.0 – how to handle quality in the industry 4.0 revolution (Report No. Number E2019:128). Gothenburg, Sweden: Chalmers University of Technology. Retrieved in 2022, June 1, from

Skrzypkowski, K., Korzeniowski, W., Zagórski, K., & Zagórska, A. (2020). Adjustment of the yielding system of mechanical rock bolts for room and pillar mining method in stratified rock mass. Energies, 13(8), 2082.

Soligo, V. (2012). Indicadores: conceito e complexidade do mensurar em estudos de fenômenos sociais. Estudos em Avaliação Educacional, 23(52), 12-25.

Sousa, J. C. C., Fernandes, R. S., Luz, R. M. N., Santos, I. R., Silva, B. L. F., & Santos Neto, I. (2022). Description of the IPCC mining process and analysis of the profile of productivity losses applied by a mining company in northern Brazil. Research Square, 1-25.

Srivastava, A. K., & Sushil. (2015). Modeling organizational and information systems for effective strategy execution. Journal of Enterprise Information Management, 28(4), 556-578.

Srivastava, A. K., & Sushil. (2017). Alignment: the foundation of effective strategy execution. International Journal of Productivity and Performance Management, 66(8), 1043-1063.

Wårell, L. (2018). An analysis of iron ore prices during the latest commodity boom. Mineral Economics, 31(1), 203-216.

Xu, L., Peng, X., Pavur, R., & Prybutok, V. (2020). Quality management theory development via meta-analysis. International Journal of Production Economics, 229, 107759.

Zanon, C., Hutz, C. S., Yoo, H. H., & Hambleton, R. K. (2016). An application of item response theory to psychological test development. Psicologia: Reflexão e Crítica, 29(1), 18.

Submitted date:

Accepted date:

63d3f89da953951dcb0fc9d4 production Articles
Links & Downloads


Share this page
Page Sections