Flow Experience as a Quality Measure in Evaluating Physically Activating Collaborative Serious Games
DOI:
https://doi.org/10.17083/ijsg.v1i3.23Keywords:
Game design, Flow experience, Exergame, Quality measurement, CollaborationAbstract
The measurement of the subjective playing experience is important part of the game development process. The enjoyment level that a serious game offers is a key factor in determining whether a player will be engaged in the gameplay and achieve the objectives of the game. In this paper we report the results of a game design process in which two prototypes of a collaborative exergame were studied. The main aim of the paper is to explore to what extend the measurement of flow experience can facilitate the game evaluation and design process. Alltogether 102 junior high school students participated in two user experience studies and played collaborative exergames designed to teach soft skills. Playing experience was measured with a flow questionnaire, playing behavior was observed and some of the players were interviewed. The results showed that flow experience can be used to evaluate the overall quality of the gameplay and it provides a structured approach to consider the quality of the game. However, flow does not provide detailed information about the shortages of the game and thus complementary methods is needed to identify the causes. The results also indicated that flow experience was independent of gender that supports its use in quality measurement.References
[1] Procci, K., Singer, A. R., Levy, K. R., Bowers, C., “Measuring the flow experience of gamers: An evaluation of the DFS-2”. Computers in Human Behavior, 28(6), 2306-2312, 2012. http://dx.doi.org/10.1016/j.chb.2012.06.039
[2] Wirth, W., Hartmann, T,. Bocking, S., Vorderer, P., Klimmt, C., Schramm., H., Saari, T., Laarni, J., Ravaja, N., Gouveia, F. R., Biocca, F., Sacau, A., Jäncke, L., Baumgartner, T., Jäncke, P., “A process model of the formation of spatial presence experiences”. Media Psychology, 9, pp. 493-525, 2007. http://dx.doi.org/10.1080/15213260701283079
[3] Kiili, K., de Freitas, S., Arnab, S. Lainema, T., “The Design Principles for Flow Experience in Educational Games”. Procedia Computer Science, 15, 78-91, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.060
[4] Cowley, A. D., Minnaar, G., “New generation computer games: watch out for Wii shoulder”, BMJ: British Medical Journal, 336(7636), 110, 2008. http://dx.doi.org/10.1136/bmj.39461.631181.BE
[5] Sweetser, P., Wyeth, P., “GameFlow: a model for evaluating player enjoyment in games”. Computers in Entertainment, 3(3), 1-24, 2005. http://dx.doi.org/10.1145/1077246.1077253
[6] Sinclair, J., Hingston, P., & Masek, M., “Exergame development using the dual flow model”. In Proceedings of the Sixth Australasian Conference on Interactive Entertainment (11), ACM, 2009. http://dx.doi.org/10.1145/1746050.1746061
[7] Bressler, D. M., & Bodzin, A. M., “A mixed methods assessment of students' flow experiences during a mobile augmented reality science game”. Journal of Computer Assisted Learning, 29(6), 505-517, 2013. http://dx.doi.org/10.1111/jcal.12008
[8] Barzilai, S., & Blau, I., “Scaffolding game-based learning: Impact on learning achievements, perceived learning, and game experiences”. Computers & Education, 70, 65-79, 2014. http://dx.doi.org/10.1016/j.compedu.2013.08.003
[9] Liu, T. Y., “Using educational games and simulation software in a computer science course: learning achievements and student flow experiences”. Interactive Learning Environments, 1-21, 2014 (ahead-of-print).
[10] Csikszentmihalyi, M., “Flow: The Psychology of Optimal Experience”. Harper Perennial, New York, 1991.
[11] Hsieh, Y. H., Lin, Y. C., & Hou, H. T., “Exploring the role of flow experience, learning performance and potential behavior clusters in elementary students' game-based learning”. Interactive Learning Environments, 1-16, 2013 (ahead-of-print).
[12] Ma, Y., Williams, D., & Prejean, L., “Designing an Electronic Educational Game to Facilitate Immersion and Flow”. Journal of Interactive Learning Research, 25(1), 27-49, 2014.
[13] Hou, H. T., & Li, M. C., “Evaluating multiple aspects of a digital educational problem-solving-based adventure game”. Computers in Human Behavior, 30, 29-38, 2014. http://dx.doi.org/10.1016/j.chb.2013.07.052
[14] Ketamo, H., Kiili, K., “Conceptual change takes time: Game based learning cannot be only supplementary amusement”. Journal of Educational Multimedia and Hypermedia, 19(4), 399-419, 2010.
[15] Miller, G., “The magical number seven, plus or minus two: some limits on our capacity for processing information”. Psychological Review, 63, 81-97, 1956. http://dx.doi.org/10.1037/h0043158
[16] Kiili, K., Perttula, A., “A design framework for educational exergames”, In Freitas S., Ott, M, Popescu, M.M. and Stanescu, I. (Eds.), New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, US: IGI Global, 2013. http://dx.doi.org/10.4018/978-1-4666-3950-8.ch008
[17] Mueller, F., Edge, D., Vetere, F., Gibbs, M. R., Agamanolis, S., Bongers, B., & Sheridan, J. G., “Designing sports: A framework for exertion games”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, Canada: ACM Press, 2011. http://dx.doi.org/10.1145/1978942.1979330
[18] Connolly, C. T., & Tenenbaum, G., “Exertion–attention–flow linkage under different workloads”. Journal of Applied Social Psychology, 40(5), 1123–1145, 2010. http://dx.doi.org/10.1111/j.1559-1816.2010.00613.x
[19] Fogg, B. J., “Persuasive technology: Using computers to change what we think and do”. San Francisco, CA: Morgan Kaufmann, 2002.
[20] Sweller, J., Van Merriënboer, J., & Paas, F., “Cognitive architecture and instructional design”. Educational Psychology Review, 10(3), 251–296, 1998. http://dx.doi.org/10.1023/A:1022193728205
[21] Tenenbaum G., “A social-cognitive perspective of perceived exertion and exertion tolerance”. In: Singer, R.N.,Hausenblas, H. and Janelle, C. (Eds.), Handbook of sport psychology, New York: Wiley, 2001.
[22] Tenenbaumb, G., Connollya, T.C., “Attention allocation under varied workload and effort perception in rowers”. Psychology of Sport and Exercise, 9(5), 704-717, 2008. http://dx.doi.org/10.1016/j.psychsport.2007.09.002
[23] Scott, L. M., Scott, D., Bedic, S. P., Dowd, J., “The effect of associative and dissociative strategies on rowing ergometer performance”. The Sport Psychologist, 13, 57-68, 1999. http://dx.doi.org/10.1123/tsp.13.1.57
[24] Zagal, J. P., Rick, J., & Hsi, I., “Collaborative games: Lessons learned from board games”. Simulation & Gaming, 37(1), 24-40, 2006. http://dx.doi.org/10.1177/1046878105282279
[25] Azadegan, A., & Harteveld, C., “Work for or Against Players: On the Use of Collaboration Engineering for Collaborative Games”. In proceedings of the 9th International Conference on the Foundations of Digital Games, 2014.
[26] Kiili, K, Tuomi, P. & Perttula, A., “Exerbraining for schools: Combining body and brain Training.”. Procedia Computer Science, 15, 163-173, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.068
[27] Sedano, C. I., Carvalho, M. B., Secco, N., and Longstreet, C. S., “Collaborative and cooperative games: Facts and assumptions”. In Proceedings of the International Conference on Collaboration Technologies and Systems (370-376), 2013.
[28] Malone, T., Lepper, M., “Making learning fun: a taxonomy of intrinsic motivation for learning”. In Snow R. E. and Farr, M. J. (Eds.), Aptitude, Learning, and Instruction, Hillsdale, N. J: Lawrence Earlbaum, 1987.
[29] Bruffee, K., “Sharing our toys- Cooperative learning versus collaborative learning”. Change, 12-18, 1995. http://dx.doi.org/10.1080/00091383.1995.9937722
[30] Romero, M., Usart, M., Ott, M., Earp, J., de Freitas, S., Arnab, S., “Learning through playing For or against each other?”. Promoting Collaborative Learning in Digital Game Based Learning, ECIS 2012.
[31] Moisant, P., “Course sans Gagnant”. Retrieved 05-08, 2014, from http://www.patrickmoisan.net/copains/course_sans_gagnant.html, 2005.
[32] Rocha, J. B., Mascarenhas, S., Prada, R., “Game Mechanics for Cooperative Games”. In ZDN Digital Game, 2008.
[33] March, S.T., Smith, G.F., “Design and natural science research on information technology”. Decision Support Systems, 15, 251-266, 1995. http://dx.doi.org/10.1016/0167-9236(94)00041-2
[34] Mueller, F., & Isbister, K., “Movement-based game guidelines”. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (2191-2200), ACM, 2014. http://dx.doi.org/10.1145/2556288.2557163
[35] Weber, R., Tamborini, R., Westcott-Baker, A. Kantor B., “Theorizing flow and media enjoyment as cognitive synchronization of attentional and reward networks”. Communication Theory, 19, 397-422, 2009.
[36] Kiili, K., Lainema, T., ”Foundation for Measuring Engagement in Educational Games”. Journal of Interactive Learning Research, 19(3), 469-488, 2008.
[37] Jackson, S. A., Eklund, R. C., “Assessing flow in physical activity: The Flow State Scale-2 and Dispositional Flow Scale-2”. Journal of Sport & Exercise Psychology, 24(2), 2002. http://dx.doi.org/10.1123/jsep.24.2.133
[38] Pasch, M., Bianchi-Berthouze, N., van Dijk, B., & Nijholt, A., “Movement-based sports video games: Investigating motivation and gaming experience”. Entertainment Computing, 1(2), 49-61, 2009. http://dx.doi.org/10.1016/j.entcom.2009.09.004
[39] Kiili, K., Lainema, T., “Power and Flow Experience in Time-Intensive Business Simulation Game”. Journal of Educational Multimedia and Hypermedia, 19(1), 39-57, 2010.
[40] Brom, C., Bromová, E., D?cht?renko, F., Buchtová, M., & Pergel, M., “Personalized messages in a brewery educational simulation: Is the personalization principle less robust than previously thought?”. Computers & Education, 72, 339-366, 2014. http://dx.doi.org/10.1016/j.compedu.2013.11.013
[41] Fu, F. L., Su, R. C., & Yu, S. C., “EGameFlow: A scale to measure learners’ enjoyment of e-learning games”. Computers & Education, 52(1), 101-112, 2009. http://dx.doi.org/10.1016/j.compedu.2008.07.004
[42] Hsiao, H. S., Chang, C. S., Lin, C. Y., & Hu, P. M., “Development of children's creativity and manual skills within digital game-based learning environment”. Journal of Computer Assisted Learning, 2014. http://dx.doi.org/10.1111/jcal.12057
[43] Kiili, K., Perttula, A., Tuomi, P., “Development of multiplayer exergames for physical education”. IADIS International Journal on WWW/Internet, 8(1), 52-69, 2010.
[44] Lai, Y. C., Wang, S. T., & Yang, J. C., “An investigation of the exergames experience with flow state, enjoyment, and physical fitness”, In proceedings of the Conference on Advanced Learning Technologies (ICALT) (58-60). IEEE, 2012. http://dx.doi.org/10.1109/icalt.2012.183
[2] Wirth, W., Hartmann, T,. Bocking, S., Vorderer, P., Klimmt, C., Schramm., H., Saari, T., Laarni, J., Ravaja, N., Gouveia, F. R., Biocca, F., Sacau, A., Jäncke, L., Baumgartner, T., Jäncke, P., “A process model of the formation of spatial presence experiences”. Media Psychology, 9, pp. 493-525, 2007. http://dx.doi.org/10.1080/15213260701283079
[3] Kiili, K., de Freitas, S., Arnab, S. Lainema, T., “The Design Principles for Flow Experience in Educational Games”. Procedia Computer Science, 15, 78-91, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.060
[4] Cowley, A. D., Minnaar, G., “New generation computer games: watch out for Wii shoulder”, BMJ: British Medical Journal, 336(7636), 110, 2008. http://dx.doi.org/10.1136/bmj.39461.631181.BE
[5] Sweetser, P., Wyeth, P., “GameFlow: a model for evaluating player enjoyment in games”. Computers in Entertainment, 3(3), 1-24, 2005. http://dx.doi.org/10.1145/1077246.1077253
[6] Sinclair, J., Hingston, P., & Masek, M., “Exergame development using the dual flow model”. In Proceedings of the Sixth Australasian Conference on Interactive Entertainment (11), ACM, 2009. http://dx.doi.org/10.1145/1746050.1746061
[7] Bressler, D. M., & Bodzin, A. M., “A mixed methods assessment of students' flow experiences during a mobile augmented reality science game”. Journal of Computer Assisted Learning, 29(6), 505-517, 2013. http://dx.doi.org/10.1111/jcal.12008
[8] Barzilai, S., & Blau, I., “Scaffolding game-based learning: Impact on learning achievements, perceived learning, and game experiences”. Computers & Education, 70, 65-79, 2014. http://dx.doi.org/10.1016/j.compedu.2013.08.003
[9] Liu, T. Y., “Using educational games and simulation software in a computer science course: learning achievements and student flow experiences”. Interactive Learning Environments, 1-21, 2014 (ahead-of-print).
[10] Csikszentmihalyi, M., “Flow: The Psychology of Optimal Experience”. Harper Perennial, New York, 1991.
[11] Hsieh, Y. H., Lin, Y. C., & Hou, H. T., “Exploring the role of flow experience, learning performance and potential behavior clusters in elementary students' game-based learning”. Interactive Learning Environments, 1-16, 2013 (ahead-of-print).
[12] Ma, Y., Williams, D., & Prejean, L., “Designing an Electronic Educational Game to Facilitate Immersion and Flow”. Journal of Interactive Learning Research, 25(1), 27-49, 2014.
[13] Hou, H. T., & Li, M. C., “Evaluating multiple aspects of a digital educational problem-solving-based adventure game”. Computers in Human Behavior, 30, 29-38, 2014. http://dx.doi.org/10.1016/j.chb.2013.07.052
[14] Ketamo, H., Kiili, K., “Conceptual change takes time: Game based learning cannot be only supplementary amusement”. Journal of Educational Multimedia and Hypermedia, 19(4), 399-419, 2010.
[15] Miller, G., “The magical number seven, plus or minus two: some limits on our capacity for processing information”. Psychological Review, 63, 81-97, 1956. http://dx.doi.org/10.1037/h0043158
[16] Kiili, K., Perttula, A., “A design framework for educational exergames”, In Freitas S., Ott, M, Popescu, M.M. and Stanescu, I. (Eds.), New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, US: IGI Global, 2013. http://dx.doi.org/10.4018/978-1-4666-3950-8.ch008
[17] Mueller, F., Edge, D., Vetere, F., Gibbs, M. R., Agamanolis, S., Bongers, B., & Sheridan, J. G., “Designing sports: A framework for exertion games”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, Canada: ACM Press, 2011. http://dx.doi.org/10.1145/1978942.1979330
[18] Connolly, C. T., & Tenenbaum, G., “Exertion–attention–flow linkage under different workloads”. Journal of Applied Social Psychology, 40(5), 1123–1145, 2010. http://dx.doi.org/10.1111/j.1559-1816.2010.00613.x
[19] Fogg, B. J., “Persuasive technology: Using computers to change what we think and do”. San Francisco, CA: Morgan Kaufmann, 2002.
[20] Sweller, J., Van Merriënboer, J., & Paas, F., “Cognitive architecture and instructional design”. Educational Psychology Review, 10(3), 251–296, 1998. http://dx.doi.org/10.1023/A:1022193728205
[21] Tenenbaum G., “A social-cognitive perspective of perceived exertion and exertion tolerance”. In: Singer, R.N.,Hausenblas, H. and Janelle, C. (Eds.), Handbook of sport psychology, New York: Wiley, 2001.
[22] Tenenbaumb, G., Connollya, T.C., “Attention allocation under varied workload and effort perception in rowers”. Psychology of Sport and Exercise, 9(5), 704-717, 2008. http://dx.doi.org/10.1016/j.psychsport.2007.09.002
[23] Scott, L. M., Scott, D., Bedic, S. P., Dowd, J., “The effect of associative and dissociative strategies on rowing ergometer performance”. The Sport Psychologist, 13, 57-68, 1999. http://dx.doi.org/10.1123/tsp.13.1.57
[24] Zagal, J. P., Rick, J., & Hsi, I., “Collaborative games: Lessons learned from board games”. Simulation & Gaming, 37(1), 24-40, 2006. http://dx.doi.org/10.1177/1046878105282279
[25] Azadegan, A., & Harteveld, C., “Work for or Against Players: On the Use of Collaboration Engineering for Collaborative Games”. In proceedings of the 9th International Conference on the Foundations of Digital Games, 2014.
[26] Kiili, K, Tuomi, P. & Perttula, A., “Exerbraining for schools: Combining body and brain Training.”. Procedia Computer Science, 15, 163-173, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.068
[27] Sedano, C. I., Carvalho, M. B., Secco, N., and Longstreet, C. S., “Collaborative and cooperative games: Facts and assumptions”. In Proceedings of the International Conference on Collaboration Technologies and Systems (370-376), 2013.
[28] Malone, T., Lepper, M., “Making learning fun: a taxonomy of intrinsic motivation for learning”. In Snow R. E. and Farr, M. J. (Eds.), Aptitude, Learning, and Instruction, Hillsdale, N. J: Lawrence Earlbaum, 1987.
[29] Bruffee, K., “Sharing our toys- Cooperative learning versus collaborative learning”. Change, 12-18, 1995. http://dx.doi.org/10.1080/00091383.1995.9937722
[30] Romero, M., Usart, M., Ott, M., Earp, J., de Freitas, S., Arnab, S., “Learning through playing For or against each other?”. Promoting Collaborative Learning in Digital Game Based Learning, ECIS 2012.
[31] Moisant, P., “Course sans Gagnant”. Retrieved 05-08, 2014, from http://www.patrickmoisan.net/copains/course_sans_gagnant.html, 2005.
[32] Rocha, J. B., Mascarenhas, S., Prada, R., “Game Mechanics for Cooperative Games”. In ZDN Digital Game, 2008.
[33] March, S.T., Smith, G.F., “Design and natural science research on information technology”. Decision Support Systems, 15, 251-266, 1995. http://dx.doi.org/10.1016/0167-9236(94)00041-2
[34] Mueller, F., & Isbister, K., “Movement-based game guidelines”. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (2191-2200), ACM, 2014. http://dx.doi.org/10.1145/2556288.2557163
[35] Weber, R., Tamborini, R., Westcott-Baker, A. Kantor B., “Theorizing flow and media enjoyment as cognitive synchronization of attentional and reward networks”. Communication Theory, 19, 397-422, 2009.
[36] Kiili, K., Lainema, T., ”Foundation for Measuring Engagement in Educational Games”. Journal of Interactive Learning Research, 19(3), 469-488, 2008.
[37] Jackson, S. A., Eklund, R. C., “Assessing flow in physical activity: The Flow State Scale-2 and Dispositional Flow Scale-2”. Journal of Sport & Exercise Psychology, 24(2), 2002. http://dx.doi.org/10.1123/jsep.24.2.133
[38] Pasch, M., Bianchi-Berthouze, N., van Dijk, B., & Nijholt, A., “Movement-based sports video games: Investigating motivation and gaming experience”. Entertainment Computing, 1(2), 49-61, 2009. http://dx.doi.org/10.1016/j.entcom.2009.09.004
[39] Kiili, K., Lainema, T., “Power and Flow Experience in Time-Intensive Business Simulation Game”. Journal of Educational Multimedia and Hypermedia, 19(1), 39-57, 2010.
[40] Brom, C., Bromová, E., D?cht?renko, F., Buchtová, M., & Pergel, M., “Personalized messages in a brewery educational simulation: Is the personalization principle less robust than previously thought?”. Computers & Education, 72, 339-366, 2014. http://dx.doi.org/10.1016/j.compedu.2013.11.013
[41] Fu, F. L., Su, R. C., & Yu, S. C., “EGameFlow: A scale to measure learners’ enjoyment of e-learning games”. Computers & Education, 52(1), 101-112, 2009. http://dx.doi.org/10.1016/j.compedu.2008.07.004
[42] Hsiao, H. S., Chang, C. S., Lin, C. Y., & Hu, P. M., “Development of children's creativity and manual skills within digital game-based learning environment”. Journal of Computer Assisted Learning, 2014. http://dx.doi.org/10.1111/jcal.12057
[43] Kiili, K., Perttula, A., Tuomi, P., “Development of multiplayer exergames for physical education”. IADIS International Journal on WWW/Internet, 8(1), 52-69, 2010.
[44] Lai, Y. C., Wang, S. T., & Yang, J. C., “An investigation of the exergames experience with flow state, enjoyment, and physical fitness”, In proceedings of the Conference on Advanced Learning Technologies (ICALT) (58-60). IEEE, 2012. http://dx.doi.org/10.1109/icalt.2012.183
Downloads
Published
2014-09-01
Issue
Section
Articles
License
IJSG copyright information is provided here.
How to Cite
Flow Experience as a Quality Measure in Evaluating Physically Activating Collaborative Serious Games. (2014). International Journal of Serious Games, 1(3). https://doi.org/10.17083/ijsg.v1i3.23
