Flow Experience as a Quality Measure in Evaluating Physically Activating Collaborative Serious Games

Kristian J. M. Kiili, Arttu Perttula, Antero Lindstedt, Sylester Arnab, Marko Suominen


The measurement of the subjective playing experience is important part of the game development process. The enjoyment level that a serious game offers is a key factor in determining whether a player will be engaged in the gameplay and achieve the objectives of the game. In this paper we report the results of a game design process in which two prototypes of a collaborative exergame were studied. The main aim of the paper is to explore to what extend the measurement of flow experience can facilitate the game evaluation and design process. Alltogether 102 junior high school students participated in two user experience studies and played collaborative exergames designed to teach soft skills. Playing experience was measured with a flow questionnaire, playing behavior was observed and some of the players were interviewed. The results showed that flow experience can be used to evaluate the overall quality of the gameplay and it provides a structured approach to consider the quality of the game. However, flow does not provide detailed information about the shortages of the game and thus complementary methods is needed to identify the causes. The results also indicated that flow experience was independent of gender that supports its use in quality measurement.


Game design; Flow experience; Exergame; Quality measurement; Collaboration

Full Text:



Procci, K., Singer, A. R., Levy, K. R., Bowers, C., “Measuring the flow experience of gamers: An evaluation of the DFS-2”. Computers in Human Behavior, 28(6), 2306-2312, 2012. http://dx.doi.org/10.1016/j.chb.2012.06.039

Wirth, W., Hartmann, T,. Bocking, S., Vorderer, P., Klimmt, C., Schramm., H., Saari, T., Laarni, J., Ravaja, N., Gouveia, F. R., Biocca, F., Sacau, A., Jäncke, L., Baumgartner, T., Jäncke, P., “A process model of the formation of spatial presence experiences”. Media Psychology, 9, pp. 493-525, 2007. http://dx.doi.org/10.1080/15213260701283079

Kiili, K., de Freitas, S., Arnab, S. Lainema, T., “The Design Principles for Flow Experience in Educational Games”. Procedia Computer Science, 15, 78-91, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.060

Cowley, A. D., Minnaar, G., “New generation computer games: watch out for Wii shoulder”, BMJ: British Medical Journal, 336(7636), 110, 2008. http://dx.doi.org/10.1136/bmj.39461.631181.BE

Sweetser, P., Wyeth, P., “GameFlow: a model for evaluating player enjoyment in games”. Computers in Entertainment, 3(3), 1-24, 2005. http://dx.doi.org/10.1145/1077246.1077253

Sinclair, J., Hingston, P., & Masek, M., “Exergame development using the dual flow model”. In Proceedings of the Sixth Australasian Conference on Interactive Entertainment (11), ACM, 2009. http://dx.doi.org/10.1145/1746050.1746061

Bressler, D. M., & Bodzin, A. M., “A mixed methods assessment of students' flow experiences during a mobile augmented reality science game”. Journal of Computer Assisted Learning, 29(6), 505-517, 2013. http://dx.doi.org/10.1111/jcal.12008

Barzilai, S., & Blau, I., “Scaffolding game-based learning: Impact on learning achievements, perceived learning, and game experiences”. Computers & Education, 70, 65-79, 2014. http://dx.doi.org/10.1016/j.compedu.2013.08.003

Liu, T. Y., “Using educational games and simulation software in a computer science course: learning achievements and student flow experiences”. Interactive Learning Environments, 1-21, 2014 (ahead-of-print).

Csikszentmihalyi, M., “Flow: The Psychology of Optimal Experience”. Harper Perennial, New York, 1991.

Hsieh, Y. H., Lin, Y. C., & Hou, H. T., “Exploring the role of flow experience, learning performance and potential behavior clusters in elementary students' game-based learning”. Interactive Learning Environments, 1-16, 2013 (ahead-of-print).

Ma, Y., Williams, D., & Prejean, L., “Designing an Electronic Educational Game to Facilitate Immersion and Flow”. Journal of Interactive Learning Research, 25(1), 27-49, 2014.

Hou, H. T., & Li, M. C., “Evaluating multiple aspects of a digital educational problem-solving-based adventure game”. Computers in Human Behavior, 30, 29-38, 2014. http://dx.doi.org/10.1016/j.chb.2013.07.052

Ketamo, H., Kiili, K., “Conceptual change takes time: Game based learning cannot be only supplementary amusement”. Journal of Educational Multimedia and Hypermedia, 19(4), 399-419, 2010.

Miller, G., “The magical number seven, plus or minus two: some limits on our capacity for processing information”. Psychological Review, 63, 81-97, 1956. http://dx.doi.org/10.1037/h0043158

Kiili, K., Perttula, A., “A design framework for educational exergames”, In Freitas S., Ott, M, Popescu, M.M. and Stanescu, I. (Eds.), New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, US: IGI Global, 2013. http://dx.doi.org/10.4018/978-1-4666-3950-8.ch008

Mueller, F., Edge, D., Vetere, F., Gibbs, M. R., Agamanolis, S., Bongers, B., & Sheridan, J. G., “Designing sports: A framework for exertion games”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, Canada: ACM Press, 2011. http://dx.doi.org/10.1145/1978942.1979330

Connolly, C. T., & Tenenbaum, G., “Exertion–attention–flow linkage under different workloads”. Journal of Applied Social Psychology, 40(5), 1123–1145, 2010. http://dx.doi.org/10.1111/j.1559-1816.2010.00613.x

Fogg, B. J., “Persuasive technology: Using computers to change what we think and do”. San Francisco, CA: Morgan Kaufmann, 2002.

Sweller, J., Van Merriënboer, J., & Paas, F., “Cognitive architecture and instructional design”. Educational Psychology Review, 10(3), 251–296, 1998. http://dx.doi.org/10.1023/A:1022193728205

Tenenbaum G., “A social-cognitive perspective of perceived exertion and exertion tolerance”. In: Singer, R.N.,Hausenblas, H. and Janelle, C. (Eds.), Handbook of sport psychology, New York: Wiley, 2001.

Tenenbaumb, G., Connollya, T.C., “Attention allocation under varied workload and effort perception in rowers”. Psychology of Sport and Exercise, 9(5), 704-717, 2008. http://dx.doi.org/10.1016/j.psychsport.2007.09.002

Scott, L. M., Scott, D., Bedic, S. P., Dowd, J., “The effect of associative and dissociative strategies on rowing ergometer performance”. The Sport Psychologist, 13, 57-68, 1999. http://dx.doi.org/10.1123/tsp.13.1.57

Zagal, J. P., Rick, J., & Hsi, I., “Collaborative games: Lessons learned from board games”. Simulation & Gaming, 37(1), 24-40, 2006. http://dx.doi.org/10.1177/1046878105282279

Azadegan, A., & Harteveld, C., “Work for or Against Players: On the Use of Collaboration Engineering for Collaborative Games”. In proceedings of the 9th International Conference on the Foundations of Digital Games, 2014.

Kiili, K, Tuomi, P. & Perttula, A., “Exerbraining for schools: Combining body and brain Training.”. Procedia Computer Science, 15, 163-173, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.068

Sedano, C. I., Carvalho, M. B., Secco, N., and Longstreet, C. S., “Collaborative and cooperative games: Facts and assumptions”. In Proceedings of the International Conference on Collaboration Technologies and Systems (370-376), 2013.

Malone, T., Lepper, M., “Making learning fun: a taxonomy of intrinsic motivation for learning”. In Snow R. E. and Farr, M. J. (Eds.), Aptitude, Learning, and Instruction, Hillsdale, N. J: Lawrence Earlbaum, 1987.

Bruffee, K., “Sharing our toys- Cooperative learning versus collaborative learning”. Change, 12-18, 1995. http://dx.doi.org/10.1080/00091383.1995.9937722

Romero, M., Usart, M., Ott, M., Earp, J., de Freitas, S., Arnab, S., “Learning through playing For or against each other?”. Promoting Collaborative Learning in Digital Game Based Learning, ECIS 2012.

Moisant, P., “Course sans Gagnant”. Retrieved 05-08, 2014, from http://www.patrickmoisan.net/copains/course_sans_gagnant.html, 2005.

Rocha, J. B., Mascarenhas, S., Prada, R., “Game Mechanics for Cooperative Games”. In ZDN Digital Game, 2008.

March, S.T., Smith, G.F., “Design and natural science research on information technology”. Decision Support Systems, 15, 251-266, 1995. http://dx.doi.org/10.1016/0167-9236(94)00041-2

Mueller, F., & Isbister, K., “Movement-based game guidelines”. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (2191-2200), ACM, 2014. http://dx.doi.org/10.1145/2556288.2557163

Weber, R., Tamborini, R., Westcott-Baker, A. Kantor B., “Theorizing flow and media enjoyment as cognitive synchronization of attentional and reward networks”. Communication Theory, 19, 397-422, 2009.

Kiili, K., Lainema, T., ”Foundation for Measuring Engagement in Educational Games”. Journal of Interactive Learning Research, 19(3), 469-488, 2008.

Jackson, S. A., Eklund, R. C., “Assessing flow in physical activity: The Flow State Scale-2 and Dispositional Flow Scale-2”. Journal of Sport & Exercise Psychology, 24(2), 2002. http://dx.doi.org/10.1123/jsep.24.2.133

Pasch, M., Bianchi-Berthouze, N., van Dijk, B., & Nijholt, A., “Movement-based sports video games: Investigating motivation and gaming experience”. Entertainment Computing, 1(2), 49-61, 2009. http://dx.doi.org/10.1016/j.entcom.2009.09.004

Kiili, K., Lainema, T., “Power and Flow Experience in Time-Intensive Business Simulation Game”. Journal of Educational Multimedia and Hypermedia, 19(1), 39-57, 2010.

Brom, C., Bromová, E., Děchtěrenko, F., Buchtová, M., & Pergel, M., “Personalized messages in a brewery educational simulation: Is the personalization principle less robust than previously thought?”. Computers & Education, 72, 339-366, 2014. http://dx.doi.org/10.1016/j.compedu.2013.11.013

Fu, F. L., Su, R. C., & Yu, S. C., “EGameFlow: A scale to measure learners’ enjoyment of e-learning games”. Computers & Education, 52(1), 101-112, 2009. http://dx.doi.org/10.1016/j.compedu.2008.07.004

Hsiao, H. S., Chang, C. S., Lin, C. Y., & Hu, P. M., “Development of children's creativity and manual skills within digital game-based learning environment”. Journal of Computer Assisted Learning, 2014. http://dx.doi.org/10.1111/jcal.12057

Kiili, K., Perttula, A., Tuomi, P., “Development of multiplayer exergames for physical education”. IADIS International Journal on WWW/Internet, 8(1), 52-69, 2010.

Lai, Y. C., Wang, S. T., & Yang, J. C., “An investigation of the exergames experience with flow state, enjoyment, and physical fitness”, In proceedings of the Conference on Advanced Learning Technologies (ICALT) (58-60). IEEE, 2012. http://dx.doi.org/10.1109/icalt.2012.183

DOI: http://dx.doi.org/10.17083/ijsg.v1i3.23


  • There are currently no refbacks.

Serious Games Society

Creative Commons LicenseThe International Journal of Serious Games (IJSG) by Serious Games Society is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.