Validation of Serious Games
DOI:
https://doi.org/10.17083/ijsg.v2i3.75Keywords:
persuasive game design, serious games, RCT, validation, game researchAbstract
The application of games for behavioral change has seen a surge in popularity but evidence on the efficacy of these games is contradictory. Anecdotal findings seem to confirm their motivational value whereas most quantitative findings from randomized controlled trials (RCT) are negative or difficult to interpret. One cause for the contradictory evidence could be that the standard RCT validation methods are not sensitive to serious games’ effects. To be able to adapt validation methods to the properties of serious games we need a framework that can connect properties of serious game design to the factors that influence the quality of quantitative research outcomes. The Persuasive Game Design model [1] is particularly suitable for this aim as it encompasses the full circle from game design to behavioral change effects on the user. We therefore use this model to connect game design features, such as the gamification method and the intended transfer effect, to factors that determine the conclusion validity of an RCT. In this paper we will apply this model to develop guidelines for setting up validation methods for serious games. This way, we offer game designers and researchers handles on how to develop tailor-made validation methods.
References
[1] Visch, V.T., Vegt, N., Anderiessen, H., van der Kooij, K., Persuasive game design: a model and its definitions. CHI. Paris, France, 2014.
[2] Anderson C.A., Shibuya A., Ihori N., Swing E.L., Bushman B.J., et al. (2010) Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: a meta-analytic review. Psychological Bulletin Vol. 136, pp. 151-173, 2014. http://dx.doi.org/10.1037/a0018251
[3] Abt, C.C., Serious Games. New York: University Press of America, Inc, 1987.
[4] Cadamuro A., & Visch V., ‘What Remains?’: A Persuasive Story Telling Game. In Games for Health. Springer Fachmedien Wiesbaden, 2013.
[5] Graafland M., Dankbaar, M., Mert, A., Lagro, J., De Wit-Zuurendonk, L., et al. How to systematically assess serious games applied to health care. JMIR Serious Games, Vol. 2, 2015.
[6] Bogost, I., Persuasive games: The expressive power of videogames: MIT Press, 2007.
[7] Brezinka V., Treasure Hunt - a serious game to support psychotherapeutic treatment of children. Studies in health technology and informatics, Vol., 136, pp. 71-76, 2008.
[8] Coyle D., Matthews, M., Sharry, J., Nisbet, A., Doherty, G., Personal Investigator: a therapeutic 3D game for adolescent psychotherapy. Journal of Interactive Technology & Smart Education, Vol., 2, pp. 73-88, 2005. http://dx.doi.org/10.1108/17415650580000034
[9] Gamberini, L., Barresi, G., Majer, A., Scarpetta, F., A game a day keeps the doctor away: a short review of computer games in mental healthcare. Journal of CyberTherapy & Rehabilitation Vol., 1, pp. 127-149, 2008.
[10] Shandley, K., Austin, D., Klein, B., Kyrios, M., An evaluation of 'Reach Out Central': an online gaming program for supporting the mental health of young people. Health Education Research, Vol., 25, pp. 563-574, 2010. http://dx.doi.org/10.1093/her/cyq002
[11] Thieme, A., Wallace, J., Johnson, P., McCarthy, J., Lindley, S., et al. Design to promote mindfulness practice and sense of self for vulnerable women in secure hospital services. CHI: changing perspectives. Paris, France, 2013. http://dx.doi.org/10.1145/2470654.2481366
[12] DeSmet. A., van Ryckeghem, D., Compernolle, S., Baranowksi, T., Thompson, D., et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Preventive Medicine, Vol., 69, pp. 95-107, 2014. http://dx.doi.org/10.1016/j.ypmed.2014.08.026
[13] Hamari, J., Koivisto, J., Sarsa, H., Does gamification work? - A literature review of empirical studies on gamification. In System Sciences (HICSS), 47th Hawaii International Conference on. IEEE, 2014.
[14] Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T., Boyle, J.M., A systematic literature review of empirical evidence on computer games and serious games. Computer and Education Vol., 59, pp. 661-686, 2012. http://dx.doi.org/10.1016/j.compedu.2012.03.004
[15] Green, M.C., Brock, T.C., Kaufman, G.F., Understanding media enjoyment: The role of transportation into narrative worlds. Communication Theory, Vol., 14, pp. 311-327, 2014. http://dx.doi.org/10.1111/j.1468-2885.2004.tb00317.x
[16] Caillois, R., Les jeux et les hommes. Paris: Librairie Gallimard, 1958.
[17] Huizinga, J., Humo Ludens - Proeve eener bepaling van het spel-element der cultuur. Amsterdam: Amsterdam University Press., (1938 / 2008). http://dx.doi.org/10.5117/9789089640031
[18] Przybylski, A.K., Rigby., C.S., Ryan, R.M., A motivational model of video game engagement. Review of General Psychology Vol., 14, pp. 154-166, 2010. http://dx.doi.org/10.1037/a0019440
[19] Baranowski, T., Buday, R., Thompson, D.I., Baranowski, J., Playing for real: video games and stories for health-related behavior change. American Journal of Preventive Medicine Vol., 34, pp. 74-82, 2008. http://dx.doi.org/10.1016/j.amepre.2007.09.027
[20] Baranowski, T., Baranowksi, J., Thompson, D., Buday, R., Jago, R., et al. Video game play, child diet, and physical activity behavior change. A randomized clinical trial. American Journal of Preventive Medicine Vol., 40., pp. 33-38, 2011. http://dx.doi.org/10.1016/j.amepre.2010.09.029
[21] Bartholomew, L.K., Parcel, G.S., Kok, G., Gottlieb, N.H., Fernandez, M.E., Planning health promotion programs: an intervention mapping approach. New York, NY: John Wiley & Sons, 2011.
[22] Kato, P., Evaluating efficacy and validating health games. In Games for Health: Research, Development, and Clinical Applications; Amsterdam; 2013.
[23] Brahmajee, K., Nallamothu, R., Hayward, E., Bates, E.R., Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies. Circulation Vol., 118, pp. 1294-1303, 2008. http://dx.doi.org/10.1161/CIRCULATIONAHA.107.703579
[24] Granic, I., O'Hara, A., Pepler, D., Lewis, M.D. A dynamic systems analysis of parent-child changes associated with successful "Real-world" interventions for aggressive children. Journal of Abnormal Child Psychology, Vol., 35, pp. 845-857, 2007. http://dx.doi.org/10.1007/s10802-007-9133-4
[25] Laurenceau, J.P., Hayes, A.M., Feldman, G.C., Some methodological and statistical issues in the study of change processes in psychotherapy. Clinical Psychology Review, Vol., 27, pp. 682-695, 2007. http://dx.doi.org/10.1016/j.cpr.2007.01.007
[26] Arnab, S., Brown, K., Clarke, S., Dunwell, I., Lim, T., et al. The development approach of a pedagogically-driven serious game to support Relationship and Sex Education (RSE) within a classroom setting. Computer and Education Vol., 69, pp. 15-30, 2013. http://dx.doi.org/10.1016/j.compedu.2013.06.01
[27] LeBlanc, M., Tools for creating dramatic game dynamics. The game design reader: A rules of play anthology, pp. 438-459, 2006.
[28] Deterding, S., Dixon, D., Khaled, R., Nacke, L., From game design elements to gamefulness: defining "gamification". Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9-15, 2011. http://dx.doi.org/10.1145/2181037.2181040
[29] Garris, R., Ahlers, R., Driskell, J.E., Games, motivation and learning: a research and practice model. Simulation gaming, Vol., 33, pp. 441-467, 2002. http://dx.doi.org/10.1177/1046878102238607
[30] Korhonen, H., Montola, M., Arrasvuori, J., Understanding playful user experience through digital games. International Conference on Designing Pleasurable Products and Interfaces, pp. 274-285, 2009.
[31] McLay, R.N., Wood, D.P., Webb-Murphy, J.A., Spira, J.L., Wiederhold M.D., et al. A randomized, controlled trials of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychology, behavior and social networking, Vol., 14, pp. 223-229, 2011. http://dx.doi.org/10.1089/cyber.2011.0003
[32] Brown, S.J., Lieberman, D.A., Gemeny, B.A., Fan, Y.C., Wilson, D.M., et al. Educational video game for juvenile diabetes: results of a controlled trial. Informatics for Health and Social Care, Vol., 22, pp. 77-89, 1997. http://dx.doi.org/10.3109/14639239709089835
[33] Kato. P.M., Cole, S.W., Bradly, A.S., Pollock, B.H., A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics, Vol., 122, e305, 2007.
[34] Granic, I., Lobel, A., Engels, R.C.M.E., The benefits of playing video games. American Psychologist Vol., 69, pp. 66-78, 2013. http://dx.doi.org/10.1037/a0034857
[35] Green, C.S., Bavellier, D. Action video game modifies visual selective attention. Nature, Vol., 423, pp. 534-537, 2003. http://dx.doi.org/10.1038/nature01647
[36] Cole, S.W., Yoo, D.J., Knutson, B. Interactivity and reward-related neural activation during a serious videogame. PLOS One, Vol., 7, 2012.
[37] Anguera, J.A., Boccanfuso, J., Rintoul, J.L., Al-Hashimi, O., Faraji, F., et al., Video game training enhances cognitive control in older adults. Nature, Vol., 501, pp. 97-101, 2013. http://dx.doi.org/10.1038/nature12486
[38] Starks, K., Cognitive behavioral game design: a unified model for designing serious games. Frontiers in Psychology, Vol., 15, 2014.
[39] Azjen, I., The Theory of Planned Behavior. Organizational behavior and human decision processes, Vol., 50, pp. 179-211, 1991. http://dx.doi.org/10.1016/0749-5978(91)90020-T
[40] Bandura, A., Social foundations of thought and action: a social cognitive theory. Englewood Cliffs, NJ: Prentice Hall, 1986.
[41] Rosenstock, I.M., Strecher, V.J., Becker, M.H., Social learning theory and the health belief model. Health Education Quarterly, Vol., 15, pp. 175-183, 1988. http://dx.doi.org/10.1177/109019818801500203
[42] Oinas-Kukkonen, H., A foundation for the study of behavior change support systems. Pers Ubiquit Comput Vol., 17, pp. 1223-1235, 2013. http://dx.doi.org/10.1007/s00779-012-0591-5
[43] Fogg, B.J., Persuasive technology: using computers to change what we think and do. Ubiquity, Vol., 5, 2002
[44] McPherson, A.C., Glazebrook, C., Forster, D., James, C., Smyth, A., A randomized, controlled trial of an interactive educational computer package for children with asthma. Pediatrics, Vol., 117, pp. 1046-1054, 2006. http://dx.doi.org/10.1542/peds.2005-0666
[45] van der Stege, H.A., van Staa, A., Hilberink, S.R., Visser, A., Using the new board game SeCZ TaLK to stimulate the communication on sexual health for adolescents with chronic conditions. Patient Education and Counseling Vol., 81, pp. 324-331, 2010. http://dx.doi.org/10.1016/j.pec.2010.09.011
[46] Cense, I.M., van der Werf, W., Haastrecht, P., Maak seks lekker duidelijk. Tijdschrift voor gezondheidswetenschappen, Vol., 90, pp. 205-208, 2012.
[47] Serret, S., Hun, S., Iakimova, G., Lozada, J., Anastassova, M., et al. Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new serious game: a pilot study. Molecular Autism, Vol., 5, pp. 37, 2014. http://dx.doi.org/10.1186/2040-2392-5-37
[48] van der Kooij K., Hoogendoorn, E., Spijkerman, R., Visch, V.T., In Games for Health, Schouten B, Schijven M, Gekker A, Fedtke S, Vosmeer M, (Eds.) Utrecht. Springer Vieweg, 2014.
[49] Shiffman, S., Stone, A.A., Hufford, M.R., Ecological Momentary Assessment. Annual Reviews in Clinical Psychology, Vol., 4, pp. 1-32, 2008. http://dx.doi.org/10.1146/annurev.clinpsy.3.022806.091415
[50] Wilkinson, N., Ang, R.P., Goh, D.H., Online video game therapy for mental health concerns: A review. International Journal of Social Psychiatry, Vol., 54, pp. 370-382, 2008. http://dx.doi.org/10.1177/0020764008091659
[51] Lewis, M.D., Lamey, A.V., Douglas, L., A new dynamic systems method for the analysis of early socioemotional development. Developmental Science, Vol., 2, 457-475, 1999. http://dx.doi.org/10.1111/1467-7687.00090
[52] Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., Berta, R., Assessment in and of serious games: an overview. Advances in Human Computer Interaction, Vol., 1, pp. 1-11, 2013. http://dx.doi.org/10.1155/2013/136864
[53] Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., et al., The research and evaluation of serious games: Toward a comprehensive methodology. British journal of educational technology, Vol., 45, pp. 502-527, 2013. http://dx.doi.org/10.1111/bjet.12067
[54] IJsselstijn, W., van den Hoogen, W., Klimmt, C., de Kort, Y., Lindley, C., et al. Measuring the experience of digital game enjoyment. Proceedings of Measuring Behavior; Netherlands: Maastricht, 2008.
[55] Jennett, C., Cox, A.L., Cairns, P., Dhoparee, S., Epps, A., et al., Measuring and defining the experience of immersion in games. International journal of human-computer studies, Vol., 66, pp. 641-661, 2008. http://dx.doi.org/10.1016/j.ijhcs.2008.04.004
[56] Brockmyer, J.H., Fox, C.M., Curtiss, K.A., McBroom, E., Burkhart, K.M., The development of the Game Engagement Questionnaire: a measure of engagement in video game-playing. Journal of Experimental Social Psychology, Vol., 45, pp. 624-634, 2009. http://dx.doi.org/10.1016/j.jesp.2009.02.016
[57] Shaker, N., Asteriadis, S., Yannakakis, G.N., Karpouzis, K., Fusing visual and behavioral cues for modeling user experience in games. IEEE Transactions on cybernetics, Vol., 43, pp. 1519-1542, 2013. http://dx.doi.org/10.1109/TCYB.2013.2271738
[58] Trepte, S., Reinecke, L., The pleasures of success: game-related efficacy experiences as a mediator between player performance and game enjoyment. Cyperpsychology, behavior and social networking, Vol., 14, pp. 555-559, 2011.
[59] Chumbley, J., Griffiths, M., Affect and the computer game player: the effect of gender, personality, and game reinforcement structure on affective responses to computer game-play. CyberPsychology & Behavior, Vol., 9, pp. 308-316, 2006. http://dx.doi.org/10.1089/cpb.2006.9.308
[60] Kivikangas, J.M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., et al., A review of the use of psychophysiological methods in game research. Journal of Gaming and Virtual Worlds, Vol., 3, pp. 181-199, 2011. http://dx.doi.org/10.1386/jgvw.3.3.181_1
[2] Anderson C.A., Shibuya A., Ihori N., Swing E.L., Bushman B.J., et al. (2010) Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: a meta-analytic review. Psychological Bulletin Vol. 136, pp. 151-173, 2014. http://dx.doi.org/10.1037/a0018251
[3] Abt, C.C., Serious Games. New York: University Press of America, Inc, 1987.
[4] Cadamuro A., & Visch V., ‘What Remains?’: A Persuasive Story Telling Game. In Games for Health. Springer Fachmedien Wiesbaden, 2013.
[5] Graafland M., Dankbaar, M., Mert, A., Lagro, J., De Wit-Zuurendonk, L., et al. How to systematically assess serious games applied to health care. JMIR Serious Games, Vol. 2, 2015.
[6] Bogost, I., Persuasive games: The expressive power of videogames: MIT Press, 2007.
[7] Brezinka V., Treasure Hunt - a serious game to support psychotherapeutic treatment of children. Studies in health technology and informatics, Vol., 136, pp. 71-76, 2008.
[8] Coyle D., Matthews, M., Sharry, J., Nisbet, A., Doherty, G., Personal Investigator: a therapeutic 3D game for adolescent psychotherapy. Journal of Interactive Technology & Smart Education, Vol., 2, pp. 73-88, 2005. http://dx.doi.org/10.1108/17415650580000034
[9] Gamberini, L., Barresi, G., Majer, A., Scarpetta, F., A game a day keeps the doctor away: a short review of computer games in mental healthcare. Journal of CyberTherapy & Rehabilitation Vol., 1, pp. 127-149, 2008.
[10] Shandley, K., Austin, D., Klein, B., Kyrios, M., An evaluation of 'Reach Out Central': an online gaming program for supporting the mental health of young people. Health Education Research, Vol., 25, pp. 563-574, 2010. http://dx.doi.org/10.1093/her/cyq002
[11] Thieme, A., Wallace, J., Johnson, P., McCarthy, J., Lindley, S., et al. Design to promote mindfulness practice and sense of self for vulnerable women in secure hospital services. CHI: changing perspectives. Paris, France, 2013. http://dx.doi.org/10.1145/2470654.2481366
[12] DeSmet. A., van Ryckeghem, D., Compernolle, S., Baranowksi, T., Thompson, D., et al. A meta-analysis of serious digital games for healthy lifestyle promotion. Preventive Medicine, Vol., 69, pp. 95-107, 2014. http://dx.doi.org/10.1016/j.ypmed.2014.08.026
[13] Hamari, J., Koivisto, J., Sarsa, H., Does gamification work? - A literature review of empirical studies on gamification. In System Sciences (HICSS), 47th Hawaii International Conference on. IEEE, 2014.
[14] Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T., Boyle, J.M., A systematic literature review of empirical evidence on computer games and serious games. Computer and Education Vol., 59, pp. 661-686, 2012. http://dx.doi.org/10.1016/j.compedu.2012.03.004
[15] Green, M.C., Brock, T.C., Kaufman, G.F., Understanding media enjoyment: The role of transportation into narrative worlds. Communication Theory, Vol., 14, pp. 311-327, 2014. http://dx.doi.org/10.1111/j.1468-2885.2004.tb00317.x
[16] Caillois, R., Les jeux et les hommes. Paris: Librairie Gallimard, 1958.
[17] Huizinga, J., Humo Ludens - Proeve eener bepaling van het spel-element der cultuur. Amsterdam: Amsterdam University Press., (1938 / 2008). http://dx.doi.org/10.5117/9789089640031
[18] Przybylski, A.K., Rigby., C.S., Ryan, R.M., A motivational model of video game engagement. Review of General Psychology Vol., 14, pp. 154-166, 2010. http://dx.doi.org/10.1037/a0019440
[19] Baranowski, T., Buday, R., Thompson, D.I., Baranowski, J., Playing for real: video games and stories for health-related behavior change. American Journal of Preventive Medicine Vol., 34, pp. 74-82, 2008. http://dx.doi.org/10.1016/j.amepre.2007.09.027
[20] Baranowski, T., Baranowksi, J., Thompson, D., Buday, R., Jago, R., et al. Video game play, child diet, and physical activity behavior change. A randomized clinical trial. American Journal of Preventive Medicine Vol., 40., pp. 33-38, 2011. http://dx.doi.org/10.1016/j.amepre.2010.09.029
[21] Bartholomew, L.K., Parcel, G.S., Kok, G., Gottlieb, N.H., Fernandez, M.E., Planning health promotion programs: an intervention mapping approach. New York, NY: John Wiley & Sons, 2011.
[22] Kato, P., Evaluating efficacy and validating health games. In Games for Health: Research, Development, and Clinical Applications; Amsterdam; 2013.
[23] Brahmajee, K., Nallamothu, R., Hayward, E., Bates, E.R., Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies. Circulation Vol., 118, pp. 1294-1303, 2008. http://dx.doi.org/10.1161/CIRCULATIONAHA.107.703579
[24] Granic, I., O'Hara, A., Pepler, D., Lewis, M.D. A dynamic systems analysis of parent-child changes associated with successful "Real-world" interventions for aggressive children. Journal of Abnormal Child Psychology, Vol., 35, pp. 845-857, 2007. http://dx.doi.org/10.1007/s10802-007-9133-4
[25] Laurenceau, J.P., Hayes, A.M., Feldman, G.C., Some methodological and statistical issues in the study of change processes in psychotherapy. Clinical Psychology Review, Vol., 27, pp. 682-695, 2007. http://dx.doi.org/10.1016/j.cpr.2007.01.007
[26] Arnab, S., Brown, K., Clarke, S., Dunwell, I., Lim, T., et al. The development approach of a pedagogically-driven serious game to support Relationship and Sex Education (RSE) within a classroom setting. Computer and Education Vol., 69, pp. 15-30, 2013. http://dx.doi.org/10.1016/j.compedu.2013.06.01
[27] LeBlanc, M., Tools for creating dramatic game dynamics. The game design reader: A rules of play anthology, pp. 438-459, 2006.
[28] Deterding, S., Dixon, D., Khaled, R., Nacke, L., From game design elements to gamefulness: defining "gamification". Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9-15, 2011. http://dx.doi.org/10.1145/2181037.2181040
[29] Garris, R., Ahlers, R., Driskell, J.E., Games, motivation and learning: a research and practice model. Simulation gaming, Vol., 33, pp. 441-467, 2002. http://dx.doi.org/10.1177/1046878102238607
[30] Korhonen, H., Montola, M., Arrasvuori, J., Understanding playful user experience through digital games. International Conference on Designing Pleasurable Products and Interfaces, pp. 274-285, 2009.
[31] McLay, R.N., Wood, D.P., Webb-Murphy, J.A., Spira, J.L., Wiederhold M.D., et al. A randomized, controlled trials of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychology, behavior and social networking, Vol., 14, pp. 223-229, 2011. http://dx.doi.org/10.1089/cyber.2011.0003
[32] Brown, S.J., Lieberman, D.A., Gemeny, B.A., Fan, Y.C., Wilson, D.M., et al. Educational video game for juvenile diabetes: results of a controlled trial. Informatics for Health and Social Care, Vol., 22, pp. 77-89, 1997. http://dx.doi.org/10.3109/14639239709089835
[33] Kato. P.M., Cole, S.W., Bradly, A.S., Pollock, B.H., A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics, Vol., 122, e305, 2007.
[34] Granic, I., Lobel, A., Engels, R.C.M.E., The benefits of playing video games. American Psychologist Vol., 69, pp. 66-78, 2013. http://dx.doi.org/10.1037/a0034857
[35] Green, C.S., Bavellier, D. Action video game modifies visual selective attention. Nature, Vol., 423, pp. 534-537, 2003. http://dx.doi.org/10.1038/nature01647
[36] Cole, S.W., Yoo, D.J., Knutson, B. Interactivity and reward-related neural activation during a serious videogame. PLOS One, Vol., 7, 2012.
[37] Anguera, J.A., Boccanfuso, J., Rintoul, J.L., Al-Hashimi, O., Faraji, F., et al., Video game training enhances cognitive control in older adults. Nature, Vol., 501, pp. 97-101, 2013. http://dx.doi.org/10.1038/nature12486
[38] Starks, K., Cognitive behavioral game design: a unified model for designing serious games. Frontiers in Psychology, Vol., 15, 2014.
[39] Azjen, I., The Theory of Planned Behavior. Organizational behavior and human decision processes, Vol., 50, pp. 179-211, 1991. http://dx.doi.org/10.1016/0749-5978(91)90020-T
[40] Bandura, A., Social foundations of thought and action: a social cognitive theory. Englewood Cliffs, NJ: Prentice Hall, 1986.
[41] Rosenstock, I.M., Strecher, V.J., Becker, M.H., Social learning theory and the health belief model. Health Education Quarterly, Vol., 15, pp. 175-183, 1988. http://dx.doi.org/10.1177/109019818801500203
[42] Oinas-Kukkonen, H., A foundation for the study of behavior change support systems. Pers Ubiquit Comput Vol., 17, pp. 1223-1235, 2013. http://dx.doi.org/10.1007/s00779-012-0591-5
[43] Fogg, B.J., Persuasive technology: using computers to change what we think and do. Ubiquity, Vol., 5, 2002
[44] McPherson, A.C., Glazebrook, C., Forster, D., James, C., Smyth, A., A randomized, controlled trial of an interactive educational computer package for children with asthma. Pediatrics, Vol., 117, pp. 1046-1054, 2006. http://dx.doi.org/10.1542/peds.2005-0666
[45] van der Stege, H.A., van Staa, A., Hilberink, S.R., Visser, A., Using the new board game SeCZ TaLK to stimulate the communication on sexual health for adolescents with chronic conditions. Patient Education and Counseling Vol., 81, pp. 324-331, 2010. http://dx.doi.org/10.1016/j.pec.2010.09.011
[46] Cense, I.M., van der Werf, W., Haastrecht, P., Maak seks lekker duidelijk. Tijdschrift voor gezondheidswetenschappen, Vol., 90, pp. 205-208, 2012.
[47] Serret, S., Hun, S., Iakimova, G., Lozada, J., Anastassova, M., et al. Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new serious game: a pilot study. Molecular Autism, Vol., 5, pp. 37, 2014. http://dx.doi.org/10.1186/2040-2392-5-37
[48] van der Kooij K., Hoogendoorn, E., Spijkerman, R., Visch, V.T., In Games for Health, Schouten B, Schijven M, Gekker A, Fedtke S, Vosmeer M, (Eds.) Utrecht. Springer Vieweg, 2014.
[49] Shiffman, S., Stone, A.A., Hufford, M.R., Ecological Momentary Assessment. Annual Reviews in Clinical Psychology, Vol., 4, pp. 1-32, 2008. http://dx.doi.org/10.1146/annurev.clinpsy.3.022806.091415
[50] Wilkinson, N., Ang, R.P., Goh, D.H., Online video game therapy for mental health concerns: A review. International Journal of Social Psychiatry, Vol., 54, pp. 370-382, 2008. http://dx.doi.org/10.1177/0020764008091659
[51] Lewis, M.D., Lamey, A.V., Douglas, L., A new dynamic systems method for the analysis of early socioemotional development. Developmental Science, Vol., 2, 457-475, 1999. http://dx.doi.org/10.1111/1467-7687.00090
[52] Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., Berta, R., Assessment in and of serious games: an overview. Advances in Human Computer Interaction, Vol., 1, pp. 1-11, 2013. http://dx.doi.org/10.1155/2013/136864
[53] Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., et al., The research and evaluation of serious games: Toward a comprehensive methodology. British journal of educational technology, Vol., 45, pp. 502-527, 2013. http://dx.doi.org/10.1111/bjet.12067
[54] IJsselstijn, W., van den Hoogen, W., Klimmt, C., de Kort, Y., Lindley, C., et al. Measuring the experience of digital game enjoyment. Proceedings of Measuring Behavior; Netherlands: Maastricht, 2008.
[55] Jennett, C., Cox, A.L., Cairns, P., Dhoparee, S., Epps, A., et al., Measuring and defining the experience of immersion in games. International journal of human-computer studies, Vol., 66, pp. 641-661, 2008. http://dx.doi.org/10.1016/j.ijhcs.2008.04.004
[56] Brockmyer, J.H., Fox, C.M., Curtiss, K.A., McBroom, E., Burkhart, K.M., The development of the Game Engagement Questionnaire: a measure of engagement in video game-playing. Journal of Experimental Social Psychology, Vol., 45, pp. 624-634, 2009. http://dx.doi.org/10.1016/j.jesp.2009.02.016
[57] Shaker, N., Asteriadis, S., Yannakakis, G.N., Karpouzis, K., Fusing visual and behavioral cues for modeling user experience in games. IEEE Transactions on cybernetics, Vol., 43, pp. 1519-1542, 2013. http://dx.doi.org/10.1109/TCYB.2013.2271738
[58] Trepte, S., Reinecke, L., The pleasures of success: game-related efficacy experiences as a mediator between player performance and game enjoyment. Cyperpsychology, behavior and social networking, Vol., 14, pp. 555-559, 2011.
[59] Chumbley, J., Griffiths, M., Affect and the computer game player: the effect of gender, personality, and game reinforcement structure on affective responses to computer game-play. CyberPsychology & Behavior, Vol., 9, pp. 308-316, 2006. http://dx.doi.org/10.1089/cpb.2006.9.308
[60] Kivikangas, J.M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., et al., A review of the use of psychophysiological methods in game research. Journal of Gaming and Virtual Worlds, Vol., 3, pp. 181-199, 2011. http://dx.doi.org/10.1386/jgvw.3.3.181_1
Downloads
Published
2015-09-18
Issue
Section
Articles
License
IJSG copyright information is provided here.
How to Cite
Validation of Serious Games. (2015). International Journal of Serious Games, 2(3). https://doi.org/10.17083/ijsg.v2i3.75
