Measuring Effectiveness of Persuasive Games Using an Informative Control Condition
DOI:
https://doi.org/10.17083/ijsg.v2i2.74Keywords:
Persuasive games, Serious games, Attitude change, Knowledge acquisitionAbstract
Research about the effectiveness of persuasive games is still emerging. This article presents a literature review of studies that empirically evaluate the effectiveness of persuasive games. The review concluded that limited empirical evidence is currently available to prove their effectiveness in attitude change. It further revealed that almost no study employed an informative control condition, making it difficult to conclude that the game was more effective than a control condition. Next, in a pretest-posttest design an empirical study tested whether change in attitude was different for people playing the persuasive game "EnerCities" compared to a control condition where participants read a document with highly similar information. No significant differences in increase of attitude or knowledge between participants that played the game and participants in the informative control condition were found. Based on the results of the literature review and the empirical study presented, it hence cannot be concluded that playing a game leads to a greater change in attitude or knowledge acquisition than experiencing conventional media would. Future work should employ designs with proper control conditions and focus on which game features lead to significant effects.References
[1] *Alhabash, S., & Wise, K., “Playing their game: Changing stereotypes of Palestinians and Israelis through videogame play.” New Media & Society, 1–19, 2014.
[2] Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M., “Investigating the impact of video games on high school students' engagement and learning about genetics”. Computers & Education, 53, 74-85, 2009.
[3] Aronson, E., Wilson, T. D., & Akert, R. M., “Social Psychology”. Upper Saddle River, N.J.: Pearson Education, 2013.
[4] Bang, M., Torstensson, C., & Katzeff, C., “The PowerHouse: A Persuasive Computer Game Designed to Raise Awareness of Domestic Energy Consumption”. In PERSUASIVE 2006 (123–132). Berlin-Heidelberg: Springer-Verlag, 2006.
[5] *Barab, S. A., Sadler, T. D., Heiselt, C., Hickey, D., & Zuiker, S., “Erratum to: Relating narrative, inquiry, and inscriptions: Supporting consequential play”. Journal of Science Education and Technology, 19(4), 387–407, 2010.
[6] *Barab, S., & Arici, A., “Mystery of Taiga River: Scientific Investigation Game”. In Escudeiro P., & Vaz de Carvalho C., (Eds.), Abstracts of Papers Presented at the 7th European Conference on Games Based Learning. Porto, Portugal, 2013.
[7] *Baranowski, T., Baranowski, J., Cullen, K. W., Marsh, T., Islam, N., Zakeri, I., De Moor, C., “Dietary Outcome Evaluation of a Multimedia Game”. American Journal of Preventive Medicine, 24(1), 52–61, 2003.
[8] Bogost, I., “Persuasive games: the expressive power of videogames”, Cambridge, MA: The MIT Press, 2007.
[9] Brands, L., “Is collaboratief spelen van een game met een script beter dan individueel”. Unpublished bachelor thesis, Universiteit Twente, 2014.
[10] *Burak, A., Keylor, E., & Sweeney, T., “PeaceMaker: A video game to teach peace”. In Intelligent Technologies for Interactive Entertainment (307–310). Springer Berlin Heidelberg, 2005.
[11] Chen, S., & Chaiken, S., "The heuristic-systematic model in its' broader context", In S. Chaiken, & Y. Trope (Eds.), Dual Process Theories in Social Psychology. New York, NY: Guilford Press, 1999.
[12] Csikszentmihalyi, M., “Flow: the psychology of optimal experience”, New York, NY: Harpers & Row, 1990.
[13] De Vries, H., Mesters, I., van de Steeg, H., & Honing, C., “The general public’s information needs and perceptions regarding hereditary cancer: an application of the Integrated Change Model”. Patient Education and Counseling, 56(2), 154–65, 2005.
[14] *De Vries, P. W., & Knol, E., “Serious Gaming as a Means to Change Adolescents’ Attitudes Towards Saving Energy; Preliminary Results from the EnerCities Case”. In EDEN Annual Conference (1–5), 2011.
[15] *Felicio, S. P. A. S., Silva, V. S. R., Dargains, A. R., Souza, P. R. A., Sampaio, F., Carvalho, P. V. R. Borges, M. R. S., “Stop Disasters Game Experiment with Elementary School Students in Rio de Janeiro?: Building Safety Culture”. In Proceedings of the 11th International ISCRAM Conference (585–591), 2014.
[16] Field, A., “Discovering Statistics Using SPSS”, London, UK: SAGE, 2009.
[17] Fisher, R. J., “Social Desirability Bias and the Validity of Indirect Questioning”. Journal of Consumer Research, 20(2), 303-315, 1993.
[18] Fogg, B. J., “Persuasive Technology: Using Computers to Change What We Think and Do”, San Francisco, CA: Morgan Kauffman, 2003.
[19] *Gerling, K., Mandryk, R., & Birk, M., “The Effects of Embodied Persuasive Games on Player Attitudes Toward People Using Wheelchairs”. In CHI’14: Proceedings of the 2014 CHI Conference on Human Factors in Computing Systems, 2014.
[20] Girard, C., Ecalle J., & Magnant, A., "Serious games as new educational tools: how effective are they? A meta-analysis of recent studies". Journal of Computer Assisted Learning, 29, 207-219, 2013.
[21] *Gonzalez, C., Kampf, R., & Martin, J. M., “Action diversity in a simulation of the Israeli–Palestinian conflict”. Computers in Human Behavior, 28(1), 233–240, 2012.
[22] *Gustafsson, A., Bång, M., & Svahn, M., “Power explorer: a casual game style for encouraging long term behavior change among teenagers”. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (pp. 182–189), 2009.
[23] *Gustafsson, A., Katzeff, C., & Bang, M., “Evaluation of a pervasive game for domestic energy engagement among teenagers”. Computers in Entertainment, 7(4-54), 1-9, 2009.
[24] Honey, M. A., & Hilton, M. (Eds.). “Learning science through computer games and simulations”, Washington, DC: The National Academies Press, 2011.
[25] *Isaacs, J. A., Dolinsky, B., Qualters, D. M., & Laird, J. T., “Engineering Decisions to Green the Automobile Supply Chain”. In Cozzi A. & Ohji T., (Eds.), Environmental Issues and Waste Management Technologies in the Materials and Nuclear Industries XII (225–234). Hoboken, NJ: John Wiley & Sons, 2009.
[26] *Kato, P. M., Cole, S. W., Bradlyn, a S., & Pollock, B. H., “A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial”. Pediatrics, 122(2), 305–317, 2008.
[27] Katsaliaki, K., & Mustafee, N., “A Survey of Serious Games on Sustainable Development”. In Proceedings of the 2012 Winter Simulation Conference (Vol. 7), 2012.
[28] Kiser, C., “Ondersteuning in Game-gebaseerd Leren: De Invloed van Testen en Feedback op Leren met Games”. Unpublished bachelor thesis, Universiteit van Twente, 2013.
[29] Klimmt, C., “Serious Games and Social Change: Why They (Should) Work”. In Ritterfeld U., Cody M., & Vorderer P., (Eds.), Serious Games: Mechanisms and Effects (248–270). New York, NY: Routledge, 2009.
[30] *Knol, E., & Vries, P. De., “EnerCities, a Serious Game to Stimulate Sustainability and Energy Conservation: Preliminary Results”. eLearning Paper, 25, 1–10, 2011.
[31] *Lavender, T., “Games Just Wanna Have Fun … Or Do They?? Measuring the Effectiveness of Persuasive Games”. In Proceedings of the CGSA 2006 Symposium (1-7), 2006.
[32] *Lavender, T. J., “Homeless: It’s No Game - Measuring the Effectiveness of a Persuasive Videogame”. Unpublished master thesis, Simon Fraser University, 2008.
[33] Liarakou, G., Sakke, E., Gavrilakis, C., & Tsolakidis, C., “Evaluation of Serious Games, as a Tool for Education for Sustainable Development”. In Proceedings of the EDEN Annual Conference, 2011.
[34] Mayer, R. E., “Multimedia Learning”. New York, NY: Cambridge University Press, 2001.
[35] Mayer, R. E., “Multimedia learning and games”. In Tobias S., & Fletcher J., D., (Eds.), Computer games and instruction (281–305), Greenwich, CT: Information Age Publishing., 2011
[36] Michalos, A. C., Creech, H., McDonald, C., & Kahlke, P. M. H., “Measuring Knowledge, Attitudes and Behaviours towards Sustainable Development: Two Exploratory Studies”.IISD, 1, 2009.
[37] *Michelin, M., “Can a game put engineering students in an active learning mode?? A first experiment in sustainable agriculture teaching”. In Elleithy K., (Ed.), Advances in Computer, Information, and Systems Sciences, and Engineering (p343–349). Springer, 2006.
[38] Murray, J., “Hamlet on the Holodeck: The Future of Narrative in Cyberspace”. Cambridge, MA: MIT Press, 1997.
[39] *Peng, W., “Design and evaluation of a computer game to promote a healthy diet for young adults”. Health Communication, 24(2), 115–27, 2009.
[40] Prensky, M., “The Motivation of Gameplay”. On the Horizon, 10(1), 1–14, 2002.
[41] Ritterfeld, U., Cody, M., & Vorderer, P., “Introduction”. In Ritterfeld U., Cody M., & Vorderer P., (Eds.), Serious Games: Mechanisms and Effects (3–9). London: Routledge, 2009.
[42] Ruggiero, D., “The Effect of a Persuasive Game on Attitude towards the Homeless”. Unpublished thesis, Purdue University, 2013.
[43] Shettleworth, S. J., “Cognition, Evolution and Behavior”. New York, NY: Oxford, 2010.
[44] Spek, E. van der, “Experiments in serious game design: a cognitive approach”. Unpublished PhD thesis, Utrecht University, the Netherlands, 2011.
[45] *Tragazikis, P., & Meimaris, M., “Engaging kids with the concept of sustainability using a commercial video game – a case study”. In Pan Z., Cheok A.D., Muller W., & Change M., (Eds.), Transactions on Edutainment III (1–12), 2009.
[46] U.S. Energy Information Administration, “Annual Energy Outlook 2014”. Retrieved from http://www.eia.gov/forecasts/aeo/pdf/0383(2014).pdf, 2014.
[47] Vermeir, I. & Verbeke, W., “Sustainable Food Consumption: Exploring the Consumer "Attitude - Behavioral Intention" Gap”. Journal of Agricultural and Environmental Ethics, 19, 169-194, 2006.
[48] *Williams, O., Hecht, M. F., DeSorbo, A. L., Huq, S., & Noble, J. M., “Effect of a novel video game on stroke knowledge of 9- to 10-year-old, low-income children”. Stroke; a Journal of Cerebral Circulation, 45(3), 889–92, 2014.
[49] Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D., “A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games”. Journal of Educational Psychology, 105(2), 249–265, 2013.
[2] Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M., “Investigating the impact of video games on high school students' engagement and learning about genetics”. Computers & Education, 53, 74-85, 2009.
[3] Aronson, E., Wilson, T. D., & Akert, R. M., “Social Psychology”. Upper Saddle River, N.J.: Pearson Education, 2013.
[4] Bang, M., Torstensson, C., & Katzeff, C., “The PowerHouse: A Persuasive Computer Game Designed to Raise Awareness of Domestic Energy Consumption”. In PERSUASIVE 2006 (123–132). Berlin-Heidelberg: Springer-Verlag, 2006.
[5] *Barab, S. A., Sadler, T. D., Heiselt, C., Hickey, D., & Zuiker, S., “Erratum to: Relating narrative, inquiry, and inscriptions: Supporting consequential play”. Journal of Science Education and Technology, 19(4), 387–407, 2010.
[6] *Barab, S., & Arici, A., “Mystery of Taiga River: Scientific Investigation Game”. In Escudeiro P., & Vaz de Carvalho C., (Eds.), Abstracts of Papers Presented at the 7th European Conference on Games Based Learning. Porto, Portugal, 2013.
[7] *Baranowski, T., Baranowski, J., Cullen, K. W., Marsh, T., Islam, N., Zakeri, I., De Moor, C., “Dietary Outcome Evaluation of a Multimedia Game”. American Journal of Preventive Medicine, 24(1), 52–61, 2003.
[8] Bogost, I., “Persuasive games: the expressive power of videogames”, Cambridge, MA: The MIT Press, 2007.
[9] Brands, L., “Is collaboratief spelen van een game met een script beter dan individueel”. Unpublished bachelor thesis, Universiteit Twente, 2014.
[10] *Burak, A., Keylor, E., & Sweeney, T., “PeaceMaker: A video game to teach peace”. In Intelligent Technologies for Interactive Entertainment (307–310). Springer Berlin Heidelberg, 2005.
[11] Chen, S., & Chaiken, S., "The heuristic-systematic model in its' broader context", In S. Chaiken, & Y. Trope (Eds.), Dual Process Theories in Social Psychology. New York, NY: Guilford Press, 1999.
[12] Csikszentmihalyi, M., “Flow: the psychology of optimal experience”, New York, NY: Harpers & Row, 1990.
[13] De Vries, H., Mesters, I., van de Steeg, H., & Honing, C., “The general public’s information needs and perceptions regarding hereditary cancer: an application of the Integrated Change Model”. Patient Education and Counseling, 56(2), 154–65, 2005.
[14] *De Vries, P. W., & Knol, E., “Serious Gaming as a Means to Change Adolescents’ Attitudes Towards Saving Energy; Preliminary Results from the EnerCities Case”. In EDEN Annual Conference (1–5), 2011.
[15] *Felicio, S. P. A. S., Silva, V. S. R., Dargains, A. R., Souza, P. R. A., Sampaio, F., Carvalho, P. V. R. Borges, M. R. S., “Stop Disasters Game Experiment with Elementary School Students in Rio de Janeiro?: Building Safety Culture”. In Proceedings of the 11th International ISCRAM Conference (585–591), 2014.
[16] Field, A., “Discovering Statistics Using SPSS”, London, UK: SAGE, 2009.
[17] Fisher, R. J., “Social Desirability Bias and the Validity of Indirect Questioning”. Journal of Consumer Research, 20(2), 303-315, 1993.
[18] Fogg, B. J., “Persuasive Technology: Using Computers to Change What We Think and Do”, San Francisco, CA: Morgan Kauffman, 2003.
[19] *Gerling, K., Mandryk, R., & Birk, M., “The Effects of Embodied Persuasive Games on Player Attitudes Toward People Using Wheelchairs”. In CHI’14: Proceedings of the 2014 CHI Conference on Human Factors in Computing Systems, 2014.
[20] Girard, C., Ecalle J., & Magnant, A., "Serious games as new educational tools: how effective are they? A meta-analysis of recent studies". Journal of Computer Assisted Learning, 29, 207-219, 2013.
[21] *Gonzalez, C., Kampf, R., & Martin, J. M., “Action diversity in a simulation of the Israeli–Palestinian conflict”. Computers in Human Behavior, 28(1), 233–240, 2012.
[22] *Gustafsson, A., Bång, M., & Svahn, M., “Power explorer: a casual game style for encouraging long term behavior change among teenagers”. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (pp. 182–189), 2009.
[23] *Gustafsson, A., Katzeff, C., & Bang, M., “Evaluation of a pervasive game for domestic energy engagement among teenagers”. Computers in Entertainment, 7(4-54), 1-9, 2009.
[24] Honey, M. A., & Hilton, M. (Eds.). “Learning science through computer games and simulations”, Washington, DC: The National Academies Press, 2011.
[25] *Isaacs, J. A., Dolinsky, B., Qualters, D. M., & Laird, J. T., “Engineering Decisions to Green the Automobile Supply Chain”. In Cozzi A. & Ohji T., (Eds.), Environmental Issues and Waste Management Technologies in the Materials and Nuclear Industries XII (225–234). Hoboken, NJ: John Wiley & Sons, 2009.
[26] *Kato, P. M., Cole, S. W., Bradlyn, a S., & Pollock, B. H., “A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial”. Pediatrics, 122(2), 305–317, 2008.
[27] Katsaliaki, K., & Mustafee, N., “A Survey of Serious Games on Sustainable Development”. In Proceedings of the 2012 Winter Simulation Conference (Vol. 7), 2012.
[28] Kiser, C., “Ondersteuning in Game-gebaseerd Leren: De Invloed van Testen en Feedback op Leren met Games”. Unpublished bachelor thesis, Universiteit van Twente, 2013.
[29] Klimmt, C., “Serious Games and Social Change: Why They (Should) Work”. In Ritterfeld U., Cody M., & Vorderer P., (Eds.), Serious Games: Mechanisms and Effects (248–270). New York, NY: Routledge, 2009.
[30] *Knol, E., & Vries, P. De., “EnerCities, a Serious Game to Stimulate Sustainability and Energy Conservation: Preliminary Results”. eLearning Paper, 25, 1–10, 2011.
[31] *Lavender, T., “Games Just Wanna Have Fun … Or Do They?? Measuring the Effectiveness of Persuasive Games”. In Proceedings of the CGSA 2006 Symposium (1-7), 2006.
[32] *Lavender, T. J., “Homeless: It’s No Game - Measuring the Effectiveness of a Persuasive Videogame”. Unpublished master thesis, Simon Fraser University, 2008.
[33] Liarakou, G., Sakke, E., Gavrilakis, C., & Tsolakidis, C., “Evaluation of Serious Games, as a Tool for Education for Sustainable Development”. In Proceedings of the EDEN Annual Conference, 2011.
[34] Mayer, R. E., “Multimedia Learning”. New York, NY: Cambridge University Press, 2001.
[35] Mayer, R. E., “Multimedia learning and games”. In Tobias S., & Fletcher J., D., (Eds.), Computer games and instruction (281–305), Greenwich, CT: Information Age Publishing., 2011
[36] Michalos, A. C., Creech, H., McDonald, C., & Kahlke, P. M. H., “Measuring Knowledge, Attitudes and Behaviours towards Sustainable Development: Two Exploratory Studies”.IISD, 1, 2009.
[37] *Michelin, M., “Can a game put engineering students in an active learning mode?? A first experiment in sustainable agriculture teaching”. In Elleithy K., (Ed.), Advances in Computer, Information, and Systems Sciences, and Engineering (p343–349). Springer, 2006.
[38] Murray, J., “Hamlet on the Holodeck: The Future of Narrative in Cyberspace”. Cambridge, MA: MIT Press, 1997.
[39] *Peng, W., “Design and evaluation of a computer game to promote a healthy diet for young adults”. Health Communication, 24(2), 115–27, 2009.
[40] Prensky, M., “The Motivation of Gameplay”. On the Horizon, 10(1), 1–14, 2002.
[41] Ritterfeld, U., Cody, M., & Vorderer, P., “Introduction”. In Ritterfeld U., Cody M., & Vorderer P., (Eds.), Serious Games: Mechanisms and Effects (3–9). London: Routledge, 2009.
[42] Ruggiero, D., “The Effect of a Persuasive Game on Attitude towards the Homeless”. Unpublished thesis, Purdue University, 2013.
[43] Shettleworth, S. J., “Cognition, Evolution and Behavior”. New York, NY: Oxford, 2010.
[44] Spek, E. van der, “Experiments in serious game design: a cognitive approach”. Unpublished PhD thesis, Utrecht University, the Netherlands, 2011.
[45] *Tragazikis, P., & Meimaris, M., “Engaging kids with the concept of sustainability using a commercial video game – a case study”. In Pan Z., Cheok A.D., Muller W., & Change M., (Eds.), Transactions on Edutainment III (1–12), 2009.
[46] U.S. Energy Information Administration, “Annual Energy Outlook 2014”. Retrieved from http://www.eia.gov/forecasts/aeo/pdf/0383(2014).pdf, 2014.
[47] Vermeir, I. & Verbeke, W., “Sustainable Food Consumption: Exploring the Consumer "Attitude - Behavioral Intention" Gap”. Journal of Agricultural and Environmental Ethics, 19, 169-194, 2006.
[48] *Williams, O., Hecht, M. F., DeSorbo, A. L., Huq, S., & Noble, J. M., “Effect of a novel video game on stroke knowledge of 9- to 10-year-old, low-income children”. Stroke; a Journal of Cerebral Circulation, 45(3), 889–92, 2014.
[49] Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D., “A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games”. Journal of Educational Psychology, 105(2), 249–265, 2013.
Downloads
Additional Files
Published
2015-06-17
Issue
Section
Articles
License
IJSG copyright information is provided here.
How to Cite
Measuring Effectiveness of Persuasive Games Using an Informative Control Condition. (2015). International Journal of Serious Games, 2(2). https://doi.org/10.17083/ijsg.v2i2.74
