Individual and collaborative Performance and Level of Certainty in MetaVals
DOI:
https://doi.org/10.17083/ijsg.v1i1.3Keywords:
Game Based Learning, Serious Games, Performance, Level of certainty, MetaVals, collaborationAbstract
In the context of Higher Education (HE) in general, and management education in particular, the use of Serious Games (SG) is spreading, and solutions are increasingly developing. Nevertheless, the implementation of this learning methodology deserves further study, in particular concerning pedagogical and psychological aspects such game performance and players’ metacognitive processes. This paper aims to study the relation among these two variables, based on the review of the results of MetaVals SG during the last 3 years. MetaVals is a collaborative, computer-based SG designed to facilitate collaboration and metacognitive awareness among HE students. It has been played by 250 students in 16 different experiences since its first version, in 2011. Overall results show a higher performance for collaborative than individual phases of the game, furthermore, students’ elicitation of their Level of Certainty (LC), although not significantly, could be related to a better performance. These results can be a basis for further studies focused on the implementation of collaborative GBL in formal and informal adult learning contexts. However, some challenges are also identified and discussed on the present version of MetaVals game, and solutions are proposed in order to continue with the design of SGs for wider application and learners’ needs in the current contexts.
References
[1] Hess, T., Gunter, G., “Serious game-based and nongame-based online courses: Learning experiences and outcomes” British Journal of Educational Technology, vol. 44, no. 3, pp. 372–385, 2013. http://dx.doi.org/10.1111/bjet.12024
[2] Bellotti, F., Berta, R., De Gloria, A., Lavagnino, E., Dagnino, F., Ott, M., Romero, M., Usart, M., Mayer, I., “Designing a Course for Stimulating Entrepreneurship in Higher Education through Serious Games." Procedia Computer Science, vol. 15, pp. 174-186, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.069
[3] Mawdesley, M., Long, G., Al-Jibouri, S., Scott, D., “The enhancement of simulation based learning exercises through formalised reflection, focus groups and group presentation”, Computers and Education, vol. 56, no. 1, pp. 44-52, 2011. http://dx.doi.org/10.1016/j.compedu.2010.05.005
[4] Dickey, M. D., “Game Design and Learning: A Conjectural Analysis of How Massively multiple Online Role-Playing Games (MMORPGs) Foster Intrinsic Motivation”, Educational Technology Research and Development, vol. 55, no. 3, pp- 253-273, 2007.
[5] Leemkuil, H., de Jong, T., de Hoog, R., Christoph, N., “KM Quest: A Collaborative Internet Based Simulation Game”, Simulation & Gaming, vol. 34, no. 1, pp. 89-111, 2003. http://dx.doi.org/10.1177/1046878102250605
[6] Herz, C. J., “Gaming the system: What higher education can learn from multiplayer online worlds. The Internet and the University”, Educause Forum on the Future of Higher Education. Retrieved [11/06/2013] from http://www.educause.edu/ir/library/pdf/ffpiu019.pdf
[7] Azevedo, R., Witherspoon, A., Chauncey, A., Burkett, C., Fike, A., “MetaTutor: A MetaCognitive tool for enhancing self-regulated learning”. In R. Pirrone, R. Azevedo, & G. Biswas (Eds.), Proceedings of the AAAI Fall Symposium on Cognitive and Metacognitive Educational Systems (pp. 14-19). Menlo Park, CA: Association for the Advancement of Artificial Intelligence (AAAI) Press, 2010
[8] Azevedo, R., “The role of self-regulation in learning about science with hypermedia”. In D. Robinson & G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning. (pp.127–156), 2010
[9] Järvelä, S., Volet, S., “Motivation in Real-Life, Dynamic, and Interactive Learning Environments: Stretching Constructs and Methodologies”. Journal European Psychologist, vol. 9, no. 4, pp. 193-197, 2004. http://dx.doi.org/10.1027/1016-9040.9.4.193
[10] Merenluoto, K., Lehtinen, E., “Number concept and conceptual change: towards a systemic model of the processes of change”, Learning and Instruction, vol. 14, no. 5, pp. 519-534, 2004. http://dx.doi.org/10.1016/j.learninstruc.2004.06.016
[11] Garner, R., “When children and adults do not use learning strategies: Toward a theory of settings”, Review of Educational Research, vol. 60, pp. 517–529, 1990 http://dx.doi.org/10.3102/00346543060004517
[12] Schraw, G., “Promoting general metacognitive awareness”, Instructional Science, vol. 26, pp. 113–125, 1998 http://dx.doi.org/10.1023/A:1003044231033
[13] Romero, M., Usart, M., Popescu, M., Boyle, E. “Interdisciplinary and International Adaption and Personalization of the MetaVals Serious Games”, Lecture Notes in Computer Science, vol. 7528, pp 59-70, 2012. http://dx.doi.org/10.1007/978-3-642-33687-4_5
[14] Peirce, N., Conlan, O., Wade, V., “Adaptive educational games: Providing non-invasive personalised learning experiences”. In Digital Games and Intelligent Toys Based Education, 2008 Second IEEE International Conference on IEEE, pp. 28-35, 2008 http://dx.doi.org/10.1109/digitel.2008.30
[15] Sawyer, B., “The serious games summit: emergent use of interactive games for solving problems is serious effort”, Computers in Entertainment (CIE) - Theoretical and Practical Computer Applications in Entertainment, vol. 2, no. 1, pp. 5-12, 2004. http://dx.doi.org/10.1145/973801.973811
[16] Moreno-Ger, P., Burgos, D., Sierra, J.L. & Fernández-Manjón, B., “Educational Game Design for Online Education”, Computers in Human Behavior, vol. 24, no. 6, pp. 2530-2540, 2008 http://dx.doi.org/10.1016/j.chb.2008.03.012
[17] Dillenbourg, P., Jarvela, S., Fischer, F., “The evolution of research on computer-supported collaborative learning: From design to orchestration”, Technology-Enhanced Learning, vol. 1, pp. 3–19, 2009 http://dx.doi.org/10.1007/978-1-4020-9827-7_1
[18] Wendel, V., Gutjahr, M., Göbel S., Steinmetz, R., “Designing collaborative multiplayer serious games”, Education and Information Technologies, vol. 18, no. 2, pp. 287-308, 2013 http://dx.doi.org/10.1007/s10639-012-9244-6
[19] Lipponen, L., Rahikainen, M., Lallimo, J., Hakkarainen, K., “Patterns of participation and discourse in elementary students’ computer-supported collaborative learning”, Learning and Instruction, vol. 13, pp. 487–509, 2003 http://dx.doi.org/10.1016/S0959-4752(02)00042-7
[20] Kreijns, K., Kirschner, P.A., Jochems, W., “Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research”, Computers in Human Behavior, vol. 19, no. 3, pp. 335-353, 2003 http://dx.doi.org/10.1016/S0747-5632(02)00057-2
[21] Davis, A., Little, P., Staward, B., “Developing an infrastructure for online learning”, In Anderson, T. (ed.) The theory and practice of online learning. Edmonton: AU Press, 2008
[22] Prensky, M., “The Motivation of Game Play or, the REAL 21st century learning revolution”, On The Horizon, vol. 10, no. 1, 2001
[23] Romero, M., Usart, M., Ott, M. ,Earp, J. , de Freitas, S., Arnab, S., “Learning through playing for or against each other? Promoting collaborative learning in digital game based learning”, ECIS 2012 Proceedings. Paper 93. Retrieved November 10, 2012, from http://aisel.aisnet.org/ecis2012/93
[24] Becker, K., “Digital game-based learning once removed: teaching teachers. British Journal of Educational Technology, vol. 38, no. 3, pp. 478-488, 2007 http://dx.doi.org/10.1111/j.1467-8535.2007.00711.x
[25] Jeffrey, L., “Learning orientations: Diversity in higher education”, Learning and Individual Differences, vol. 19, pp. 195–208, 2009 http://dx.doi.org/10.1016/j.lindif.2008.09.004
[26] Munneke, L., Andriessen, J., Kanselaar, G., Kirschner, P., “Supporting interactive argumentation: Influence of representational tools on discussing a wicked problem”, Computers in Human Behavior, 23, no. 3, pp. 1072, 2007 http://dx.doi.org/10.1016/j.chb.2006.10.003
[27] Ducheneaut, N., Yee, N., Nickell, E., Moore, R. J., "Alone Together?" Exploring the Social Dynamics of Massively Multiplayer Online Games (CHI Conference), pp. 407-416, 2006 http://dx.doi.org/10.1145/1124772.1124834
[28] Roebers, C. M., “Confidence judgments in children’s and adults’ event recall and suggestibility”, Developmental Psychology, vol. 38, pp. 1052–1067, 2002 http://dx.doi.org/10.1037/0012-1649.38.6.1052
[29] Kirschner, P.A., Erkens, G., “Cognitive Tools and Mindtools for Collaborative Learning”, Journal of Educational Computing, vol. 35, no. 2, pp. 199–209, 2006 http://dx.doi.org/10.2190/R783-230M-0052-G843
[30] Valdez, A., “Student Metacognitive Monitoring: Predicting Test Achievement from Judgment Accuracy”, International Journal of Higher Education, vol. 2, no. 2, 141-146, 2013 http://dx.doi.org/10.5430/ijhe.v2n2p141
[31] Karpicke, J. D., Butler, A. C., Roediger III, H. L., “Metacognitive strategies in student learning: do students practise retrieval when they study on their own?”, Memory, vol. 17, no. 4, 471-479, 2009 http://dx.doi.org/10.1080/09658210802647009
[32] Swanson, H. L., “Influence of metacognitive knowledge and aptitude on problem solving”, Journal of educational psychology, vol. 82, no. 2, pp. 306, 1990 http://dx.doi.org/10.1037/0022-0663.82.2.306
[33] Swartz, S. M., “Acceptance and accuracy of multiple choice, confidence-level, and essay question formats for graduate students”, Journal of Education for Business, vol. 81, no. 4, pp. 215-220, 2006 http://dx.doi.org/10.3200/JOEB.81.4.215-220
[34] Flannelly, L., “Using feedback to reduce students' judgment bias on test questions”, Journal of Nursing Education, vol. 40, no. 1, 10-16, 2001
[35] Efklides, A., Akilina, S., Petropoulou, M., “Feeling of difficulty: an aspect of monitoring that influences control”, European Journal of Psychology of Education, vol. 15, pp. 461–476, 1999 http://dx.doi.org/10.1007/BF03172973
[36] Brennan, S.E., Chen, X., Dickinson, C.A., Neider, M.B., Zelinsky, G.J., “Coordinating Cognition: The Costs and Benefits of Shared Gaze during Collaborative Search”, Cognition vol. 106, no. 3, pp. 1465– 1477, 2008 http://dx.doi.org/10.1016/j.cognition.2007.05.012
[37] Bluemink, J., Hämäläinen, R., Manninen, T., Järvelä, S., “Group-level analysis on multiplayer game collaboration: how do the individuals shape the group interaction?” Interactive Learning Environments, vol. 18, no. 4, pp. 365-383, 2010 http://dx.doi.org/10.1080/10494820802602444
[38] O'Reilly, T., McNamara, D.S., “The Impact of Science Knowledge, Reading Skill, and Reading Strategy Knowledge on More Traditional "High-Stakes" Measures of High School Students' Science Achievement”, American Educational Research Journal, vol. 44, no. 1, pp. 161-196, 2007 http://dx.doi.org/10.3102/0002831206298171
[39] Hailikari, T., Nevgi, A., Komulainen, E., “Academic self?beliefs and prior knowledge as predictors of student achievement in Mathematics: a structural model”, Educational Psychology: An International Journal of Experimental Educational Psychology, vol. 28, no. 1, pp. 59-71, 2008 http://dx.doi.org/10.1080/01443410701413753
[2] Bellotti, F., Berta, R., De Gloria, A., Lavagnino, E., Dagnino, F., Ott, M., Romero, M., Usart, M., Mayer, I., “Designing a Course for Stimulating Entrepreneurship in Higher Education through Serious Games." Procedia Computer Science, vol. 15, pp. 174-186, 2012. http://dx.doi.org/10.1016/j.procs.2012.10.069
[3] Mawdesley, M., Long, G., Al-Jibouri, S., Scott, D., “The enhancement of simulation based learning exercises through formalised reflection, focus groups and group presentation”, Computers and Education, vol. 56, no. 1, pp. 44-52, 2011. http://dx.doi.org/10.1016/j.compedu.2010.05.005
[4] Dickey, M. D., “Game Design and Learning: A Conjectural Analysis of How Massively multiple Online Role-Playing Games (MMORPGs) Foster Intrinsic Motivation”, Educational Technology Research and Development, vol. 55, no. 3, pp- 253-273, 2007.
[5] Leemkuil, H., de Jong, T., de Hoog, R., Christoph, N., “KM Quest: A Collaborative Internet Based Simulation Game”, Simulation & Gaming, vol. 34, no. 1, pp. 89-111, 2003. http://dx.doi.org/10.1177/1046878102250605
[6] Herz, C. J., “Gaming the system: What higher education can learn from multiplayer online worlds. The Internet and the University”, Educause Forum on the Future of Higher Education. Retrieved [11/06/2013] from http://www.educause.edu/ir/library/pdf/ffpiu019.pdf
[7] Azevedo, R., Witherspoon, A., Chauncey, A., Burkett, C., Fike, A., “MetaTutor: A MetaCognitive tool for enhancing self-regulated learning”. In R. Pirrone, R. Azevedo, & G. Biswas (Eds.), Proceedings of the AAAI Fall Symposium on Cognitive and Metacognitive Educational Systems (pp. 14-19). Menlo Park, CA: Association for the Advancement of Artificial Intelligence (AAAI) Press, 2010
[8] Azevedo, R., “The role of self-regulation in learning about science with hypermedia”. In D. Robinson & G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning. (pp.127–156), 2010
[9] Järvelä, S., Volet, S., “Motivation in Real-Life, Dynamic, and Interactive Learning Environments: Stretching Constructs and Methodologies”. Journal European Psychologist, vol. 9, no. 4, pp. 193-197, 2004. http://dx.doi.org/10.1027/1016-9040.9.4.193
[10] Merenluoto, K., Lehtinen, E., “Number concept and conceptual change: towards a systemic model of the processes of change”, Learning and Instruction, vol. 14, no. 5, pp. 519-534, 2004. http://dx.doi.org/10.1016/j.learninstruc.2004.06.016
[11] Garner, R., “When children and adults do not use learning strategies: Toward a theory of settings”, Review of Educational Research, vol. 60, pp. 517–529, 1990 http://dx.doi.org/10.3102/00346543060004517
[12] Schraw, G., “Promoting general metacognitive awareness”, Instructional Science, vol. 26, pp. 113–125, 1998 http://dx.doi.org/10.1023/A:1003044231033
[13] Romero, M., Usart, M., Popescu, M., Boyle, E. “Interdisciplinary and International Adaption and Personalization of the MetaVals Serious Games”, Lecture Notes in Computer Science, vol. 7528, pp 59-70, 2012. http://dx.doi.org/10.1007/978-3-642-33687-4_5
[14] Peirce, N., Conlan, O., Wade, V., “Adaptive educational games: Providing non-invasive personalised learning experiences”. In Digital Games and Intelligent Toys Based Education, 2008 Second IEEE International Conference on IEEE, pp. 28-35, 2008 http://dx.doi.org/10.1109/digitel.2008.30
[15] Sawyer, B., “The serious games summit: emergent use of interactive games for solving problems is serious effort”, Computers in Entertainment (CIE) - Theoretical and Practical Computer Applications in Entertainment, vol. 2, no. 1, pp. 5-12, 2004. http://dx.doi.org/10.1145/973801.973811
[16] Moreno-Ger, P., Burgos, D., Sierra, J.L. & Fernández-Manjón, B., “Educational Game Design for Online Education”, Computers in Human Behavior, vol. 24, no. 6, pp. 2530-2540, 2008 http://dx.doi.org/10.1016/j.chb.2008.03.012
[17] Dillenbourg, P., Jarvela, S., Fischer, F., “The evolution of research on computer-supported collaborative learning: From design to orchestration”, Technology-Enhanced Learning, vol. 1, pp. 3–19, 2009 http://dx.doi.org/10.1007/978-1-4020-9827-7_1
[18] Wendel, V., Gutjahr, M., Göbel S., Steinmetz, R., “Designing collaborative multiplayer serious games”, Education and Information Technologies, vol. 18, no. 2, pp. 287-308, 2013 http://dx.doi.org/10.1007/s10639-012-9244-6
[19] Lipponen, L., Rahikainen, M., Lallimo, J., Hakkarainen, K., “Patterns of participation and discourse in elementary students’ computer-supported collaborative learning”, Learning and Instruction, vol. 13, pp. 487–509, 2003 http://dx.doi.org/10.1016/S0959-4752(02)00042-7
[20] Kreijns, K., Kirschner, P.A., Jochems, W., “Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: a review of the research”, Computers in Human Behavior, vol. 19, no. 3, pp. 335-353, 2003 http://dx.doi.org/10.1016/S0747-5632(02)00057-2
[21] Davis, A., Little, P., Staward, B., “Developing an infrastructure for online learning”, In Anderson, T. (ed.) The theory and practice of online learning. Edmonton: AU Press, 2008
[22] Prensky, M., “The Motivation of Game Play or, the REAL 21st century learning revolution”, On The Horizon, vol. 10, no. 1, 2001
[23] Romero, M., Usart, M., Ott, M. ,Earp, J. , de Freitas, S., Arnab, S., “Learning through playing for or against each other? Promoting collaborative learning in digital game based learning”, ECIS 2012 Proceedings. Paper 93. Retrieved November 10, 2012, from http://aisel.aisnet.org/ecis2012/93
[24] Becker, K., “Digital game-based learning once removed: teaching teachers. British Journal of Educational Technology, vol. 38, no. 3, pp. 478-488, 2007 http://dx.doi.org/10.1111/j.1467-8535.2007.00711.x
[25] Jeffrey, L., “Learning orientations: Diversity in higher education”, Learning and Individual Differences, vol. 19, pp. 195–208, 2009 http://dx.doi.org/10.1016/j.lindif.2008.09.004
[26] Munneke, L., Andriessen, J., Kanselaar, G., Kirschner, P., “Supporting interactive argumentation: Influence of representational tools on discussing a wicked problem”, Computers in Human Behavior, 23, no. 3, pp. 1072, 2007 http://dx.doi.org/10.1016/j.chb.2006.10.003
[27] Ducheneaut, N., Yee, N., Nickell, E., Moore, R. J., "Alone Together?" Exploring the Social Dynamics of Massively Multiplayer Online Games (CHI Conference), pp. 407-416, 2006 http://dx.doi.org/10.1145/1124772.1124834
[28] Roebers, C. M., “Confidence judgments in children’s and adults’ event recall and suggestibility”, Developmental Psychology, vol. 38, pp. 1052–1067, 2002 http://dx.doi.org/10.1037/0012-1649.38.6.1052
[29] Kirschner, P.A., Erkens, G., “Cognitive Tools and Mindtools for Collaborative Learning”, Journal of Educational Computing, vol. 35, no. 2, pp. 199–209, 2006 http://dx.doi.org/10.2190/R783-230M-0052-G843
[30] Valdez, A., “Student Metacognitive Monitoring: Predicting Test Achievement from Judgment Accuracy”, International Journal of Higher Education, vol. 2, no. 2, 141-146, 2013 http://dx.doi.org/10.5430/ijhe.v2n2p141
[31] Karpicke, J. D., Butler, A. C., Roediger III, H. L., “Metacognitive strategies in student learning: do students practise retrieval when they study on their own?”, Memory, vol. 17, no. 4, 471-479, 2009 http://dx.doi.org/10.1080/09658210802647009
[32] Swanson, H. L., “Influence of metacognitive knowledge and aptitude on problem solving”, Journal of educational psychology, vol. 82, no. 2, pp. 306, 1990 http://dx.doi.org/10.1037/0022-0663.82.2.306
[33] Swartz, S. M., “Acceptance and accuracy of multiple choice, confidence-level, and essay question formats for graduate students”, Journal of Education for Business, vol. 81, no. 4, pp. 215-220, 2006 http://dx.doi.org/10.3200/JOEB.81.4.215-220
[34] Flannelly, L., “Using feedback to reduce students' judgment bias on test questions”, Journal of Nursing Education, vol. 40, no. 1, 10-16, 2001
[35] Efklides, A., Akilina, S., Petropoulou, M., “Feeling of difficulty: an aspect of monitoring that influences control”, European Journal of Psychology of Education, vol. 15, pp. 461–476, 1999 http://dx.doi.org/10.1007/BF03172973
[36] Brennan, S.E., Chen, X., Dickinson, C.A., Neider, M.B., Zelinsky, G.J., “Coordinating Cognition: The Costs and Benefits of Shared Gaze during Collaborative Search”, Cognition vol. 106, no. 3, pp. 1465– 1477, 2008 http://dx.doi.org/10.1016/j.cognition.2007.05.012
[37] Bluemink, J., Hämäläinen, R., Manninen, T., Järvelä, S., “Group-level analysis on multiplayer game collaboration: how do the individuals shape the group interaction?” Interactive Learning Environments, vol. 18, no. 4, pp. 365-383, 2010 http://dx.doi.org/10.1080/10494820802602444
[38] O'Reilly, T., McNamara, D.S., “The Impact of Science Knowledge, Reading Skill, and Reading Strategy Knowledge on More Traditional "High-Stakes" Measures of High School Students' Science Achievement”, American Educational Research Journal, vol. 44, no. 1, pp. 161-196, 2007 http://dx.doi.org/10.3102/0002831206298171
[39] Hailikari, T., Nevgi, A., Komulainen, E., “Academic self?beliefs and prior knowledge as predictors of student achievement in Mathematics: a structural model”, Educational Psychology: An International Journal of Experimental Educational Psychology, vol. 28, no. 1, pp. 59-71, 2008 http://dx.doi.org/10.1080/01443410701413753
Downloads
Additional Files
Published
2014-02-03
Issue
Section
Articles
License
IJSG copyright information is provided here.
How to Cite
Individual and collaborative Performance and Level of Certainty in MetaVals. (2014). International Journal of Serious Games, 1(1). https://doi.org/10.17083/ijsg.v1i1.3
