Serious Games are a Serious Tool for Team Research
DOI:
https://doi.org/10.17083/ijsg.v4i1.141Keywords:
Game engines, serious games, simulations, team research, teamsAbstract
Serious games are an attractive tool for education and training, but their utility is even broader. We argue serious games provide a unique opportunity for research as well, particularly in areas where multiple players (groups or teams) are involved. In our paper we provide background in several substantive areas. First, we outline major constructs and challenges found in team research. Secondly, we discuss serious games, providing an overview and description of their role in education, training, and research. Thirdly, we describe necessary characteristics for game engines utilized in team research, followed by a discussion of the value added by utilizing serious games. Our goal in this paper is to argue serious games are an effective tool with demonstrated reliability and validity and should be part of a research program for those engaged in team research. Both team researchers and those involved in serious game development can benefit from a mutual partnership which is research focused.
References
[1] Townsend, A. M., DeMarie, S. M., & Hendrickson, A. R. Virtual teams: Technology and the workplace of the future. The Academy of Management Executive, 12(3), 17–29, 1998. https://doi.org/10.5465/ame.1998.1109047
[2] Coovert, M. D. Technological changes in office jobs: What we know and what we can expect. In A. Howard (Ed.), The changing nature of work: Frontiers of industrial and organizational psychology (pp. 175-208). San Francisco, CA: Jossey-Bass, 1995.
[3] Coovert, M. D., & Foster Thompson, L. F. Computer-supported cooperative work: Issues and implications for workers, organizations, and human resource management. Thousand Oaks, CA: Sage, 2001.
[4] Coovert, M. D. and Foster Thompson, L. (Eds.) The Psychology of Workplace Technology. New York, NY: Routledge Taylor & Francis Group, 2014.
[5] Pavlova, E., Coovert, M. D., & Bennett, W. Trust development in computer-mediated teams: A latent change score model. Multivariate Behavioral Research, 48, 166-167, 2013. https://doi.org/10.1080/00273171.2013.752265
[6] Cooke, N. J., Kiekel, P. A., Salas, E., Stout, R., Bowers, C., & Cannon-Bowers, J. A. Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory, Research, and Practice, 7 (3), 179-199, 2003. https://doi.org/10.1037/1089-2699.7.3.179
[7] Brannick, M.T., Roach, R.M., & Salas, E. Understanding team performance: A multidimensional study. Human Performance, 6(4), 287-308, 1993. https://doi.org/10.1207/s15327043hup0604_1
[8] Dickenson, T.L., & McIntyre, R.M. A conceptual framework for teamwork measurement. Team performance assessment and measurement (19-43). In M.T. Brannick, E. Salas, C.W. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications. Taylor & Francis, 1997.
[9] Coovert, M. D. Petri nets: Modeling the complexity of modern jobs. In J. M. Cortina and R. S. Landis (Eds.), Modern research methods for the study of behavior in organizations (pp. 381-404). New York, NY: Routledge Taylor & Francis Group, 2013.
[10] Coovert, M. D., Salas, E., & Cannon-Bowers, J. A. Process models of team behavior. Proceedings of the American Control Conference. IEEE Press, 1991.
[11] Yagoda, R. E., & Coovert, M. D. How to work and play with robots: An approach to modeling human-robot interaction. Computers in Human Behavior, 28(1), 60-68, 2012. https://doi.org/10.1016/j.chb.2011.08.011
[12] Marks, M.A., Mathieu, J.E., & Zaccaro, S.J. A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356-376, 2001.
[13] LePine, J.A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. A meta-analysis of teamwork process: Tests of a multidimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61(2), 273 – 307, 2008. https://doi.org/10.1111/j.1744-6570.2008.00114.x
[14] Cooke, N.J., Salas, E., Cannon-Bowers, J. A., & Stout, R. J. Measuring team knowledge. Human Factors, 42, (1), 151-173, 2000. https://doi.org/10.1518/001872000779656561
[15] Klimoski, R. & Mohammed, S. Team mental model: Construct or metaphor? Journal of Manage,ent, 20, 403-437, 1994.
[16] Langan-Fox, J., Anglim, J. & Wilson, J. R. Mental models, team mental models, and performance: Process, development, and future directions. Human Factors and Ergonomics in Manufacturing, 14(4), 331-352, 2004. https://doi.org/10.1002/hfm.20004
[17] Mohammed, S. & Dumville, B. C. Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22, 89-106, 2001. https://doi.org/10.1002/job.86
[18] Mohammed, S., Klimoski, R. & Rentsch, J. R. The measurement of team mental models: We have no shared schema. Organizational Research Methods, 3(2), 123-165, 2000. https://doi.org/10.1002/job.86
[19] Hess, K. P., Freeman, J., & Coovert, M. D. CENTER: Critical thinking in team decision making. In M. P. Letsky, N. W. Warner, S. M. Fiore, & C.A.P. Smith (Eds.). Macrocognition in teams: Theories and methodologies (pp. 239-258). Burlington, VT: Ashgate Publishing, 2008.
[20] Rosen, M.A., Bedwell, W.L., Wildman, J. L., Fritzsche, B.A., Salas, E, & Burke, C.S., Managing adaptive performance in teams: Guiding principles and behavioral markers for measurement. Human Resource Management Review, 21, 107-122, 2011. https://doi.org/10.1016/j.hrmr.2010.09.003
[21] Burke, C.S., Stagl, K.C., Klein, C., Goodwin, G.F., Salas, E., & Halpin, S.M. What type of leadership behaviors are functional in teams? A meta-analysis. Leadership Quarterly, 17, 288-307, 2006. https://doi.org/10.1016/j.leaqua.2006.02.007
[22] Tuckman, B.W. Developmental sequence in small groups. Psychological Bulletin, 63(6), 384-399, 1965. https://doi.org/10.1037/h0022100
[23] Morgan, Jr. B.B., Glickman, A.S., Woodard, E.A. Blaiwes, A.S., & Salas, E. Measurement of team behaviors is a Navy environment. Battell Columbus Labs, Research Triangle Park, NC, 1986.
[24] Prince, C., & Salas, E. Training and research for teamwork in the military aircrew (337-366). In E. L. Wiener, B. G. Kanki, and R.L. Helmreich (Eds.), Cockpit resource management, Elsiver, 1993.
[25] Fowlkes, J. E., Lane, N.E., Salas, E., Franz, T., & Oser, R., Improving the measurement of team performance: The TARGETs methodology. Military Psychology, 6(1), 47-61, 1994. https://doi.org/10.1207/s15327876mp0601_3
[26] Herz, J. C., & Macedonia, M. R. (2002). Computer games and the military: Two views. Defense Horizons, (11-12), 1.
[27] Shilling, R., Zyda, M., & Wardynski, E. C. (2002, November). Introducing Emotion into Military Simulation and Video Game Design America’s Army: Operations and VIRTE. In GAME-ON.
[28] Moon, I. C., Schneider, M., & Carley, K. M. (2006). Evolution of player skill in the America’s army game. Simulation, 82(11), 703-718. https://doi.org/10.1177/0037549706073689
[29] Farrell, C. M., W. K. Klimack, and C. R. Jacquet. 2003. Employing interactive multimedia instruction in military science education at the U.S. Military Academy. In Proceedings of the Interservice/Industry Training, Simulation, and Education (I/ITSEC) Conference, Orlando, FL.
[30] Alhadeff, E. (2016), Serious Games Market. Retrieved Feb 3, 2017 from: http://seriousgamesmarket.blogspot.com/2016/12/2016-winners-announced-serious-games.html
[31] Crookall, D. Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41(6), 898-920, 2010. https://doi.org/10.1177/1046878110390784
[32] de Freitas, S., & Ketelhut, D.J. Introduction to the Journal of Information Sciences special issue on serious games. Information Sciences, 264, 1-3, 2014. https://doi.org/10.1016/j.ins.2014.01.036
[33] Greco, M., Baldissin, N., & Nonino, F. An exploratory taxonomy of business games. Simulation & Gaming, 44(5), 645-682, 2013. https://doi.org/10.1177/1046878113501464
[34] MONOPOLOGS [Developed by Rand Logistic Department.] Santa Monica: CA. Rand Corporation, 1955.
[35] Abt, C.A. Serious games. Viking, New York, 1970.
[36] Faria, A.J. The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming, 32(1), 97-110, 2001. https://doi.org/10.1177/104687810103200108
[37] Bragge, J., Thavikulwat, P., & Töyli, J. Profiling 40 years of research in Simulation & Gaming. Simulation & Gaming, 41(6), 869-897, 2010. https://doi.org/10.1177/1046878110387539
[38] Feinstein, A. H., & Cannon, H. M. Constructs of simulation evaluation. Simulation & Gaming, 33(4), 425-440, 2002. https://doi.org/10.1177/1046878102238606
[39] Sauvé, L., Renaud, L., Kaufman, D., & Marquis, J.S. Distinguishing between games and simulations: A systematic review. Educational Technology & Society, 10(3), 247-256, 2007.
[40] Pohl, A., & Walker, J. Leveraging game technology: Where are games the right answer to augment military training? Material Science & Technology, 2, 22-26, 2010.
[41] Werbach, K., & Hunter, D. For the win: How thinking can revolutionize your business. Wharton Digital Press: Philidelphia, 2012.
[42] Deterding, S., Dixon, D., Khaled, R., & Nacke, L. From game design elements to gamefullness: Defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. Tampere, Finland, 2011. https://doi.org/10.1145/2181037.2181040
[43] Bailey, P., Pritchard, G., & Koenohan, H. Gamification in market research: Increasing enjoyment and richness of data, but what of data validity? International Journal of Market Research, 57 (1), 17-28, 2015. https://doi.org/10.2501/IJMR-2015-003
[44] Deterding, S. Gamification: Designing for motivation. Interactions, 19(4), 14-17, 2012. https://doi.org/10.1145/2212877.2212883].
[45] Paharia, R. Gamification means applying intrinsic value, 19(4), 14-17, 2012. https://doi.org/1010,1145/2212877.2212883]
[46] Landers, R.N., & Landers, A.K. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simulation & Gaming, 45(6), 769-785, 2015. https://doi.org/10.1177/1046878114563662
[47] Mitgutsch, K., & Alvarado, N. Purposeful by design? A serious game design assessment framework. Proceedings of Foundations of Digital Games (121-128). Association for Computing Machinery, Raleigh: N.C., 2012.
[48] Salen, K., & Zimmerman, E. Rules of play: Game design fundamentals. Cambridge: MIT Press, 2003.
[49] De Gloria, A., Bellotti, F., & Berta, R. (2014). Serious games for education and training. International Journal of Serious Games, 1(1). https://doi.org/10.17083/ijsg.v1i1.11
[50] Connolly, T.M., Boye, E.A., MacArthur, E., Hainey, T., & Boyle, J.M. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661-686, 2012. https://doi.org/10.1016/j.compedu.2012.03.004
[51] Ortiz, S. (2015). Video game self-efficacy and its effect on training performance. International Journal of Serious Games, 2(3). https://doi.org/10.17083/ijsg.v2i3.89
[52] Sitzmann, T. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528, 2011. https://doi.org/10.1111/j.1744-6570.2011.01190.x
[53] Bedwell, W.L., Pavlas, D., Heyne, K., Lazzara, E. H., & Salas, E. Toward a taxonomy linking game attributes to learning: An empirical study. Simulation & Gaming, 43(6), 729-760, 2012. https://doi.org/10.1177/1046878112439444
[54] Landers, R.N. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752-768, 2015. https://doi.org/10.1177/1046878114563660
[55] Ghannem, A. Characterization of serious games guided by the educational objectives. Proceedings of TEEM’14 (pp. 227-233). Association for Computing Machinery, 2014. https://doi.org/10.1145/2669711.2669904
[56] Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., van Ruijven, T., Lo, J., Kortmann, R., & Wenzler, I. The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Psychology, 45(3), 502-527, 2014. https://doi.org/10.1111/bjet.12067
[57] de Klerk, S., Veldkamp, B.P., & Eggen, T.J.H.M. Psychometric analysis of the performance data of simulation-based assessment: A systematic review and a Bayesian network example. Computers & Education, 85, 23-34, 2015. https://doi.org/10.1016/j.compedu.2014.12.020
[58] Chorianopoulos, K., Giannakos, M.N., & Chrisochoides, N. Design principles for serious games in mathematics. Proceedings of the 18th Panhellenic Conference on Informatics (pp. 1-5). Association for Computing Machines, 2014. https://doi.org/10.1145/2645791.2645843
[59] Margolis, M.J., & Clauser, B.E. A regression-based procedure for automated scoring of a complex medical performance assessment. In. D.M. Williamson, I. I. Bejar, and R.J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123-168). Mahwah, NJ: Lawrence Erlbaum Associates, 2006.
[60] Coovert, M. D., Winner, J., & Bennett, W. Construct development and validation in game-based research. Simulation & Gaming, 47(6),1-13, 2016. https://doi.org/10.1177/1046878116682661
[61] Mäyrä, F., Holopainen, J., Jakobsson, M. Research methodology in gaming: An overview. Simulation & Gaming, 43(3), 295-299, 2012. https://doi.org/10.1177/1046878112439508
[62] de Caluwé, Geurts, J., & Kleinlugtenbelt, W.J. (2012). Gaming research in policy and organization: An assessment from the Netherlands. Simulation & Gaming, 43(5), 600-626. https://doi.org/10.1177/1046878112439445
[63] Eladhari, M.P., & Ollila, E.M.I. Design for research results: Experimental prototyping and play testing. Simulation & Gaming, 43(3), 391-412, 2012. https://doi.org/10.1177/1046878111434255
[64] Protopsaltis, A., & Auneau, L. Scenario-based serious game repurposing. Proceedings of the 29th ACM International Conference of Design of Communications (37-43). Association for Computing Machinery, 2011. https://doi.org/10.1145/2038476.2038484
[65] Campbell, D.T., & Fiske, D.W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105, 1959. https://doi.org/10.1037/h0046016
[66] Widaman, K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9(1), 1-26, 1985. https://doi.org/10.1177/014662168500900101
[67] Jöreskog, K. G. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183-202, 1969. https://doi.org/10.1007/BF02289343
[68] Marsh, H.W., Balla, J.R., & McDonald, R.P. Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391-410, 1988. https://doi.org/10.1037/0033-2909.103.3.391
[69] Lewis, M., & Jacobson, J. Game engines in scientific research. Communications of the ACM, 45(1), 27-31, 2002.
[70] Hendrix, M., Al-Sherbaz, A., & Victoria, B. (2016). Game based cyber security training: are serious games suitable for cyber security training?. International Journal of Serious Games, 3(1). https://doi.org/10.17083/ijsg.v3i1.107
[71] Gorman, J. C., Cooke, N. J., Pederson, H. K., Connor, O. O., & DeJoode, J. A. Coordinated awareness of situation by teams (CAST): Measuring team situation awareness of a communication glitch. Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting,Orlando, FL, 274-277, 2005. https://doi.org/10.1177/154193120504900313
[72] Gorman, J. C., Cooke, N. J., & Winner, J. L. Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312-1325, 2006 https://doi.org/10.1080/00140130600612788
[73] Aptima, Inc. Distributed Dynamic Decision-making: The DDD Team simulation. https://github.com/aptima/ddd (2005)
[74] Crookall, D. Debriefing. Simulation & Gaming, 23(2), 141-142, 1992. https://doi.org/10.1177/1046878192232001
[75] Lederman, L.C. Debriefing: Toward a systematic assessment of theory and practice. Simulation & Games, 23(2), 145-160, 1992. https://doi.org/10.1177/1046878192232003
[76] Stewart, L.P. Ethical issues in postexperimental and postexperiential debriefing. Simulation & Gaming, 23(2), 196-211, 1992. https://doi.org/10.1177/1046878192232007
[77] APA Ethical principles of psychologists and code of conduct. American Psychological Association. APA, Washington D.C, 2010.
[78] Lederman, L.C. Debriefing: A critical reexamination of the postexperience analytic process with implications for its effective use. Simulation & Games, 15(4), 415-431, 1984. https://doi.org/10.1177/0037550084154002
[79] Charsky, D. From edutainment to serious games: A change in the use of game characteristics. Games and culture, 5(2), 177-198, 2010. https://doi.org/10.1177/1555412009354727
[80] Van Ments, M. The effective use of role-play. London: Kogan Page, 1989.
[81] Steinwachs, B. How to facilitate a debriefing. Simulation & Gaming, 23(2), 186-195, 1992. https://doi.org/10.1177/1046878192232006
[82] Thiagarajan, S. Using games for debriefing. Simulation & Gaming, 23(2), 161-173, 1992. https://doi.org/10.1177/1046878192232004
[83] Academy, Serious Games Society. http://academy.seriousgamessociety.org/about. Retrieved March 15, 2017.
[2] Coovert, M. D. Technological changes in office jobs: What we know and what we can expect. In A. Howard (Ed.), The changing nature of work: Frontiers of industrial and organizational psychology (pp. 175-208). San Francisco, CA: Jossey-Bass, 1995.
[3] Coovert, M. D., & Foster Thompson, L. F. Computer-supported cooperative work: Issues and implications for workers, organizations, and human resource management. Thousand Oaks, CA: Sage, 2001.
[4] Coovert, M. D. and Foster Thompson, L. (Eds.) The Psychology of Workplace Technology. New York, NY: Routledge Taylor & Francis Group, 2014.
[5] Pavlova, E., Coovert, M. D., & Bennett, W. Trust development in computer-mediated teams: A latent change score model. Multivariate Behavioral Research, 48, 166-167, 2013. https://doi.org/10.1080/00273171.2013.752265
[6] Cooke, N. J., Kiekel, P. A., Salas, E., Stout, R., Bowers, C., & Cannon-Bowers, J. A. Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory, Research, and Practice, 7 (3), 179-199, 2003. https://doi.org/10.1037/1089-2699.7.3.179
[7] Brannick, M.T., Roach, R.M., & Salas, E. Understanding team performance: A multidimensional study. Human Performance, 6(4), 287-308, 1993. https://doi.org/10.1207/s15327043hup0604_1
[8] Dickenson, T.L., & McIntyre, R.M. A conceptual framework for teamwork measurement. Team performance assessment and measurement (19-43). In M.T. Brannick, E. Salas, C.W. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications. Taylor & Francis, 1997.
[9] Coovert, M. D. Petri nets: Modeling the complexity of modern jobs. In J. M. Cortina and R. S. Landis (Eds.), Modern research methods for the study of behavior in organizations (pp. 381-404). New York, NY: Routledge Taylor & Francis Group, 2013.
[10] Coovert, M. D., Salas, E., & Cannon-Bowers, J. A. Process models of team behavior. Proceedings of the American Control Conference. IEEE Press, 1991.
[11] Yagoda, R. E., & Coovert, M. D. How to work and play with robots: An approach to modeling human-robot interaction. Computers in Human Behavior, 28(1), 60-68, 2012. https://doi.org/10.1016/j.chb.2011.08.011
[12] Marks, M.A., Mathieu, J.E., & Zaccaro, S.J. A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356-376, 2001.
[13] LePine, J.A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. A meta-analysis of teamwork process: Tests of a multidimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61(2), 273 – 307, 2008. https://doi.org/10.1111/j.1744-6570.2008.00114.x
[14] Cooke, N.J., Salas, E., Cannon-Bowers, J. A., & Stout, R. J. Measuring team knowledge. Human Factors, 42, (1), 151-173, 2000. https://doi.org/10.1518/001872000779656561
[15] Klimoski, R. & Mohammed, S. Team mental model: Construct or metaphor? Journal of Manage,ent, 20, 403-437, 1994.
[16] Langan-Fox, J., Anglim, J. & Wilson, J. R. Mental models, team mental models, and performance: Process, development, and future directions. Human Factors and Ergonomics in Manufacturing, 14(4), 331-352, 2004. https://doi.org/10.1002/hfm.20004
[17] Mohammed, S. & Dumville, B. C. Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22, 89-106, 2001. https://doi.org/10.1002/job.86
[18] Mohammed, S., Klimoski, R. & Rentsch, J. R. The measurement of team mental models: We have no shared schema. Organizational Research Methods, 3(2), 123-165, 2000. https://doi.org/10.1002/job.86
[19] Hess, K. P., Freeman, J., & Coovert, M. D. CENTER: Critical thinking in team decision making. In M. P. Letsky, N. W. Warner, S. M. Fiore, & C.A.P. Smith (Eds.). Macrocognition in teams: Theories and methodologies (pp. 239-258). Burlington, VT: Ashgate Publishing, 2008.
[20] Rosen, M.A., Bedwell, W.L., Wildman, J. L., Fritzsche, B.A., Salas, E, & Burke, C.S., Managing adaptive performance in teams: Guiding principles and behavioral markers for measurement. Human Resource Management Review, 21, 107-122, 2011. https://doi.org/10.1016/j.hrmr.2010.09.003
[21] Burke, C.S., Stagl, K.C., Klein, C., Goodwin, G.F., Salas, E., & Halpin, S.M. What type of leadership behaviors are functional in teams? A meta-analysis. Leadership Quarterly, 17, 288-307, 2006. https://doi.org/10.1016/j.leaqua.2006.02.007
[22] Tuckman, B.W. Developmental sequence in small groups. Psychological Bulletin, 63(6), 384-399, 1965. https://doi.org/10.1037/h0022100
[23] Morgan, Jr. B.B., Glickman, A.S., Woodard, E.A. Blaiwes, A.S., & Salas, E. Measurement of team behaviors is a Navy environment. Battell Columbus Labs, Research Triangle Park, NC, 1986.
[24] Prince, C., & Salas, E. Training and research for teamwork in the military aircrew (337-366). In E. L. Wiener, B. G. Kanki, and R.L. Helmreich (Eds.), Cockpit resource management, Elsiver, 1993.
[25] Fowlkes, J. E., Lane, N.E., Salas, E., Franz, T., & Oser, R., Improving the measurement of team performance: The TARGETs methodology. Military Psychology, 6(1), 47-61, 1994. https://doi.org/10.1207/s15327876mp0601_3
[26] Herz, J. C., & Macedonia, M. R. (2002). Computer games and the military: Two views. Defense Horizons, (11-12), 1.
[27] Shilling, R., Zyda, M., & Wardynski, E. C. (2002, November). Introducing Emotion into Military Simulation and Video Game Design America’s Army: Operations and VIRTE. In GAME-ON.
[28] Moon, I. C., Schneider, M., & Carley, K. M. (2006). Evolution of player skill in the America’s army game. Simulation, 82(11), 703-718. https://doi.org/10.1177/0037549706073689
[29] Farrell, C. M., W. K. Klimack, and C. R. Jacquet. 2003. Employing interactive multimedia instruction in military science education at the U.S. Military Academy. In Proceedings of the Interservice/Industry Training, Simulation, and Education (I/ITSEC) Conference, Orlando, FL.
[30] Alhadeff, E. (2016), Serious Games Market. Retrieved Feb 3, 2017 from: http://seriousgamesmarket.blogspot.com/2016/12/2016-winners-announced-serious-games.html
[31] Crookall, D. Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41(6), 898-920, 2010. https://doi.org/10.1177/1046878110390784
[32] de Freitas, S., & Ketelhut, D.J. Introduction to the Journal of Information Sciences special issue on serious games. Information Sciences, 264, 1-3, 2014. https://doi.org/10.1016/j.ins.2014.01.036
[33] Greco, M., Baldissin, N., & Nonino, F. An exploratory taxonomy of business games. Simulation & Gaming, 44(5), 645-682, 2013. https://doi.org/10.1177/1046878113501464
[34] MONOPOLOGS [Developed by Rand Logistic Department.] Santa Monica: CA. Rand Corporation, 1955.
[35] Abt, C.A. Serious games. Viking, New York, 1970.
[36] Faria, A.J. The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming, 32(1), 97-110, 2001. https://doi.org/10.1177/104687810103200108
[37] Bragge, J., Thavikulwat, P., & Töyli, J. Profiling 40 years of research in Simulation & Gaming. Simulation & Gaming, 41(6), 869-897, 2010. https://doi.org/10.1177/1046878110387539
[38] Feinstein, A. H., & Cannon, H. M. Constructs of simulation evaluation. Simulation & Gaming, 33(4), 425-440, 2002. https://doi.org/10.1177/1046878102238606
[39] Sauvé, L., Renaud, L., Kaufman, D., & Marquis, J.S. Distinguishing between games and simulations: A systematic review. Educational Technology & Society, 10(3), 247-256, 2007.
[40] Pohl, A., & Walker, J. Leveraging game technology: Where are games the right answer to augment military training? Material Science & Technology, 2, 22-26, 2010.
[41] Werbach, K., & Hunter, D. For the win: How thinking can revolutionize your business. Wharton Digital Press: Philidelphia, 2012.
[42] Deterding, S., Dixon, D., Khaled, R., & Nacke, L. From game design elements to gamefullness: Defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. Tampere, Finland, 2011. https://doi.org/10.1145/2181037.2181040
[43] Bailey, P., Pritchard, G., & Koenohan, H. Gamification in market research: Increasing enjoyment and richness of data, but what of data validity? International Journal of Market Research, 57 (1), 17-28, 2015. https://doi.org/10.2501/IJMR-2015-003
[44] Deterding, S. Gamification: Designing for motivation. Interactions, 19(4), 14-17, 2012. https://doi.org/10.1145/2212877.2212883].
[45] Paharia, R. Gamification means applying intrinsic value, 19(4), 14-17, 2012. https://doi.org/1010,1145/2212877.2212883]
[46] Landers, R.N., & Landers, A.K. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simulation & Gaming, 45(6), 769-785, 2015. https://doi.org/10.1177/1046878114563662
[47] Mitgutsch, K., & Alvarado, N. Purposeful by design? A serious game design assessment framework. Proceedings of Foundations of Digital Games (121-128). Association for Computing Machinery, Raleigh: N.C., 2012.
[48] Salen, K., & Zimmerman, E. Rules of play: Game design fundamentals. Cambridge: MIT Press, 2003.
[49] De Gloria, A., Bellotti, F., & Berta, R. (2014). Serious games for education and training. International Journal of Serious Games, 1(1). https://doi.org/10.17083/ijsg.v1i1.11
[50] Connolly, T.M., Boye, E.A., MacArthur, E., Hainey, T., & Boyle, J.M. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661-686, 2012. https://doi.org/10.1016/j.compedu.2012.03.004
[51] Ortiz, S. (2015). Video game self-efficacy and its effect on training performance. International Journal of Serious Games, 2(3). https://doi.org/10.17083/ijsg.v2i3.89
[52] Sitzmann, T. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528, 2011. https://doi.org/10.1111/j.1744-6570.2011.01190.x
[53] Bedwell, W.L., Pavlas, D., Heyne, K., Lazzara, E. H., & Salas, E. Toward a taxonomy linking game attributes to learning: An empirical study. Simulation & Gaming, 43(6), 729-760, 2012. https://doi.org/10.1177/1046878112439444
[54] Landers, R.N. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752-768, 2015. https://doi.org/10.1177/1046878114563660
[55] Ghannem, A. Characterization of serious games guided by the educational objectives. Proceedings of TEEM’14 (pp. 227-233). Association for Computing Machinery, 2014. https://doi.org/10.1145/2669711.2669904
[56] Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., van Ruijven, T., Lo, J., Kortmann, R., & Wenzler, I. The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Psychology, 45(3), 502-527, 2014. https://doi.org/10.1111/bjet.12067
[57] de Klerk, S., Veldkamp, B.P., & Eggen, T.J.H.M. Psychometric analysis of the performance data of simulation-based assessment: A systematic review and a Bayesian network example. Computers & Education, 85, 23-34, 2015. https://doi.org/10.1016/j.compedu.2014.12.020
[58] Chorianopoulos, K., Giannakos, M.N., & Chrisochoides, N. Design principles for serious games in mathematics. Proceedings of the 18th Panhellenic Conference on Informatics (pp. 1-5). Association for Computing Machines, 2014. https://doi.org/10.1145/2645791.2645843
[59] Margolis, M.J., & Clauser, B.E. A regression-based procedure for automated scoring of a complex medical performance assessment. In. D.M. Williamson, I. I. Bejar, and R.J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123-168). Mahwah, NJ: Lawrence Erlbaum Associates, 2006.
[60] Coovert, M. D., Winner, J., & Bennett, W. Construct development and validation in game-based research. Simulation & Gaming, 47(6),1-13, 2016. https://doi.org/10.1177/1046878116682661
[61] Mäyrä, F., Holopainen, J., Jakobsson, M. Research methodology in gaming: An overview. Simulation & Gaming, 43(3), 295-299, 2012. https://doi.org/10.1177/1046878112439508
[62] de Caluwé, Geurts, J., & Kleinlugtenbelt, W.J. (2012). Gaming research in policy and organization: An assessment from the Netherlands. Simulation & Gaming, 43(5), 600-626. https://doi.org/10.1177/1046878112439445
[63] Eladhari, M.P., & Ollila, E.M.I. Design for research results: Experimental prototyping and play testing. Simulation & Gaming, 43(3), 391-412, 2012. https://doi.org/10.1177/1046878111434255
[64] Protopsaltis, A., & Auneau, L. Scenario-based serious game repurposing. Proceedings of the 29th ACM International Conference of Design of Communications (37-43). Association for Computing Machinery, 2011. https://doi.org/10.1145/2038476.2038484
[65] Campbell, D.T., & Fiske, D.W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105, 1959. https://doi.org/10.1037/h0046016
[66] Widaman, K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9(1), 1-26, 1985. https://doi.org/10.1177/014662168500900101
[67] Jöreskog, K. G. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183-202, 1969. https://doi.org/10.1007/BF02289343
[68] Marsh, H.W., Balla, J.R., & McDonald, R.P. Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391-410, 1988. https://doi.org/10.1037/0033-2909.103.3.391
[69] Lewis, M., & Jacobson, J. Game engines in scientific research. Communications of the ACM, 45(1), 27-31, 2002.
[70] Hendrix, M., Al-Sherbaz, A., & Victoria, B. (2016). Game based cyber security training: are serious games suitable for cyber security training?. International Journal of Serious Games, 3(1). https://doi.org/10.17083/ijsg.v3i1.107
[71] Gorman, J. C., Cooke, N. J., Pederson, H. K., Connor, O. O., & DeJoode, J. A. Coordinated awareness of situation by teams (CAST): Measuring team situation awareness of a communication glitch. Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting,Orlando, FL, 274-277, 2005. https://doi.org/10.1177/154193120504900313
[72] Gorman, J. C., Cooke, N. J., & Winner, J. L. Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312-1325, 2006 https://doi.org/10.1080/00140130600612788
[73] Aptima, Inc. Distributed Dynamic Decision-making: The DDD Team simulation. https://github.com/aptima/ddd (2005)
[74] Crookall, D. Debriefing. Simulation & Gaming, 23(2), 141-142, 1992. https://doi.org/10.1177/1046878192232001
[75] Lederman, L.C. Debriefing: Toward a systematic assessment of theory and practice. Simulation & Games, 23(2), 145-160, 1992. https://doi.org/10.1177/1046878192232003
[76] Stewart, L.P. Ethical issues in postexperimental and postexperiential debriefing. Simulation & Gaming, 23(2), 196-211, 1992. https://doi.org/10.1177/1046878192232007
[77] APA Ethical principles of psychologists and code of conduct. American Psychological Association. APA, Washington D.C, 2010.
[78] Lederman, L.C. Debriefing: A critical reexamination of the postexperience analytic process with implications for its effective use. Simulation & Games, 15(4), 415-431, 1984. https://doi.org/10.1177/0037550084154002
[79] Charsky, D. From edutainment to serious games: A change in the use of game characteristics. Games and culture, 5(2), 177-198, 2010. https://doi.org/10.1177/1555412009354727
[80] Van Ments, M. The effective use of role-play. London: Kogan Page, 1989.
[81] Steinwachs, B. How to facilitate a debriefing. Simulation & Gaming, 23(2), 186-195, 1992. https://doi.org/10.1177/1046878192232006
[82] Thiagarajan, S. Using games for debriefing. Simulation & Gaming, 23(2), 161-173, 1992. https://doi.org/10.1177/1046878192232004
[83] Academy, Serious Games Society. http://academy.seriousgamessociety.org/about. Retrieved March 15, 2017.
Downloads
Published
2017-03-22
Issue
Section
Articles
License
IJSG copyright information is provided here.
How to Cite
Serious Games are a Serious Tool for Team Research. (2017). International Journal of Serious Games, 4(1). https://doi.org/10.17083/ijsg.v4i1.141
