Serious Games are a Serious Tool for Team Research
Keywords:Game engines, serious games, simulations, team research, teams
Serious games are an attractive tool for education and training, but their utility is even broader. We argue serious games provide a unique opportunity for research as well, particularly in areas where multiple players (groups or teams) are involved. In our paper we provide background in several substantive areas. First, we outline major constructs and challenges found in team research. Secondly, we discuss serious games, providing an overview and description of their role in education, training, and research. Thirdly, we describe necessary characteristics for game engines utilized in team research, followed by a discussion of the value added by utilizing serious games. Our goal in this paper is to argue serious games are an effective tool with demonstrated reliability and validity and should be part of a research program for those engaged in team research. Both team researchers and those involved in serious game development can benefit from a mutual partnership which is research focused.
 Coovert, M. D. Technological changes in office jobs: What we know and what we can expect. In A. Howard (Ed.), The changing nature of work: Frontiers of industrial and organizational psychology (pp. 175-208). San Francisco, CA: Jossey-Bass, 1995.
 Coovert, M. D., & Foster Thompson, L. F. Computer-supported cooperative work: Issues and implications for workers, organizations, and human resource management. Thousand Oaks, CA: Sage, 2001.
 Coovert, M. D. and Foster Thompson, L. (Eds.) The Psychology of Workplace Technology. New York, NY: Routledge Taylor & Francis Group, 2014.
 Pavlova, E., Coovert, M. D., & Bennett, W. Trust development in computer-mediated teams: A latent change score model. Multivariate Behavioral Research, 48, 166-167, 2013. https://doi.org/10.1080/00273171.2013.752265
 Cooke, N. J., Kiekel, P. A., Salas, E., Stout, R., Bowers, C., & Cannon-Bowers, J. A. Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory, Research, and Practice, 7 (3), 179-199, 2003. https://doi.org/10.1037/1089-26184.108.40.206
 Brannick, M.T., Roach, R.M., & Salas, E. Understanding team performance: A multidimensional study. Human Performance, 6(4), 287-308, 1993. https://doi.org/10.1207/s15327043hup0604_1
 Dickenson, T.L., & McIntyre, R.M. A conceptual framework for teamwork measurement. Team performance assessment and measurement (19-43). In M.T. Brannick, E. Salas, C.W. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications. Taylor & Francis, 1997.
 Coovert, M. D. Petri nets: Modeling the complexity of modern jobs. In J. M. Cortina and R. S. Landis (Eds.), Modern research methods for the study of behavior in organizations (pp. 381-404). New York, NY: Routledge Taylor & Francis Group, 2013.
 Coovert, M. D., Salas, E., & Cannon-Bowers, J. A. Process models of team behavior. Proceedings of the American Control Conference. IEEE Press, 1991.
 Yagoda, R. E., & Coovert, M. D. How to work and play with robots: An approach to modeling human-robot interaction. Computers in Human Behavior, 28(1), 60-68, 2012. https://doi.org/10.1016/j.chb.2011.08.011
 Marks, M.A., Mathieu, J.E., & Zaccaro, S.J. A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356-376, 2001.
 LePine, J.A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. A meta-analysis of teamwork process: Tests of a multidimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61(2), 273 – 307, 2008. https://doi.org/10.1111/j.1744-6570.2008.00114.x
 Cooke, N.J., Salas, E., Cannon-Bowers, J. A., & Stout, R. J. Measuring team knowledge. Human Factors, 42, (1), 151-173, 2000. https://doi.org/10.1518/001872000779656561
 Klimoski, R. & Mohammed, S. Team mental model: Construct or metaphor? Journal of Manage,ent, 20, 403-437, 1994.
 Langan-Fox, J., Anglim, J. & Wilson, J. R. Mental models, team mental models, and performance: Process, development, and future directions. Human Factors and Ergonomics in Manufacturing, 14(4), 331-352, 2004. https://doi.org/10.1002/hfm.20004
 Mohammed, S. & Dumville, B. C. Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22, 89-106, 2001. https://doi.org/10.1002/job.86
 Mohammed, S., Klimoski, R. & Rentsch, J. R. The measurement of team mental models: We have no shared schema. Organizational Research Methods, 3(2), 123-165, 2000. https://doi.org/10.1002/job.86
 Hess, K. P., Freeman, J., & Coovert, M. D. CENTER: Critical thinking in team decision making. In M. P. Letsky, N. W. Warner, S. M. Fiore, & C.A.P. Smith (Eds.). Macrocognition in teams: Theories and methodologies (pp. 239-258). Burlington, VT: Ashgate Publishing, 2008.
 Rosen, M.A., Bedwell, W.L., Wildman, J. L., Fritzsche, B.A., Salas, E, & Burke, C.S., Managing adaptive performance in teams: Guiding principles and behavioral markers for measurement. Human Resource Management Review, 21, 107-122, 2011. https://doi.org/10.1016/j.hrmr.2010.09.003
 Burke, C.S., Stagl, K.C., Klein, C., Goodwin, G.F., Salas, E., & Halpin, S.M. What type of leadership behaviors are functional in teams? A meta-analysis. Leadership Quarterly, 17, 288-307, 2006. https://doi.org/10.1016/j.leaqua.2006.02.007
 Tuckman, B.W. Developmental sequence in small groups. Psychological Bulletin, 63(6), 384-399, 1965. https://doi.org/10.1037/h0022100
 Morgan, Jr. B.B., Glickman, A.S., Woodard, E.A. Blaiwes, A.S., & Salas, E. Measurement of team behaviors is a Navy environment. Battell Columbus Labs, Research Triangle Park, NC, 1986.
 Prince, C., & Salas, E. Training and research for teamwork in the military aircrew (337-366). In E. L. Wiener, B. G. Kanki, and R.L. Helmreich (Eds.), Cockpit resource management, Elsiver, 1993.
 Fowlkes, J. E., Lane, N.E., Salas, E., Franz, T., & Oser, R., Improving the measurement of team performance: The TARGETs methodology. Military Psychology, 6(1), 47-61, 1994. https://doi.org/10.1207/s15327876mp0601_3
 Herz, J. C., & Macedonia, M. R. (2002). Computer games and the military: Two views. Defense Horizons, (11-12), 1.
 Shilling, R., Zyda, M., & Wardynski, E. C. (2002, November). Introducing Emotion into Military Simulation and Video Game Design America’s Army: Operations and VIRTE. In GAME-ON.
 Moon, I. C., Schneider, M., & Carley, K. M. (2006). Evolution of player skill in the America’s army game. Simulation, 82(11), 703-718. https://doi.org/10.1177/0037549706073689
 Farrell, C. M., W. K. Klimack, and C. R. Jacquet. 2003. Employing interactive multimedia instruction in military science education at the U.S. Military Academy. In Proceedings of the Interservice/Industry Training, Simulation, and Education (I/ITSEC) Conference, Orlando, FL.
 Alhadeff, E. (2016), Serious Games Market. Retrieved Feb 3, 2017 from: http://seriousgamesmarket.blogspot.com/2016/12/2016-winners-announced-serious-games.html
 Crookall, D. Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41(6), 898-920, 2010. https://doi.org/10.1177/1046878110390784
 de Freitas, S., & Ketelhut, D.J. Introduction to the Journal of Information Sciences special issue on serious games. Information Sciences, 264, 1-3, 2014. https://doi.org/10.1016/j.ins.2014.01.036
 Greco, M., Baldissin, N., & Nonino, F. An exploratory taxonomy of business games. Simulation & Gaming, 44(5), 645-682, 2013. https://doi.org/10.1177/1046878113501464
 MONOPOLOGS [Developed by Rand Logistic Department.] Santa Monica: CA. Rand Corporation, 1955.
 Abt, C.A. Serious games. Viking, New York, 1970.
 Faria, A.J. The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming, 32(1), 97-110, 2001. https://doi.org/10.1177/104687810103200108
 Bragge, J., Thavikulwat, P., & Töyli, J. Profiling 40 years of research in Simulation & Gaming. Simulation & Gaming, 41(6), 869-897, 2010. https://doi.org/10.1177/1046878110387539
 Feinstein, A. H., & Cannon, H. M. Constructs of simulation evaluation. Simulation & Gaming, 33(4), 425-440, 2002. https://doi.org/10.1177/1046878102238606
 Sauvé, L., Renaud, L., Kaufman, D., & Marquis, J.S. Distinguishing between games and simulations: A systematic review. Educational Technology & Society, 10(3), 247-256, 2007.
 Pohl, A., & Walker, J. Leveraging game technology: Where are games the right answer to augment military training? Material Science & Technology, 2, 22-26, 2010.
 Werbach, K., & Hunter, D. For the win: How thinking can revolutionize your business. Wharton Digital Press: Philidelphia, 2012.
 Deterding, S., Dixon, D., Khaled, R., & Nacke, L. From game design elements to gamefullness: Defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. Tampere, Finland, 2011. https://doi.org/10.1145/2181037.2181040
 Bailey, P., Pritchard, G., & Koenohan, H. Gamification in market research: Increasing enjoyment and richness of data, but what of data validity? International Journal of Market Research, 57 (1), 17-28, 2015. https://doi.org/10.2501/IJMR-2015-003
 Deterding, S. Gamification: Designing for motivation. Interactions, 19(4), 14-17, 2012. https://doi.org/10.1145/2212877.2212883].
 Paharia, R. Gamification means applying intrinsic value, 19(4), 14-17, 2012. https://doi.org/1010,1145/2212877.2212883]
 Landers, R.N., & Landers, A.K. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simulation & Gaming, 45(6), 769-785, 2015. https://doi.org/10.1177/1046878114563662
 Mitgutsch, K., & Alvarado, N. Purposeful by design? A serious game design assessment framework. Proceedings of Foundations of Digital Games (121-128). Association for Computing Machinery, Raleigh: N.C., 2012.
 Salen, K., & Zimmerman, E. Rules of play: Game design fundamentals. Cambridge: MIT Press, 2003.
 De Gloria, A., Bellotti, F., & Berta, R. (2014). Serious games for education and training. International Journal of Serious Games, 1(1). https://doi.org/10.17083/ijsg.v1i1.11
 Connolly, T.M., Boye, E.A., MacArthur, E., Hainey, T., & Boyle, J.M. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661-686, 2012. https://doi.org/10.1016/j.compedu.2012.03.004
 Ortiz, S. (2015). Video game self-efficacy and its effect on training performance. International Journal of Serious Games, 2(3). https://doi.org/10.17083/ijsg.v2i3.89
 Sitzmann, T. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528, 2011. https://doi.org/10.1111/j.1744-6570.2011.01190.x
 Bedwell, W.L., Pavlas, D., Heyne, K., Lazzara, E. H., & Salas, E. Toward a taxonomy linking game attributes to learning: An empirical study. Simulation & Gaming, 43(6), 729-760, 2012. https://doi.org/10.1177/1046878112439444
 Landers, R.N. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752-768, 2015. https://doi.org/10.1177/1046878114563660
 Ghannem, A. Characterization of serious games guided by the educational objectives. Proceedings of TEEM’14 (pp. 227-233). Association for Computing Machinery, 2014. https://doi.org/10.1145/2669711.2669904
 Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., van Ruijven, T., Lo, J., Kortmann, R., & Wenzler, I. The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Psychology, 45(3), 502-527, 2014. https://doi.org/10.1111/bjet.12067
 de Klerk, S., Veldkamp, B.P., & Eggen, T.J.H.M. Psychometric analysis of the performance data of simulation-based assessment: A systematic review and a Bayesian network example. Computers & Education, 85, 23-34, 2015. https://doi.org/10.1016/j.compedu.2014.12.020
 Chorianopoulos, K., Giannakos, M.N., & Chrisochoides, N. Design principles for serious games in mathematics. Proceedings of the 18th Panhellenic Conference on Informatics (pp. 1-5). Association for Computing Machines, 2014. https://doi.org/10.1145/2645791.2645843
 Margolis, M.J., & Clauser, B.E. A regression-based procedure for automated scoring of a complex medical performance assessment. In. D.M. Williamson, I. I. Bejar, and R.J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123-168). Mahwah, NJ: Lawrence Erlbaum Associates, 2006.
 Coovert, M. D., Winner, J., & Bennett, W. Construct development and validation in game-based research. Simulation & Gaming, 47(6),1-13, 2016. https://doi.org/10.1177/1046878116682661
 Mäyrä, F., Holopainen, J., Jakobsson, M. Research methodology in gaming: An overview. Simulation & Gaming, 43(3), 295-299, 2012. https://doi.org/10.1177/1046878112439508
 de Caluwé, Geurts, J., & Kleinlugtenbelt, W.J. (2012). Gaming research in policy and organization: An assessment from the Netherlands. Simulation & Gaming, 43(5), 600-626. https://doi.org/10.1177/1046878112439445
 Eladhari, M.P., & Ollila, E.M.I. Design for research results: Experimental prototyping and play testing. Simulation & Gaming, 43(3), 391-412, 2012. https://doi.org/10.1177/1046878111434255
 Protopsaltis, A., & Auneau, L. Scenario-based serious game repurposing. Proceedings of the 29th ACM International Conference of Design of Communications (37-43). Association for Computing Machinery, 2011. https://doi.org/10.1145/2038476.2038484
 Campbell, D.T., & Fiske, D.W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105, 1959. https://doi.org/10.1037/h0046016
 Widaman, K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9(1), 1-26, 1985. https://doi.org/10.1177/014662168500900101
 Jöreskog, K. G. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183-202, 1969. https://doi.org/10.1007/BF02289343
 Marsh, H.W., Balla, J.R., & McDonald, R.P. Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391-410, 1988. https://doi.org/10.1037/0033-2909.103.3.391
 Lewis, M., & Jacobson, J. Game engines in scientific research. Communications of the ACM, 45(1), 27-31, 2002.
 Hendrix, M., Al-Sherbaz, A., & Victoria, B. (2016). Game based cyber security training: are serious games suitable for cyber security training?. International Journal of Serious Games, 3(1). https://doi.org/10.17083/ijsg.v3i1.107
 Gorman, J. C., Cooke, N. J., Pederson, H. K., Connor, O. O., & DeJoode, J. A. Coordinated awareness of situation by teams (CAST): Measuring team situation awareness of a communication glitch. Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting,Orlando, FL, 274-277, 2005. https://doi.org/10.1177/154193120504900313
 Gorman, J. C., Cooke, N. J., & Winner, J. L. Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312-1325, 2006 https://doi.org/10.1080/00140130600612788
 Aptima, Inc. Distributed Dynamic Decision-making: The DDD Team simulation. https://github.com/aptima/ddd (2005)
 Crookall, D. Debriefing. Simulation & Gaming, 23(2), 141-142, 1992. https://doi.org/10.1177/1046878192232001
 Lederman, L.C. Debriefing: Toward a systematic assessment of theory and practice. Simulation & Games, 23(2), 145-160, 1992. https://doi.org/10.1177/1046878192232003
 Stewart, L.P. Ethical issues in postexperimental and postexperiential debriefing. Simulation & Gaming, 23(2), 196-211, 1992. https://doi.org/10.1177/1046878192232007
 APA Ethical principles of psychologists and code of conduct. American Psychological Association. APA, Washington D.C, 2010.
 Lederman, L.C. Debriefing: A critical reexamination of the postexperience analytic process with implications for its effective use. Simulation & Games, 15(4), 415-431, 1984. https://doi.org/10.1177/0037550084154002
 Charsky, D. From edutainment to serious games: A change in the use of game characteristics. Games and culture, 5(2), 177-198, 2010. https://doi.org/10.1177/1555412009354727
 Van Ments, M. The effective use of role-play. London: Kogan Page, 1989.
 Steinwachs, B. How to facilitate a debriefing. Simulation & Gaming, 23(2), 186-195, 1992. https://doi.org/10.1177/1046878192232006
 Thiagarajan, S. Using games for debriefing. Simulation & Gaming, 23(2), 161-173, 1992. https://doi.org/10.1177/1046878192232004
 Academy, Serious Games Society. http://academy.seriousgamessociety.org/about. Retrieved March 15, 2017.
How to Cite
IJSG copyright information is provided here.