Serious Games are a Serious Tool for Team Research

Michael D. Coovert, Jennifer Winner, Winston Bennett, David J. Howard

Abstract


Serious games are an attractive tool for education and training, but their utility is even broader. We argue serious games provide a unique opportunity for research as well, particularly in areas where multiple players (groups or teams) are involved. In our paper we provide background in several substantive areas. First, we outline major constructs and challenges found in team research. Secondly, we discuss serious games, providing an overview and description of their role in education, training, and research. Thirdly, we describe necessary characteristics for game engines utilized in team research, followed by a discussion of the value added by utilizing serious games. Our goal in this paper is to argue serious games are an effective tool with demonstrated reliability and validity and should be part of a research program for those engaged in team research. Both team researchers and those involved in serious game development can benefit from a mutual partnership which is research focused.


Keywords


Game engines, serious games, simulations, team research, teams

Full Text:

PDF

References


Townsend, A. M., DeMarie, S. M., & Hendrickson, A. R. Virtual teams: Technology and the workplace of the future. The Academy of Management Executive, 12(3), 17–29, 1998. https://doi.org/10.5465/ame.1998.1109047

Coovert, M. D. Technological changes in office jobs: What we know and what we can expect. In A. Howard (Ed.), The changing nature of work: Frontiers of industrial and organizational psychology (pp. 175-208). San Francisco, CA: Jossey-Bass, 1995.

Coovert, M. D., & Foster Thompson, L. F. Computer-supported cooperative work: Issues and implications for workers, organizations, and human resource management. Thousand Oaks, CA: Sage, 2001.

Coovert, M. D. and Foster Thompson, L. (Eds.) The Psychology of Workplace Technology. New York, NY: Routledge Taylor & Francis Group, 2014.

Pavlova, E., Coovert, M. D., & Bennett, W. Trust development in computer-mediated teams: A latent change score model. Multivariate Behavioral Research, 48, 166-167, 2013. https://doi.org/10.1080/00273171.2013.752265

Cooke, N. J., Kiekel, P. A., Salas, E., Stout, R., Bowers, C., & Cannon-Bowers, J. A. Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory, Research, and Practice, 7 (3), 179-199, 2003. https://doi.org/10.1037/1089-2699.7.3.179

Brannick, M.T., Roach, R.M., & Salas, E. Understanding team performance: A multidimensional study. Human Performance, 6(4), 287-308, 1993. https://doi.org/10.1207/s15327043hup0604_1

Dickenson, T.L., & McIntyre, R.M. A conceptual framework for teamwork measurement. Team performance assessment and measurement (19-43). In M.T. Brannick, E. Salas, C.W. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications. Taylor & Francis, 1997.

Coovert, M. D. Petri nets: Modeling the complexity of modern jobs. In J. M. Cortina and R. S. Landis (Eds.), Modern research methods for the study of behavior in organizations (pp. 381-404). New York, NY: Routledge Taylor & Francis Group, 2013.

Coovert, M. D., Salas, E., & Cannon-Bowers, J. A. Process models of team behavior. Proceedings of the American Control Conference. IEEE Press, 1991.

Yagoda, R. E., & Coovert, M. D. How to work and play with robots: An approach to modeling human-robot interaction. Computers in Human Behavior, 28(1), 60-68, 2012. https://doi.org/10.1016/j.chb.2011.08.011

Marks, M.A., Mathieu, J.E., & Zaccaro, S.J. A temporally based framework and taxonomy of team processes. Academy of Management Review, 26(3), 356-376, 2001.

LePine, J.A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., & Saul, J. R. A meta-analysis of teamwork process: Tests of a multidimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61(2), 273 – 307, 2008. https://doi.org/10.1111/j.1744-6570.2008.00114.x

Cooke, N.J., Salas, E., Cannon-Bowers, J. A., & Stout, R. J. Measuring team knowledge. Human Factors, 42, (1), 151-173, 2000. https://doi.org/10.1518/001872000779656561

Klimoski, R. & Mohammed, S. Team mental model: Construct or metaphor? Journal of Manage,ent, 20, 403-437, 1994.

Langan-Fox, J., Anglim, J. & Wilson, J. R. Mental models, team mental models, and performance: Process, development, and future directions. Human Factors and Ergonomics in Manufacturing, 14(4), 331-352, 2004. https://doi.org/10.1002/hfm.20004

Mohammed, S. & Dumville, B. C. Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22, 89-106, 2001. https://doi.org/10.1002/job.86

Mohammed, S., Klimoski, R. & Rentsch, J. R. The measurement of team mental models: We have no shared schema. Organizational Research Methods, 3(2), 123-165, 2000. https://doi.org/10.1002/job.86

Hess, K. P., Freeman, J., & Coovert, M. D. CENTER: Critical thinking in team decision making. In M. P. Letsky, N. W. Warner, S. M. Fiore, & C.A.P. Smith (Eds.). Macrocognition in teams: Theories and methodologies (pp. 239-258). Burlington, VT: Ashgate Publishing, 2008.

Rosen, M.A., Bedwell, W.L., Wildman, J. L., Fritzsche, B.A., Salas, E, & Burke, C.S., Managing adaptive performance in teams: Guiding principles and behavioral markers for measurement. Human Resource Management Review, 21, 107-122, 2011. https://doi.org/10.1016/j.hrmr.2010.09.003

Burke, C.S., Stagl, K.C., Klein, C., Goodwin, G.F., Salas, E., & Halpin, S.M. What type of leadership behaviors are functional in teams? A meta-analysis. Leadership Quarterly, 17, 288-307, 2006. https://doi.org/10.1016/j.leaqua.2006.02.007

Tuckman, B.W. Developmental sequence in small groups. Psychological Bulletin, 63(6), 384-399, 1965. https://doi.org/10.1037/h0022100

Morgan, Jr. B.B., Glickman, A.S., Woodard, E.A. Blaiwes, A.S., & Salas, E. Measurement of team behaviors is a Navy environment. Battell Columbus Labs, Research Triangle Park, NC, 1986.

Prince, C., & Salas, E. Training and research for teamwork in the military aircrew (337-366). In E. L. Wiener, B. G. Kanki, and R.L. Helmreich (Eds.), Cockpit resource management, Elsiver, 1993.

Fowlkes, J. E., Lane, N.E., Salas, E., Franz, T., & Oser, R., Improving the measurement of team performance: The TARGETs methodology. Military Psychology, 6(1), 47-61, 1994. https://doi.org/10.1207/s15327876mp0601_3

Herz, J. C., & Macedonia, M. R. (2002). Computer games and the military: Two views. Defense Horizons, (11-12), 1.

Shilling, R., Zyda, M., & Wardynski, E. C. (2002, November). Introducing Emotion into Military Simulation and Video Game Design America’s Army: Operations and VIRTE. In GAME-ON.

Moon, I. C., Schneider, M., & Carley, K. M. (2006). Evolution of player skill in the America’s army game. Simulation, 82(11), 703-718. https://doi.org/10.1177/0037549706073689

Farrell, C. M., W. K. Klimack, and C. R. Jacquet. 2003. Employing interactive multimedia instruction in military science education at the U.S. Military Academy. In Proceedings of the Interservice/Industry Training, Simulation, and Education (I/ITSEC) Conference, Orlando, FL.

Alhadeff, E. (2016), Serious Games Market. Retrieved Feb 3, 2017 from: http://seriousgamesmarket.blogspot.com/2016/12/2016-winners-announced-serious-games.html

Crookall, D. Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41(6), 898-920, 2010. https://doi.org/10.1177/1046878110390784

de Freitas, S., & Ketelhut, D.J. Introduction to the Journal of Information Sciences special issue on serious games. Information Sciences, 264, 1-3, 2014. https://doi.org/10.1016/j.ins.2014.01.036

Greco, M., Baldissin, N., & Nonino, F. An exploratory taxonomy of business games. Simulation & Gaming, 44(5), 645-682, 2013. https://doi.org/10.1177/1046878113501464

MONOPOLOGS [Developed by Rand Logistic Department.] Santa Monica: CA. Rand Corporation, 1955.

Abt, C.A. Serious games. Viking, New York, 1970.

Faria, A.J. The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming, 32(1), 97-110, 2001. https://doi.org/10.1177/104687810103200108

Bragge, J., Thavikulwat, P., & Töyli, J. Profiling 40 years of research in Simulation & Gaming. Simulation & Gaming, 41(6), 869-897, 2010. https://doi.org/10.1177/1046878110387539

Feinstein, A. H., & Cannon, H. M. Constructs of simulation evaluation. Simulation & Gaming, 33(4), 425-440, 2002. https://doi.org/10.1177/1046878102238606

Sauvé, L., Renaud, L., Kaufman, D., & Marquis, J.S. Distinguishing between games and simulations: A systematic review. Educational Technology & Society, 10(3), 247-256, 2007.

Pohl, A., & Walker, J. Leveraging game technology: Where are games the right answer to augment military training? Material Science & Technology, 2, 22-26, 2010.

Werbach, K., & Hunter, D. For the win: How thinking can revolutionize your business. Wharton Digital Press: Philidelphia, 2012.

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. From game design elements to gamefullness: Defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. Tampere, Finland, 2011. https://doi.org/10.1145/2181037.2181040

Bailey, P., Pritchard, G., & Koenohan, H. Gamification in market research: Increasing enjoyment and richness of data, but what of data validity? International Journal of Market Research, 57 (1), 17-28, 2015. https://doi.org/10.2501/IJMR-2015-003

Deterding, S. Gamification: Designing for motivation. Interactions, 19(4), 14-17, 2012. https://doi.org/10.1145/2212877.2212883].

Paharia, R. Gamification means applying intrinsic value, 19(4), 14-17, 2012. https://doi.org/1010,1145/2212877.2212883]

Landers, R.N., & Landers, A.K. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simulation & Gaming, 45(6), 769-785, 2015. https://doi.org/10.1177/1046878114563662

Mitgutsch, K., & Alvarado, N. Purposeful by design? A serious game design assessment framework. Proceedings of Foundations of Digital Games (121-128). Association for Computing Machinery, Raleigh: N.C., 2012.

Salen, K., & Zimmerman, E. Rules of play: Game design fundamentals. Cambridge: MIT Press, 2003.

De Gloria, A., Bellotti, F., & Berta, R. (2014). Serious games for education and training. International Journal of Serious Games, 1(1). https://doi.org/10.17083/ijsg.v1i1.11

Connolly, T.M., Boye, E.A., MacArthur, E., Hainey, T., & Boyle, J.M. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661-686, 2012. https://doi.org/10.1016/j.compedu.2012.03.004

Ortiz, S. (2015). Video game self-efficacy and its effect on training performance. International Journal of Serious Games, 2(3). https://doi.org/10.17083/ijsg.v2i3.89

Sitzmann, T. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489-528, 2011. https://doi.org/10.1111/j.1744-6570.2011.01190.x

Bedwell, W.L., Pavlas, D., Heyne, K., Lazzara, E. H., & Salas, E. Toward a taxonomy linking game attributes to learning: An empirical study. Simulation & Gaming, 43(6), 729-760, 2012. https://doi.org/10.1177/1046878112439444

Landers, R.N. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752-768, 2015. https://doi.org/10.1177/1046878114563660

Ghannem, A. Characterization of serious games guided by the educational objectives. Proceedings of TEEM’14 (pp. 227-233). Association for Computing Machinery, 2014. https://doi.org/10.1145/2669711.2669904

Mayer, I., Bekebrede, G., Harteveld, C., Warmelink, H., Zhou, Q., van Ruijven, T., Lo, J., Kortmann, R., & Wenzler, I. The research and evaluation of serious games: Toward a comprehensive methodology. British Journal of Educational Psychology, 45(3), 502-527, 2014. https://doi.org/10.1111/bjet.12067

de Klerk, S., Veldkamp, B.P., & Eggen, T.J.H.M. Psychometric analysis of the performance data of simulation-based assessment: A systematic review and a Bayesian network example. Computers & Education, 85, 23-34, 2015. https://doi.org/10.1016/j.compedu.2014.12.020

Chorianopoulos, K., Giannakos, M.N., & Chrisochoides, N. Design principles for serious games in mathematics. Proceedings of the 18th Panhellenic Conference on Informatics (pp. 1-5). Association for Computing Machines, 2014. https://doi.org/10.1145/2645791.2645843

Margolis, M.J., & Clauser, B.E. A regression-based procedure for automated scoring of a complex medical performance assessment. In. D.M. Williamson, I. I. Bejar, and R.J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123-168). Mahwah, NJ: Lawrence Erlbaum Associates, 2006.

Coovert, M. D., Winner, J., & Bennett, W. Construct development and validation in game-based research. Simulation & Gaming, 47(6),1-13, 2016. https://doi.org/10.1177/1046878116682661

Mäyrä, F., Holopainen, J., Jakobsson, M. Research methodology in gaming: An overview. Simulation & Gaming, 43(3), 295-299, 2012. https://doi.org/10.1177/1046878112439508

de Caluwé, Geurts, J., & Kleinlugtenbelt, W.J. (2012). Gaming research in policy and organization: An assessment from the Netherlands. Simulation & Gaming, 43(5), 600-626. https://doi.org/10.1177/1046878112439445

Eladhari, M.P., & Ollila, E.M.I. Design for research results: Experimental prototyping and play testing. Simulation & Gaming, 43(3), 391-412, 2012. https://doi.org/10.1177/1046878111434255

Protopsaltis, A., & Auneau, L. Scenario-based serious game repurposing. Proceedings of the 29th ACM International Conference of Design of Communications (37-43). Association for Computing Machinery, 2011. https://doi.org/10.1145/2038476.2038484

Campbell, D.T., & Fiske, D.W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105, 1959. https://doi.org/10.1037/h0046016

Widaman, K.F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9(1), 1-26, 1985. https://doi.org/10.1177/014662168500900101

Jöreskog, K. G. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183-202, 1969. https://doi.org/10.1007/BF02289343

Marsh, H.W., Balla, J.R., & McDonald, R.P. Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103(3), 391-410, 1988. https://doi.org/10.1037/0033-2909.103.3.391

Lewis, M., & Jacobson, J. Game engines in scientific research. Communications of the ACM, 45(1), 27-31, 2002.

Hendrix, M., Al-Sherbaz, A., & Victoria, B. (2016). Game based cyber security training: are serious games suitable for cyber security training?. International Journal of Serious Games, 3(1). https://doi.org/10.17083/ijsg.v3i1.107

Gorman, J. C., Cooke, N. J., Pederson, H. K., Connor, O. O., & DeJoode, J. A. Coordinated awareness of situation by teams (CAST): Measuring team situation awareness of a communication glitch. Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting,Orlando, FL, 274-277, 2005. https://doi.org/10.1177/154193120504900313

Gorman, J. C., Cooke, N. J., & Winner, J. L. Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312-1325, 2006 https://doi.org/10.1080/00140130600612788

Aptima, Inc. Distributed Dynamic Decision-making: The DDD Team simulation. https://github.com/aptima/ddd (2005)

Crookall, D. Debriefing. Simulation & Gaming, 23(2), 141-142, 1992. https://doi.org/10.1177/1046878192232001

Lederman, L.C. Debriefing: Toward a systematic assessment of theory and practice. Simulation & Games, 23(2), 145-160, 1992. https://doi.org/10.1177/1046878192232003

Stewart, L.P. Ethical issues in postexperimental and postexperiential debriefing. Simulation & Gaming, 23(2), 196-211, 1992. https://doi.org/10.1177/1046878192232007

APA Ethical principles of psychologists and code of conduct. American Psychological Association. APA, Washington D.C, 2010.

Lederman, L.C. Debriefing: A critical reexamination of the postexperience analytic process with implications for its effective use. Simulation & Games, 15(4), 415-431, 1984. https://doi.org/10.1177/0037550084154002

Charsky, D. From edutainment to serious games: A change in the use of game characteristics. Games and culture, 5(2), 177-198, 2010. https://doi.org/10.1177/1555412009354727

Van Ments, M. The effective use of role-play. London: Kogan Page, 1989.

Steinwachs, B. How to facilitate a debriefing. Simulation & Gaming, 23(2), 186-195, 1992. https://doi.org/10.1177/1046878192232006

Thiagarajan, S. Using games for debriefing. Simulation & Gaming, 23(2), 161-173, 1992. https://doi.org/10.1177/1046878192232004

Academy, Serious Games Society. http://academy.seriousgamessociety.org/about. Retrieved March 15, 2017.




DOI: http://dx.doi.org/10.17083/ijsg.v4i1.141

Refbacks

  • There are currently no refbacks.





Serious Games Society


Creative Commons LicenseThe International Journal of Serious Games (IJSG) by Serious Games Society is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.