Experience, Experiment, Evaluate: A Framework for Assessing Experiential Games
The design of effective educational games has proven itself difficult for many years, leading to sparse and somewhat inconsistent insight into the principles governing such systems. While attempts at constructing frameworks for educational games certainly exist, their nature is often quite general (limiting the practical utility) or noticeably specific (limiting the scope of projects to which that framework might be applied). We present a design framework for a broad, but well-defined genre known as experiential games. We have named our framework the Experience, Experiment, Evaluate (EEE) framework and believe it to be an adequate lens under which to analyze such games. This article presents the EEE framework in detail and provides example analyses of three games (a U.S. Civil War history game, a medical diagnosis game, and The Legend of Zelda: Ocarina of Time). We present empirical results for two of these games showcasing evidence that presumed adherence to EEE provides benefits in the classroom. In particular, the medical diagnosis game, Rashi, is shown to elicit higher quantity and quality of student responses when features were added that more tightly bound the game to our framework. Additionally, we provide evidence that activities within our U.S. Civil War game, ‘A Nation Divided’, are more successful in providing learning gains to students when those activities more carefully apply the ideas within our framework. We do not present any empirical results regarding Ocarina of Time, but include it as an anecdotal example of how commercial games have applied these principles successfully in order to teach the mechanics of the game to players, and argue that this is, in many ways, an exercise in pedagogy. We end by offering suggestions for strategically incorporating elements of our framework in the development and design of future systems.
 Winn, B.. "The design, play, and experience framework." Handbook of research on effective electronic gaming in education 3 (2008): 1010-1024.
 Hunicke, R, LeBlanc, M., and Zubek, R.. "MDA: A formal approach to game design and game research." Proceedings of the AAAI Workshop on Challenges in Game AI. Vol. 4. 2004.
 Kelly, H., et al. "How to build serious games." Communications of the ACM 50.7 (2007): 44-49. https://doi.org/10.1145/1272516.1272538
 Westera, W., et al. "Serious games for higher education: a framework for reducing design complexity." Journal of Computer Assisted Learning 24.5 (2008): 420-432. https://doi.org/10.1111/j.1365-2729.2008.00279.x
 Burgos, D., Tattersall, C., and Koper, R. "Re-purposing existing generic games and simulations for e-learning." Computers in Human Behavior 23.6 (2007): 2656-2667. https://doi.org/10.1016/j.chb.2006.08.002
 Dondlinger, M. J. "Educational video game design: A review of the literature." Journal of applied educational technology 4.1 (2007): 21-31.
 Akkerman, S., Admiraal, W., and Huizenga, J. “Storification in History Education: A Mobile Game in and about Medieval Amsterdam.” Computers & Education 52.2 (2009): 449–459. Web. 22 Feb. 2016.
 Carvalho, M. B., et al. "An activity theory-based model for serious games analysis and conceptual design." Computers & Education 87 (2015): 166-181. https://doi.org/10.1016/j.compedu.2015.03.023
 Roungas, B. "A Model-driven Framework for Educational Game Design." International Journal of Serious Games 3.3 (2016): 19-37. https://doi.org/10.17083/ijsg.v3i3.126
 Nicholson, S. “Completing the Experience: Debriefing in Experiential Educational Games.” 11.6 (2013): 27–31. Print.
 Arnab, S., et al. "Mapping learning and game mechanics for serious games analysis." British Journal of Educational Technology 46.2 (2015): 391-411. https://doi.org/10.1111/bjet.12113
 Dragon, T., et al. "Coaching within a domain independent inquiry environment." Intelligent Tutoring Systems. Springer Berlin Heidelberg, 2006. https://doi.org/10.1007/11774303_15
 Murray, T., Woolf, B. and Marshall, D. "Lessons learned from authoring for inquiry learning: A tale of authoring tool evolution." Intelligent Tutoring Systems. Springer Berlin Heidelberg, 2004.
 Floryan, M., and Park Woolf, B. "Improving the efficiency of automatic knowledge generation through games and simulations." Artificial Intelligence in Education. Springer Berlin Heidelberg, 2013. https://doi.org/10.1007/978-3-642-39112-5_36
 Floryan, M. Evolving expert knowledge bases: Applications of crowdsourcing and serious gaming to advance knowledge development for intelligent tutoring systems. Diss. University of Massachusetts Amherst, 2013.
 Baker, R., et al. "Why students engage in “gaming the system” behavior in interactive learning environments." Journal of Interactive Learning Research 19.2 (2008): 185-224.
 Baker, R S., et al. "Off-task behavior in the cognitive tutor classroom: when students game the system." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2004. https://doi.org/10.1145/985692.985741
 Cocea, M., Hershkovitz, A. and Baker R. S. "The impact of off-task and gaming behaviors on learning: immediate or aggregate" (2009): 507-514.
 Lytle N., Floryan. M., “A Design Framework for Experiential History Games.” Games and Learning Alliance Conference. Rome, Italy. December 10, 2015.
 Gee, J. P., Good video games + good learning: Collected essays on video games, learning, and literacy. New York: P. Lang, 2007. https://doi.org/10.3726/978-1-4539-1162-4
IJSG copyright information is provided here.