A Framework for Research in Gamified Mobile Guide Applications using Embodied Conversational Agents (ECAs)


  • Ioannis Doumanis University of Middlesex
  • Serengul Smith University of middlesex




Gamification, Mobile Guide Applications (MGA), Embodied Conversational Agents (ECAs), Player experience, Cognitive Accessibility, Usability


Mobile Guides are mobile applications that provide players with local and location-based services (LBS), such as navigation assistance, where and when they need them most. Advances in mobile technologies in recent years have enabled the gamification of these applications, opening up new opportunities to transfer education and culture through game play. However, adding traditional game elements such as PBLs (points, badges, and leaderboards) alone cannot ensure that the intended learning outcomes will be met, as the player’s cognitive resources are shared between the application and the surrounding environment. This distribution of resources prevents players from easily immersing themselves into the educational scenario. Adding artificial conversational characters (ECAs) that simulate the social norms found in real-life human-to-human guide scenarios has the potential to address this problem and improve the player’s experience and learning of cultural narratives [1]. Although significant progress has been made towards creating game-like mobile guides with ECAs ([2], [3]), there is still a lack of a unified framework that enables researchers and practitioners to investigate the potential effects of such applications to players and how to approach the concepts of player experience, cognitive accessibility and usability in this context. This paper presents a theoretically-well supported research framework consisted of four key components: differences in players, different features of the gamified task, aspects of how the ECA looks, sound or behaves and different mobile environments. Furthermore, it provides based on this framework a working definition of what player experience, cognitive accessibility and usability are in the context of game-like mobile guide applications. Finally, a synthesis of the results of six empirical studies conducted within this research framework is discussed and a series of design guidelines for the effective gamification of mobile guide applications using ECAs are presented. Results show that an ECA can positively affect the quality of the player’s experience, but it did not elicit better player retention of cultural narratives and navigation of routes.


[1] Doumanis, I. (2013): Evaluating Humanoid Embodied Conversational Agents in Mobile Guide Applications. Thesis (PhD). Middlesex University, London UK.
[2] Krüger, A., Baus, J., Heckmann, D., Kruppa, M., (2007): Adaptive Mobile Guides. The Adaptive Web, P. Brusilovsky, A. Kobsa & W. Nejdl (Eds.), Springer, 2007, pp. 521 – 549. http://dx.doi.org/10.1007/978-3-540-72079-9_17
[3] Wagner, D. and D. Schmalstieg (2006): Handheld Augmented Reality Displays. In Proceedings of the 2006 IEEE conference on Virtual Reality (VR2006) Alexandria, Virginia, USA, March 25 – 29, 2006, pp. 321 http://dx.doi.org/10.1109/vr.2006.67
[4] Apple Inc. (2015): iPad Tablet Device. Available at: http://www.apple.com/ipad/ [Accessed March 11 2015].
[5] Apple Inc. (2015): Siri Mobile Application. Available at: http://www.apple.com/ios/siri/ [Accessed March 11 2015].
[6] Waze Mobile (2015): Waze Mobile application. Available at: https://www.waze.com/.
[7] Mobee Inc. (2015): Mobee mobile application. Available at: http://www.mobeeapp.com/ [Accessed March 11 2015].
[8] Falke, C. (2014). Ghosts Games Wartburg. Available at: http://goo.gl/vd3fQ3 [Accessed March 11 2013].
[9] Next OS (2015): Virtual Assistance Denise. Available at: https://guile3d.com/en/ [Accessed March 11 2015].
[10] Dow, S., Mehta, M., Lausier, A., MacIntyre, B., Mateas, M., (2006): Initial Lessons from AR Façade, An Interactive Augmented Reality Drama. Proceedings of the International Conference on Advances in Computer Entertainment Technology, ACE 2006, Hollywood, California, USA, June 14-16, 2006. http://dx.doi.org/10.1145/1178823.1178858
[11] Miyake, S., Ito, A., (2012): A Spoken Dialogue System Using Virtual Conversational Agent with Augmented Reality. Proceedings of Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012 Asia-Pacific December 3-6, 2012, pp. 1-4.
[12] Cassell, J., & Stone, M. (1999): Living hand to mouth: Psychological theories about speech and gesture in interactive dialogue systems. Proceedings of the AAAI Fall symposium '99, pp. 34-42.
[13] Reeves, B., & Nass, C. (1996): The media equation: How people treat computers, television, and new media like real people and places. Cambridge University Press
[14] Lee, E.-J. & Nass, C. (1998): Does the ethnicity of a computer agent matter? An experimental comparison of human-computer interaction and computer-mediated communication. Proceedings of the Workshop on Embedded Conversational Characters Conference. Lake Tahoe, CA, pp. 123-128
[15] Rickenberg, R., Reeves, B. (2000): The effects of Animated Characters on Anxiety, Task Performance, and Evaluations of User Interfaces. Proceedings of CHI 2000, The Hague, The Netherlands, Letters, 2(1), pp.49-56. http://dx.doi.org/10.1145/332040.332406
[16] Xiao, J. (2006): Empirical Studies on Embodied Conversational Agents. Thesis (PhD). Georgia Institute of Technology.
[17] GamEffective (2015): GamEffective Platform. Available at: http://www.gameffective.com/ products/overview/
[18] Mayer, R., & Moreno R. (2002): Animation as an Aid to Multimedia Learning. Educational Psychology Review 14-1, pp. 87-99. http://dx.doi.org/10.1023/A:1013184611077
[19] Moreno, R., Mayer, R. (2000): Engaging Students in Active Learning: The Case for Personalized Multimedia Messages. Journal of Educational Psychology Vol 92(4), pp. 724-733. http://dx.doi.org/10.1037/0022-0663.92.4.724
[20] Baylor, A. L. and S. Kim (2008): The Effects of Agent Nonverbal Communication on Procedural and Attitudinal Learning Outcomes. International Conference on Intelligent Virtual Agents. Tokyo, Springer: pp. 208-214. http://dx.doi.org/10.1007/978-3-540-85483-8_21
[21] Campos, H., Campos, J., Martinho, C., Paiva A. (2012): Virtual Agents in Conflict, 12th International Conference on Intelligent Virtual Agents (IVA 2012), September 20-22, 2012, Santa Cruz, CA, USA pp. 105-111. http://dx.doi.org/10.1007/978-3-642-33197-8_11. http://dx.doi.org/10.1007/978-3-642-33197-8_11
[22] Korre, D., (2015): Usability Engineering: Embodied Conversational Agents in Serious Games. Available at: http://goo.gl/TR72Kj
[23] Adams, R. and Langdon, P. (2003): SIMPLEX: a simple user check-model for Inclusive Design. Universal Access in HCI: Inclusive Design in the Information Society. Stephanidis, C. (Ed.). 4, pp. 13-17. Mahwah: NJ: Lawrence Erlbaum Associates.
[24] Adams, R. (2006): Decision and stress: cognition and e-accessibility in the information workplace. Universal Access in the Information Society, 5 (4). pp. 363-379. ISSN 1615-5297
[25] Sternberg, R. J. (Ed.). (2000). Handbook of intelligence. New York, USA: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511807947
[26] Chou, Y. (2015): Octalysis: Complete Gamification Framework. [Blog] Yu-Kai Chou & Gamification. Available at: http://goo.gl/Qa1Vm2 [Accessed 29 March 2015].
[27] Adams, R. (2005): (R. G. Adams@ mdx.ac.uk) (25 June 2005). RE: Accessibility Evaluation based on Simplex. Personal email to I. Doumanis (idoumanis@mdx. ac.uk)
[28] ISO 9241-11 (1998). Ergonomics requirements for office work with visual display terminals (VDTs) – Part 11: Guidance on usability.
[29] Hollan, J., Hutchins, E., & Kirsh, D. (2000): Distributed Cognition: Toward a New Foundation for Human Computer Interaction Research. ACM Trans. on Computer-Human Interaction. 7 (2), pp. 174-196. http://dx.doi.org/10.1145/353485.353487
[30] Kaptelinin, Victor and Bonnie Nardi (1997): Activity Theory: Basic Concepts and Applications. Proceedings of CHI ’97, pp. 74–77. http://dx.doi.org/10.1145/1120212.1120321
[31] Craig, P., Roa-Seiler, N., Diaz, M. D, Rosano, L. R., (2013): The Role of Embodied Conversational Agents in Collaborative face to face Computer Supported Learning Games, Proceedings of the 25th International Conference on System Research, Informatics & Cybernetics, Baden-Baden, Germany, July 29 –August 2, 2013.
[32] Katz, J. (2000). Up, up, down, down. Slashdot.org. Originally published November, 30, 2000. (http://slashdot.org/features/00/11/27/1648231.shtml)
[33] Prensky, M. (2000). Digital Game-Based Learning. New York: McGraw Hill.
[34] Raessens, J.F.F. 2012: Homo Ludens 2.0. The Ludic Turn in Media Theory” Available at: http://goo.gl/38nPtz [Accessed March 11 2013]
[35] Cowell, A. J., Stanney, K. M. (2002): User demographics for embodiment customization. AAAI Fall Symposium on Personalized Agent. Cambridge, MA: American Association for Artificial Intelligence. November 15-17, 2002, pp. n1-n2.
[36] Chou, Y. (2015): Gamification Design: 4 Phases of a Player’s Journey [Blog] Yu-Kai Chou & Gamification. Available at: http://goo.gl/UYxuNN [Accessed 29 March 2015]
[37] Catrambone, R., Stasko, J., & Xiao, J. (2002): Anthropomorphic agents as a user interface paradigm: Experimental findings and a framework for research. Proceedings of the 24th Annual Conference of the Cognitive Science Society, Fairfax, VA, August 2002, pp. 166-171.
[38] Phan M. H., Jardina J. R., & Hoyle W. S. (2012): Video Games: Males prefer violence while Females prefer social. Available at: http://goo.gl/z4rbvY [Accessed March 11 2013]
[39] Bartle, R. (1996). Hearts, clubs, diamonds, spades: players who suit MUDS. Available at: http://goo.gl/mp8oct [Accessed March 11 2015
[40] Gonzalez, Jr., P.A (2014): Race and Ethnicity in Video Games: A Reflection of Social Reality. Racism, hate speech and prejudice: a manifestation of social stereotypes. Available at: http://goo.gl/5EKi2q [Accessed March 11 2015].
[41] Gans, H. J. (1999). Popular culture and high culture: an analysis and evaluation of taste. New York: Basic Books.
[42] Levine, L. (1988) Highbrow/Lowbrow: the emergence of cultural hierarchy in America. Cambridge: Harvard University Press.
[43] Schilling, C. (2009): Art house video games. The Daily Telegraph (London). Available at: http://goo.gl/QUjUIV
[44] Griffiths, S. (2013): 'Granddad' Theft Auto: Middle-class, middle-aged parents are most likely to play violent crime game. Available at: http://goo.gl/BWpOC7
[45] Octalysis 2013. Enterprise Gamification Consultancy wiki [Online]. Available at: http://goo.gl/3khrVy [Accessed March 11 2015].
[46] Game Mechanics 2010. Gamification Wiki. [Online]. Available at http://goo.gl/ yTHT81
[47] Likert, R. (1932): A Technique for the Measurement of Attitudes. In: Archives of Psychology 140, p.55.
[48] Doumanis I., and Smith S (2014): Evaluating the impact of Embodied Conversational Agents attentional behaviours on user retention of cultural content in a simulated mobile environment. Proceedings of 16th ACM International Conference on Multimodal Interaction (ICMI 2014), November 16th, 2014, Istanbul, Turkey.
[49] Doumanis I., Serengul, S., “Evaluating an Intelligent Q&A system for Mobile Cultural Learning” In Journal of EAI Endorsed Transactions on Future Intelligent Educational Environments, Vol.1, Issue 2, June 2015, pp. 1-11.
[50] Doumanis, I., Serengul, S., (2013): An empirical study on the effects of Embodied Conversational Agents (ECAs) on user retention performance and perception in a simulated mobile environment. Proceedings of the 9th International Conference Intelligent Environments (IE’13) 16-17 July 2013, Athens Greece, pp. 431-442.
[51] Doumanis, I., Serengul, S., (2013): Evaluating the Impact of ECAs on User navigation performance and perception. Proceedings of the 13th International Conference on Intelligent Virtual Agents (IVA 2013) 29-31 August 2013, Edinburgh, UK, pp. 474-475.
[52] Virtual People Factory (VPF) (2013): Available at: http://goo.gl/MKA1W3 [Accessed March 11 2013].
[53] MacDorman, K. F., & Ishiguro, H. (2006): The uncanny advantage of using androids in social and cognitive science research. Interaction Studies, Volume 7, Issue 3(2006), pp. 297–337. http://dx.doi.org/10.1075/is.7.3.03mac
[54] Hile, H., Vedantham, R., Liu, A., Gelfand, N., Cuellar, G., Grzeszczuk, R., Borriello, G. (2008): “Landmark-Based Pedestrian Navigation from Collections of Geotagged Photos”. In: Proceedings of ACM International Conference on Mobile and Ubiquitous Multimedia (MUM 2008), December 3 – 5 2008, Umea, Sweden, pp.145-152. http://dx.doi.org/10.1145/1543137.1543167
[55] Microsoft Inc. (2015): Kinect for Windows. Available at: https://www.microsoft.com/en-us/kinectforwindows/ [Accessed March 11 2015].
[56] Chou, Y. (2015): Game Design Analysis of Diablo II through Level II Octalysis [Blog] Yu-Kai Chou & Gamification. Available at: http://goo.gl/5dlMP7 [Accessed 29 March 2015].


Additional Files



How to Cite

Doumanis, I., & Smith, S. (2015). A Framework for Research in Gamified Mobile Guide Applications using Embodied Conversational Agents (ECAs). International Journal of Serious Games, 2(3). https://doi.org/10.17083/ijsg.v2i3.79