Risk Game: Capturing impact of information quality on human belief assessment and decision making

Abstract

This paper presents the Risk Game, a general methodology to elicit experts’ knowledge and know-how, in their ability to deal with information provided by different types of sources (sensors or humans) of variable quality, to take into account the information quality and to reason about concurrent events. It is a contrived technique capturing data expressing human reasoning features during a specific task of situation assessment. The information is abstracted by cards and its quality, which varies along the three dimensions of uncertainty, imprecision and falseness, is randomly selected by dice roll. The game has been played by experts of maritime surveillance, mostly marine officers from several nations. The Risk Game is domain-independent and can be designed for any specific application involving reasoning with multi-sources. The preliminary results obtained are promising and allow validating the efficiency of the elicitation method in capturing the link between information quality and human belief assessment. Besides the positive feedback collected from the players and their perceived effectiveness of the method, the data effectively capture the impact some specific information quality dimensions on belief assessment. We highlight, for instance, that the relevance of information perceived by the players may differ from the effective information relevance, that a high ratio of false information increases the uncertainty of the player before decision and may lead to wrong decisions, or that the context has a high impact on the decision made. Future extensions of the Risk Game are finally sketched.