BORBS XXIV: Black Box or Glass Box? Overconfidence and the hidden cost of decision support interfaces
Authors: Ayşegül Engin, Department of Business Decisions and Analytics, University of Vienna
Abstract: Digital transformation has expanded the business applications of decision support and recommender systems. However, optimizing user-system interaction remains challenging due to cognitive biases — such as hindsight bias and overconfidence — that are not easily addressed through design modifications alone. This study explores how overconfidence, driven by hindsight bias, influences the effectiveness of personalized recommendation and decision support systems. Adopting an experimental approach and employing an Experience-Weighted Attraction (EWA) reinforcement learning model, I investigate the impact of these systems on repeated decision-making under uncertainty, where outcomes can only be improved through experience. Three conditions are compared: no decision support, a black-box recommender system providing only a recommendation, and an explicit decision support system providing information-summarizing support without an explicit recommendation. Findings reveal that users benefit from different DSS interfaces in markedly different ways depending on their cognitive biases. Notably, even when the underlying decision rules are held constant across interfaces, the form in which support is delivered leads to substantial differences in decision quality and learning behavior. The results contribute to the behavioral OR literature by demonstrating that interface design — not only algorithmic sophistication — is central to whether users realize the promised benefits of AI-augmented decision support, and by introducing a decision-training perspective that goes beyond single-shot outcomes.
Why should you join?
A counter-intuitive finding for an AI-investment era. Organizations are pouring substantial sums into AI-augmented decision support — with finance firms commonly investing
more than US$10 million, and high-profile failures such as the IBM Watson Healthcare project reaching the multi-billion-dollar range. This talk shows why the standard assumption that “all users benefit equally” can quietly undermine these investments, and what the moderating role of overconfidence looks like empirically.
A clean experimental test of an under-examined design question. Most comparisons of recommender systems and explicit decision support confound interface differences with
algorithmic differences. By holding the underlying decision rule constant across conditions, this study isolates what the interface alone does to decision quality — a methodological choice that makes the behavioral mechanism interpretable rather than buried in algorithmic noise.
A reinforcement-learning lens on user behavior. The talk introduces an Experience-Weighted Attraction (EWA) model adapted to the DSS context, allowing us to estimate not only whether users make better choices, but how they learn — separating learning rate, decision consistency, and experience-weight decay. This computational layer offers a richer behavioral diagnostic than outcome-based comparisons alone.
From single-shot outcomes to a decision-training perspective. The talk reframes DSS evaluation away from “did the user pick correctly this time?” and toward “is the user becoming a better decision-maker over repeated interactions?” — a perspective that opens up new design and managerial questions for behavioral OR and information systems researchers alike.
June 11th, 2026
12 PM to 12.40 PM (UK, London)
1 PM; to 1.40 PM (CET, Berlin)
https://us02web.zoom.us/j/89143663283?pwd=kxdJrqXHfZ0O2nJVPL7TgfCBfuJtRq.1
Meeting-ID: 891 4366 3283
Kenncode: 1
Comment: In case of technical problems, please visit https://www.euro-online.org/websites/bor/behavioral-operation-research-brown-bag-seminar-series/ before the start of the meeting.