756. Do you forgive your algorithm? The impact of self-confidence, perceived accuracy, and trust on the use of decision support advice
Invited abstract in session MD-28: Human-AI Collaboration and Ethics, stream Decision Support Systems.
Monday, 14:30-16:00Room: Maurice Keyworth 1.03
Authors (first author is the speaker)
| 1. | Maren Suffel
|
| Department of Business Decisions and Analytics, University of Vienna | |
| 2. | Ayşegül Engin
|
| Business Aministration, University of Vienna | |
| 3. | Rudolf Vetschera
|
| Dept. of Business Decisions and Analytics, University of Vienna |
Abstract
Artificial intelligence (AI) is transforming decision support by enabling faster, more accurate decision-making in tasks such as forecasting, scheduling, and hiring. However, many AI systems lack transparency due to their “black-box” nature, raising concerns about users’ ability to understand and trust their outputs. This can impact decision quality, as users must judge when to rely on AI or override its recommendations. Effective human-AI collaboration requires careful judgment, but this process is often influenced by cognitive biases. Overconfidence in one's abilities or uncritical acceptance of AI (overreliance) can undermine collaboration, while rejecting correct AI outputs (underreliance) limits potential benefits. These behaviors are especially relevant in high-stakes environments like air traffic control, where AI-based decision support can reduce delays and fuel consumption. Nonetheless, controllers' expertise remains essential, requiring effective joint human-AI decision-making. This study examines how self-confidence, perceived system accuracy, and trust evolve as participants interact with system recommendations in a simplified air traffic control task. We focus on how users respond to system errors, including how they adapt reliance and recover trust. Initial observations suggest that unexpected AI accuracy drops or shifts in task structure trigger different biases, with some users becoming overly cautious, while others persist in following incorrect advice.
Keywords
- Decision Support Systems
- Artificial Intelligence
Status: accepted
Back to the list of papers