EURO 2024 Copenhagen
Abstract Submission

EURO-Online login

1269. The optimal input-independent baseline for binary classification

Invited abstract in session MB-4: Recent Methodologies in Explainable AI (XAI) 2, stream Recent Advancements in AI .

Monday, 10:30-12:00
Room: 1001 (building: 202)

Authors (first author is the speaker)

1. Sandjai Bhulai
Department of Mathematics, Vrije Universiteit Amsterdam
2. Joris Pries
CWI
3. Etienne van de Bijl
CWI
4. Jan Klein
CWI
5. Rob van der Mei
CWI

Abstract

Before any binary classification model is taken into practice, it is important to validate its performance on a proper test set. Without a frame of reference given by a baseline method, it is impossible to determine if a score is “good” or “bad.” The goal of this paper is to examine all baseline methods that are independent of feature values and determine which model is the “best” and why. By identifying which baseline models are optimal, a crucial selection decision in the evaluation process is simplified. We prove that the recently proposed Dutch Draw baseline is the best input-independent classifier (independent of feature values) for all order-invariant measures (independent of sequence order) assuming that the samples are randomly shuffled. This means that the Dutch Draw baseline is the optimal baseline under these intuitive requirements and should therefore be used in practice.

Keywords

Status: accepted

Back to the list of papers