EURO 2024 Copenhagen
Abstract Submission

EURO-Online login

4238. Unboxing Tree ensembles for interpretability: A hierarchical visualization tool and a multivariate optimal re-built tree

Invited abstract in session TD-27: Feature attribution and selection for XAI, stream Mathematical Optimization for XAI.

Tuesday, 14:30-16:00
Room: 047 (building: 208)

Authors (first author is the speaker)

1. Giulia Di Teodoro
Department of Information Engineering, University of Pisa
2. Marta Monaci
Institute for System Analysis and Computer Science "Antonio Ruberti" (IASI), National Research Council of Italy
3. Laura Palagi
Department of Computer, Control, and Management Engineering A. Ruberti, Sapienza University of Rome

Abstract

The interpretability of models has become a crucial issue in Machine Learning because of algorithmic decisions' growing impact on real-world applications. Tree ensemble methods, such as Random Forests or XgBoost, are powerful learning tools for classification tasks. However, while combining multiple trees may provide higher prediction quality than a single one, it sacrifices the interpretability property resulting in “black-box” models. In light of this, we aim to develop an interpretable representation of a tree-ensemble model that can provide valuable insights into its behavior. First, given a target tree-ensemble model, we develop a hierarchical visualization tool based on a heatmap representation of the forest's feature use, considering the frequency of a feature and the level at which it is selected as an indicator of importance. Next, we propose a mixed-integer linear programming (MILP) formulation for constructing a single optimal multivariate tree that accurately mimics the target model predictions. The goal is to provide an interpretable surrogate model based on oblique hyperplane splits, which uses only the most relevant features according to the defined forest's importance indicators. The MILP model includes a penalty on feature selection based on their frequency in the forest to further induce sparsity of the splits. The natural formulation has been strengthened to improve the computational performance of mixed-integer software.

Keywords

Status: accepted

Back to the list of papers