A Framework Quantifying Trustworthiness of Supervised Machine and Deep Learning Models

Trusting Artificial Intelligence (AI) is controversial since models and predictions might not be fair, understandable by humans, robust against adversaries, or trained appropriately. Existing toolkits help data scientists to create fair, explainable, robust, and transparent Machine and Deep Learning (ML/DL) models. However, tools to quantify AI trustworthiness according to pillars and metrics relevant for heterogeneous scenarios are still missing. This work proposes a novel algorithm that quantifies the trustworthiness level of supervised ML/DL models according to their fairness, explainability, robustness, and accountability. The algorithm is deployed on a Web application to allow the general public to calculate the trustworthiness of their models. Finally, a validation scenario with models classifying cyberattacks demonstrates the applicability of the Web application and algorithm.

Recommended citation: Huertas Celdrán, Alberto, Kreischer, Jan, Demirci, Melike, Leupp, Joel, Sánchez Sánchez, Pedro Miguel, et al. (2023). "A Framework Quantifying Trustworthiness of Supervised Machine and Deep Learning Models." In SafeAI 2023: The AAAI Workshop on Artificial Intelligence Safety, CEUR Workshop Proceedings, 3381.
Download Paper