Maybe Explainable AI Is a Spectrum

Artificial Intelligence’s (AI) growing influence raises urgent questions about fairness, trustworthiness, transparency, and broader ethical goals. Explainable AI (XAI) tries to address these problems through human-interpretable justifications for the decisions these frequently ‘black-boxed’ AI systems make. XAI’s plausibility as an ethical goal hinges on the balance it achieves between its two evaluation metrics: practical feasibility and ethical importance. Specifically, XAI can only be regarded as a plausible ethical goal if it remains ethically crucial enough to address the core principles of transparency and fairness while bound to the technical and resource constraints of the real world. This paper argues that XAI is a plausible ethical goal because it upholds several essential principles including fairness, transparency, and accountability while supporting a nuanced, context-sensitive approach to account for the technical realities of implementing this explainable framework that counterbalances aforementioned ethical imperatives.

• The ethical case for explainability
• Fairness, accountability, and the failures of opacity
• The technical constraints
• A context-sensitive approach
• In conclusion

I first expound on the ethical importance of XAI, focusing on the aforementioned ethical principles. I then put forward the technical limitations accompanying XAI, followed by a discussion of a context-sensitive approach to XAI, finally ending with addressing long-term ethical imperatives while responding to intermediary counter-arguments along with each sub-topic to support the indispensability of XAI despite its challenges.

The ethical case for explainability

Explainable AI is an ethical necessity across all applications (to ensure fairness, accountability, and transparency), but its importance becomes critical for high-stakes domains where AI decisions can significantly impact the rights of individuals, public trust, or even societal structures. In domains like healthcare, finance, and criminal sentencing, every decision can have serious implications. Without explainability, decisions risk being unchallenged, eroding trust in the system and perpetuating harm. Enabling explainability enables all users of the system — from developers to regulators — to better scrutinize and improve AI systems to make sure they are up to code with ethical standards. From the lens of transparency, stakeholders and anyone impacted have the ability to access and interpret the logic, methodology, and the rationale behind decisions taken to ensure that any decision is understandable and can always be contested. This requirement is further warranted by Seth Lazar’s argument that “automated decision making intensifies existing power structures”.¹ For example, transparency in facial recognition systems revealed disproportionate effect on marginalized groups. These power imbalances would not be intervened upon without transparency. More centrally, opaqueness in these systems is a threat to individual autonomy. The impacted parties can only instill trust in a system through rightful influence on decisions taken in the system. Transparency provides the “informed self-advocacy”² needed to ensure fairness and maintain public support. For example, transparent hiring algorithms allow applicants to challenge discriminatory evaluations of resumes, something that instills reliance and certainty in the system. Another example of this logic is in how GDPR-compliant³ loan-denial systems ensure equitable behavior by providing decision criteria, allowing applicants to contest decisions too. A common counter-argument to this position is that ‘gaming the system’ becomes ever easier and that transparency undermines proprietary systems by exposing sensitive algorithmic information. There are however many barricades and safety nets to prevent this from happening. Ethical frameworks like the GDPR balance transparency with protecting proprietary information by requiring sufficient explanation without full disclosure of intellectual properties, treating transparency as a spectrum instead of an analog decision. Further, processes can always be set up to provide explainability selectively to relevant authorities while maintaining confidentiality from the general public.

Fairness, accountability, and the failures of opacity

Just like transparency, fairness also advocates for the indispensability of explainability in AI. We have multiple examples of opaque systems perpetuating systemic biases instead of equitable treatment. ProPublica’s analysis⁴ showed how the COMPAS tool disproportionately labeled Black defendants as high risk. Similarly, Buolamwini and Gebru’s Gender Shades study revealed higher error rates for dark-skinned women in facial recognition systems. This lack of explainability obscuring systemic inequities is not the only reason to warrant explainability. The Like Cases Maxim asserts that individuals with morally equivalent characteristics should be treated equally. Furthermore, Zimmerman and Stronach argue that avoiding doxastic negligence — making decisions based on incomplete or biased data — demands the need for transparency in AI systems. Lastly, accountability also creates a strong case for explainability. Accountability ensures that decisions can be traced back to their sources, assigning responsibility for harmful outcomes. AI systems need to be held at the same standard as their human counterparts. For instance, in healthcare, AI-assisted diagnoses should be held at the same level as doctors; explainability allows for patient safety and reduced chances of flawed outputs. Accountability through explainability allows for the “articulation and justification of decisions,”¹ empowering stakeholders to hold systems accountable. It additionally also ensures regulatory backing, like through GDPR Article 30 which requires justification for all data processing activities. A common rebuttal to this line of thought is justifying decisions through outcomes-based evaluations, but I argue such alternatives are insufficient. Explainability allows for traceability and a deeper scrutiny of logic and processes, not post-hoc confirmation of righteousness. For instance, in healthcare, understanding why an AI system makes a diagnosis is much more important than just simply noting the error. This applies for all high-stakes contexts.

The technical constraints

While I present multiple rationales on the imperativeness of explainable AI, its implementation is constrained by substantial technical challenges including complexity, trade-offs with accuracy, and resource demands too. Complex models, like deep neural networks, are inherently opaque, which makes their decision-making processes very difficult to interpret, and achieving explainability often involves simplifications that could obscure critical contexts or nuances. There do exist simpler, more explainable models like decision trees, but these models often lack the precision that is needed for a complex action. Taylor and Vredenburgh however underscore that the complexity that explainability brings in the former cases still does not outweigh the moral importance of transparency, foundational to accountability and trust. Especially in high-stakes scenarios like healthcare and criminal justice, I do not believe that marginal accuracy gains trump the ethical importance of explainability. Another challenge in practical feasibility lies in the cost and resources needed to implement XAI. Explainable systems not only require significant computational power but also human supervision. While these requisites might be easily fulfilled in high-resource domains like finance and healthcare, underfunded and low-resource settings risk perpetuating structural inequities through opaque and inscrutable systems. Finally, explainability often involves oversimplification, which is a significant hurdle. Simplified explanations often omit very crucial details or nuances about the decisions made, leading to misinterpretations, misuses, and worse: a false sense of transparency. While critics argue that the technical bottlenecks make XAI impractical, I believe the recent advances from continued research have addressed these challenges significantly. Methods like SHAP and LIME enable scalable and meaningful explainability.

A context-sensitive approach

With the ethical imperative and the practical feasibility of XAI acting like counterbalances, I believe a nuanced, context-sensitive approach is ideal — ensuring that ethical imperatives are prioritized in high-stakes domains while practical limitations help guide the scope and depth of XAI elsewhere.

Domains like healthcare, criminal justice, and finance demand the highest level and standard of explainability due to their profound impact on individual rights and on the structures of society. In this context, ethical principles like accountability and fairness, achieved through XAI, must be at the forefront, and minor trade-offs to accuracy and efficiency are acceptable. For example, we would much rather have an AI system used for cancer diagnosis provide rigorous explanations to build patient safety and trust in the healthcare system than get only the diagnosis back just a little faster. On the other side, low-stakes applications like recommendation systems or algorithms used in social media may be controlled by less rigorous standards for explainability. In low-stakes scenarios, I believe scalability and usability should be prioritized, given they do not compromise trust. For example, a recommendation system for movies might only need to reveal basic insights instead of a detailed rationale into its logic, overwhelming the users. Finally, I do want to explicitly state that explainability should not be treated as a binary requirement but as a spectrum, with the depth and detail of explanations tailored to the specific use case being considered. Just like GDPR exemplifies a tiered transparency model, allowing selective disclosure, high-stakes scenarios can similarly necessitate full transparency while low-stakes scenarios can operate with partial transparency because the harmful outlook is minimal. One may argue that such a varying approach might bring confusion, but clear guidelines and frameworks (like the tiered model in GDPR) can provide consistency while being adaptable to a variety of use cases.

In conclusion

I want to establish XAI as a cornerstone of ethical artificial intelligence, ensuring fairness, transparency, and accountability across systems that have a deep influence on our lives. I reiterate my argument that XAI is a plausible ethical goal because it upholds these baser, foundational ethical principles, even as it navigates its technical limitations in reality. While current challenges in the implementation of XAI exist, these challenges are surmountable with ongoing advances in research and innovation. A context-sensitive framework offers a unique pragmatic solution, balancing the ethical rigor needed with AI systems along with the challenges that accompany it, based on the potential for impact an AI system holds — treating this trade-off as a spectrum instead of a binary decision.