There’s a Russian proverb that cuts straight to the heart of human nature: Having an ugly face, don’t blame the mirror (На зеркало неча пенять, коли рожа крива).

We like to blame LLM models for their lack of transparency. Calls for Explainable AI get louder every single day. Politicians demand it, regulators require it, and businesses claim it’s essential for winning consumer trust. And yet, when an AI tool provides us with a transparent explanation of its reasoning, we tend to ignore this explanation, especially if it makes us uncomfortable or, worse, if it threatens to undermine our financial gains.

The Loan Officer Experiment: Seeking Predictions, Avoiding Truth

New research from Harvard Business School highlights this unsettling truth in two experiments.

The first experiment placed participants in the role of loan officers at a private U.S. lender. Their task was straightforward: allocating a real, interest-free $10,000 loan between two unemployed borrowers. An AI system had classified one borrower as low risk and the other as high risk. Participants could access the AI’s risk predictions and also choose whether to view an explanation of how the model reached its assessment.

The results were striking. Roughly 80% of participants eagerly accepted the risk scores: they wanted the AI’s predictions to help them make profitable decisions. But only about 45% chose to see the explanations. The gap widened dramatically when participants’ financial incentives were aligned with the lender’s interests: they earned more money if the loans were repaid. (Lender-aligned participants were about 10 percentage points more likely to skip explanations than neutrally compensated participants.) These participants were even more likely to seek out predictions, but significantly more likely to avoid explanations, particularly when told that those explanations might involve considerations of race and gender.

The pattern was clear: when financial incentives conflict with fairness concerns, people don’t just make questionable decisions; they strategically avoid information that would force them to confront the ethical dimensions of those choices.

Critically, this wasn’t about disliking extra information in general. When race and gender information was removed from explanations and replaced with arbitrary technical details, the gap in explanation avoidance between different incentive conditions almost vanished. People weren’t shunning explanations as such; they were avoiding what the explanations might reveal about discrimination and their own profit-maximizing behavior.

The Risk Experiment: Failing to See What Helps

The second experiment removed moral conflicts entirely to focus on pure decision quality. Here, participants evaluated a loan application that an AI had labeled “high risk” because of a two-year employment gap in the borrower’s work history. The researchers first asked participants how much they would be willing to pay for an explanation that would reveal whether the employment gap was indeed the primary driver of the AI’s high-risk classification.

Then came the crucial twist. Participants received free private information: the employment gap resulted from the borrower pursuing a full-time professional certificate, a benign reason that shouldn’t increase credit risk (unlike, say, a job termination). This private information should have made the AI explanation significantly more valuable: If participants knew that the AI’s high-risk label stemmed from the employment gap, and they also knew the gap resulted from pursuing education rather than being fired, they could integrate both pieces of information to reach a more accurate risk assessment.

Logic suggests that after receiving this private information, participants should value the AI explanation more highly, not less. But that’s not what happened. When asked for the second time about their willingness to pay (that is, after receiving the private information about the certificate) valuations actually dropped by 26%. People systematically failed to recognize that the explanation would help them integrate their own knowledge with the AI’s output to make a better decision.

Only when researchers explicitly walked participants through the logic, spelling out exactly how the private information and AI explanation could be combined, did the valuations increase. This revealed a novel behavioral bias: people don’t naturally see when explanations would be most useful to them, even when there’s no moral conflict involved.

The Black Box We Refuse to Open

We often complain that LLM models are like a black box and criticize AI Labs for creating them. The metaphor has become ubiquitous in debates about artificial intelligence: mysterious algorithms making consequential decisions while we’re left to wonder what’s happening inside.

But this research reveals an uncomfortable irony. When an AI algorithm gives us an opportunity to open the lid of that black box—to peer inside and understand its reasoning—we hesitate. We look away. Sometimes it’s because we’re lazy and don’t want to make the cognitive effort. More often, it’s because we don’t want to see what’s there.

In high-stakes decisions spanning credit, hiring, pricing, healthcare, and safety, people may eagerly consume AI predictions while quietly avoiding the explanations that would expose uncomfortable trade-offs or discriminatory patterns. That avoidance can skew outcomes, undermine fairness, and create hidden risks for every organization. Meanwhile, even well-intentioned professionals may systematically under-invest in explanations that would improve their forecasting by helping them combine their domain expertise with AI insights.

Building transparent AI systems is necessary but not sufficient. The real challenge isn’t engineering better explanations or making black boxes more transparent; it’s ensuring that people actually use the transparency that’s already available. Organizations must design decision-making environments and incentive structures that encourage opening the lid, even when what’s inside might be uncomfortable.

Because the black box isn’t the problem. The problem is our unwillingness to look inside.