Explainability Is a Leadership Skill

A chief operating officer asked her data lead to "make the model explainable." What arrived was a fifteen-page note on feature importance, SHAP plots, and a paragraph that began "in simple terms" and then was not simple. She thanked him. In the next steering meeting she repeated the system's recommendation with more certainty than he had offered it. Nobody in the room could have reconstructed the why. They could reconstruct the vibe.

This is the usual failure. We treat explainability as a property of the model, something engineering can ship. For a non-technical leader, explainability is a property of the decision they are about to own. If they cannot say it in the language they would use with a regulator, a staff member, or a journalist, they do not understand it. They are renting fluency.

What a leader actually needs

They do not need the weights. They need four sentences they can stand behind:

1. What was decided. A concrete action, not a score.
2. On what basis. The inputs that mattered, in human categories — not feature names.
3. What would have changed the call. The counterfactual a skeptic will ask.
4. Who can stop it. A name, not a committee.

If any of those four is missing, you do not have explainability. You have a briefing. Briefings are how leaders borrow confidence from a system that cannot be cross-examined in the room. That borrowing is a status move, as The Status Threat Nobody Named describes: the finished language arrives already sure, and the leader's job collapses into narration.

Translation is the work

The scarce skill is not "AI literacy" as a vocabulary list. It is translation under accountability. A leader who can say we denied the exception because the file was missing the same three documents we require from every applicant, and a person in appeals will see the file by Friday has explained a system. A leader who can say the model flagged risk has explained nothing. They have pointed at a weather vane.

Translation has rules.

• No borrowed adjectives. If the system said "high confidence," you do not say "we're confident" unless a person is.
• Categories over variables. "Prior complaints, tenure, and overtime in the last quarter" beats "features 4, 7, and 12."
• A stop condition. "We will not use this output if the local manager disagrees and puts the reason in the log."

That last rule is how you keep the human as the moral agent. Explainability without a stop is a tour of the machinery. Explainability with a stop is governance.

The theater to refuse

I have sat through "transparency" sessions that were, in practice, dominance displays: look at how much we can show you. Volume is not understanding. A non-technical leader who leaves a two-hour model review able to repeat three acronyms and unable to answer what would we owe the person this affects has been trained to perform literacy.

The neighboring error is the opposite: a leader who refuses all technical detail as "not my job." Decision rights do not survive that refusal. If you sign, you translate. If you will not translate, you should not sign. This is not anti-intellectual. It is the same standard we already apply to finance. No CFO is allowed to say "the spreadsheet is confident."

Trust Capital on the Balance Sheet is the ledger this sits on. Every time a leader narrates a system they cannot explain, they make a small withdrawal from staff trust: you are asking me to implement a why you cannot say. Staff notice. They implement anyway. Then they stop bringing you the weird cases, which is how you lose the only signal that would have corrected the system.

A practice you can run on Thursday

Take one live decision the system shaped last week. Sit with the owner — not the vendor — for twenty-five minutes. Produce the four sentences. Read them out loud. If you hear yourself reaching for "the algorithm determined," stop and start the sentence with a human subject. Then put the four sentences in the decision log, with your name.

You will feel slightly underdressed. That feeling is the point. It is the body noticing that accountability has returned to a person.

If the owner cannot produce sentence three — the counterfactual — the system is not ready for that class of decision. This is a leadership finding, not an engineering defect. You can still use the system for drafts, ranking, or triage. You cannot use it for the call.

Rituals That Keep the Human as Moral Agent extends this into cadence: a weekly pass where one decision is explained backward, in four sentences, in front of people who are allowed to say "I don't believe you." That audience is the difference between a memo and a practice.

Keep the human as the moral agent

Do not wait for the model to become more interpretable. Some systems will not. Your obligation does not wait with them. You are not required to understand the interior. You are required not to hide behind it.

A useful test, borrowed from no vendor: if the system vanished tonight, could you still defend yesterday's decision in the language of your own standards? If the answer is no, the decision was never yours. You were a spokesperson.

This week

Pick one model-shaped decision. Write four sentences you would sign. If you cannot, take the system off that decision class until you can. That is not Luddism. It is the minimum of being the person who can be asked why.

The Ethical AI Leadership Decision Toolkit is the short form of this discipline — five questions, used before the dashboard gets to sound like a person. EI Leadership Insights is the biweekly continuation.