A while back, I argued that there are two ways to put AI to work. You can treat it as an oracle — an answer machine you query and obey — or as an interlocutor, a thinking partner that sharpens your judgment instead of replacing it. I still think the distinction stands. But a recent essay by Ethan Mollick, Choosing to Stay Human, forced me to revisit it — because Mollick pointed to a troubling cost of the oracle mode that I had not fully appreciated back then, and because the evidence of this cost that has piled up since is hard to ignore.

So let me start where I left off.

The wrong question, answered beautifully

The most common way organizations use AI is as an oracle. You ask, it answers, and the answer arrives fast, fluent, and confident. This is genuinely useful — until you remember that AI has no way of knowing whether you asked the right question. If your problem is poorly defined, AI will not tell you so. It will produce a polished, sophisticated, and occasionally brilliant response to the wrong question, and it will do it at a speed and price that makes the output feel authoritative. Garbage in, gospel out.

That is the first danger, and it’s the one I’ve written about before: AI doesn’t move you toward a good decision; it moves you faster in whatever direction you’re already pointed. Point it wrong, and it becomes an efficient engine for going wrong. For organizations that already struggle to define their problems clearly — which applies to most small businesses and nonprofits, stretched thin and rarely afforded the luxury of a strategy team — this isn’t a small risk. It is the risk.

The subtler cost: cognitive surrender

Here is what Mollick’s essay made me see more clearly. Even when you ask the right question, the oracle mode exacts a second, quieter tax. Mollick’s colleagues at Wharton have a name for it: “cognitive surrender” — the documented tendency of people to stop thinking about a problem and simply defer to the machine, even when the machine is wrong.

The evidence is accumulating from every direction. A  study from researchers at Carnegie, Oxford, MIT, and UCLA found that just ten minutes of AI-assisted problem solving measurably reduced people’s ability to work through problems on their own afterward. Once the AI was gone, they skipped more and solved less — across both math and reading. The crucial detail: the damage was concentrated among those who used AI to get direct answers. People who asked only for hints or clarification showed no meaningful drop-off. That is the oracle and the interlocutor, separated inside a single experiment.

The same issue appears elsewhere. A study in Nature found that collaborating with generative AI boosted immediate performance, but that the gain vanished once people returned to working alone, and that the hand-off left them less motivated and more bored. In software development, where AI tools became standard fast, novices posted real productivity gains while quietly compromising their own grasp of the fundamentals: conceptual understanding, code reading, and debugging. AI-enhanced output, it turns out, is not a shortcut to competence.

Mollick offers the cleanest illustration of all, drawn from two studies by overlapping research teams. In one, about a thousand high-school students in Turkey used plain ChatGPT to study math; they did their homework better, felt they were learning more, and then underperformed their AI-free classmates on the test. In the other, an AI tutor that delivered a personalized sequence of problems rather than answers, across schools in Taipei, produced gains equivalent to months of extra schooling. Same technology, opposite results. The entire difference was whether the AI did the thinking or provoked it.

What the chess players reveal

The study that ties all of this together comes from Wharton and INSEAD, and I’m a little embarrassed it took Mollick’s essay to send me back to it. Researchers ran a large experiment with around three hundred chess players, giving them two different kinds of AI help. The first was an action signal: here is the best move. The second was an attention signal: this is a critical moment — slow down and think.

The action signals did work: players made better moves. But the help came with a hidden bill: in the moves that followed, performance dropped. The players had gone passive; they had stopped engaging, and when the AI fell silent, they struggled. The attention signals produced the opposite effect: immediate gains were more modest, but players thought harder, stayed in the game, and played better afterward. The AI had functioned as a complement to their judgment rather than a substitute for it.

Read that again with our two words in mind. The action signal is the oracle. The attention signal is the interlocutor. The experiment doesn’t merely support the distinction — it measures the cost of choosing wrong. And it points to something more useful than “AI good” or “AI bad”: the right kind of support depends on who is deciding, what they’re deciding, and how fast they need to decide.

When a single decision is high-stakes and speed is everything — aviation alerts, real-time fraud — give me the answer. When decisions unfold over time and expertise compounds, protect the thinking.

Which brings me back to small organizations

For a nonprofit or a small business, the answer to “who is deciding, what, and how fast” is almost always the second case. These are not high-frequency, single-shot decision environments. They’re places where judgment compounds — where the same handful of people will face the same class of problem again next quarter, and where there is no second analyst to catch the error. Which is exactly why the oracle mode is the wrong default for them, and why the surrender temptation bites hardest precisely where it does the most harm. When you’re stretched thin, the machine that simply hands you an answer is almost irresistible.

But this is also where the real opportunity lives — and it’s bigger than avoiding harm. Used as an interlocutor, inside a disciplined process rather than as a replacement for one, AI becomes the thinking partner these organizations have never been able to afford. The loop is simple to state: define the problem, surface and stress-test your assumptions, reframe, and only then generate and weigh options.

AI earns its place at each step not by answering but by pushing back. The equalizing power of AI for small organizations isn’t that it hands them the same answers as the big players. It’s that it gives them the ability to ask better questions — and that ability, embedded in a process, is a genuine change in what a small team can do.

And the interlocutor can do something the oracle can’t

It would be easy to read all of this as a counsel of caution: use AI gingerly, keep your hands on the wheel. But the most exciting part is what becomes possible when a sharply defined problem meets a model you’re genuinely engaging with.

Consider the combinatorial geometry problem that the mathematician Paul Erdős posed in 1946. For eighty years, it resisted the tools of combinatorial geometry. Last month, OpenAI reported that a general-purpose reasoning model — not one trained for the task — found a solution by reaching into an entirely different field, algebraic number theory. It questioned an assumption the human researchers had quietly shared and produced an answer no one expected.

Notice two things. First, that result was available only because the problem was defined with absolute precision, which is the whole argument of this piece, running in reverse. Second, questioning a shared assumption and proposing an unexpected framing is not what an oracle does. It is the defining move of an interlocutor. The model didn’t just retrieve a solution; it challenged the way the problem had been seen. That is the capability worth reaching for — and you can only reach it if you’ve done the work of defining the problem and stayed engaged enough to recognize a good reframe when it arrives.

Choosing to stay human, as an organization

Mollick’s essay is titled Choosing to Stay Human, and his point is that the choice is ours: intentional use, rather than reflexive dependence or reflexive avoidance. He notes, rightly, that the defaults are being set right now — by the companies designing AI to be frictionless, by employers deciding what “using AI well” means, by everyone teaching the slippery thing we call AI literacy. Once a generation builds its habits, those defaults will be hard to reverse.

For a small organization, the stakes of that choice are concentrated and immediate. What you choose to hand to the oracle, and what you choose to keep as your own thinking, will shape the institution you become. The discipline I’m describing — staying in the problem, even with a capable machine at hand — is not nostalgia. It’s the difference between an organization that gets answers and one that gets better at asking.

Protect that, and AI stops being a way to move faster in the wrong direction. It becomes the partner you could never afford.