“AI makes you smarter but none the wiser” is the phrase from the Fernandez et al paper that stood out.
Their point that AI can improve task performance while leaving people poorly calibrated about their own ability matters. This is because a great deal of AI adoption is currently being judged primarily through output.
And, of course, questions such as: did we produce more? Did it take less time? Was the draft better? Was the analysis cleaner? are important.
But these questions alone are not enough. They tell us what the human -AI system produced. But they do not tell us what the human learned at a deeper level about the challenge under investigation.
And this gap is where the trouble starts. Better outputs generate confidence. But confidence can reduce scrutiny and become dependence. And dependence can start to look suspiciously like competence – especially from the outside.
This is why I’m wary of AI capability programmes that focus primarily on tools, prompts and productivity gains.
Prompting matters of course, but it is not the same as the deeper capability of knowing how to think with the tool without letting the tool think for you.
The challenge is that most AI interfaces are built for smoothness. The user asks and the system answers.
But in this way the friction that often promotes learning disappears. We lose the edgy creative friction where deep learning and true understanding takes place.
We want to encourage and reward processes that ask: what did you think before you asked the machine? Why do you believe that? What evidence would change your mind? What might be missing? Where could this answer fail?
Recent work on AI “provocations” is useful here: systems that challenge assumptions, surface counter arguments and force a pause for thinking before acceptance are important.
If AI is going to build capability it has to make us think harder at the right moments.
The aim should not be to use AI less. It should be to use it in ways that make our human thinking stronger.
Read the Latest Polymath Mind Substack from Adam Riley and I here.