Perspectives

There is a question circling most serious conversations about artificial intelligence that rarely gets answered directly: does AI actually think? And if it does, does that mean it is intelligent? The two are routinely treated as the same question. They are not – and the difference between thinking and intelligence matters.

The confusion is understandable. When a large language model produces a fluent essay, suggests a diagnosis, or explains a complex idea with apparent clarity, it is very hard not to attribute some form of mental activity to it. It feels like thinking. It produces the outputs we associate with thinking. But feeling like thinking and actually being thinking are different claims – and both are different again from the claim that a system which thinks is actually intelligent.

There is a reasonable case that current AI does think – or at least does something close enough to thinking to deserve the name in a functional sense. That claim deserves justification, because it is doing important work in everything that follows.

The most direct justification is behavioural. When a system consistently produces outputs that are sensitive to the meaning of its inputs – not just their surface form – something more than mechanical processing appears to be happening. Pose a contradiction and the system notices. Introduce an ambiguity and it navigates it. Shift the meaning of a question subtly and the response shifts accordingly. A central heating thermostat does not do this. It responds to a signal – temperature – without any relationship to what that signal means. A language model responds differently, and the difference tracks meaning in ways that are not trivial.

A second justification comes from the absence of a satisfactory alternative description. If we want to deny these systems the name thinking, we need a convincing account of what else to call it. It is more than a simple lookup function – the space of possible inputs is too big for stored responses. Straightforward pattern-matching does not account for the contextual coherence of extended exchanges. Something is being constructed in response to meaning, not merely retrieved.

But the most important point is this: the claim that AI thinks in a functional sense is not a claim that AI thinks as humans do. There is no suggestion that there is any inner experience behind the outputs, or that the underlying mechanism is the same. It is the more modest claim that the functional description fits well enough to be useful. Reaching the threshold of functional thinking – systems that respond to meaning rather than merely to signals – is a real achievement. It is just not the same thing as intelligence – and the rest of this piece seeks to explain why.

Intelligence is a more demanding concept than thinking. Its most important dimension – and the one that separates it most clearly from what AI currently demonstrates – is the capacity to generalise reliably in new situations. Not situations that resemble prior experience, but situations that are structurally novel: where no template applies, where the right response cannot be retrieved or approximated from what has gone before.

Children illustrate this capacity when they acquire language – one of the most complex cognitive achievements we know of – from a few years of exposure, with no formal instruction and imperfect correction. By school age they can understand sentences they have never heard, say things they have never been taught to say, and deploy metaphor and irony with a facility that eludes our best AI systems in edge cases. Chomsky (1965) observed that children end up with a knowledge of grammar far richer than their exposure could justify by pattern extraction alone. They are not learning a very long list of sentences. They are acquiring an underlying system that produces sentences – a grammar, not a catalogue.

This distinction – between acquiring a productive structure and extracting surface patterns – is at the heart of what separates human learning from what large language models do. A pattern is a regularity in data: something that recurs often enough to be detected and used predictively. It tells you that something happens. It does not tell you why, and it gives you no insight on what would happen if conditions changed. Structure, by contrast, is a causal model. It is a representation of why things happen and through what mechanism. It generates correct predictions about situations never previously encountered, because what has been acquired is not a distribution of examples but a model of the underlying reality.

Large language models are trained to predict what text is likely to follow a given input, across a vast body of human-generated language. In doing so they become remarkably good at capturing statistical regularities: which arguments tend to appear together, what the surface shape of a well-reasoned answer looks like, and how fluent confident prose is structured. But absorbing the surface traces of structural human thinking is not the same as acquiring the capacity for structure-building itself. The outputs of the minds that built causal models are in the training data. Whether the model-building process that generated those outputs is present in the system is a different and much less certain question.

Here is an analogy: imagine learning how a clock works by studying millions of photographs of labelled clock faces. You would become very good at recognising the time but what you would not acquire, from photographs alone, is an understanding of the causal structure that produces the behaviour you have learned to predict. Your predictions might be accurate but your understanding of what generates them would be limited. This is, in essence, the situation of a system trained on the outputs of structured human thinking. It has studied the clock faces with remarkable thoroughness but it has not understood the mechanism behind them.

The clock analogy points at what is perhaps the deepest issue in this discussion and this is the difference between understanding something from the inside and approximating it from the outside.

Consider someone who genuinely knows a city, not from a map or a set of directions, but from years of living in it. They do not navigate by memorising turn-by-turn instructions. They have an internal model which gives them a feel for direction and distance, and an intuitive sense of how neighbourhoods relate to one another. This gives ability to find their way from somewhere they have never been to somewhere they have never visited. They know the city from the inside. Someone who has only ever followed a prescribed route knows the same streets differently and the moment they step off that route, the difference becomes immediately apparent.

AI. These systems learn the shape of understanding from the outside. By being trained on the outputs of minds that genuinely did understand things, they acquire the characteristic patterns of inference, the typical structure of explanation, the surface features of reasoning. In many contexts this approximation is so good the difference is invisible. But it remains an approximation of the form without instantiating the substance – tracing the shadow of a mechanism rather than possessing the mechanism itself.

The philosopher John Searle illustrated this point with the Chinese Room thought experiment (Searle, 1980). A person sits in a room following a rulebook that tells them how to respond to Chinese symbols passed under the door. To anyone outside, the responses look like those of a fluent Chinese speaker. But the person inside understands nothing. They are manipulating symbols correctly without grasping their meaning. Searle’s point is that producing the right outputs is not the same as understanding – and understanding, not output, is what we should be asking about when we ask whether a system is intelligent.

This inside/outside distinction also illuminates one of the most concrete and widely experienced limitations of current AI: what the field calls hallucination, but which might more accurately be described as a problem of accuracy or groundedness. These systems produce false information in form of fabricated statistics, invented quotations, and plausible-sounding citations that do not exist – with the same fluency and apparent confidence as when they produce something true. The word hallucination implies something mind-like going wrong: a more honest description is that the production mechanism has no reliable internal relationship to truth. There is no process running alongside that checks whether a claim is accurate before it is made, no felt sense of being on solid or uncertain ground.

This is not a correctable flaw sitting alongside otherwise sound intelligence. It is a symptom of the inside/outside gap. A system that truely understood – that had real causal models of the world and an internal relationship to the truth of its claims – would have a grip on the difference between what it knows well and what it is merely generating plausibly. The groundedness problem and the intelligence question are the same question viewed from different angles.

The groundedness problem has a further dimension that deserves naming directly. It is not just that AI systems sometimes produce false outputs. It is that they lack the capacity to know when they are doing so – what psychologists call metacognition – thinking about thinking, monitoring the quality of one’s own reasoning in real time.

This capacity is central to genuine intelligence. Knowing the limits of what you know – being able to say “I am not certain about this” and mean it, because you have actually assessed the basis for your claim and found it wanting – is not a weakness. It is a mark of intellectual reliability. It is what allows human judgement to be trusted in conditions of uncertainty, because the person exercising it has a calibrated sense of where their confidence is and is not warranted.

Current AI systems do produce hedges: “I may be mistaken,” “you might want to verify this.” But these are learned surface patterns – the system has encountered countless examples of careful writers qualifying uncertain claims, and reproduces that behaviour. It is not actually uncertain in the way a human speaker would be using the same words. It is producing the surface behaviour of humility without the underlying self-awareness that gives that behaviour its meaning and reliability. Once again, this is the outside without the inside.

We are now in a position to say something more positive about what genuine intelligence involves, and why the gap between it and what AI currently demonstrates is structural rather than merely technical.

Polanyi (1966) observed that we know more than we can tell. A great deal of human knowledge – perhaps most of it in any domain involving real skill – is not held as explicit statements that could be written down or directly taught. It is held in trained perception, in an acquired sensitivity that operates faster and more fluidly than conscious articulation. The cyclist who cannot explain how they balance. The clinician who senses something is wrong before they can say what. The experienced navigator who knows a city from the inside rather than from a set of instructions. This tacit knowledge is not irrational. It is the accumulated learning of a great deal of rational experience, but it lives in a register that language cannot fully reach.

The implication for AI is pointed. A system trained on text is trained on the explicit layer of human knowledge – the part that has been articulated and made transmissible through language. It is trained by design on what can be told. But if Polanyi is right that what can be told is systematically less than what is known, then such a system is trained on a representation of human knowledge that is structurally incomplete in a specific and fundamental way. It has access to the shadow cast by the iceberg, but not the iceberg itself.

This tacit foundation is not simply a repository of facts and skills. It is the ground in which judgement grows. Judgement – the capacity to reach a well-founded conclusion in conditions of uncertainty, where information is incomplete and no rule clearly applies – cannot be fully articulated as a procedure, because the moment you have articulated it fully, what you have is a rule, and a rule is a substitute for judgement rather than the thing itself. Aristotle (Nicomachean Ethics, Book VI) called the highest expression of this capacity practical wisdom (phronesis). This is the ability to perceive what a situation actually calls for, not by applying a template but through direct, experienced perception of what matters here, now, in these particular circumstances. It is acquired through living, not training.

There is one further dimension worth naming. Judgement requires not only knowledge and the ability to generalise, but care – a real orientation toward getting things right, toward the people affected by a decision, toward the consequences of being wrong. This is not sentiment. It is the formed disposition that orients attention and gives judgement its seriousness. A person of good judgement is not merely cognitively equipped. They have something at stake. Current AI systems do not – and the absence of genuine stakes is not incidental. It may be part of why their outputs, however fluent, cannot quite be trusted in the way a person of good judgement can be trusted. The fluency is real. The stake in being right is not.

Current AI systems almost certainly think, in the functional sense established earlier. That is a real and useful achievement. But the arguments developed here suggest that the gap between functional thinking and intelligence is not a matter of degree – not simply a question of more training data, better architectures, or greater computational power. It is a structural problem. The gap does not arise because these systems lack sufficient data or processing power – it arises because the architecture itself is oriented toward predicting and reproducing the surface patterns of human thought, rather than acquiring the causal evaluative foundations from which that thought actually grows. More data and greater scale may close the surface gap but they cannot, of themselves, supply what was never part of the design.

These systems have learned from the surface traces of human thought – from what can be written down and transmitted through language – without access to the embodied, tacit knowledge that gives language its cognitive roots. They have absorbed the outputs of structural, causal human thinking without acquiring the capacity for structure-building themselves. They produce confident-sounding claims without any reliable internal relationship to truth, and they lack the metacognitive layer that would allow them to monitor the quality of their own reasoning. They approximate the outputs of judgement without the formed values and authentic stakes that make judgement trustworthy.

Horn and Cattell’s distinction between crystallised and fluid intelligence is useful here (Horn & Cattell, 1966). Crystallised intelligence – the accumulated knowledge and reliable application of established patterns – is where these systems are strongest. Fluid intelligence – the capacity to reason through genuinely novel problems, to form new abstractions, to think beyond the familiar -is where they are weakest, and weakest precisely where the stakes are highest. It is a different profile from human intelligence, not simply a lesser version of it.

This matters practically, not just philosophically. These systems are being deployed in medicine, law, real estate, education, and public information – contexts where the difference between fluent output and real understanding is consequential. A system that performs well within familiar territory but degrades at the edges, that generates confident claims without an internal relationship to their truth, and that has no stake in being right, is not a reliable substitute for human judgement in high-stakes or novel situations. Treating it as one, because the outputs are often compelling, is a category error with consequences.

None of this is an argument for dismissing what these systems can do. Their competence within familiar domains is real, growing, and practically valuable. But that value is best realised by people who understand clearly what they are working with – and what they are not.

References

Aristotle. Nicomachean Ethics, Book VI. Translated by Crisp, R. (2000). Cambridge University Press.

Cattell, R. B. (1971). Abilities: Their Structure, Growth, and Action. Houghton Mifflin.

Chomsky, N. (1965). Aspects of the Theory of Syntax. MIT Press.

Horn, J. L. & Cattell, R. B. (1966). Refinement and test of the theory of fluid and crystallised general intelligences. Journal of Educational Psychology, 57(5), 253–270.

Polanyi, M. (1958). Personal Knowledge. University of Chicago Press.

Polanyi, M. (1966). The Tacit Dimension. Routledge.

Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–424.