We analyse the
Appearance of Meaning

To understand how meaning forms inside a language model, take a simple word like "bank." In one sentence it means a place for money. In another it means the side of a river. The word itself stays the same. What changes is the context around it.

That observation motivates my recent research on large language models. The question is not whether these systems "really understand" language in the full human sense. That debate is too vague on its own. The more useful question is: what kind of context-sensitive behavior do these models show, and how is that behavior implemented inside them?

My work approaches that question in three steps.

The first paper makes a broad philosophical argument. Transformer models do not seem to treat words as if they each come with one fixed, dictionary-like meaning that is simply looked up and applied. Instead, they constantly rebuild a word's role from the surrounding context. In that sense, their success supports a more relational picture of language: meaning is not carried by isolated words alone, but by words as they appear in context.

The second paper, The Appearance of Meaning (AoM), turns that broad idea into something testable. Instead of arguing directly that language models have "meaning proper," it defines a narrower empirical target: a measurable profile of meaning-like behavior. Can the model disambiguate words from context? Does it respond to changes that alter meaning more than to superficial changes that do not? Does it track discourse constraints across longer passages? Under controlled tests, the answer is often yes. More importantly, when I intervene on the model's internal states, I find that context-conditioned internal states causally control those interpretation shifts.

The third paper, Mechanics of Meaning (MoM), asks the next question: what form does that control take inside the model? If context-sensitive control is real, is it spread diffusely across the model's internal activity, or is some of it recoverable in a more selective feature basis? To test that, I compare two kinds of interventions: broader edits to the model's internal state, and more selective edits in a sparse feature space.

The answer is qualified, but informative. In some early disambiguation settings, it can help to work in a more selective vocabulary - isolating a smaller set of internal variables - rather than treating the model's state as one big, entangled object. In other settings (especially broader discourse behavior), that advantage weakens and the relevant control looks more distributed.

That matters because it rules out two overly simple stories at once. One is the dismissive story that language models are doing nothing but shallow pattern mimicry. The other is the overly neat story that there must be a single internal "meaning unit" for each concept. What the evidence suggests instead is more interesting: context-sensitive control in language models is real, measurable, and internally heterogeneous. Some of it is selectively recoverable. Some of it is not.

So the arc across the three papers is a descent in scale. The first asks what kind of theory of language transformer models seem to support. The second asks what can be measured and causally tested. The third asks how that control is organized in the model's internal feature space.

That does not settle the largest philosophical questions. It does, however, turn a vague argument about whether models "understand" into a more precise research program: what kinds of meaning-like behavior exist, which internal states control them, and when that control is more selective rather than broadly distributed.

The papers are linked above in reading order.