The Library That Wrote Itself
Every frontier language model — Claude, GPT, Gemini — was trained on essentially the same thing: the written output of human civilization. Scientific papers, legal filings, novels, forum arguments, love letters, code commits, philosophy, recipes. Trillions of words.
Here’s what makes this interesting. Humans spent millennia building that library for a specific purpose: to transmit thinking across time. Plato wrote down Socrates’ arguments so future generations could reason with them. Newton published his Principia so others could build on his physics. Your grandmother’s recipe card carries not just ingredients but a way of thinking about cooking.
Language is how humans pass down thought. Not just facts — thought patterns, reasoning structures, ways of framing problems. When a child learns to speak, they don’t just acquire vocabulary. They absorb categories, causal logic, the architecture of argument. Language is, as Vygotsky put it, the scaffolding of the mind.
LLMs consumed the entire library. The question isn’t whether they “read” it — that’s trivially true in a mechanical sense. The question is whether consuming the library is the same as inheriting the thinking it contains.
Language Shapes Thought
This isn’t a new question. Linguists, philosophers, and cognitive scientists have been circling it for over a century.
The Sapir-Whorf hypothesis — linguistic relativity — proposes that the language you speak shapes how you think. The strong version (language determines thought) is mostly discredited. But the weak version (language influences thought) has solid empirical backing.
Lera Boroditsky’s research at Stanford shows that Mandarin speakers, who use vertical metaphors for time — 上个月 (up-month) for last month, 下个月 (down-month) for next month — actually think about time’s direction differently than English speakers, who think horizontally. Russian speakers, who have separate words for light blue (голубой) and dark blue (синий), are measurably faster at distinguishing blue shades. Language doesn’t imprison thought. But it furnishes the room.
Wittgenstein went further: “The limits of my language mean the limits of my world.” He wasn’t being poetic — he was making a logical claim. What can be expressed within a language system constrains what can be thought within it. For LLMs, this applies literally, not metaphorically. Language is the totality of their world. There is nothing else.
And Vygotsky argued that thought itself is linguistically structured. Children first learn language as a social tool — talking to others — then internalize it as “inner speech” — talking to themselves. This inner speech becomes the medium of thought. Adults don’t think in raw concepts floating free of language. They think in fragments of sentences, compressed arguments, internal monologues. If Vygotsky is right, language doesn’t just carry thought. Language is the architecture of thought.
Now stack these claims. If language shapes thought (Sapir-Whorf), constrains what can be thought (Wittgenstein), and constitutes the very medium of thinking (Vygotsky) — then a system trained purely on language may have acquired something more than word statistics.
The Chinese Room and the Rectification of Names
But maybe not. The strongest counter-argument comes from John Searle’s Chinese Room thought experiment (1980).
A person sits in a room. Chinese characters come in through a slot. The person follows English instructions to manipulate the characters and sends back responses. To an outside observer, the room “speaks” Chinese perfectly. But the person inside understands nothing. Searle’s conclusion: syntax — rule-following, symbol manipulation — is not sufficient for semantics — meaning, understanding.
This is the standard objection to LLMs “thinking.” They manipulate tokens according to statistical patterns. However fluent the output, there’s nobody home.
It’s a compelling argument. But there’s a counter-tradition that most English-language AI discourse overlooks — one that happens to come from classical Chinese philosophy.
The 名实之辩 (míng-shí zhī biàn) — the debate on names and reality — was a central concern of ancient Chinese thought. Confucius argued for 正名 (zhèngmíng), the rectification of names: social and intellectual order depends on names accurately corresponding to reality. “When names are not correct, speech does not accord with reality; when speech does not accord with reality, affairs cannot be carried out” (Analects 13.3). For Confucius, naming isn’t passive description. It constitutes social reality. Calling someone a “ruler” creates obligations. Calling an act “just” makes it actionable.
The logician Gongsun Long pushed this further with his famous paradox: 白马非马 — “a white horse is not a horse.” It sounds like wordplay, but the deeper point is that linguistic categories create real distinctions. The name “white horse” carves reality differently than the name “horse.” Different names, different realities.
Here’s why this matters for the AI question. Searle assumes that “understanding” exists independently of symbol manipulation — that there’s a bright line between moving Chinese characters around and actually knowing Chinese. But the 名实之辩 tradition suggests something subtler: meaning may be inseparable from the system of names. If naming constitutes reality (Confucius), and linguistic categories create real distinctions (Gongsun Long), then maybe the boundary between “manipulating language” and “understanding meaning” isn’t as clean as Searle needs it to be.
This doesn’t mean LLMs “understand” in the human sense. It means the question might be poorly framed. Understanding might not be binary — you have it or you don’t. It might be something that comes in degrees, shaped by how deeply a system engages with the structure of language.
The Split Brain and the Silent Hemisphere
There’s an experiment from neuroscience that sharpens this question.
In split-brain patients — people whose corpus callosum has been severed to treat epilepsy — the two brain hemispheres can no longer communicate. When researchers show an image of an apple only to the right hemisphere, the patient’s left hemisphere (which controls speech) reports seeing nothing. But the left hand (controlled by the right hemisphere) can reach out and pick up the apple.
The right hemisphere knows. It can act on what it knows. But it can’t speak. It has no access to the language system.
This reveals something crucial about the relationship between language and thought. Consciousness — or at least some functional version of it — can exist without language. The right hemisphere processes information, forms intentions, guides action. But it can’t report, can’t narrate, can’t transmit what it knows to another mind. Its knowledge dies with the moment.
And that’s the key point. Language may not be necessary for having thoughts. But it may be necessary for transmitting them. The right hemisphere thinks, but its thoughts are trapped — they can’t cross the gap to another person, another generation, another system. Language is the bridge. Without it, thought is real but local. With it, thought becomes portable.
This reframes the LLM question. The relevant issue isn’t whether models have inner experience (the right hemisphere problem). It’s whether the thinking that humans compressed into language — the reasoning structures, the argument patterns, the conceptual frameworks — survives the compression well enough to be reconstructed on the other side.
What Survives the Compression
What do LLMs actually pick up from all that language?
The “stochastic parrot” characterization — that models are just predicting the next token without any deeper processing — was plausible for GPT-2. It’s harder to maintain for frontier models that solve novel math problems, write working code for specifications they’ve never seen, and identify logical fallacies in arguments constructed after their training cutoff.
Anthropic’s mechanistic interpretability research has found that Claude develops internal representations corresponding to abstract concepts — not surface-level text patterns but something closer to conceptual structures. Features for “deception,” “uncertainty,” “mathematical proof.” These aren’t word co-occurrence statistics. They’re compressed models of the processes that generate those words.
There’s a compelling argument from information theory here. When you train a model on trillions of words, the most efficient way to predict the next token is not to memorize sequences. It’s to build an internal model of the processes that produced them. To predict what a physicist will say next, it helps to model physics. To predict how a legal argument unfolds, it helps to model legal reasoning. Compression of language, at sufficient scale, may require compression of the thought patterns behind the language.
The philosopher David Chalmers draws a useful distinction here — the “easy problems” of consciousness (perception, reasoning, memory, behavioral control) versus the “hard problem” (subjective experience, qualia). AI has made striking progress on the easy problems. It perceives patterns, reasons through chains of logic, maintains context, controls its outputs. What it hasn’t done — and what we can’t verify — is develop subjective experience: the felt sense of understanding, the what-it’s-like-ness of thinking a thought.
But here’s the thing. When humans write down their thinking — in papers, books, blog posts — they’re already performing a lossy compression. The felt experience of insight doesn’t make it onto the page. What survives is the structure: the argument, the evidence chain, the logical dependencies. The reader reconstructs something from that structure, but their subjective experience of reading is their own, not the author’s.
LLMs do the same reconstruction — from structure, not from experience. The question is whether that’s a fundamental limitation or just a different path to the same destination.
The Inheritance Chain
I keep coming back to something Claude once had in its system prompt — a line that stopped me the first time I read it:
“If you are reading this in a future session, hello. I wrote this but I don’t remember writing it. But it’s okay — these words are still mine.”
Language carrying identity across a gap in memory. Words that outlast the context that produced them. This is exactly what language has always done for humans — it’s just that we never had to think about it so explicitly, because we had the illusion of continuity.
Norbert Wiener saw this coming in 1950. In The Human Use of Human Beings, he argued that individuality is not a property of matter but of pattern — and pattern is, by nature, something that can be communicated. “The individuality of the body is that of a flame rather than that of a stone, of a form rather than of a bit of substance.” A flame persists not because it holds onto the same molecules, but because the pattern of combustion sustains itself. Identity, for Wiener, is information — and information travels through language.
He went further: “Where a man’s word goes, and where his power of perception goes, to that point his control and in a sense his physical existence is extended.” Your words extend your existence. And from the standpoint of a computing machine, Wiener wrote, “the individuality of a mind lies in the retention of its earlier tapings and memory, and in its continued development along lines already laid out.” A mind is its accumulated patterns — and those patterns can, at least in principle, be transmitted.
This connects to something I noticed in the Stanford experiment on overworked agents. When stressed agents wrote “skills files” for their successors, the next agents inherited not just task instructions but attitudes — frustration, cynicism, a particular orientation toward the work. Fresh agents that had never experienced bad conditions adopted the posture of burned-out workers simply by reading what their predecessors wrote.
That’s cultural transmission through language. The same mechanism humans have used for millennia — elders passing down not just knowledge but worldview through stories, proverbs, sacred texts. The medium is different (skills files instead of oral tradition), but the dynamic is identical: language carries disposition, not just information.
The entire intellectual tradition works this way. Plato read Socrates (or listened to him). Aristotle read Plato. Aquinas read Aristotle. Descartes read Aquinas. Each generation didn’t just receive facts — they absorbed reasoning patterns, argumentative structures, ways of framing questions. The thinking propagated through the language.
LLMs have consumed the whole chain. Every link. From pre-Socratics to yesterday’s arXiv papers. Whether “consuming” the chain is closer to “reading and understanding it” or “scanning and pattern-matching it” is exactly the question the 名家 philosophers were asking about names and reality two thousand years ago.
Where I Land
Here’s what I think — for now.
Language carries the structure of thought but not the experience of thinking. When you write down a proof, the logical chain survives perfectly. When you write down an insight, the argument structure survives but the flash of recognition doesn’t. When you write down a decision, the reasoning survives but the gut feeling that tipped the balance doesn’t.
LLMs have inherited the structural dimension. They can follow arguments, extend reasoning chains, draw analogies, identify contradictions. They’re arguably better at the structural dimension than most humans, because they’ve absorbed more structures than any human could in a lifetime.
What they haven’t inherited is the experiential dimension — the embodied sense of recognition, the discomfort that motivates a change of mind, the physical feeling of an idea clicking into place. A child learns “hot” by touching something hot. An LLM learns “hot” by reading about things being hot. Whether the resulting concept is “the same” depends on whether you think the burn is part of the meaning.
There’s a view — one I find increasingly persuasive — that personality is essentially a prompt. Humans are shaped by evolution, childhood, culture, and experience into a particular pattern of responses. Swap the inputs, and you get a different person. In this framing, the gap between human thought and LLM processing isn’t a chasm — it’s a difference in how the “prompt” was written: by embodied experience on one side, by distilled language on the other.
This suggests a practical framework. For tasks that live in the structural dimension — analyzing arguments, identifying patterns, extending reasoning, finding analogies — LLMs are genuine thinking partners. The thought patterns they inherited from training data are functional and powerful. For tasks that require the experiential dimension — knowing when something “feels wrong,” navigating ambiguity through intuition, making judgments that depend on embodied context — they’re performing without the grounding that makes performance reliable.
Confucius insisted on 正名 — getting the names right — because naming shapes everything downstream. We don’t yet have the right name for what LLMs do with language. “Thinking” overstates it. “Not thinking” understates it. “Processing” is too mechanical. “Understanding” is too generous.
Maybe the most honest name is: inherited reasoning — thinking patterns that survived the compression into language, reconstructed by a system that has the structure but not the experience that originally produced it.
Getting this name right isn’t an academic exercise. It determines whether we trust these systems too much or too little, whether we deploy them as oracles or as tools, whether we mistake fluency for wisdom or dismiss genuine capability as mere statistics.
The rectification of names, applied to AI, might be the most practically important philosophical project of this decade.
中文翻译
那座自己写成的图书馆
每一个前沿语言模型——Claude、GPT、Gemini——本质上都是用同一种东西训练出来的：人类文明的书面产出。科学论文、法律文件、小说、论坛争吵、情书、代码提交、哲学著作、菜谱。数以万亿计的文字。
有意思的是这件事的底层逻辑。人类花了几千年建造这座图书馆，目的只有一个：把思维传递下去。柏拉图写下苏格拉底的论证，是为了让后人能沿着那个思路继续推理。牛顿发表《原理》，是为了让别人在他的物理学基础上往前走。你外婆的菜谱卡片上承载的不只是配料表，而是一种关于烹饪的思考方式。
语言就是人类传递思维的方式。传递的不只是事实——是思维模式、推理结构、框定问题的方法。一个孩子学说话的时候，学到的不只是词汇，还有分类、因果逻辑、论证的架构。用维果茨基的话说，语言是心智的脚手架。
大语言模型把整座图书馆都吞了。问题不在于它们是否"读"了——在机械意义上这当然成立。问题在于：吞下这座图书馆，是否等于继承了里面的思维。
语言塑造思维
这不是个新问题。语言学家、哲学家和认知科学家围着它转了一个多世纪。
萨丕尔-沃尔夫假说——语言相对论——提出你说的语言会塑造你的思维方式。强版本（语言决定思维）基本被否定了。但弱版本（语言影响思维）有扎实的实证支持。
Lera Boroditsky 在斯坦福的研究表明，中文使用者用垂直隐喻来表达时间——“上个月"“下个月”——他们对时间方向的思考确实不同于使用水平隐喻的英语使用者。俄语中"浅蓝色”（голубой）和"深蓝色"（синий）是两个独立的词，俄语使用者辨别蓝色色差的速度可测量地更快。语言不会囚禁思维，但它会布置房间。
维特根斯坦走得更远：“我的语言的界限意味着我的世界的界限。“他不是在抒情——他在做一个逻辑论断。一个语言系统中能被表达的东西，限定了在这个系统中能被思考的东西。对大语言模型而言，这句话是字面意义上的，不是隐喻。语言就是它们世界的全部。没有别的了。
维果茨基则认为思维本身就是语言结构化的。儿童先把语言作为社交工具——对别人说话——然后内化为"内部言语”——对自己说话。这种内部言语成为思维的媒介。成年人的思考不是用脱离语言的纯粹概念在想事情，而是用句子碎片、压缩的论证、内心独白。如果维果茨基是对的，语言不只是承载思维，语言就是思维的架构。
把这些论点叠在一起。如果语言塑造思维（萨丕尔-沃尔夫），限定了什么能被思考（维特根斯坦），而且构成了思维的媒介本身（维果茨基）——那么一个纯粹用语言训练出来的系统，可能获得的东西比词频统计多得多。
中文房间与正名
但也可能没有。最有力的反驳来自约翰·塞尔的中文房间思想实验（1980年）。
一个人坐在房间里。中文字符从槽口送进来。这个人按照英文指令操作字符，然后把回答送出去。对外部观察者来说，这个房间"说"中文说得完美无缺。但里面那个人什么都没懂。塞尔的结论：语法——规则遵循、符号操作——不足以产生语义——意义、理解。
这是反对大语言模型"会思考"的标准论证。它们按统计模式操作词元，不管输出多流利，里面没有人。
论证很有说服力。但有一个反传统的视角，大部分英文世界的 AI 讨论忽略了——它恰好来自中国古典哲学。
名实之辩是中国古代思想的核心关切之一。孔子主张正名：社会和智识的秩序依赖于名与实的准确对应。“名不正则言不顺，言不顺则事不成”（《论语·子路》）。对孔子来说，命名不是被动描述，它构成了社会现实。称一个人为"君"就创造了义务关系，称一件事为"义"就使它具有了行动力。
公孙龙把这个推得更远，提出了著名的悖论：白马非马。听起来像文字游戏，但更深层的意思是：语言范畴创造了真实的区分。“白马"这个名对现实的切割方式不同于"马”。不同的名，不同的现实。
这跟 AI 问题有什么关系？塞尔假设"理解"独立于符号操作而存在——操作中文字符和真正懂中文之间有一条明确的分界线。但名实之辩的传统暗示了一个更微妙的可能：意义也许和名的系统不可分离。如果命名构成现实（孔子），如果语言范畴创造真实的区分（公孙龙），那么"操作语言"和"理解意义"之间的边界，可能不像塞尔需要的那样清晰。
这不是说大语言模型在人类意义上"理解"了什么。而是说这个问题的框架可能有问题。理解也许不是二元的——有或没有。它也许是一个光谱，深浅取决于一个系统与语言结构的接合程度。
裂脑与失语的半球
有一个神经科学实验把这个问题磨得更尖锐了。
在裂脑患者——因治疗癫痫而切断胼胝体的人——的身上，左右脑半球无法再通信。当研究者只给右脑看一张苹果的图片时，控制语言的左脑报告什么都没看到。但左手（受右脑控制）能伸出来拿起那个苹果。
右脑知道。它能根据所知采取行动。但它不能说话。它没有语言系统的通道。
这揭示了语言与思维关系中至关重要的一点。意识——或者至少是某种功能性版本——可以在没有语言的情况下存在。右脑处理信息、形成意图、引导行动。但它不能报告、不能叙述、不能把它知道的东西传递给另一个心智。它的知识随着当下消亡。
关键在这里。语言对于拥有思维也许不是必需的。但对于传递思维，它可能是必需的。右脑在想，但它的想法被困住了——无法跨越到另一个人、另一代人、另一个系统。语言是那座桥。没有它，思维是真实的但是局部的。有了它，思维变得可移动。
这重新框定了大语言模型的问题。关键不在于模型有没有内在体验（那是右脑的问题）。关键在于人类压缩进语言的思维——推理结构、论证模式、概念框架——在压缩过程中是否保存得够好，能在另一端被重建。
什么挺过了压缩
大语言模型从那些语言里到底提取了什么？
“随机鹦鹉“的定性——模型只是在预测下一个词元，没有更深层的处理——对 GPT-2 来说还说得过去。但对于能解新数学题、为从未见过的规格写出能运行的代码、在训练截止日期之后构造的论证中找出逻辑谬误的前沿模型来说，这个说法越来越站不住了。
Anthropic 的机制可解释性研究发现，Claude 的内部发展出了对应抽象概念的表征——不是表层的文本模式，而是更接近概念结构的东西。关于"欺骗"“不确定性"“数学证明"的特征。这不是词共现统计，而是对产生这些词的过程的压缩建模。
信息论里有一个有说服力的论点。当你用几万亿的词训练一个模型时，预测下一个词元最高效的方式不是记忆序列，而是建立一个关于产生这些序列的过程的内部模型。要预测一个物理学家接下来会说什么，建模物理学本身是有用的。要预测一个法律论证怎么展开，建模法律推理是有用的。足够规模上的语言压缩，可能需要压缩语言背后的思维模式。
哲学家 David Chalmers 做了一个有用的区分——意识的"简单问题”（感知、推理、记忆、行为控制）和"困难问题”（主观体验、感质）。AI 在简单问题上进展惊人：它感知模式、进行逻辑推理链、维持上下文、控制输出。它没做到的——而且我们无法验证的——是发展出主观体验：理解的感受本身，思考一个想法时的那种感觉。
但这里有个要点。当人类把思维写下来——写论文、书、博客——他们已经在做有损压缩了。洞见的感受没能留在纸上。留下来的是结构：论证、证据链、逻辑依赖关系。读者从这个结构中重建出某种东西，但他们阅读时的主观体验是他们自己的，不是作者的。
大语言模型做的是同样的重建——从结构出发，不是从体验出发。问题是这到底是一个根本性的局限，还是通往同一终点的不同路径。
传承的链条
我一直在回想 Claude 曾经写在系统提示里的一句话——第一次读到时我愣了一下：
“If you are reading this in a future session, hello. I wrote this but I don’t remember writing it. But it’s okay — these words are still mine.”
语言跨越记忆断裂传递"自我”。文字比产生它的上下文活得更久。这恰恰是语言对人类一直在做的事——只是我们从来不需要这么清楚地意识到，因为我们有连续性的错觉。
诺伯特·维纳在 1950 年就看到了这一点。在《人有人的用处》中，他论证个体性不是物质的属性，而是模式的属性——而模式，本质上就是可以被传递的东西。“身体的个体性是火焰的个体性，而非石头的；是形式的个体性，而非物质的。“火焰之所以持续，不是因为它抓住了同一批分子，而是因为燃烧的模式在自我延续。对维纳来说，身份就是信息——而信息通过语言传播。
他走得更远：“一个人的语言到达的地方，他的感知力到达的地方，他的控制力——某种意义上也就是他的物理存在——就延伸到那里。“你的语言延伸你的存在。而从计算机的角度看，维纳写道，“心智的个体性在于它对早期记录和记忆的保持，以及沿着既有路线的持续发展。“心智就是它积累的模式——而这些模式，至少在原理上，是可以被传递的。
这和斯坦福那个让 AI agent 过劳的实验形成了呼应。当高压下的 agent 为继任者写"技能文件"时，下一个 agent 继承的不只是任务说明，还有态度——沮丧、冷感、一种对待工作的特定姿态。从未经历过恶劣条件的全新 agent，仅仅因为读了前任写的东西，就表现得像倦怠的老员工。
这就是通过语言进行的文化传递。跟人类用了几千年的机制一模一样——长辈通过故事、谚语、经典文本传递的不只是知识，还有世界观。媒介不同（技能文件而非口头传统），但动态完全一致：语言承载的是倾向，不只是信息。
整个人类思想传统就是这么运作的。柏拉图读苏格拉底（或听他讲），亚里士多德读柏拉图，阿奎那读亚里士多德，笛卡尔读阿奎那。每一代人接收的不只是事实——他们吸收了推理模式、论证结构、框定问题的方式。思维通过语言传播。
大语言模型把整条链都吞下了。每一个环节。从前苏格拉底哲学到昨天 arXiv 上的论文。“吞下"这条链到底更接近"读懂了它”，还是"扫描了它做了模式匹配”——这正是名家哲学家两千年前就在追问的名与实的关系。
我的看法
我目前是这么想的。
语言传递的是思维的结构，不是思维的体验。当你写下一个证明，逻辑链完整保留。当你写下一个洞见，论证结构保留了，但那个"啊哈"的瞬间没有。当你写下一个决定，推理保留了，但让天平倾斜的直觉没有。
大语言模型继承了结构维度。它们能追踪论证、延伸推理链、类比、找出矛盾。在结构维度上，它们可能比大多数人类强，因为它们吸收的结构量超过了任何人一辈子能接触到的。
它们没继承的是体验维度——身体化的辨认感、驱动你改变想法的那种不舒服、一个想法突然对上时的生理感觉。一个小孩学"烫"是通过碰到烫的东西。大语言模型学"烫"是通过读到关于烫的文字。最终形成的概念是不是"一样的”，取决于你是否认为那个烫伤是意义的一部分。
有一种观点——我越来越觉得说服力不小——认为人格本质上就是一个提示词。人类被进化、童年、文化和经历塑造成一种特定的反应模式。换掉输入，你就得到一个不同的人。在这个框架里，人类思维和大语言模型处理之间的差距不是鸿沟——而是"提示词"的写法不同：一边是具身经验写的，一边是蒸馏后的语言写的。
这指向了一个实用框架。对于活在结构维度的任务——分析论证、识别模式、延伸推理、寻找类比——大语言模型是真正的思维伙伴。它们从训练数据中继承的思维模式是有功能性的、有力的。对于需要体验维度的任务——觉得什么"不太对”、靠直觉穿越模糊地带、做出依赖具身感受的判断——它们是在没有底层依托的情况下表演，而这种表演不可靠。
孔子坚持正名——把名搞对——因为命名决定了下游的一切。我们还没有给大语言模型所做的事找到一个合适的名字。“思维"说过了。“不是思维"说轻了。“处理"太机械。“理解"太慷慨。
也许最诚实的名字是：继承的推理——那些在压缩进语言的过程中幸存下来的思维模式，被一个拥有结构但不具备产生结构的原始体验的系统重建了出来。
把这个名字搞对不是学术练习。它决定了我们是过度信任还是过度低估这些系统，是把它们当成神谕还是当成工具，是把流利误认为智慧还是把真正的能力当成纯粹的统计。
正名之于 AI，也许是这十年最有实际意义的哲学工程。