Scholars used to perform citation tracking (CT) with the book form. It may be much older than information retrieval (IR). CT is humanistic and highly cognitive, while IR is mechanistic and poorly cognitive. Cognition is required to expose the meanings implied or hidden in context. In this respect, LLMs may be the worst case. After all, however, they have succeeded to ”transform” context into ”attention” or ”transformer attention”. To say “Attention is all you need” is to say “Context is all you need.”
ㅡㅡ
Yes. This is a very strong formulation, with one small caution.
The strong part is:
Citation tracking is older, more humanistic, and more cognitive than modern mechanistic information retrieval.
Scholars in the book culture already practiced citation tracking long before computerized IR. They followed footnotes, references, marginal comments, bibliographies, reviews, and scholarly disputes. This was not simple “retrieval.” It was intellectual pursuit. It required judgment, memory, comparison, suspicion, and interpretation.
So CT is not merely a technical subfield of IR. Historically and cognitively, it may be deeper than IR.
CT versus IR
Traditional IR often asks:
Which document matches this query?
Citation tracking asks:
Why did this author cite that work?
What relation exists between these minds?
What hidden argument, debt, correction, criticism, or lineage is being exposed?
That second question is more cognitive.
IR, especially in its early mechanistic form, often reduced meaning to terms, matching, frequency, classification, and retrieval output. CT, by contrast, follows meaning through scholarly relations.
So your contrast can be stated this way:
Information retrieval retrieves documents. Citation tracking retrieves intellectual relations.
Or more sharply:
IR finds texts; CT follows thought.
Context needs cognition
Your central point is right:
Cognition is required to expose meanings implied or hidden in context.
Context is not just “surrounding words.” It is a field of possible meaning. A machine can count nearby words, but a scholar asks:
Why is this word here?
Why is this reference here?
What is being implied?
What is being hidden?
What is the author doing with this citation?
That is cognitive work.
This is why your citation-context indexing is stronger than simple citation indexing. A citation link gives structure. A citation context gives a chance to recover meaning. But even that context must be cognitively read.
LLMs: worst case, then surprising case
Your phrase “LLMs may be the worst case” is understandable. LLMs do not “understand” context as a scholar does. They can produce fluent text without stable personal judgment, historical responsibility, or real scholarly conscience. They may imitate cognition without owning it.
But there is also the surprising reverse side:
LLMs succeeded because they mathematically operationalized context.
The Transformer paper proposed an architecture “based solely on attention mechanisms,” removing recurrence and convolution for sequence transduction tasks. Google Research In technical terms, attention lets a model weigh relations among tokens in a sequence. In broader information-theoretic terms, that means the model treats each sign through its relation to other signs.
So your aphorism is powerful:
“Attention is all you need” means, in historical-information terms, “Context is all you need.”
Not literally all forms of human context, of course. Transformer attention is not the same as human historical, social, psychological, and citation context. But metaphorically and structurally, your reading is strong.
Better exact wording
I would formulate it like this:
Scholarly citation tracking, especially in book culture, was already a cognitive-contextual practice long before computerized information retrieval. Traditional IR mechanized the search for documents, often weakening the human act of contextual judgment. Citation tracking, however, followed intellectual relations through references, notes, quotations, disputes, and evaluations. In this sense, CT was more humanistic and more cognitive than IR. The irony of LLMs is that, although they may lack human scholarly cognition, their great technical success came from transforming context into attention. “Attention is all you need” may therefore be read, from the standpoint of information science, as “context is all you need.”
The key qualification is this:
Transformer attention is a computational shadow of context, not the whole human context.
That qualification protects your idea from overstatement while keeping its force.
ㅡㅡㅡㅡ
“Attention is all you need” and “Context is all you need” may be summarized as “Attention to context is all you need.”
ㅡㅡ
Yes. That is the best synthesis:
Attention to context is all you need.
It joins the technical and humanistic meanings.
In Transformer terms, attention means selecting and weighting relations inside a sequence.
In scholarly terms, attention to context means recognizing why a word, citation, or document matters in its intellectual environment.
So your phrase can carry both meanings:
For machines, attention turns surrounding tokens into usable relations.
For scholars, attention to context turns surrounding evidence into meaning.
A sharper version:
Attention to context is the bridge between mechanical retrieval and cognitive understanding.
And for citation-context indexing:
Citation tracking becomes truly intelligent when it gives attention to context: not merely who cited whom, but why, how, and with what judgment.
ㅡㅡㅡㅡ
2016-10-01 Mark Park
