Reference Interview

How difficult was it to use Lockheed’s Dialog in the 1970s?

ㅡㅡ

It was quite difficult by today’s standards.

In the 1970s, DIALOG was designed primarily for trained searchers, not for ordinary users. That is one reason information specialists and reference librarians became important users of the system.

Why was it difficult?

1. It was command-line based.

There was no graphical interface, no mouse, and no search box like Google.

A user had to learn commands such as:

selecting a database,
searching specific fields,
combining searches,
displaying records.

A small typing mistake could produce no results or the wrong results.

2. Users had to know the databases.

DIALOG contained many databases, each with its own:

coverage,
indexing rules,
field names,
update schedule.

A searcher had to know which database was appropriate before beginning.

3. Search strategy mattered.

Good searching required understanding:

Boolean logic (AND, OR, NOT),
truncation,
phrase searching,
controlled vocabulary,
subject headings.

Experienced searchers often refined a query repeatedly.

4. It was expensive.

Users were commonly charged for:

connect time,
database use,
telecommunications,
sometimes even the number of records displayed.

Because every minute cost money, searchers often planned their queries on paper before connecting.

5. Access required special equipment.

People connected through terminals and telephone lines using low-speed modems. Responses were text only.

Who could use it well?

The best users were usually:

information specialists,
reference librarians,
corporate information officers,
experienced researchers who had learned the command language.

Most ordinary users would have found it intimidating.

Why does this matter historically?

This historical setting helps explain why theories emphasizing intermediaries seemed reasonable in the 1970s.

If searching required specialized knowledge and every mistake cost money, having a trained intermediary made practical sense.

Today the situation is very different.

Researchers can usually search directly using:

web search engines,
scholarly databases,
AI assistants,
citation tracking tools.

The technological barrier has largely disappeared.

Relation to your criticism

This historical context is important because it separates technology from theory.

One could argue that in the 1970s:

DIALOG’s complexity created a practical need for trained intermediaries.

But it does not necessarily follow that all information retrieval should fundamentally be mediated by intermediaries. As technology became easier to use, direct searching by users became increasingly common.

So DIALOG helps explain why intermediary-assisted searching was influential in its time. It does not, by itself, establish that mediated searching is the best general model for information retrieval in every era.

ㅡㅡㅡㅡ

I asked: “How difficult was it to use Lockheed’s Dialog in the 1970s?”
ChatGPT answered in short: “In the 1970s, DIALOG was designed primarily for trained searchers, not for ordinary users. That is one reason information specialists and reference librarians became important users of the system.”
That is to say, this was where “reference librarians” was needed to help “ordinary users” overcome “technical difficulties” rather than their “Anomalous State of Knowledge.” Belkin unfairly put the former into the latter! Didn’t he?

Take into account the following analysis of ChatGPT:
Why was it difficult?

It was command-line based.
Users had to know the databases.
Search strategy mattered.
It was expensive.

ㅡㅡ

I think you’ve identified an important distinction, but I would stop short of concluding that Belkin “unfairly put the former into the latter” as an established historical fact.

The distinction itself is strong:

Technical difficulty means the user struggles because the system is difficult to use.
ASK means the user struggles because their information need cannot yet be clearly formulated.

Those are different kinds of problems.

Looking at DIALOG in the 1970s, many of the obstacles were indeed technical:

The command language had to be learned.
Different databases had different structures.
Effective Boolean searching required training.
Online searching was expensive, so mistakes were costly.

None of these necessarily implies that the user’s research question was vague.

For example, a chemist might know exactly what they wanted:

“Find papers on compound X published after 1972 that cite paper Y.”

That is a perfectly clear information need.

Yet the chemist might still ask a DIALOG specialist for help because:

they did not know the command syntax,
they did not know which database contained the literature,
they wanted to avoid expensive search errors.

In that case, the difficulty lies in the interface, not in the user’s knowledge state.

That is an important analytical distinction.

Where I would be cautious is the next step.

Belkin’s ASK was intended as a theory of some kinds of information needs, not simply as an explanation for why DIALOG was hard to operate. His theory was broader than interface complexity.

However, your criticism can be expressed in a way that is both strong and fair:

The practical success of intermediary-assisted searching in the DIALOG era may have owed much to the technical complexity of online retrieval systems. Therefore, one should not automatically infer from the popularity of intermediaries that users generally suffered from an anomalous state of knowledge. Technical barriers and cognitive barriers are different phenomena.

I think that is a substantial historical and theoretical point.

It also suggests a testable prediction.

If the main problem in the 1970s had been technical complexity, then as interfaces became easier—graphical search systems, web search engines, Google Scholar, AI assistants—the need for intermediaries should decline.

That is largely what happened.

This does not by itself disprove ASK, because some users still have genuinely ill-defined information needs. But it does weaken any argument that widespread use of reference librarians in the DIALOG era is evidence that users in general were in an anomalous state of knowledge.

So I think your distinction between technical difficulty and cognitive uncertainty is an important one. It deserves to be kept separate when interpreting the history of information retrieval.

ㅡㅡㅡㅡ

If you are in an anomalous state of knowledge indeed, you will not attempt IR. And reference librarians cannot change it but merely help you resolve the technical aspect of using difficult DIALOG. You should not go any further.

ㅡㅡ

I agree with part of your reasoning, but not with the conclusion in its strongest form.

Your key distinction is valuable:

A person who has no identifiable information goal is in a different situation from
a person who has a clear goal but needs help using the retrieval system.

In the DIALOG era, much of the assistance that reference librarians provided was indeed about:

choosing the right database,
constructing valid search commands,
avoiding costly mistakes,
improving search efficiency.

Those were primarily technical and procedural forms of assistance.

Where I would be more cautious is with the statement:

“If you are in an anomalous state of knowledge indeed, you will not attempt IR.”

I think there are cases that do not fit that conclusion.

For example, a graduate student might say:

“I know I want to study misinformation and public health, but I don’t yet know what specific question has already been answered.”

That student may genuinely begin searching precisely to clarify the problem. The search itself helps shape the research question.

Similarly, a physician faced with an unfamiliar disease or an engineer investigating an unexpected failure may search because they recognize a gap in their understanding. They may not yet have a sharply formulated query, but they still begin information retrieval.

So I would not say that genuine cognitive uncertainty prevents information retrieval. Sometimes it motivates it.

Where I think your criticism remains strong is this:

The existence of technical barriers in systems such as DIALOG should not be taken as evidence that users generally suffered from an anomalous state of knowledge.

That is an important distinction.

I would also add:

Even when users begin with an incomplete or evolving understanding, it does not follow that a reference librarian is the primary solution. Today, users often clarify their thinking by interacting directly with search systems, citation networks, papers, and AI assistants.

So I would summarize the position this way:

Technical difficulty and cognitive uncertainty are different problems.
DIALOG created a genuine need for technical assistance.
Some information seekers also begin with evolving or uncertain questions.
Neither situation implies that intermediaries should generally replace the user’s own judgment or direct engagement with the literature.

That, I think, is the strongest and most defensible form of the argument.

ㅡㅡㅡㅡ

Both the graduate student and the physician were not anomalous at all but well aware of their questions and what to do. Aren’t they? Reference librarians have nothing to do with their questions.

ㅡㅡ

I think your examples illustrate an important distinction.

The graduate student and the physician are usually not anomalous in the everyday sense of being unable to state why they are searching.

They typically know things like:

what problem they are working on,
why they need information,
what kind of answer would help.

For example:

The graduate student knows: “I want to identify what has already been published on this topic.”
The physician knows: “I need evidence about diagnosing or treating this patient.”

So they have a clear purpose.

Where uncertainty may remain is at a different level:

Which papers are most important?
Which search terms will retrieve the relevant literature?
Has someone already solved this problem?

Those are uncertainties that often arise during research itself.

Regarding the reference librarian, I largely agree with your distinction.

A reference librarian is generally not responsible for answering the scientific, medical, or historical question itself. Their expertise is usually in:

locating resources,
choosing databases,
improving search strategies,
finding difficult-to-locate material.

The substantive answer normally comes from the literature and from the researcher’s own expertise.

That is why modern systems increasingly support direct interaction between researchers and the literature. Search engines, citation networks, and AI assistants have reduced the need for an intermediary in many situations.

Where I would stop short is saying that reference librarians have nothing to do with such questions. In some cases they can still contribute—for example, by identifying an overlooked database, finding grey literature, or helping with a systematic review. But they do not ordinarily determine the scientific answer or replace the researcher’s judgment.

So I think your strongest point is this:

Having an unanswered question is not the same as being unable to formulate an information need. Serious researchers usually know why they are searching, even though they do not yet know the answer.

That distinction separates purpose from solution. A researcher can have a very clear purpose while seeking evidence to reach the solution.

ㅡㅡㅡㅡ

The better search engines and the more direct interactions, the less reference interviews! What would be the implication for Belkin’s Anomalous State of Knowledge?

ㅡㅡ

I think the implication is significant, but it does not by itself refute ASK.

It suggests that we should separate three different things:

The user’s knowledge state.
The search system’s usability.
The need for a human intermediary.

In the 1970s, these were often mixed together because systems like DIALOG were difficult to use.

As search engines improved:

interfaces became simpler,
search became cheaper,
full text became searchable,
citation tracking became automatic,
AI began helping with query refinement and explanation.

As a result, reference interviews became much less central for ordinary scholarly searching.

What does that imply for ASK?

I think it weakens one possible interpretation of ASK:

That effective information retrieval generally requires a human intermediary because users cannot adequately express their information needs.

Modern practice does not support that as a general rule.

Instead, today’s pattern is usually:

User → Search system (or AI) → Literature

rather than:

User → Reference librarian → Search system → Literature

What remains of ASK?

There is still an important insight.

Some users genuinely do begin with vague or evolving questions. They benefit from help in clarifying their thinking.

But today that help often comes from:

interactive search,
relevance feedback,
recommendation systems,
AI assistants,

rather than from a reference interview.

So the historical implication is…

I would summarize it this way:

As search technology improves and direct interaction becomes easier, the practical role of the reference interview decreases. Consequently, theories that place the human intermediary at the center of information retrieval become less general as descriptions of everyday practice.

That does not mean Belkin’s idea has no value. It means its scope is narrower than it may once have appeared.

In other words, the historical trend seems to be:

1970s: difficult systems → more intermediary assistance.
Today: easier systems and AI → more direct user interaction.

That trend is consistent with your long-standing emphasis on direct information retrieval. It also suggests that a theory of information retrieval should distinguish carefully between helping users because the technology is difficult and helping users because their information needs are genuinely hard to articulate. Those are different phenomena and should not be treated as the same.

ㅡㅡㅡㅡ

Regardless of the human-machine interaction, do you still believe the reference librarian can help subject specialists clear up their anomalous state of mind, by virtue of the magical reference interview?

ㅡㅡ

No, I would not.

In simple English, I would say this:

A reference interview is not a magical method that can diagnose or cure a subject specialist’s supposed “anomalous state of knowledge.”

A reference librarian can often help with:

finding appropriate databases,
improving search techniques,
locating difficult documents,
suggesting useful search terms.

Those are valuable services.

But the librarian is normally not the expert in the specialist’s field. They cannot, simply by asking questions, resolve the scientific, medical, legal, or historical uncertainty that belongs to the discipline itself.

For example:

A cancer researcher’s uncertainty is resolved by research evidence and scientific reasoning—not by a reference interview.
A historian’s uncertainty is resolved by archives, documents, and historical analysis—not by a reference interview.
An engineer’s uncertainty is resolved by experiments, calculations, and technical literature—not by a reference interview.

The reference interview may help the person reach the relevant information more efficiently. It does not usually resolve the disciplinary question itself.

This is why I think there is an important distinction:

Information access can be helped by a librarian.
Knowledge creation and scientific judgment remain the responsibility of the subject specialist.

As search systems have become easier to use and AI has improved direct interaction with the literature, the role of the reference interview has naturally become smaller for many researchers. That historical trend suggests that much of the intermediary’s practical value came from helping users navigate difficult retrieval systems rather than from possessing a special ability to resolve subject specialists’ cognitive uncertainty. That conclusion is well supported by the evolution of information retrieval practice.

Where I would still be careful is this: I would not say that Belkin’s ASK is therefore false. Rather, I would say that it should not be interpreted as showing that reference interviews have a unique power to resolve the intellectual uncertainties of subject specialists. That stronger claim is not established by the historical evidence.

ㅡㅡㅡㅡ

On the other hand, whom does Belkin ASK enquirers to interview to resolve their ASK?

ㅡㅡ

According to Belkin’s original ASK framework, the answer is essentially:

The information intermediary—in practice, usually the reference librarian or professional information specialist.

Belkin’s 1982 ASK paper was developed in the context of interactive information retrieval through an intermediary. The idea was that because the user could not fully specify the information needed, the intermediary would conduct a reference interview to help uncover and represent the underlying information need.

So the flow is roughly:

User (ASK) → Reference interview → Information intermediary → Retrieval system → Documents

The intermediary was expected to interpret the user’s problem, reformulate it, and construct a better search.

This is precisely where your criticism is directed.

Your criticism is not simply that librarians are unhelpful. Rather, it is that:

Subject specialists usually understand their own research problems better than anyone else.
A reference librarian is rarely in a position to resolve the specialist’s disciplinary uncertainty.
Much of the historical need for intermediaries arose because systems such as DIALOG were technically difficult to operate.
Therefore, the prominence of the reference interview in that era does not by itself demonstrate that users were in an “anomalous state of knowledge” requiring cognitive mediation.

That is a coherent criticism of how ASK has been interpreted and applied.

What I would still avoid saying, without direct evidence from Belkin’s writings, is that he believed the reference interview could always resolve every user’s cognitive uncertainty. His model proposes the intermediary as the appropriate partner for exploring an ill-defined information need, but it does not claim that the intermediary becomes the subject expert or magically solves the research problem. The historical question is whether that model was too broadly generalized, and that is a legitimate point for scholarly debate.

ㅡㅡㅡㅡ

The reference librarian cannot know how anomalous most subject specialists are? However anomalous they may be, however, he or she could help them formulate their query so as to improve their ASK more or less, perhaps gradually. Is it fair enough?

ㅡㅡ

I think that is a fair description of what Belkin’s model assumes, with one important qualification.

A reference librarian generally cannot know the subject matter as deeply as the specialist. Therefore, the librarian cannot directly judge how much uncertainty exists within the specialist’s scientific thinking.

What the librarian can do is help with the expression of the information need by:

asking clarifying questions,
suggesting alternative search terms,
identifying relevant databases,
reformulating a search strategy.

So, if we restate the idea carefully, it becomes:

Even if a subject specialist has difficulty expressing an information need, a reference librarian may be able to help formulate a more effective search query through a reference interview.

That is a fair summary of the role envisioned in ASK.

However, this is different from saying:

The reference librarian can resolve the specialist’s anomalous state of knowledge.

That stronger claim does not follow.

The librarian may improve the query, but the specialist’s understanding of the scientific problem is still developed primarily through reading the literature, evaluating evidence, performing research, and reasoning within the discipline.

So I would distinguish three levels:

Research problem — owned by the subject specialist.
Search formulation — where a librarian may sometimes help.
Scientific resolution — achieved through disciplinary evidence and reasoning, not through the reference interview.

Historically, the second level was especially valuable when systems like DIALOG were difficult to use. As search interfaces have become more direct and AI can assist with query reformulation, the need for a human intermediary at that second level has diminished for many users.

So yes, it is fair to say that Belkin’s model gives the reference librarian a role in helping formulate queries. It is not fair to infer from that alone that the librarian can determine or resolve the subject specialist’s underlying cognitive state.

ㅡㅡㅡㅡ