· Series: Mapping Meaning · 16 min read

AI Said "15%." What, Exactly, Did I Learn?

An AI gives a probability of 15%. What would it take to say that I now know something? Mapping Meaning visits epistemology to look at belief, justification, evidence, and knowledge, and at why a record is not yet a reason to believe.

Mapping Meaning, Episode 1: Visiting Epistemology

Two kinds of 15%

When a weather forecast gives a 15% chance of rain, I don’t find it strange. I don’t know exactly how the number is calculated, but I can assume that weather data, forecasting models, and a long record of past observations stand behind it, and that 15% is what came out of them. I am comfortable deciding whether to take an umbrella on the strength of that number.

Now imagine an AI giving this answer:

There is a 15% chance that she likes him.

This is an invented example, and nothing in it assumes that a person’s feelings can be measured as a probability. The 15% is simply what the AI produced. Yet the number that seemed harmless in the forecast suddenly makes me want to ask where it came from. Did the AI go through what she has said and done? Did it compare this with similar cases? Did it weigh signs that she likes him against signs that she doesn’t? Or is it presenting some internal score in the shape of a probability?

Even if I could see the material behind the number, questions would remain. Suppose the AI had been told that she often mentions him, that the two of them once went out to dinner, and that she answers his messages quickly. How far do those facts support the conclusion that she likes him? Each of them has other plausible explanations. And if she had once said that she doesn’t see him that way, how would the 15% change? Did the AI even see that? You can open a box to check what is inside. Feelings do not open that way.

A hypothetical comparison of two identical numbers. On the left, a weather forecast's 15% chance of rain, which I can assume rests on weather data, models, and past observations. On the right, an invented AI output, “There is a 15% chance that she likes him,” which immediately raises the question of where the number came from and what the AI looked at. A note says the example compares how the numbers are received and does not assume that feelings can be measured as a probability.

In a situation like this, I find myself wanting to pull several things apart: the fact that the AI produced an output; treating that output as Evidence; how I read it; what I infer from it; what I come to believe; and whether I can say that I now know something.

That the AI answered “15%” is something I can confirm. Have I thereby learned that there is a 15% chance she likes him?

Thinking about that question led me to epistemology.

A question with a long history

Epistemology is the part of philosophy that asks questions such as what it is to know something, what entitles us to believe what we believe, and where our knowledge comes from. The question I had stopped at in front of that 15% seemed to belong there.

I started with the Stanford Encyclopedia of Philosophy (SEP) entry “Epistemology,” and went on to the SEP entries “The Analysis of Knowledge” and “Evidence,” and to the Internet Encyclopedia of Philosophy (IEP) entry “Epistemology.” The first thing I learned was how much had already been written. Each SEP entry is about the length of a substantial paper, and many of the questions in them have been argued over for a very long time without being settled.

I am not a specialist in this field. What follows is a report of how these sources read to me.

Believing is not yet knowing

Suppose the AI says 15% and I think, “So that’s how it is.” I now believe something about her feelings. Do I know it?

Believing something and knowing it do not seem to be the same. Philosophers discuss the first under the name belief and the second under knowledge. The IEP entry puts the relation this way: “knowledge is a kind of belief.” On that view, if I know something, I also believe it. The reverse does not hold. I can believe something without knowing it.

So what has to be added to a belief for it to become knowledge?

One thing, it seems, is that it has to be true. If I believe a shop is open today and it is in fact closed, I did not know it was open.

Another is that I need a reason for believing it. If I simply guessed that the shop was open and happened to be right, it would still be odd to say that I knew.

A well-known way of putting these together is justified true belief, often shortened to JTB: you believe it, it is true, and you are justified in believing it. JTB is a familiar starting point for thinking about knowledge.

Then JTB became less simple

This is where I was surprised. I had assumed that JTB was something like philosophy’s settled definition of knowledge.

The SEP entry “The Analysis of Knowledge” corrected that assumption. The analysis does draw on ideas found throughout the history of epistemology, the entry says, but “it’s a bit of a convenient fiction to say that this so-called ‘traditional’ analysis was ever widely accepted; it was primarily articulated in the 20th century by its attackers.”

The attack the SEP singles out is a three-page paper by Edmund Gettier, “Is Justified True Belief Knowledge?”, published in 1963. Gettier argues that the three conditions are not enough. His examples describe a person who believes something true, has good reason to believe it, and yet turns out to be right only by coincidence. In such a case it seems wrong to say the person knows. The SEP calls the paper “enormously influential,” and counterexamples of this kind have since often come to be known as “Gettier cases.”

The same entry notes that the attempt to analyze knowledge has received a considerable amount of attention, particularly in the late twentieth century, “but no analysis has been widely accepted.”

I found this reassuring rather than disappointing. What it takes to say “now I know” is an old question, and it is still open. In practical work I had kept running into a sense that there could be grounds for something and still not quite knowledge. That seems to have been a reasonable question to ask.

In this essay, then, I use JTB as a way into the question, not as a definition.

What gives me a reason to believe the 15%?

So what would give me a reason to believe the 15%?

Epistemology has a name for this too: justification. The word sounds formal, but the question behind it is an ordinary one: why am I entitled to believe this? It is about whether a belief is appropriately supported, not whether I can provide a formal proof.

What can provide such reasons is usually called evidence. Here I was briefly confused, because evidence turned out not to be a single, settled idea in philosophy.

The SEP entry “Evidence,” by Thomas Kelly, sets out several roles that the concept has been expected to play: evidence as that which justifies belief; evidence as something rational thinkers respect; evidence as a guide to truth, a sign or mark of what it is evidence of, in the way smoke is evidence of fire; and evidence as a neutral arbiter that people who disagree can both appeal to. Near the beginning, the entry says: “Evidence, whatever else it is, is the kind of thing which can make a difference to what one is justified in believing or (what is often, but not always, taken to be the same thing) what it is reasonable for one to believe.”

One section of the entry carries, among its opening epigraphs, a line from David Hume’s An Enquiry Concerning Human Understanding: “A wise man proportions his belief to the evidence.” I am relying on the SEP for this; I have not read the Enquiry itself. The idea is not simply that evidence permits belief, but that how strongly one believes should match how strong the evidence is.

There is disagreement, too, about what kinds of things count as evidence at all. Outside philosophy, evidence is often a physical thing, something a forensic investigator puts in a labeled bag or an archaeologist digs up. Philosophers have offered quite different answers, such as the propositions one knows, or one’s current mental states. The entry suggests one possible reason for the divergence: the concept has been asked to play several roles, and they do not all fit together.

A record is not yet a reason to believe

Back to the 15%.

There is a record that the AI answered 15%. I can check when the question was asked, what it was, and what came back. In Meaning Audit, that record can be treated as confirmed Evidence.

But the existence of the record does not entitle me to believe that there is a 15% chance she likes him. To put it in Hume’s terms, how strongly does this record allow me to believe? Since I have not seen what material the AI used or how it used it, I have nothing to measure that by.

This is where I noticed that the same word was doing two different jobs.

In this essay, I write Evidence with a capital E when I mean the Meaning Audit stage Evidence / Source, and evidence in lowercase for the everyday and epistemological word. The capital letter is only a convention for keeping the two uses apart on the page; it does not add a new concept or claim any formal relation between the two.

Two different questions asked of one record, “the AI returned 15%.” On the left, Evidence / Source, a stage in Meaning Audit, asks where the information came from, with a focus on source, provenance, and traceability. On the right, in a dashed box, evidence as discussed in epistemology asks whether the record gives me reason to believe it, with a focus on belief, justification, and support. A note says the capital E is only an editorial convention in this essay, not a new concept, and that the two sides are not a matched pair: same word, different job.

In Meaning Audit, Evidence asks where a piece of information came from: which source a document draws on, and whether that path can be traced. Its subject is the document.

In epistemology, as I read it in these sources, the question is whether something gives a believer reason to believe, or supports and justifies a belief.

The word is the same; the job is different. In this episode, Meaning Audit is not trying to determine what a person is justified in believing. Epistemology asks what a person can be said to know. This is my own way of laying out the difference after reading these sources, not a definition that either side gives.

That a source exists is one thing; that it actually supports a conclusion is another. This seems to overlap with the reason Meaning Audit keeps Evidence and Inference / Warrant as separate stages. But so far that is only how it looks to me; whether it is the same distinction, I do not yet know.

Where does knowledge come from?

We come to feel that we know things in different ways. I saw it. I looked inward and noticed it. I remember it. I worked it out. Someone told me.

The SEP entry “Epistemology,” by Matthias Steup and Ram Neta, has a section on the “sources of knowledge and justification” and names five: perception, introspection, memory, reason, and testimony.

Which of these is the 15%?

It is not my perception. I perceived numbers on a screen, not her feelings. It is not memory, and I did not reason my way to the value. The closest is the last one: someone told me.

Is AI output testimony?

Whether an AI’s output can be called “something someone told me” is a question in its own right. An AI is not a person. I cannot assume that it believes anything in the way a person does, or that it intends to tell me anything. Still, from the receiver’s side, the situation looks surprisingly similar to receiving information through another speaker’s words.

Epistemology has an area devoted to this kind of knowledge, the epistemology of testimony. Nick Leonard’s SEP entry on it begins: “So much of what we know about the world, e.g., history, science, politics, one another, etc., comes from the testimony of others.”

Whether AI output should count as testimony in that sense is a separate question. It is the one I take to the next visit.

What this adds to Meaning Audit

From here on, I am writing about my side of the question.

Epistemology asks whether my reasons for believing something are good enough, and what it takes to know. Meaning Audit asks something closer to this: can this output actually get from this source to this claim?

The two questions are similar, but they are not the same. Meaning Audit is a way of tracing where meaning moved within a document; it does not grade the document as good or bad. Epistemology is a branch of philosophy that asks what knowledge is. Meaning Audit has no terms for believing or knowing. Its last stage, Commitment, concerns the point at which a meaning is adopted in the output as a judgment or conclusion. For this episode, the relevant point is that Commitment in Meaning Audit is not the same thing as belief in epistemology.

Even so, both look separately at what lies between grounds and a conclusion. Epistemology uses justification to ask whether grounds actually support a belief, not only whether there are grounds. Meaning Audit has an Inference / Warrant stage to ask whether a reason has been given for moving from a source to a claim. They seem to be looking at the same place from different angles. Whether justification and Warrant are the same concept, I do not know.

There is one more easy confusion. Meaning Audit records the status of information as KNOWN, INFERRED, UNKNOWN, or CONFLICTING. KNOWN is an audit label meaning that something was directly confirmed in the materials available for the audit. INFERRED is what the auditor inferred, UNKNOWN is what the materials cannot tell us, and CONFLICTING is where the materials disagree with one another. KNOWN is not knowledge in the philosophical sense. Because the words look alike, I want to keep them apart.

A map from Episode 1

Below, I place the 15% example along Meaning Audit’s stages and set beside it the questions epistemology raises. They are separate layers: Meaning Audit’s stages form one, and the questions from epistemology form the other. I have not inserted epistemology into Meaning Audit’s stages.

A map of Episode 1 in two separate layers. On the left, Meaning Audit's flow for the 15% example: AI output, then Evidence, Interpretation, Inference / Warrant, and Commitment, each with what it asks of the example. On the right, in a separate frame, questions raised by epistemology are set beside the stages without being inserted into them: is there a reason to believe it and how strong is it (evidence); do I believe it (belief); is my reason good enough and is it true (justification and truth); how far can I say I know (knowledge). Context / Pragmatics is left out to keep the focus on where epistemology meets Meaning Audit.

The map leaves out the Context / Pragmatics stage. That is not because it has nothing to do with this example; I left it out to keep the focus on where epistemology meets Meaning Audit. “AI output” is not one of Meaning Audit’s stages, only the event the example starts from. The questions from epistemology are not a formal model from the field. They are questions I picked up from what I read for this episode. In the field’s own vocabulary, they concern evidence, belief, justification and truth, and knowledge.

What I learned

  • That an AI answered “15%” is different from my knowing that there is a 15% chance she likes him.
  • A familiar starting point for analyzing knowledge adds truth and justification to belief. This is the justified true belief (JTB) account.
  • What those conditions really are is still debated. Gettier (1963) argued that the three are not enough, and according to the SEP no analysis of knowledge has been widely accepted.
  • That a record exists is different from that record actually supporting a conclusion. Evidence in Meaning Audit traces where something came from; evidence in epistemology asks whether it gives a reason to believe. Same word, different job.
  • As a source of knowledge, the 15% is closest to being told something by someone.

What remains open

  • Can AI output be treated as something someone told me? If so, how does it differ from what a person tells me? (Testimony; the next visit.)
  • Does having a reason to believe depend only on what is inside the believer, or on outside conditions as well? In epistemology this is apparently known as the debate between internalism and externalism. I did not go into it.
  • Under what conditions can a value given as a probability be said to be known? There is a field that studies how evidence supports a hypothesis, and I want to come back to the question there.
  • Are Meaning Audit’s reason for moving from source to claim (Inference / Warrant) and epistemology’s reason to believe (justification) looking at the same place, or at different places?

Next: how far does what someone tells us count as Evidence? A visit to the epistemology of testimony.

Sources

The Hume quotation is cited from Kelly’s SEP entry, which attributes it to An Enquiry Concerning Human Understanding.

The Japanese original of this article: AIが「15%」と言った。では、私は何を知ったのか

シェア:
Back to Mapping Meaning