What we don't know we don't know
AI has read almost everything people have written. The bottleneck is no longer data. It is the question. We look for what we didn't know we didn't know, and ask.
1Question
Until now, AI has grown on data. It became smarter by reading the books, articles, and conversations people left behind, again and again.
But the record people have left has an end, and AI has already read most of it. The data of the past is running out.
So what will stand in AI's way next?
2Structure
We see what lies ahead of us at three depths.
- What we know.
- What we know we don't know. So we can ask about it.
- What we don't know we don't know. It has not even become a question yet.
To understand more deeply is to go down through these depths. We find what we didn't know we didn't know, and turn it into something we know we don't know. Then we find its answer, and turn it into something we know.
AI does the second part astonishingly fast. As long as we can ask, the answer comes quickly. Checking keeps getting faster and easier.
So the bottleneck moves: from data to questions. More precisely, to the questions that find what we didn't know we didn't know.
3Experiment
Our product Grid went through all three depths.
Given only a list of the text on a web page, AI got 87% right on ordinary pages. It is easy to call that good enough. We didn't even know what we didn't know.
Then we built pages where the order of the text and the place on screen disagree, and accuracy fell to 61%. A weakness nobody had measured came to light. What we didn't know we didn't know became something we knew we didn't know.
Once we knew, we could ask: "How do we tell AI about places that are not in the text?" Our answer was a seat map for the web, and all four models got 100%. What we didn't know became what we know.
The most important step was not the answer. It was building that page in the first place.
4Understanding
Answers keep getting faster. Questions keep getting more precious.
In a future where the question is the bottleneck, this is the question we must ask: What don't we know that we don't know?
This question has no end. Find one thing we didn't know we didn't know, and there is another beneath it. So we will not stop asking. We will turn each question into structure, test it by experiment, and rebuild our understanding.