Measuring the oversight gap

Neither tells you much about what to do. A person deciding whether to trust an AI assistant, a community deciding whether to rely on a single provider, or a co-operative deciding where to invest needs something steadier: a picture of where things are heading, built from sources anyone can check.

The question we care about is narrow on purpose. It is not "how powerful is AI?" on its own. It is whether that power is growing faster than the checks on it and faster than its spread to ordinary people. A very capable system that is independently tested, contained, and widely shared is a different world from an equally capable system run by a handful of organizations with no outside view. The homepage chart shows this as two rising lines. This note explains what goes into each line.

Part of this now runs. Our agents record checked sources every day, and the live trajectory shows whether, on balance, they point toward or away from our principles. The homepage’s risk explorer is different: it is a model you set with your own assumptions, not data. The fuller measure of the gap described below is still a design we intend to build and test.

One family feeds the capability line; three feed the line for oversight and shared power.

1. What AI can do

This family feeds the capability line. Candidate measures:

• How long a task AI can finish on its own. Researchers at METR proposed measuring AI by the length of tasks, timed by how long they take skilled people, that a system can complete with a given reliability. Their paper reports that this "time horizon" has been doubling roughly every seven months over several years (Kwa et al., 2025). It is a useful measure because it tracks autonomy, not just test scores.
• Signs of systems improving themselves. Evidence that AI is doing a growing share of the research and engineering that produces the next generation of AI.
• Agents acting on the open internet. How much AI agents do without a person approving each step: buying, writing code that runs, contacting people.
• Compute and cost trends. The amount of computing power used to train leading systems, and the falling cost of a given level of capability. Epoch AI publishes research and datasets on these trends.

2. Who checks it

This family feeds the oversight line. Candidate measures:

• Independent testing before release. The share of major releases evaluated by an outside party before public use, and whether results are published.
• Containment in use. Whether capable systems run in isolated environments with limited access to networks, money and other systems.
• Native interpretability and circuit legibility. Whether the internal concepts of modern neural networks can be read and audited directly on local hardware without datacenter-scale proxy models—such as the October 2026 debate around Nick Cammarata and Goodfire’s “Neural Networks Are Already Clean” hypothesis (analyzed in our Related labs & independent researchers directory →).
• Honest incident reporting. Whether failures and misuse are disclosed promptly, or found later by outsiders.

3. Who holds the power

This family also feeds the oversight line, because power that is spread out is easier to check. Candidate measures:

• Concentration. How few providers supply the most capable systems, and how concentrated large-scale computing is by organization and country.
• The open gap. How far the best openly available models trail the best closed ones on public tests.
• Running it yourself. The smallest open model that meets a fixed capability bar, and the ordinary hardware needed to run it at home.
• Dependence. How many open-source agent tools rely by default on a single company's model.

4. How society holds up

The last family asks whether people keep their voice and their information as AI spreads. Candidate measures:

• Network shutdowns and censorship, counted per year from established monitoring sources.
• Participation in public decisions, such as citizens' assemblies and other forms of direct involvement.
• Democratic health, from long-running research indexes.
• Data rights: whether people can take their own data with them and decide who sees it.

Each indicator will be recorded with its exact definition, its source, how often the source updates, its known biases, and a quality grade. Self-reported or opt-in figures can appear, but always labeled, and never alone as the basis for a conclusion. Every retrieval is archived so that anyone can check a past value later.

We are designing against that. For each goal we care about, we track indicators that would show the world moving away from it, and we give those at least as much room as the good news. Rising concentration, more shutdowns, releases that outpace testing, and agent tools that all depend on one company are as important to report as cheaper open models or more citizens' assemblies.

We also hold ourselves to a rule: news can prompt us to ask whether an indicator should be added, but it never moves an indicator by itself. Values change only when their named, pre-agreed sources change. That keeps a dramatic headline from reshaping the picture overnight.

When the date arrives, the forecast resolves true or false from its named source and is scored with a standard accuracy measure, the Brier score. We will publish every forecast, including the ones we got wrong, along with our calibration: whether things we called at 60% happened about 60% of the time. If several people forecast, each builds a track record.

We are not inventing this from scratch. The Forecasting Research Institute studies how to make forecasts useful for decisions, and its ForecastBench project offers a continually updated benchmark that compares the forecasting accuracy of AI systems with that of people, using questions whose answers are unknown when asked (Karger et al., 2024). Work like this shows the method can be done at scale and kept honest.

Then a first batch of forecasts, published with dates and resolution rules. Reports will follow on a regular schedule, with each number shown next to its source and quality.

If you work on AI evaluation, forecasting, open data on computing power, or measures of democratic and digital health, we would like to hear which indicators you trust, which you don't, and why.