Back to blog

How will we ask AGI?

Aug 12, 2026 - llms, agi

Prompting a model today often means writing a paragraph of precautions: follow these steps, avoid those mistakes, use this format, and keep the numbers real. A system that understands the business needs fewer words. "Close the quarter" takes just three, with dozens of decisions left implicit.

Take that far enough and two words plus a shrug are enough for the system to work out the rest.

That shrug depends on a substantial history. The system needs context to recover everything you have omitted. A request this short assumes the listener already knows the fuller version. That knowledge comes from eighteen months of decisions, discarded drafts, and details such as which customer must never get a call on Friday. The request takes little effort because someone spent those eighteen months recording the decisions and keeping the record current.

Requirements will change in much the same way. Engineers spent forty years getting better at specifying what software should do. The machine carrying out those instructions had no way to ask for clarification.

When the system can ask questions quickly, the specification becomes an ongoing conversation as the work progresses. It might ask three things: two you had overlooked and a third that exposes a flaw in the plan. A good contractor already works this way, including occasionally turning the job down.

An early refusal can save more than people expect. These systems can carry out most requests. They can also recognize when the request itself is misguided. The first thing to test when evaluating one is its willingness to challenge you. Would you welcome a coworker who questions your plan at nine in the morning, while there is still time to avoid the work and expense? Almost everyone says they would. For most, that enthusiasm lasts about a week.

Once execution is effectively free, choosing the work takes up most of your time.

That choice calls for taste and judgment: knowing which of forty possible projects would actually help the business. Developing it takes years of making choices and living with the results. Companies have always needed more of this judgment, though the shortage of people to do the work used to be more pressing.

Each delegation leaves a record of the authorized action, the person who approved it, the spending limit, and whether it can be reversed. Sending a contract commits you in a way that rescheduling a meeting usually does not.

A company that operates these systems well keeps an unremarkable list of irreversible actions, with a person responsible for each one. A decade of talking about AI safety has brought us to that list.

Software takes on the work. Responsibility stays with whoever defined its scope and whoever approved it. The scope is now part of the written record.

Each organization will develop its own style for that document. We will settle on terms for budgets, scopes, escalation routes, and required approvals. Addressing the system will resemble briefing a general counsel: be precise about its authority, explicit about the limits, and deliberate about what gets recorded and which decisions remain yours.

Learning to delegate means giving someone work, watching them take a different approach, and judging whether that difference affected the result. Most of us are practicing on small tasks with systems still far from general intelligence. We will bring the habits formed here to more consequential work. At the moment, the habit almost everyone is developing is approval without reading.

These systems become steadily better at carrying out our requests. As they improve, the requests become a more revealing record of the people making them. The projects you fund, the formats you require, and the customers you protect show how authority works and what takes priority in practice. Your refusals say just as much about what you value. Delegation at this scale puts those choices in writing.