Where will the next experts come from?

The story

AI makes us faster. It also makes us understand less.

Earlier this year Anthropic ran a randomized trial with 52 junior software engineers. Everyone had to build something using a tool they had never touched. Half worked with an AI assistant, half worked alone. Then everyone took a quiz on what they had just built.

The AI group finished a little faster. They also scored 50 percent on the quiz, against 67 percent for the people who worked alone, a gap of roughly two letter grades. They had produced the work without thinking or analyzing.

The exception is the useful part. Some people in the AI group scored well anyway. They were the ones who asked the AI to explain itself, questioned its answers, and fixed errors on their own instead of handing them back. They used the tool to understand the work, and they kept what they built.

Why it matters for you: your staff are living this trial every day. The work AI takes over first is the first draft, the proofing, the first pass at the numbers, the junior analysis. That is the same work that used to turn a junior person into a senior one. Every trusted editor, analyst, and teacher you know spent years doing the unglamorous version first. A new paper by researcher Nolan Lovett, "The Tragedy of the Cognitive Commons", makes the point at the scale of whole professions: choices that make sense for each organization on its own could, added together, leave a field with nobody who learned the craft well enough to tell when the machine is wrong.

Here's the question: can we embrace these tools and still form people who think? Put concretely: ten years from now, who in your building will be able to check the machine's work?

The practice

Check yourself before you check them

The trial's dividing line was not the tool. It was whether the person made the machine explain itself.

The next thing you craft or refine using AI, ask yourself whether you can source, defend, or explain every claim in it.

And next time you prompt, consider three principles:

1. Ask for the reasoning before the answer. "Before you draft this, tell me how you plan to approach it and where you are least certain."
2. Argue with it once, every time. "Where is this weakest? Make the strongest case against it."
3. Fix edits or errors yourself. When you find a mistake, correct it rather than handing it back for another pass. In the trial, that one habit separated the people who scored under 40 from the people who scored above 65.

Require this of your staff the week after you require it of yourself, not before.

Higher ground

Psalm 145 describes faith moving from one generation to the next: one generation tells the next what God has done. That is how faith has always traveled, person to person, at cost, with patience. Craft travels the same way. Good judgment has always been learned from someone further along who took the time. Mentoring the person who will one day check the machine's work is that same duty, and it is work only a person can do. Ask yourself who took that time for you, and who you are taking it for now.

The next step

On October 8–9 in Boulder, we are taking a small group of founders and executives through exactly this terrain. The Summit is two days at Whisper Ranch to build your organization's approach to AI with the formation questions kept in view — including who you are raising up to check the machine's work. If this issue named a gap you feel, that room is where it closes.

These issues are still new, so hit reply and tell us what helped and what would help more. A real person reads every one.

Not anti-tech. Pro-flourishing.

Climbing with you,
Jeff and Jason