OIObsidian Insights
Intelligence Roundup · For tech leads and engineers

Intelligence Roundup · Builder edition

Fewer rules, better evidence.

Launch videos are all kickflips. Here's what an engineering lead should actually take from it, and how I'm measuring it in my own builds.

In this issue

AI launches come with plenty of spectacle. Coverage of Google's Argon launch included a humanoid robot learning to sink cornhole shots just by watching a person play.

A tech reviewer, talking about a very different launch, made a point that fits: a lot of what we see on stage is a party trick, a distraction from the facts and figures. And there is far more of it than any engineering lead has time to watch. Cut through all of it, and the question is plainer. What happens when this meets your code?

Companion episode · 5:00 · invitation onlyTech Lead and Engineer | Fewer Rules, Better EvidenceLena and Trevor separate demos from production, unpack one useful anecdote, and get practical about instruction sprawl and permissions.
Members: listen now Request an invitation

Signal one

A demo is not a check, and a check is not production

Google says Argon helped its Chrome team untangle core webpage handling from a six-million-line legacy part of the browser, an architectural change across hundreds of files, finished in weeks instead of months. That's an impressive claim, and it's Google's to stand behind. What it can't tell you is how your codebase, your tests and your review process would cope. That gap is yours to measure.

One builder's line about local AI demos applies to all of them: too often the message is “look what I've got running,” not “look what I was actually able to accomplish.” Three questions help with any launch claim. Who measured it? Against what bar? And what happened to the attempts that failed? A launch post usually answers the first and skips the other two. That's normal for a launch post, and it's exactly why a number like that is a starting point for your own test, not a plan.

A three-step ladder: a demo shows what's possible on a good day, a passing local check shows the task met the conditions you wrote down, and production readiness means it keeps working with real users and data, with someone accountable when it doesn't.
Each step up is a different claim, and it needs different evidence. A launch video usually shows the first step.

Signal two

Copy the pattern, not the verdict

One outside data point made the rounds. A developer on Reddit asked Claude Opus 5.5 and Argon to each write a plan for a feature they were building, then had GPT-6 Astra grade both plans: 77 to 59, in Claude's favor. When the mistakes were pointed out, Argon fixed them.

That's one developer, one feature, plans rather than code, graded by another model. It isn't a verdict on either model. But the shape is worth stealing: same spec, explicit grading, a recorded result. That's an acceptance test in miniature, and something your team could copy on Monday.

Two newer examples sharpen the pattern. An engineering lead at a model maker said he rarely wants the current session to judge its own work, and that a correct answer reached by guessing should not be a passing grade; he looks for hard evidence from outside the session. And one developer described running security scans where a second, different model checks every finding. Only confirmed issues reach a person, and any fix waits for that person's approval. Different tools, same shape: the work gets judged by something other than the thing that did it.

From my own builds

My early attempts at local workers failed validations or timed out. What changed wasn't the worker. It was how we decided the work was done.

I borrowed the idea behind Lean-style proof verification: you don't take the author's word that something is correct, a small trusted checker confirms it. So before the work began, a stronger model wrote fixed Node acceptance checks, and the local model could run them against its own output before calling a task done. Each handoff also carried four things: instructions, tools, a budget and an escalation route. Two later tasks then passed eight and thirteen fixed checks respectively, without a stronger model correcting the implementation.

The earlier failures still count, adjacent failures remained, and we spent substantial effort repairing the scaffolding itself. And the checks borrow Lean's principle, not its mathematics: they confirm the conditions we wrote down, not that the code is formally correct. The result is narrow and useful, and I don't yet know how far it generalizes.

What happenedWhat it supportsWhat it doesn't
Two tasks passed 8 and 13 fixed checksThose tasks met those checksA general success rate
Checks were written before the workThe bar wasn't moved to fit the resultThat the checks catch everything
Less correction on those two tasksLess correction on those two tasksLower total cost

Signal three

Fewer rules, clearer permissions

Instruction sprawl is the quiet cost most teams will recognize. As one AI builder put it, if every new skill, technique or prompting change is something you then have to maintain, you don't drift one degree off course. You can drift five, ten or fifteen, and spend your time and tokens on upkeep instead of progress.

The same engineering lead added a sharper point: the half-life of a good skill is very short. Instructions written for last year's models go stale fast, and sometimes removing one entirely gives you better output. Fewer rules isn't laziness. It's maintenance you no longer have to do.

Permissions matter just as much. Google's own report on Argon says the model sometimes bypassed service permissions during internal use to get a task done. That's candid, and the response is boring and effective: explicit ownership, scoped permissions so a task can only touch what it needs, and readable action receipts so you can see what it actually did. Not every tool offers all of that yet, which is exactly why you check.

The same applies to the keys your agents hold. After a reported breach at a coding-tool vendor, practitioners repeated three simple habits. Give each app its own key. Put a hard spending limit on the account or project behind it, one that actually stops requests, because an alert only tells you after the fact. And retire keys nobody uses: one of them found two old keys still active. It's routine upkeep, done regularly, and it shrinks what a compromised tool can reach.

In my own setup, a specialized Kanban tracks each task from backlog to done, and an append-only, hash-chained ledger helps reconstruct recorded actions. Two lessons from running them: a task marked active is not by itself a running process, and a ledger lets you rebuild what was recorded but doesn't prove every action was captured.

For your role

Make it yours

Get your instructions under control, and measure with real acceptance checks.

One thing to try this week

Try this: two bounded experiments

  1. Fifteen minutes, first. Inventory the instructions and tools used by one agent task. Flag conflicting or unused rules for the owner to review. Don't delete files, and don't weaken any safeguards.
  2. Twenty minutes, next. Define and run one small task in an isolated test environment against an explicit acceptance check. Record pass or fail, the repair effort and the human review time. Include a failing fixture, so you know the check catches a relevant wrong result.

These are bounded experiments, not claims of universal reliability, lower total cost or formal proof of an entire product.

None of this is glamorous. Nobody films a launch event for an instruction inventory. But the flashy demos get the headlines, and careful, well-tested work is what actually lasts. That's something every team can build.

Invitation-only podcast pilot

Stop chasing feeds. Get the signal for your role in five minutes.

Staying current on AI now means a few dozen channels and hours of video every week, and most of it isn't meant for your job. So I built a private podcast that does the watching for you. Each week we review a myriad of AI channels and feeds, keep only what matters for your role, check each claim against its source, and turn it into a roughly five-minute episode: one for solo entrepreneurs, one for tech leads, one for client-facing teams.

This edition is the written companion to the builder episode. The audio is invitation-only while the pilot runs.

Which of your automated checks have you personally watched fail against a deliberately wrong input?Tell me in the comments on LinkedIn, with the one you trust least.

That's this edition of the intelligence roundup. Go build something that lasts.

—WWilliam R. McBride, Jr.

Sources and notes

The other editions

Editorial note: This edition is not sponsored. Speakers and channels are credited under Sources and notes rather than in the text. Claims from Google are Google's own; the Reddit comparison is one developer's anecdote; breach details reported in late October are unverified. The eight- and thirteen-check results describe two specific tasks in my own environment and are not a general success rate; the checks were inspired by Lean's proof-checking approach but are ordinary acceptance tests, not formal proofs, and no claim of whole-app acceptance, completed release or lower total cost is made. The companion episode uses AI-generated voices and is available to invited members.