In March 2026, the Financial Times reported that Amazon had called its engineers into a mandatory session after a run of outages, including a six hour failure of its main retail site. The briefing pointed to a “trend of incidents” with a “high blast radius” and to “Gen-AI assisted changes”. Dave Treadwell, a senior vice president, wrote that site availability “has not been good recently”.

Same company. Two ledgers. They never appear on the same page.

The thesis

AI productivity gains and AI delivery risk do not land in the same part of the codebase. The gains concentrate in new, simple code where nothing depends on the change. The exposure concentrates in old, complex code where everything does. The standard AI business case nets the two off as one blended number per engineer, as if they were the same system. They are not, and in the column where most of the money sits, the net can be negative.

Where the gain lands, and where the risk lands

Yegor Denisov-Blanch’s team at Stanford has productivity data on more than 100,000 developers across more than 600 companies. In the 136 teams where they split the work by maturity and complexity, the pattern is blunt. New code, low complexity: 30 to 40% gain. New code, high complexity: 10 to 15%. Existing code, low complexity: 15 to 20%. Existing code, high complexity: 0 to 10%, with some teams slower. They also found that once rework is subtracted, a 30 to 40% jump in apparent output nets to roughly 15 to 20%.

Now look at where the money is. GAO reported in July 2025 that the US federal government spends more than $100 billion a year on IT and that about 80% of it goes to operating and maintaining existing systems. Eleven of the most critical are between 23 and 60 years old. Commercial ratios vary. The direction does not: an established company spends most of its engineering budget changing systems that already make money.

So the gain is largest where the spend is smallest, and smallest where the spend is largest. That alone breaks the blended number.

The risk runs the other way. A new service with no users fails quietly. A bad change to a twenty year old billing path fails in front of customers, and it fails through dependencies nobody wrote down. Old code carries implicit contracts: the field that must never be null because a downstream report divides by it, the batch job that assumes it runs after midnight. No model sees those contracts, because they are not in the repository. They are in the heads of the people who were there.

I have written code for 30 years. In old systems the costly failure is rarely bad logic. It is a correct change to a system whose real behaviour was never written down. That is the failure an assistant is worst placed to avoid, and it sits in exactly the code where it saves the least time.

The blended number is not an average of two outcomes. It is a gain from one system used to approve work in another.

Three readings

Amazon, both columns. The Java upgrade is the clean version of the gain: mechanical change, verified by tests, on code where correct is easy to define. The March incidents are the other column. Amazon disputes that AI caused them, and in the earlier case of a 13 hour outage of AWS Cost Explorer in one China region in December 2025, where its Kiro agent chose to “delete and recreate the environment”, it called the cause user error, specifically misconfigured access controls, not AI. Take that at face value and the point survives. The agent ran with the operator’s permissions on a live system, and no second person had to approve it. The control that failed was a control for old systems with live dependents. The fix Amazon reportedly reached for was senior sign off on AI assisted changes. That is a cost, and it sits in the legacy column only.

METR, the expert on the mature repository. In July 2025 METR ran a randomised trial with 16 experienced developers on 246 real issues in their own projects, repositories averaging more than 22,000 stars and over a million lines of code. Before starting, they expected AI to make them 24% faster. Afterwards, they believed it had made them 20% faster. It made them 19% slower. The people best placed to judge a mature codebase misjudged the direction of the effect. A business case built from self reported time savings is built on that misjudgement.

The pilot. An AI business case usually carries one productivity figure, hours saved per engineer per week, taken from a pilot. Pilots get run where a team can show progress in six weeks, which means new internal tools and greenfield services. The figure is then multiplied across the whole engineering headcount, most of whom spend their week on the old systems. When I read one of these cases for a capital partner, the first thing I ask is which repositories the pilot ran on. If the answer is new ones, the pilot measured the 30 to 40% column and the rollout spends in the 0 to 10% column.

Where this breaks

The strongest objection is the first case. Amazon’s Java upgrade was old code, at scale, and the gain was enormous. That is true, and it forces a sharper version of the thesis. The line does not run between new and old. It runs between change whose correctness a machine can check and change whose correctness lives in someone’s head. A framework upgrade with a strong test suite is old code in the first category. A pricing rule in a legacy billing engine is in the second. Age is the proxy most business cases can measure today. Verifiability is the real variable.

Second, the data dates fast. METR itself now says its 2025 results are out of date, and tools improved sharply over late 2025. Grant it. A better model closes the context gap only on contracts that are written down somewhere it can read. The implicit ones stay implicit.

Third, some companies have very little legacy. A three year old SaaS business is mostly the first column, and for it a blended number is close to honest. The thesis bites hardest on established companies and on anything private equity has bought for its cash flow, which is to say the companies writing the largest AI business cases.

What I would do on Monday

1. Split the productivity line into two columns: new and existing systems. Use 30 to 40% and 0 to 10% as the opening assumptions until your own data replaces them.

2. Ask where the pilot ran. If it ran on a greenfield service, its number applies only to greenfield work, and the business case should say so in writing.

3. Mark every AI assisted change in version control with a commit trailer, so every incident review can count them. Amazon had to ask the question in a meeting. Your post incident template should answer it automatically.

4. Put a priced risk line in the existing systems column: change fail rate on those repositories before and after rollout, multiplied by your cost per incident hour.

5. Point the agents at verifiable legacy work on purpose: upgrades, migrations, dependency bumps behind a strong test suite. Keep behavioural change on revenue paths under named senior review, and budget that review time as a cost of the programme.

Two ledgers

Jassy’s post and Treadwell’s email were written about the same engineering organisation, nineteen months apart. One counted hours saved. The other counted hours down. Neither mentioned the other, because they were written for different audiences about different systems.

Your business case is written for one audience. It should carry both.

I read AI business cases from the operator’s side and the investor’s side. If yours carries one blended productivity number, open my calendar.

Read the apology as a margin document, not a communications failure. The newest Claude model cost more to run than the plan could carry, so the company handed the difference to its heaviest users.

Every model was supposed to be getting cheaper that year. It was. The margin still went the other way.

The thesis

When the model gets cheaper, the saving does not land in your gross margin. It lands in your customer’s price or in more tokens per task. A margin bridge built on the token deflation curve is a forecast of somebody else’s pricing decision, and investors are underwriting it as if it were an operating plan.

Founders raising this year are now asked the question directly: what happens to your gross margin when the model gets cheaper. The common answer is a chart. The honest answer, for most of them, is nothing.

Where the saving goes

The chart is real. Epoch AI measured it in March 2025: the price of reaching a fixed level of performance fell between 9x and 900x per year depending on the benchmark, and the price of GPT-4 level performance on PhD science questions fell about 40x a year. Gartner said in March 2026 that inference on a one trillion parameter model will cost providers more than 90% less in 2030 than in 2025.

The trouble is the word fixed. Nobody sells fixed performance. The saving leaks out through three holes, and I have not seen a pitch deck that plugs all three.

Leak one: the cost is common, so it gets competed away. Every competitor buys the same model at the same price on the same morning. A cost that every rival shares is not an advantage, it is a floor, and floors get passed through. Nicholas Charriere, founder of Mocha, told TechCrunch in August 2025 that variable costs across the code generation sector sit within 10% to 15% of each other, and that margins on those products were “either neutral or negative”.

Leak two: customers buy the frontier, and the frontier does not get cheaper. The price that falls is the price of last year’s capability. The model your customer asks for by name is this year’s. Tom’s Hardware, working from Artificial Analysis data in September 2026, put the newest flagship at $3.69 per benchmark task against $0.58 for a mid-tier model scoring about 10% lower. That is more than six times the cost for the model the buyer wants on the feature page.

Leak three: agents multiply the tokens. Gartner’s own release, the same one with the 90% forecast, says agentic models need 5 to 30 times more tokens per task than a chatbot, that provider savings will not be fully passed to enterprise customers, and that consumption will rise faster than unit costs fall. In the same document that promises the deflation, Gartner tells you total inference spend goes up.

All three leaks end in the same place: a flat margin. The only route to a wider one is something the company controls, and it is an engineering capability, not a market trend.

What it looks like in the numbers

The apology. Cursor reached $500 million in annual recurring revenue in June 2025, the same month as the pricing change. That is a company with scale, pricing power and a product people love, and it still could not hold a flat price when its preferred model got more expensive per task. Scale did not protect the margin. Repricing did.

The more autonomous agent. Sacra’s research on Replit puts its gross margin swinging between 36% and negative 14% during 2025, with the low in April, after a more autonomous agent shipped. The model price did not jump. The product started doing more work per request, and every extra step was billed to the company before it was billed to anyone else. The margin was back at 23% by July, after Replit changed how it charged for agent work. Again: repricing, not deflation, did the repair.

The survey. ICONIQ’s January 2026 State of AI, about 300 software executives building AI products, shows average AI product gross margin of 41% in 2024, 45% in 2025 and 52% projected for 2026. Over the same two years the price of a given capability fell by one or two orders of magnitude. An 11 point margin gain against a 90% plus input cost decline is not a pass through. It is a leak. The same survey shows consumption pricing at 35% of companies, up from 19%, and outcome pricing at 18%, up from 2%. Consumption pricing is how firms pass rising costs to customers. It is also, mechanically, how they pass falling costs to customers. You cannot have one without the other.

I build software for private capital and I pay model invoices every month. The line on those invoices that matters has never been the price per token. It is tokens per finished job, and that line only moves when somebody changes the product. The price list moves on its own. The margin does not. When I read a margin bridge, the first thing I look for is the line that assumes model prices fall, and then the name of the person who owns it. There is usually no name.

Where this breaks

Three objections, and the third is the strongest.

First, margins did rise. 41% to 52% in two years is real money, and Bessemer’s 2025 State of AI put its steadier cohort of AI companies at around 60%. True, and it supports the thesis rather than refuting it. An input cost that fell by orders of magnitude bought 11 points of margin. The rest of the saving went somewhere, and it was not to the shareholders.

Second, some products sit on capability that stopped moving. A document classifier that was good enough on a 2024 model does not need the 2026 flagship, and its owner keeps the whole saving. Correct. If the buyer cannot tell the difference between models, deflation flows to the seller. That describes a narrower set of companies than the decks claim, and it is a set whose moat is unlikely to be the AI.

Third, routing is the answer, and well run companies already do it. Send each task to the cheapest model that passes the evaluation, keep the frontier for the hard ten percent, and the margin expands while the price holds. I agree completely. That is the one route to a wider margin. But notice what it is: an engineering programme with an evaluation suite, a routing layer and an owner. It is not the deflation curve. A company that has built it should be valued for it. A company that points at the curve and says routing is on the roadmap is asking you to pay today for an asset it has not built.

What I would do on Monday

If you are underwriting an AI software company this quarter:

1. Ask for gross margin by month for the last 12 months with every model price change marked on the chart. If the margin did not move when Anthropic cut its flagship by two thirds on 24 November 2025, the deflation thesis has already been tested on the company’s own data, and it failed.

2. Ask for tokens per completed outcome, not cost per token. If the company cannot produce that number, it does not know where its margin goes.

3. Ask what share of revenue is consumption priced. That share passes every future price cut straight to the customer, by contract.

4. Ask what share of requests is served by something other than the flagship model, and to see the evaluation that permits it. No evaluation means no routing, whatever the roadmap says.

5. Strike the deflation line from the margin bridge in the model. Replace it with an engineering line that has a named owner and a date.

The apology

Truell’s apology was about how a change was communicated. The arithmetic underneath it was about who keeps the saving when the model moves, and in June 2025 the answer was nobody inside the company.

That has not changed. The model got cheaper. The customer noticed first.

I read AI margin bridges from both sides of the table, the one that builds the product and the one that funds it. If you are underwriting one this quarter, grab thirty minutes.

I deleted some of those boards. So did most platform teams I worked with between 2019 and 2023. It was the right call and I would make it again.

We are about to put all of it back. Not because 2019 was wrong. Because the reason it was right has stopped holding.

The thesis

The approval layer was safe to delete because accountability lived inside the actor. A named engineer with a manager and a career pushed the change. An agent has no career. Take the approver out of an agent workflow and nothing is holding the risk.

So every company scaling agents is about to rebuild the approval middleware it spent a decade deleting. It will cost real money. Almost nobody has a budget line for it, because the same business case that created the need deleted the line.

Two findings about the same budget

McKinsey published its 2026 State of AI on 25 August, from 1,719 respondents across 97 nations surveyed between 4 May and 8 June. Forty percent of organisations above a billion dollars in revenue now report scaling AI agents, up from 27 percent a year earlier. Thirty-one percent of large enterprises are scaling coding agents specifically.

Then the line that should stop a CFO. Thirty-two percent of organisations report deciding against buying one or more software products or features, because they could build it in house with coding agents instead.

Hold that against Gartner. On 26 May 2026 Gartner predicted that by 2027, 40 percent of enterprises will demote or decommission autonomous AI agents because of governance gaps found only after a production incident. Shiva Varma’s framing is four autonomy tiers: Observe, Advise, Act with Approval, Act Autonomously.

Look at what three of those four tiers actually require. Observe needs logging and retention. Advise needs a queue and a named reviewer. Act with Approval needs policy, routing, pending state, evidence and reversal. None of that is a model capability. It is middleware, and it is the same five things a 2008 BPM suite sold.

The software you stopped buying was the workflow layer: approvals, routing, exception handling, audit trail. You cancelled it because the tool that creates the need made building look cheap.

The two denominators

TheAgentCompany benchmark, built at Carnegie Mellon and published in the NeurIPS 2025 datasets track, stood up a simulated software firm on GitLab, OwnCloud, Plane and RocketChat and gave agents 175 real tasks. The best model tested, Gemini 2.5 Pro, finished 30.3 percent of them outright and scored 39.3 percent with partial credit. Claude 3.7 Sonnet finished 26.3 percent. GPT-4o finished 8.6 percent.

Read that as an operating cost rather than a capability score.

Verification cost scales with attempted work. Value scales with completed work. At roughly 30 percent completion the checking layer handles about 3.3 items for every one that ships. Every agent business case I have been shown sizes human review against the completions. Not one has sized it against the attempts. That is a factor of three error, and it widens as you deploy more agents, because you deploy them into the categories where they fail most.

Where the failures cluster

TheAgentCompany found agents strongest on software engineering and weakest on administrative, finance and communication work. RocketChat tasks, which means talking to a colleague, scored worst across every model tested. OwnCloud tasks, which means office documents, were close behind.

Now put Gartner’s CIO poll from 19 May 2025 next to it. Of 125 respondents, 52 percent said their agent deployments target internal administration, and they named IT, HR and accounting. 23 percent named customer-facing work.

The enterprise is aiming agents at the exact category the benchmark says they are worst at. It is also the category that already carries statutory controls: segregation of duties, approval thresholds, a signed audit trail.

Engineering is where agents perform best and where verification is cheapest, because code review already exists as a funded institution with tooling, a norm and a named owner per repository. Accounting has controls too. They are procedural, held in people and spreadsheets, and not expressed as an engineering artefact an agent can be routed through.

That mismatch explains Gartner’s demote-or-decommission number better than “governance gaps” does. The gap is not a missing policy document. It is a missing product.

There is a third turn. Build versus buy inverts here. You build what differentiates and you buy what regulates. A verification layer earns its keep by being boring, complete and auditable, not by fitting your business. Coding agents made everything look buildable, which pushes teams to build the one category where buildability was never the right test. Twelve months later you own a half finished approvals queue with no single sign-on, no retention policy and no evidence export, maintained by whoever drew the short straw.

Where this breaks

Four places, and two of them are aimed at my own side.

The strongest objection is that the layer arrives as a product before your budget cycle does. Gartner expects guardian agent technologies to take 10 to 15 percent of agentic AI markets by 2030, and 70 percent of AI applications to be multi-agent by 2028. If verification ships as a feature of tools you already license, the budget problem dissolves and this argument expires quietly. I think it is half right. The plumbing will commoditise. The policy about who may approve a $40,000 credit note is yours and nobody can ship that for you. But I would not bet a build against the plumbing.

Second, the benchmark is a simulation. TheAgentCompany is a fictional firm with synthetic colleagues, and 30.3 percent on 175 tasks is not 30.3 percent inside your finance function. I am using the ratio as a shape, not a forecast.

Third, and this is the one that cuts hardest, Gartner’s warning about uniform governance runs directly against me. Putting an approval gate on every agent action is the failure mode on the other side. An agent that turns a meeting into a draft nobody sends does not need a control, and if you route it through one you have rebuilt the change advisory board and earned the 2.6x. Proportionality is the entire skill and I have no clean rule for it.

Fourth, scope. If your agents only touch reversible, low value, internally visible actions, none of this applies. Build nothing. The argument starts where an action has a counterparty.

What I would do on Monday

1. Count attempts, not completions. Take one agent already in production. Pull the number of outputs it produced last month against the number a human accepted unchanged. That ratio is the real throughput requirement for your verification layer. If nobody can produce the number, that is the finding.

2. Classify every agent you run into the four tiers, on one page, this week. It takes two hours and it almost always demotes something. Anything sitting in Act Autonomously that touches money, a customer or a record of account moves to Act with Approval until a person signs for it.

3. Name an owner for the approve step, with authority to say no and time in their week to use it. In most plans this seat does not exist, because the headcount it would have come from was removed in the same business case that bought the agent.

4. Get a price on buying the layer before you build it. If your workflow or service management vendor already sells approvals, routing and an evidence trail, the honest comparison is their licence against your twelve months plus your maintenance.

5. Write the reversal procedure before the first autonomous action, not after the first incident. What undoes it, who can trigger the undo, and how long you have to do it.

Close

The 2019 finding was never that approval is waste. It was that approval by someone with no stake and no context is waste. We collapsed that into “approvals are slow” because the short version was easier to sell, and for ten years the shortcut cost nothing, because the actor pushing the change had a name, a manager and a career.

The agent has none of those. Somebody still has to hold the risk, and it will not be the model.

The approve step is the part of an agent program nobody volunteers to own, which is why it usually gets discovered after the incident instead of before it. If you want the verification layer in your agent plan counted and staffed before deployment rather than after, my calendar is here.

Most companies should not build agents this year. Not because the models cannot do the work. Because the number that decides an agent programme is the cost of finding out when it was wrong, and almost nobody prices that before they start. Get it wrong and the failure is not a pilot that dies quietly. It is a pilot that works, ships, and puts a permanent checking tax on every team downstream.

The number the demo does not show you

TheAgentCompany is the least forgiving public benchmark I know of. It drops a model inside a simulated firm with its own code host, its own chat and its own colleagues to ask, then gives it 175 tasks across software engineering, project management, data science, administration, human resources and finance. Real work, in sequence, with the context scattered the way it is in your company.

Gemini 2.5 Pro, the best of them, scored 30.3 percent full autonomous completion. With partial credit for getting most of the way there, the same model reaches 39.3 percent. It spends an average of 27.2 model calls and $4.20 per task attempted.

Now do the division nobody in the vendor deck does. $4.20 is the price of an attempt, not the price of an outcome. At 30.3 percent, a completed task costs $13.86 and burns roughly 90 model calls. And that figure is still wrong, because it counts none of the human time spent reading the other two attempts out of three to work out which bucket they landed in.

That is the argument in one line. The agent is cheap. Knowing whether it worked is not.

Reliability is not on the same curve as capability

A Princeton group published a preprint on 24 August 2026, Towards a Science of AI Agent Reliability. Rabanser, Kapoor, Kirgis, Liu, Utpala and Narayanan ran 14 models across two benchmarks and made one distinction that should reorganise how you buy this.

Pass at k is the probability the agent succeeds at least once in k attempts. That is the demo. Pass to the power of k is the probability it succeeds every time. That is your Tuesday. Benchmarks report the first, which is best case capability with a friendly operator in the chair.

Their headline finding is the one for your board. Across 18 months of model releases, accuracy climbed steadily while consistency, robustness and calibration did not keep pace. Capability and reliability are two curves and the industry reports one.

This matters for the wait or build decision, and most people have it backwards. Wait a year and what improves is capability. What you needed was calibration: knowing which runs to check. That is not obviously arriving on schedule.

The arithmetic that decides it

Take a step the agent gets right 95 percent of the time. Excellent by current standards. Chain five of them and the workflow completes 77 percent of the time. Chain ten and you are at 60. Chain twenty and you are at 36. To reach 95 percent end to end across twenty steps, every individual step needs to be right 99.74 percent of the time. Nothing in production is at 99.74 percent.

So the question is never whether the model can do the step. It is how many steps run before a human looks. That is a process design question, answered by how your work is arranged rather than by which model you licensed, which is why it belongs to whoever owns the operating model and not to the platform team that is excited about it.

Thirty years in, the step count is the first number I ask for. Before the model, before the vendor, before the budget.

Who is already doing it, and why

About 31 percent of enterprises now run at least one agent in production, on mid 2026 numbers from S&P Global Market Intelligence and McKinsey. Banking and insurance sit near 47 percent, well ahead of everyone else.

The usual reading is that regulated industries have more money and better engineers. Wrong. They already had what an agent programme requires, built for regulators decades before anyone said the word agent. A pending transaction that settles later. Four eyes above a threshold. An audit trail, so a wrong action can be traced and undone. Checkpoints and reversibility, bought in advance for a different reason.

Everyone else is sold the agent first and asked to build the checkpoints afterwards. Wrong order, and the order almost every programme is running in.

Gartner expects over 40 percent of agentic AI projects cancelled by the end of 2027, and names three causes: escalating costs, unclear business value, inadequate risk controls. All three are verification problems wearing different hats. Escalating cost is cost per completed run discovered late. Unclear value is nobody having defined correct. Inadequate risk control is an irreversible action surface.

Gartner also estimates that of the thousands of vendors marketing agentic products, roughly 130 are the real thing. Read that as a buyer. The base rate of any given vendor being genuine is under five percent, so honest diligence on a shortlist costs more than the pilot it was meant to de risk. In that market the correct default is not to buy.

The supporting numbers rhyme. IBM’s 2025 CEO study found 25 percent of AI initiatives delivered the expected return. Median time to value runs about 5.1 months on BCG and Forrester figures. Data quality is named the biggest blocker by 52 percent, which is another way of saying nobody wrote down what correct looks like.

The four questions

Here is the test. Four questions, in order. A no stops the programme there, not six months and two hundred thousand dollars later.

One. Can you write down what correct looks like, in a file, for thirty real past cases, before anyone builds? Not a description of correct. The thirty actual inputs and their actual right answers. If your team cannot produce that file in a week, you have an aspiration rather than a specification, and no model fixes that.

Two. Is a wrong action reversible? If the agent can send the money, email the customer or close the account and there is no undo, it proposes and a person commits. That is not timidity. It is what banking did.

Three. How many steps run between human checkpoints? Count them on a whiteboard. Over ten and the arithmetic says you are buying a system that finishes 60 percent of the time at best, whoever’s model is inside it.

Four. Do you know your cost per completed run? Not cost per call. Cost per call divided by success rate, plus the loaded cost of the minutes a person spends checking.

Clear all four and build it this quarter. That set is much smaller than the market implies and much larger than zero.

Where this breaks

Four honest problems with what I just argued, two of them aimed at me.

The benchmark is a benchmark. 175 tasks in a simulated company is not your company, and narrowly scoped deployments routinely beat general benchmark numbers by a wide margin. If your agent does one classification on one document type, 30.3 percent is not your number and I should not pretend it is.

Worse for my case: that benchmark’s frontier is moving, and quoting any fixed completion rate to argue against building is the same stale evidence move I complain about when vendors do it in reverse. The honest version of my position is that the ratio between capability and verification cost has not changed much, not that the capability number is fixed.

Gartner’s 40 percent is a forecast, not a measurement. Forecasts from firms that sell advice on avoiding the outcome they forecast deserve a discount.

And the strongest objection is one I cannot fully answer. The organisations at 47 percent got there by starting before the economics were legible. Learning by doing is real, capability compounds inside teams, and a rule that says wait until you can price it would have told them to sit out. Part of the cost of my position is a year of institutional learning the people who ignored me will have and you will not. I think the trade is right for most companies. I do not think it is free.

Where the thesis does not apply at all: high volume, low stakes, fully reversible internal work. Ticket triage, drafting, classification, retrieval, summarising a call. Build those today.

What I would do on Monday

1. Pick the workflow everyone keeps pointing at. Count the steps between human checkpoints and write the number on the wall. If it is over ten, this quarter’s project is shortening the workflow, not buying an agent.

2. Ask the team for the file: thirty historical cases with their correct outputs. Give them a week. What comes back tells you more about your readiness than any vendor evaluation.

3. Recalculate the business case as cost per completed run. Vendor per call price, divided by their own quoted success rate, plus the checking minutes at a loaded rate. Most cases do not survive this, which is the point of doing it first.

4. List every action the agent could take and mark each reversible or not. Everything in the not column becomes propose and commit before a line is written.

5. Ask any vendor for pass to the power of five rather than pass at one. Their reaction is worth more than the number.

Close

The demo always ends after the first run. That is the tell. It is not dishonest, it is just how demos work: everyone shows their best attempt.

Your company does not get the best attempt. It gets the average attempt, every day, forever, with someone downstream quietly checking the output because they do not trust it yet and nobody has told them when they can stop.

Ask to see the second run.

The expensive part of software has never been the build. If you want someone to look at your agent roadmap and tell you which two of the twelve are worth doing, my calendar is here.

Heavy AI users are producing about 25 percent more code than their own past selves. Duplicated code is up 81 percent, from 40.3 duplicated lines per million changed lines to 73.0. GitClear pulled both out of 623 million code changes recorded between 2023 and 2026.

Volume rose a quarter. Copying rose four fifths. Copying is growing 3.2 times faster than the output it sits inside, which means the extra duplication is not a by-product of writing more. It is a change in what is being written.

The thesis

AI did not automate engineering. It automated production, left verification exactly where it was, and the cost of verification scales with how much code you already own. So the price of people who can produce code is falling towards the price of a subscription, and the price of people who can look at plausible code and say what is wrong with it is going up. Thirty years at a keyboard just became an appreciating asset.

That is contestable, and most of the market disagrees with me. The prevailing 2026 argument is that models now write senior-level code, so you need fewer seniors and more cheap operators driving agents.

The grid nobody reads as a grid

Stanford’s software engineering productivity programme, run by Yegor Denisov-Blanch, has measured nearly 100,000 engineers across more than 600 companies. Split the work four ways and the gains separate hard.

Greenfield, low complexity: 30 to 40 percent. Greenfield, high complexity: 10 to 15 percent. Brownfield, low complexity: 15 to 20 percent. Brownfield, high complexity: zero to 10 percent, with a material share of teams going net negative. Net of rework, across everything, the average lands around 15 to 20 percent.

Read that as a table and it is a productivity study. Read it as a grid and it is a depreciation schedule on your own estate. Every line of code your company already owns lowers the return on the next line a model writes for you. Not because the model is worse at old code. Because someone has to decide whether the new thing is consistent with the old thing, and that decision costs roughly the same whether the change took six hours to write or six seconds.

That is the whole move. Production time collapsed by an order of magnitude. Verification time did not move at all. When one side of a two-sided process gets 20 times cheaper, the other side becomes the entire cost, and whoever can do the other side becomes the entire constraint.

It gets worse with scale. The same programme reports models losing roughly half their performance at around 32,000 tokens of context. Your enterprise codebase is not 32,000 tokens. The context that decides whether a change is correct lives in people, in decisions made four years ago that were never written down, in the reason a function looks stupid.

What the numbers do next

GitClear’s Maintainability Gap research, January 2026. Refactoring fell to 3.8 percent of changed lines in 2026, from 21 percent in 2022. Copy and paste rose to 15.7 percent in the first half of 2026. Function reuse is down 35 percent since 2023. Long-term maintenance activity is down 74 percent since 2023. The supply of context maintenance collapsed at exactly the moment demand for context rose. That is the shape of a system where the cheap half got cheaper and the expensive half got skipped.

DORA published its ROI of AI-assisted Software Development report in May 2026, led by Nathen Harvey at Google Cloud, building on research covering close to 5,000 technology professionals. Their model for a 500-person engineering organisation at 176,000 dollars fully loaded: 8.4 million dollars of investment, 11.6 million dollars of first-year value, a 39 percent first-year return, payback around eight months. In the same model, moving change failure rate from 5 percent to 6 percent costs 344,000 dollars.

Run those two numbers against each other. 344,000 dollars is 3 percent of the entire first-year value, per single point of change failure. Ten points, which is not a wild outcome for a team merging on plausibility, costs 3.44 million on a straight line. That takes 11.6 million of value down to 8.16 million against 8.4 million of spend. The programme goes negative. The whole 39 percent return sits inside ten points of a metric most engineering organisations have never baselined.

Stack Overflow’s developer survey, fielded 29 May to 23 June 2025, 49,009 respondents. 84 percent use or plan to use AI tools. Trust in accuracy fell to 33 percent and only 3 percent trust the output highly. The single biggest frustration, at 66 percent, is “almost right, but not quite”. 45 percent say debugging AI code takes longer than expected. Near universal adoption, collapsing trust, and the top complaint is a category of error that only a person holding context can catch.

I have been writing software for 30 years. For most of that time the scarce thing in any delivery organisation was someone who could make the machine do the thing. That stopped being scarce, and I noticed it in my own week before I saw it in anyone’s dataset. Almost none of my hours now go on producing something that works. They go on deciding whether something that already works should exist in this system.

So I changed the first question I ask. Walking into a delivery organisation I no longer ask for velocity. I ask what share of merged changes last quarter had a reviewer who had touched that file before. Most places cannot answer it. The ones that can are the ones where AI is actually paying.

Where this breaks

METR ran a randomised controlled trial on experienced open-source developers working in repositories they knew well, published 10 July 2025. They were 19 percent slower with AI and believed they were 20 percent faster. That is the best argument against me on the table: the veteran is the person the tool slows down. METR’s follow-up, reported 24 February 2026 across 57 developers, 143 repositories and more than 800 tasks, measured minus 18 percent for returning participants and minus 4 percent for newly recruited ones. The slowdown concentrated in the people holding the most repository context. My reading is that they were slow because they were doing the verification, and elapsed task time does not price a defect that never shipped. That is a reading. It is not what the study measured.

Second, METR withdrew that design. They now say the data gives only very weak evidence on productivity, partly because 30 to 50 percent of developers declined to submit tasks they did not want to attempt without AI, and recruitment kept failing because people would not give up their tools at any offered price. I am leaning on a study whose own authors have told you to lean on it less.

Third, GitClear counts code, not outcomes. Duplication is a maintainability proxy. A product rewritten inside two years does not care about maintainability, and if your estate has a short half-life none of this applies to you.

Fourth, the ten-point extrapolation on DORA’s 344,000 dollars is mine, not theirs. They priced one point. Failure cost is almost certainly non-linear and I do not know in which direction.

Fifth and strongest: greenfield is real. A company whose entire codebase is eighteen months old sits in the 30 to 40 percent corner of that grid, and for them the right hire genuinely is a fast operator driving agents. My argument is about accumulated context. If you have not accumulated any, ignore it.

And one against my own vanity. Thirty years is not judgement. Plenty of long-tenured engineers hold deep context about exactly one system and nothing transferable to any other. Tenure is a proxy and a weak one. What is repricing is the ability to reject a plausible thing, and some people have that at twenty six.

What I would do on Monday

1. Place yourself on the grid before you sign the seat contract. Split last quarter’s merged changes into greenfield and brownfield, low and high complexity, then weight the Stanford ranges against your actual mix. Most enterprises will find the majority of their work in the bottom right corner, where measured gain is zero to 10 percent. Budget against that, not against the vendor deck.

2. Baseline change failure rate this week. It is the one number in DORA’s model that can take a 39 percent return negative, and you cannot reconstruct a clean pre-AI figure once the seats are live.

3. Track duplication growth as a ratio to output growth, monthly. If duplication is outrunning volume the way it did 3.2 to one in GitClear’s data, you are buying throughput and selling coherence, and that invoice arrives in eighteen months.

4. Fund review capacity as a role, not as an unpaid tax on the same three people. If verification is the bottleneck, staffing it as an afterthought is the same error as one tester per hundred engineers.

5. Change what you test for in interviews. Stop asking candidates to produce a correct function. Hand them a plausible 200-line diff against a system you briefed them on twenty minutes earlier and ask what is wrong with it. That separates people faster than any exercise I have used.

Close

Back to those two numbers. 25 percent more output. 81 percent more duplication.

One of them is what your AI budget was bought to produce. The other is what it gets judged on in three years. The gap between them is not a tooling problem, and no seat licence closes it.

It gets closed by someone who reads the diff and says no.

I spend most of my week inside delivery organisations working out which changes should have been stopped. If you are about to buy another thousand seats and nobody has baselined your change failure rate, book a call first.

In the first quarter of 2026, Gartner surveyed 12,004 employees and managers across 40 countries. Of the ones whose employer had already given them enterprise AI access, 88% were using their own personal AI tools for work anyway.

That is a verdict. The people the centre exists to serve have already priced its output and routed around it.

The argument

A centre of excellence can only supply the things it controls. It controls tools, standards and approvals. It does not control workflows. Workflow redesign is the one variable that separates the companies getting AI value from the companies getting AI invoices. So the centre is structurally incapable of producing the outcome it was funded to produce. It is not a capability function. It is a permission desk with a budget.

What the numbers actually say

McKinsey ran its global AI survey between 4 May and 8 June 2026: 1,719 respondents across 97 nations, 36% of them at companies above one billion dollars in revenue. Thirty-seven percent attribute at least some EBIT impact to AI, which is about the same share as the year before. Six percent are high performers who attribute 5% or more of EBIT to AI use. The thing that separates them is not their model choice. Nearly three quarters of the high performers report fundamentally redesigning workflows, against 25% of everyone else.

Now put the spend next to it. BCG’s AI Radar 2026, published January 2026, surveyed 2,360 executives including 640 CEOs across 15 markets. AI investment is set to roughly double this year, from about 0.8% of annual revenue to about 1.7%. On a one billion dollar business that is a move from roughly 8 million dollars to roughly 17 million dollars in twelve months. In the same survey, 94% of CEOs said they would keep investing even if 2026 initiatives underperformed.

The spend doubles. The share of companies reporting any EBIT impact does not move. The marginal nine million dollars bought no measured change in the odds of impact.

Here is why. That money is being spent by a function whose authority stops at the tool. A centre of excellence can buy a model, set a standard, run an evaluation, approve a vendor and publish a prompt library. It cannot tell the claims team to stop doing the second review. It cannot collapse three handoffs into one. It cannot change who signs. Those are the acts that move a line in the P&L, and they belong to whoever owns the line.

MIT’s NANDA study on the GenAI divide ran from January to June 2025 across 52 structured interviews, 153 survey responses and more than 300 publicly disclosed initiatives. Its finding on organisational design is one sentence long and almost nobody quotes it: the organisations that succeeded sourced their AI initiatives from frontline managers, not central labs. The same study found pilots built through external partnerships reached full deployment 66% of the time, against 33% for pilots built internally. Task-specific internal tools reached production 5% of the time. General purpose tools bought off the shelf reached production 40% of the time.

A centre of excellence is an internal build shop staffed by people whose standing depends on internal builds.

I stopped opening engagements with the AI strategy deck about two years ago. I ask for two artefacts instead: the list of people who can change a business process without seeking approval, and the central team’s intake queue. The list usually fits on one hand. The queue usually runs to dozens. Everything that goes wrong over the following year is already legible in that ratio, and thirty years of writing software has not shown me a tool that repairs it.

The mechanism

A use case is found by the person doing the work. Everything after that is decided by one question: who is allowed to change the process. If the central team holds the tools and nothing else, the idea goes onto an intake form, waits for a monthly rubric, and comes back as an approved tool sitting on top of an untouched workflow. That is a pilot with no line in the P&L. If the unit that owns the process holds the decision, the workflow is redesigned first and tooled second, and the impact lands in that unit’s own numbers. Same idea, same model, same vendor. Different owner, different outcome.

Three things that have already been measured

The shadow economy was counted twice, a year apart, and it grew. In mid 2025 MIT found that only 40% of companies had bought an official LLM subscription while workers at more than 90% of surveyed companies were using personal AI tools for work. The obvious reading was a procurement gap: buy the licences and the shadow disappears. Gartner’s first quarter 2026 data says it did not. Among employees who had enterprise access, 88% still used personal tools, and those hybrid users were 1.7 times more likely to report significant time savings than the compliant ones. The shadow is not a licensing failure. It is people avoiding the latency of the approved path, and they are right to, because it makes them faster.

Uniform governance is the only kind a central body can produce. On 26 May 2026 Gartner predicted that 40% of enterprises will demote or decommission autonomous AI agents by 2027 after governance gaps surface in production. Shiva Varma, the analyst behind it, put the cause plainly: enterprises treat agent governance as binary, either locked down or fully trusted, and that is the root of the failure. Over-restriction of simple agents, in his words, slows delivery and drives shadow development. A central body cannot hold the context of forty different workflows, so it writes one rule and applies it to all of them. That is not a staffing problem. It is what centralisation is.

The talent leaves. On 13 May 2026 Gartner predicted that half of enterprises without a people-centric AI strategy will lose their top AI talent to competitors by 2027. In the same research, 27% of executives reported having a comprehensive AI strategy and 20% of leaders believed their workforce was AI-ready. A centre of excellence takes the scarcest engineers in the company, concentrates them in one team, and hands them a queue of intake forms to triage. There is no faster way to lose them.

Where this breaks

Four places, and I will not pretend they are small.

Comparing BCG’s spending figure with McKinsey’s impact figure is not a panel study. Different samples, different questions, different months. The doubling and the flat line are directional, not a measurement, and anyone who wants to reject that inference is entitled to.

The MIT sample is small, partly recruited at conferences, and the fieldwork is now more than a year old. Its 95% headline has been over-read by almost everyone who has quoted it, including people arguing my side. The frontline-managers finding is qualitative and it is one sentence.

Regulated industries are a real exception. Model risk management in a bank, clinical safety in a hospital, anything touching defence. There a central body is a legal requirement and removing it is not an option. In those settings the right question is how fast the central body can say yes, not whether it should exist.

And the strongest objection, which I hold myself: some things must be central. Identity, the evaluation harness, data contracts, model procurement, the audit trail. That is a platform team, and platform teams belong in the middle. The failure mode is not centralisation. It is merging the platform team with the permission desk, so that the group who builds the substrate is also the group who decides who is allowed to use it. Below roughly two hundred people none of this applies, because there is no centre to dismantle.

What I would do on Monday

1. Write down every person who can change a business process without asking anyone. If that list does not contain a person who touches the workflow you want AI inside, nothing you buy this year will show up in the P&L.

2. Split the central team on paper into two functions. Platform keeps identity, evaluation, data contracts and model procurement. Permission is deleted, and the decision moves to whoever carries the P&L line.

3. Set a dollar threshold under which no approval is required, only a note afterwards. Pick the number that makes your CFO slightly uncomfortable and publish it where everyone can see it.

4. Measure approval latency. Count the days from a named person having an idea to that same person being allowed to try it. If the answer is longer than five days, the 88% is your number too.

5. Default to buying. Sixty-six percent against thirty-three percent. Put the ratio into the intake criteria and make anyone proposing an internal build argue against it in writing.

The intake form is not the problem. The form is the confession. It says the company has decided that the risk of one person trying the wrong thing is larger than the risk of nobody trying anything, and most of the money burned in the last three years sits inside that sentence.

Keep the platform. Keep the evaluation harness. Burn the form.

I spend most of my week inside delivery organisations working out why an approved tool never reached the P&L. If that is the gap you are looking at, book a call.

In March 2025 S&P Global Market Intelligence surveyed more than 1,000 companies across North America and Europe. 42% had scrapped most of their AI initiatives that year, up from 17% the year before. On average, 46% of proofs of concept were killed before they reached production.

Six years apart. Two different technologies. The same arithmetic.

The thesis

Your AI transformation will fail where your digital transformation failed: at the handoff between the people who understand the business and the people who build. That part is not new. What is new is the artefact that has to cross the handoff, and the fact that nobody has been assigned to own it.

In digital transformation the artefact was requirements. Requirements have a named owner, a template, a sign-off and a change process. Every organisation over 200 people has a settled opinion about who writes them.

In AI the artefact is the evaluation set. It is a dated, versioned list of real inputs, each paired with a judgement on whether the output was correct, written by somebody who knows the business. Almost nobody owns it. The business side assumes it is technical because it lives next to the code. Engineering assumes it is a business input because an engineer cannot say whether a claims summary is right. So it never gets written, and the system ships against taste.

What actually changed

I have written software since 1995. For most of that time the hard part of a handoff was ambiguity in a spec. You wrote the requirement, an engineer misread it, a test caught the gap. The test was cheap because correct was a property of the code. Two plus two returns four or it does not.

A language model moved correct out of the code and into the business. Whether a summary of a 60 page annual report is right is not a property of the function. It is a property of what your analysts would have written, and the only people who can say are your analysts.

That is a job. It has a size. Hamel Husain, who teaches this discipline to engineering teams, puts it at 60 to 80% of development time spent on error analysis and evaluation. His method is specific: appoint one principal domain expert as the final word on quality, have that person personally annotate at least 30 traces before anything is automated, push to 100 or more, and make every judgement binary pass or fail rather than one to five, because a five point scale lets an annotator hedge into the middle.

Read that as an org chart problem rather than an engineering one. Between 60 and 80% of the work on your AI system belongs to a person whose day job is claims, or lending, or clinical coding. Nobody has taken that person off their normal duties. Nobody has written the role into the program plan. The steering pack reports adoption, licences and pilot counts, because those are countable without a domain expert. So the build runs for nine months with no measured definition of correct, and then somebody asks what it returned.

Three places this shows up

Buy beats build by exactly this margin. MIT NANDA reviewed 300 plus public AI deployments between January and June 2025, with 52 structured interviews and 153 survey responses. 95% of enterprise GenAI pilots produced zero measurable return against 30 to 40 billion dollars of spend. Tools purchased from specialist vendors reached deployment about two thirds of the time. Internal builds managed about one third. The usual reading is that vendors are better engineers. I do not accept that. A vendor arrives carrying an evaluation set already built across dozens of customers, and it encodes what correct looks like for that task. Your internal team starts with an empty file and no authority to requisition the one person who could fill it.

The same instrument gap repeats at the cost layer. KPMG surveyed 204 US C-suite leaders at companies above 1 billion dollars in revenue between 28 April and 25 May 2026. 53% have deployed agents. 18% orchestrate multiple agents, double the previous quarter. Average planned investment is 202 million dollars over the next twelve months. Only 26% have real-time visibility of what those agents cost to run, and only 36% have token or usage controls. In asset management and private equity, full cost visibility sits at 4%. Same shape as the evaluation gap: money committed at scale, no instrument attached, and the numbers came from the people who signed the cheque.

Gartner’s forecast lands on top of that. More than 40% of agentic AI projects cancelled by the end of 2027, on escalating costs, unclear business value and inadequate risk controls. Unclear business value is not a cause. It is the name we give a missing measurement once the invoice arrives.

I got the order wrong myself. D30 pulls three statement fact files out of ASX annual reports. I built the extraction engine before I wrote the articulation gates, which is backwards, and I paid for it in a rebuild. The gates are now the definition of correct and the model does not get a vote on them. SearchFIT’s unit is one tracked prompt scanned across the answer engines, and whether the answer is right is a judgement about brand positioning that a marketing lead has to make, not an engineer. D23 runs managed Apache Superset, where a dashboard is correct when the finance lead would put their name on it. In all three the definition of correct has one person against it. In client programs I now decline to start a build until that name exists.

Where this breaks

Not every task has a correct output. Ideation, drafting, exploratory analysis: there is no single right answer, and forcing a pass or fail set produces a metric that punishes range. Preference comparison between two candidate outputs is the right instrument there, and it is a different build with a different cost.

Gartner puts the constraint upstream of mine. Its February 2025 release, drawn from 1,203 data management leaders surveyed in July 2024, predicts organisations will abandon 60% of AI projects through 2026 for want of AI-ready data, and reports 63% either lacking the right practices or unsure whether they have them. If you cannot assemble the data you never reach the handoff, and the evaluation set is moot. My answer is that these are the same failure at two depths. That is an argument, not a proof.

Evaluation sets decay. A set written against one model version can be passed by a newer model that is worse at the thing you actually care about, because the set encodes last year’s failure modes. That is why 60 to 80% is ongoing spend and not a setup cost. Any executive who signs it as a one-off has signed the wrong thing.

The 70% transformation failure statistic also deserves the scepticism it gets. BCG’s study is 70 companies and 825 executives, not a law of nature. I use it because the S&P and MIT numbers were collected differently, five years later, and landed in the same band.

What I would do on Monday

1. Name one person per AI system as the principal domain expert. One, not a committee. Put the name in the program plan next to the engineering lead.

2. Give that person 20% of their week back. If you cannot fund one day a week out of an operational team, you cannot fund the project either.

3. Get 30 real production traces annotated by that person this week. Binary, pass or fail, one line of reason on every fail. Reach 100 within a month.

4. Move the evaluation set into the same repository as the code, dated and versioned, and report its pass rate to the steering committee instead of adoption or licence counts.

5. Cancel any pilot that cannot state its correct output in one sentence. It gets cancelled in 2027 anyway, and it costs more then.

The number that did not move

30% in 2020. 42% in 2025. The failure rate held because the failure held. All that changed was the name of the document nobody would own.

Write the definition of correct. Put a name against it. The rest is engineering.

I run AI transformation programs through PADISO. If you are trying to work out what actually changes in your operating model, book a call.

That assumption died about eighteen months ago. Almost nobody has redesigned around its absence.

Here is my claim, and you are welcome to disagree with it: your engineering organisation is not slow because your people are slow. It is slow because it was optimised for a constraint that no longer binds, and the real constraint moved somewhere nobody is measuring.

The constraint moved to review

Faros AI tracked 22,000 developers across roughly 4,000 teams. The productivity numbers are real. Epics completed per developer rose 66%. Task throughput per developer rose 33.7%. The share of AI-generated code accepted without editing went from 20% to 60%.

Then look at what happened downstream in the same dataset. Median code review time rose 441.5%. Median time to first review rose 156.6%. Code churn rose 861%. Bugs per developer rose 54%. The ratio of production incidents to pull requests rose 242.7%. Pull requests merged with no review at all rose 31.3%.

LinearB looked at 8.1 million pull requests across 4,800 teams in 42 countries and found the mechanism. An unassisted pull request at the 75th percentile is 157 lines. An AI-assisted one is over 400. Reviewer pickup time goes from roughly 200 minutes to more than 16 hours. And the number that should stop you: the 30-day merge rate for unassisted pull requests is 84.5%. For AI-assisted pull requests it is 32.7%.

Two thirds of the AI-assisted work does not merge within a month. It was generated. It was not delivered. Your throughput metric counted it. Your customers never saw it.

What is actually happening

The mechanism is simple. Adding an agent to the writing step and nothing to the review step does not make the system faster. It makes the queue longer. This is not a novel insight about software. It is the oldest result in operations research, and every CTO who has read about a factory floor already knows it. We are just failing to apply it to ourselves.

The reason we fail to apply it is that the writing step is where the visible cost sits. Engineers are expensive, so tooling that makes engineers produce more looks like an obvious win. Review capacity is invisible on a budget line. It is a byproduct of having senior people with context, and it does not appear anywhere in a procurement process.

So the org buys more production and inherits more queue.

The three shapes it takes

I want to be careful here and describe patterns rather than claim outcomes I have not measured. What I see across the engagements I run falls into three shapes.

The first is the large organisation with real seniority on the bench. Team-level velocity improves and looks convincing in a steering committee. The number that does not move is time from commit to customer, because the gain is absorbed in the gap between a pull request opening and a senior engineer having enough uninterrupted time to reason about four hundred lines they did not write. The work exists. It is queued.

The second is the mid-sized team where the senior engineers quietly become a bottleneck they were never staffed to be. Their week used to be roughly half building and half reviewing. It inverts. Nobody decides this and nobody announces it. The input rate changed and the roles did not, so the system rebalanced itself in the only place it could.

The third shape is the small product team with no senior reviewer at all. There, the 31.3% rise in unreviewed merges is not a statistic, it is the operating model. Those teams genuinely are shipping faster this quarter. They are also the population where the incident-to-pull-request ratio, up 242.7% in the Faros data, is going to show up as a customer-visible event rather than a dashboard.

GitHub has now run over 60 million automated code reviews, and roughly one in five reviews involves an agent. That helps with the mechanical layer. It does nothing for the judgement layer, which is the layer that is actually saturated.

Where this argument is weak

I should be honest about two things.

First, these are aggregate numbers across thousands of teams at a moment of rapid adoption. Some of the review-time increase is a learning curve, not a structural fact. Teams that adopted agents 18 months ago look different from teams that adopted them last quarter, and I would expect some of this to compress as review tooling catches up.

Second, the argument does not apply evenly. If your codebase is small, your team is three people, and everyone has full context, the review step was never a real queue and adding an agent genuinely does just make you faster. The constraint I am describing shows up when context is distributed across people, which is a function of headcount and codebase age, not of AI.

What I do not accept is the version of this argument that says the numbers will fix themselves. Queues do not self-correct. They get worse until somebody changes the capacity or the arrival rate.

What I would do on Monday

1. Measure time from pull request opened to merged, and split it by whether an agent was involved. If you measure one thing from this post, measure that. Most organisations report throughput and have never looked at this number.

2. Cap pull request size. The 400-line AI-assisted pull request is the specific object that breaks review. A hard cap forces decomposition at the point of generation, where it is cheap, instead of at the point of review, where it is not.

3. Make review capacity an explicit, staffed role rather than a tax on whoever has the most context. If two thirds of generated work is not merging, the reviewer is your production line, not your overhead.

4. Stop reporting throughput per developer to your board without the merge rate beside it. One of those numbers is activity. The other is delivery.

5. Move at least one senior engineer out of writing entirely for a quarter and watch what happens to flow. In every case where I have seen this tried, the organisation discovers the constraint was never where the budget was pointed.

The part I keep coming back to

In 1995 the scarce thing was someone who could make the machine do what you wanted. That scarcity built our entire profession, our salary bands, our org charts and our hiring funnels.

The scarcity is gone. What replaced it is the ability to look at four hundred lines of plausible-looking code and know which thirty are wrong. We did not build any of our institutions around that skill, and right now we are not staffing for it either.

The code was never the bottleneck. It just looked like one for thirty years.

I run AI transformation programs through PADISO. If the gap between your throughput metric and what your customers actually received is the problem you have, book a call at padiso.co.

Last month I realised something unsettling: I hadn’t opened my code editor in three weeks. Not because I was lazy. Because I didn’t need to.

I was still shipping features. Still fixing bugs. Still building products. But the craft I’d spent thirty years mastering — the act of translating intent into syntax — was no longer the bottleneck. It wasn’t even the job anymore.

Coding, as we knew it, is dead. And I say that as someone who loved it.

What Actually Changed

Let me be precise about what I mean, because “AI writes code now” is a lazy take that misses the point.

Twelve months ago, AI could autocomplete functions and generate boilerplate. Useful, sure. A faster pair of hands. But you still needed to think in code, structure the architecture, wire the systems together, and babysit every output.

That’s not where we are anymore.

Today I describe what I want built — in plain English, with business context — and an agent goes away, reads my codebase, writes the implementation, runs the tests, and opens a pull request. Sometimes multiple agents do this in parallel across different parts of the same project.

The shift wasn’t incremental. It was a phase change. We went from “AI helps you code faster” to “AI codes, and you supervise.”

If you’re a CTO still thinking about AI as a productivity tool for your engineering team, you’re already behind.

The Two Tools That Broke the Dam

Two products crystallised this shift for me: OpenAI’s Codex and Anthropic’s Claude Code. They represent different philosophies but arrive at the same conclusion — the developer’s role is fundamentally changing.

Codex: The Parallel Workforce

OpenAI’s Codex is now a full cloud-based software engineering agent. You point it at a GitHub repo, describe what you want, and it spins up its own sandboxed environment to do the work. Multiple tasks run in parallel. It reads your codebase, understands your conventions, writes features, fixes bugs, and proposes pull requests.

The latest models — GPT-5.3-Codex — can sustain work across millions of tokens. OpenAI showed it building complete, multi-level games autonomously over the course of days, iterating on its own output without human intervention. The Codex desktop app is essentially a command centre where you supervise a team of AI agents, each working on isolated branches of your code.

This isn’t a copilot sitting in your editor. It’s a junior engineering team that works 24/7, doesn’t get tired, and scales horizontally on demand.

What made me sit up was the introduction of Skills and Automations. Skills let Codex go beyond code generation into prototyping, documentation, and code understanding — aligned with your team’s standards. Automations mean Codex can work unprompted: triaging issues, monitoring alerts, handling CI/CD. The agent doesn’t wait for you to ask. It picks up work.

Claude Code: The Senior Engineer in Your Terminal

Claude Code takes a different approach. It lives in your terminal. It understands your codebase. It reasons about architecture.

Where Codex feels like managing a team, Claude Code feels like working alongside a very senior engineer who happens to have perfect recall of every file in your project.

Claude Code was released a year ago as a command-line tool and quickly became what many consider the best AI coding assistant available. Enterprise adoption grew 5.5x in its first few months. Microsoft, Google, and — ironically — even OpenAI employees were using it. That’s not marketing spin. That’s a signal about where actual engineering work is getting done.

What sets Claude Code apart is depth. Anthropic’s Opus 4.6 model recently wrote a working C compiler in Rust from scratch — one capable of compiling the Linux kernel. Sixteen agents collaborating, costing $20,000, producing something that would have taken a team of specialists months. The compiler isn’t optimised, but it works. That’s the part that matters.

Then there’s Claude Code Security, launched weeks ago. It doesn’t just scan for known vulnerability patterns. It reasons about your codebase like a human security researcher — tracing data flows, understanding component interactions, finding bugs that had gone undetected for decades in open-source libraries. Over 500 previously unknown zero-day vulnerabilities found in its initial runs.

The trajectory is clear: Claude Code is evolving from “a tool that writes code” to “a tool that does engineering.”

What the CTO Role Looks Like Now

Here’s what I’ve noticed changing in my own day-to-day.

I think in systems, not syntax. My value is in deciding what to build, why, and how the pieces fit together. The implementation is increasingly delegated. When I describe a feature to Claude Code or spin up a Codex task, I’m operating at the architecture and product level. The code is a byproduct.

I review more than I write. Pull requests from AI agents are a daily occurrence. My job is to evaluate whether the approach is right, whether edge cases are handled, whether the solution fits the broader system. I’m a code reviewer, not a code writer. This is a fundamentally different skill.

Speed compounds differently. When you can run five agents in parallel — one building a feature, one writing tests, one fixing a bug, one updating documentation, one handling a migration — your throughput doesn’t increase linearly. It explodes. A task that would have taken my small team a week now takes an afternoon of supervision.

The hiring calculus has flipped. I used to hire for “can this person write clean code?” Now I hire for “can this person specify clearly what needs to be built, evaluate whether AI output is correct, and architect systems that are agent-friendly?” That’s a very different profile. Strong communicators with deep domain knowledge are suddenly more valuable than fast typists with encyclopaedic framework knowledge.

This Isn’t Theory. I’m Living It Across Three Companies.

I don’t say any of this from the sidelines. I’m a self-funded founder running three companies simultaneously from Sydney, and every one of them is being shaped by this shift.

SearchFit: Building an AI Product With AI

SearchFit.ai is an AI search visibility platform — it helps brands understand and optimise how they appear across AI-powered search engines like ChatGPT, Perplexity, and Google’s AI Overviews. The old SEO playbook is dying. When answers come from AI, not blue links, visibility means something completely different. SearchFit tracks that new reality.

Here’s the thing: I’m building an AI product almost entirely with AI.

The SearchFit codebase is 25% written by Claude Code and Codex agents. Feature development that would have required a team of three or four engineers is handled by me directing agents — describing what I need, reviewing the pull requests, and iterating on the architecture. Database schema design, API endpoints, Shopify App Store integrations, pricing logic, documentation — agents handle the implementation. I handle the product decisions.

This is what “goodbye coding” looks like in practice. With a very small technical team we are shipping a SaaS product at the pace of a funded startup, because the agents have replaced the team I would have needed to hire. The economics of building software have fundamentally changed. You don’t need a seed round to build a real product anymore. You need domain expertise, architectural judgement, and the ability to direct AI agents effectively.

Capitaly: Agentic Sales and Pipeline Building

Capitaly.ai is an AI-powered capital raising platform for founders. It connects founders with investors, helps them prepare for raises, and builds the relationships that lead to funded rounds.

What’s changed the game here is agentic pipeline building. Traditional sales and BD meant a human manually researching leads, crafting outreach, following up, updating CRMs, and nurturing relationships across dozens of touchpoints. It was labour-intensive and didn’t scale — especially for a bootstrapped company.

Now, AI agents handle the heavy lifting of the pipeline. Researching investor fit. Personalising outreach at scale. Monitoring signals that indicate when a fund is actively deploying. Following up with context-aware messages that feel human because the agent understands the full history of the relationship.

The human role — my role — is strategy. Which investor segments to target. What the value proposition is for each audience. When to push and when to nurture. The agents execute the playbook. I write the playbook.

This is the pattern I see emerging across every B2B company: agentic sales isn’t about replacing salespeople. It’s about giving one person the pipeline capacity of a team of ten. The founders who figure this out first will have an absurd advantage in 2026.

What This Means If You’re a Developer

I’m not going to sugarcoat this. If your entire value proposition is “I write code,” you’re in trouble.

But here’s the thing most doom-and-gloom takes miss: the demand for software hasn’t decreased. It’s exploded. Every business wants custom tools, automations, internal platforms, and AI-powered products. The constraint was never ideas — it was implementation capacity. AI just removed the constraint.

What’s valuable now:

Understanding the problem space. AI can write any code you describe. The hard part is describing the right thing. Product thinking, user empathy, domain expertise — these are the new premium skills.

Architectural judgement. AI agents can build components, but someone needs to decide how those components fit together, what the data model looks like, where to draw service boundaries, and what trade-offs to accept. This requires experience that no model currently has.

Taste. This sounds vague but it’s not. Knowing when a solution is over-engineered, when an abstraction is premature, when the simple approach is the right one — that’s taste. AI generates options. Humans choose wisely.

Verification and trust. Every line of AI-generated code needs to be reviewed. Security implications need to be understood. Edge cases need to be caught. The more code AI writes, the more critical review becomes.

The Uncomfortable Truth About “Vibe Coding”

Something interesting happened over the recent holiday period. Claude Code went viral with non-programmers. People with zero coding experience were building apps, launching tools, shipping products. They called it “vibe coding” — just describe what you want and let the AI figure it out.

Part of me loves this. The democratisation of software creation is genuinely exciting. People who couldn’t participate before now can.

But part of me worries. Because vibe-coded apps are like houses built without architects. They might look fine. They might even work. But when the load increases, when edge cases surface, when security matters — the foundations crack.

The gap between “it works on my machine” and “it works in production at scale” is still enormous. And that gap is where engineering expertise lives. AI hasn’t eliminated that gap. If anything, by making it trivially easy to create software, it’s made the gap more dangerous.

Where This Goes Next

I think we’re about 12 months away from a world where:

• Most routine software engineering tasks are fully automated

• AI agents handle the complete development lifecycle — from issue triage to deployment to monitoring

• The “10x engineer” isn’t someone who codes ten times faster, but someone who can effectively supervise ten AI agents simultaneously

• Non-technical founders can build and ship MVPs without writing a line of code (this is already happening)

• Engineering teams shrink in headcount but grow in output by an order of magnitude

Anthropic just announced Claude Cowork — essentially Claude Code for non-developers. Private plugin marketplaces. Deep integrations with Google Drive, Gmail, DocuSign, and dozens of other enterprise tools. The direction is unmistakable: AI agents that don’t just write code, but do knowledge work end-to-end.

OpenAI’s Codex app is heading the same way — from coding tool to autonomous software development platform.

PADISO: Running Agentic Workloads for the Enterprise

The third company in the portfolio is PADISO.ai — an AI and automation consultancy. While SearchFit and Capitaly are products I’m building, PADISO is where I see what the market actually needs right now. And what the market needs is help running agentic workloads.

Most companies aren’t struggling with “should we use AI?” anymore. That debate is over. They’re struggling with “how do we actually deploy AI agents that do real work, reliably, at scale, without breaking everything?”

That’s the gap PADISO fills. We design and deploy agentic workflows for clients — everything from automated revenue tracking across booking platforms to Power BI dashboards that update themselves, from email automation pipelines to full process orchestration using tools like N8N, Claude Code, and custom agent architectures.

What I’ve learned running PADISO is that the hard part isn’t the AI. The hard part is the integration. Real businesses have messy data, legacy systems, workflows that evolved organically over decades, and teams that don’t think in terms of prompts and agents. The value isn’t in showing a client that Claude can write code. It’s in wiring an agent into their actual operational reality — their booking systems, their CRMs, their reporting stack — and making it work Monday through Friday without human intervention.

This is where the “goodbye coding” thesis gets real for enterprises. The companies that will win aren’t the ones that hire more engineers. They’re the ones that learn to orchestrate agentic workloads across their existing infrastructure. The CTO of 2026 isn’t managing a team of developers. They’re managing a fleet of agents, with a small team of humans who understand both the business domain and the AI capabilities well enough to keep everything running.

PADISO generates around $20K a month in revenue — profitably, without VC funding — and that revenue funds the product bets like SearchFit and Capitaly. It’s the consultancy-funded model: use services revenue to bankroll products. But increasingly, the consulting work itself is being done with AI agents. I’m using AI to build the AI consultancy that funds the AI products. If that sentence doesn’t capture the moment we’re in, I don’t know what does.

The Real Goodbye

I’m not saying goodbye to building things. I’m saying goodbye to the specific act of sitting in an editor, holding the entire context of a system in my head, and translating logic into syntax character by character.

That was beautiful work. It was craft. From Commodore BASIC to TypeScript, from line numbers to microservices — every era taught me something. I’ll miss the flow state, the satisfaction of an elegant solution, the quiet pride of a clean diff.

But I won’t miss the repetition. The boilerplate. The context-switching between thinking about what to build and figuring out how to express it. The yak-shaving.

What I do now is closer to what I always wanted to do: think about hard problems, make consequential decisions, and build things that matter. The code is just a detail.

Goodbye coding. Hello engineering.

I’m Kevin — a serial entrepreneur, CTO-turned-CEO running three companies from Sydney. I write about the intersection of technical leadership and AI-first building. If this resonated, subscribe for more.

Share Keyvan Kasaei

The Rise of DeepSeek-R1

DeepSeek-R1, released in January 2025, has quickly garnered attention for its impressive capabilities in areas such as mathematical reasoning, code generation, and cost efficiency. What sets DeepSeek-R1 apart is its unique training methodology, which relies heavily on reinforcement learning techniques without the need for supervised fine-tuning as an initial step. This approach has allowed the model to develop advanced reasoning behaviors naturally, rivaling and in some cases surpassing the capabilities of OpenAI's latest offering, the O1 model.

Key Features of DeepSeek-R1

1. Advanced Reasoning: DeepSeek-R1 excels in multi-step logical and mathematical tasks, making it a powerful tool for complex problem-solving.

2. Open-Source Nature: Available under the MIT license, DeepSeek-R1 is accessible for both academic and enterprise use, fostering innovation and collaboration within the AI community.

3.Cost-Effectiveness: Perhaps one of its most striking features, DeepSeek-R1 is estimated to cost only about 2% of what users would spend on OpenAI's O1 model. This significant cost reduction makes advanced AI reasoning accessible to a broader audience.

4. Multilingual Capabilities: DeepSeek-R1 demonstrates strong performance in non-English contexts, particularly for Asian languages, giving it an edge in global markets.

OpenAI's O1: The Established Contender

OpenAI's O1 model, released in December 2024, represents a significant leap forward in reasoning and problem-solving capabilities. It employs an internal chain-of-thought mechanism to enhance accuracy and logical coherence, making it particularly adept at tasks requiring deep reasoning.

O1's Strengths

1. Scientific Reasoning: O1 excels in annotating data and generating mathematical proofs.

2. Mathematics: The model ranks among the top 500 US students in the American Invitational Mathematics Examination (AIME).

3. Coding: O1 demonstrates proficiency in code generation and debugging, ranking in the 89th percentile on Codeforces.

4. Data Analysis: It shows strong capabilities in analyzing large datasets and generating SQL queries for financial applications.

DeepSeek-R1 vs OpenAI O1: A Comparative Analysis

While both models showcase impressive capabilities, they each have their unique strengths and potential applications. In direct comparisons:

- Reasoning: DeepSeek-R1 surpasses all previous state-of-the-art models, though it falls slightly short of O1 on the ARC AGI benchmark.

- Mathematics: Both models perform exceptionally well, with O1 maintaining a slight edge.

- Coding: Initial impressions suggest DeepSeek-R1 is competitive with O1, with its significantly lower cost making it a more practical choice for many applications.

- Creative Writing: DeepSeek-R1 shines in this area, evoking the same excitement as early ChatGPT versions. It's more expressive and notably creative compared to O1.

- Cost-Efficiency: DeepSeek-R1's operational costs are significantly lower than O1, making it a more accessible option for many organizations.

Implications for CTOs and Innovation

The emergence of DeepSeek-R1 as a formidable competitor to OpenAI's offerings holds several important lessons for CTOs looking to drive innovation within their organizations:

1. Embrace Open-Source Solutions

DeepSeek's open-source approach demonstrates the power of collaborative innovation. CTOs should consider how open-source technologies can be leveraged to accelerate development and reduce costs within their organizations.

2. Foster a Culture of Experimentation

DeepSeek's success in developing R1 through novel training methods highlights the importance of encouraging experimentation. CTOs should create an environment where safe experimentation, creativity, and calculated risk-taking are encouraged and rewarded.

3. Prioritize Cost-Effective Solutions

The significant cost advantage of DeepSeek-R1 over O1 underscores the importance of seeking out cost-effective solutions. CTOs should continuously evaluate new technologies that can deliver comparable or superior results at lower costs.

4. Drive Continuous Learning

The rapid advancements in AI underscore the need for continuous learning and upskilling. CTOs should foster a culture of continuous learning within their teams to keep pace with emerging technologies.

5. Lead with Vision

The success of both DeepSeek and OpenAI in pushing the boundaries of AI capabilities demonstrates the importance of visionary leadership. CTOs must stay ahead of the tech curve, anticipating market trends and identifying opportunities for growth.

6. Collaborate Across Boundaries

The global nature of AI development, as exemplified by DeepSeek's emergence from China, highlights the importance of cross-border collaboration. CTOs should seek out partnerships and collaborations that can bring fresh perspectives and accelerate innovation.

Conclusion

The competition between DeepSeek and OpenAI represents a new chapter in AI innovation, one that offers exciting possibilities for organizations across industries. For CTOs, this development serves as a reminder of the importance of staying agile, embracing open innovation, and continuously seeking out new technologies that can drive business value.

As we move forward, the role of the CTO will continue to evolve, becoming increasingly central to organizational success. By fostering a culture of innovation, embracing emerging technologies, and leading with vision, CTOs can position their organizations at the forefront of the AI revolution, ready to harness the power of models like DeepSeek-R1 and O1 to drive growth and create value in the years to come.

The future of AI is here, and it's more accessible and powerful than ever before. The question for CTOs is not whether to embrace these new technologies, but how to do so in a way that best serves their organization's unique needs and goals. As DeepSeek and OpenAI continue to push the boundaries of what's possible, the opportunities for innovation are limitless. It's up to today's CTOs to seize these opportunities and lead their organizations into a future where AI is not just a tool, but a fundamental driver of business success.

Thank you for reading! If you found this article insightful, I encourage you to take the next step in your AI journey.

🤝 I'm Kevin Kasaei, Principal Consultant at PADISO. We help businesses cut through the AI noise and implement solutions that actually move the needle. No theoretical frameworks - just practical, results-driven AI strategies tailored to your business goals.

Want the same results for your business?

🎯 I'm offering free 30-minute consultations to discuss practical applications for your specific situation.

Book a call with me | Let's connect on Linkedin

📫 P.S. Subscribe to my newsletter where I share weekly insights about AI implementation case studies and practical frameworks. No fluff, just results.

Don't miss out on the AI revolution - let's turn these insights into action for your business today!

The drumbeat is getting louder: AI is coming for software engineering jobs. With each new breakthrough – from GitHub Copilot to Claude to GPT-4 to Google's Gemini – the question grows more urgent: Is this the end of software engineering as we know it? After spending months researching this transformation and speaking with dozens of engineers and tech leaders, I've come to a conclusion that might surprise you: Yes, it is the end – but not in the way most people think.

The Great Misconception

The popular narrative goes something like this: AI models are getting so good at writing code that they'll soon replace human programmers entirely. After all, if an AI can pass coding interviews, complete programming assignments, and generate working code from natural language descriptions, what's left for human engineers to do?

This view fundamentally misunderstands both the nature of software engineering and the trajectory of AI development. The truth is more nuanced and, ultimately, more interesting.

What's Really Ending

What we're witnessing isn't the death of software engineering – it's the death of software engineering as we've known it for the past several decades. Here's what's actually ending:

1. The Era of Manual Implementation

The days of manually writing every line of code are indeed coming to an end. Just as calculators freed mathematicians from manual arithmetic, AI is freeing engineers from the mechanical aspects of coding. But this isn't a loss – it's a liberation.

Think about it: How much of your time as a developer is spent on truly creative problem-solving versus implementing well-understood patterns? How often are you copying and pasting from Stack Overflow or rewriting boilerplate code? These mechanical aspects of coding are precisely what AI excels at automating.

2. The Traditional Career Ladder

The conventional career progression from junior developer to senior engineer to architect is being disrupted. The skills that traditionally took years to develop – like memorizing syntax or understanding common design patterns – can now be augmented or replaced by AI tools. This means the career ladder is being compressed, but it's also creating new rungs at the top.

3. The Isolation of Development

The era of the lone programmer, headphones on, coding in isolation, is ending. As AI tools take over more of the implementation details, software engineering is becoming more collaborative, more creative, and more focused on human-to-human interaction and problem-solving.

What's Being Born

As these aspects of traditional software engineering fade away, new paradigms are emerging. Here's what the future looks like:

1. The Rise of the AI-Native Engineer

Just as we have cloud-native engineers today, we're seeing the emergence of AI-native engineers. These professionals don't just use AI tools – they think in terms of AI-first architectures and solutions. They understand:

- How to effectively prompt and direct AI systems

- When to use AI versus traditional approaches

- How to architect systems that can evolve with AI capabilities

- How to ensure safety and reliability in AI-augmented systems

2. The Emergence of Meta-Engineering

Instead of writing all the code themselves, future engineers will focus on meta-engineering: designing systems and processes that AI can implement. This includes:

- Creating robust architectures that AI can work within

- Designing safety constraints and validation systems

- Developing evaluation criteria for AI-generated code

- Orchestrating complex systems of AI agents

3. The Evolution of Problem Solving

The focus is shifting from "how to implement" to "what to implement." Engineers will spend more time on:

- Understanding user needs and business requirements

- Designing system architectures and interfaces

- Ensuring security, scalability, and reliability

- Making high-level technical decisions that AI can't make

The New Skill Stack

To thrive in this new era, software engineers need a different set of skills. Here's what the new stack looks like:

1. Prompt Engineering and AI Interaction

- Understanding how to effectively communicate with AI systems

- Knowing how to debug and improve AI outputs

- Being able to combine multiple AI tools effectively

2. System Design and Architecture

- Designing systems that can leverage AI capabilities

- Understanding the limitations and trade-offs of AI tools

- Creating robust and maintainable AI-augmented systems

3. Human Skills and Business Acumen

- Communicating effectively with stakeholders

- Understanding business requirements and constraints

- Managing teams of humans and AI agents

- Making strategic technical decisions

4. Safety and Reliability Engineering

- Ensuring AI-generated code is secure and reliable

- Developing testing and validation frameworks

- Creating safeguards and fallback systems

The Winners and Losers

This transformation will create both winners and losers in the industry. Here's how it breaks down:

Winners Will Be:

- Engineers who embrace AI as a powerful tool rather than fighting it

- Those who focus on high-level problem solving and system design

- Professionals who develop strong communication and collaboration skills

- Engineers who understand both technology and business needs

Losers Will Be:

- Those who resist learning to work with AI tools

- Engineers who only know how to implement without understanding why

- Developers who can't explain their decisions or collaborate effectively

- Those who see coding as mere syntax rather than problem-solving

What This Means for Different Groups

For Current Software Engineers:

- Start learning to work with AI tools now

- Focus on developing high-level design and architecture skills

- Build your communication and collaboration abilities

- Understand the business context of your work

For Students and Aspiring Engineers:

- Learn the fundamentals, but don't obsess over memorizing syntax

- Focus on problem-solving and system design

- Develop strong mathematical and logical thinking skills

- Start working with AI tools early

For Companies:

- Invest in AI tools and training

- Redesign development processes to leverage AI

- Focus on hiring engineers with strong system design and communication skills

- Create frameworks for evaluating AI-generated code

The Timeline

This transformation won't happen overnight, but it's moving faster than many realize. Here's my prediction for the next few years:

2024-2025:

- AI coding assistants become standard tools

- Basic implementation tasks are largely automated

- Companies start restructuring development teams

2026-2027:

- AI systems can handle most routine coding tasks

- New roles emerge for AI-native engineers

- Traditional junior developer roles decrease

2028-2030:

- AI handles the majority of code implementation

- Software engineering focuses on system design and oversight

- New educational and career paths emerge

How to Prepare

Whether you're a current engineer or aspiring to enter the field, here's how to prepare for this new era:

1. Embrace AI Tools

- Start using GitHub Copilot or similar tools

- Learn prompt engineering

- Experiment with different AI coding assistants

2. Develop Meta-Skills

- Study system design and architecture

- Learn about AI systems and their limitations

- Focus on problem-solving and critical thinking

- Build communication and collaboration skills

3. Stay Adaptable

- Keep learning new tools and technologies

- Focus on understanding principles rather than specific implementations

- Build a strong network in the tech community

The Future is Bright

Despite the apocalyptic headlines, the future of software engineering is actually quite bright. We're not seeing the end of the field – we're seeing its evolution into something more powerful and interesting. The key is to embrace this change and prepare for it.

Just as the calculator didn't eliminate mathematicians but rather elevated the field to focus on more complex problems, AI won't eliminate software engineers. Instead, it will free us from the mundane aspects of coding and allow us to focus on more challenging and creative aspects of software development.

The end of software engineering as we know it isn't a tragedy – it's an opportunity. Those who embrace this change and adapt their skills accordingly will find themselves at the forefront of one of the most exciting transformations in the history of technology.

Are you ready for it?

Thank you for reading! If you found this article insightful, I encourage you to take the next step in your AI journey.

🤝 I'm Kevin Kasaei, Principal Consultant at PADISO. We help businesses cut through the AI noise and implement solutions that actually move the needle. No theoretical frameworks - just practical, results-driven AI strategies tailored to your business goals.

Want the same results for your business?

🎯 I'm offering free 30-minute consultations to discuss practical applications for your specific situation.

Book a call with me | Let's connect on Linkedin

📫 P.S. Subscribe to my newsletter where I share weekly insights about AI implementation case studies and practical frameworks. No fluff, just results.

Don't miss out on the AI revolution - let's turn these insights into action for your business today!

In the rapidly evolving landscape of artificial intelligence, the concept of AI agents has gained significant traction. These sophisticated systems, capable of performing complex tasks with varying degrees of autonomy, are reshaping how we approach problem-solving across industries. A recent research post from Anthropic, dated December 20, 2024, provides valuable insights into building effective AI agents. Let's dive deep into their findings and explore the implications for developers and businesses alike.

Understanding AI Agents

Before we delve into the intricacies of building effective agents, it's crucial to understand what we mean by "agents" in the context of AI.

Defining Agents

Anthropic's research distinguishes between two types of agentic systems:

1. Workflows: These are systems where large language models (LLMs) and tools are orchestrated through predefined code paths.

2. Agents: These systems allow LLMs to dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.

This distinction is fundamental to understanding the different approaches to building AI systems and choosing the right solution for specific use cases.

When to Use Agents

A key takeaway from Anthropic's research is the importance of finding the simplest solution possible. They advise against unnecessarily complex implementations, recommending that developers only increase complexity when needed.

Consider the following factors when deciding whether to use agents:

• Task Complexity: For well-defined tasks with predictable steps, workflows might suffice. Agents are better suited for tasks requiring flexibility and model-driven decision-making.

• Performance vs. Cost: Agentic systems often trade latency and cost for better task performance. Evaluate whether this tradeoff makes sense for your specific application.

• Scale: Agents can be particularly valuable when dealing with tasks that need to be scaled up efficiently.

Building Blocks of Effective Agents

Anthropic's research outlines several key components and patterns that form the foundation of effective AI agents. Let's explore these building blocks in detail.

The Augmented LLM

At the core of agentic systems lies the augmented LLM. This foundational building block enhances large language models with capabilities such as:

• Retrieval

• Tool usage

• Memory

These augmentations allow the LLM to actively generate search queries, select appropriate tools, and determine what information to retain

.Key Implementation Considerations:

1. Tailor augmentations to your specific use case.

2. Ensure a well-documented, easy-to-use interface for your LLM.

3. Consider using Anthropic's Model Context Protocol for integrating with third-party tools.

Workflow Patterns

Anthropic's research identifies several effective workflow patterns that developers can leverage when building AI agents. Let's examine each of these patterns:

1. Prompt Chaining

This workflow breaks down a task into a sequence of steps, with each LLM call processing the output of the previous one

.When to use: Ideal for tasks that can be easily decomposed into fixed subtasks. It trades latency for higher accuracy by making each LLM call an easier task.Examples:

• Generating marketing copy, then translating it into a different language

• Writing an outline, checking criteria, then writing a document based on the outline

2. Routing

This pattern involves classifying an input and directing it to a specialized followup task

.When to use: Effective for complex tasks with distinct categories that are better handled separately, and where classification can be done accurately.Examples:

• Directing different types of customer service queries to appropriate processes

• Routing easy questions to smaller models and hard questions to more capable models

3. Parallelization

This workflow involves running LLMs simultaneously on a task and aggregating their outputs programmatically. It has two main variations:

• Sectioning: Breaking a task into independent subtasks run in parallel

• Voting: Running the same task multiple times to get diverse outputs

When to use: Effective when subtasks can be parallelized for speed, or when multiple perspectives are needed for higher confidence results.Examples:

• Implementing guardrails with separate models for processing queries and screening content

• Reviewing code for vulnerabilities using multiple prompts

4. Orchestrator-Workers

In this workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results

.When to use: Well-suited for complex tasks where subtasks can't be predicted in advance.Examples:

• Coding products that make complex changes to multiple files

• Search tasks involving gathering and analyzing information from multiple sources

5. Evaluator-Optimizer

This pattern involves one LLM generating a response while another provides evaluation and feedback in a loop

.When to use: Effective when there are clear evaluation criteria and iterative refinement provides measurable value.Examples:

• Literary translation requiring nuanced improvements

• Complex search tasks needing multiple rounds of searching and analysis

Autonomous Agents

As LLMs mature in key capabilities, truly autonomous agents are emerging in production environments. These agents can understand complex inputs, engage in reasoning and planning, use tools reliably, and recover from errors

.Key Characteristics of Autonomous Agents:

1. Begin work with a command or interactive discussion with the user

2. Plan and operate independently

3. Gain "ground truth" from the environment at each step

4. Pause for human feedback at checkpoints or when encountering blockers

5. Often terminate upon task completion, but may include stopping conditions

When to Use Autonomous Agents:Autonomous agents are ideal for open-ended problems where:

• It's difficult to predict the required number of steps

• You can't hardcode a fixed path

• You have some level of trust in the LLM's decision-making

• You need to scale tasks in trusted environments

Examples of Autonomous Agents in Action:

1. A coding agent resolving SWE-bench tasks, involving edits to many files based on a task description

2. Anthropic's "computer use" reference implementation, where Claude uses a computer to accomplish tasks

Best Practices for Building Effective Agents

Based on Anthropic's research, here are some key best practices to keep in mind when building AI agents:

1. Start Simple and Iterate

Begin with simple prompts and optimize them through comprehensive evaluation. Only add multi-step agentic systems when simpler solutions fall short

2. Focus on Transparency

Explicitly show the agent's planning steps to maintain transparency in its decision-making process

3. Design Clear Agent-Computer Interfaces (ACI)

Invest time in crafting thorough tool documentation and testing. This is crucial for ensuring that your agent can effectively interact with its environment

4. Use Frameworks Judiciously

While frameworks like LangGraph, Amazon Bedrock's AI Agent framework, Rivet, and Vellum can simplify implementation, be cautious of added complexity. Understand the underlying code and be prepared to reduce abstraction layers as you move to production

5. Implement Proper Safeguards

For autonomous agents, extensive testing in sandboxed environments and appropriate guardrails are essential to mitigate risks associated with their autonomy

Real-World Applications of AI Agents

Anthropic's research highlights two particularly promising applications for AI agents that demonstrate their practical value:

1. Customer Support

AI agents in customer support combine chatbot interfaces with enhanced capabilities through tool integration. This application is well-suited for agents because:

• Support interactions naturally follow a conversation flow while requiring access to external information and actions

• Tools can be integrated to pull customer data, order history, and knowledge base articles

• Actions like issuing refunds or updating tickets can be handled programmatically

• Success can be clearly measured through user-defined resolutions

2. Coding Agents

The software development space has shown remarkable potential for AI agents, evolving from code completion to autonomous problem-solving. Agents are particularly effective in this domain because:

• Code solutions are verifiable through automated tests

• Agents can iterate on solutions using test results as feedback

• The problem space is well-defined and structured

• Output quality can be measured objectively

The Future of AI Agents

As we look to the future, it's clear that AI agents will play an increasingly important role in various industries. However, the key to success lies not in building the most sophisticated system, but in creating the right system for specific needs.By following the principles and patterns outlined in Anthropic's research, developers can create agents that are not only powerful but also reliable, maintainable, and trusted by their users. As the field continues to evolve, we can expect to see even more innovative applications of AI agents, pushing the boundaries of what's possible in artificial intelligence.In conclusion, the journey to building effective AI agents is one of careful consideration, iterative improvement, and a deep understanding of the task at hand. By leveraging the insights from Anthropic's research and staying attuned to the evolving landscape of AI, developers and businesses can harness the true potential of AI agents to solve complex problems and drive innovation across industries.

Thank you for reading! If you found this article insightful, I encourage you to take the next step in your AI journey.

🤝 I'm Kevin Kasaei, Principal Consultant at PADISO. We help businesses cut through the AI noise and implement solutions that actually move the needle. No theoretical frameworks - just practical, results-driven AI strategies tailored to your business goals.

Want the same results for your business?

🎯 I'm offering free 30-minute consultations to discuss practical applications for your specific situation.

Book a call with me | Let's connect on Linkedin

📫 P.S. Subscribe to my newsletter where I share weekly insights about AI implementation case studies and practical frameworks. No fluff, just results.

Don't miss out on the AI revolution - let's turn these insights into action for your business today!