A Dark Turn in Coding

on May 1, 2026

A Trillion-Dollar Frenzy

Like a kangaroo on mescalin everything related to the AI world is oscillating between an uncontrollable splurge for expansion just as Mister Creosolt’s waist in Monty Python’s Meaning of Life, with the massive consequences of an almost unlimited feeding of the machine and its explosion of output (I’m not entirely happy with the image that metaphor plays out). In one direction is the financial momentum, the calls for uncontrolled investment, evident when we saw Anthropic’s release of plug-ins for Claude Cowork trigger a staggering “trillion-dollar” sell-off. Investors gripped with a kind of mania calling for the total obsolescence of the traditional Software-as-a-Service (SaaS) giants, like the many moneyed David’s throwing fiscal rocks at the Goliaths. In this frenzy of land-grabbing and revenue targeting even Sam Altman’s OpenAI has apparently jettisoned the non-profitable “side quests” to focus on enterprise-grade coding tools that are grabbing the headlines and the cash flow.

Beneath this feeding frenzy a “grim truth” is beginning to sharpen its cutting surfaces. While the rush to automate production, creation and maintenance grows so does an abyss between the promises of “frictionless efficiency” and the reality of what is actually being shipped. We are standing at the dawn of an uncertain age. One that has the promises, or maybe not, of perfect automation. But also one that contains murky seas polluted with unverified, hallucinatory architectures and a confusion of responsibility.

The problem has become undeniable. In the race to optimize we have compromised the authority of our digital world.

It might look good, but it is bad work

There is a deceptive enticement to AI-generated code, Morpheus offering us a pill choice updated to the modern world where red-pilling is bad. To the casual prisoner, or the overtasked tech employee, it can be indistinguishable to professional-grade software. It has created a land where industry leaders, and the meme lords in their bro-palaces, might declare no difference to anything a human created, even boasting an overall improvement. Those claims can be true. Machine generated/constructed output is usually syntactically precise in a flash, but it hides logical rot and cul-de-sacs, mysterious loops like a bowl of sphaghetti and redundant functions.

This is multiplied when we use the well known and lamented failure that is too often deployed in legacy metrics from uneducated managers and over-enthusiastic project coordinators. The industry has unwisely measured productivity with “lines of code”, “event attendance” or “pull requests”, the lie of quantity, or representation, equalling quality. For machine-generated output these metrics aren’t just inaccurate, they are downright dangerous. You applauded a toddler who tossed a grenade into the pool party for the distance the bodies flew. When an LLM can generate thousand of lines of code, create oodles of functions in the time it takes an engineer to sip their coffee, volume becomes a turd avalanche of “technical debt.”

This is an even more tragic understanding to every good development team. They know it isn’t how much you put in that matters, it is knowing what you can take out. The best pull requests are the ones that optimise and reduce the code overhead. The best meetings those where you discuss extractions.
Developers are now forced to “fly by the seat of their pants,” frantically engaging new AI tools which seem to metastasize in order to verify the code of the original ones.

We have run blindly into a closed loop. A repeat of reality where fallibility of the underlying process is never interrogated. Without the reasoning gained from an understanding of the whole business (the domain model) which we couple with the developers internal monologue (self assessment and internal judgement) it feels impossible to verify trust. These systems are merely automating the appearance of being right while eviscerating the team understanding that helps sustain a product and make the business thrive.

Experiments Can Cause Real Outages

The dangers of unverified code are no longer confined to theoretical examples, this is true for humans and machines. It was bad enough when we were stuck in the dystopian lands of “move fast and break stuff” but now we have moved on to a procedurally generated island surrounded by sharks, with cannibalistic natives eating any code that disagrees while playing dice with corporate environments.

The recent emergency at Meta which began as a software engineer using an in-house agent devolved into a heady brew of AI hallucination and a dangerous “game of telephone.” The AI provided inaccurate information, this was then acted upon by a human employee who made the mistake of trust without secondary verification. It all resulted in a critical security breach that lasted for nearly two hours. Meta shifted almost all the blame to “human error,” illustrating the growing burden we are placing on those with ungoverned use of AI. It was a human error, some human decided we didn’t need to verify what the machine thought was truth.

An island surrounded by sharks, with the natives being AI code generators that are eating any code that disagrees while playing dice

” It was bad enough when we were stuck in the dystopian lands of “move fast and break stuff” but now we have moved on to a procedurally generated island surrounded by sharks, with cannibalistic natives eating any code that disagrees while playing dice with orporate environments.”

Amazon faced similar issues as outages at its online retail business were described in internal notes as having a “high blast radius,” citing “Gen-AI assisted changes”. In one instance at AWS an AI tool deleted and then attempted to recreate an entire coding environment. These aren’t just one-off glitches; they are failures born of “novel GenAI usage for which best practices and safeguards are not yet fully established.”

A Million-Line Backlog

Companies can drown in “code glut,” the surge in output that creates a deluge of unexamined logic, functions, operations and libraries. Often when the final code that is generated is revised, re-revised and checked only by other AI-assisted systems there is no insight that a human can comfortably gain to have any authority, or any real responsibility. There are reports of firms that see manifold increases in coding output after adopting AI tools resulting in staggering bottlenecks of code that can be a million lines in length. Code that no human has actually read, let alone understood.

This is an engineering crisis. A weight that will eventually affect sales and marketing teams who are forced to sell products built on potential vulnerabilities. We are building systems that no human actually understands, making them potentially unfixable when they fail.

The Watcher Scarcity

In a move of spectacular corporate dissonance the tech industry is laying off its valuable employees, while simultaneously demanding more oversight. Companies have announced thousands of redundancies touting “pivots to AI,” as the principal cause, unaware that AI requires more human vigilance, not less. This has created a scarcity that undercuts the very cost-cutting promise of AI. At Amazon, the response to AI-led outages was a mandate that every AI-assisted change be signed off by a senior engineer creating a massive productivity drag: if every line of “cheap” automated code requires a high-priced senior human to vet it, the efficiency gains evaporate. Even Microsoft have made statements declaring that material produced with Co-Pilot is not fit for business or production. So what is the value of all the integration of these tools if we cannot have any trust in their product?

And, we are facing a talent vacuum that no algorithm can fill. We are firing the builders while the few remaining inspectors are spread thin across near epoch-level layers of unreviewed code.

Deep-Fried Brains of the AI Feedback Loop

The scarcity of oversight is a bodily crisis leading to “brain fry”. The engineers who remain can see the nature of work has shifted joy of creation to the drudgery of constant supervision. Too many people are reporting working harder and producing more but there is no coresponding increase in investment that matches this. The most succesful AI integrations see an upturn of a few percent for a significant input.

Engineers are no longer problem-solvers, they are safety nets for a tool that can hallucinate, and lie, at any moment. Locked in a state of vigilance like a cocaine-fueled roadrunner with ADHD constantly forced to play a deranged game of whack-a-mole to “spot the error” in a spew of AI-generated text (we’re back to Mister Creosolt just as he has finished the “wafer thin mint”). We are potentially facing a torrent of unprecedented burnouts. The human element is being squeezed into non-existence like the last bit of liquid in a tube, forced to act as the cognitive glue for a system that doesn’t understand the rules of the game it’s playing.

a cocaine-fueled roadrunner with ADHD who is being forced to play a game of whack-a-mole surrounded by a deluge of AI-generated code

Locked in a state of vigilance like a cocaine-fueled roadrunner with ADHD constantly forced to play a deranged game of whack-a-mole to “spot the error”

Calling Time

We are at a critical point where the marketing of AI coding has far outpaced its technical reality. The problems of LLMs remain unsolved. They lack specific reasoning (inductive or whole domain related); they do not reliably retrieve facts; and they possess no internal monologue, the anachronistic internal dimension (“creatures from the ID”), to check their own work. They are engines of mathematics with logic based on vectors not generators of truth. We could dismiss outages and backlogs as “growing pains,” but reality looks more like inherent failure, a tale filled with more plot holes and inconsistencies than a Fast and Furious sequel. We are currently building the world’s most critical infrastructure on a shaky foundation of potentially “hallucinatory architecture.”

Every tech and business leader must confront a question that no benchmark will answer: Can you trust your whole software environment, your user data, your company’s future to a system that by design doesn’t know if the answer it gave can be trusted?

Sources

AI Declaration

The author used NotebookLM for research and collation/searching of articles and Gemini’s NanoBanana image generation model for the images in the creation of this article.

Categories:

AI Software

Tags:

AI Business Development Environments Software