Will AI Agents Run Amok Across Business Information in 2027?

0
18

Document intelligence expert John Bates pictured is concerned about how readily organizations are  giving AI systems access to critical business repositories

As businesses navigate the year-end rush, it’s easy to lose sight of what lies ahead. Against a backdrop of economic uncertainty, AI-driven superabundance and fears of job losses and market turmoil, here’s what I expect to see in the run-up to 2027 and beyond.

Tokenmaxxing loses its appeal

2027 will mark a shift away from making general-purpose LLMs bigger and toward purpose-built AI systems that harness models for specific tasks. Clearly, we are on the brink of the next strategic inflection point in AI, where tokenmaxxing, i.e., trying to use as many AI tokens as possible, is recognised as a failure and something better is needed. 

Join The European Business Briefing

New subscribers this quarter are entered into a draw to win a Rolex Submariner. Join 40,000+ founders, investors and executives who read EBM every day.

Subscribe

What that looks like will, I believe, look much like a harness or framework around a model—a platform built around an LLM and purpose-built for a particular task. Our internal experiments and our ongoing conversations with customers suggest this topology is, on average, at least six times more productivity-yielding than a raw LLM. 

In other words, the future will increasingly bring a synthesis between core workflows, business content systems such as ECM, and AI. What is emerging is more accurately described as ‘document intelligence’: moving beyond simply storing and retrieving documents to using AI to understand the information within them and proactively derive business insight.

AI agents get governance, permissions and accountability

As agents become more autonomous, 2027 will see a major rise in AI governance and mechanisms for supervising agents at scale. We may soon have whole departments or business units effectively run by AI in the shape of agents; maybe even some small companies. At that scale, what does adequate supervision look like? 

At this agentic or AI scale, there’s a whole new market segment now coming to the forefront around AI governance and AI and what might be called AI ‘caretaking’. This means putting strong internal guardrails in place around what agents can do, which data they can access and which applications they can use. We can’t rely on model makers to do this for us.

Without these safeguards, how can we stop an agent running amok with salary data, confidential IP or other sensitive information?

An AI agent causes a business-ending incident

Over the course of the next few quarters I fear an autonomous AI agent will cause a sufficiently serious failure that it becomes a major corporate cautionary tale, potentially contributing to the collapse of a business. 

I’m hesitant to make this prediction, but I think we’re moving too fast. There is close to a 100% probability that we will read headlines about an agent-related issue that will cause a business to explode, trigger a business-ending event, or at the very least cause some serious corporate embarrassment for a major brand or public institution. 

Ultimately, this comes down to the problem of autonomy creep: the gradual expansion of what an AI agent is allowed and able to do without human approval. The danger is that as approval steps disappear, agents gain greater access and their practical authority expands without corresponding oversight. At that point, chaos might become inevitable—like it or not.

The human user interface is evolving

In 2027, enterprise software will increasingly be designed for AI agents, not just directly for human users. When you want to do some banking, do you go to your bank’s website, or just do it in seconds on your phone? Agents are, in this analogy, the mobile phone: they’re there to get things done as fast as possible. For them, it’s all about MCP and the ability for an agent to talk to whatever service it needs to get a result.

That creates a fascinating new direction for UX: our most important user is no longer necessarily human. 

Good user interfaces won’t disappear, but the nature of UX is changing because humans are no longer always interacting directly with the form or system behind it. So perhaps we’ll see entirely new ways of interfacing with business systems, with speech, for example, becoming much more of a default in business IT than it has really ever been.

Europe back in the AI race?Contrary to the prevailing view that the US and China have left Europe behind in AI, I think Europe could make a meaningful comeback in AI in 2027, driven partly by sovereignty and distrust of foreign models. Geopolitical realignments may mean that European organisations increasingly don’t want to put their data into American or Chinese models. 

I expect a number of previously quiet, perhaps even pan-European, projects to emerge, with European LLMs that are ‘good enough’ gaining significant traction. But I have to say that none of these trends will do much to slow the US’s remarkable progress in AI. Its huge capital base (US private AI investment was $285.9 billion last year, 23 times China’s), leading AI companies and unrivalled data-centre infrastructure give it a formidable head start, making it overwhelmingly likely to remain at the centre of the AI race for the foreseeable future.

LLMs move onto the phone

Rather than relying entirely on specialist data-center hardware, could 2027 be the year we see the first widespread LLM running directly on a smartphone? While Apple hasn’t really engaged seriously with AI yet, 2027 will mark the 20th anniversary of the iPhone, so what better moment to make a quantum leap? 

An LLM that runs on mobile phones could be a real coup for Tim Cook’s successor, cap Steve Jobs’s legacy in one stroke, and even be great for consumers too.

Business software in space

AI has created such an extraordinary demand for power and computation that companies are beginning to investigate space as another data-center location. If Elon Musk’s ambitions play out and he starts laying the foundations for data center computing in low Earth orbit as early as this year, then like the proverbial horse and carriage, he’s going to need proper software to run up there. 

And document intelligence is a good place to start.

Turbulent times lie ahead, but one thing is certain, whatever 2027 brings, it won’t be dull.

The author is the CEO of Doxis, a leader in document intelligence

LEAVE A REPLY

Please enter your comment!
Please enter your name here