Hierarchies of Gen AI agents for solving valuable problems

What do we know about AI in general? It’s great at mimicking things that have been done. It’s not so great (yet) at inventing new ways to do (most) things. This means we still need humans to solve new problems, and there are infinite new problems.

So, at least until AGI nirvana is reached at an unknown point in the future, it is unlikely that a general-purpose AI will be able to solve all of a person’s problems or a business’ issues.

So the human problem solver will be combined with tech to solve issues, and as the capabilities of technology and the problems that need to be solved progress, this frontier will expand. This may sound familiar.

My thesis is that software is not dead, despite the publicity. And point solutions will remain necessary.

What does this have to do with hierarchies of agents?

First, let’s look at a naturally occurring multi-agent system - human businesses.

An oversimplification of a business is a hierarchy of agents with the following characteristics:

• agents specialize in different tasks

• the CEO (agent at the top of the hierarchy) is the chief orchestrator of the execution of the company’s objective

• middle management prioritizes work coming from above, and manages agents below

• agents at the lowest tiers work on their individual tasks

• an agent’s work exists within a scope defined by an agent at a higher tier

• agents interact with peers to complete their jobs - either direct peers or by messages traversing through the hierarchy to indirect peers

Now, is a business a single large agent that’s multifaceted, or are individual agents working as a collective? Mostly semantics, at least in my oversimplified description.

Similar to software design macro vs. micro architecture - useful concepts but in reality most solutions are somewhere along a gradient.

Bringing it back to Generative AI products…

I think the above model can be applied to the mature Gen AI products we’ll see from 2026 onwards. Up until ~now the products that have scaled are chatbots. Chatbots are terrible products for most use cases. But they were low-hanging fruit for getting cool new products to market.

2025 is the year of actually good generative ai point solutions. Agents that do the thing they're supposed to, with little or no manual input from the user.

Once these products work well and scale, we’ll be looking at an orchestration layer above - circa 2026. Right now, there are a lot of products that claim to solve multiple use cases and orchestrate multiple agents to solve a cluster of similar problems for your business but in reality, they might solve a single problem well.

Bringing it back to the thesis

A general pattern that I believe will hold true as Gen AI products evolve: Humans solve a problem, the solution is AI-ified, and the AI solution is added as a bottom-tier agent in a hierarchy. Agents of similar functionality are grouped and managed by an orchestration agent “middle management”. There may be multiple layers of middle management. A human only interacts with the top-level agent - the “CEO” - to set an objective, and the top-level agent sets its workers in motion to accomplish this task.

As there are infinite new problems, and AI is not (yet) good at solving brand new problems - humans and point solutions remain a necessary part of the system for the foreseeable future.

The biggest question mark for me is how trivial does it become to teach a general-purpose AI to solve a brand-new problem? Does it need agents? I think so. The goalposts may move but you’ll always need a frontier point solution for complex problems - even if the definition of complex evolves.

Or we get AGI…

Additional note: Worth noting that one characteristic of multi-agent systems that I haven’t mentioned is that they can speak to each other with “fuzzy” APIs, which is an interesting concept and might mean traditional API products have a limited lifespan.

James is the co-founder and CEO of Coso.ai - the social media co-pilot for businesses without a social media expert.

Reach out to james @ domain to discuss this post or anything else.