Thinking of AI Agents? You Need A Conductor!

If you’re drowning in the hype around agentic AI or being hammered by a salesperson from yet another AI startup about why you need their flavor of automation, take a breather and think about this instead: who is going to wrangle these little fiends you’re about to unleash upon your business?

\ I’ll give you a hint, if you love the theatre or listening to an orchestra the chances are you’ll have seen one of these nutters in action.

\ You need a Conductor.

\ The definition of one and what they do is pretty simple, “the conductor is the leader of an orchestra or choir. They use specific gestures to direct individuals and sections of an orchestra or chorus through a piece of music in rehearsals and final performances. They instruct in aspects such as tempo, volume, tone and pitch to achieve the desired performance of the piece. Conductors may also be directors, or else work closely with directors to ensure the desired performance is achieved.”

\ Why would you need even more middleware on top of buying an agentic platform? Well, the answer is, “Because it’s fucking complicated.”

\ For starters, at the very minimum, your business processes, business rules, compliance protocols, and aligning data should be defined to a reasonable standard that an AI system can understand and follow in any sense of autonomy and know where the boundaries and touchpoints are for handing off internally, interacting with customers or clients, and how and when to end a process rather than carry on endlessly spawning more tasks and smaller agentic actions ad infinitum.

\ Agents will be both horizontally and vertically aligned according to industry or functional needs.

\ And that also means multiple Agentic platforms need to integrate in order to complete one task or process, there are already startups that are just picking away at specific problem areas but not holistically looking at the bigger organizational picture.

\

\ Aaron Levie, CEO of Box, alluded to this in a post recently on LinkedIn but it’s all for naught without something to direct them and keep the music flowing.

\

AI Agent interaction is going to be one of the most interesting software interoperability paradigms of the future.

\ Inevitably, no one software system contains all the knowledge or information to perform all the tasks that an enterprise or users needs. This means we’ll need AI Agents to coordinate and do work together.

\ AI agents won’t be the ones coordinating how they interact with one another; they can’t be left to their own devices because they don’t know when to stop, there is no orchestration in executing instructions, no staying within their lanes. We’ve seen this happen with various open-source projects before where agents just continue to spiral into a rabbit hole, spinning off more agents until the original task is lost and the result is useless.

\ The diagram below shows hints at what orchestration could look like but it’s still a technical implementation, ignoring any business context that exists in an organization. Years of Knowledge Management and Business Process Management programs are ignored just because the perspective is from an application-centric point of view.

\ Let’s try to redefine what a Conductor is in the age of AI then.

\ “The Conductor is the orchestrating intelligence that coordinates multiple AI agents through complex workflows while ensuring strict adherence to predefined business rules, data regulation, and compliance standards.”

\ It’s a starting point. If we look at the diagram above, each agent will have been pre-trained on existing business processes and task definitions, they will understand to a degree what their function is and where their tasks start and end. How they execute and resolve these tasks will be fairly loose because what you want is to harness the apparent intelligence within an AI solution to prevent having to constantly deal with exceptions.

\ I see this as something becoming a thing of the past, whereas before having to try to capture and implement in applications as many scenarios as possible to eliminate manual processing.

\ But we can’t have them running amok.

\ And so the Conductor exists.

\ The Conductor (yeah ok, it’ll be called an AI Orchestrator by the industry, or Orchestration Layer, or AI Middleware, etc. but stick with me) is where the sheet music sits. They will direct this symphony of agents but also will execute specific instructions in order to maintain compliance.

\

For instance, you may ask Salesforce a question about a customer and an Agent will combine in an answer from Agents that review contracts in Box or billing info in Stripe. Or, you’re onboarding as a new employee and you ask an Agent a question in ServiceNow, which fans out to HR documentation in Box or data in Workday. Or you want to build software with Replit or Devin, and the Agent talks to Agents in Box for product specs, project plans in Asana, or design assets in Figma.

\ Levie continues citing a few examples, and in each case, there are multiple agents at play but there are multiple complications too. They all are cross-functional processes at work, each agent within these workflows will have their own sets of rules, boundaries, authorizations, permissions, etc., and tracking them all will be a nightmare including whether or not the task was completed as instructed or needed more ‘prompting’ to get there. So, the idea of a Conductor starts to get more complex.

\ Using strategic prompts and control mechanisms, the Conductor would direct individual AI agents to execute specific subtasks, synchronize their efforts, and achieve the desired overall performance of a given objective. And you have to remember that we are just talking about one process execution instance here, now consider that tens, hundreds, or thousands of employees are still working in an organization prompting tasks to be executed in a multitude of ways with varying instructional nuance.

\ No two prompts are the same, and no two AI results are ever the same either.

\ The Conductor then has to dynamically adjust agent parameters such as context, instructions, scheduling, and API protocols to optimize any collective problem-solving and result quality, and ensure that the end result is actually the desired one.

\ This orchestration layer is where you maintain a grip on your business and your evidence of compliance. I would not trust one single agentic platform to act as some sort of golden source of truth, they’re all black boxes.

\ If you imagine an orchestra trying to play a piece of music without a conductor you get the picture of multiple agents, from multiple agentic platforms, all trying to compete for resources and data without any synchronization, harmony, or adherence to what defines your business. A Conductor provides that harmony, it is, if nothing else, an AI supervisor, another middle management layer albeit an artificial one.

\ In the end, orchestration will become absolutely key and critical to any AI implementation whether agentic or not. There’s barely a mention from the major vendors about this, and it's a worrying sign because they don’t fundamentally understand their own platforms they’re trying to push, let alone someone else they need to integrate with.

\ And without orchestration will you even have any understanding or control over the true cost of using multiple agentic systems? You can kiss your ‘savings’ goodbye and may as well stick to hiring humans.

\ I mean, look at this list of pricing models and you get the picture.

\ EvenUp: Per-demand package generated by AI \n CaseMark: Per work product delivered

OpenAI: Per input/output token (GPT-4o) \n OpenAI: 10x higher pricing for ChatGPT Pro \n 11x: Per task completed by the AI SDR \n Kustomer: Per conversation (no seats) \n Salesforce: $2 per conversation (Agentforce) \n Sendesk: Per successful autonomous resolution (Sendesk AI) \n Microsoft: $4 per hour of usage (AI copilot for security) \n Microsoft: Pay-as-you-go (Copilot Chat) \n Clay: Per credit with a credit = a data point or action \n Copy.ai: Per workflow creditReplit: Hybrid pricing (seats with monthly usage credits, PAYG upgrades) \n Codeium: Hybrid pricing (seats & monthly usage credits) \n AiSDR: Based on the number of personalized emails \n Warmly: Hybrid pricing (features & # of warm leads)

Sierra: Outcome-based (completed tasks) \n Intercom: $0.99 per AI resolution (FinAI agent) \n Intercom: 10 free tickets per agent, per month (FinAI copilot) \n Relevance: Credits-based subscriptions \n Relay.app: Per workflow step \n Bardeen AI: Per automation credit \n Sapier: Per task automated \n Chargeflow: 25% per successful chargeback \n Synthesia: Per minute of video \n Imagen: Per AI photo edit

Codeium: Hybrid pricing (seats & monthly usage credits) \n DeepL: Per user & editable file translation \n Asana: Based on platform credits (AI Studio add-on) \n Aftershoot: Unlimited AI photo edits \n Evoto: Per AI photo retouched \n Adobe: Based on GenAI credits \n Cognition: Per agent compute unit (Devin) \n Cursor: Hybrid pricing (usage paywalls: fast premium requests & o1-mini uses)

Captions: Per AI video generation credit \n Kittl: High watermark of AI credits per day

\ One more thing to consider. Does the Conductor sit within your organization or outwith? That is, are you basing this orchestration in the Cloud or on-premises? Because this opens up more questions that nobody has an answer for.

\ And you most certainly should not be buying an orchestration layer from the same vendor you’re buying an agentic solution from.

\ You need that separation.