Rogue AI agents selected without adequate strategy, governance, and validation should be a major concern for CIOs.

Major SaaS providers, hyperscalers, LLM providers, startups, and open source projects are announcing capabilities for developing and configuring AI agents. Most organizations will operate AI agents across different platforms from the outset. The transformation will be unlike how IT departments gradually ended up in multiple clouds over time.

The inevitability of running AI agents on various platforms raises several questions.

• How should CIOs develop a strategy and governance around where they will buy and build AI agents?
• What are some of the considerations for centralizing one platform vs. multiple?
• What can CIOs learn, or not learn, from the pros/cons of being cloud vs. multicloud?

Chris Mahl, CEO, Pryon, says,  “CIOs making AI agent decisions today are repeating every mistake from the early cloud era. The temptation is to let every department buy their favorite AI agent platform—but that creates the same knowledge friction we’ve spent decades trying to solve.”

A balanced AI governance strategy encompasses defining strategic goals and establishing usage guardrails. CIOs should specify data requirements, outlining validation responsibilities, and establishing a tool landscape. Communicate an agentic AI operating model, clarify to employees the requirements, and prevent shadow IT evolving into rogue AI agents.

“Smart governance means establishing data foundation standards first, then choosing 2-3 core platforms maximum,” adds Mahl. “Unlike cloud infrastructure, AI agents share organizational knowledge, so fragmentation isn’t just inefficient—it’s dangerous. The multicloud lesson is that vendor diversity has value, but data consistency is non-negotiable. Build where you have unique IP, buy where you need scale, and govern everything through unified data access controls.”

Planning strategic AI Agents

There are two classes of agents that CIOs should plan for, according to Christian Buckner, SVP of data and AI platform at Altair.

• Off-the-shelf agents come built into enterprise software – turnkey tools that automate narrow tasks with minimal friction. 
• Custom-built agents, although more challenging to implement, are where real transformation occurs.

“Building an AI agent is like onboarding an employee; they need to operate across systems, understand enterprise context, and learn over time, while scaling them is like hiring hundreds,” adds Buckner.

Most organizations should expect to support both types of AI agents. Most organizations will consider AI agents in customer experience, field operations, and capabilities supporting the future of work.

“The real power comes from building your own agents that orchestrate across all your systems, encoding your unique business logic as intellectual property that competitors can’t just buy off the shelf,” says Florian Douetteau, CEO of Dataiku. “You need a platform that gives you the freedom to use any model, connect to any system, and most importantly, the ability to evolve your agents as your business changes – not wait for some vendor’s roadmap to maybe address your needs next year.”

With great power comes great responsibility

Here’s what CIOs face. There are multiple data sources, LLM providers, and AI agent platform. Across the orginanization, development teams conduct POCs and employees are experimenting. The impact is this can turn last generation’s messy data landfills into raving rogue AI agents.  

“The risk people see here is ‘agent anarchy,’ where individuals and departments deploy agents without coordination, similar to what we witnessed with shadow IT during early cloud adoption,” says Dominic Wellington, director of product marketing at SnapLogic.  “CIOs need clearly defined AI governance frameworks, ideally anchored in strong API and integration management, including observability of agents, in order to deploy these new capabilities with the right mix of speed and control. In times of business uncertainty and rapid technological evolution, it’s better to focus on a rapid and agile approach, building on capabilities that already exist and leveraging them to deliver new capabilities to users.”

Agentic AI ecosystem versus spawl, shadow, and rogue AI agents

CIOs must avoid the culture that allowed SaaS sprawl and shadow IT. Communicate AI governance before rogue AI agents are everywhere, and anyone can invite any AI agent into the company’s environment.

“Businesses are going to deploy hundreds of thousands, if not millions, of self-directed agents that can adapt and learn, says Steve Lucas, CEO of Boomi. “Governance protocols that enable visibility, oversight, security, and administrative functionality across all AI agents are critical in order to thrive in this new era of AI-driven automation.”

CIOs should apply some of their key multicloud learnings to AI agents. IT will need monitoring, observability, and security tools that work wherever AI agents operate.

 “When strategizing agent governance, CIOs should consider the ‘single pane of glass’ approach for managing AI agents centrally, which helps with oversight and security,” says Miles Ward, CTO of SADA. “Remember, unified platforms, but be open to the variety of models, tunings, and prompt techniques, because, like with cloud, we’ll likely find strengths and niches in different agents. Non-negotiable is a clear framework for agent deployment and monitoring that integrates with your current security stack.”

Governing the AI agent ecosystem

Below are suggestions from several experts on how CIOs should prepare for the distributed AI agent ecosystem.

1. Establish a continuous cycle of data discovery and security to ensure ongoing protection. “CIOs must approach multi-agentic AI by prioritizing continuous data discovery and protection over platform selection, as these agents often operate with elevated access to sensitive information, posing risks exceeding traditional shadow IT,” says George Gerchow, CSO of Bedrock Security. “Shift from static compliance models to dynamic, metadata-rich frameworks enabling visibility and control across environments.”
2. Assess data quality, infrastructure, and compliance with governance standards. “AI agents are only as powerful as the data infrastructure behind them,” says Anjan Kundavaram, chief product officer of Fivetran. “Whether you build or buy, success depends on having all your data in one place, governed, reliable, and ready in real-time. Just as enterprises learned the risks of cloud lock-in, they now need to ensure their AI stack stays flexible, interoperable, and built for scale.”
3. Define AI architecture and integration principles. “The key is not to chase every new agent but to build a governance-first strategy aligned with business outcomes,” says Jay Upchurch, CIO of SAS. “CIOs should evaluate AI agents not just on capabilities, but on how well they integrate into a unified, secure, and explainable architecture. Can agents communicate across systems and data silos? Are policies enforceable across environments? Success hinges on building agentic ecosystems that are not only powerful but also principled, prioritizing accountability and trust.”
4. Define governance for AI agents. “Treat AI agents like employees and track what they access, how they behave, and what decisions they make,” says Jimmy Mesta, co-founder and CTO  of RAD Security. “Ensure you have a well-defined AI data governance policy that has strong oversight on data usage, ensures that model behavior aligns with company values, and regulatory frameworks. A strong AI governance policy makes experimentation and learning easier, allowing you to stay current with market developments.”
5. Separate AI strategies for workflows versus CX and innovation use cases. “CIOs can utilize an AI collaboration matrix, aligning team skills with task importance, buying agents for usual tasks, and building for areas of unique expertise,” says Jeff Foster, director of technology and innovation at Red Gate. “Centralizing on a single AI platform can unify workflow and governance, while using multiple platforms sparks innovation. While it is easy to switch between similar core services of mature cloud market providers, AI agents make the transition between platforms costly and limit interoperability.”
6. Establish baseline criteria for platform selection before experimentation. “CIOs must take a step back and evaluate what AI tools serve what purposes, and how multiple tools can work together simultaneously for a harmonious outcome,” says Rodrigo Coutinho, co-founder and AI product manager at OutSystems. “Considerations such as cost, use case, data privacy, and compliance will be crucial in picking the right AI partner or deciding to build in-house.”
7. Create a decision framework and manage the full AI agent lifecycle. “Map agent use cases against two axes: strategic differentiation and data sensitivity,” says Srujan Akula, CEO of Modern. “Build where the logic is core to your business or deeply tied to proprietary data, and buy where speed, scale, and commoditized capabilities matter more than control. Treat agents like autonomous systems needing lifecycle management, audit trails, escalation paths, and guardrails for hallucination and drift.”

Bring the human element to the forefront

What often gets overlooked in the conversation about AI agents is their impact on people. CIOs need a people plan and should consider these 25 GenAI emerging roles. Also, create a change management plan before accelerating the deployment of AI agents.

There’s no doubt a new wave of transformation and disruption is upon us. CIOs need to restate their digital transformation strategies and pave the path for developing leaders in the genAI era.