There are LLMs answering questions, vibe-coding tools building prototypes, and simple role-based workflows that AI agents are taking over. Then there is mission-critical process management, where making mistakes can be costly and create regulatory issues.  

Should an insurance company leave it to an autonomous AI agent to decide which claims to approve? What should AI’s role be in reviewing loan applications? Should procurement officers outsource the final decision on whether to approve requests and where to source from?

These are the strategic questions Matt Calkins, CEO of Appian, shared at last week’s Appian World. Calkin says that large enterprises in regulated industries that require error-intolerant processes must pursue “serious AI” built on defined business processes.

He categorized Appian’s AI approach as “East-Coast AI.” AI companies touting fully autonomous, agentic AI capabilities may be focusing on much simpler use cases.

Here’s what this alternative, non-East-Coast AI vision sounds like. Build role-based AI agents that can be fully autonomous. Insert the human in the middle, but slowly remove this oversight as people trust the AI’s results and recommendations. More complex workflows are then orchestrated auto-magically through coordinated AI agent interactions.

It sounds too good to be true, and it very well may be for mission-critical workflows. As Calkins explained to me, errors in a network multiply as they propagate, so AI-driven recommendations are less accurate and more expensive when AI agents are tasked with performing a complex, multi-step process and are left to fend for themselves.

Calkins proposes a more pragmatic AI approach to critical process management in the AI era. Here’s what CIOs need to know.

Serious applications start with a spec

CIOs should review their first principles with their teams. Does an agile development team start coding an application with only a static mockup? Or do they go through a process of documenting requirements, non-functional acceptance criteria, and agile user stories? If you don’t support coding off an idea and a diagram, then maybe vibe coding isn’t for you. Advise your teams to consider spec-driven development.

Appian’s vision goes beyond low-code development to AI-assisted development based on spec-driven practices. Appian Composer is their tool for business teams to go from specs to applications, while developers can use Claude, Codex, or Kiro for more significant development efforts.

Why this matters: AI-developed apps require collaboration

The problem I have with vibe coding is that the experience is designed for a one-on-one collaboration between a developer and the AI. So if a product manager wants to connect requirements to the AI’s implementation, or a security professional wants to validate whether DevSecOps non-negotiables were implemented, they have to assert themselves into the vibe coding platform/process. Spec-driven development allows a team to create business and technical specifications with AI and review them before the AI starts coding off of them.

Critical workflows are defined, then AI-enabled

Workflow design starts with an objective, which is then broken down into a flow of activities, task responsibilities, and exception-handling rules. Continuous improvement starts by measuring time-on-task, error rates, and other operational metrics, then applying technology, training, and other improvement methods.

Now, add AI in a process or AI agents as evolutionary techniques. In one example at Appian World, a loan intake step in a loan origination process was improved by connecting a fraud AI agent, property search capability, and data from the customer ledger. 

“You insert AI into the process with a specific job defined, not any job, and you make sure that the inputs match the job specification,” says Calkins. “You must give the AI agent a narrow range of possible outputs, instead of saying, you’re an agent, do whatever you want.”

One way to validate an AI agent’s effectiveness is by implementing a second LLM as a judge. Appian has LLM as a judge implemented in DocCenter, its end-to-end document automation solution.

“In DocCenter, there’s a toggle you can select to leverage LLM as a judge,’ says  Mark Talbot,  Appian’s director of architecture, AI. “You can use LLM as a judge, use computer vision to review the document, or implement advanced intelligent document processing. It will increase your token consumption, but research shows you get higher accuracies, and  we’ve seen this with our customers.”

Why this matters: Minimize risks when introducing AI to critical workflows

There’s a lot of hype around fully automated agentic AI solutions and agent-to-agent orchestration. But regulated companies are unlikely to embrace these approaches in their mission-critical operations or when automating document processing. A bottom-up approach introduces AI at each step of a defined business process. This enables tracking changes, monitoring for risks, and measuring improvements.  

Evolving the enterprise’s context layer for AI

Enterprises have historically struggled to build and maintain knowledge bases. Defining taxonomies, integrating content, and documenting workflows can be expensive and require ongoing updates.  

Sanat Joshi, EVP of product and innovations at Appian, described a very different, bottom-up approach that’s derived from Appian’s data fabric and process automation.

“The data fabric does a beautiful job of encompassing three concepts needed to create applications and processes: the data catalog, the data model, and data access”, says Joshi. “But now add business rules, process models, APIs, security groups, the organizational model, and their interrelationships into one unified view of the enterprise  — that becomes your context layer. We created it for people, but now it’s phenomenally valuable because those same assets feed LLMs and AI agents.”

Why this matters: Centralize a context layer for AI agents

The accuracy of LLMs and AI agents in recommending or automating decisions depends on giving them comprehensive information about the business process. Having an integrated data fabric, structured information extracted from documents, and instrumented business processes in a single platform can accelerate the deployment of accurate AI capabilities.

Why review Appian’s AI capabilities

I’ve been covering Appian for several years, including articles on

• How Appian is transforming low-code app development
• Where DocCenter is enabling enterprises to save millions.

But what impresses me most is their authentic leadership and pragmatic solutions, led by CEO Calkins and founding CTO Michael Beckley. Who is your organization going to partner with on evolving critical process management with AI using a risk-based approach?