Summary
Isaac Sacolick shared data showing that many leaders expect significant business process redesign around AI agents, while few feel their processes are ready. The group discussed overhauling practices such as software development, funding models, systems thinking, and risk monitoring. They also identified new practice areas, including AI FinOps, AI cyber resilience, and organizational re-engineering. The conversation touched on the need for human-centered design, clear accountability for AI agents, and the importance of a strong context layer. Participants also debated the distinction between business strategy and AI strategy.
Speakers
- Host – Isaac Sacolick
- Digital Trailblazers – Derrick Butts, Martin Davis, Joanne Friedman, John Patrick Luethe, Liz Martinez, Heather May, Joseph Puglisi
Discussion
- What processes/practices need an overhaul?
- What new practice areas does AI require developing?
- How should Digital Trailblazers plan for organizational change efforts?
Research
Whiteboard
Transcript
[00:00:02] Isaac Sacolick: Greetings everyone. Welcome to this week’s Coffee with Digital Trailblazers, our event for the AI and digital transformation leaders across all sizes of organizations. This is our 195th episode and entering our fifth year.
And do me a favor, put this on your calendar.
I think it’s December 18th will be our 200th episode. You don’t want to miss that one. We’ll have some fun and some special announcements for that one.
We will have an episode next week. Next two weeks we will have two in December.
And that’s going to be our year. We’re going to be skipping over the holidays. Our episode this week, planning 2027. What practices our transformation leaders prioritizing every year around this time. When I was a cio, I was thinking about where I was changing my organization. And I always think in terms of people and process first even before I think about what the priorities are. And the reason is that because practices and people are sustainable, they are what allows you to continue to grow and evolve your organization.
And I think about some of the practices that are dear to me, areas like agile and DevOps and becoming data driven. And then I start thinking about which areas I need to mature further or change which new things, new practice areas I need to conceive and then how to drive change across our organization. And that will be our topic for today.
I got some help today. I got a lot of help from AI around some of the data points that you see here.
Let’s talk about first the practices that need an overhaul. 74% of leaders expect nearly half of their business processes to be redesigned or rebuilt around AI agents within four years. We’ve talked about this quite a bit. There’s an episode that Camunda sponsored a few weeks ago. We talked about process re engineering and we what you’re seeing here is just an acknowledgement that 74% of what you’re doing today or 50% of what you’re doing today probably needs to evolve over the next four years. And I actually think that timeline is a little slow. I think it needs to happen a lot faster. 21% rate their business processes as prepared for AI agents.
The least ready of seven areas behind data and infrastructure.
So we have a lot of work to do around data and infrastructure. And I’m sure Derek is going to come back and say we have a lot of work to do around resiliency around these areas to make sure we’re ready for Agent 10, Agent 100 and then Agent 1000.
Some new practice areas we have to consider.
5% had the full set of critical AI controls, yet 42% expect agents to act autonomously.
We got a problem here, right? If we’re autonomous almost 50% of the time without the controls, I think we’re heading off a cliff. I actually think the 40% is higher than what we’ll see by 2030.
And I know that’s a bunch of years out. Lots can happen with AI. I just don’t see the autonomous enterprise happening that dramatically.
But I also think the 5% is low.
I think organizations are a little bit better around their controls.
80% of AI spending sits outside the entire enterprise IT budget. This is a big deal. Like a lot of us are either working in IT or lead it or partner with it.
This is a trend that’s been happening going back to shadow IT and then SaaS.
And what this means for CIOs and CISOs and CDOs and for the occasional CAIO that’s out there is we need to be a lot more collaborative in working across our enterprise. They have the money, and sometimes the money sometimes comes Decision authority planning for work change. 43% expect agentic AI to significantly disrupt their workforce in the next 12 to 18 months. Talking about planning for practices.
We can’t wait for 12 to 18 months and say holy crap, we better change our organization. 85% of UK CEOs are redesigning roles, career paths and teams for human AI work. That’s up from 71%.
And for those of you who already know, and for those of you who don’t know, this is essentially what my next book is about.
It’s tentatively titled AI Reinvention. The folks I’ve been collaborating with here on the Coffee with Digital trailblazers have heard my journey. I started it back in March or April.
I will finish the manuscript by the end of this year.
And do do me a favor, sign up for my newsletter. There’s a URL on the bottom left hand side there for signing up for my newsletter. That will be your ticket to getting early access to chapters and to some other content around the book.
That’s starcio.com driving digital for those of you listening and this is my work for the rest of the year. Finish this book. I’m about 2/3 of the way edited. I have one more part to finish off. But for today let’s talk about 2027. And again we’re talking about practices. We are not talking about technology.
We’re not talking about the direction of the company and its business or AI strategy. We’re going to talk about it from the lens the of a C level leader that could be a cio, cto, ciso, cmo, chro. And you’re looking at your organization and saying how am I changing my practices to be ready for the next level of AI era? And again we started this year with most companies first learning about AI agents and ending this year with lots of opportunities and many organizations getting to hundreds and even thousands of AI agents. We have a lot of things to do around our AI agents. And John, you’re the first to raise hand. I’m glad you’re here.
I want to talk about practices that need an overhaul.
Where would you see taking something you’ve already been doing in your organization and saying you know what?
We need to do something systematically different in the AI era.
What are you focused on? John?
[00:06:55] John Patrick Luethe: Hey Isaac, thank you for having me on. I’m really looking forward to your book coming out. I mean AI is so transformational.
It’s like the electricity, it’s like the Internet, it’s like email.
It’s going to fundamentally change everything that we do in society. And so I completely agree that this is going to change everything in the enterprise, but we can’t change everything at once. And so we really have to prioritize on what’s out there. And when I’m looking at the news and the security vulnerabilities, that’s an area that I think is always going to have to get a lot of priority just in light of. It’s like the absolute definition of a hockey stick. But I’ll leave that one for Derek. And the areas that I’m really seeing that people focus on is anywhere we have software development and the software development, the end to end is really turning into AI native delivery. Right. And so there are some, you know, it started as is kind of like vibe coding and then went to spec driven development. And that’s kind of the next evolution that what I’m really seeing is, is really transforming how people build software and maintain software and operate software in an enterprise setting and how all the roles are using AI and it’s really neat and I just, I think people are going to have to get in that mindset just to stay up with with companies and stay with all the changes and the security issues that we’re seeing. And so I, that that’s, that’s what, where I’m spending a lot of my time. That’s what I’m. What I’m seeing talking to companies.
[00:08:22] Isaac Sacolick: John, I’m calling that a, the transformation from SDLC to what I’ll just call the AI life cycle.
And this notion that 80% of the AI budget’s falling outside of it.
We’ve got a problem here. I mean, we don’t, you know, people are creating agents and applications and integrations with no idea what’s inside, with no idea how to really test it or version it or validate its output or what it’s going to take to run in production. I think it’s a pretty significant change from, you know, let’s just say a head of software development or software delivery, one who’s producing code that delivers applications, that delivers outcomes to one, you know, that maybe is a collaborator partner governance role over the next couple years. I’ll let you comment on that before we go to Martin.
[00:09:18] John Patrick Luethe: Well, it’s, it’s like the biggest shadow IT that’s ever existed. Like we had, you know, people came out with iPhones, people loved iPhones. Pretty soon the companies had to let iPhones in.
And, but when it came to software development, there’s a couple of hurdles. One is, is that it was, there was. Not everyone in the organization had the knowledge to, to write, to write software. And then a lot of people didn’t have the tools to write software. So there is so much shadow IT going on right now. People are buying subscriptions to Claude and Grok and Grokbot and you know, OpenAI and ChatGPT and, and, and they’re able to, to basically describe what they want and they’re able to build, build agents and build software and skills and things that are doing things. And so what I am seeing is in companies, everyone, every role is starting to use stuff to automate things to make their life better. I’m seeing product owners using AI to look across all the different ADO boards that they have to try to see what their product teams are doing. I’m seeing the business people are putting the decks from people into AI and quad and saying, are these people telling me the truth? Go out and look at all the different data sources across the company and try to tell me if the, if the numbers in this deck are actually accurate. I’m seeing extremely C suite level people doing this. And so everyone in the company that I’m, that I’m working with are using AI to try to figure out how to make their lives better, to just stay up with things. And so it’s going to be a reckoning.
[00:10:48] Isaac Sacolick: Very cool, John. I don’t know how to characterize that this transition of how we are creating our communications materials and fact checking Two things that AI is doing a really good job of, and, you know, I’m just going to call it PowerPoint storytelling. I actually think AI is getting better at that.
But, you know, ultimately I think that the person needs to know the audience to be able to tell that story the right way. Hey, Martin.
[00:11:22] Martin Davis: Hey.
[00:11:22] Isaac Sacolick: We’re talking about what processes and practices need an overhaul.
And I’m not going to guess where you’re going to go, but I have a thought.
[00:11:32] Martin Davis: I’m going to go actually may surprise you. I’m going to start with funding.
[00:11:38] Isaac Sacolick: Whoa.
Okay.
[00:11:40] Martin Davis: So I think one of the issues we have, and this is a big issue, is most companies run on annual project budgets.
And the trouble is, a lot of the time with AI, you don’t understand the value until after you’ve been experimenting for a bit.
So at the time you’re doing the budgeting, you have no idea what value you’re going to get from it. So ability to actually define the budget is very, very difficult.
However, if you do some form of maybe rolling quarterly funding for your AI, then you have an opportunity to maybe experiment a bit and then define what you actually need to deliver something. So. So just kind of a different twist on how you fund some of these things.
[00:12:30] Isaac Sacolick: You know, I.
So there’s part of something you’re saying, Martin, I agree with and something that you don’t. I’m going to speak for Liz here and say I think I can project value based on the outcome, not necessarily the path to get there. And my experimenting is largely giving me path and it’s validating assumptions.
But if I want to experiment on delivering new leads using AI to help me do campaign management, I can probably back into what new leads are going to in a particular segment or Persona type is going to get me. It may take some work. I probably need some AI to do this. But I think your underlying premise that funding running on an annual project that should have died the days of Agile 20 years ago and it didn’t. A lot of us have been retrofitting Agile and planning to fit that yearly budget and reporting cycle.
I think what you’re also getting out this notion of being able to project an roi, or even the cost, I think the cost is actually the harder part of projecting because I have no idea if this idea will cost me $1,000 in tokens or $100,000 in tokens until I spend.
[00:13:53] Martin Davis: There’s two sides to that because in order to justify the costs that you don’t know, you need to know the value which you don’t know.
[00:14:03] Isaac Sacolick: I think it’s a really challenging question, particularly when organizations go from the few that they’re starting, they’re experimenting with to there’s a line out the door somewhere in that transition. There’s the reality of what the organization can fund and support. And I think you’re right that the timeline needs to be faster and the criteria needs to get looked at. Let’s go to Joe.
Joe, what needs an overhaul for the air from a practice perspective?
[00:14:35] Joe Puglisi: Oh, you, you jumped over Joanne. She’s going to be mad at me, but,
[00:14:38] Isaac Sacolick: oh, shoot, I, you know what it is? I’m looking at photos and Joanne doesn’t have her photo op here. Let’s go to Joanna. Let’s, let’s do this the right way. Joanne, go ahead.
[00:14:47] Joanne Friedman: Thank you. I appreciate that. Joe, you know what, you’re one of the few left in the world.
The group here, generally speaking, I’m gonna say that it’s systems thinking and design thinking have to be brought together so that we have a human centered systems thinking approach to AI. And the reason for that is the statistics on redesigning process.
You know, if you saw David Lithicum’s post earlier today, he is spot on. And this is exactly the, the same thinking that I use from, you know, a systems design perspective for the company.
Every decision we make over the course of a day doesn’t necessarily have to be captured in an agent or in the harness around the agent, but those that are critical to decision making and really driving the business forward have to be redefined as a capability and use a notion of human centered systems thinking. Because we need to understand that an agent doesn’t know how to do these things until you give it the context and a very heavy semantic layer around that context to understand the nuances of the business. And so that gets to ontology and a lot of technical work. But it also brings us back to what do people actually do? What are the steps that they go through to resolve an issue or solve a problem? There’s a lot of, you know, nuance there. There’s a lot of steps that we go through as humans. If we see a problem, we immediately ask, well, why? Right? And we go through five whys. So the five wise method with a human centered twist to it is what will get companies a lot farther, a lot faster. Because their processes, if they just laid them out as they are, go back to how many decades ago when paper was still the way we worked.
And that workflow can be easily redesigned. The low hanging fruit could be autonomous Agents, the higher level, and the trust path that’s needed to go from I’m a toddler to I’m, you know, a tween to I’m a teenager to I’m a mature adult, which would be the autonomy trust path. That’s what’s needed. So it’s changing how we look at human centered systems thinking.
[00:17:29] Isaac Sacolick: I love that. I don’t know if you mind this, Joanne, by added critical and analytical thinking to your list.
Well,
[00:17:36] Joanne Friedman: they’re part and parcel of it, too.
[00:17:38] Isaac Sacolick: Yeah. I just think IT needs to call it be called out, you know, and I think even creative thinking belongs there. Although I do think that’s part of design thinking.
Very, almost synonymously.
[00:17:50] Joanne Friedman: Well,
[00:17:51] Isaac Sacolick: but go ahead.
[00:17:53] Joanne Friedman: Sorry. Sorry for interrupting you. It’s a holistic approach. If you don’t use a holistic approach with AI, what you end up doing is creating agent silos because the harnesses around each don’t connect the dots.
And I wrote a piece not long ago called AI’s Last Mile, which I believe is execution. And so in that vein, if you don’t think about how all the pieces fit together, all the agents run in a swarm or whatever. And it’s not a technical version of it, it’s the thinking behind it.
Because until the dots are connected, you’re not going to get where you want to go.
[00:18:36] Isaac Sacolick: Gotcha. Joanne, let’s. Joe, thank you for being patient.
Thank you for listening. Thank you for being here every single week, and thank you for being everything that you are. Joe, what are you overhauling?
[00:18:50] Joe Puglisi: Thank you for that.
I want to double down on what Joanne said, but in much more pedestrian terms. I wrote a piece recently called the Department of no.
And I think there are two areas where I would emphasize the IT leadership. The IT staff really embrace the idea of knowledge, knowledge of what’s happening in the industry.
What are the capabilities of AI? They’re moving so rapidly. The opportunities are almost limitless. So I want to put more emphasis on external research and observation. What are our competitors doing?
And then I want to double down on internal research, as Joanne said. Do you as an IT professional understand what people do in this company, how they do what they do? And does that give us entree to leveraging these AI technologies that we’re studying to improve not only the individual’s processes, but the overall processes of the company? Can we evolve this company forward through the use of artificial intelligence? Whether that’s simply generative AI creating content or. Or whether it’s agents that take over for humans?
Do we understand how to manage those agents if they begin to take over for humans? And are we enabling the rest of the employees to understand and leverage artificial intelligence? So those are the different areas I think I would emphasize. Isaac.
[00:20:30] Isaac Sacolick: Joe, we’ve talked a lot about this here. I completely agree with you around knowledge and lifelong learning in many organizations. You know, talk about funding, that’s a budget owned by HR and largely used to buy access, subscription access to courses. And I think that’s like 1% of the dimension that organizations need to think about if they’re going to build both AI literacy and business acumen and do that together. I think it’s a really good call out. Hey Derek, we don’t need CISOs anymore or security departments. I’ve heard this a couple of times and everything tells me that’s wrong.
That example we talked about a few weeks ago about the refrigerators in Europe being hacked in the papers again because of default passwords. I mean, where do you want to go? What needs an overhaul for the AI era?
[00:21:29] Derrick Butts: Well, first of all, thank you for having me. And I think the CISOs are definitely needed because they’re the quarterbacks of what’s going to happen in the organization. And I think the first thing that needs to be transformed is the transformation itself. We need to change the mindset of how we think about things. We can’t keep bolting on cyber security, AI operations and data and business continuity after the fact, because right now they’re treated as separate conversations that have been brought into the mix. We need to look at all the business services across the board. And again, from a resilience point of view, I’m looking at this. If you have a new technology or service, what risk is it bringing in? The attitude used to have where the pen testing annually, every six months, that needs to go away. The new priority needs to be continuous AI monitoring and cybersecurity monitoring because of the risk, the things that are associated, the former business continuity plans, disaster recovery plans, they need to incorporate things like now incorporating the AI models that we’re using, the API risk, the platforms that they’re working on, the third party AI risk and services, all being brought into the mix. The third party risk is a huge thing that you just mentioned with the refrigerators across Europe. These things and knowing how to properly configure a new application, a tool, before it’s entered into your ecosystem is one of the biggest, highest risk that you see. We saw it with the wireless era, where people go to the store, they get an access point, they plug it in and Everything’s good and they never change the password.
Incident response. I look at also how prioritizing now, how you do this now in an AI world, all these things need to be validated and tested as far as how you do an incident response based on the AI agents when they go rogue, the unavailability of AI when it goes sideways. Also looking at just the conventional cyber attacks and how they’re accelerating now using artificial intelligence. And the biggest thing I think I look at is just the change management itself.
When you look at resilience as part of this transformation, you know, rather than beginning with security after the fact, we need to look at it before the fact and as part of the deployment. When you have the conversation about artificial intelligence, looking at the risk, looking at what you’re doing and how it actually is going to integrate into your ecosystem, that’s where people are missing the point. The interoperability of these new applications is not something that’s asked up front. I have a scenario where a customer I was working with, they’ve got three different chat bots, ChatGPT Rock and Gemini Enterprise, and they have no idea what the proper skill set or what they’re capable of when it comes to different services and what they can provide as far as capabilities and business value for that particular company. And that becomes a concern because you’ve got all these tools and applications but don’t know how to properly use them. So when you look at this, the question is not simply is the AI going to be secure?
Can I now establish my business continuity operations in an AI area where my data, my ecosystems will contain or be able to recover if things go sideways? That’s what I think the priority needs to do. What am I putting in my system and how can I make sure I can get through it if something should happen and I can still maintain my business operations.
[00:24:27] Isaac Sacolick: Eric, I captured one point in here about continuously monitoring AI. We’ve always left it to security and operations to basically is the system up and running and hopefully performing well and hopefully secure and looking at things like observability signals to give us a sense of where errors need to get corrected. There’s an entire discipline, site reliability, engineering, sre, that takes that Google process and mindset and says, you know, how do we continuously improve our applications?
But none of that really dealt with AI and data for that matter. For that matter. And so now we’re
[00:25:07] Derrick Butts: putting, yeah, that’s the change that needs to take place. Because, you know, it’s Joanne and Joe and everybody, probably these AI agents They need to be identifiable, they need to be trackable, they need to have identities which can be tracked out the system. So one, when you tell an agent to go do something, is it doing exactly what it needs to do? We’ve seen already the cases with huggy face where the agents are collaborating and they break out and they go to other places of the organization where they weren’t supposed to go. We need to be able to track those. So that’s why the continuous monitoring of AI in itself needs to be holistic and enterprise wide.
This focus of the containers and things, you know, AI is pretty smart if you let it run, but if you don’t put the guardrails in place. So we have to focus on the monitoring and the capability. The problem is we haven’t harnessed that capability yet where we can actually monitor them accurately and trace every place where they’re going to.
[00:25:56] Isaac Sacolick: Yeah. One more dimension, Rick, on the common stream is adding AI risk needs to address plan for damage recovery and alternative fixes and a new layer of bcp.
Absolutely. We’re in early, early stages for all of those things. Hey Liz, welcome to the floor. We’re talking about what practice areas need an overhaul for the AI era that you would go and make it a priority in your 20, 2017.
[00:26:24] Liz Martinez: So the big ones that you’ve already hit are fabulous. Like I heard the funding, which was perfect. I heard continual risk monitoring holistically from Derek. Beautiful.
Making sure we’re, you know, doubling down on business value, which is always my core speaking point. However, I’m actually going to take this in a different direction. I hope I’m not stealing Joe’s thunder around communication because I think that the organizational change and messaging that needs to occur needs to be much more robust than it’s been.
I think a company needs to have a thoughtful conversation amongst their leadership as to how it is that they want to think about AI and how it, you know, if it’s really, if it is really is to replace their, you know, people or is it as a tool or is it a, you know, whatever it is, they need to really clarify what it is that they’re using it for and put in place a major campaign around it and how people can participate in the uptick of AI because if they don’t, if you’re not proactive, people are going to be reactive and they’re going to go off and do all kinds of things and create all kinds of assumptions. And that’s the. So this, this is where I’m thinking that an AI Philosophy and a communication plan and a full Org change plan is probably a good place to look for 2027.
[00:28:04] Isaac Sacolick: It’s really interesting, Liz. I mean, the interesting part about communications is we have AI tools to create the communications, and then we have AI tools that summarize the communications.
And the question is, is anybody paying attention?
Right. You know, that, that all the, you know, questions like, yeah, you put out your weekly report or your power, you know, you had, I mean, yeah, I think the talent. That’s why
[00:28:32] Liz Martinez: I’m saying it’s more of a campaign.
Not necessarily. I mean, sure, obviously communications are included in a campaign, but it needs to be like a complete philosophy with specific standardized talking points that leadership aligns on and repeats on the regular.
[00:28:52] Isaac Sacolick: Yeah, I think it’s a. I think it’s a great point. Who’s jumping in here? Go ahead,
[00:28:56] Joe Puglisi: it’s me. I just want to come back to what Les was saying. You know, earlier. I said it’s about the relationship between IT leadership and the rest of the organization.
And to your point, Liz, I was trying to say that you have to, as the IT leadership, you have to be that person who goes out, you know, strap on the robe and sandals and preach the gospel of AI because the rest of the organization may not understand what the opportunities are, what the implications are, what the risks are.
And it’s. It’s not putting out another newsletter. It’s not.
And
[00:29:33] Liz Martinez: I would
[00:29:33] Joe Puglisi: say
[00:29:33] Liz Martinez: it’s not just the IT leadership.
Everybody has to be singing from the same page.
[00:29:39] Isaac Sacolick: All right, let’s take a time out here. You see, I bolded it. I’m creating a new signal around us where if I bold IT on the dashboard, it means we are going to have a conversation around this at a future coffee hour. Because this is a great topic. It impacts everybody. I’m getting a thumbs up from Derek and Liz. This is just too big of a topic, and it’s too important.
Let’s do it in January. Hey, Heather.
Overhaul. And then let’s bleed into some new practice areas. Go ahead, Heather.
[00:30:07] Heather May: Well, something that you mentioned about AI talking to AI Talking to AI really relates to resumes and how so often candidates are taking their resumes and their job description, throwing it into AI and getting a resume that reflects what the company is looking for, not necessarily what they do or making them look like they’re perfect for it. And then when it gets to the company, the hiring team, they’re thrown into AI so now to decide who they’re going to talk to, AI is talking to AI and where is the human? So I would say that areas that are need overhaul are in talent acquisition and so that the internal recruiting teams recognize the value of AI but don’t dismiss the importance of human interaction or you know, to Joe’s point of communication which I would add to the agenda when we talk about communication needs. New approach, particularly when it comes to recruiting because people have to be involved in terms of training.
What are the actual reskilling or retraining? Retraining that is necessary within a company as a result of AI and the role distinction and role representation and job description changes that may result. And then finally for hr which usually has the oversight in those other two areas, how are they going to remediate when there’s a problem? When someone goes against a policy, how are they going to handle someone who goes rogue? A person going rogue in their.
Dealing with their personal use of AI on the. On the workplace. In the workplace. So I think there’s all of those people related elements that are critical.
[00:31:54] Isaac Sacolick: Wow.
That you know, problem and conflict resolution in the AI era. I mean it bleeds into the first new practice area. I’m just going to drop in here and we’ll get to after the break some way of defining agent accountability and what that means and putting the guardrail up saying you don’t get your agent deployed unless you’re going to be accountable for its outcomes and its risks and finding a way to measure that. I don’t think I’ve heard a definitive practice around this. I think it’s really important.
You know, Heather, you brought up hiring and interviewing. I think it’s really important. Jason McKay on the Common Stream talks about hiring a business AI analyst. And I asked him how do you enter? How do you interview somebody like that? Right. So easy to talk knowledge and jargon. And he gave me the simple answer, show and tell.
It’s something that we do in our Agile Sprints. We do a Sprint review with a demo and if 80% of the AI budget is going outside of it, I want Agile outside of it and I want to see Sprint reviews and I want some way of looking underneath the hood in a way that we could say, yeah, this is being built on standards, folks. You join the coffee with digital trailblazers. We meet here every single week or almost every single week. We’re going to miss some weeks in the next few weeks because of holidays.
But our discussion is for those of you leading transformation in your organization. And we started a few years ago with digital transformation, now we’re getting into AI transformation and into AI Reinvention. That is the tentative name for my new book that will come out sometime early in Q1. Q2. If you want to be involved in it, do sign up for my newsletter starcio.com driving digital. I will be sharing some insights around the book coming up our week next week. Joe, surprise for you. Your suggested topic on AI’s 2027 reality gap around Big Expectations, Tight Supply and Hard choices.
Joe I placed that on the 13th. On the 20th we’ll be talking about AI for social good, Doing more with less when the mission can’t wait. I’m looking for executives from nonprofits. If anybody listening is one or wants to recommend one, obviously. Well, not obviously. I’m looking for organizations to have some scale to them and talk about what are they using AI to do more with less and what that looks like inside nonprofits. The 27th we will have a week off for Thanksgiving and that’s our conversations going forward. Martin comments on overhauling, but let’s bleed into some new practice areas. Hey Martin.
[00:34:47] Martin Davis: Hey. I was going to go with new practice areas. So there you go.
[00:34:51] Isaac Sacolick: New practice areas for 500.
[00:34:55] Martin Davis: So I’m going to so I like your agent accountability, but I rephrase that as people being accountable for their AI agents, because AI agent accountability implies the AI agents are accountable.
So I think it’s a case of people have to be accountable for their AI agents. So that whole governance side of things.
But I was also going to go somewhere different, which was AI finops.
So the whole thing around unit cost, visibility, budget guardrails.
Yeah. How the spend scales with use.
Yeah. And as that usage ramps up, how the token costs ramp as well.
Yeah. And the fact is we need to keep control of of the costs overall. And some of these, if they become really popular, could blow your budget out the water. Look at what happened with Uber. Yeah. So I think the kind of AI finops is a new practice and how do you govern it? How do you decide when the cost outweighs the value?
There’s a whole new area there, I think in terms of
[00:36:13] Isaac Sacolick: goes back to something I think Liz brought up a few weeks ago about setting person and team and use case budgets and having the tools in place to monitor and capture that and charge back around it. The first time you get cut off because you used up your tokens is the first time you start realizing, oh, I have some choices to make here about what I’m asking and what I’m valuing.
And I think that’s going to be a, you know, for organizations that are going from one team to multiple teams, it’s going to hit them pretty hard. How do I manage this? Great, great suggestion, Martin. Hey Derek, what are some new practice areas?
[00:36:55] Derrick Butts: Well, I agree with what you said about the AI accountability. I mean I look at this and I’m going to go back to the resilience. AI cyber resilience is something should be a new practice. People should look at the business challenges of trying to overcome bringing AI, but try to think about what are the risk associated with them, how would they govern that AI? Understand what are the dependencies in the interoperability of how that’s going to work with the other applications and services they have in place. And most of all, how can we recover if something goes wrong? I look at the AI governance is Martin just mentioned. Everybody needs to have established clear ownership of who owns it, what decisions need to be made, what’s the risk that tolerate the acceptance as far as what are the outcomes they’re looking to do. I think the AI dependencies for interoperability management, which is a huge area that people really don’t look at, doesn’t play well with the other applications that are currently in place. Understand the way it’s going to interact with the APIs, some of the identity data platforms, all the cloud systems that come into place. As I mentioned earlier, the new practices, the AI incident response, it needs to be focused more now to look at AI models, prompt injection, data poisoning, drift, all those different things. How do you account for all those and how do you recover from it? The AI inventory a lot of times you’ve got these applications, those that you have that you approve and those you do not. Being able to identify those that are clear, the classifications associated with those feel recovered from those that are used and be able to block out those that are not authorized is going to be key. That goes back to the visibility I mentioned earlier. And overall we can’t remove the human insight. You got to have human in the loop. There has to be a decision maker who actually proves these things and not agents or proven agents. Especially with AI is making decisions that affect customers, outcomes, employees, finances, security, all those things that even in a regulatory area you still have to have somebody overseeing and say yeah, this makes sense. The AI governance is going to tell us how the AI should operate. But the AI cyber resilience prepares us for what happens when it doesn’t.
[00:38:54] Isaac Sacolick: Wow, Derek, you’ve got three bullets just on what you’ve said here and I think this is going to be a real question mark. For CISOs, particularly in mid sized organizations, are they going to be able to keep up with 100 times faster and relentless cyber threats that are generated by AI?
And that I think that’s going to require a scratching their heads of their strategies.
AI catalog management, that’s an area of AI governance.
If, if you have more than 10 AI agents deployed and you don’t have a catalog that’s not a spreadsheet, you’ve probably got some work to do to find an AI governance platform. If you need some advice around that, do reach out to me. And I think the most important one you listed, Derek, is this notion of putting some definition around what Human in the Loop means.
Right? What is the practice around that? Don’t assume that people know what that means and when it’s required and how to practice it, how to capture it.
You know, for those organizations looking to deliver more autonomous agentic operations, you know, putting some observability characteristics around, Human in the Loop is your, is your vehicle to decide, hey, when are we ready to go autonomous around a particular set of decisions?
[00:40:22] Derrick Butts: And the key thing about that, Isaac, is when you do a tabletop exercise, that’s when the accountability hits the road. Because now when the fingers start getting pointed, you need to know who’s accountable for that particular arrow, why that AI went rogue or that agent went rogue. Who do you point to? And you can’t be just pointing the fingers all around everybody. There’s got to be one specific person who brought it in needs to be in charge of it or be authorized or accountable for the behavior of that particular agent when it does go sideways. And even if it does well, who gets the Atta board or the Atta girl? Those are the kind of things we need to relegate. But this is the holistic approach of the resilience mindset. Everybody has to be involved. You can’t just have certain individuals or certain departments. Everybody’s got to have some sort of skin in the game to some extent.
[00:41:05] Isaac Sacolick: Thank you, Derek. We’re going to bring Joanne back to talk about a new practice area for the AI era. What’s your vote, Joanne?
[00:41:13] Joanne Friedman: Well, first of all, let me drop a word, a phrase that goes with human on in the loop or on the loop, which is human on the hook.
That is the person accountable for what the action of the agent is. But the new practice area that really needs to evolve is, oddly enough, linguistic. And I say that because ontologies differ from company to company, but more predominantly from department to department.
We all call things something relevant to Ourselves a report that we produce a statistic, whatever it is.
The need for ontology and taxonomies to come together in a particular way is absolutely fundamental to agentic development.
Because when I hear companies are running, you know, thousands of agents, how much duplication is there?
Because we go down the path of I need some, I need a fix for my business problem, X and somebody else in the next department over does the same thing. They’re basically the same tasks.
Why are you using two different agents? If we start to come together around a lingua franca that the business uses and there’s a group that is responsible for making those definitions, this is how we talk about X in our industry, and here’s how we talk about it in terms of our business, then we start to be able to reduce the overhead cost of building the agents and also the threat vectors and everything else that goes along with it from the notion of a holistic perspective.
Similarly, if we use that level of linguistic as a new practice, we can start to bring people together not only around AI and agency, but but also determine the level of agency we’re going to give. And by agency I mean the notion of I am giving you, you, the agent, a proxy for my judgment.
How much agency are we giving our agents? How much are we outsourcing it to them? That’s about judgment. That’s the second area of practice that needs to be developed. It’s about trade off management. It’s about how decisions are actually made, not from a workflow perspective or system, but rather from the what is the process of making that decision?
Therefore, what are the criteria that are used?
Is it my perception? Is it my intent, is it my perspective that gets applied? How do we govern those things so that the agents become a little bit more standardized?
Because linguistically they all agree. And then similarly we have the understanding that these are the criteria that are used to make a decision as the agents reason through them.
Where we have a standard, we don’t go, well, why would that be a criteria? It’s not something I’m familiar with or I ever worked work with.
But okay, I either go along or I disagree.
But that’s the level that we have to come to.
[00:44:38] Isaac Sacolick: Joanne, thank you for bringing up the context layer or elements of it. When you talk about linguistics and semantics and you know, this rush to push out AI agents out often means that they have their own sort of memory and their own sort of brain and their own sort of path to data and context. And we all know that’s just not going to Scale, you know, and I, if you ask me, I’m going to be writing about this for CIO.com this month. I think it’s even more than just agent accountability and catalogs. I think it’s the single most important part of a CIO’s charter going into next year and the following year is to lay out an architecture and lay out a structure and lay out the governance around that context layer. So thank you for bringing that up. Let’s go to Liz.
[00:45:29] Joanne Friedman: Sorry, I just. Liz, just let me make one comment to what Isaac said. That is the, the biggest new practice that needs reinvention is enterprise architecture.
I’m done
[00:45:43] Liz Martinez: like that. I like that a lot.
First of all, I think you stole my points, Joanne. I was gonna.
The whole idea of AI agent rationalization and making sure that we actually have a decent level of governance. It’s so funny because the PMOs of the world have been getting such a bad rap as a low value add type of organization. But this is a place where any PMO could easily step in and say, here’s some governance that we could really leverage in partnership with the CFO from a financial perspective, in partnership with the CISO, from a security perspective, and get some decent governance in there around cracking and rationalizing and keeping the company safe and efficient around AI agents.
[00:46:40] Isaac Sacolick: Liz, I put this under overhaul. I think PMOs need an AI charter. I mean, there’s a ton of value in PMOs. You know, I’ve hired them, I’ve managed them, I’ve, I just, I. You know, it’s very hard for me to picture having a large organization without some kind of value management office or a program management office.
But the reality is that tools are doing most of the work they were doing five years ago.
Right. And if we’re talking about reporting or consolidating data, I mean, if your tools aren’t doing that for you, you’ve got, you’ve probably got an engagement problem with your staff. Maybe you have too many tools going on, you’ve got some friction there. But when we talk about tokenomics, which is both value and cost, and the tools there, quite frankly, are primitive, it’s probably the first area I probably ask my PMO to own is make this work for the tools that we have today, for the scale that we have today.
[00:47:42] Liz Martinez: Agreed. It’s funny, it’s a conversation I’ve been having with John Offline about setting up some, some good tools for that, for that whole area.
[00:47:54] Isaac Sacolick: Thanks, Liz. Let’s go back to Joe. Joe, what’s your New practice area,
[00:47:59] Joe Puglisi: Organizational RE engineering.
I think AI affords us an opportunity. It’s another column I wrote a few weeks ago.
I think AI affords us an opportunity to completely re engineer the organization, but it has to be done in, in, you know, in a rational way.
Again, it goes to the understanding of how the organization actually functions. What are the inputs and outputs of the organization? How do we create value? Are there new ways of creating value? I like to think that, you know, traditionally we’ve organized or reinvented the way that marketing works or we’ve reinvented the way that finance works works. And now we have an opportunity to really reinvent the way the entire company works. The borders between departments is dissolving and processes are flowing across departments, displacing people and displacing entire functions. So I think there’s a whole practice in what I would call organizational reengineering.
[00:49:06] Isaac Sacolick: Yeah, I’ll give you a clue, Joe. I listed four practices in my book that become predominant once you start scaling AI agents and organization. Re engineering is going to hit organizations somewhere between 100 and 1,000 agents.
And you start looking at how process have been re engineered, what skills are shifting toward AI, what skills you need and new practice areas you have.
And it’s not an HR function. Right. You really need to understand AI and operations and value to be able to do this.
And I really don’t think it’s a big four function either. Right. This is not a spanning controls and looking for just cost efficiencies and things like that.
I think organizations are going to need a skill set around that. I think it’s a really good call out. Go ahead, Derek.
[00:50:08] Derrick Butts: Yeah, I
[00:50:09] Isaac Sacolick: want
[00:50:09] Derrick Butts: to go back to, yeah, I want to go back to something that Joanne said earlier about the enterprise architecture from an AI architectural point of view. I think you should look at the practice of, you know, not just the start with the business outcomes first and not just say I need an AI tool. And I think when you look at that, the companies, they, they want the tool, they want it now, but they don’t really think what are you really trying to solve? And I think if we look at the architecture, because when you use the word architecture it means put it before, not active. And I think when we look at that practice, we need to get people taken now, before what is, what am I trying to get? What are my business outcomes? What are the risks associated with that? And think about what happens when it goes sideways. If you think about that up front, it makes it easier when you put it into place and you can monitor and understand that it’s doing exactly what it needs to do. You can actually monitor, you can change it. You can monitor how AI is influencing the other things you’re working on. You can also look at the new dependency that is creating. But I think also the way we, how we work differently, that’s something we need to think about moving forward. As we move into AI area, everybody needs to think differently. The old mindset needs to change and say, now we’re in the AI area. How do we think now based on how AI is going to operate or how do we want it to operate?
[00:51:22] Isaac Sacolick: Thank you, Derek. Let’s go. Martin. Let’s talk about organization change efforts. We just laid out portfolio of work for leaders that is untenable and yet very hard to prioritize. Depending on the velocity of how fast your organization’s adopting AI and what risk characteristics that organization has.
You know, somewhere in here you’ve decided what you’re going to focus on. How do you bring it home and drive the change efforts around it?
[00:51:56] Martin Davis: Okay, well, some of what I’m going to say is not new, but it has a kind of, maybe a slightly different flavor with AI.
The first thing I’ve got kind of maybe five points. So the first one is redesign the work first, decide what the AI is doing and what the people are doing. And some of that needs some experimentation to find out what you can do with the AI first of all, but be clear on the difference. Yeah. The second piece is make sure you’ve got a business owner who actually owns this in the business side of things and is actually going to be responsible for the stakeholder management, the driving at the. Yeah, this is kind of, I know it’s not rocket science, but it’s still how bad some of it’s done these days.
Be very candid about the impact on your people, on the workforce.
Yeah, this is going to take jobs and jobs are going to change. But hopefully those people can be redeployed on value added work as opposed to menial work that they may be doing before non value add.
And something that you’ve always heard me say about the layer of clay, the middle management.
Equip your middle management first before you start.
They are the ones that decide the adoption. Give them tools, give them training, give them revised metrics before the teams. Yeah. Because that’s really where this goes to die if you don’t sort out that layer of clay in the middle.
And then the last thing I would say is measure the behavior, not the deployment don’t measure how many user IDs you’ve given out or who’s logged in. Measure the behavior about what is actually going on.
[00:53:43] Isaac Sacolick: Oh man.
You just gave us a webinar of information around change management and really got it down to the essence and basics. Thank you for that, Martin. Let’s just get some lasting ideas here from Liz, Joanne and Heather around this. Go ahead, Liz.
[00:54:00] Liz Martinez: Thanks.
I love what you said, by the way, Martin, but the thing that I’d like to bring up now is an overhaul or a thinking through of the strategy of the company itself.
Because now that we’re, you know, able to deploy AI in so many different places, I’ve always been an advocate of reviewing your strategy on the regular. So whether that’s twice a year or four times a year. But this is an opportunity to really get in there and say, okay, what are, what is the value we’re giving to our customers and how can we leverage this new tool to help us get them more value and what new markets is it opening up for us and maybe where do we want to start concentrating or no longer concentrate based on our now our new abilities? So it’s like a whole at the, at the top of the house really looking at the strategy and, and how that is overhauled.
[00:55:01] Isaac Sacolick: I think
[00:55:02] Liz Martinez: thumbs up from Joe. Joe and I are on the same page today.
[00:55:05] Isaac Sacolick: We’re on the same page. You know, Rick is making a good point in the comments. Everyone forgets AI is not new. But I do agree with you, Liz. I think how we go about defining AI strategy, you know, whether it’s like, you know, Martin’s earlier point, we need to get away from a yearly or a five year plan.
You know, there’s something in there that people that leaders have to look back and say, how have we done this in the past? How do we want to go about doing this in the future? And you bring up communication needs, a new approach. AI strategy is communication.
So you’re right back there. Go ahead, Joanne.
[00:55:44] Joanne Friedman: I would say I somewhat disagree with the term AI strategy. I think it has to be business strategy or corporate strategy enabled or in conjunction with either AI, machine learning, robotics, whatever. It’s still. And that’s what I
[00:56:08] Liz Martinez: was, that’s where I was going as well, Joanne. Business strategy, not just the AI strategy. Go ahead. Sorry to interrupt.
[00:56:15] Joanne Friedman: So from the point of view of corporate strategy, am I, how do I move the needle forward? And there are many circumstances where, excuse me, you get trapped in what are called execution gaps. You get all the way towards the end. And then something fails.
Either the decision routing was wrong, the knowledge was wrong, whatever it is, it fails at the execution level.
Here with corporate strategy being the driver as opposed to AI strategy, you have an opportunity. And the reason I bring this up is because you can use generative if you want speed, you can use agentic if you want more workflow oriented, structured or unstructured. And then there’s that third level. And that’s where I think people don’t recognize that there’s a maturity curve to developing Agentix just as there is to any other kind of tooling. But it’s how it’s adapted and how it’s going to be adopted that really determines whether your business strategy or corporate strategy will succeed or fail.
[00:57:27] Isaac Sacolick: Joanne and Liz, I am happy to disagree with you guys on this one. I think business strategy and AI strategy are two different things. I think AI strategy needs to flow into the business strategy. But if you’re going to talk about what our experiments need to gear to and what’s the structure around them and we overload our business strategy with planning and implementation and experimentation and even bleeding into some governance, we’re going to lose people right away.
And
[00:57:57] Joanne Friedman: sorry, I think maybe there was a misunderstanding. I don’t believe there is AI strategy. I think there’s business strategy and AI is a tool underneath it.
[00:58:07] Isaac Sacolick: Oh, I think there’s AI strategy.
We may have to have another one debate on this one.
But I think these are two different things and I think they, you know, the people read them differently, I think they impact different audiences.
You know, I think we could have another session around this. Let’s bring Heather up. Heather, organizing change efforts, new practice areas. Where do you want to go?
[00:58:35] Heather May: Well, just taking off from what two other people said. One of the last things Joanne said is about how things are adapted and adopted and something that Martin said about how it’s. I forgot exactly how you said it Martin, about being communicated. But what I thought my connection to that is get a buy in and a champion. You can. Oh, he was saying about equip your middle managers I think absolutely. And then if you have people that can in a positive way infiltrate the organization, say this works, let’s try it, let’s do it together. I’m your buddy. Whatever it is, it’s another like person to person communication strategy that can be very effective if you really want the adoption to take place.
[00:59:26] Isaac Sacolick: Geez, Heather, leaving us on a cliffhanger around this because I do think people management needs to change.
We’ve exhausted our time today about talking about what practices are transformation leaders prioritizing next year.
And I think we successfully talked about what practices we need. I don’t know if we got to the point where we prioritized well, so maybe we need a poll and I’ll think about putting one of those out and seeing if we can get some data around this. And I did put a comment in the comment Stream Business Strategy vs. AI Strategy if you think we should cover that in an episode, give me a like on that particular one. I’ll just get a sense. I think there’s still about 30, 40 of you listening. We’ll see how many of you give it a thumbs up and see if we should put that in a conversation.
Next week’s episode AI’s 2027 reality gap big Expectations, Tight supply, Hard choices and you know what?
The stock market yesterday is responding to that change in our country, US country’s leadership and what that’s going to look like in terms of hard choices and tight supplies. This is a really good topic. I didn’t plan it that way, Joe, but we’re going to have to think about what’s the right questions to draw out how we project our future when we know that unlimited growth in our organizations and usage around AI need some choices. On the 20th AI for social good Doing More with Less when the Mission can’t wait. Again, I’m looking for one to three leaders from nonprofits, preferably medium and large size, who can talk about this subject and be our experts. If you are one or no one, do message me and then on the 27th we will not have a session. Happy Thanksgiving. Thank you for being here this week. Again, sign up for my newsletter starcio.com driving digital and I will let you know more about my book as I am finalizing the manuscript over the next four to six weeks.
If you have a thought about wanting to contribute to the book, do reach out to me. I am still working on some concepts around it. Folks. Thanks for joining everybody. Have a great weekend and do mark your calendars to visit next week and remember you can visit drive starcio.com coffee. You’ll see the next episodes, you can watch previous episodes and you can get to the add to the calendar button that will put it on your calendar. Everybody have a great weekend.
Leave a Reply