AI Agents for Skeptical Boards: What's Actually Working in Production

Coffee With Digital Trailblazers

AI Agents for Skeptical Boards: What’s Actually Working in Production

00:00 / 01:01:24

Summary – AI Agents

This was the 178th episode of Coffee with Digital Trailblazers, focusing on what boards are thinking about AI Agents investments and what’s actually working in production. Isaac moderated a panel discussion with Tim Crawford, Martin, Derek, Joanne, Joe, Liz, and John to explore board sentiment around AI, paths to deliver value, and how to set realistic expectations. The panelists discussed how boards are conflicted between FOMO and concerns about cost and risk, with Tim noting a shift from replacing people with AI to augmenting human capabilities, particularly in customer experience and employee engagement. The group identified that while many AI investments struggle to prove ROI, companies are increasing budgets despite missing cost savings targets, and emphasized the importance of governance, realistic timelines, and focusing on augmentation rather than replacement of human workers.

Participants

Hosted by Isaac Sacolick, CEO of StarCIO

Special Guests

Tim Crawford

Digital Trailblazers

StarCIO Research

What Boards are Reading

•s AI ‘one big bubble’? Behind the tech sell-off – NPR
“Over $1 trillion spent. Are we going to start to see returns?“
AI sticker shock hits corporate America – Axios
“… spent half a billion dollars in a single month after failing to put usage limits on Claude licenses for employees”
“Consumer sentiment around AI is also nosediving, and employees are rebelling”
Your AI Budget Is Growing. Your Returns Aren’t. Here’s Why. – Bain
“Nearly 40% of companies that measured AI cost savings landed below 10%, despite targeting 11% to 20%, yet 90% are increasing their budgets again”
The AI proof gap: Why AI isn’t delivering the performance leaders expected – GrantThorton
78% lack strong confidence that they could pass an independent AI governance audit in 90 days

What’s working in production?

CX: $9M/year increased cross-sell revenue, 3-week decrease in agent speed to competency – British Telecom via Verint
Shopping Advisor: 98% neutral-to-positive customer reactions achieved with Mylow, Lowe’s AI-powered home improvement virtual advisor via NVIDIA
Medical Diagnosis: AI-enabled brain scan slashes stroke diagnosis from about 30 minutes to just 7 minutes. Apollo Hospitals, Chennai
Software Code generation: Productivity at Bancolombia, JP Morgan, EchoStar Hughes quantified. Exceeds
Contract Management: Contract-review agents save roughly 360K human hours per year, with compliance costs down more than 50% – JP Morgan via FinTecBuzz

Of note:

Only 18% of respondents say their organizations collect metrics around ROI from AI. – Thomson Reuters, AI in Professional Services

Transcript

[00:00:01] Speaker A: Greetings, everyone.

Welcome to this week’s coffee with digital trailblazers. June 26th, our 178th episode. I’m really excited to be here.

For those of you joining, come say hello in the comment stream on LinkedIn.

Let us know you’re here, where you’re from.

Share an insight.

Drop a question for us.

I’ve got a super expert panel today.

Joined special guest of Tim Crawford. Tim and I have known each other for a good number of years and I was with him at a conference this week. We’ll talk a little bit about that. And I’m sitting here thinking, who should I have as a special guest to talk about boards and what’s working in AI in production? I’m like, holy cow. Tim sitting three seats over from me. I just need to ask him. And here he is. Tim, I’m glad to have you here.

Thank you for joining. Hello, Tony. Thanks for saying hello.

Let us get grounded in our conversation today. I want to talk about what boards are thinking about when it comes to AI agents. I want to have a conversation about what is actually delivering financial returns today.

And then what are the agents that we are deploying that are starting to deliver business value above and and below just what financial returns give us in the immediate sense. We’ll talk about boards, we’ll talk about agents, we’ll talk about revenue. Let’s look on the left side of this slide. What boards are reading.

And this is just a collection of sources that board members are likely to be reading.

And I deliberately chose the ones that are not hyping up AI. There’s plenty of material out there telling us AI is going to transform our businesses and we better keep up with all the hype and make sure that we’re going agentic. But there’s also a lot of evidence that they’re reading that says is AI a bubble over a trillion dollars spent? When are we going to get our money back?

Some sticker shock over the costs that are materializing. CIOs are certainly feeling that part of it.

And a question of, you know, how do we contain our costs but not necessarily slow down innovation?

Some numbers here from Bain around the AI budget. Nearly 40% of companies that measured AI cost savings landed 10% that’s landed below 10% despite targeting 11 to 20%. Yet 90% are increasing their budgets again. So we’re targeting big numbers. We’re not making our objectives, but we’re still increasing our rev our budgets around them. And then just a proof gap From Grant Thornton, 78% lack strong confidence that they could pass an independent AI governance audit in 90 days.

Derek, that quotes for you to comment on.

Certainly that’s going to be on the risk profile of our board members. And remember our board members are supposed to be skeptical. That’s their job, right? Is to evaluate the strategy and evaluate the risk profile of the company that they’re on the board of and ask a lot of questions. And this isn’t the first rodeo many board members are on.

They’ve seen technology promises, they’ve seen transformations, they’ve seen a lot of things that say look, invest here.

And then the question is are they actually seeing the transformation and growth come out of that. Now on the right hand side I have some examples of actual numbers from returns on AI investments and believe it or not, it was actually pretty difficult to find published examples of money being made from AI whether it’s cost or revenue. Plenty of examples of KPI improvements, lots of even more examples of experiments and just good areas to invest in AI agents. But when it came to money, it was hard to find.

Tim and I saw this example on stage earlier this week.

A company, British Telecom talked about $9 million a year increased cross sell revenue over a three week period. Decrease in agent speed to competency where that basically means is in their call centers they the reducing the amount of time it takes to onboard a new call center agent and it’s driven new revenue for them. A shopping Advisor from Nvidia 98% neutral to positive customer reactions using their tool. Milo, this is Lowe’s and I encourage you to go check this out. We’re always looking for B2C examples.

A medical diagnosis in cancer dropping from 30 minutes to to 7 minutes is a good example of something that’s hard to quantify but obviously very important for life when it comes to software code generation. Checkout exceeds.

They have a really good paper on there around Bank Columbia, JP Morgan, EchoStar just quantifying some of the benefits they’re getting in software code generation and then JP Morgan reporting back on the number of hours saved in contract management.

Just of note, 18% of respondents say their organizations collect metrics around ROI from AI which means 80% don’t and we certainly can comment on that number. Not that I love the metric around roi, but I do think we have to start talking about financials when it comes to our AI investments.

So Tim, welcome to the floor.

This is my opening slide. I’ll leave it up here until our board starts filling up. We’re talking about skeptical boards. What’s actually Working. And so I have three questions we’re all welcome to comment on. Number one, what are you seeing around boards and strategic leadership teams around AI investments, value and financial returns? What is the sentiment going on?

Then your thoughts on quicker paths to deliver value with AI and AI agents?

And then this is the softball for Joe Puglisi. How should digital trailblazers set realistic expectations with their boards around AI opportunities, risks, benefits, timelines and costs? We’re going to go all over the board today around these questions. We’ll start. Tim, your what are you hearing from boards and what do you know that’s working in production? Hi, Tim.

[00:06:51] Speaker B: Yeah, hey. So first off, thanks for having me. It was great to spend some time with you this week in Las Vegas.

You know, if you look at the, if you look at the stats for just a minute, take AI out of the equation and just look at the numbers. So we’re having trouble proving out the roi, but we’re still going to throw more money into the pot in terms of trying to go after whatever it is.

You would look at this and go, you’re crazy. What are you doing here?

The reality is that there’s a lot of FOMO that is driving the frothiness around AI today. And so that’s both scary and exciting at the same time.

One of the things that I am seeing from boards is kind of leaning into, or has been leaning into this concept of AI replaces people.

And that narrative started out as a marketing narrative to support a number of vendors, products and where they were going. And it kind of stuck. Unfortunately, the reality is, and I’m trying to pick apart a couple of things that you mentioned, Isaac, the reality is that that’s not where the real value is. And at the same time, we, they’ve been hitting some significant headwinds from a financial standpoint. And we know for decades a good way to buoy your stock is to announce layoffs. So AI becomes a convenient sparring partner to be able to say, okay, we’re using AI, we’re letting people go, and therefore the stock is, is going to go up accordingly.

That’s not where I think the real money is.

I don’t think that’s where the real opportunity is. And we’re starting to see some of these lighthouse companies shift momentum into how do we get more value from our existing employees. So, for example, it’s not replace an employee with, it’s not replacing Tim with AI, but rather to say, how do we get 2x or 3x of the value out of Tim?

Because he has the institutional knowledge, and we’re starting to see this in the call centers too, but in other spaces within your organization, how do you get more value out of an employee? By augmenting what they’re doing with AI. And that’s the new narrative and that’s actually starting to prove out. But to your other point here about delivering value, in order to truly understand value, we have to understand the costs.

And I’ll be honest, it’s really hard to truly understand what your costs are around AI today.

And so when you’re having these straightforward conversations with your board or even your executive team to try and compare and contrast what you’re getting in terms of value from the cost you’re putting in, you have to truly understand the cost and you have to truly understand the value. And, and those are still really, really squishy concepts today.

So there are a couple of things there, Isaac, that you know, we could double click on and talk about.

But I would say that the, the bottom line here that I’m seeing around the lighthouse boards is the shift from AI doesn’t replace people, AI augments people.

And the number one area, and there are a couple of areas that are providing the best value there is in customer experience and customer engagement first.

And then we can get into other aspects of value around cybersecurity and whatnot as well.

[00:10:46] Speaker A: You know, Tim, you and I did were at a lot of conferences the last four months. It gave both of us a pretty wide and deep picture of where investments are being made, what are the new capabilities, who’s taking advantage of them, what are some of the early wins that customers are facing. And one of the things that I noticed is exactly to your point, there are some solution providers who are embracing this idea of AI augmenting people.

Their keynotes were very people centric and very deliberate in their messaging around their AIs are there to drive growth in the ways that they can help people become successful with AI as a partner. And there were a few others that were very much in the automation agentic. They didn’t necessarily embrace cost savings right there front and center, although they talked about it.

But they were definitely promoting a, a sentiment that, look, AI is here to automate a lot more than what we’ve been able to do in the past. And there was a lot, there was a strong absence of people in their presentation. So let’s go down a layer. I just before I hand it off to the board a little bit here, the speaker board, given everything that you’ve seen in there, let’s give me a couple of concrete examples of the conferences that you went to. What struck you out as the most promising agent paths to value that you really got excited over?

[00:12:26] Speaker B: Yeah, there were a couple, Isaac, and you’re absolutely right. I mean, there’s been a lot in the last couple of months. There’s been a lot.

And it’s really hard for.

Forget about boards for a minute, just the broader enterprise. It’s really hard for them to increase the velocity in which they are consuming, the amount of innovation that is being presented to them on a daily basis.

But there were a couple things that kind of stood out for me. I mean, obviously the conference we were at this week was really interesting in terms of how they’re bringing automation into customer engagement.

But another one that was also really interesting is when you look at this concept of personal agents and you have to get comfortable with a degree of automation in what’s being done.

One of the concerns around personal agents, especially when you start talking about tools like OpenClaw and a couple of the others that initially came out in that vein, was that they lacked the appropriate guardrails for the enterprise.

So the risk was you’re essentially giving your credentials to an untrusted tool outside of the realm of the enterprise constraints and guardrails and allowing it to take action on your behalf.

Now, when you started to see things like Anthropic’s Cowork come up, and most recently Amazon’s Quick, you started to see essentially similar capabilities, but with those protection mechanisms built in and of course, some advancement in the space, Quic is something that I’m finding really, really interesting because it’s a very low hurdle, it’s very easy to start to use, but it still provides those constructs that give the enterprise some comfort around the guardrails.

So that was one that definitely stuck out.

The other thing that in the last few weeks that I’ve attended, and it kind of comes back to my earlier comment about just understanding value, is I was at FinOps X in San Diego, and if you’re not familiar with FinOps X, it kind of started out in understanding cloud costs and truly putting your arms around the cost around cloud, and then has evolved into understanding the true costs around AI.

And that’s a moving target, to be fair, it’s a really complicated moving target.

But what was clear to me is there’s still a lot of work to be done to truly understand the cost. Because when you look at the entire value chain, from training the models, building the models, training the models, all the way down to operating them, updating them, providing them, consuming them.

That whole value chain is being subsidized from top to bottom. And so the true cost that you see your $20 a month or 200 bucks a month is not really representative of the true costs that go into that entire value chain. The reason why I bring this up is that we’re working under a model of.

And model is probably not the right term in this context, but we’re working under a construct that suggests that we can build our value equations off of what we’re seeing today in terms of the pricing. And that’s a flawed assumption. And so once you start to understand what the true cost would be, it causes you to think differently about where and how you might use AI. And then once we start to get into the governance of IT and the security around it, that also starts to create more constraint around it, which doesn’t inhibit innovation. And I want to be clear about this, because this is something that comes up in board discussions, is how do we embrace the innovation but not introduce undue risk into the equation?

And that’s a stark balance right now with AI and AI tools, unless it’s something that’s built into an existing enterprise tool. But we have to start to take the frothiness out of the conversation and start talking about some of the brass tacks. And some of the brass tacks include some of these fundamental conversations that aren’t sexy, but it’s a reality of what we need to consider. And one of those is governance.

So there are a couple of areas, I think, like I mentioned, customer engagement is a. Is a hot area, employee engagement, especially when you think about policies and trying to understand policies and guide employees and provide what they need and how they need it, when they need it.

And then of course, just understanding your overall business operations, whether it’s from an efficiency standpoint or going after new markets, going after new areas, that and doing things and elevating insights that you never had access to because the sheer amount of data you had to process was just completely overwhelming. And the tools were just completely incapable of being able to analyze that.

AI has opened a lot of new avenues and pathways in each of these three areas. CX EX and business operations.

And I would encourage enterprises to think about not just the efficiency opportunity, but also what can it do for you that you couldn’t do before. And that’s where things really start to get interesting.

[00:18:22] Speaker A: Thank you, Tim. There’s a lot there to unpack. I want to be able to hand the mic over to my speaker board. Derek, hold off a second. I’m going to go to Martin first. We’re only having him for another 10 or 11 minutes. Martin, love to hear about what you are hearing from boards. Your ideas on paths to value and how you set realistic expectations. Hi, Martin.

[00:18:46] Speaker C: Hey, thanks for having me on as always. Isaac, I think when you look at

[00:18:51] Speaker A: the role of the board, the board

[00:18:53] Speaker C: has two viewpoints on things. One is risk, one is strategy.

And when you look at AI, you see, Tim mentioned it, there’s a lot of fomo, so that comes into things as well. But the board’s looking at, okay, well, does this AI stuff actually drive us forward or is it going to actually lead to our demise? So that’s one aspect the board will be looking at.

Another aspect is the whole strategy, which is, yeah, how are we driving this company forward? How are we actually going to increase its revenue, increase its shareholder value or whatever it might be?

So then you look at, from those perspectives, what’s the board thinking?

And I think it falls into two camps, really. I think the, the first camp is, oh, wow, it’s. This looks fantastic. Yeah, the, the possibilities with AI are endless.

Yeah. It could really take us to a new level.

The ability for us to actually make more of our data, to provide a better service to our customers.

And I think the potential there is amazing.

But then you’ve got the other side of the board which says, hang on. Yeah, is this just another shiny object? Is this just a.

Another kind of technology advancement that’s. That’s going to be more hyped than deliverables? And, you know, obviously we’re seeing, at the moment that. We’re seeing both of those things. We’re seeing fantastic creation, fantastic opportunities, but we are seeing an awful lot of hype and not a lot of value being delivered at times.

So I think if you kind of put all those together, you get a board, you get boards that are very conflicted. They’re kind of, yeah, how do we move this forward? But we see the risk, we see the masses of money that’s been invested. We’re not seeing the value coming back. So I think we’re in this kind of, yeah.

Kind of still early stages in some ways where we’re not seeing exactly how to get all the value out of it.

I think there’s a lot of great examples of fantastic things that are being done, but they’re not the norm. Yeah, there’s a lot of different ways to be doing it. Yeah. And I think, yeah, Tim mentioned it as Well, I think you mentioned it, Isaac. There’s a lot of augmentation going on there. As opposed to replacement of people, you’re seeing people that are far more able to do a really good job or a better job using AI tools as opposed to necessarily getting rid of their job. You know, it’s a case of, right across a lot of industry, a lot of enterprises, people are struggling to cope with the amount of work they have to do and the amount of things they have to deliver. And that’s just got worse year on year on year. So I think what you’re seeing is, you’re seeing AI as an augmentation, enabling people to get the work done that they should have been doing anyway. And it’s. There’s less things falling off the table and I think you’re seeing that kind of, that buffer happening there. So it’s not removing work. People are just managing to actually cope with their day to day.

[00:22:19] Speaker A: Hey, Martin, you know, something you said strikes me as something I’ve seen and it’s different than in the past. You know, when you were pitching board ideas in the past, you made sure you had some really good internal stakeholders at the executive level and you knew that translated up to the board. Right. You need at least one person on the board who is going to echo and champion that investment because you knew there were at least two or three people on the board that were either cost or risk driven and were going to slow the conversation down. So it’s pretty, you know, you had a bunch of neutral people, yet you had tried to get one or two supporters. You knew you’re going to have two or three people talking about risk and cost.

But then you said, today boards are conflicted on risk and reward. And so that’s something I’ve seen where, you know, there isn’t a true person worried about just risk or reward and there isn’t true people are going to champion. It’s like everybody is looking at both sides and saying, does this even make sense?

Are you seeing it that way as well?

[00:23:23] Speaker C: Yeah, I think there’s definitely. You’ve got management teams that are saying, we’ve got to go forward with this because our competitors are doing this. And you’ve got boards saying, well, yeah, I agree, but concerned about the amount of money this is costing us and how much time we’re investing. Is. Are we really moving things forward?

So I think boards are conflicted.

[00:23:48] Speaker A: Thank you, Martin. Let’s go to Derek. Derek, you obviously look at it from a risk and security and operational resiliency standpoint, what are you seeing that boards are.

What are you seeing as the board sentiment when the boards you work with.

[00:24:04] Speaker D: Well, thank you and good morning everybody. The things that Tim mentioned and Martin mentioned are spot on. I mean the boards that I’m seeing right now, they’re looking at the type of artificial intelligence, a lot of interest, but they have very little patience when it comes to what’s the outcome they’re looking at now, what are the measured business values, how is it going to increase revenue, where are my cost reductions? Most of all, as you mentioned earlier, what are the risk reductions and what are the risks are now being exposed and these kind of things they’re funding into, but they’re tied up looking at the innovation funding aspect and saying where’s the return on the investment? You mentioned earlier about the 90 day position that companies are looking to get a return on investment, which is totally unrealistic because the companies that I’m working with today, they go in there, they’re looking to introduce artificial intelligence in the application and realize their own current infrastructure doesn’t support some of the cyber security services they have today, which AI needs to sit on top of. So now they need to look at as, as Tim mentioned earlier, the other cost they need to do introduced to how can I now increase the, the governance, the compliance, the resilience of my current network to support artificial applications. And by doing that now when I introduce artificial int, Artificial intelligence applications, what’s going to be my internal investment?

So now they’re looking at from the hype to now accountability. Where are the metrics, how can I measure the cost savings, how this cycle is going to be reduced, where the risk adjustments, returns based on the risk that I’m finding, what is the clarity to find, Figure out what I’m doing at as far as this roadmap. This roadmap, you know, you’re telling me it’s three years, I need something quicker or sooner with milestones that actually can be measurable. And all these things come into, as mentioned by both of them, the hidden risk they’re seeing all these things that are going wrong with the models they’re introducing, what’s being exposed as far as their data. They want to scale faster, but they can’t do without introducing more risk. And they’re now starting to ask the questions and I think the augmentation that was mentioned earlier is some of the earlier values what I’m seeing with my customers. There was one customer I’m working with now, they said how can we show low Hanging pull low hanging fruit to show immediate risk. And one of the applications is augment your incur employees to do the mundane jobs they were doing beforehand. So for example, 27 FY, 27 budgets are coming up. They asked there’s the things you can do to help. I said, well, what are some of the challenges you have today? They said, well, it takes me about a week to consolidate 20 programs to figure out where I need to go. After writing a prompt that took me 30 minutes, they’re able to get what took them seven days done in literally 30 minutes. So now they’ve reduced by all user AI to augment their work. They’re able to see now the value and augmentation using the product and services, reducing the risk because now it’s just taking what they’ve done. But it’s no longer just an agentic version, it’s now a human in the loop version. And now they’re seeing the quicker return on investments in doing that by seeing the growth, by seeing the realization yes, we can get to faster ROIs, but we need to shift what we’re thinking and not look at putting everything in the hands of artificial intelligence. And I think when they change the mindset and look at the strategy, because they’re trying again, they’re trying to put all these new services and operations into an existing infrastructure that can’t support adequately what they have today.

So the governance piece of it has to be in the forefront. The risk element needs to have to be in the forefront. The strategy, the services, the technology and the education of the employees with AI literacy also has to be in the forefront. All these things are moving targets that involve cost. And as Tim mentioned earlier, as the analogy of looking at what it took for us to introduce and use cloud services, artificial intelligence and other cloud services now, but it creates more risk. These are things that the organizations from the board level need to be concerned with all the way across all the business units because it is going to be an impact. And I think when they look at this then they can see there is clarity, you know, but they need to get people thinking in the same direction at the same time because everybody’s all over the place, we’re still at the hype and it’s just now they’re starting to see the realization it’s not what it script out to be what it was planned to be originally.

[00:27:55] Speaker A: Thanks, Derek. I mean one point I think is worth counting here. I mean we’re talking a lot about experiments and which experiments lead to production. You know, underneath that is, you know, what’s the value? And now Tim is reminding us how important it is to make sure that we have a realistic projection, or at least a projection on where costs will land out, that we have value that’s significantly going to exceed what our cost expectations are. But there’s another conversation with the board saying, look, there’s a bunch of capabilities that are AI enablers that we need to move up and increase our capabilities around, whether it’s legacy infrastructure, lots of data debt out there, security gaps, and one that you didn’t mention, Derek, is just change management capabilities. Right? With all the, you know, every tool going from forms and reports and, you know, manual workflows to automation, conversational AI, agent driven, it’s a completely different way of working.

And Tim and I got to see a lot of those demos during the conferences that we went to.

We’re going to have the same spectrum of people who embrace it and people who are fearful of it. And underpinning that is

[00:29:11] Speaker E: all of the

[00:29:12] Speaker A: layoffs that we’re seeing that come from lots of different angles, not just AI driven. And that’s what our employees are dealing with. Let’s bring Joanne up. Hi, Joanne.

I don’t know which are you going to take board for a thousand AI agents past the value for 500 or setting realistic expectations. Where do you want to go, Joanne?

[00:29:32] Speaker F: All of the above.

Because what I hear from boards. First of all, let me start this out by saying there is still FOMO and a lot of confusion, and I would say it’s definitely a lot of confusion that’s leading to the FOMO about what AI is, what kind of AI you need to be using and what it can actually bring to the table.

And I think that’s part of the issue that really, as educators, thought leaders, influencers, whatever we need to help dispel, because everybody goes by what they see, like foundational models. Oh, there’s chatgpt, oh, there’s anthropic, there’s cloud, there’s cloud code, there’s cloud cowork, there’s, you know, every variation on the theme, whether it comes from Google or anybody else, tons and tons of open source models. And they don’t necessarily understand, particularly at the board level, that it’s not about the tooling, it’s not even about the foundation models that you’re choosing, it’s about what are you trying to do, what is the business value or the business outcome you’re trying to achieve and how can AI, or whether it’s augmented machine learning, generative AI or agentic or even physics AI can actually bring to the table. So that’s the first part of the conversation. Once they start to understand that there’s a lot of different cats in the herd, they start to realize that one of the, and this is the biggest change that I’ve seen recently is boards are starting to recognize that, that they’re paying a huge amount for infrastructure and they can’t say what it’s contributing to in terms of business value. They can talk on a project level, maybe from an ROI perspective, but generally speaking, there’s a gap between what they understand the outcome they’re trying to achieve is, and how AI fits.

And it’s a connect the docs exercise. And that’s going on a lot because the things that come out in projects that are sort of pilots or even proofs of concept that are not translating well to the enterprise level, which I’ll get to in a second, are those that were not, I don’t know, were not well defined in terms of what kind of AI needed to be used. So that’s part of it. And that’s leading to a discussion about, hey, we’re paying a million dollars a month for cloud.

What is that data?

How much return are we getting on that data?

What value does it create for us in, in accomplishing a business outcome?

So that’s where the conversation is starting to turn. And because of security and to a lot of the points that Derek made, it’s I should. We should be owning our own AI, not buy a frontier model or have an opportunity or put so many rules around it that we can’t use a frontier model. But we should be leveraging our own infrastructure, our own data, to try and drive more business value out of it with the help of AI.

And that’s where the conversation is starting to shift a lot.

So in terms of generating value, it’s also about understanding that there’s a world of difference between generative AI and agentic AI and agents are not one thing.

They don’t have one specific role. They work together.

So it’s multiple agents doing multiple things and it’s designed to be used in very dynamic environments. That’s the best rule of thumb that I would say is coming to light at the board level.

Do we do this across the board, across every employee and every part of our operation, or do we start looking at cherry picking, where this is going to drive value in the most significant way in the shortest period of time, where we’re actually throwing the leverage into our own already paid for sunk cost of infrastructure and data.

And so I would say, you know, the realistic expectation is if you’re looking for something that’s going to give you an ROI in two days, forget about it.

If you’re looking for something that is going to help you eliminate some of your technical debt, put in more governance than you already have and more execution reliability, then look at choosing the outcomes that are maybe a three to six month return and are doing so because agents learn as they go. Frontier models are constantly changing and it’s a way to find the single path that’s going to get you farther, faster with a reasonable rate of value being returned. Not necessarily hard dollars, but value improvement.

[00:34:50] Speaker A: Thank you, Joanne. I mean, Joanne, you know, listening to all that, one of the things I’d remind all of our digital trailblazers here is there’s a lot of confusing jargon out there that means different things to different people depending on what two or three contexts they’ve read them about.

And you know, Tim can attest to this. Every four to six months, you know, the, you know, the solution providers, the analyst firms are introducing even more terms to confuse things.

You know, I’m not suggesting go up and educate the board, that’s usually a bad recipe, but being able to make sure that in the context of your presentation, you explain, you know, what your organization means by an agent, what orchestration means, where value is being determined. I think the Most important thing CIOs can do is make sure that there’s a set of principles that you’re using for how you’re making decisions around AI investments. That’s a good conversation with the board, you know, and what are your criteria around risk? What do you mean by growth?

If you’re going to invest in customer experience, what are you focused on in terms of objectives?

What are your cost guardrails so that you make sure that there’s some discipline in place all the way from top to down. These are things that the board is going to be interested in when you’re talking about AI strategies. We’re going to go to Joe, Liz and John next, folks. Remember you are at this week’s Coffee with Digital Trailblazers. We meet almost every week to talk about transformation, AI capabilities, how our organizations are evolving, where growth is coming from. We’ve had some really spectacular episodes that you can find and listen to@drive.star cio.com coffee and you can always find our next episode by going to starcio.com coffee that’s a redirect to our upcoming episodes. We are off next week, July 3rd, for the 4th of July weekend. We’ll be back on the 10th talking about reskilling mid career leaders, what senior talent need to stay relevant. And on the 17th, we’ll be bringing back a conversation that I think needs to be revisited in the AI era. Does IT leadership in the AI era require an IT background anymore? And I’m sure we’re going to get a little bit of a debate around that at that episode. I’ll announce the 24th and the 31st episodes next week and that’ll be our program. Folks. We’re talking about board skeptical boards around AI. Remember, that’s their job is to be skeptical and to ask questions, which means you need to have a good understanding of what AIs are working in production, particularly for your objectives and in your industry.

And we’re going to go straight to Joe.

Joe, you want to talk about boards, AI agents or realistic expectations? I have a good guess where you’re going to go.

[00:38:06] Speaker E: Well, I think trailblazers should take a lesson from history.

Tim brought up the cloud phenomenon, right? I’m going to move to the cloud and sunshine and rainbows. Everything’s going to work faster and cheaper.

And what did we learn through that experience? We learned that lift and shift, just moving the workloads to the cloud. Long term, that didn’t really help.

It might have been a short term alleviation and some capital investment perhaps, but over the long haul, we suddenly started feeling the pain of very expensive and clunky cloud operations.

We had to learn that you needed to reinvent your applications, take advantage of the cloud in a way that delivered value.

What Joanne would say, right? It has to deliver some value, not simply be a substitute for what you’re doing today.

And we need to take that lesson into the AI world.

AI is an opportunity to reinvent the business.

That’s hard to do. And the board has to be clear about, as I think Tim and Martin said, what is the objective? What are we trying to do and why does AI fit the bill?

How do we reinvent our workforce so that we deliver more value to our customers so we can deliver new or different products so that we can operate more efficiently? And I’m sure, as Derek would say, how we can either mitigate or reduce existing risk? How do we leverage our data? How do we get more value out of what we have through this new set of tools?

It’s not a simple monolithic thing. It’s not AI. As Joanne said, it’s a lot of different components, a lot of different pieces and parts and ways in which we can accelerate the business through enabling our employees to do more or different things, do what they do now easier or faster.

But really it’s how do we re engineer this whole friggin business so that we can just be a better operation overall.

[00:40:26] Speaker A: Joe, I agree with you with one caveat and that is I think we, you know, when you’re the CIO or reporting to the board in another capacity, you need to have some specifics about what you’re targeting. Because they’ve heard the word transformation before, they’re hearing reinvention more.

There are starting to examples like the ones I showed earlier. Customer experience of shopping, going on medical areas.

I’m sure when Joanne speaks next she’s going to talk some. In manufacturing you need to have some concrete examples about what you’re focused on so that they’re not just seeing, hey, this is just another new capability. It’s expensive, it’s going to take some cost out. I want to, I don’t want to be left behind. But you know, what is it that we’re exactly targeting? And I think it’s okay to be wrong about these things in that you’re going to share four or five areas that you’re focusing and maybe two are going to be ones that you really bring to production and deliver value. But I think we have to be much stronger about our articulation about why, who and what and then where AI is fitting that equation. That’s a softball.

[00:41:51] Speaker E: I completely agree with you, Isaac. I’m talking from a board perspective. I’m not talking about the CIO going and presenting something. I’m saying that the boards should learn from history, that it’s not so simple to take a new technology trend and slap it into their existing operation and expect to get value.

[00:42:12] Speaker A: So combat the myth that this is a, you know, I don’t know, 1 to 3 year, you know, transformation event going to take 10% of cost out and 3% lift and growth.

[00:42:25] Speaker E: Exactly.

[00:42:27] Speaker A: Thank you. Thank you Joe.

I think I have Liz next. Liz, go ahead.

[00:42:33] Speaker G: Hey. Yeah, so I’m gonna kind of bring us back down to basics. Okay.

So I know I always say governance is not a four let word but I, and we were talking about how oh the cloud like revolutionized everything and we should have learned from that. This, this seems more like the dot com era to me, like just insanity, you know, over overvalue, over hyped expectations. And the maturity curve associated with AI is a real thing. And I don’t think that we’re necessarily like, we’re not even mature. Mature enough to know that there’s a maturity curve. Right? I mean, I hear Joanne talking about AI agents and, and the level of complexity, nuance that is required to engage in those conversations is.

I’m 100 positive is way beyond what average board members can handle, right? So if we back up and start talking about what a board is actually looking for, especially when it comes to AI, there’s basically two types of members, or maybe three. One is fomo, right? Running after everything.

The other one is fear of being overtaken and being completely blotted out. So they’re just afraid of the whole concept and don’t want any of it. And then there’s the ones in between who really are looking to see, how can I make money out of this? And if you go back to that, if you just look at why you’re in business to begin with, you’re here to provide value at a profit.

So how come we have abandoned the basic governance structure of doing POCs or pilots that actually demonstrate with real business and success criteria what they’re looking to find out about and making sure that we’re actually looking at this in a small way that’s not throwing millions of dollars after it. I love a comment that Jay Cohen put in the. In the chat, he talks about identify the boring wins, right? Find pain points where, you know, you can actually make a difference for a small amount of money and experiment with AI in a way that doesn’t blow millions of dollars, but still gives you an anchor in moving up the maturity path as long as those success criteria are created as part of the project.

[00:44:58] Speaker A: You know, Liz, I think part of the issue is that there’s sort of a, you know, a feedback loop the better leaders are doing, you know, in terms of positioning what the value is in an experiment or a POC up front. I think they know how to do that. Okay. I think in terms of connecting that to the agents and the AI capabilities they want to experiment with, not so easy to do just yet without making an upfront investment and saying, let’s go get a bunch of people to try these things in a governed way and then see whether or not it’s proving value.

And, you know, when we talk about 20 or 30% making it from value stage into production, it’s that lack of understanding until you try it.

This is a much less deterministic set of work that needs to happen in the organization.

And, you know, is your data ready? Are people ready?

Is the, are the agents that you’re working with, which predominantly I think was Cheryl on the comment she was asking are people actually building agents? The answer is absolutely yes. But the majority of them are trying buy agents built into platforms that are available before they’re building them and if they’re using them and finding that they need to get better results out of it. They’re investing in data, in some cases, they’re extending them. They’re doing a lot of things with new things this year with orchestration, connecting low level bots into more complex workflows to see how those work out. So there’s a lot of work to bring the capability, connect it with people, then connect it to, with value and then seeing if that equation is working or not.

[00:46:50] Speaker G: I’ll, I’ll even that, even that requires a definition of success.

Even that require. Right. And that’s the rigor that basic governance is.

[00:47:01] Speaker A: Thank you, Liz. Let’s go to John. John, good to see you again.

[00:47:05] Speaker H: Hey Isaac, happy Friday. Thank you for having me on. I just, when I hear this conversation, I’m largely agreeing with everyone and I think it’s just really important for, for people to remember that AI is a general purpose technology and it’s as transformative as like electricity. Like when electricity was brought to our society, it took a time to use it, but it impacted every part of our society. And when we got phones that impacted every part of our society and when we got the Internet, that it, you know, impacted every part of our society and took a long time to roll out. And so this widespread adoption of AI and making it so it’s accessible by, by almost everyone on the planet, it’s going to take some real time for us to figure out.

And so it’s just, it’s, it’s to the, to the people like I think Martin and Joe that made comments about, you know, how, you know, there, there’s going to be an adoption for this and in time for us to learn how to use it. That’s, that’s absolutely correct. One of the things I’m seeing is, is anytime an issue is raised to a board, I’m seeing the board members, you know, like an objection to maybe there’s, there’s, maybe not enough people to do something or a process is not working right. Almost every, every time I’m hearing anything raised and it’s an objection or a challenge board members are proposing like is there a way to use AI for this? I’ve seen that anytime that there’s a new project that, you know, they’re asking like how is AI in this when we’re working with international users? To the comment on Martin, I’m seeing a huge advantage to, to workers and, and where I personally see it is in the Philippines and, and India. And so that when people are talking on the phone, having live transcripts of what the, what people are saying so they can see. Right, right. You know, a translation of, of what people are saying, having automatic summaries generated for the conversation, having hints provided to people, action items summarized after the fact. So any, any company that has a support for us, this AI has been a huge, huge benefit.

And on the costs, I, I think anybody that’s using a monthly subscription plan, I think we’re getting a massive discount on it.

I was using the Mythos API from Claude and, and when you use an API, you’re paying a lot more of a true cost. And, and it’s probably estimated that compared to the, the flat subscriptions compared to the API costs are savings of 10 to 30 times.

And so it’s kind of like Uber when we were using the car share stuff, it was so cheap in the beginning. But we’re gonna, at some point we’re gonna have to really start paying the true cost of these things and then it’ll probably change our behavior.

[00:49:45] Speaker A: Thank you, John. Tim, I want to bring you back.

You’ve been hearing everybody speaking here. I know you have a hard cut off coming up.

Just your thoughts, just connecting the dots between value and, you know, production and experimentation in between.

Any parting thoughts for everybody before you drop off?

[00:50:08] Speaker B: Yeah, there’s, you know, it’s all great.

There’s some great perspectives that have been shared in the conversation today, and I would encourage folks to kind of go back and rethink how those comments kind of apply within their organization, whether it’s at a board level or even further into the Org. You know, one of the things that, that I always come back to with boards is what is the role of the board? And I think sometimes the board members, especially when they get excited about new technology and innovation, they can kind of lose focus on, on what their role really is. Or likewise, if they’re not getting the information they need, then they feel like they have to go a step further or two further beyond what their role typically is.

But at the end of the day, we all, whether, whether you’re a board member or an executive leadership team member, you have to think about your business and what you’re doing. And, and one of the things that AI allows us to do, whether in small ways or further into the organization or at a macro level, at a board level, is rethinking and reimagining our business.

So it opens doors that we haven’t had access to in the past.

And some of the questions, some of the really good hard questions that I’ve seen come up amongst boards is if we were to do things differently, what could we do today that we couldn’t do before?

And how does this lead to differentiating our business and accelerating our business in some ways or going after a new market, Revenue growth, revenue expansion, TAM expansion, How do we use technology, whether it’s AI, black magic or something else, but how do we start to use the tools that are available to us to go after these new opportunities? And what that really means is we have to, number one, understand our business, which arguably a lot of folks further into the company, they don’t understand truly what their business is.

They might understand it at a layperson’s level, but they truly don’t understand the dynamics of their business. And then the second is how could you think about this differently? And so it’s more of a thought experiment initially rather than kind of getting caught up in the technology and saying, ooh, ooh, AI, AI, AI, how do we use AI?

But backing away and saying what are we really trying to do here and what new opportunities do we have available to us and at what cost? And that kind of gets to that equation of understanding the value more so than just the cost. I think one problem that I see at a board level as well as at an ELT level is they get over fixated on the cost number rather than the value number.

And so it’s important to make sure that, that you, whatever level you’re at, you are helping change that narrative and change that dynamic to focus on what that business outcome is and that you’re focused on the value that it brings to achieving the objectives that, that you and your team, whether your team is the board or, or the team is the elt, are trying to set out to do?

So those would be a couple of my kind of parting thoughts, but a lot of really great conversation today for sure.

[00:53:45] Speaker A: Thanks Tim. Thanks for joining us. I do like this leaving comment. What couldn’t we do before?

I wrote a CIO article about that about a year and a half ago saying, you know, whether you used to yearly planning cycles or three year plans or six month planning cycle, you know, think of blue sky thinking almost every monthly, the capabilities are changing so much.

Think about getting your experts to really understand what’s available and Platforms that you have already and how that’s changing.

An article I wrote about SAP Sapphire. Last year they introduced 40 AI agents. This year they’re up to 200 AI agents Varant introduced, you know, talked about 50 bots around customer experience that we saw earlier this week. These are things that you can go try. You talk about low hang fruit, but even before trying it, you know, connecting the dots between what are your pain points and what is available to you without having to go build an overly experiment with. I think that’s the opportunity facing most enterprise leaders today and ways to have a story and a conversation with the board saying this is what people are doing with the platforms that we have and ideally in the industry that we have. Derek and Joanne, you’re going to give us our last words. We got five minutes. Go ahead Derek.

[00:55:10] Speaker D: Sure thing. Tim was spot on when he talked about the value and I think a lot of companies overlook that because they’re looking for the roi. But I think also the value is set in realistic expectations and all the challenges that AI brings to it. Now when you look at you know the, the quick WINS, you know, 0 to 6 months and then looking at the process transformation 6 to 12 months and then the strategic revenue models sometimes 12, 18 months out, those are things that you look at from day one. But also on the bad side or the risky side is know that AI introduces now new risk and it expands your attack surface throughout your enterprise and your business culture and they need to understand what that’s going to bring. So now as the board is looking for awareness they need to understand the control and the governance associated with that. I think also when you look at the business metrics as they’re rolling out these new business cases, everything should be tied to a business metric. The growth revenue, the cost, the risk reduction, the customer experience. All these things are measurable and they should be part of every business case that the word rolling out. But I think also the biggest thing is the resilience aspect. You know, everything that they do now build AI resilience and has an embedded and not after the fact and, and we’re seeing too many companies now, they’ve already done the agents, they’ve done the AI, they’ve seen where the holes in the risk and how their things have been leaked out into the ether and now they can’t get it back. So when you look at establishing the governance models, establish what you know there’s going to be data, data lineage and access control. Look at the fail back, look at the monitoring AI threat Intelligence and AI threats that are lurking within your ecosystem and what your business culture will introduce. I think we also looking at also how do you now adopt identify incident response based on AI failures or AI exploits. Those are gonna be a little bit different than what you’ve experienced before. So you gotta change your policies and your models. What you used to do to now to work with a new infrastructure, new type of threat, work with artificial intelligence. Now when you look at this across the board, I think the bottom line is they want to use AI, they want to trust AI, but they have to put it in a sense that’s going to make sense for them where they can have real, tangible and measurable outcomes.

[00:57:09] Speaker A: Thank you, Derek. Joanne, last word?

[00:57:12] Speaker F: Last. Well, okay, the last word is differentiable value. It’s something that we haven’t heard for a very, very long time since we had the democratization of data and B2B back in the, you know, turn of the century, so to speak. But AI gives boards the opportunity to think about competitive advantage and differentiable value in a very new way. Because the capabilities inherent in AI give us the tools and give us the capabilities to start driving new revenue streams to do innovation in a different way using technology. And what I mean by that is innovation around a whole lot of data that they’re not doing anything with.

And I’m talking about commercializing data and data products, but also looking for the gaps in their execution and closing them. Because you have a, you have a bunch of different areas where gaps exist. And I’m talking about time to data, time to value, all of those kinds of things. The faster you can close those gaps, the more opportunity with AI, the more opportunity you have to create differentiable value. And those old school thoughts of being easier to do business with, being more aware of your customer and contributing back to your industry, those are all coming full circle now.

Companies are looking for that because over the last 20 years we all pushed digital transformation and technology and the like. But what we missed in the process, or what a lot of organizations missed, was what made them so successful previously needs to come back. And whether it’s under the guise of customer experience, better product quality, as in manufacturing, all of those things are being contributed by AI as the vehicle to get you there. They need to start focusing on those things and capitalizing on them.

[00:59:20] Speaker A: Thank you, Joanne. And to all my speakers, folks, not every company is as far down their AI journey as it seems. I work with many companies through their journeys, as does Tim and Derek. All three of us run advisory programs. Tim is focused on Fortune 2000. I tend to work with small to large enterprises and Derek works on the operational resiliency and security side.

Do reach out to us if you need help or on your journeys. I do have an AI strategy and governance workshop. I have a world class IT workshop. I know Tim does advisory with many CIOs and Derek works with many CISOs. We also have a good number of us who provide thought leadership. Joanne has her writing that she does. She has her company real AI that you should check out.

We have Jo myself who do a lot of writing. Liz has been joining in. She’s gotten a couple really interesting posts on LinkedIn recently and Tim has his podcast. So all of us are sharing our knowledge around AI in different formats. Reach out to us if you don’t know how to find them if you don’t know how to find the Coffee with digital trailblazers, we’re@drive.star cio.com Coffee is where you can find previous episodes. Starcio.com Coffee is where you can find the next episode. And and the best way to stay up on my writing is@star cio.com driving-digital is my newsletter. Tim, thank you for joining us. Can’t wait to have you again everybody. Have a great weekend. Remember, we’re off next week July 3rd.

Our July 10th episode on reskilling Mid Career Leaders. What senior talent needs to stay relevant? And then on the 17th, this IT leadership in the AI era require an IT background anymore? Two upcoming topics folks. Have a great weekend. Speak soon.

StarCIO Digital Trailblazer Community