Episode Summary

This was the 185th episode of “Coffee with Digital Trailblazers,” in which Isaac hosted a discussion on shadow AI with special guest Wayne Saden and panelists Joe, Joanne, Derek, Elena, Heather, and John. The conversation focused on the real risks and practical guardrails around shadow AI, which goes beyond traditional shadow IT with new risks including data leakage, agentic AI poor decisions, and potential legal compliance issues.

Wayne explained that shadow AI creates risks through data leakage into AI training models, unintended disclosure violations, and the rapid spread of agentic swarms that can cause damage faster than human oversight could prevent. The panelists discussed various scenarios and solutions, with Derek emphasizing the need for threat intelligence monitoring tools and real-time visibility, while Joanne highlighted the importance of understanding where human judgment needs to remain in the loop when deploying AI agents. The group agreed that effective governance requires moving beyond simple policies to a holistic understanding of how AI initiatives affect the entire organization, spanning departments and systems.

Speakers

• Host – Isaac Sacolick
• Guest speaker – Wayne Sadin, Transformational/Turnaround CIO
• Digital Trailblazers – Derrick Butts, Martin Davis, Joanne Friedman, John Patrick Luethe, Liz Martinez, Heather May, Joseph Puglisi, Elena Putilina

Discussion

1. Shadow IT focused on tools. Shadow AI has a tool challenge and more. What are the other real risks around shadow AI that we’re not talking about enough?
2. Beyond writing and communicating policies, what practical guardrails should every company implement—not just large regulated enterprises?
3. How should Digital Trailblazers approach a Shadow AI problem and turn a ‘no’ into a “yes, but” while identifying the underlying business need?

Research

Whiteboard

Transcript

[00:00:02] Speaker A: Greetings everyone. Welcome to this week’s August 21 episode of the Coffee with Digital Trailblazers. So happy you are here during your summer months, your lazy Fridays. Before we hit Q4 and joining us on one of our many topics around AI, today we’re going to be talking about shadow AI, real risks, practical guardrails and hinted innovations.

I’m thrilled to be here. Do say hello on the Common Stream. We’ve had some really active ones the last few weeks.

I want to hear about your shadow AI stories, I want to hear your questions around it, I want to hear about your solutions that have worked and I want to hear about your turnaround stories, how you’ve taken a shadow IT or a shadow AI issue, had a conversation with the department or the person who has been in the front of that and turned a risk into an opportunity. And that’s going to be our conversation today.

I have a special guest today. Wayne Sedan and I have been friends going back to the days of CIO chat, which is still up and running every week on Thursday, 2:00pm Eastern Time. Our good friend Diane runs that program, hires highly, highly recommend it.

And we met back then. We met at one of the get together events that we had back in the day. And I’m so glad he’s getting back into social media and talking about what is, talking about what is happening here. I can’t get this graphic to disappear. There it goes. Okay, so I want to talk about shadow AI and share some of the details that I found during my research. Hello Amit. Hello Fadi. Hello Steve. Hello John. You know, my fear of doing an episode around shadow AI is that when it came to shadow it, we had an easy job as CIOs. If we wanted to do a conversation starter at a conference, we would just say certain trigger words and shadow it was probably one of them. And you would get into a huge gripe session among every CIO and CISO about this problem. And so I’m going to ask my pals, I have a full suite of people here today. Let’s not get into a gripe session. But on the other hand, when I did my research, I was like, oh, this is a bigger, wider, deeper, more complex problem than somebody just installing an application or a plugin or going to a website.

And so I’m sharing some details over here from Black Fog. 63% of respondents believe it is acceptable to use AI tools, tools without IT oversight. 51% admit to integrating AI tools with other systems without it’s approval.

We’ve got a problem here. This is everybody in our organization.

IBM’s data breach security incidents involving shadow IT and organizations have more than doubled in 2026 to 43% and it’s costing breaches are costing real money.

Some of the real issues data loss, operational disruption and reputational damage.

Some news stories in here and I’ll be honest with you, it’s a little hard to separate shadow AI from other stories. But we’ve had our first SEC filing around the use of shadow AI.

74% of organizations had more AI tools running than they expected.

I am sure that number is actually higher. Shadow AI has also led to data compromises at 63% of organizations. This is coming from Israel Defense. They’re soaring inside breach costs in the healthcare area.

And then the story on CX today, a certain company that I won’t name had a data breach and it highlights the issues with customer experience around shadow AI tools. I also left you a couple things here around some practical guardrails. Gartner has a framework there.

It says applying uniform governance across AI agents will lead to enterprise AI agent failure. And what that basically means is saying no doesn’t work.

Okay, saying this policy applied to all of AI doesn’t work.

Use controlled AI environments. Vendor agreements should address whether inputs may be retained, reviewed or used for models. I shared you two links in here coming from two different law firms talking about shadow AI behaviors.

This is one involving vendors. So those of you on the procurement side might want to click into here. And then we use AI notetakers all the time, but there are places and times where you should not be using them. Anybody in HR might want to click into this architect Mayor Brown study giving you some guardrails about locations, states, countries, types of meetings where you probably should turn off the AI NoteTaker and how to set policies around that. I also shared you some links in here about hidden innovation. How to govern shadow AI without Stifling innovation comes from Forbes, the seven Steps for Turning Shadow IT into a Competitive Edge. That is my article that’s specifically around it. Hope you’ll click into that one that I published on cio I think it was last year and how to Come up with AI Use Cases, an ideation technique developed based on the analysis of shadow AI T usages. That’s really the punchline here is when you start finding shadow AI in your organization, it’s probably a risk that you haven’t communicated. It’s probably a speed factor. Your organization’s trying to move faster than you can keep up with and it’s also an opportunity, taking a wrong and finding a way to say yes, but here’s the better way of doing that. Wayne that’s our conversation today. Hello, Wayne, Derek, Elena, Heather, Joanne, John and Joe are all here today to talk about this exciting topic about Shadow AI.

As you can see, it is a bigger problem than just somebody using an LLM when they’re supposed to using a different one that the organization sanctioned.

Wayne, provide us a quick introduction and let’s talk about this question.

It’s more than an issue around tools. Shadow AI has an issue around data, around how we’re using these different tools. What are the real risks around Shadow AI that we’re not talking about enough. That’s my first prompt today. Wayne, welcome to the floor.

[00:07:03] Speaker B: Thank you, Isaac. And thank you for asking me to introduce myself so very briefly. Wrote my first computer program in 1968 in high school.

I’ve been a CIO since the early 1980s. I have seen Shadow IT for many, many years. I want to tell you, Isaac, I tend to call it Rogue IT rather than Shadow It. I think Rogue captures more the spirit of what’s going on.

It gets people’s attention more. But you know, back in the day, Shadow it or Rogue IT was somebody doing a giant macro in an Excel spreadsheet. In fact, as a new cio, I’d often find a system running when the person who wrote the macro left. And they would then say, hey, it, we can’t close the books, come fix it. And I got really good at million row excel spreadsheets about 30 years ago.

And the damage there was, of course, people going around it could do all sorts of things within the walled garden called the company, and then fast Forward into the 21st century and we have everything open. We’re connecting to the world. And so a lot of Shadow IT allowed us to make connections around it. In fact, when Salesforce first started, I think their whole business plan was let’s work around the cio. I worked on several projects when Salesforce just started with the sales rep trying to get me to help them go around the IT department and go to the CFO or the CMO or somebody. So it was a credit card. Give me a credit card and we’re golden. And that sums up the state of Shadow IT typically.

Now, Isaac, I’m not blaming anybody. The real reason people do Shadow IT and Shadow AI is not because they want to be evil or they want to be dumb. It’s because it generally is overworked, overstretched, and under optimized. To solve problems and so IT can’t get to them. Is that an IT problem? No, that’s a board of directors problem. That’s a CEO problem. That’s a resource allocation problem.

But you’re the person running a big team, big department. You got to get stuff done. So now there are tools, IT and now AI, that allow people to solve their own problem.

And so look, at best, they are duplicating resources. They are maybe using unapproved databases and fragmenting the data estate. They potentially are opening up attack surface. It’s not good, but that’s just shadow it. So, Isaac, I don’t know if you want to, we can stop there. I see a hand up and other people comment on this. No, keep going, keep going.

[00:09:37] Speaker A: I want to hear your vantage point about what’s happening in shadow AI that goes beyond these typical problems from shadow it.

[00:09:46] Speaker B: I have to be careful. I’m from New York, so I tend to talk over people.

So here’s the deal. With shadow it, you could damage the company. You could not be able to close the books. You could have multiple databases and so on.

Sh. I break into two categories, the speed of which this whole industry is moving. You have the LLM problem, which is tends to be data leakage, and then you have the agentic problem, which could be active destruction.

So let’s do the first one. If I’m using an unapproved AI, I am leaking company data into that AI’s training model. I’m leaking it into the workspace of the AI and making it available to everybody.

And that’s a problem. So I used to tell people I worked in a multinational organization, if somebody wrote a blog in Bolivia, it’s not going to be seen by that many people. But now in the world of AI, that Spanish language, Bolivian blog becomes available to somebody in Paris, France, or in China or on Wall street, who says, wait, I didn’t know that.

And you have the law firm articles. I want to point out. In a public company, there is a regulation, an SEC regulation called REG fd, which stands for full disclosure, which says a company cannot favor a group of investors by leaking information to certain people without making it available to everybody. And AI allows that to happen inadvertently. I say something, people hear it, other people don’t. They can bring a claim, and that could be a claim against the company, or what’s known as a derivative lawsuit against directors themselves for not overseeing it properly. That’s bad enough. Now that we’re moving into the agentic world, we have the ability to make Mistakes faster than any human could ever make them. In the past. If you made a mistake, you type some keystrokes, something happened. The security people looked at their logs. You didn’t type the keystrokes again because you went for coffee. They had you blocked by the time you came back with the coffee cup.

Now I can start up an agent which spawns other agents which spawns other agents. And all of a sudden I have an agentic swarm potentially contacting all of my customers, telling them something erroneous, or potentially throwing data out into the open world, or potentially doing any other kind of thing that I’m not controlling. So you get the attack surface problem, the data leak problem, the inadvertent disclosure problem. And agents tend to operate in both directions. They’re sending data and receiving data. All of a sudden, maybe I’ve opened a back door into the company’s cyber security vault. And so I’ve created every kind of problem you can create in it at a faster rate than anybody’s ever done it using technology that’s changing about every day and a half.

So just imagine the nightmare for the cio, for the CFO, and for the chief legal officer. And that’s the. That, to me, is the state of the art right now. Isaac.

[00:12:40] Speaker A: I think the only thing I would disagree with you on, Wayne, is I would start with a nightmare for the chief legal officer and the cfo.

When these things go wrong, it’s going to be their doors that are knocked on first in terms of the business implications.

Of course, they’ll come bulldozing down the CIO and CISO’s door soon after that.

But like you said, you know, the people, the humans are responding too late. Let’s go around the room.

Derek, I’m going to ask you to prioritize because you probably could write a small research paper with like 18 things that can go wrong with shadow AI.

Let’s start with what keeps you up at night.

[00:13:28] Speaker C: Yeah, absolutely. It really depends on the environment that you’re working in as well, because I’ve worked with everything from small business to universities where shadow it was students creating businesses out of the room and sucking the bandwidth out of the university. But, you know, looking at shadow AI from, you know, it’s not just employees use an unauthorized tool. That’s kind of like the tip of the iceberg. I look at it as uncontrolled decision making. So now you’re using. You’re talking about gentic AI. You’ve got these bots making decisions they’re actually utilizing and Getting into other systems without the authorization to do so. They’re generating content, they’re analyzing data, then they’re helping other decisions make employees that may not be correct. This is going to impact the operations, it’s looking to impact the revenue and everything else is going in. I also look at the fact that now it’s making decisions and pulling data now creates knowledge leak. We now have the point where data, intellectual property, other things that may be confidential are now being seeped into the ether without the business knowing what’s taking place or how much is being put out there. And once you put it out there, you can’t get it back. So things going from business models to strategic plans, all these different things come into play. I also look at the fact that it’s Wayne mentioned, you know, when you got agentic AI and all this misinformation, all this stuff now affects the enterprise. And as you mentioned earlier, you know, the biggest impact is not going to be just the business, but the people who are going to be on the hooks for it. So you’ve got your AI, chief AI officer, you’ve got your legal team, you’ve got your cfo. In some cases, the CEO was taking place. I look at this as a business resilience issue.

When you look at it, it’s not just it, but it’s unmanaged AI. When you look at AI having the ability to now search and query things when I haven’t opened up browser or anything of that nature, that’s scary.

So these things that are being affected, the things that need to be talked about, how is it going to affect operations? How’s it going to impact legal compliance? How’s it going to affect the reputation and the branding of the company? But also how is it going to introduce more cybersecurity risk simultaneously at speeds never seen before? If these entities and businesses don’t have the ability to detect it with the tools that they need to detect artificial intelligence threats at the speed at which they move, it’s really a big concern.

[00:15:36] Speaker A: Thank you, Derek. I mean, I’m glad we’re bringing up this issue of both generating content and I would probably add generating content, interpreting content without researching the sources. A lot of legal issues around that one.

Generating content is again, you know, pushing something on your website that’s anywhere from slop to plagiarizing and then protecting your ip. Right. Somebody pushing something into a language model that’s an open model, it’s not under enterprise contract.

And now that model can do what it wants with it, depending on or not, depending on how the user configured it. All problems, Joe. And Joe, my data points somewhere around 50 to 60% of employees saying that’s okay.

That same research paper didn’t have a data point, but it said that boards and CEOs are opting for speed over safety. So I’d love to hear your general comments and then go one step further and tell me what are you saying to the board? Saying this is a real problem.

[00:16:48] Speaker D: So I want to go back to Wayne’s opening commentary and the term rogue.

I think in part the shadow it or use of AI sort of under the covers, is indeed rogue. But I think to understand this problem fully, I want to break it into two major components.

There’s innocent use of AI, and that’s derived from people who are curious to see what they can do with it.

And to some degree, I think that should be encouraged.

There are people who have knowledge of AI and can use it effectively.

Again, I would say that should be encouraged. And there’s innocent use because of just frustration.

So hard to manipulate this stupid spreadsheet. And I read somewhere that Gemini can do it for me and I’ll figure out how to do that and it’ll help me out. Those are all innocent and I think, you know, sort of positive uses.

Then there’s the intentional use to circumvent rules or regulations.

And there it’s, you know, I’ve often talked about where it has to be the department of Know K N O W and not the department of no, don’t do that. No, the solution in all cases is to understand the landscape, who’s using it and why, and address all of these things through effective communication. And Isaac, to go back to your second part of the question. For me, this, this derives from board level, right? One has to make the board aware of the threats that we’ve heard very well articulated by, by Derek, as always, and, and Wayne mentioned quite a number of them. You’ve cited some statistics in your opening commentary. The board has to understand the risks and the potential value, the potential advancements and frame that as something that is to be encouraged, but encouraged in a framework that preserves the safety and security of data and minimizes the risk.

[00:18:53] Speaker A: What are you saying to the board, Joe?

I say the board doesn’t care about overwhelming the organization with policies, the education that goes with it, the tools that are required to monitor things. What are you saying to the board? Wayne’s raising his hand. He’s going to come in on this one, too. But let’s hear your answer. On this?

[00:19:14] Speaker D: Well, to some degree, if the board is.

I’ll tell you a war story.

I was called to a board meeting at a company that should remain nameless. And I was asked about our cyber security posture. And, and I said that I looked at cyber security at the time in, in a very simple framework, there’s detect, there’s defend, and there’s remediate. And I said as. As far as detecting, we’re pretty good at that. As far as defending, we’re doing all the right stuff. But in terms of remediation, I said, we have no recovery posture. We don’t have a bcp, we don’t, we don’t have a recovery plan. Right. And the reaction of the board was, okay, thank you very much. We’ll make note of that.

What can you say if they’re not willing to recognize that there is a threat and act upon it?

You know, they’re sort of putting their heads in the news.

[00:20:10] Speaker A: That’s my test, Joe. I come up with the headline.

I may even doctor up a printout of what one might look like in a newspaper, depending on the age of my board members, and show them the headline test. That’s going to freak them out. Let’s bring Wayne for 30 seconds. Wayne, what are you telling the board?

[00:20:33] Speaker B: Well, the board is coming to me. Every board reads the articles about AI and says, what’s the risk? What’s the opportunity? So the good news is you don’t have to make them aware of the topic. And the number one thing they want to know is how do I keep myself from being sued? A derivative lawsuit, everybody’s worried about that. And so you’ve got to give them enough comfort that they are understanding the risk and managing the risk. Because with the board comes the risk posture in the company. They may decide that AI is not the biggest risk. It’s hurricanes, it’s war in a country they operate in, it’s employee theft, whatever it may be. As long as they’ve made thoughtful, informed decisions, they’re probably lawsuit proof. And I say that not as an attorney, and I don’t play one on tv.

[00:21:17] Speaker A: Hey, Wayne. Thank you, Joanne. Your definition of the real risks around shadow AI. And then let’s let you jump into this second question. Right. You know, shadow AI often has solutions around communicating and writing policies, but we need a little bit more than that. We call that guardrails.

It’s becoming an ambiguous term.

What should every company be doing other than just writing and communicating policies that can help implement guardrails to protect our organizations? When we want sanctioned AI and not shadow AI. Go ahead, Joanne.

[00:22:00] Speaker E: Well, first of all, let me give you my, my premise and my thesis for this discussion.

Excuse me, which is shadow. It was about tools, for sure.

Shadow AI is about outsourcing judgment.

Because every time you use it, you are inadvertently or by osmosis, bringing the judgment that you’ve outsourced to AI to help you make a decision back into the organization. And it is becoming inculcated in tribal knowledge.

So that being the premise, the answer to your question is, and I support the research finding that Gartner, my ex employer, chose to put. There is a lot of work that needs to be done around understanding where those guardrails need to be with AI. So let me give you a very short feet on the street example.

So let’s assume that somebody decides that they want to write an agent or have an AI intervene when a maintenance order needs to come up.

In manufacturing, of course, or any business that repairs anything or makes anything.

That being the case, and that that would include content, by the way, where exactly does the human in the loop have to go? So here’s the quick story example.

So particular company wanted to do a maintenance agent. And I said to them, you know, that’s all well and good, but if you want to do governed execution and eventually have autonomy in your agents, think about the business processes that this involves and think about the categories and stages. For example, there is something on the shop floor. It’s an anomaly. You run a root cause analysis, you come to the point that, you know, a bearing needs to be replaced in a machine, okay, if autonomy kicks in right then and there, and it’s not a human decision, does that agent have the right, the agency, the authority, the accountability?

It’s an alliteration of six A’s and I’ll put them in, you know, LinkedIn.

But basically what you’re giving, what you’re saying is by proxy, that agent has the right to then or should have the right to go check your ERP and see what your inventory of spare parts is. It should then figure out, based on the governance policy that you’re giving it, do I have the right to deplete that inventory? If the stock is not there, but it happens to be at another location, does it have the right to check that location’s inventory and so forth and so on? And literally, if you go down the entire checklist of all the actions that are required, you really start to begin to see how important it is that your governance cannot be universal. It has to Be based on specific instances. And what you really need to do is a lot of business process examination and potentially reformation. Because if you don’t, that’s where your security comes in, that’s where your operational environments come in. That’s where human safety comes in. Now I’m using maintenance as one example. The same could be true of a stock trade.

You need to understand where the provenance and the lineage are going. Who made the decision is one thing you may be able to find, like who, who hit. Okay.

But if you don’t know what went into that decision making, in other words, you have outsourced your judgment, that is exactly where the biggest risk comes from because it gets inculcated either in an operational sense or in a decision making sense. And that goes to your brand value, your trust. What I’m hearing from boards is not only about the lawsuit issue that Wayne surfaced, but also how do we make sure that things that come back from AI don’t automagically become part of our culture or part of our operating procedure. That’s board level decisions that need to filter down not just to technology leaders, but to business leaders.

[00:26:39] Speaker A: Joanne, I like the direction you’re going because we know we have tools in IT to protect against usage of applications models that we don’t want people to use. So if I’m a quad shop and I don’t want people using OpenAI, I have tools to protect against that. I have proc. Proxies that can put, I can put in place to start looking at is the information being leaked out of the organization. So there’s data leakage tools that I can use.

You know, things get much more interesting when we start getting into business processes.

And now, you know, this notion of shadow AI becomes a little bit more fungible. Right? It’s not binary, you know, it’s, you know, somebody is in that innocent category that Joe brought up and trying to use AI to do something. And now the question is, how do we make sure that what they’re trying to do makes sense? And that goes back to the decision making. Go ahead, Joanne.

[00:27:46] Speaker E: Right, sorry. Sorry for interrupting you, Isaac.

The other part of that is there’s a third category that wasn’t mentioned, which is people who are, they may fall into the innocent category, but there’s innocence and then there’s, I need an answer.

There’s, there’s a, you know, like a, a person who walks with purpose. I need to get a solution. I need it now. I don’t have time to wait. This is going to impact how I’m measured or how something else is measured. And that’s where the biggest risk starts to come in. Because they’re people who are sincerely trying to solve a problem. It could be in healthcare. Right? 90%. And I, I just became involved in an organization that’s looking at this.

One of the things that we don’t talk about very often is people who end up in a very bad way in hospitals because they get the wrong medication or have an allergic reaction.

These are people who think about it in the, in the sense of AI. I go to AI, I’m looking up, well, what medication can I give this person in an emergency where the ones that are normal that I would normally use are unavailable or the person will have a reaction and they give a medication that that person then reacts to as well.

This is one of the areas where governance and I believe execution, particularly in the agentic era is.

That’s AI’s last mile problem.

How do you govern it? How do you govern the execution of it? How do you prevent damage from being done, particularly in the case of human life?

[00:29:35] Speaker A: Yeah, they’re purposeful. I don’t think we’re at a last mile problem. I think we’re still discovering AI and that means there’ll be new problems that are coming.

Before I bring Elena into the conversation, there’s two comments here that are worth me echoing. There’s one from Keith who is worried about swarm AI.

And I would apply that in the IT area itself, this notion of swarming for finding a solution and sort of the cultural byproduct of that term is I’m going to use whatever means available to me to find solutions. And so he brings up that as a problem. And Dana Sanderson, our good friend, brings up the other extreme where your culture is highly distributed and open to, in his case, faculty becoming their own IT departments in their labs and being used to using any tool they want to be able to do their research.

And it’s not just in higher ed that that problem exists. It’s in a lot of organizations that just want to provide their businesses in departments with a lot of autonomy.

You know, construction has that either. I’ll let Joe and Wayne comment on that. Joe, Joanne, give me a second. I want to bring Elena in and then John and we’ll go back to Wayne after that. Hello, Elena.

[00:31:04] Speaker E: Hello.

[00:31:05] Speaker F: So I would like to out inadvertent usage and second to some degree what Joanne was saying that a lot of users are using AI tools to genuinely answer their questions. And coming from the commercial end of the business, I would say, well, that would be us, right? Because the board and the management is putting pressure on getting things done. And very often in the organizations they say, well, you go figure it out. And that’s where we would run into problems. And also our organizations typically are often siloed. And how often do commercial leaders talk to IT leaders outside the annual holiday party? Question mark? And that is something that we need to change. Right. The role of technical management of AI, whoever is wearing that title needs to stay very close organizationally and change their role and their view of people who execute it.

[00:32:07] Speaker D: Right.

[00:32:08] Speaker F: And that would be a lot of commercial functions. And therefore the organizations shouldn’t be afraid of over educating. They should culturally embrace IT and implement change management processes. Right. That would allow commercial people to understand, while I’m trying to solve my problem, find my answer, analyze my data, I’m exposing something. And I might not know what I’m exposing, but my IT counterpart does. And therefore, if we have a close, open, close, close, open relationship, tight and open and honest relationship, we would be in a position to educate both sides.

[00:32:49] Speaker A: Wow. We’re going to have to have a whole nother conversation around educating the workforce around using AI and probably revisit that probably every six months as the capabilities change.

I want to hear from John and then I’m going to take my break and come back to Wayne. Go ahead, John.

[00:33:07] Speaker G: Isaac, thank you for having me on. And just reflecting back, like the iPhones were really like a great example of shadow IT that became the mainstream standard. And it wasn’t really that long ago that the companies were encouraging people to use as many tokens as possible.

They were saying, go fast, use it, use AI, try to figure out how do you do your job more effectively, how to do your job more efficiently, burn as many tokens as possible.

And that was before people really started looking at the bills. But the way you really burn a lot of tokens is either you have an agent or you start using AI on your code to start doing things on code. And so it’s like we were really were encouraging people to use a ton of AI just really recently. And so people were moving faster than the corporate tools, people were moving faster than the governance, the regulations. And so now I think we just have to go back and retrofit all that stuff in. And so I think we’ve done a great job on the risks. Some of the other risks I’ll just call out are I was talking to my friend and he was trying to get, get really good, basically a search on his notes. And so he actually spent $6,000 of credit, $6,000 of credits through the corporate tool just so he could try to figure out how to search his notes better. And that was like one month. And so it’s just like that’s going everywhere. We’re hearing about stories about people accidentally triggering a huge code, refactoring and having bills being in the millions of dollars and so that it spans a real thing that could happen.

But the other thing that can happen is when people start really looking at the IT spend is that they say, well, is there a cheaper way to do this? And if they start moving off of the corporate AI sources and start looking at other AR sources, that’s really a bad thing. Or if people can’t do what they want with the company tools and they start looking at what can I do with the tools? Outside of our company now we have all sorts of data leaking. And so the faster that we can get people on to bless tools and keep them in the corporate tool system, the better.

[00:35:16] Speaker A: You know, a lot of terms here from that mean different things. You know, whether it’s shadow AI or rogue AI or the impact of swarming AI or the AI cost exposure, right? So now we’re not leaking data. We’re not using AI with tools that we’re not supposed to be using.

Maybe we’re not at the point where we’re getting into the agentic issues of is it, you know, making decisions that we need to put some guardrails on it. We’ve got people just making really bad decisions about how they’re using it and the underlying expense around it.

And, you know, maybe Wayne and Derek and I and Joe will come up with some language so that when we talk about it with the board, we can classify the risk differently. Folks, welcome to this coffee with digital trailblazers. Entering our fourth year are. What is this 185th episode. We meet here every week, Fridays at 11am Eastern Time. If you miss an episode and want to listen to it, please visit drive.starcio.com Coffee the episodes are there for you to listen to.

The research papers that I do, the one screeners that I do, they’re available there.

The whiteboards, I cleaned them up. We pushed those over there. There are links to sign up on Apple and Spotify. If you want to listen to this in your car, there’s links to the driving digital newsletter. My newsletter is coming out in just about a week.

You can get there. And the most important thing is something asked by many of you is the add this to your calendar button so that you don’t miss an upcoming episode. Our episode next week is going to be around SaaS, cloud and AI contracts, where technology leaders lose leverage. I don’t have a special guest there for that one yet, so if you want to speak up, if you have an angle around procurement or legal or just have had to deal with a lot of contracts and you want to be our special guest, reach out to me and let me know. We are going to take off for Labor Day weekend, September 4th.

We won’t be here, but we’ll be back on the 11th. I don’t have the topic title just yet, but it will surface around AI governance, particularly around the context layer. So we have a nice group of people here today. Mark your calendars for the 11th to rejoin us. It’ll be something like AI governing the context layer. I am sure Joanne’s gonna have a lot to say about that.

That will be on the 11th.

Mark your calendars, folks. Let’s bring Wayne back. A lot of different comments here, so I’m just going to let you go wherever you want to go and then we’ll hear from Joanne and Derek after that.

[00:38:12] Speaker B: So I’ll make comments on what John said. The token maxing idea, I think that’s a temporary aberration. I mean, everybody saw it as a mark of being smart and being up there and then they realized how much it costs. And between open weight models, between tools that can find the appropriate model at the appropriate costs, and the whole AIOps movement, I think we’re seeing an aberration and engineering problem that will fix it in a pretty short order.

That’s one comment. The other one is to Joanne’s comment about healthcare and final mile. Final mile is when the robot is giving them the pill or the robot is giving them a shot, or the robot is doing the operation. When we take the human out of the loop, that’s when it gets really interesting, which also means really scary.

And an environment like that, you don’t allow shadow AI and you lock it down like crazy. I will point out I was in financial services for many years. There was a study done that said if people had to choose between their healthcare and their money, which would they rather protect? And overwhelmingly people said they want to protect their money even more than their health, which I think is kind of backwards. But I use the example of financial services and compare it to healthcare.

You’ve got to lock down shadow AI because you can do too many things that get you in trouble. As I said, To Isaac before the meeting started. I work with a company that wasn’t regulated and I had to tell the board, if we were a bank, we’d all be in jail. Our industry doesn’t regulate that way. But if you’re in an industry that regulates very tightly or that involves human safety, OSHA kind of stuff, manufacturing with large heavy machines, explosives, whatever, you better be darn careful that there’s no shadow anything going on and that every model is vetted within an inch of its life.

[00:39:59] Speaker A: Go ahead, Joanne. I’ve been holding you off, so I’m not going to even prompt you.

[00:40:03] Speaker E: Well, I’m not sure. First of all, I want to address the swarm comment.

Swarms are basically, in our vernacular, I would say, agents that have very specific tasks that are deployed in parallel or in a distributed model so that they accomplish their goal, but they’re constrained not only by governance, but also by the execution by the model that they’re using. We are in a part of the business that is small language or specialized language models.

Would I trust a large language model to do half of what we’re doing with our agents? Not a chance.

Because very recently, and it’s still emerging, you haven’t heard a lot about this yet. There, there is a lot of research going on, not only by the frontier model makers, but by a lot of folks in academia and in business where there are instances that the agents or the AI invents secret spaces where it does its planning and you don’t know what it’s thinking, where it’s coming from, what its perspective is, what its intent is. I’m not talking about malicious or, you know, something back to the future ish. I’m talking about how models drift and why you need to have guardrails around model drifting.

When you choose your tool set, don’t rely on one LLM, always use one against the other. You will be amazed at the difference in the results. You will also be amazed when you start looking at a context layer and giving it a very strong semantic spine.

So take that as a comment on a comment and also a move towards your what would I tell trailblazers.

[00:41:57] Speaker A: Yeah, I’m starting to make a list of scenarios here. And Heather, if you’re listening, I do want you to chime in on at least one of them. I know you’re on. So the person with little patient would be my bet, which came up earlier. Derek, I’m going to give you a scenario. Okay, so Wayne has brought up large regulated, and large regulated enterprises are the easier ones. They have A history of putting technology and process and governance in place because they have to. And this is just, you know, expanding the surface over what things they have to cover.

I want to talk about that mid sized organization that’s unregulated, that has to make choices over what they put investments in their technology. And you mentioned threat intelligence. I want to give you get your 1, 2 or 3. Okay, what am I investing in as the CEO in force ranked order that allows me to get some of the benefits over AI while minimizing the risks.

[00:43:12] Speaker C: Yeah, and that’s a great question because I look at it from a human nature point of view. You got to realize that people are going to do things that they want to do based on either if it is given a permission or not. Human nature says I’d rather ask for forgiveness than ask for permission. So by doing that, the question is, do I have visibility to see what my employees and my teams may be doing? That’s what I would invest in first. So the AI threat intelligence monitoring tool will give me visibility into those AI tools that are being used, those AI threats and challenges that may exist, but also how it may impact my organization. If you can’t see it coming, you can’t prepare for it. And I think a lot of times companies miss out on the fact that the people they know people are using it, but they have no idea where or to what extent what information is being leaked outside or inside their particular area. That’s a huge concern.

I think when you look at from a mid sized business point of view. You know, we talk about the governance and the policies, procedures, those don’t have any play unless they have an impact in action to it. If we know that people are going to click on stuff and do things they shouldn’t be doing, we need to move from a risk first mindset to more of a resilience first mindset, knowing that it’s going to happen them people are going to use AI. So I need to figure out as a business entity, as a CEO, what do I need to do to protect my business and my business culture. And by doing that is I need to put the policies in place, I need to put the guardrails in place. But I also need to figure out am I hindering or am I affecting and making a better place for my business. In most cases it’s going to be a challenge and some people are not going to like it, some people are. But if everybody’s on board, and that’s the key thing, everybody has to be on board when you’re looking at this.

If everybody’s working and pulling in the same direction with the same mindset, we’re going to have a successful outcome based on the strategies put in place. But the strategy has to be put in place and you have to have buying from. It can’t be done in a vacuum. If I’m not getting the information and feedback from my developers, my marketing team, my hr, my legal team, then it’s not done with everybody’s mindset involved to help create the best factor and resilience strategy moving forward. So I would look at it from that perspective.

[00:45:13] Speaker A: All right, so we’re starting with basics. Your strategy, your governance, your policies, all the communications that go around that. So we know what box we should be living in from a. From an implementation standpoint. Where are you starting from, Derek?

[00:45:30] Speaker C: Yeah, implementation. Put in the threat intelligence monitoring because then you’ll see exactly what’s taking place. You know, I know companies when it comes to it, they do these pen tests of vulnerability scans. That’s the one shot with a threat intelligence AI threat intelligence monitoring tool, you’re seeing in real time what is happening in your network and see what’s building. That would be my first investment. And by doing that, even though you have some systems in place, tightening your configurations systems and configurations of all your current applications and services, I would make sure they’re at the highest grade. I would also make there be upgraded to the highest level, including AI services.

And that would give me better understand of what I need to do.

Things can’t be done in serial. Things have to be done in parallel efforts. And by doing that in parallel, that’s when you’re going to catch it. It’s the people that take the time or delay and putting these things in place are the ones that are going to get caught. Because AI and the speed of AI, if you’re not moving or thinking ahead of it, you’re going to get caught. As I tell my customers, you need to be playing chess, not check. There’s no way for AI to make the move before you actually put something in place, folks.

[00:46:31] Speaker A: Our CISO has prioritized only one tool and then back to basics in terms of communications and setting policies. I love it. Let’s move into our third question. I think this is the one where we can provide the most value to our listeners. Heather, I’m glad you’re raising your hand here because at the end of the day, Derek is right. I’m going to put a strategy and governance program in place. I’m going to Set my policies and then people are going to go back to their desk and do what’s easy for them or what they want to do or what’s convenient for them or what they think is the right thing to do. And you’re welcome to pick one of these scenarios that I put up on the dashboard. The person with little patience, the DevOps team that wants to swarm without restrictions, the person who wants to do then ask permission. Someone just brought up, I think there’s also the person just like, you know what? I’ve got chatgpt on my phone, I’m just going to use it whenever I want.

Heather, how do you approach pick a scenario and how do you approach them?

[00:47:34] Speaker H: Well, I think sometimes it really has to do with training and there’s a bunch of sound bites that I have heard and that I want to comment on. But the collaboration with it and communicating what could possibly happen.

Some people that just have it on their phone, well, what’s the problem? What’s the big deal? Well, if you show them what some scenarios, not only ones that have actually happened through the news and you can talk about what the implications of those situations and how it impacted the business, people can stop and say, oh wow, that really is a big deal. It’s not a minor thing. So if you have different training sessions and reminders, then I think it really is helpful when you collaborate with it, that involvement will always give you that sounding board because you also have to collaboration with legal and that’s why having an environment where everyone is participating and making sure that they are involved in not only what is being suggested but what the implications are and being able to communicate that.

I had a situation where I got called by a law firm where someone was in a loss of one of my candidates and they asked me for everything that I ever did with this, with this candidate and I’m like, like what?

And they said, well, did you ever record your conversations on the phone? I said no, that I never do. And if I’m doing an interview or vetting a candidate and I’m recording they a they can see that I’m doing it because it’s on the button that your session is being recorded. But I also ask them and that’s something that I think Wayne mentioned very early on. You know, there’s a kindness and there’s a courtesy that you offer to people. This is what I’m doing because it helps me evaluate you better and I can be very present with, with our time together.

But so I just want you to know that I’m recording this or I’m taking a transcript and I just want to make sure it’s okay with you.

But down the road, I have to worry now about what the legal implications are. And there was something else someone said about thinking.

One of my friends who’s a nurse practitioner at Sloan Kettering Hospital in New York City was given a tool that not only does it scribe and take the notes when you’re with your patient, but you also get prompted by questions that could be asked. Now, this is a policy. It’s a known service that they offer to their medical professionals. But what her concern was, and this is something else to be considered from a business standpoint and a training standpoint, does that reduce the amount of thinking that the medical professional has to do now? Because someone is just going to push it out at me. So there’s a lot of other issues that have to be considered then that go into the policy and setting that policy because it’s not, as we’ve just learned, not straightforward at all.

[00:50:39] Speaker A: Thank you, Heather. I love this bring legal in to play the bad cop so you can play the good cop. I think it’s a really smart move. I want to go to Wayne. Wayne, I’d love to hear your turnaround scenario. And if you don’t pick the one I want, I’m going to give you another one.

[00:50:54] Speaker B: Okay. So I was going to comment on something Derek said when he said threat intelligence to me that normally looks at outside in with AI, a lot of it is inside out. So when you say threat intelligence, I think you have to be thoughtful about what threat you’re talking about.

The tool that I haven’t heard beyond policy, governance, that sort of thing, which is cheap for even a small company, is an MCP model, content context protocol. You’ve got to be able to gateway, guardrail, whatever, restrict these AIs so that you’re talking about the right things to the right AI at the right time. And so if I was going to tell a company what to invest in beyond the normal hygiene and CMDB and all the good stuff for cyber, it would be one new piece of gear would be an mcp.

[00:51:45] Speaker A: I agree with Dan. For those of you don’t know what this is, it is MCP and AI air tools or really protocols for agents to speak to each other and exchange information.

And a gateway is just like it sounds. Every communication goes through the gateway and that’s your opportunity for implementing governance. Wayne, I want to throw you a scenario that you sort of brought up earlier. I’m going to call it the salesforce scenario, for lack of a better term. It’s the GM that wants to pick their own tools. It’s, you know, the professor that wants to run the lab on their own.

What’s the, where are you starting in trying to unwind that behavior, particularly when it’s an AI risk.

[00:52:28] Speaker B: Yeah. So in industry I have more control. If it’s university or medical, they tend to operate with their own grants in their own lab. And you’re basically chasing after them, trying to keep them protected. That’s just the reality when I’ve worked in those areas. But in a corporate, corporate setting, you start with the Enterprise Risk Register. Here’s the Enterprise Risk Register. These are the biggest risks.

If you go on your own, this is what might happen. So there’s, the first step is telling them what might happen. It’s a lot of it is ignorance. They are innocent but ignorant. If you go on your own, this could happen. Your team could do X and that would cause Y and we’d all be in the newspaper.

Beyond that, you’re working with the general counsel, you’re working with the board and the Enterprise Risk Register. And if the answer is we have decided to control the AI tools because the G, the GC or the CFO and the CEO agree, or the risk committee of the board, then you use the hammer, as you said, let’s let the chief legal officer be the hammer. And so it’s a combination. You try to work with the people that’ll work with you. And then you also offer, well, if that tool is fundamentally better than the tools we’re using, let’s spend a little money and investigate. Maybe it is better in your use case. And we can build some guardrails, we can put an MCP around it, we can put the, the framework around it that you were saying, Isaac. And so we can say, okay, if you’re going to spend a dollar, spend a dollar twenty. And you can do it in a protected and governed and tracked way. And most people are pretty reasonable about that. They don’t want to cause the company extra risk, extra problem. They just don’t get the attention from it that they want. And so they’re going it alone. Same thing with Shadow It. It’s cheaper to do it yourself. Why? Because you eliminate cybersecurity, you eliminate governance, you eliminate key management, you eliminate all the messy stuff that we all do behind the scenes that keep everybody else safe.

[00:54:18] Speaker A: I’m going to unpack that for everybody because there’s three data points that are really important. Number one is listening. Right? Why are they doing this? What is their thought process? What are they optimizing for? Number two, goes back to what Derek was saying earlier.

You know, what are your policies that hopefully has some legal and risk management grounding behind it. Right. Your policy may say you can use tools, but you have to use, have these precautions in place. You can use mcp, but it must go through the corporate gateway. And number three, the way you really get a GM listing is cost.

They own the P and L. So yes, I can do this for you, but there’s that 20% overhead to create the structure for your particular tool. I got five minutes left, Joanne and John, let’s hear about it.

[00:55:09] Speaker E: Okay, so if I’m going first, John.

[00:55:12] Speaker A: Yes, Joanne.

Sorry, I said John.

[00:55:17] Speaker E: That’s okay. So I guess I would go with the CEO choosing speed over safety as the scenario to turnaround. And this goes back to part of what I was saying earlier.

Having done this now for a little bit close to two years, especially in the agentic side, one of the things that finally pops in your brain is you cannot look at use case scenarios the way we used to. You have to look more holistically, look at the initiative, look at what you’re trying to accomplish as a business, whether it’s top line value or bottom line value, or in Derek’s terms, resilience, innovation, all of those lovely words. You really need to look holistically because the turnaround to somebody who wants speed over thoughtful progression, as I would put it, and does not want to outsource judgment, understands that systems may be siloed or departments may be siloed, but the impact of one action in one department has a cascade effect across the entire organization and even its upstream and downstream ecosystem of trading partners.

Here’s a great example from what I mentioned before. That maintenance record and allowing an agent to introspect whether or not you have inventory for a spare part or you have to order it from another facility. What happens if you have to get it directly from a supplier that’s outside your organization’s boundaries? And what happens if there’s no price ceiling put around that governance or that execution that says, oh well, it’s a five thousand dollar part and no, I’d rather put this out for bid.

So agents talk to agents. We can’t look at a single use case and plan for that eventuality in traditional IT tooling. Oh, we’re going to, you know, choose six vendors, shortlist three.

A year and a half later, we’ll finally Make a decision and Bob’s your uncle. Doesn’t work that way. You have to be very focused on the organization as a whole, on the initiative and all the piece parts that go with it. Does it go back to business process? To a large extent, but it also goes to common sense.

You know that finance is going to get involved in every other department. You know that legal is. You also know that security is take the top three, start looking holistically and plan what you want these tools to do for you in terms of business value.

[00:58:02] Speaker A: Really good stuff. Joanne. I love this quote from Dana. Using AI to detect shadow AIs like installing a smoke detector. It only alerts you after the fire has started. As all A bunch of other questions here that I will pull together from the Common stream to think through our next sessions in September.

John giving you the last word today.

[00:58:25] Speaker G: Yeah, Isaac, thanks for having me on. And so when I look at this, we have to go back to the basics. We have to educate people, continue to reinforce the message.

These agents, they can do everything that the, that the user has access to do. So, so look at the basics, like what can users do, what, what systems do they have access to what traffic’s going on your network, what traffic’s going outside your network.

And just make sure that like if the users are doing the right things and having access to things, that’s great. But if, if they shouldn’t be having access to things, take that access away. We should be going down to a model of the Lewis Permissions, the lowest access needed for everything in the company.

And then we got to get people so they have governed AI that they can use and we have to be encouraging people to use AI for all the good things. But figure out how we can have guardrails and systems in place so that they can use AI safely. And we’re never going to be able to, we’re never going to have regulations stop people from using AI because there’s so many ways to get around it. So the faster we have ways to do it safely and govern, the better.

[00:59:31] Speaker A: Thanks to everybody who spoke today and commented on our comment stream. Thank you Wayne, Joanne, Derek, Elena, Heather, John and Joe for all your comments.

If you missed part of this session or a previous session, do visit drive.storecio.com Coffee, which has previous episodes. It has the whiteboards, it has the dashboards, it has the newsletter, it has the calendar link to sign up for. For one, if you are struggling to get your strategy and governance in place, I have Star cio. My company has an AI Strategy and Governance Workshop is one of the things that we can use to educate people on AI governance and policies. We will have that as a future session here, maybe September, October, the 28th. We’ll be talking about losing leverage on your AI contracts, SaaS contracts and cloud contracts do register for that particular event.

There won’t be a session on the 4th. And like I said, I don’t have the title for it yet. But the 11th, our topic will be about AI governance of the context layer of semantic layers. And I have a very special guest coming on for that particular episode. Folks, enjoy the weekend. Happy August. Happy Summer. We’ll be back here next week for more discussions here at the coffee with Digital Trailblazers. And thank you again, Wayne, for being our special guest today. Have a good weekend, everyone.