You’re likely to read about the major innovations from the largest tech companies through their conferences and media coverage, while startups can get attention for their forward-looking innovations and the venture capital money funding them.

However, many other breakthrough capabilities receive less notice. Marketers work hard to ensure that customers, end-users, and prospects learn about the new capabilities, but with so many tech companies and platforms promoting their feature releases, it’s challenging to build awareness.

So, I’m starting a new column on the blog about breakthrough innovations and, more specifically, shining a light on the CTOs and founders working tirelessly to release groundbreaking capabilities. I plan to cover innovations a few times a year, so look out for these “breakthrough” posts.

GenAI breakthroughs focus on quality

Several months ago, I wrote an article about how to test large language models (LLMs), and quite frankly, I was disappointed with the approaches that relied on significant manual efforts to build test questions and validate results. Organizations looking to develop LLMs and leverage retrieval augmented generation (RAGs) with their intellectual property need a more scalable approach, so I was thrilled to hear about innovations around responsible AI.

“John Snow Labs recently released automated Responsible AI testing in the Generative AI Lab, a first-of-its-kind no-code tool to test and evaluate the safety and efficacy of language models,” says David Talby, CTO of John Snow Labs. “The tool helps healthcare organizations build AI models that are safe, fair, robust, and transparent. This enables human expert oversight, something increasingly important to ensure the quality and safety of LLM solutions. Second, it allows non-data scientists, such as medical doctors, to use an LLM and train a smaller, task-specific model that can be more accurate and far cheaper to run.”

Talby’s recent article on bias and fairness in healthcare sheds light on the challenges of implementing a more equitable AI experience.

Another example of efforts to improve genAI quality is from Meaning, which launched its “soften” feature for its genAI-powered voice technology. “This capability enables users, primarily off-shore contact center agents, to enhance the clarity of conversations without losing their identity,” says Yishay Carmiel, CEO of Meaning. “Born out of the need from real agent feedback, Soften offers another gradient of speech augmentation that meets users where they are, empowering them to focus on excellent customer service rather than conversational barriers.”

Site reliability engineering reliability diagnostic capabilities

My recent writing on DevOps focused on advanced CI/CD practices, release management principles, and improving the developer experience. The innovations below focus on reliability and scalable operations, which, in my opinion, are really the heart and soul of DevOps practices.

The first capability should help site reliability engineers (SREs) manage SLOs and error budgets.

“In the process of driving updates to our platform this year, I have learned that when it comes to reliability, teams want to know not only whether their services are reliable and how reliable they are but also why,” says Alex Nauda, CTO of Nobl9. “We built Composite SLO 2.0 to effectively summarize error budget burn events, like downtime and incidents, across an entire tree of SLOs — potentially hundreds at once — to clearly show how each of the underlying services in that tree contributes to a loss in reliability.”

Nauda shares more details on how the feature enables holistic reliability management across an ecosystem of monitoring tools.

“Composite SLO 2.0 unites data from a growing ecosystem of sources like Dynatrace and Amazon CloudWatch for enhanced flexibility, providing engineering and management with a straightforward reliability metric for an entire product or service,” he says. “This allows our users to drill down into the behavior of complex systems with composite SLOs of near-infinite size and complexity while still receiving high-level business intelligence to inform executive decision-making.”

DevOps innovations focus on scalable operations

If improving reliability is a primary objective, finding ways to help DevOps team members improve operations is a key success factor. The next two features highlight opportunities for DevOps teams.

 “We introduced GitOps for IT and scheduled maintenance on your calendar for system updates,” says Luke Heath, CTO of Fleet. “Both features are the first of their kind in the space and represent a new way of managing IT at a scale that puts user experience first.”

Another example comes from Fabian Kramm, CTO and co-founder of Loft Labs. “I was proud to drive the release of vCluster for Rancher, which enables self-service virtual Kubernetes cluster creation and management for teams using Rancher and helps customers securely virtualize clusters for multi-tenancy from a single central location,” he says. “Rancher users can now provision virtual clusters just as they would traditional clusters, drilling down from within Rancher to inspect, manage, install applications, and more.”

Kramm shared more details on securing Kubernetes multi-tenancy and unifying management of virtual clusters.

What about improving the developer experience and open source innovations? Kramm shares his involvement in our open source projects and excitement about seeing how users leverage recent updates to DevPod, an open-source tool for creating and managing dev environments without a heavy server-side setup. “We made the UI more powerful, including easier workspace filtering and bulk workspace actions, as well as built-in experimental support for a new VS Code Insiders IDE so our users can take advantage of the latest VS Code updates.”

I love hearing about brilliant innovations that drive business outcomes, improve customer experiences, and provide real value to engineers. What innovations will pique my interest over the next few months? Stay tuned!