Enterprise AI grows its management layer: measure, govern, vet

The measurement gap

Dataiku's latest installment in its CIO confessions series, based on a survey of 685 enterprise CIOs conducted with The Harris Poll, lands on an uncomfortable number: only 24% say their organization has mature, highly capable ROI measurement across all or most AI initiatives, and even for select use cases the figure is just 41%. Julia Berman's summary line: "Accountability for AI is rising faster than the ability to measure its value" Or, more memorably: "A healthy agent and a valuable agent look the same on paper." m:…4899

The gap has two halves. On outcomes: 72% of CIOs don't consistently measure both operational performance and business outcomes across every AI agent. On cost: only 21% have full, near-real-time visibility into agent and advanced-workload costs attributed by business unit, team, or use case. The consequences are already showing — token and compute costs have significantly changed how 39% of CIOs plan, approve, or execute AI strategy, and another 33% say unpredictable AI usage has disrupted their plans. Dataiku's framework for the fix is four questions every CIO should be able to answer: what is being consumed at what cost, is the agent running reliably, is it doing its job, and is it improving the business outcome it was built to affect. m:…4899

Skills that improve — with a human in the loop

Glean's answer to an adjacent problem — skills going stale — is "Skill reflection," announced for Glean Assistant. Skills package reusable capabilities that AI can discover and load on demand, but as the company notes, a skill can become out of date as expertise evolves, processes change, and underlying data shifts. The system watches for signals as skills are used — a repeated missing step, a correction that maps cleanly to an instruction, a pattern in how users steer the work — evaluates the evidence at the end of a chat session, and suggests improvements. Crucially, learning is separated from reflection: "A skill change only takes effect once the user explicitly approves it," says Miribel Wu, and learnings are previewed before they're applied. m:…8606

The feature is live in Glean Assistant, and reflection can be triggered two ways: proactively cued by the Assistant itself, or invoked directly by the user — with the suggested learnings previewed before anything is applied. The design intent is to stop one-off task variations or experimental formats from being permanently baked into what the company calls established best practices. In other words, the skill gets to learn, but the human still owns the curriculum. m:…8606

Vetted models and six-month migrations

Oracle's contribution is the model layer itself. In a guide to selecting and managing AI models in its Fusion Cloud Applications suite, the company says it vets a set of models for performance, safety, and suitability for enterprise tasks, shipping Oracle-built agents with defaults chosen for accurate, reliable results — while letting users compare and change models in AI Agent Studio, including at the agent or workflow-node level. The defaults, the company notes, may not be the lowest-consumption option, which is exactly the trade-off the Dataiku respondents are struggling to measure. m:…2883

The operational details matter. When an Oracle-provided model is scheduled for retirement, the company announces a removal date, makes a replacement available, and provides a window of up to six months for testing and migration; if an agent still references a removed model at runtime, the platform falls back to an available model in the same class. Starting with Release 26C, users can monitor AI Unit consumption in both AI Agent Studio and Cloud Console — a direct response to the cost-visibility gap. For bring-your-own models, the responsibility flips: the organization manages governance, credentials, and the provider's charges. m:…2883

Read together, the three pieces describe an enterprise AI management layer taking shape from three directions — and that reading is the author's synthesis, not any single vendor's claim. Dataiku names the measurement deficit that makes AI budgets hard to defend, Glean adds governed change control for the skills agents actually run, and Oracle supplies model vetting plus a migration clock. The uncomfortable implication is sequencing: the governance tooling is arriving faster than the measurement discipline that would tell a CIO which agents deserve to be governed at all. m:…4899 m:…8606 m:…2883