Databricks opens the lakehouse to AI assets and puts a meter on Genie

Sharing moves beyond tables

Databricks and the Linux Foundation have evolved Delta Sharing into OpenSharing, an open, vendor-neutral protocol for exchanging AI assets — not just tables, but agent skills, AI models, and unstructured data. The specification is Apache 2.0 licensed and governed by the Linux Foundation AI & Data Foundation, though OpenSharing is currently at the sandbox stage within the foundation. Existing Delta Sharing clients and integrations are compatible with OpenSharing, and the protocol adds support for Apache Iceberg REST Catalog compatible clients. m:…8492

The scale explains why the change matters. Delta Sharing reached more than 28,000 data recipients by June 2026, and roughly one-third of Delta Sharing traffic flows across platforms via open connectors. SecureConnect, a Databricks-managed proxy for OpenSharing, is generally available. The direction is clear: the lakehouse is becoming a network for sharing the things AI systems need — data, models, and the skills that tell agents how to use them — rather than a warehouse a single company queries. m:…8492

The rough edges are visible too. Global Distribution for OpenSharing is in private preview, Genie Sharing is in beta, and Lakebase table and change data feed sharing and foreign Iceberg table sharing are in public preview. Email-based sharing is planned for a future release. A protocol at sandbox stage with several features in preview is a direction, not a finished standard — and teams that build on it should expect the edges to move. m:…8492

Getting live data in without Kafka

The second change is about getting data in. Databricks and NVIDIA collaborated on a Debezium sink for Zerobus Ingest, included in Debezium Server 3.7, that connects change-data-capture feeds directly to the lakehouse without a Kafka cluster. Zerobus Ingest is a serverless, push-based service that writes records directly into Unity Catalog Delta tables. Debezium reads transaction logs like PostgreSQL WAL or MySQL binlog, and NVIDIA contributed an envelope mode to the sink, which requires durable acknowledgment before source offsets advance and uses a fixed nine-column table structure. m:…1894

The reason to remove Kafka is operational, not ideological. Restricted network policies often block inbound connections to databases, while outbound connectivity is often easier to permit in enterprise environments — and a push interface fits that reality. Kafka requires capacity planning and dedicated management resources, and polling databases for change data scales poorly. Delivery through the Zerobus sink is at-least-once, which means downstream consumers must handle duplicates. The trade is explicit: less infrastructure to run, more care needed in the consumer. m:…1894

Databricks' own guide to scaling Apps built on its platform makes the same point about discipline. Scaling involves four stages — design, build, test and deploy, and operate — and the advice is to reserve application compute for UI serving and call coordination rather than heavy lifting, split session state into soft disposable state and hard durable state, and write telemetry to Unity Catalog tables via OpenTelemetry from the start. Load testing should use real user journeys at expected peak loads, and performance targets should be based on percentiles like p95 to capture slower experiences. m:…1103

The meter arrives for Genie

The third change is the one administrators will feel first. Genie is a consumption-based service rather than a seat-based one, and Databricks has introduced a pay-as-you-go model with budget controls. Each identified user on a paying account receives 150 DBUs of free Genie LLM usage per month, Genie One and Genie Agents usage by human users is free through January 31, 2027, and billed Genie usage receives a 25% promotional discount through the same date. Genie Code usage above the 150-DBU allowance is billed, service principals get no free allowance, and compute used to run work resulting from Genie is billed separately from the LLM portion. m:…8348

The controls are correspondingly fiddly. Genie budgets are managed via Unity Gateway using the resource tag for Genie, the system billing usage table is updated every few hours and is the source for final billing analysis, and setting a paid Genie budget to zero does not disable included free usage. The community guide recommends managing budgets by user group to give different populations different controls. The pattern is the same one this edition keeps finding: the AI assistant is easy to deploy and hard to govern, and the governance arrives as a billing table and a tag. m:…8348