Read, don't copy: the data layer stops moving tables around

OpenSharing: read the table where it lives

Databricks' two-part post this week makes a case against the way most teams still share data: SFTP file dumps, API pulls, or managed connectors. The post argues those patterns carry hidden costs that grow with customer count, from maintenance and data drift to backfills that re-poll APIs in chunks and trip rate limits. The replacement it proposes is OpenSharing, an open protocol for reading a table in place, built on open formats such as Delta Lake and Iceberg. m:…1737

The mechanics are what make it more than a slogan. Providers share a table instead of building and babysitting an export, and they can see an audit trail of when their shared tables are queried. Reading through OpenSharing does not slow down the customer's system because it uses the consumer's compute, and it eliminates storage duplication between provider and consumer. The second part of the post shows the implementation: Unity Catalog governs the share, Structured Streaming processes changes incrementally with checkpoints tracking the last read version, and the provider manages a single governed source table instead of separate export jobs. Databricks Runtime 19 and above supports automatic feed computation for tables with row tracking enabled. m:…1737 m:…1736

Cross-platform reads, without the copy

A Snowflake build report shows the same idea working across vendors. Databricks Lakeflow pipeline tables can be queried directly from Snowflake using Apache Iceberg, through Unity Catalog's Iceberg REST endpoint and catalog-linked databases. The author accessed 13 Lakeflow tables this way, with a 60-second sync interval on the gold layer. The catch is in the details: Lakeflow tables need the pipelines.externalMetadata.enabled property set to true for external access, full refreshes of a pipeline can change a table's Iceberg identity, and the error codes are specific, such as Snowflake error 004174 when external data access is not enabled on the metastore. m:…5337

Onehouse's OneFlow applies the same logic to the oldest data problem of all: getting changes out of an Oracle database. OneFlow automates Oracle change data capture into Apache Iceberg tables, provisioning an Amazon MSK cluster in the user's AWS account when the first Flow is created, and writing one copy of the table that Iceberg engines can read. Hand-built Oracle CDC pipelines, Onehouse notes, require Kafka, Debezium, a schema registry, Spark, and table upkeep jobs. OneFlow resumes synchronization automatically after an Oracle outage without manual intervention, recovering in 4 minutes in the company's test, and supports a Mutable write mode that applies updates and deletes directly to existing rows. m:…2784

The governance stays where the data is

The governance story this week is that the app now lives where the data lives. A Snowflake App Runtime build report describes rebuilding a business glossary and definition-approval app in five days, running entirely inside the Snowflake account, without external Docker containers, registries, or a separate authentication layer. As the author put it: "The data stays in the account, and users sign in with the SSO they already have." The build's hard part was permissions: the client account required 46 CALLER grants on the catalog role, and the author recommends deploying as a dedicated role with limited grants rather than as SYSADMIN or ACCOUNTADMIN. m:…8781

The pattern across all four pieces is the same: stop moving data to the tool, and move the tool to the data. OpenSharing reads the table in place. Snowflake reads Lakeflow tables through Iceberg. OneFlow writes one Iceberg copy instead of a pipeline of copies. The catalog app runs inside the account. The copy was always the expensive part, in storage, in drift, and in the export jobs nobody wanted to maintain. m:…1737 m:…5337 m:…2784 m:…8781