← All writing
Delta / Iceberg

Your Delta table is Iceberg-compatible. Your readers just don't know it yet

Your Delta table is Iceberg-compatible. Your downstream readers just don’t know that yet.

We had a Databricks pipeline writing Delta tables with Iceberg compatibility enabled. The job finished. The data landed. Everything looked fine.

But Iceberg readers downstream were getting stale data, sometimes hours behind. Sometimes it only updated on the next job run.

The cause: the sync between _delta_log and the Iceberg metadata path isn’t immediate. After a Delta write completes, the metadata update has a lag.

The drawback is serious. If you have Trino, Athena, or Spark with an Iceberg catalog reading that table, they can silently consume an outdated snapshot. No error. No warning. Just wrong numbers flowing downstream.

What actually helped wasn’t the docs, it was reading the Delta source code directly. The docs say “Iceberg compatible.” The code shows you exactly when and how that compatibility is applied. Two very different things.

If you are running Delta and Iceberg together in production and haven’t validated the metadata sync timing end to end, do it before your consumers run into it in a pipeline that’s already live.

Have you hit metadata consistency issues between Delta and Iceberg, or found a workaround for the sync lag?

This is the kind of problem I get hired to fix. If it sounds like your pipeline, let's talk.

Message me on LinkedIn