Lakehouse table formats in 2026: revisit Apache Iceberg, Delta Lake and Apache Hudi before you pick another format

Picture a payments and lending analytics platform in Hyderabad (an invented example, not a real client). Card authorisations and EMI schedules still land in PostgreSQL 16. Nightly dumps and CDC already fill an S3-compatible object store. The data platform team wants one set of Parquet tables that Spark batch jobs, Flink streaming upserts and Trino ad-hoc SQL can share without each engine inventing its own folder layout. One architect proposes Apache Iceberg for its REST catalog and multi-engine reader story. A second prefers Delta Lake because the team already knows Databricks-style MERGE and wants UniForm for Iceberg readers later. A third argues for Apache Hudi because the Flink CDC path needs record-level upserts and incremental pulls every few minutes.

All three are open table formats for lakehouse data: they add ACID commits, schema evolution, time travel and maintenance on top of files in object storage. They are not OLAP engines. If your pain is dashboard latency on local columnar storage, see my real-time OLAP post first. This post compares Iceberg, Delta Lake and Hudi as table formats: concurrency, catalogs, compaction, engine support, licences and day-two operations. Facts come from release notes, official docs, licence texts and CVE records, checked on 12 October 2026. The drill near the end is my own run on this Linux box.

The short version

• A table format is the contract between writers and readers over files in a lake. It is not a warehouse, not a message bus, and not a substitute for PostgreSQL OLTP.
• Apache Iceberg 1.12.0 (29 September 2026 on the project releases page; GitHub tag 30 September 2026 UTC) is the current tip. Spec versions 1, 2 and 3 are adopted; version 4 is under active development. Spark runtimes ship for Spark 3.5, 4.0 and 4.1; Flink for 1.20, 2.1, 2.2 and 2.3. Apache 2.0.
• Delta Lake 4.4.1 (5 October 2026 UTC, morning of 6 October IST) is the current 4.x tip on Spark 4.0/4.1/4.2. The 3.3.3 line (12 August 2026) remains for Spark 3.x users. Apache 2.0 under the Linux Foundation Delta Lake project. UniForm can expose Iceberg metadata beside Delta commits.
• Apache Hudi 1.2.1 (24 September 2026) is a bug-fix release on the 1.2 platform (table version 9, same as 1.2.0). Copy-on-Write and Merge-on-Read, record-level index, Flink and Spark writers, and new VECTOR/VARIANT/BLOB types in 1.2.0. Apache 2.0.
• Pick by writer shape and catalog politics first: multi-engine lake with REST catalogs leans Iceberg; Spark-centric MERGE and Unity Catalog lean Delta; continuous Flink upserts and incremental consumers lean Hudi. Many teams run more than one format.
• In my drill, pyiceberg 0.12.0 wrote and time-travelled 100,000 rows, and deltalake 1.6.6 did the same plus a 1,000-row merge. Hudi writes needed Spark or Flink; the hudi 0.5.0 Python bindings on this box are read-oriented only.

Where things stand on 12 October 2026

Dates are the release dates printed on each project’s notes or GitHub release pages. GitHub timestamps, converted to IST, agree within a day.

Project

Latest release

Notes

Licence

Apache Iceberg

1.12.0 on 29 September 2026

Current tip; Spark 3.4 and Flink 2.0 support removed; Flink 2.2/2.3 added; V4 foundations and deletion-vector work continue

Apache 2.0

Apache Iceberg

1.11.0 on 19 May 2026

Prior minor; Spark 4.1 and Flink 2.1 support landed here

Apache 2.0

Delta Lake

4.4.1 on 5 October 2026

Patch on 4.4.0; safer maintenance for catalog-managed tables; Spark 4.2/4.1/4.0.1 artifacts

Apache 2.0

Delta Lake

4.4.0 on 20 August 2026

Spark 4.2 support; identity and generated columns in SQL DDL; UC Delta API work in Kernel and Flink

Apache 2.0

Delta Lake

3.3.3 on 12 August 2026

Maintenance line for Spark 3.x stacks

Apache 2.0

Apache Hudi

1.2.1 on 24 September 2026

Bug-fix on 1.2.0; Trino connector now in-repo against Trino 483; table version still 9

Apache 2.0

Apache Hudi

1.2.0 on 23 May 2026

VECTOR, VARIANT, BLOB; Lance base-file option; Flink RLI and Source V2; Spark 4.1 and Flink 2.1 bundles

Apache 2.0

Date

Event

14 December 2016

Apache Hudi repository created on GitHub (Uber origins; ASF later)

19 November 2018

Apache Iceberg repository created on GitHub (Netflix origins; ASF later)

22 April 2019

delta-io/delta repository created on GitHub

11 December 2024

Hudi 1.0.0

9 June 2025

Delta Lake 4.0.0

11 September 2025

Iceberg 1.10.0

23 May 2026

Hudi 1.2.0

19 to 20 May 2026

Iceberg 1.11.0

20 August 2026

Delta Lake 4.4.0

24 September 2026

Hudi 1.2.1

29 to 30 September 2026

Iceberg 1.12.0

5 October 2026

Delta Lake 4.4.1

First question: do you need a lakehouse table format at all?

Table formats solve coordinated writers and readers over object storage. They do not replace an OLTP database, and they do not make a random folder of Parquet into a product by themselves.

Stay with plain Parquet or engine-native tables when:

• one engine owns the path end to end, and a simple partition layout is enough;
• you have no concurrent writers, no need for time travel, and schema changes are rare;
• the dataset is small enough that rewriting whole partitions on every update is honest.

Consider a table format when:

• Spark, Flink, Trino, DuckDB or an OLAP engine must share the same files safely;
• you need upserts, deletes, or CDC into the lake without nightly full reloads;
• auditors ask for “what did this table look like last Tuesday”;
• you want compaction and snapshot expiration as first-class jobs, not ad-hoc scripts.

Object storage still matters. My MinIO and alternatives post is about the S3-compatible layer under the lake. The format sits above that layer. If the Hyderabad team’s pain is only bucket layout and credentials, fix storage first. If the pain is conflicting Spark jobs and broken Hive partitions after every schema change, pick a format.

For streaming ingest into the lake, treat Kafka or Redpanda as the bus and the table format as the landing contract. My Kafka and Redpanda post is about the bus, not the table.

What a table format actually is

A table format tracks which data files belong to a table version, how the schema looks, and how a commit becomes visible. Writers add or remove files and then atomically publish a new metadata pointer. Readers pin a snapshot or version and ignore in-flight writers.

Idea

Iceberg

Delta Lake

Hudi

Metadata core

Table metadata JSON, manifest lists, manifests, snapshots

_delta_log transaction log (JSON, optional checkpoints)

Timeline under .hoodie with instant actions

Data files

Parquet, ORC, Avro (Parquet common)

Parquet (primary)

Parquet, ORC, HFile; Lance option in 1.2 for AI shapes

Commit model

Optimistic swap of metadata pointer; sequence numbers

Optimistic concurrency on log versions; optional coordinated commits

Instant-based commits; OCC with lock providers for multi-writer

Row-level changes

Position deletes, equality deletes, deletion vectors (v3+)

Merge/update/delete; deletion vectors in modern lines

Copy-on-Write rewrites; Merge-on-Read log blocks + compaction

Time travel

Snapshot ID or timestamp

Version or timestamp

Instant time / commit time queries

Iceberg’s adopted spec covers format versions 1 (analytic tables), 2 (row-level deletes) and 3 (variant, geo, row lineage, binary deletion vectors, defaults). Version 4 restructures metadata (relative paths and related work) and is not yet formally adopted. Delta documents ACID transactions, scalable metadata, time travel, schema enforcement and DML (MERGE, update, delete). Hudi documents Copy-on-Write and Merge-on-Read table types, snapshot / incremental / read-optimized query types, and a rich set of table services (clean, compact, cluster, archive).

Apache Iceberg

Iceberg separates planning from listing every file in a directory. Manifests carry metrics so readers skip files that cannot match a filter. Partitioning is a transform of column values (identity, bucket, truncate, year/month/day/hour), not a frozen Hive folder contract, so you can evolve the partition spec without rewriting history as a new table name.

Catalogs are first-class. Spark configuration documents Hive, Hadoop (directory), REST, Glue, JDBC and Nessie under SparkCatalog, plus SparkSessionCatalog when you want Iceberg and non-Iceberg tables in one Spark session. REST catalogs (including Apache Polaris and vendor REST endpoints) are where many 2026 lakehouses centralise auth and storage credentials.

1.12.0 highlights that matter for operators: Spark 3.4 support removed; Flink 2.0 support removed; Flink 2.2 and 2.3 added; more deletion-vector and variant work; REST catalog OpenAPI additions (unregister table, labels, finer loadTable restrictions). Treat a jump to 1.12 as a tested upgrade, not a casual dependency bump.

Delta Lake

Delta’s _delta_log is the source of truth. Each commit is a numbered JSON action file; checkpoints compact history for faster starts. Readers see a consistent snapshot; writers retry on conflict. The open project ships Delta Spark, Delta Kernel (connector libraries), Delta Sharing, experimental Kernel-based Flink connectors, and UniForm (delta-iceberg) so Iceberg clients can read Delta tables when metadata sync is enabled.

4.4.x targets Spark 4.x (Scala 2.13 artifacts). 4.4.0 added Spark 4.2 support, identity columns and generated columns in SQL DDL, and broader UC Delta API wiring in Kernel and Flink. 4.4.1 tightens maintenance on catalog-managed tables (block unsupported cleanups, skip automatic maintenance that the catalog should own) and fixes snapshot construction when S3A fast listing meets staged commits.

If the Hyderabad team lives in Spark SQL and already practises MERGE INTO for SCD and CDC, Delta is the shortest path. If every reader must be a pure Iceberg client tomorrow, plan UniForm carefully and verify engine versions; UniForm is interoperability, not “Delta disappears”.

Apache Hudi

Hudi grew from upsert-heavy ingestion. Copy-on-Write (CoW) rewrites Parquet file groups on update so snapshot reads stay simple. Merge-on-Read (MoR) writes deltas to log files and compacts later, trading read merge cost for write latency. Indexes (bucket, record-level, and others) steer updates to file groups. Timeline actions record commits, cleans, compactions and clustering.

1.2.0 opened unstructured types: VECTOR (fixed-dimension embeddings), VARIANT (semi-structured), BLOB (inline or out-of-line bytes), plus hudi_vector_search() and read_blob() in Spark SQL, and optional Lance base files. 1.2.1 is the bug-fix tip: Flink streaming correctness fixes, Spark datasource fixes, Trino connector published from the Hudi repo against Trino 483, Hive/Glue sync leak fixes. Known issue on 1.2.x: Spark 4.1 PushVariantIntoScan can mis-read VARIANT on MoR; the project documents spark.sql.variant.pushVariantIntoScan=false as the workaround until a later fix lands.

Hudi still shines when the primary writer is a continuous upsert stream and consumers want incremental pull by commit time. It is heavier to operate than “just append Parquet” because table services are part of correctness, not optional polish.

ACID and concurrent writers

All three aim for atomic table commits so readers do not see half-written jobs. The conflict behaviour differs.

Topic

Iceberg

Delta Lake

Hudi

Isolation story

Serializable isolation goals in the spec; snapshot reads pinned at plan time

Serializable isolation on Spark path (project docs)

Instant visibility; query types choose snapshot vs incremental vs read-optimized

Concurrent writers

Optimistic retries; commit validation rules per operation

Optimistic concurrency on log; catalog-managed / coordinated commit options in 4.x

optimistic_concurrency_control with a lock provider; storage-based locks including Azure in 1.2

Typical failure mode

Commit conflict, retry or abort

ConcurrentModification-style failure, retry

Writer abort / rollback of failed instants; heartbeat tuning (1.2.1 default tolerable misses raised to 10)

Practical notes for Hyderabad:

1. Two Spark jobs rewriting the same Iceberg partition need tested retry settings and, ideally, disjoint write sets.
2. Delta catalog-managed tables in 4.4.1 deliberately stop some client-side maintenance; do not fight the catalog with scripts that vacuum what the catalog owns.
3. Hudi multi-writer without a lock provider is a foot-gun. Prefer Writer Conflict Resolution Strategy docs and a real lock (ZooKeeper, HMS, or storage-based) before you scale Flink tasks.

Catalogs

The catalog is where table names become storage locations and credentials. Format wars often turn into catalog wars.

Catalog style

Iceberg

Delta Lake

Hudi

Hive Metastore

First-class HiveCatalog

Common with Spark; metastore sync patterns exist

Hive sync / HMS JDBC paths; HMS 4.x fallback in 1.2

REST

First-class REST catalog (+ Polaris and vendor REST)

Unity Catalog and UC Delta APIs; Iceberg REST endpoints on some platforms for UniForm reads

Less central than Iceberg’s REST story; meta sync to HMS/Glue

Cloud native

AWS Glue, GCS/BigQuery metastore integrations

Unity Catalog (Databricks and open UC efforts); Glue in AWS stacks

Glue sync supported

Git-like / transactional meta

Nessie

Not the same model

Not the same model

JDBC / SQL catalog

JDBC and SQL (for example sqlite in pyiceberg labs)

Filesystem log can stand alone; external catalogs optional

Timeline on storage; SQL catalog not the centre

Directory / Hadoop

HadoopCatalog for labs

Path-based tables common

Path-based tables common

Rules of thumb:

• If multiple engines and clouds must share one auth story, Iceberg REST (or a Polaris-style catalog) is the usual design centre in 2026.
• If you are standardised on Unity Catalog, Delta is native; Iceberg tables may also live there depending on platform support. Verify your vendor’s matrix, not a slide.
• Hudi teams often keep HMS or Glue in sync for discovery while the timeline remains authoritative for commits.

Security note: Apache Polaris (an Iceberg REST catalog implementation) published several CVEs in 2026 (see Security below). A catalog bug can be as serious as a format bug because catalogs mint storage credentials.

Time travel, schema evolution, compaction

Capability

Iceberg

Delta Lake

Hudi

Time travel

Snapshot ID, as-of-timestamp, branches/tags (refs)

Version number, timestamp; RESTORE

Commit / instant based; incremental queries by time range

Schema evolution

Add/drop/rename/reorder; type promotion rules; defaults in v3

Enforcement and evolution; identity/generated columns in 4.4 Spark SQL

Schema-on-write evolution; Streamer helpers; VARIANT/VECTOR/BLOB in 1.2

Partition evolution

Supported via new specs; void transform patterns for v1

Partitioning is part of table layout; evolve with care

Partition path schemes; slash-separated dates option in 1.2

Compaction / rewrite

rewrite_data_files, manifest rewrite, expire snapshots, remove orphans

OPTIMIZE, vacuum, checkpointing; auto-compaction settings (respect catalog-managed rules in 4.4.1)

Compaction (MoR), clustering, cleaning, archival; pre-write cleaner options in 1.2

Delete vectors

Spec v3 binary deletion vectors; 1.12 continues DV work

Deletion vectors used in modern merge strategies

MoR log blocks; different mechanism

Operational habit that saves weekends: schedule expiration and vacuum only after you know the slowest consumer’s lag. Time travel is not free storage. Orphan-file removal must see the same catalog the writers use, or you will delete live data.

Engine support (verified directions, not a promise matrix)

Engine support changes by minor release. Confirm the exact runtime JAR or connector version before you commit a platform standard.

Engine

Iceberg

Delta Lake

Hudi

Apache Spark

Official runtimes for 3.5, 4.0, 4.1 in 1.12.0

Delta Spark 4.4.x for Spark 4.2/4.1/4.0.1; 3.3.x for Spark 3

Spark bundles including 4.0/4.1 in 1.2; Java 17+ for Spark 4.x

Apache Flink

Runtimes for 1.20, 2.1, 2.2, 2.3 in 1.12.0

Experimental Kernel-based delta-flink for Flink 2.0.2 to 2.3.0 in 4.4.1 notes

Mature Flink writers; 2.1 bundle in 1.2; Source V2 opt-in

Trino

Widely used Iceberg connector (Trino project)

Delta connector in Trino ecosystem

Hudi connector; 1.2.1 builds hudi-trino against Trino 483

ClickHouse

Iceberg table engine / integration docs

Separate paths; not the same as native MergeTree

Not the centre of Hudi’s story

DuckDB

Iceberg extensions / readers in the DuckDB ecosystem

Delta readers in the DuckDB ecosystem

Limited compared with Spark/Flink

Python

pyiceberg (I used 0.12.0)

deltalake (I used 1.6.6)

hudi-rs Python bindings (0.5.0 read-oriented); full writers via PySpark

Doris and StarRocks expose Iceberg catalogs for lakehouse queries; that is query-side support for Iceberg data, covered in the OLAP post, not a reason to skip choosing a write format.

Operations

Concern

Iceberg

Delta Lake

Hudi

Upgrade mindset

Spec version and engine runtime must move together; 1.12 drops Spark 3.4 / Flink 2.0

Spark major lines split 3.3.x vs 4.4.x; catalog-managed tables change who may vacuum

Table version 9 across 1.1.1 to 1.2.1 for many upgrades; still read behaviour-change notes

Small files

Rewrite data files + manifest maintenance

OPTIMIZE / bin-pack style maintenance

Clustering and compaction services

Streaming landing

Flink sink, Kafka Connect sink in-tree

Spark Structured Streaming + Flink experimental

Hudi Streamer, Flink source/sink, Kafka sources

Observability

Commit metrics reporters; REST metrics

Spark UI + log history

Rich metrics in 1.2 (clean/archive/rollback/postCommit families)

Kubernetes

Run engines and catalogs as you already do; format is in the data plane

Same

Same; watch Flink checkpoint compatibility when toggling Source V2

Capacity practise that matters more than format slogans:

• measure commit conflict rate under two writers;
• measure query planning time as file counts grow past a few hundred thousand;
• rehearse snapshot expiration on a clone before production;
• keep a documented “who is allowed to vacuum” rule when catalogs manage tables.

Licences

Project

Licence (verified)

Notes

Apache Iceberg

Apache License 2.0 (LICENSE in apache-iceberg-1.12.0 tree)

ASF project

Delta Lake

Apache License 2.0 (LICENSE.txt in v4.4.1 tree)

Linux Foundation Delta Lake; copyright line names The Delta Lake Project Authors

Apache Hudi

Apache License 2.0 (LICENSE in release-1.2.1 tree)

ASF project

All three cores are Apache 2.0 on 12 October 2026. That does not make every surrounding catalog or cloud feature Apache 2.0. Unity Catalog deployments, managed Polaris offerings and vendor support contracts have their own terms. Read those separately.

Security and CVEs

I queried the CVE Program API (cveawg.mitre.org) for IDs found via NVD keyword search and project dependency notes. Do not treat this as a full audit.

CVE

Product (CNA)

Published

Relevance to lakehouses

CVE-2025-30065

Apache Parquet Java (parquet-avro <= 1.15.0)

1 April 2025

Critical schema-parsing RCE class issue; Iceberg and Hudi release notes record Parquet bumps to pick up fixes

CVE-2025-46762

Apache Parquet Java (through 1.15.1 in CNA text)

6 May 2025

Follow-on trusted-package issue for specific/reflect Avro models

CVE-2026-42812

Apache Polaris (< 1.4.1)

4 May 2026

write.metadata.path style property changes could bypass commit-time location checks for Iceberg metadata in Polaris

CVE-2026-42810

Apache Polaris (< 1.4.1)

4 May 2026

* in namespace/table names affecting S3 IAM resource patterns for vended credentials

CVE-2026-64640

Apache Polaris (<= 1.6.0)

6 August 2026

Registration path could read caller-selected Iceberg metadata with catalog credentials before location checks

CVE-2026-97395

Apache Polaris (< 1.8.0)

29 September 2026

Table properties such as s3.endpoint influencing server-side FileIO

I did not find a distinct Delta Lake core CVE in the NVD keyword pass that clearly targets the delta-io project itself (noise from unrelated “delta” kernel CVEs appeared). Dependency CVEs in Spark, Parquet and connector stacks still apply. Pin engine and format versions, and patch catalogs as carefully as you patch writers.

Side-by-side comparison

Dimension

Prefer Iceberg when…

Prefer Delta when…

Prefer Hudi when…

Engines

Many engines must write or read the same tables

Spark is the centre of gravity

Flink/Spark upsert pipelines dominate

Catalog

REST / Polaris / Nessie style control plane

Unity Catalog or Delta log + metastore

HMS/Glue sync beside timeline is enough

Upserts

Increasingly strong (v2/v3 deletes, Flink DV work)

Mature MERGE story in Spark

Record-level upserts and incremental pull are the product

Interop

Native Iceberg everywhere

UniForm toward Iceberg/Hudi readers

Primary Hudi; readers via Spark/Flink/Trino

AI-ish types

Variant/geo in v3; 1.12 expands engine support

Strong structured + some variant paths via Spark

VECTOR/VARIANT/BLOB and vector_search in 1.2

Ops cost

Catalog + rewrite/expire jobs

Log checkpointing + OPTIMIZE/vacuum discipline

Table services are mandatory for MoR health

Migration sketches

Moving formats is a project, not a flag.

Hive / plain Parquet directories to Iceberg. Use engine migrate or snapshot procedures (Spark migrate / snapshot style flows in Iceberg docs) so you register existing files without a blind copy when possible. Validate partition transforms and field IDs. Keep the old location read-only until checksum-style counts and spot queries match.

Delta to Iceberg readers without copying data. UniForm (delta-iceberg) can sync Iceberg metadata for Iceberg-only tools. Confirm your Delta version, column mapping requirements, and whether readers are read-only. Dual-write is usually worse than UniForm or a planned cutover.

Hudi MoR to CoW (or the reverse). Treat it as a table-type change with compaction catch-up first. Consumers that relied on incremental pull must be retested. Do not flip hoodie.datasource.write.table.type on a busy table without a rehearsal.

Between clouds. Copying Parquet without copying metadata leaves you with orphans. Prefer export/register tools that rewrite metadata pointers, or use catalog-level registration after a bulk copy with a freeze window.

Away from a format. If you abandon a lakehouse format for an OLAP engine’s native tables, that is an ETL project. Keep the lake as cold storage if auditors still need time travel.

Compaction and maintenance in practise

Small files kill planning time and object-store LIST bills.

For Iceberg, run rewrite of data files on hot partitions, rewrite manifests when manifest counts climb, expire snapshots on a retention policy tied to the slowest time-travel consumer, then remove orphan files with the same FileIO and warehouse the writers use. Partial progress and max-file limits exist so jobs can finish inside a maintenance window.

For Delta, checkpoint the log so readers do not replay thousands of JSON commits, OPTIMIZE to bin-pack small Parquet files, and VACUUM only after you understand the retention setting and any catalog-managed restrictions. On 4.4.1 catalog-managed tables, let the catalog own destructive cleanup.

For Hudi, MoR without compaction is a slow read waiting to happen. Schedule compaction and clustering, watch clean and archive durations (1.2 adds more metrics here), and keep failed-write rollback policy explicit (LAZY vs eager). Storage-based locks help multi-writer Azure and cloud object paths without standing up ZooKeeper only for locks.

Shared practise: one ownership chart for maintenance jobs, paging on conflict rate and on “last successful compaction age”, and a ban on ad-hoc vacuum from laptops in production warehouses.

Decision guide

Situation

Lean toward

Why

Greenfield multi-engine lake on S3-compatible storage

Iceberg

REST catalog ecosystem and broad reader support

Existing Databricks / Unity Catalog estate

Delta Lake

Native integration; UniForm if Iceberg readers appear later

Flink CDC into lake with incremental BI pulls

Hudi

Upsert + incremental query model

Mostly append-only facts, Trino/Spark SQL

Iceberg or Delta

Both fine; pick by catalog standard already in the company

Need embeddings beside facts in one table

Hudi 1.2 (VECTOR) or Iceberg v3 variant/geo paths

Match the type system you will actually query

Team strength is Spark SQL DML

Delta Lake

Least new mental model

OLAP engine will own hot data, lake is cold

Iceberg as lake + OLAP catalogs

Matches the split in the OLAP post

For the Hyderabad payments platform, my calm recommendation is:

1. Keep PostgreSQL as the system of record for authorisations and schedules.
2. Land CDC and batch facts into one primary table format. If the company is standardising a REST catalog and Trino, start with Iceberg. If the lake is already Delta on Unity Catalog, stay on Delta and enable UniForm only when an Iceberg-only tool appears.
3. Introduce Hudi only for streams that truly need record-level upsert latency and incremental consumers, or accept Hudi as the primary format if that is most of the lake’s write path.
4. Do not run three formats “for flexibility” without a catalog and ownership map. Two is already expensive.

Checklist before you standardise a format

1. Write down the engines that must write, and those that only read, for the next 18 months.
2. Choose the catalog (REST, Glue, Unity Catalog, HMS, Nessie) before you choose cute folder names.
3. Prove one upsert or MERGE path with production-shaped keys, not only appends.
4. Prove time travel for the retention you promise auditors, and price the storage.
5. Schedule compaction/clustering/OPTIMIZE and snapshot expiration as owned jobs with alerts.
6. Align Spark/Flink/Trino versions with the format release’s supported matrix.
7. Patch Parquet and catalog dependencies; track Polaris or UC advisories if you use them.
8. Test concurrent writers on one table until conflict behaviour is boring.
9. Document who may VACUUM / expire snapshots / remove orphans.
10. Run a kill test: interrupt a writer mid-job and confirm readers still see the last good commit.

Common mistakes

1. Treating Iceberg, Delta and Hudi as interchangeable Parquet folders.
2. Picking a format from a vendor benchmark without labelling it as the vendor’s own run.
3. Enabling MoR (Hudi) or deletion vectors without budgeting compaction.
4. Letting every job create its own catalog configuration and warehouse path.
5. Vacuuming while a Trino gateway still serves week-old snapshots.
6. Upgrading to Iceberg 1.12 or Delta 4.4 on the same day as a Spark major upgrade without a staging catalog.
7. Assuming Python bindings (pyiceberg, deltalake, hudi-rs) cover the same write features as Spark.

Drill: what I ran on this box (12 October 2026)

Environment: shared Linux VM, 8 vCPU, about 15 GiB RAM, no root, no Docker for engines. Python 3.13.5. Packages: pyiceberg 0.12.0 (with SQL/sqlite extra), deltalake 1.6.6, pyarrow 26.0.0, hudi 0.5.0. Scripts and JSON results live under /workspace/revisit/drill29/.

Workload: 100,000 synthetic order rows (order_id, city, amount, status), then append 100 rows, then time travel to the first snapshot/version. For Delta, also merge 1,000 rows to set status to PAID.

Iceberg (pyiceberg 0.12.0, sqlite SQL catalog, identity partition on city)
  write 100000 rows: 0.0788 s
  append 100 rows:   0.0277 s
  snapshots: 2
  current rows: 100100
  time-travel snapshot v1 rows: 100000
  city=Pune rows (current): 20100
  warehouse bytes: 325106

Delta (deltalake 1.6.6, local path)
  write 100000 rows: 0.0196 s
  append 100 rows:   0.0070 s
  versions: 0 -> 1 -> 2 (after merge)
  time-travel version 0 rows: 100000
  merge 1000 statuses: 0.0386 s, paid_rows=1000
  warehouse bytes: 621082

Hudi (hudi 0.5.0)
  installed; HudiTable exposes read/timeline APIs only
  no create/append/upsert writer without Spark or Flink on this box

Process RSS max for the Python run was about 410 MiB (ru_maxrss on Linux). Timings are a smoke test on local disk, not a benchmark.

Limits of the drill:

• No S3, no Glue, no REST catalog, no Unity Catalog.
• No Spark or Flink, so no Hudi write path and no full SQL DML matrix.
• Single process writers; no conflict stress test.
• File sizes and wall times are not comparable across clouds.
• Heavyweight engine installs were skipped on purpose; Python packages stayed.

Unlearn / re-learn

Unlearn

• “Lakehouse format” means one winner for every team.
• Hive partitions alone give you ACID.
• Time travel is free.
• Catalog CVEs are somebody else’s problem.
• A Python quickstart equals production Spark behaviour.

Re-learn

• Commits, catalogs and maintenance jobs are the product.
• MoR and deletion vectors shift cost from write path to read/compaction path.
• Engine matrices move every minor release; pin them.
• Practise expire/vacuum on a clone with a real consumer lag assumption.

Closing

Open lakehouse table formats in 2026 are mature enough that the hard part is organisational: which engines write, which catalog owns names and credentials, and who runs compaction. Learn how Iceberg snapshots, Delta logs and Hudi timelines actually commit. Unlearn the idea that a folder of Parquet is already a lakehouse. Re-learn licence and CVE lines for formats and catalogs together. Practise with a small write/read/time-travel drill on your own storage, as above. Apply one primary format for shared facts, keep PostgreSQL on the transactional path, and let OLAP engines query the lake when interactive speed needs a different core.

Sources

Apache Iceberg

• Iceberg releases (1.12.0)
• Iceberg 1.12.0 GitHub release
• Iceberg table spec
• Iceberg Spark configuration (catalogs)
• Iceberg LICENSE (apache-iceberg-1.12.0)
• pyiceberg project

Delta Lake

• Delta Lake home
• Delta Lake 4.4.1 GitHub release
• Delta Lake 4.4.0 GitHub release
• Delta Lake 3.3.3 GitHub release
• Delta introduction docs
• Delta LICENSE.txt (v4.4.1)
• Delta blog: Kernel and UC Delta APIs (20 Aug 2026)

Apache Hudi

• Hudi overview
• Hudi 1.2 release notes (1.2.0 and 1.2.1)
• Hudi 1.2.1 GitHub release
• Hudi 1.2.0 GitHub release
• Hudi LICENSE (release-1.2.1)
• ANNOUNCE Apache Hudi 1.2.1

Security (CVE Program)

• CVE-2025-30065
• CVE-2025-46762
• CVE-2026-42812
• CVE-2026-42810
• CVE-2026-64640
• CVE-2026-97395

Internal Revisit.Tech links used in the body

• Real-time OLAP in 2026: ClickHouse, Doris, StarRocks
• MinIO and its alternatives
• Kafka and Redpanda