Schema registries in 2026: revisit Confluent Schema Registry, Apicurio Registry and Karapace before you ship another event schema
An event schema looks harmless in a pull request. A few fields, a namespace, maybe a default value. But once a producer starts writing with it, that schema is a contract with every consumer that reads the topic today and every consumer that will replay it next year. The schema registry is the place where that contract is stored, versioned and checked. When it is set up casually, the damage shows up late: a consumer that cannot deserialise old messages, a field number reused in Protobuf, or a registry nobody can upgrade because nobody knows what depends on it.
This post compares the three registries most Kafka teams shortlist when they run things themselves: Confluent Schema Registry, Apicurio Registry and Karapace from Aiven. We also mention the registry built into Redpanda, because many teams meet it first. If you are still choosing the broker itself, start with Kafka and Redpanda: revisit the choice, and if you are not sure Kafka is the right bus at all, see RabbitMQ, NATS and Kafka queues. Change data capture is one of the biggest producers of schemas, which we covered in change data capture in 2026. Database schema changes are a close cousin of event schema changes, and the same discipline applies; see schema migrations in 2026. For the login and token side of securing a registry, see Keycloak and its alternatives.
We end with a small, safe drill on Avro schema evolution and a plain list of what we did not run.
A scenario to keep us honest
Imagine a company in Vadodara that builds dispatch, cold-chain tracking and trucking software for pharmaceutical and chemical distributors across Gujarat. The company is invented for this post. It has about 50 engineers, a four-person platform team and a small data team.
The setup today:
- A self-managed Apache Kafka cluster on rented VMs, running in KRaft mode.
- One Confluent Schema Registry instance from the Confluent Platform 7.6 community Docker image, installed in early 2024 and not upgraded since.
- About 300 subjects. Most are Avro, written by Java services and by Debezium from PostgreSQL. The Go team that builds the driver mobile backend uses Protobuf.
- Compatibility was never changed from the default, so every subject is on BACKWARD.
- The registry has no authentication. Anyone on the internal network can register or delete a schema.
Four things push the team to revisit this. First, the Confluent support table shows that Confluent Community software support for 7.6.x ended on 9 February 2026, so the registry is on an unsupported line. Second, product managers want to offer a hosted partner integration service where distributors subscribe to dispatch events, and somebody asks whether the Confluent Community License allows that. Third, in August a Go engineer removed a Protobuf field and later reused its field number for a different meaning, and an analytics consumer quietly read wrong values for two days. Fourth, the security team wants every registry call to go through the company Keycloak, with write rights only for owning teams.
We will come back to Vadodara in the worked plan.
The short version
- A schema registry stores schemas by subject, gives each one an ID, and checks new versions against compatibility rules. Producers put the schema ID in each message, so the registry and its IDs become part of your data.
- Confluent Schema Registry (Confluent Platform 8.4, released 27 October 2026) is the reference implementation. Its server is under the Confluent Community License, while its client serialisers are Apache 2.0. Fine-grained authorisation needs a Confluent Enterprise licence.
- Apicurio Registry (3.3.3, 8 September 2026) is Apache 2.0, Java and Quarkus based, stores data in PostgreSQL or Kafka, and offers a Confluent-compatible API at /apis/ccompat/v7 and /apis/ccompat/v8 besides its own API. It also stores OpenAPI, AsyncAPI and other artefacts.
- Karapace (6.2.4, 2 October 2026) is Apache 2.0, written in Python, and aims to be a drop-in replacement for Confluent Schema Registry and REST Proxy. It stores schemas in a Kafka topic like Confluent does. Its README says it is compatible with Schema Registry 6.1.1 at the API level.
- Redpanda has a registry built into the broker binary, with the same wire format and API style.
- The compatibility mode you choose matters more than the product. Transitive modes and a check in CI prevent most incidents.
- Migration between registries is mostly about keeping schema IDs. If the IDs change, old messages point at the wrong schema.
Where things stand on 28 October 2026
Confluent Schema Registry. It ships as part of Confluent Platform. The Confluent support table lists Confluent Platform 8.4.x (with Kafka 4.4.x) as released on 27 October 2026, with Community software support until 27 October 2027. Before that, 8.3.x came out on 17 June 2026 and 8.2.x on 4 March 2026. The 8.1.x line, released on 15 October 2025, reached the end of Community support on 15 October 2026. The GitHub repository does not use GitHub releases; it has tags, and the v8.4.0 tag points to a commit dated 16 October 2026. The 8.4 release notes add Logical Types (define a schema with SQL-like types such as STRUCT, INT and VARCHAR, and let the registry translate it to Avro, Protobuf or JSON Schema), a new LOGICAL compatibility policy that catches changes that would break Flink or Iceberg table mappings, inline data quality rules, and explicit topic to subject associations. Confluent Platform 8.4 supports Java 25, 21 and 17.
Apicurio Registry. The latest release is 3.3.3, published on GitHub on 8 September 2026 at 8:12 PM IST. Before it came 3.3.2 (27 August 2026), 3.3.1 (27 July 2026) and 3.3.0 (8 June 2026). The 3.2 line had its last release, 3.2.6, on 3 July 2026. The 3.3.0 release added a large set of data contract features (an Open Data Contract Standard parser, a CEL rule executor and tag extraction for Avro, Protobuf and JSON Schema), a command line tool and autoscaling support in the operator. It also lists a fix for “incompatibilities with Confluent Schema Registry identified by compatibility harness”. The project is licensed under Apache 2.0.
Karapace. The latest release is 6.2.4, published on GitHub on 2 October 2026 at 7:44 PM IST. It adds an import mode (register schemas with their original IDs and versions) and a migration script. The 6.x line began with 6.0.0 on 5 March 2026, which removed the built-in backup tool. 6.1.0 (12 March 2026) added Avro schema references, and 6.2.0 (5 June 2026) split OIDC authentication from authorisation and added native OAUTHBEARER support for its Kafka connection. Karapace is licensed under Apache 2.0. It is not published on PyPI; you install from source or use the container image at ghcr.io/aiven-open/karapace.
Redpanda. The Redpanda documentation for version 26.2 lists v26.2.6 as the latest tag. Its registry is part of the broker binary, listens on port 8081 by default, supports Avro, Protobuf and JSON Schema, and stores data in a compacted _schemas topic.
Security in 2026
Karapace. Two advisories matter:
- CVE-2025-61673 (high, CVSS 8.6). In Karapace 5.0.0 and 5.0.1, when OAuth 2.0 bearer token authentication was enabled, a request with no Authorization header skipped token validation completely, so an unauthenticated user could read and write schemas. Fixed in 5.0.2.
- CVE-2026-29190 (medium). A path traversal in the backup reader could let a malicious backup file read arbitrary files on the host. The fix was to remove the backup feature altogether in 6.0.0.
The first one is a good reminder to test the negative case. Send a request with no token and a request with a wrong token, and confirm both fail, before you trust any auth configuration.
Apicurio Registry. NVD lists three CVEs published on 25 and 26 June 2026, all about XML handling, because Apicurio also stores XML, WSDL and XSD artefacts:
- CVE-2026-12975: XML parsing without secure processing could allow blind server-side request forgery or entity expansion. The description says an attacker needs artefact-write permission, or no permission at all when the registry runs with its default configuration.
- CVE-2026-12992: with the validity rule set to FULL, a WSDL document with attacker-controlled imports could make the registry call internal URLs.
- CVE-2026-12993: a “billion laughs” style entity expansion could exhaust CPU and memory (CVSS 6.5).
The project also published GHSA-m2hr-xvqh-qwfq on 14 October 2026 (no CVE ID at the time we checked): with GitOps storage and dry-run validation enabled, a symbolic link in validated content could make cleanup delete files outside the task directory. It affects 3.3.0 and 3.3.1 and is fixed in 3.3.2, with more hardening in 3.3.3. Separately, the 3.2.0 notes list a fix for “non-owners are able to update artifacts through the ccompat API”. If you run Apicurio on an older 3.x release, move to 3.3.3.
Confluent Schema Registry. The GitHub repository’s security advisories page lists none, and an NVD keyword search found no CVE that names Confluent Schema Registry. That does not mean no bugs; Confluent publishes its own security notices, and the registry depends on Jetty, Jackson and many other libraries that receive fixes through regular releases. The bigger risk we see in practice is not a CVE but an open registry: no auth, so anyone can delete a subject.
Lessons. Turn on authentication. Limit who can change compatibility settings and delete subjects. Keep the registry off the internet. Stay on a supported release line.
The first question: what is the contract?
Before comparing products, write down three things for each important topic:
- Which format. Avro, Protobuf or JSON Schema. Each has different evolution rules. Avro needs defaults for added fields. Protobuf identifies fields by number, so a removed field’s number must be reserved and never reused. JSON Schema compatibility depends a lot on whether the schema allows additional properties.
- Who upgrades first. If consumers always deploy before producers, BACKWARD is the natural fit. If producers always go first, FORWARD. If you cannot control the order, FULL. If consumers may replay old data from the start of a topic, use the transitive variant, which checks against all earlier versions and not only the latest one.
- Which subject naming strategy. The default topic name strategy gives one schema line per topic. The record name and topic record name strategies allow several event types in one topic. Changing this later is painful, because the subject names change.
Most incidents with registries come from these three choices, not from the registry software.
What each tool is
Confluent Schema Registry
The original Kafka schema registry, started by Confluent in 2014 (the GitHub repository was created on 9 December 2014). It is a Java service with a REST API. All schemas, versions, IDs and compatibility settings are written as records to a Kafka topic, by default _schemas, with a single partition. Every instance reads that topic into memory. One instance is the primary and handles writes; the others forward writes to it and serve reads themselves. Confluent’s documentation describes this as a single-primary architecture with primary election through Kafka (or ZooKeeper in older setups).
Confluent also ships the serialisers and deserialisers (serdes) that most clients use, for Java and, through librdkafka-based clients, for Python, Go, .NET and others. Newer features include schema contexts, data contracts with rules, client-side field encryption and, from 8.4, Logical Types.
Apicurio Registry
A general registry for schemas and API designs, started in 2019 (repository created on 16 July 2019) and sponsored by Red Hat. It is a Java application on Quarkus with a separate web UI and a Kubernetes operator. It stores Avro, Protobuf and JSON Schema, and also OpenAPI, AsyncAPI, OpenRPC, GraphQL, XML-based types and, in recent releases, AI artefacts such as model schemas and prompt templates. Content is organised as groups, artefacts and versions, with rules at global, group and artefact level.
Its documentation lists three storage options: embedded H2 (the default, in memory, for development only, data lost on restart), PostgreSQL (recommended for production) and Kafka. Release notes in 2026 also mention GitOps storage and a KubernetesOps variant that keeps artefacts in ConfigMaps. Apicurio has its own serdes, and it also exposes the Confluent API so Confluent clients can use it.
Karapace
An open source implementation of Confluent Schema Registry and Kafka REST Proxy, started by Aiven in 2019 (repository created on 7 January 2019). It is written in Python, with the registry built on FastAPI. Like Confluent, it keeps everything in a Kafka topic (_schemas by default), and its instances use a Kafka group to elect a leader that handles writes. The instance whose client_id sorts first among those with master_eligibility set to true becomes leader.
Its README describes it as a drop-in replacement on both the client and the server side, supporting Avro, JSON Schema and Protobuf, with caveats: schema normalisation and error messages may not always match Confluent exactly.
Redpanda’s built-in registry
Redpanda runs the registry inside each broker. Its documentation says every broker accepts writes, with optimistic concurrency on the _schemas topic, so there is no leader to configure. It uses the same wire format (a magic byte of 0, then a four-byte schema ID) and the Confluent-style API. It supports only the metadata properties part of Confluent’s data contracts, not rule sets. If you already run Redpanda, this is often enough, and it is one less service.
Side by side
Licence and governance
- Confluent Schema Registry: the server is under the Confluent Community License 1.0. The README lists the Apache 2.0 modules: the client libraries, the Avro, Protobuf and JSON Schema serialisers and serdes, the encryption client modules, schema types and rules, and the Maven plugin. The Community License lets you use, modify and redistribute the software, but excludes “making available any software-as-a-service, platform-as-a-service, infrastructure-as-a-service or other similar online service that competes with Confluent products or services that provide the Software”. For most companies running a registry for their own applications, this does not bite. For a company that wants to sell a hosted event streaming service, it needs a lawyer’s reading. The project is controlled by one company.
- Apicurio Registry: Apache 2.0 for everything. The project is open source with Red Hat as the main sponsor, and Red Hat offers a supported build commercially.
- Karapace: Apache 2.0. The project is led by Aiven, which runs it as part of its managed Kafka service.
None of the three is under a neutral foundation such as the Apache Software Foundation or the CNCF. Look at release pace and who merges pull requests. All three shipped several releases in 2026.
Architecture and storage
- Confluent: stateless Java processes plus the _schemas Kafka topic as the source of truth. Backup means protecting that topic (replication factor, and a copy elsewhere). Kafka must be healthy for writes to work.
- Karapace: same model as Confluent: _schemas topic, leader election through a Kafka group, in-memory state rebuilt by reading the topic at startup. Because the built-in backup tool was removed in 6.0.0, you need your own way to copy the topic or export schemas.
- Apicurio: application nodes are stateless; state lives in PostgreSQL or in Kafka. With PostgreSQL, you use your normal database backups, replicas and point-in-time recovery. Apicurio also has an export and import API that writes a zip file.
- Redpanda: the registry is inside the broker; state is in the _schemas topic.
A practical point: if your team already runs PostgreSQL well, Apicurio on PostgreSQL gives a registry that backs up and restores like any other database. If your team trusts Kafka more than its databases, the topic-based registries feel more natural.
Formats and compatibility rules
All three support Avro, Protobuf and JSON Schema, schema references, and the familiar compatibility levels: BACKWARD (the default in Confluent and Karapace), FORWARD, FULL, their transitive variants, and NONE.
- Confluent: compatibility can be set globally and per subject, and in recent versions per context. Normalisation on register and lookup is available. The 8.4 release adds a LOGICAL compatibility policy alongside STRICT and LENIENT; the release notes say it conflicts with FORWARD and FORWARD_TRANSITIVE, and advise checking which subjects inherit the global policy before setting it globally. Data contracts add metadata, tags and rules (CEL based) that run in the client serdes.
- Apicurio: rules are VALIDITY (is the content well formed), COMPATIBILITY (same level names) and INTEGRITY (are references valid), at global, group or artefact level. Through the Confluent compatible API, Confluent style config calls map onto these rules. Its 3.3.0 data contract features are its own design, based on the Open Data Contract Standard; its documentation says Confluent’s data contracts, schema linking and client-side encryption are not supported through the compatibility API.
- Karapace: global and per-subject compatibility through the same API calls as Confluent, including the transitive modes. Avro references arrived in 6.1.0. Its README notes that normalisation may not always match Confluent’s.
Whatever the product, the compatibility checker is implemented separately in each one. A schema change that passes in one registry is very likely, but not guaranteed, to pass in another. If you migrate, re-run your schema checks against the new registry before switching.
Client and API compatibility
This is where most migration surprises live.
- Wire format. Confluent serdes write a magic byte of 0 and a four-byte schema ID before the payload. Karapace and Redpanda use the same IDs and format, so Confluent serdes work unchanged against them.
- Apicurio’s Confluent API. Point Confluent clients at /apis/ccompat/v7 or /apis/ccompat/v8 instead of the root URL. The 3.2.0 release filled many gaps in the v7 API (mode, exporters, pagination, normalisation), and 3.3.0 fixed more differences found by a compatibility test harness. Test your exact client versions and the calls they make.
- Apicurio’s own serdes. These are separate libraries with their own settings. In 3.x the default is a four-byte ID that carries the contentId, which is compatible with the Confluent format. But there is also a legacy eight-byte ID handler, a globalId option, and an option to put the ID in Kafka headers instead of the payload. Older Apicurio 1.x and 2.x setups often used the eight-byte globalId. Mixing producers and consumers with different settings gives deserialisation errors or, worse, wrong schemas. Pick one serde family per topic and write down its settings.
- IDs during migration. Messages already in topics carry IDs. A new registry must give the same schema the same ID. Confluent and Karapace (from 6.2.4) support an IMPORT mode for this, and Apicurio added the Confluent mode API in 3.2.0. Karapace’s README says IMPORT mode skips compatibility checks and content deduplication, and that a failed import is resumed by replaying it, not rolled back.
- REST Proxy. Karapace also includes a Kafka REST Proxy. If you use Confluent REST Proxy too, Karapace can replace both.
Security and auth
- Confluent: TLS, mutual TLS and HTTP basic authentication through Jetty are available in the community build, and the registry can talk to a secured Kafka with TLS or SASL. Authorisation of registry operations (who may write which subject) comes from the Schema Registry Security Plugin, RBAC or ACL authorisers, and the documentation says these security plugins require a Confluent Enterprise licence. So on the free build, authentication is easy but fine-grained authorisation is not.
- Apicurio: OIDC authentication (documented with Keycloak), role-based access with admin, developer and read-only roles, and owner-based authorisation for artefacts, all in the open source build. Basic authentication through client credentials is also supported.
- Karapace: HTTP basic authentication and Read or Write rules matched by regular expression on subjects, through a registry_authfile, plus OIDC bearer token validation through a JWKS endpoint, with optional role-based authorisation from token claims (split into separate authentication and authorisation settings in 6.2.0). The REST Proxy can forward the caller’s bearer token to the registry. Remember the 5.0.x bypass above, and test that missing tokens are rejected.
- Redpanda: registry authentication and authorisation are configured on the brokers. We did not review them in detail for this post; check the Redpanda documentation for your version and which features need an enterprise licence.
For Vadodara, this is the deciding factor more than anything else: the security team’s request for per-team write rights through Keycloak is possible in all three, but on Confluent it needs a paid licence.
Operations and upgrades
- Confluent: upgrades follow Confluent Platform versions, with a published support window (two years of Community support from the minor release date). You can run the registry from a newer Confluent Platform release against your existing Kafka, but read the compatibility tables. The 8.4 release also upgraded the bundled JMX Prometheus exporter to 1.6.0, which changed the metrics path and some metric names, so dashboards may need changes.
- Apicurio: upgrades are mostly a new application image against the same PostgreSQL or Kafka storage, with an export taken first. Read release notes for breaking changes: 3.2.0 moved health and metrics endpoints to a management port, 9000, which breaks probes and ServiceMonitors that still point at the main port. It also dropped the native (GraalVM) server build. The operator now supports horizontal pod autoscaling.
- Karapace: a Python service with a small configuration file or environment variables prefixed with KARAPACE_. Upgrades are frequent and usually small, but 6.0.0 removed the backup tool, and its README notes that the default log level is DEBUG, which you will want to change in production. A config flag, kafka_schema_reader_strict_mode, decides whether invalid records in _schemas stop the service or are skipped.
- High availability: Confluent and Karapace run several instances with one leader for writes; reads work on all. If the leader is lost, writes pause until a new one is elected; reads continue. Apicurio runs several stateless replicas behind a load balancer, and availability depends on PostgreSQL (or Kafka). Redpanda accepts writes on every broker.
- Caching: serdes cache schemas by ID, so a short registry outage usually does not stop running producers and consumers that already know their schemas. A new deployment or a new schema version during an outage will fail. Design for that rather than assuming the registry is never down.
Performance
We ran no benchmarks and quote no numbers. Registry load is usually small: clients fetch a schema once and cache it. The cases where performance matters are startup storms (hundreds of pods starting together and all fetching schemas), very large numbers of subjects, and Avro serialisation in a REST proxy. Karapace’s README has a latency comparison with Confluent, but it measures the REST Proxy, not the registry, and it is the project’s own run. Apicurio’s 3.2.0 release fixed a caching bug where its v3 API serdes called the registry for every record, which is a good reason to test cache behaviour on the version you deploy.
A worked plan for Vadodara
Step 1. Make an inventory. Export the list of subjects, their versions, formats, compatibility settings and the schema IDs in use. Map each subject to its topics, producers and consumers. In Vadodara, this shows about 300 subjects, 40 of them with no traffic in the last six months.
Step 2. Fix the rules before the product. Set BACKWARD_TRANSITIVE on topics that consumers replay from the start, such as the Debezium topics feeding the data warehouse. Set FULL_TRANSITIVE on the few topics shared with partners. For Protobuf, add a review rule: removed fields must be marked reserved, by number and by name. Add a CI step that calls the registry’s compatibility endpoint for every changed schema before merge. This step alone would have stopped the August incident.
Step 3. Turn on authentication now. Even on the old registry, put HTTP basic authentication and TLS in front, and stop anonymous deletes. This buys time.
Step 4. Read the licence for the hosted product. Give the Confluent Community License text, especially the Excluded Purpose clause, to the company’s legal adviser together with a clear description of the partner service. This is not a decision for engineers to make alone.
Step 5. Choose a path.
- Path A: upgrade Confluent Schema Registry to the 8.4 line. Lowest risk for clients. Use the Confluent Community build, keep the _schemas topic, and run two or three instances. Accept that per-team write rights need either a Confluent Enterprise licence or a gateway in front of the registry.
- Path B (chosen for trial): move to Karapace. Same storage model and wire format, Apache 2.0, OIDC with Keycloak and per-subject authorisation in the open source build. Run it on a copy of the _schemas topic first. Its README claims server-side drop-in; we did not test reading an existing Confluent _schemas topic, so treat this as something to prove on a copy.
- Path C: move to Apicurio Registry. Choose this if the team also wants one place for OpenAPI and AsyncAPI documents, prefers PostgreSQL for storage, or wants the richer UI and governance features. Clients must point at /apis/ccompat/v7 or v8, and the migration needs an ID-preserving import.
The Vadodara team does Path A immediately to get back onto a supported release, and time-boxes a Path B trial for one quarter.
Step 6. Trial with mirrored data. Copy _schemas to a test Kafka cluster. Start the candidate registry on it. Compare subject lists, versions, IDs and compatibility settings with the source registry, using a script that calls both APIs. Then run the CI compatibility checks for the last 50 real schema changes against both, and compare answers.
Step 7. Move clients gradually. Change the registry URL in one low-risk consumer, then one producer, then the rest. Keep the old registry read-only during the move. Watch deserialisation error counts.
Step 8. Test failure and restore. Stop the leader instance and confirm writes resume. Restore the registry state from backup into an empty cluster and confirm IDs match. Time both.
When each is better
- Confluent Schema Registry is better when you already use Confluent clients and features such as data contracts, client-side encryption or schema linking, when you may buy Confluent support, or when you want the reference behaviour that every client is tested against.
- Apicurio Registry is better when you want an Apache 2.0 registry with OIDC and role-based access in the free build, PostgreSQL storage, a web UI, and one place for event schemas and API documents, especially in Red Hat or Strimzi based environments.
- Karapace is better when you want an Apache 2.0 replacement that behaves like Confluent on the wire and in storage, with minimal change to clients, plus a REST Proxy in the same project.
- Redpanda’s built-in registry is better when you already run Redpanda and do not need rule sets or advanced governance.
When not to change
- If your registry is on a supported release, has authentication, and your compatibility rules are checked in CI, the product is probably not your problem.
- If your incidents came from wrong compatibility modes or careless Protobuf changes, a new registry will not fix them. Fix the rules.
- If you depend on Confluent-only features such as client-side field encryption, schema linking or rule sets, moving away means rebuilding those features or living without them.
- If nobody has time to test ID-preserving migration properly, do not migrate. A wrong schema ID is worse than an old registry.
A short history
- Confluent Schema Registry appeared alongside the first Confluent Platform releases; its repository dates from December 2014. It supported only Avro for years, and added Protobuf and JSON Schema in Confluent Platform 5.5 in 2020. In December 2018 Confluent moved several components, including the registry server, from Apache 2.0 to the Confluent Community License, while keeping client libraries under Apache 2.0.
- Karapace was started by Aiven in early 2019, around the time of that licence change, as an Apache 2.0 implementation of the same API. It later moved to the Aiven-Open GitHub organisation, and in 2026 went through a 6.x major version that removed backups and added references, OIDC authorisation and import mode.
- Apicurio Registry grew out of Red Hat’s Apicurio API design tools in 2019. Version 1.0.0 was tagged in October 2019, 2.0.0 in April 2021 and 3.0.0 in September 2024. The 3.x line changed the core API to v3 with groups and versions as first-class objects, and the 2026 releases added data contracts and AI artefact types.
- Redpanda chose to build the registry into the broker itself, so that teams moving from Kafka would not need a separate registry service.
Decision guide
- Do you run Redpanda and need only basic schema storage and checks? Use its built-in registry.
- Do you use or plan to buy Confluent features such as data contracts, encryption or schema linking? Stay on Confluent Schema Registry and keep it on a supported line.
- Do you need fine-grained authorisation without a commercial licence, and want minimal client change? Trial Karapace.
- Do you want PostgreSQL storage, a web UI, OIDC roles and API documents in the same registry? Trial Apicurio Registry with the ccompat API.
- Do you plan to offer a hosted event streaming service to customers? Get the Confluent Community License reviewed before you build on it, and consider an Apache 2.0 registry.
Checklist
- Every important topic has a chosen format, compatibility mode and subject naming strategy, written down.
- Topics that are replayed from the start use a transitive compatibility mode.
- Protobuf reviews check that removed field numbers and names are reserved.
- CI checks every schema change against the registry before merge.
- The registry has TLS and authentication; anonymous writes and deletes are impossible.
- Only owning teams can change compatibility or delete subjects.
- The registry is on a supported release; Confluent users know their Community support end date.
- The _schemas topic (or Apicurio database) has a tested backup and restore that keeps schema IDs.
- All producers and consumers of a topic use the same serde family and ID settings.
- Health probes and dashboards match the current release’s endpoints and metric names.
- At least two registry instances run in production, and leader failover has been tested.
Common mistakes
- Leaving every subject on BACKWARD by habit. It checks only against the latest version. A consumer replaying old data can still break.
- Reusing a Protobuf field number. Old data is then read with the new meaning, silently.
- Letting producers auto-register schemas in production. Auto-registration is handy in development, but in production it lets any service create versions. Register through CI and set auto.register.schemas to false in producers.
- Running Apicurio on the default H2 storage. It is in memory, and all schemas disappear on restart.
- Mixing Apicurio and Confluent serdes on one topic without matching ID settings. Eight-byte versus four-byte IDs, globalId versus contentId, and payload versus headers must all match.
- Migrating without keeping IDs. Old messages then point to missing or wrong schemas.
- Assuming auth works because it is configured. Test that requests with no token and a bad token are rejected.
- Hard deleting subjects to clean up. Soft delete first, wait, and check that no consumer still needs the schema.
The drill we ran
We wanted to see Avro evolution rules in action without running any registry or Kafka on a shared machine. So we did a small drill as an ordinary user, in a scratch folder, with no sudo, no system packages, no services and no network listeners.
- A Python virtual environment with fastavro 1.13.1 on Python 3.13.5.
- An invented Dispatch record with shipment_id, weight_kg and temp_c, written with version 1 and read back with several reader schemas, using Avro schema resolution.
- Results. Adding an optional field with a null default worked. Adding a required field without a default failed with “No default value for field”. Dropping a field in the reader worked. An old reader that expected temp_c could not read data written without it, because temp_c had no default; this is exactly what FORWARD checks catch. Changing weight_kg from double to string failed with a schema mismatch. Renaming a field with an alias worked. An old reader ignored an extra optional field written by a newer writer.
- Wire format check. We built a message with a magic byte of 0, a four-byte schema ID of 42 and the Avro payload, and read the ID back correctly. Reading a message framed with an eight-byte ID using the four-byte rule gave ID 0 and a payload offset four bytes off. This is the kind of silent mismatch that mixed serde settings cause.
Not run: no Confluent Schema Registry, Apicurio Registry, Karapace or Redpanda; no Kafka; no compatibility REST calls; no Protobuf or JSON Schema checks; no authentication tests; no failover or migration test; no benchmarks. We tried to fetch Karapace 6.2.4 with pip and found no PyPI package, so we did not install it. fastavro’s schema resolution shows how Avro readers behave; each registry has its own compatibility checker, which may differ in edge cases. For the real test, run Steps 6 and 8 of the plan on a throwaway Kafka cluster.
Unlearn, re-learn
- Unlearn “the schema registry is just a small lookup service”. Re-learn that its IDs are inside every message, so it is part of your data.
- Unlearn “BACKWARD is safe because it is the default”. Re-learn to choose modes by who deploys first and who replays.
- Unlearn “Confluent Schema Registry is open source in the Apache sense”. Re-learn that the server is source-available under the Confluent Community License, while the clients are Apache 2.0.
- Unlearn “drop-in means no testing”. Re-learn to compare IDs, versions and compatibility answers on mirrored data.
- Unlearn “auth is on, so we are safe”. Re-learn to test the request that should fail.
- Unlearn “a registry outage stops Kafka”. Re-learn that cached clients keep working, and only new schemas and new deployments fail.
Sources
- Confluent Schema Registry: schema-registry repository and licence notes, Confluent Community License text, Confluent Platform 8.4 release notes, supported versions and interoperability, Schema Registry fundamentals, Schema Registry security overview, schema evolution and compatibility
- Apicurio Registry: Apicurio Registry releases, 3.3.0 release notes, 3.2.0 release notes, introduction to Apicurio Registry 3.3, Kafka client serdes configuration, security advisories, CVE-2026-12975, CVE-2026-12992, CVE-2026-12993
- Karapace: Karapace releases, Karapace README, 6.0.0 release notes, 6.2.4 release notes, GHSA-vq25-vcrw-gj53 (CVE-2025-61673), GHSA-rw4j-p3jg-4fxq (CVE-2026-29190)
- Redpanda: Redpanda Schema Registry overview, Redpanda releases
- Avro and drill: Apache Avro specification, schema resolution, fastavro
- Related posts on Revisit.Tech: Kafka and Redpanda, RabbitMQ, NATS and Kafka queues, change data capture in 2026, schema migrations in 2026, Keycloak and its alternatives
Releted Posts
RabbitMQ, NATS and Kafka queues: revisit your message queue before you add another broker
A health insurance company in Hyderabad processes claims through RabbitMQ: three nodes on 3.13 with classic mirrored queues, about 40 lakh messages a day, from claim documents to OCR, OCR results to fraud checks and approvals to payments.
Read moreKafka and Redpanda: revisit the choice before you treat them as the same bus
Many teams say “we run Kafka” when they mean “our services talk to a Kafka-compatible broker”. That shortcut was useful when one open source broker dominated the protocol.
Read moreConfiguration management in 2026: revisit Ansible, Salt and Puppet before you write another server setup script
Almost every team has one: a setup.sh or bootstrap.sh written for the first production server and grown every year since. It installs packages, edits configuration files with sed, creates users and starts services.
Read more