CDC Schema Evolution at Zero Downtime: A Practical Playbook

Schema changes are a fact of life. Columns get added, types get widened, fields get renamed, and tables get restructured. When you have an active CDC pipeline streaming changes from that table, every ALTER TABLE becomes a coordination problem between the source database, the streaming layer, and every downstream consumer.

This article is a practical playbook for handling schema evolution in CDC pipelines without downtime, data loss, or broken consumers. It covers what happens at each layer when schema changes occur and which changes are safe, risky, or dangerous.

The Schema Change Lifecycle in CDC

When you run ALTER TABLE orders ADD COLUMN discount DECIMAL(10,2), here is what happens across the pipeline:

1. Source database applies the DDL. New rows include the discount column. The WAL/binlog now emits events with the new column.

2. CDC engine detects the schema change. Most engines (including Streamkap’s CDC engine) read the table schema from the database and update their internal mapping. Events before the change have N columns; events after have N+1.

3. Kafka topic now receives messages with two different schemas — the old one and the new one. If you use a schema registry, the new schema is registered and checked for compatibility.

4. Downstream consumers receive events in the new schema. If the consumer code expects exactly N columns, it may break on the N+1 column event. If it is schema-aware (reads the schema from the registry or uses a flexible format like JSON), it adapts.

The goal is to make every step in this chain handle the transition without intervention.

Classifying Schema Changes by Risk

Not all schema changes are equal. Here is how common DDL operations affect CDC pipelines:

Safe Changes (Low Risk)

Change	Why It’s Safe
Add nullable column	Old events have null, new events have values. Backward compatible.
Add column with default	Same as nullable — old events get the default on the consumer side.
Widen numeric type (INT → BIGINT)	New type is a superset. Old values still valid.
Widen string type (VARCHAR(50) → VARCHAR(200))	Same — superset.
Add a new table to CDC	No impact on existing streams.

Risky Changes (Medium Risk)

Change	Why It’s Risky
Add NOT NULL column without default	May require table rewrite. Existing events don’t have the column.
Change column type to something wider but semantically different (INT → DECIMAL)	Depends on consumer parsing.
Add index (on source)	Safe for CDC data, but can slow the ALTER and affect source performance.

Dangerous Changes (High Risk)

Change	Why It’s Dangerous
Drop column	Consumers expecting the column will fail. Schema registry rejects it under BACKWARD compatibility.
Rename column	Appears as drop + add. Breaks column-name-based consumers.
Narrow type (BIGINT → INT)	Values may not fit in the new type. Schema registry rejects under BACKWARD.
Change primary key	CDC uses PK for row identity. Changing it can cause duplicate or missing rows downstream.
Drop table and recreate	Pipeline loses its position. Must re-snapshot.

Schema Registry Compatibility Modes

If your pipeline uses a schema registry (Confluent Schema Registry, AWS Glue, or Apicurio), compatibility modes determine which schema changes are allowed.

How Compatibility Works

When a new schema version is registered, the registry checks it against previous versions:

Mode	Rule	Safe Operations	Blocked Operations
BACKWARD	New schema can read old data	Add optional fields, widen types	Drop fields, rename fields, narrow types
FORWARD	Old schema can read new data	Drop optional fields, add fields with defaults	Add required fields without defaults
FULL	Both directions	Add/remove optional fields with defaults	Most other changes
NONE	No checks	Everything	Nothing (no safety net)

Recommended Setting

Use BACKWARD compatibility for CDC topics. This is the most practical mode because:

• Adding columns is the most common schema change, and it is always allowed.
• Consumers are always reading data from the past (older schema), so backward compatibility is the natural direction.
• It blocks the dangerous changes (drops, renames) that break consumers.

# Set compatibility mode for a specific subject
curl -X PUT \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  -d '{"compatibility": "BACKWARD"}' \
  http://localhost:8081/config/cdc.myapp.orders-value

Testing Compatibility Before Deploying

Before running the ALTER TABLE, test the new schema against the registry:

# Test if a new Avro schema is compatible
curl -X POST \
  -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  -d '{
    "schema": "{\"type\":\"record\",\"name\":\"orders\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"customer_id\",\"type\":\"int\"},{\"name\":\"total\",\"type\":\"double\"},{\"name\":\"status\",\"type\":\"string\"},{\"name\":\"discount\",\"type\":[\"null\",\"double\"],\"default\":null}]}"
  }' \
  http://localhost:8081/compatibility/subjects/cdc.myapp.orders-value/versions/latest

Response: {"is_compatible": true}

If it returns false, the schema change will break your pipeline. Fix the schema before touching the database.

The Playbook: Safe Schema Changes Step by Step

Adding a Column (The Easy Case)

This is the most common change and the simplest to handle.

Step 1: Verify the change is backward compatible.

The new column must be nullable or have a default. In Avro, this means a union type with null: ["null", "string"].

Step 2: Run the ALTER TABLE.

ALTER TABLE orders ADD COLUMN discount DECIMAL(10,2) DEFAULT 0.00;

Step 3: Verify the pipeline continues.

The CDC engine picks up the new column automatically. Check that events after the change include the new field. Events that were in flight during the change may or may not include it, depending on timing — this is normal and consumers should handle null gracefully.

Step 4: Update consumers (if needed).

If your consumer destructures events into a fixed struct, update the struct to include the new field. If it uses dynamic schema loading, it already handles it.

Dropping a Column (The Hard Case)

Dropping a column is the most disruptive change. Here is the safe multi-step approach:

Step 1: Stop consumers from depending on the column.

Update all downstream consumers to not read the column. Deploy the consumer change. Verify it works without the column.

Step 2: Stop writing to the column in the application.

If the source application writes to the column, update it to stop. This is optional but reduces confusion.

Step 3: Consider your schema registry mode.

With BACKWARD compatibility, dropping a column is blocked. You have two choices:

• Temporarily set NONE compatibility, drop the column, then set BACKWARD again (risky — other bad changes could sneak through).
• Leave the column in the database but stop populating it. This is often the safest long-term approach.

Step 4: If you must drop it, execute the ALTER TABLE.

ALTER TABLE orders DROP COLUMN discount;

The CDC engine detects the schema change. Events after this point no longer include the column. If your schema registry is set to NONE, the new schema registers. Consumers that you updated in step 1 continue working.

Step 5: Re-enable BACKWARD compatibility.

Renaming a Column

Column renames are the trickiest because CDC sees them as “column A disappeared, column B appeared.” The safe approach:

-- Step 1: Add the new column
ALTER TABLE orders ADD COLUMN total_amount DECIMAL(10,2);

-- Step 2: Backfill from old column
UPDATE orders SET total_amount = total WHERE total_amount IS NULL;

-- Step 3: Update application to write to BOTH columns
-- (deploy this code change)

-- Step 4: Update all consumers to read the new column name
-- (deploy this change)

-- Step 5: Stop writing to the old column
-- (deploy this code change)

-- Step 6: Drop the old column (when safe)
ALTER TABLE orders DROP COLUMN total;

Yes, this is six steps for a rename. That is the cost of zero-downtime schema evolution. If you are making many renames at once, consider if a single coordinated downtime window is more practical.

Type Changes

Safe Widening

Widening a type (INT → BIGINT, VARCHAR(50) → VARCHAR(200)) is generally safe:

-- Safe: widening integer type
ALTER TABLE orders ALTER COLUMN quantity TYPE BIGINT;

-- Safe: widening string type
ALTER TABLE orders ALTER COLUMN status TYPE VARCHAR(100);

The schema registry accepts these as backward compatible because the old type’s values fit in the new type.

Unsafe Narrowing

Narrowing a type (BIGINT → INT, VARCHAR(200) → VARCHAR(50)) is not safe. Existing values may not fit, and the schema registry rejects it under BACKWARD or FULL compatibility.

If you must narrow a type, follow the same multi-step pattern as a column rename: add a new column with the narrow type, migrate data, update consumers, drop the old column.

Type Class Changes

Changing the type class (e.g., STRING → INTEGER, or INTEGER → BOOLEAN) is almost never backward compatible. Treat it as a column rename: add a new column, migrate, and deprecate the old one.

Handling Schema Changes in Practice

Monitor Schema Versions

Keep track of schema versions over time. If you use Confluent Schema Registry:

# List all versions for a subject
curl http://localhost:8081/subjects/cdc.myapp.orders-value/versions

# Get a specific version
curl http://localhost:8081/subjects/cdc.myapp.orders-value/versions/3

Automate Compatibility Checks in CI

Add a step to your CI/CD pipeline that tests migration scripts against the schema registry before they run in production. This catches incompatible changes before they reach the database.

For more on managing schema changes across your data stack, see schema change management and schema drift detection.

The Friday Deploy Question

Should you deploy schema changes on Friday?

If the change is a safe addition (nullable column, type widening), yes — these should be boring, non-events with a well-tested pipeline.

If the change is risky (drop, rename, type change), schedule it for a time when the team is available to monitor and respond. That means Tuesday morning, not Friday at 5 PM.

How Managed CDC Handles Schema Evolution

Managed CDC platforms like Streamkap handle schema evolution automatically for safe changes. When you add a column, the platform detects it, updates the schema, and propagates the change to the destination — no manual intervention needed. The platform’s connectors are tested against the common schema change patterns, and the documentation specifies which changes are automatic and which require action.

The playbook in this article still applies: make changes backward compatible, avoid drops and renames when possible, and test compatibility before deploying. The difference is that with a managed platform, the middle of the pipeline (CDC engine, Kafka, schema management) is handled for you — you focus on the source DDL and the destination adaptation.

---

Ready to stop worrying about schema changes breaking your CDC pipeline? Streamkap automatically detects and propagates schema changes for supported ALTER TABLE operations, so your pipeline keeps running while your schema evolves. Start a free trial or learn more about schema change management.