Handling Postgres Schema Changes Without Pipeline Downtime

Handling Postgres schema changes without pipeline downtime requires automatic schema evolution for safe changes and explicit review for unsafe ones. Postgres logical replication streams only DML, so a CDC pipeline must detect schema changes, update compatible targets, and preserve delivery guarantees across the transition. The governing rule is to automate nullable column additions while treating drops, renames, and type changes as coordinated migrations.

Why Postgres logical replication does not stream DDL

Postgres logical decoding turns row-level changes in the WAL into events that a CDC consumer can process. DDL statements such as ALTER TABLE do not produce equivalent logical replication messages. This gap is a deliberate property of the protocol. A stream that carries changes to rows is not also a stream of instructions for changing destination tables.

That distinction separates data replication from schema coordination. A source table can change while the destination retains its previous definition, even when replication is otherwise working correctly. Postgres sends table definitions through Relation messages only when data activity occurs. Operations therefore depend on when schema information becomes available through logical replication and what the consumer does with it.

What the WAL does carry — Relation messages and when they arrive

A Relation message describes the current schema of a table to the CDC consumer. It precedes the first row event for that table in a given session and is sent again when Postgres detects that the previously transmitted schema no longer matches the current definition. But that signal comes only when the data changes.

For example, adding a column to a table that receives no writes for hours leaves the consumer with a stale schema throughout that interval. If no subsequent write occurs, the stream never tells the consumer about the change. This is delayed, conditional signaling rather than total silence. It is correct behavior for a protocol built around DML, but it does not provide proactive DDL awareness.

A CDC platform that detects source schema changes proactively removes this waiting window rather than relying solely on the next Relation message. Artie handles automatic schema evolution in its managed CDC service so teams do not have to manage the timing of Relation messages by hand. Neither, by itself, establishes whether applying the change downstream is safe.

How each type of schema change fails differently downstream

Column additions are usually the most recoverable changes. Log-based connectors emit the new column on subsequent events. The destination must accept it without breaking, typically by making the target column nullable. Trigger-based CDC has a different dependency because the trigger itself must be updated. Missing that update silently omits the new column from the changelog. Additions are therefore the safest class, but only when capture and destination handling both accommodate the new field.

Column removals introduce a stricter sequencing requirement. Once the source drops a column, the log stops emitting it, while a consumer that still expects the field may receive null or report an error. The outcome depends on how strictly the consumer validates incoming events. The recommended order reverses the usual source-first assumption by removing the column from the consumer before removing it from the source. That order prevents the destination from demanding a field the stream can no longer supply.

Renames are harder because a CDC consumer sees a drop followed by an add. That shape is indistinguishable from removing an unrelated field and adding a new one. Most consumers therefore treat the old column as lost and the new column as empty, without carrying historical values forward. The mitigation is a staged migration that adds the new column, dual-writes during a migration window, and removes the old column after all consumers switch. This makes renames the highest-friction schema change in a CDC pipeline.

Type changes can cause the most dangerous failures because a pipeline can keep running while its results become wrong. Typed consumers, including Parquet files, Avro topics, and warehouse table definitions, may reject new events or silently coerce their values. Rejection produces a visible interruption, but coercion can corrupt aggregations while row counts still look correct.

The two other structural traps that compound schema-change failures

A replication slot holds WAL from its last confirmed LSN, so a stopped consumer allows WAL to accumulate on the source disk. If that accumulation exhausts disk space, PostgreSQL can stop accepting application writes as well as CDC traffic. Setting max_slot_wal_keep_size caps retention at the cost of possible slot invalidation and a forced re-snapshot. In one documented case, recovery took almost 10 hours and used a lot of I/O before analytics could run again. Checking pg_replication_slots for active = false and the size of retained WAL shows when the source is at risk during migration windows.

Postgres logical replication publishes changes under partition names rather than the root table name unless configured otherwise. So a connector watching public.orders can receive nothing when writes go to orders_2026_q2. The publication parameter publish_via_partition_root, available since Postgres 13, addresses this routing mismatch. A migration that adds partitions must account for publication routing as well as column definitions, because correct schema handling cannot recover events the connector never receives.

How AWS DMS handles — and mishandles — DDL events during CDC

AWS DMS illustrates how a production tool works around the DDL gap through supported automation, event triggers, and documented constraints. It is an appropriate choice for AWS-native targets and migration workloads, although its documented CDC uses also include disaster recovery, cross-region replication, and analytics. AWS requires at least one endpoint to be on an AWS service. Its suitability therefore depends on the endpoint architecture and the schema operations involved.

AWS DMS 3.5.0 fixed a pg_logical column misalignment issue for a PostgreSQL source and S3 target when LOB support was disabled and LOBs were present. The release note covered dropped columns during CDC, which shows why teams need to check how the tool handles removals. Separately, AWS DMS 3.6.1 added PostgreSQL read replicas as CDC source endpoints for PostgreSQL version 16.x and later. That capability reduces replication load on the primary, but it does not replace schema-change coordination.

AWS documents another constraint when the source database is also a target for a third-party replication system. That setup can keep the awsdms_intercept_ddl event trigger from firing, so DDL changes may not migrate during CDC. The documented workaround alters the trigger to enable always. Each DMS task also creates a source replication slot, making slot count and retention part of the operating cost of task restarts during schema-change windows. These are predictable design constraints to include in a deployment plan.

The practitioner debate — auto-evolve additive changes versus halt on all DDL

The strongest argument for halting on DDL is that visible interruption is preferable to silent corruption. A consumer that uses a mismatched schema may produce wrong data without a clear sign of failure. From this perspective, production schema changes are coordinated events, and stopping replication forces the required review. The policy favors correctness over continuity when the system cannot establish that a change is compatible.

The strongest argument for automatic evolution is that not every change creates that uncertainty. Column additions are the most common schema change, and a new nullable column can propagate safely when the destination supports it. The tool can issue ALTER TABLE … ADD COLUMN, record the field in its schema history, and carry subsequent data forward without an operator-managed interruption. Halting for every such addition creates unnecessary work and can train operators to treat alerts as noise.

The practical compromise is to automate safe additions and require explicit review for drops, renames, and type changes. This is the operating principle behind Artie’s approach, rather than a promise to propagate every DDL operation without interruption. The objection to unrestricted auto-evolution remains valid because table rewrites and transactional integrity can complicate apparently simple changes.

The zero-downtime migration pattern that relies on the DDL gap being predictable

Logical replication also provides a zero-downtime Postgres migration pattern precisely because schema setup and row replication are separate. First, stand up the target instance and apply the schema with pg_dump --schema-only. Next, create a publication on the source for every table and a subscription on the target. The first COPY does the bulk load, and replication then carries the row changes. Wait for lag to approach zero, then cut over.

During that migration window, apply any schema changes manually to both sides in the correct order. The logical stream will keep carrying DML, but it will not carry those DDL statements. The target’s readiness therefore depends on coordinated definitions as well as replication lag.

Postgres 17 added related reliability improvements, but their scopes differ. Slots marked failover = true synchronize to physical standbys, allowing physical failover without invalidating logical subscriptions. Postgres 17 also extended pg_upgrade to migrate valid logical slots and subscriptions. That upgrade benefit requires the old cluster to be version 17 or later, so it is not a shortcut for carrying logical replication state forward from every older version. Neither capability removes the need to coordinate DDL during a logical migration.

How do you handle upstream schema changes without stopping your CDC pipeline?

The first requirement is current schema knowledge. A CDC tool fit for production must update its internal schema registry when replication starts and on every Relation message after that. Otherwise, the stream can supply the correct definition while the consumer continues using an obsolete one. Proactive detection addresses the waiting period before the next data event, while Relation-message handling keeps decoding aligned with the definitions supplied by Postgres.

The next requirement is selective target evolution. A new nullable column should reach the downstream warehouse through a destination schema update and a matching schema-history change, without an operator having to step in. By contrast, a destructive change should halt replication with an actionable error naming the table, column, and change. A generic replication failure does not provide enough information to coordinate recovery. Automatic schema evolution is useful only when its safety boundary is explicit.

Delivery guarantees must hold across that boundary. Checkpointing must wait at a schema transition until the destination write is confirmed, because at-least-once delivery through a type change can silently corrupt data without idempotent writes. In a managed service such as Artie, preventing risky schema-change patterns and providing exactly-once delivery reduces the burden of coordinating source and destination transitions manually. Artie also provides automatic target schema updates and schema history tracking as part of managed replication.

Built-in slot lag and WAL retention metrics make pauses visible before they threaten source availability, while correct partition routing prevents missing changes during partition rollover. Together, these requirements define a practical evaluation standard for live CDC pipelines.