CDC Replication Slot Lag and Postgres Database Bloat

Replication slot lag in Postgres is a storage risk. A stalled slot can keep WAL with no limit until the disk fills and applications can no longer write.

How replication slots make WAL retention a storage commitment, not just a lag number

A replication slot is a durable promise by Postgres to keep WAL that a consumer may still need, even under storage pressure. That promise starts at a specific Log Sequence Number, or LSN, which identifies a position in the Write-Ahead Log. Postgres writes changes to this log before writing them to data files, and logical replication clients read changes from it. The slot therefore connects the consumer’s progress directly to the database’s ability to reclaim log storage.

Each slot tracks a restart_lsn, which establishes the oldest WAL position it still needs. The checkpointer can recycle segments only when they are older than every slot’s restart_lsn. Progress by most consumers does not override the needs of the one furthest behind. A disconnected consumer’s slot also retains this obligation until the slot advances or is removed.

Why a stalled slot causes unbounded WAL growth, independent of max_wal_size

max_wal_size does not cap slot-bound WAL retention. A stalled slot can force Postgres to retain WAL beyond that setting because the consumer’s retention requirement takes precedence. The relevant limit is max_slot_wal_keep_size, whose default is -1, meaning unlimited. Without an explicitly configured finite value, the slot has no retention ceiling. What looks like a growing lag number is an open-ended claim on disk.

max_slot_wal_keep_size has been configurable since PostgreSQL 13, but an upgraded cluster can carry the unlimited default forward. The presence of a familiar WAL setting is therefore not evidence that slot growth is bounded. Nor does a large disk settle the question, because the important relationship is between available space and the rate of WAL generation. On a busy cluster, the current WAL position can advance rapidly while a frozen restart_lsn holds the retention floor in place. The gap grows continuously for as long as that combination persists.

Partial-table replication as the hidden accelerant

Partial-table replication makes this relationship counterintuitive because a busier database can make a quiet slot more dangerous. A CDC task may capture only a defined set of tables, while Postgres generates WAL at the cluster level. Heavy writes to tables outside that set move the cluster’s WAL position forward but may leave the slot where it is. The consumer can read the stream without encountering matching records frequently enough to move its retained position.

This can happen when one task covers only a subset of tables and writes concentrate elsewhere. It can also happen with multiple CDC tasks, where a quieter task’s slot stalls while other tasks continue processing changes. The same mismatch can occur between databases on one cluster, because writes in a non-replicated database still move the cluster WAL forward. A slot tied to a quieter database can hold that growing log behind it. In each case, the replication scope is narrow but the storage obligation covers the intervening WAL.

Other slots may be progressing normally while one quiet slot determines how much WAL the cluster must retain. Monitoring only the visibly active pipelines can therefore miss the source of disk growth.

The second-order damage: VACUUM suppression and table bloat

WAL retention is not the only storage consequence of a stalled slot. A slot’s retained xmin, its transaction ID horizon, can prevent VACUUM from reclaiming dead tuples that remain visible from that horizon. Those row versions then remain in the heap while WAL accumulates separately. The database can therefore develop table bloat as well as a growing log directory. This widens the damage beyond the files that first attract attention.

Removing the stale slot releases the retention barrier, but it does not automatically remove the table bloat already created. A team that stops WAL growth before an outage may still inherit bloated high-write tables. Manual VACUUM or VACUUM FULL may be required afterward.

The complete failure path: from frozen slot to write rejection

The outage begins when a slot stops advancing while the cluster continues generating WAL. With unlimited slot retention, old segments remain required and new segments consume the remaining disk. Once storage fills completely, Postgres cannot complete checkpoints or write new WAL. Because database changes depend on WAL, the failure then becomes write rejection. Every application on the cluster is affected.

Azure Database for PostgreSQL Flexible Server goes read-only when the amount of storage in use hits a high threshold. At a separate, higher threshold, an unused logical slot may be dropped automatically. That forced release of retention still leaves the CDC consumer requiring full re-initialization. An automated storage response is not a substitute for preserving replication continuity.

AWS RDS and Aurora expose the buildup through OldestReplicationSlotLag and TransactionLogsDiskUsage in CloudWatch. The first measures WAL not yet consumed by the most lagging slot, while the second measures storage used by WAL. Both are instance-level metrics, so a single stalled slot can drive them upward even when every other slot is healthy.

Diagnosing a stalled slot before the disk fills

Start by distinguishing database growth from other disk growth. Compare the size reported by pg_database_size() with actual disk usage, and establish whether table sizes are changing. If tables stay the same size while disk use climbs, the added space may be taken up by WAL, logs, or temporary files. That comparison narrows the investigation but does not identify a slot on its own. The next step is to inspect the retention state directly.

Query pg_replication_slots for inactive slots and growing retention gaps. The expression pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) measures the distance from the current WAL position to the slot’s retained position. A query such as SELECT slot_name, active, restart_lsn, pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) FROM pg_replication_slots; exposes those signals together. Repeated results distinguish a large retained gap from one that is still expanding. Rows where active is false also identify slots without a streaming walsender.

Inactivity and lag require different interpretations. An inactive stale slot points toward removal, whereas an active slot with growing lag still has a consumer that is not acknowledging enough progress. Match pg_replication_slots.active_pid with pg_stat_replication.pid to find the attached consumer before treating the slot as abandoned. Check WAL status as well, because a status of lost means required WAL has already been removed and the slot is permanently unusable. Slot names often match hostnames or connector names, so deployment logs for deleted tasks or rebuilt replicas can help identify orphaned retention commitments.

Remediation options and their trade-offs

Dropping a stale slot is the most direct way to remove its retention barrier. The command is SELECT pg_drop_replication_slot('slot_name');. Eligible WAL segments are not removed instantly, because the next checkpoint evaluates them for removal. When disk is critically full, running CHECKPOINT manually accelerates that step. But any consumer that needs the dropped slot again must be re-initialized from a full snapshot.

A finite max_slot_wal_keep_size provides a preventive alternative on PostgreSQL 13 and later. If the distance between a slot’s restart_lsn and the current LSN exceeds the limit, Postgres can invalidate the slot and remove older WAL. This bounds the storage commitment by sacrificing the consumer’s ability to resume from that slot. It protects the disk, but the resulting unusable slot still requires re-initialization. A retention cap is therefore a choice about which failure to accept.

For partial-table replication, a heartbeat table addresses the reason the slot is not advancing. A small dedicated table within the CDC task’s replication scope receives periodic timestamp updates. The consumer reads those records and advances its confirmed LSN, allowing Postgres to discard older WAL safely. This supplies progress even when the application tables in scope are quiet. Unlike dropping or invalidating the slot, the pattern preserves continuity while addressing the mismatch between cluster activity and replication scope.

An orphaned slot has no useful consumer to preserve, while an active lagging slot calls for attention to the remote consumer. The full-snapshot cost of losing a slot is what makes that distinction operationally important.

What PostgreSQL 17 changes about slot durability across failovers and upgrades

In PostgreSQL 16 and earlier, pg_upgrade did not migrate logical replication slots, so a major upgrade lost them and required subscriber re-initialization. With PostgreSQL 17, upgrades can keep logical replication slots on publishers and full subscription state on subscribers. That capability applies only when upgrading from PostgreSQL 17 or later to a future version. The upgrade from PostgreSQL 16 to PostgreSQL 17 still follows the older behavior.

Failover handling also requires a distinction between community Postgres and a hosted service. Managed services implement slot durability features differently, so teams on RDS or Aurora should apply community PostgreSQL 17 failover slot procedures only after validating equivalent service behavior. Whether a slot survives an upgrade or a failover, the WAL it retains still needs to be managed.

Why pipeline lag is now a product reliability problem, not just a data freshness concern

The consumer side of this failure matters as much as the database side. AI agents, fraud detection systems, and customer-facing dashboards inherit delays in the data they depend on. When a pipeline freezes, features, predictions, and decisions can be based on state that is hours old. Systems that stream changes within sub-minute latency support fraud detection and recommendation uses that hours-old batch feeds cannot serve in the same way. Replication latency is therefore a product reliability criterion.

Delivering that reliably requires more than keeping a connector running. The work includes checking slot health, handling schema evolution, moving LSNs forward correctly, and recovering from consumer failures without starting from a full snapshot. Partial-table lag, retained transaction horizons, heartbeat management, failover eligibility, and retention limits all belong to that same responsibility.

Managed CDC services take on this work, but someone still has to be responsible for it. For a self-managed pipeline, it sits with the team operating the replication lifecycle. The practical implication is that slot health belongs within production reliability because a stalled pipeline can both stale the product’s data and exhaust the database that serves it.