Database Technology

Netflix engineers deployed a metadata-driven dynamic partition splitting mechanism to resolve wide partition issues in time series workloads. The system automatically detects oversized partitions, divides them into smaller units, and routes read requests across these child partitions. This approach successfully lowered read latency to the millisecond range, reduced timeouts, and enhanced cluster stability without requiring application-level changes.

• Dynamic splitting addresses wide partition hotspots that cause second-level read latencies in time series data.
• Metadata-driven detection allows automatic partitioning without manual schema changes or application code updates.
• Read routing across child partitions ensures load distribution and improved cluster stability under heavy load.
• Timeouts are significantly reduced as the system prevents single-node bottlenecks from oversized partitions.

Michael Paquier committed a patch for PostgreSQL 20 that introduces per-backend lock statistics. This feature exposes lock wait counts, wait times, and fast-path exceeded counts for individual backends. The data mirrors existing information available through pg_stat_lock but allows for more granular monitoring at the session level.

• PostgreSQL 20 gains per-backend visibility into lock contention metrics.
• Monitor lock wait counts and durations to pinpoint specific blocked sessions.
• Track fast-path lock acquisition failures to identify performance bottlenecks.
• No immediate action required for current releases; plan for PG 20 upgrade.

The presence of .ready files in pg_wal/archive_status indicates that WAL segments have been generated and are available for transport via streaming replication or archive_command. These files accumulate before being marked as .done once the receiving side confirms consumption. A growing pile of .ready files is a strong signal of stalled WAL delivery rather than a broken replica, often pointing to network or consumer latency issues.

• .ready files mean WAL is generated and waiting for transport, not necessarily that replication is broken.
• Consumption by streaming replicas or archive_command moves files to .done status.
• Accumulating .ready files usually indicate slow transport or a stalled consumer.
• Check network latency and replica replay speed when disk usage climbs due to .ready files.

Alibaba Group has open-sourced zvec, a lightweight vector database designed to run in-process within applications. The latest v0.5.0 release introduces native full-text search and hybrid retrieval capabilities, allowing combined queries across dense vectors, sparse vectors, and scalar filters without external search engines. It utilizes a DiskANN index for production-grade similarity search.

• Embeds directly into apps, eliminating external vector DB infrastructure overhead.
• Native full-text search removes dependency on separate search engines like Elasticsearch.
• Supports hybrid retrieval combining vector, sparse, and scalar filters in one query.
• Uses DiskANN indexing for low-latency, scalable similarity search.