OffNet Newsroom

Daily topic roundup

Database Technology

Thursday, October 01, 2026 · 8 stories, curated & summarized — click any story for the source.

Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data stored in S3, S3 Tables, or the AWS Glue Data Catalog. This feature allows operational and lake data to be accessed together through PostgreSQL foreign tables, eliminating the need for ETL pipelines and data duplication. Existing applications can leverage this capability without modification to access fresh lake data.

  • Eliminates ETL pipelines and data duplication for lake analytics
  • Query Iceberg/Parquet via PostgreSQL foreign tables in S3 or Glue
  • No application code changes required for standard PostgreSQL clients
  • Reduces engineering overhead as source schemas evolve over time
  • Enables real-time joins between operational and analytical datasets
CHECKLISTStreamline Lake AnalyticsQuery Iceberg and Parquet via foreign tablesEliminate ETL pipelines and data duplicationNo application code changes requiredJoin operational and analytical datasets in real time
AWS Database Blog awsdatabase

PostgreSQL CDC Lag: Fixing WAL Pileup with Heartbeat Tables

High source activity during Change Data Capture migrations can cause counterintuitive replication lag, leading to Write-Ahead Log accumulation. This AWS Database Blog post details a solution using heartbeat tables implemented via AWS DMS or Debezium to mitigate this issue. It also outlines methods for monitoring replication slot health using Amazon CloudWatch and SQL diagnostics.

  • WAL pileup often occurs during CDC even when the source database is under heavy load.
  • Heartbeat tables provide a mechanism to resolve replication lag in PostgreSQL CDC scenarios.
  • Implement heartbeat logic using either AWS DMS or the Debezium connector.
  • Monitor replication slot health proactively with CloudWatch metrics and SQL queries.

Chris van Eijk benchmarks the performance cost of Row-level Security in PostgreSQL 17, identifying specific architectural pitfalls. The analysis highlights how function volatility, SELECT wrappers, and membership subqueries can degrade query efficiency. A critical finding is that the leakproof rule can inadvertently disable index usage, impacting throughput.

  • Function volatility settings directly influence RLS execution overhead in PG17.
  • Using SELECT wrappers for RLS policies introduces measurable query latency.
  • Membership subqueries can become bottlenecks under high-concurrency loads.
  • The leakproof rule may disable index scans, leading to sequential scan penalties.
  • Review RLS policies for volatility and wrapper usage to optimize query plans.
CHECKLISTOptimize PostgreSQL 17 RLSAudit function volatility to reduce overheadAvoid SELECT wrappers for lower latencyMonitor membership subqueries under loadReview leakproof rules for index usageRegularly review RLS policies for optimization

This analysis measures Row Level Security behavior when PgBouncer operates in transaction pooling mode. It identifies specific scenarios where tenant context may leak between connections and explains how SET LOCAL behaves outside an active transaction block. Additionally, it highlights the risk of plans being shared across different tenant contexts, potentially impacting performance or security.

  • Transaction-mode PgBouncer can cause tenant context leakage if not managed carefully.
  • SET LOCAL has no effect outside a transaction block, leading to unexpected policy application.
  • Shared plans across tenants may mask performance differences or security boundaries.
  • Verify RLS policies are robust against connection reuse in pooled environments.
CHECKLISTSecuring PgBouncer RLSVerify RLS policies against connection reuseManage tenant context carefully in poolingExpect SET LOCAL to fail outside transactionsMonitor for shared plan security risks

AWS details a migration path for IBM Db2 on z/OS to Amazon Aurora PostgreSQL by deploying a gateway server on EC2. The process leverages AWS Database Migration Service (DMS) to handle full loads, periodic refreshes, and data replication. Validation steps are included to ensure data integrity during the transition from mainframe to cloud-native PostgreSQL.

  • Deploy a dedicated EC2 instance as a Db2 gateway to bridge z/OS and DMS.
  • Use DMS for both initial full load and ongoing change data capture.
  • Implement periodic full-load refreshes to handle large delta updates.
  • Validate data consistency between source Db2 and target Aurora PostgreSQL.
Planet PostgreSQL database

PostgreSQL 19 Adds Query Plan Hints: Implementation Details

PostgreSQL 19 introduces query plan hints, allowing explicit SQL instructions to override the default query planner's execution path. This feature ends years of community resistance to hints, which were previously avoided in favor of relying on accurate table statistics. While the planner generally makes optimal choices, hints provide a fallback for specific scenarios where plans unexpectedly degrade.

  • PostgreSQL 19+ supports explicit query plan hints for the first time.
  • Hints override the default planner, useful for sudden plan regressions.
  • Community previously resisted hints, preferring accurate statistics.
  • New capability gives DBAs an explicit tool for plan control.
TRADE-OFFPostgreSQL 19 Hint StrategyPrevious ApproachRelied on accurate table statisticsCommunity resisted explicit hintsPlanner made optimal choicesNew ApproachExplicit SQL instructions…Provides fallback for regressionsGives DBAs explicit controlvs

Amazon S3 Tables now natively supports all Apache Iceberg V3 data types, including geometry, geography, unknown, and nanosecond-precision timestamps. This eliminates the need to encode geospatial coordinates and high-precision event times as strings or integers, simplifying data architecture. The update enhances storage efficiency and query performance for workloads requiring precise location and time data.

  • Native support for geometry and geography types enables efficient filtering on location data.
  • Nanosecond timestamps improve precision for telemetry and financial event processing.
  • Column default values are now supported alongside existing Variant and deletion vectors.
  • Reduces architectural complexity by removing the need for custom encoding of V3 types.
CHECKLISTKey Iceberg V3 BenefitsNative geometry and geography supportNanosecond timestamp precisionColumn default values enabledSimpler data architecture

pgEdge introduced Starfleet, a cloud database platform designed to move agentic AI applications from prototype to production. The service offers a unified developer experience with built-in AI tooling and supports flexible deployments, including on-premises and multi-region clusters. This release addresses the industry-wide challenge where less than half of AI prototypes successfully reach production environments.

  • Starfleet bridges the gap between AI prototyping and production with native agentic AI tooling.
  • Supports zero-downtime, highly available multi-region clusters for global scalability.
  • Enables hybrid and on-premises deployments to satisfy strict data sovereignty requirements.
  • Aims to solve the 54% prototype failure rate cited in recent IDC and Lenovo research.
  • Provides a 100% Postgres-native cloud experience for enterprise AI workloads.
BY THE NUMBERSAI Prototype Failure Rate54%Prototypes failing to reach productionCited from recent IDC and Lenovo research
↑