OLTP core: API and adaptive tuning in DBMS

Building an OLTP Core: API Contracts, Page Size, and Adaptive Tuning

Developers of a new OLTP database implemented a strict architecture with isolated layers using trait interfaces in Rust. Each layer has clear API contracts, page size is configurable per tablespace, and configuration is managed by a Resource Broker without manual tuning. Diagnostics integrate via USDT and eBPF.

Page Size and Tablespace Settings

Data page sizes range from 4-8 KB for pure OLTP to 16-32 KB for HTAP on NVMe. Each tablespace is a separate file with a fixed page size after creation. Default is 16 KB. PageId is composite: [tablespace_id:16][page_index:48]. The BufferPool routes requests to sub-caches by size.

The superblock at the file start stores tablespace metadata independently of page size. On mount, it checks configuration; mismatches prevent startup without degradation.

Architectural Layers and API Contracts

The core is divided into four layers:

• Adapter Layer (async tokio): terminates TLS, parses wire protocol.

#[async_trait]
pub trait NetworkAdapter {
    async fn handle_connection(&self, stream: Box<dyn ConnectionStream>) -> Result<(), NetworkError>;
}

• CompatLayer: parses SQL to AST, emulates pg_catalog, translates to LogicalPlan, rejects unsupported features (error 0A000).

pub trait CompatLayer {
    fn translate_query(&self, ast: SqlAst) -> Result<LogicalPlan, CompatError>;
}

• Core Engine (sync): optimizer, executor, transaction manager. Runs in a spawn_blocking pool.

pub trait ExecutionEngine {
    fn execute_plan(&self, plan: LogicalPlan, session: &Session) -> Result<ResultSet, ExecutionError>;
}

• StorageManager: facade over PageProvider and TransactionLogSink. Core requests pages and UNDO records.

pub trait StorageManager {
    fn pin_page(&self, page_id: PageId, mode: LockMode) -> Result<PageGuard, StorageError>;
    fn append_undo(&self, txn_id: TxnId, record: UndoRecord) -> Result<UndoPtr, StorageError>;
}

Layer boundaries prevent detail leaks: lock metrics exclude async waits.

Adaptive Tuning and Resource Broker

Around 60 parameters are split into budgets, guardrails, and overrides. Operators set high-level limits:

[resources]
memory_budget = "16GB"
cpu_budget    = "auto"
io_iops       = 5000

Advisors (MemoryAdvisor, IoAdvisor, CpuAdvisor) redistribute resources every second:

• CpuAdvisor: DOP and thread pool vs throttling.
• MemoryAdvisor: BufferPool vs work_mem under load.
• IoAdvisor: transaction priority, burst awareness in the cloud.

Safeguards

• Hysteresis: 5s window, 5% steps to avoid oscillations.
• Hard Floors: min. 128 MB for BufferPool/UNDO.
• Graceful Transition: new limits for new allocations.
• Expert overrides override autotuning.

Background UNDO log purge instead of VACUUM. Runtime changes without restart.

Diagnostics: USDT Probes and eBPF

Diagnostic mechanisms are baked into the binary. USDT probes for runtime analysis, eBPF for tracing with no overhead.

Key Takeaways:

• Page size per-tablespace with startup checks.
• Strict trait contracts between layers prevent leaks.
• Resource Broker: budgets → autotuning with guardrails.
• Diagnostics via eBPF/USDT in the core.
• Ditch manual tuning for adaptive mechanisms.

— Editorial Team