Loading TOML config with Pydantic in Python

Reading and Validating settings.toml with Pydantic in Python Apps

This module handles reading settings.toml, parsing it with tomllib, caching the result via lru_cache, and validating sections using Pydantic. It keeps config logic separate from your app parameters, cuts down on repeated disk reads, and ensures type safety.

from functools import lru_cache
from pathlib import Path
from tomllib import load
from typing import Any, Type, TypeVar

from pydantic import BaseModel, SecretStr

ConfigType = TypeVar("ConfigType", bound=BaseModel)

class BotConfig(BaseModel):
    token: SecretStr

@lru_cache
def parse_config_file() -> dict[str, Any]:
    config_path = Path(__file__).resolve().parent.parent.joinpath("settings.toml")

    if not config_path.exists():
        error = "Could not find settings file"
        raise ValueError(error)

    with open(config_path, "rb") as file:
        config_data = load(file)

    return config_data

def get_config(model: Type[ConfigType], root_key: str) -> ConfigType:
    config_dict = parse_config_file()

    if root_key not in config_dict:
        error = f"Key {root_key} not found"
        raise ValueError(error)

    return model.model_validate(config_dict[root_key])

Why TOML and Pydantic for Configuration

TOML files are human-readable and parse cleanly into Python dictionaries. Sections like [bot] turn into nested dicts. Pydantic validates the structure, handles type conversions, and masks secrets with SecretStr.

Example TOML:

[bot]
token = "123456:ABCDEF"

Parsing yields {"bot": {"token": "123456:ABCDEF"}}. Validation creates a BotConfig object.

Caching with @lru_cache skips repeat reads: the first call parses, later ones hit the cache. Reload with parse_config_file.cache_clear().

Handling Paths with pathlib

Path(__file__).resolve().parent.parent.joinpath("settings.toml") builds an absolute path to the file in your project's parent directory. exists() checks and parent properties simplify validation.

• Path.joinpath(): appends path segments cross-platform.
• resolve(): resolves symlinks to absolute paths.
• Open in "rb": required by tomllib.load() for binary reading.

Type Hints with TypeVar and Pydantic

ConfigType = TypeVar("ConfigType", bound=BaseModel) lets get_config work with any Pydantic model, returning an instance of the passed type.

• Type[ConfigType]: expects a model class, not an instance.
• model_validate(): turns a dict into a validated object, raising ValidationError on failures.
• Any for dicts: TOML mixes types (str, int, bool, dict).

Extend for other sections:

class DatabaseConfig(BaseModel):
    host: str
    port: int
    user: str

config = get_config(DatabaseConfig, "database")

Caching and Decorators

@lru_cache stores results by arguments (none here, so it's simple). LRU evicts old entries on limits, but one slot suffices for config.

Pros:

• Cuts I/O: disk read once.
• Idempotent: multiple get_config calls share one parse.
• Thread-safe: the stdlib decorator handles it.

Cons: Disk changes to TOML are ignored until cache_clear().

Module Structure

• Models (BotConfig): independent of loaders.
• parse_config_file(): parses full TOML.
• get_config(): extracts and validates a section.

This separation of concerns boosts testability: mock paths or TOML strings.

Key Tips:

• Use tomllib (Python 3.11+), fall back to tomli for older versions.
• Always validate config with Pydantic before use.
• Cache parsing, but add reload support.
• Store secrets in SecretStr to mask them in logs/debug.
• Build paths with pathlib for cross-platform compatibility.

— Editorial Team