Articles by tag: spark

Declarative Data Pipeline in Spark | Classes and Decorators

How to Build a Maintainable Data Pipeline Using Classes and Decorators in Python. Implementation on Spark for Data Engineering Projects. Read the Guide.

Iceberg Optimization: sorting and deletion vectors

Configuring write order, statistics, and vectorized deletion in Apache Iceberg. Speed up queries, reduce load on storage. For data engineers.

ETL Architecture: asapBI, Trino, Spark, Airflow for data engineers

How asapBI combines Trino, Spark, and Airflow into a single interface for ETL processes. Code autogeneration, orchestration, and Open Source solutions for data engineers.