Spring Batch for Excel reports without timeouts

Batch Processing Excel Reports with Spring Batch: Architecture and Implementation

Spring Batch solves the problem of synchronously generating hundreds of Excel documents within a single HTTP request. The framework breaks the process into chunks, manages state, and ensures recovery from failures. In a real-world project automating aircraft engine repair, this allowed users to trigger batch printing of work cards and routing sheets without timeouts.

Core Components of Spring Batch

Job orchestrates the entire cycle: Excel generation, archiving, storage upload, and cleanup. Step is a processing stage, Tasklet is an atomic operation.

The process is divided into chunk-oriented steps (read/process/write in batches) and tasklet-based steps (single action). This ensures stage independence and centralized management.

Steps for Report Generation

Step PRINT: Creating Excel Files

Data is split into chunks of 10 items. Each passes through reader → processor → writer:

@Bean
@JobScope
public Step printTrpTagsStep(
        ItemReader<TrpTagForPrintModel> excelTrpTagItemReader,
        ItemProcessor<TrpTagForPrintModel, ExcelItem> excelTrpTagItemProcessor,
        ItemWriter<ExcelItem> excelTrpTagItemWriter
) {
    return new StepBuilder(BatchExcelPrintStep.PRINT.getCode(), jobRepository)

            .<TrpTagForPrintModel, ExcelItem>chunk(10)
                    .transactionManager(transactionManager)
                    .reader(excelTrpTagItemReader)
                    .processor(excelTrpTagItemProcessor)
                    .writer(excelTrpTagItemWriter)
                    .build();
}

Spring Batch tracks writeCount for progress. Files are saved to /tmp/engine/{timestamp}/zip/ with subfolders for document types.

Step ZIP: Creating the Archive

A Tasklet scans the directory and creates a ZIP using standard Java utilities.

Step UPLOAD: Uploading to S3

The archive is uploaded to S3, and the link is recorded in the ExecutionContext.

Step CLEAN_UP: Removing Temporary Data

A JobExecutionListener checks the status and cleans up /tmp on success or failure.

Concurrency and Scaling

Multiple Jobs are launched in a Thread Pool to handle concurrent user requests. Within a Job, parallel steps were not used, but the framework supports them.

Frontend Integration: Monitoring Progress

POST /trp-tag/print/excel/batch/start returns a jobExecutionId. GET /trp-tag/print/excel/batch/{id} returns the status.

Progress is calculated as follows:

• PRINT: proportional to writeCount (90% of the time).
• Remaining steps: 0% or 100%.

The frontend receives:

• progressPercentage for the progress bar.
• currentStepNumber/totalStepsNumber.
• status (STARTED/COMPLETED/FAILED).
• downloadUrl.
• failureDetails.
• durationMillis.

Implementation Benefits

• Asynchronous Processing: Instant response instead of timeouts.
• Persistence: History stored in PostgreSQL without additional infrastructure.
• Transactional Integrity: Automatic state management, unlike @Async.

Key Takeaways

• A chunk size of 10 minimizes memory usage and ensures progress tracking.
• WriteCount is a built-in counter for precise monitoring.
• ExecutionContext stores references to artifacts.
• JobExecutionListener guarantees cleanup.
• Thread Pool scales multiple Jobs concurrently.

Spring Batch is suitable for any batch document generation with archiving and storage. Implementation takes less time than custom logic with queues.

— Editorial Team