Spring Batch 5: Chunk Processing, Restartability and Partitioning in Practice

Introduction

Batch processing is one of those engineering problems that looks deceptively simple until you have to explain why the nightly job silently ingested half a million rows and stopped without a trace. Spring Batch was built specifically to avoid that outcome. It provides a structured model for every concern you eventually discover on your own the hard way: job state persistence, chunk-level transaction boundaries, skip and retry semantics, and the ability to resume exactly where a failed run left off.

This tutorial walks through Spring Batch 5 — the version that ships with Spring Boot 3.x — building a realistic batch job that processes employee salary records from CSV files. You will see how chunk-oriented processing works inside the framework, how to configure restartability so a failed run picks up from the last committed chunk, and how to split large datasets across parallel workers using local partitioning. Everything here runs on a single machine; the approach scales to remote workers over message queues once the local patterns are solid.

What you will build: a job that reads employee salary records from one or more CSV files, applies a tax calculation, and writes the results to a MySQL database. The job is restartable, skips malformed rows without failing the whole run, and can process multiple files in parallel using a partitioned step.

Prerequisites

Java 21 or later
Spring Boot 3.3.x (bundles Spring Batch 5.1.x)
MySQL 8.x (or any JDBC-compatible database)
Maven 3.9+
Familiarity with Spring Boot auto-configuration

A note on versions: Spring Batch 5.0 (released with Spring Boot 3.0 in November 2022) removed all classes deprecated in 4.x. If you are coming from a 4.x codebase, the migration guide is essential reading — the builder factory classes JobBuilderFactory and StepBuilderFactory are gone; you inject JobRepository and PlatformTransactionManager directly instead.

Project Setup

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.3.5</version>
    <relativePath/>
  </parent>

  <groupId>com.vimleshpandey.demo</groupId>
  <artifactId>spring-batch-demo</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <properties>
    <java.version>21</java.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
      <groupId>com.mysql</groupId>
      <artifactId>mysql-connector-j</artifactId>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>com.h2database</groupId>
      <artifactId>h2</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
      <optional>true</optional>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <configuration>
          <excludes>
            <exclude>
              <groupId>org.projectlombok</groupId>
              <artifactId>lombok</artifactId>
            </exclude>
          </excludes>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

application.yml

spring:
  datasource:
    url: jdbc:mysql://localhost:3306/batch_demo
    username: ${DB_USER:root}
    password: ${DB_PASS:secret}
    driver-class-name: com.mysql.cj.jdbc.Driver

  jpa:
    hibernate:
      ddl-auto: validate
    show-sql: false
    open-in-view: false

  batch:
    jdbc:
      initialize-schema: always
    job:
      enabled: false

logging:
  level:
    org.springframework.batch: INFO
    com.vimleshpandey: DEBUG

Setting spring.batch.job.enabled=false is important. The default is true, which causes Spring Boot to run every discovered job immediately on startup — fine for simple scripts, problematic in a service that starts on a schedule or in response to an event.

Understanding Chunk-Oriented Processing

Spring Batch's most important design choice is the chunk-oriented step. Rather than reading all items into memory at once and writing them in a single transaction, it reads and processes items one at a time, accumulates them in a buffer, and commits the buffer in a single write transaction once the chunk size is reached.

READ item → PROCESS item → (repeat N times) → WRITE chunk → COMMIT

Each commit advances a cursor stored in BATCH_STEP_EXECUTION_CONTEXT. If the job fails after three committed chunks of 100 items each, a restart skips the first 300 rows and begins processing item 301. This is the core of restartability: the framework tracks exactly how far each step progressed.

The step execution context is serialized as JSON into BATCH_STEP_EXECUTION_CONTEXT.SHORT_CONTEXT. When a job restarts, Spring Batch loads the previous context and passes it to the ItemReader via ItemStream.open().

Spring Batch manages six BATCH_* metadata tables that persist job and step state. These tables are your audit log, your restart checkpoint store, and your diagnostic tool — query them freely when debugging a failed run.

Domain Model

// src/main/java/com/vimleshpandey/demo/model/EmployeeRecord.java
package com.vimleshpandey.demo.model;

public record EmployeeRecord(
    String employeeId,
    String name,
    String department,
    double grossSalary
) {}

// src/main/java/com/vimleshpandey/demo/model/ProcessedSalary.java
package com.vimleshpandey.demo.model;

import jakarta.persistence.*;

@Entity
@Table(name = "processed_salaries")
public class ProcessedSalary {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(nullable = false)
    private String employeeId;

    private String name;
    private String department;
    private double grossSalary;
    private double taxAmount;
    private double netSalary;

    public ProcessedSalary() {}

    public ProcessedSalary(String employeeId, String name, String department,
                           double grossSalary, double taxAmount, double netSalary) {
        this.employeeId  = employeeId;
        this.name        = name;
        this.department  = department;
        this.grossSalary = grossSalary;
        this.taxAmount   = taxAmount;
        this.netSalary   = netSalary;
    }

    public Long getId() { return id; }
    public String getEmployeeId() { return employeeId; }
    public String getName() { return name; }
    public String getDepartment() { return department; }
    public double getGrossSalary() { return grossSalary; }
    public double getTaxAmount() { return taxAmount; }
    public double getNetSalary() { return netSalary; }
}

Database schema

CREATE TABLE processed_salaries (
    id           BIGINT AUTO_INCREMENT PRIMARY KEY,
    employee_id  VARCHAR(50)    NOT NULL,
    name         VARCHAR(255)   NOT NULL,
    department   VARCHAR(100),
    gross_salary DECIMAL(15,2)  NOT NULL,
    tax_amount   DECIMAL(15,2)  NOT NULL,
    net_salary   DECIMAL(15,2)  NOT NULL,
    INDEX idx_employee_id (employee_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Building the Chunk Step

ItemReader — FlatFileItemReader

// src/main/java/com/vimleshpandey/demo/batch/SalaryItemReader.java
package com.vimleshpandey.demo.batch;

import com.vimleshpandey.demo.model.EmployeeRecord;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.core.io.Resource;

public class SalaryItemReader {

    public static FlatFileItemReader<EmployeeRecord> create(Resource resource) {
        return new FlatFileItemReaderBuilder<EmployeeRecord>()
            .name("salaryItemReader")
            .resource(resource)
            .linesToSkip(1)
            .delimited()
            .names("employeeId", "name", "department", "grossSalary")
            .fieldSetMapper(fs -> new EmployeeRecord(
                fs.readString("employeeId"),
                fs.readString("name"),
                fs.readString("department"),
                fs.readDouble("grossSalary")
            ))
            .saveState(true)
            .build();
    }
}

saveState(true) is on by default for FlatFileItemReader and is what allows the framework to record the file position in the step execution context. After a failure, the reader seeks to the last committed line rather than starting from line 1.

ItemProcessor — Tax Calculation

// src/main/java/com/vimleshpandey/demo/batch/SalaryItemProcessor.java
package com.vimleshpandey.demo.batch;

import com.vimleshpandey.demo.model.EmployeeRecord;
import com.vimleshpandey.demo.model.ProcessedSalary;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.lang.NonNull;

public class SalaryItemProcessor implements ItemProcessor<EmployeeRecord, ProcessedSalary> {

    private static final double TAX_RATE = 0.22;

    @Override
    public ProcessedSalary process(@NonNull EmployeeRecord item) {
        if (item.grossSalary() <= 0) {
            return null;
        }
        double taxAmount = item.grossSalary() * TAX_RATE;
        double netSalary = item.grossSalary() - taxAmount;
        return new ProcessedSalary(
            item.employeeId(), item.name(), item.department(),
            item.grossSalary(), taxAmount, netSalary
        );
    }
}

Returning null from process() is the idiomatic way to filter items. The framework does not write null results, and the skip/retry mechanism does not count them as errors.

ItemWriter

// src/main/java/com/vimleshpandey/demo/batch/SalaryItemWriter.java
package com.vimleshpandey.demo.batch;

import com.vimleshpandey.demo.model.ProcessedSalary;
import com.vimleshpandey.demo.repository.ProcessedSalaryRepository;
import org.springframework.batch.item.Chunk;
import org.springframework.batch.item.ItemWriter;
import org.springframework.lang.NonNull;

public class SalaryItemWriter implements ItemWriter<ProcessedSalary> {

    private final ProcessedSalaryRepository repository;

    public SalaryItemWriter(ProcessedSalaryRepository repository) {
        this.repository = repository;
    }

    @Override
    public void write(@NonNull Chunk<? extends ProcessedSalary> chunk) {
        repository.saveAll(chunk.getItems());
    }
}

Note the Chunk<? extends T> signature — this changed in Spring Batch 5.0. If you are copying examples from 4.x, replace the old List<? extends T> parameter with the new Chunk type.

JPA Repository

// src/main/java/com/vimleshpandey/demo/repository/ProcessedSalaryRepository.java
package com.vimleshpandey.demo.repository;

import com.vimleshpandey.demo.model.ProcessedSalary;
import org.springframework.data.jpa.repository.JpaRepository;

public interface ProcessedSalaryRepository extends JpaRepository<ProcessedSalary, Long> {}

Fault Tolerance: Skip and Retry

A realistic batch job will encounter rows it cannot process — malformed numbers, missing fields, constraint violations on the write side. Spring Batch's fault-tolerance DSL handles these without aborting the entire job.

// src/main/java/com/vimleshpandey/demo/batch/JobConfig.java
package com.vimleshpandey.demo.batch;

import com.vimleshpandey.demo.model.EmployeeRecord;
import com.vimleshpandey.demo.model.ProcessedSalary;
import com.vimleshpandey.demo.repository.ProcessedSalaryRepository;
import org.springframework.batch.core.*;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.file.FlatFileParseException;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.FileSystemResource;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
public class JobConfig {

    private final JobRepository jobRepository;
    private final PlatformTransactionManager transactionManager;
    private final ProcessedSalaryRepository salaryRepository;

    public JobConfig(JobRepository jobRepository,
                     PlatformTransactionManager transactionManager,
                     ProcessedSalaryRepository salaryRepository) {
        this.jobRepository      = jobRepository;
        this.transactionManager = transactionManager;
        this.salaryRepository   = salaryRepository;
    }

    @Bean
    public Step salaryProcessingStep() {
        return new StepBuilder("salaryProcessingStep", jobRepository)
            .<EmployeeRecord, ProcessedSalary>chunk(100, transactionManager)
            .reader(SalaryItemReader.create(
                new FileSystemResource("data/employees.csv")))
            .processor(new SalaryItemProcessor())
            .writer(new SalaryItemWriter(salaryRepository))
            .faultTolerant()
                .skipLimit(50)
                .skip(FlatFileParseException.class)
                .skip(NumberFormatException.class)
                .retryLimit(3)
                .retry(java.net.ConnectException.class)
            .listener(new SkipLoggingListener())
            .build();
    }

    @Bean
    public Job salaryJob() {
        return new JobBuilder("salaryJob", jobRepository)
            .start(salaryProcessingStep())
            .build();
    }
}

Skip vs. retry: skip is for input data errors that are not transient — a malformed CSV row will always fail on retry, so retrying it wastes resources. retry is for transient errors — a momentary network blip connecting to a downstream service may succeed on the second attempt.

skipLimit: once the job has skipped 50 items in a step, the step fails. This prevents a misconfigured reader from silently swallowing an entire bad file.

Skip Listener

// src/main/java/com/vimleshpandey/demo/batch/SkipLoggingListener.java
package com.vimleshpandey.demo.batch;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.SkipListener;

public class SkipLoggingListener implements SkipListener<Object, Object> {

    private static final Logger log = LoggerFactory.getLogger(SkipLoggingListener.class);

    @Override
    public void onSkipInRead(Throwable t) {
        log.warn("Skipped item during read: {}", t.getMessage());
    }

    @Override
    public void onSkipInProcess(Object item, Throwable t) {
        log.warn("Skipped during process: item={}, error={}", item, t.getMessage());
    }

    @Override
    public void onSkipInWrite(Object item, Throwable t) {
        log.warn("Skipped during write: item={}, error={}", item, t.getMessage());
    }
}

Partitioned Steps: Processing Multiple Files in Parallel

When your data can be split into independent slices — by ID range, by date, or by file — a partitioned step processes each slice in a separate worker step execution, all running concurrently on a thread pool.

The partitioning abstraction has two roles:

Partitioner: decides how to split the data and creates an ExecutionContext for each partition, encoding the slice boundary (the file path, the row ID range, etc.).

PartitionHandler: decides how to run the worker steps — locally on a thread pool, or remotely over a message queue.

File-based Partitioner

// src/main/java/com/vimleshpandey/demo/batch/MultiFilePartitioner.java
package com.vimleshpandey.demo.batch;

import org.springframework.batch.core.partition.support.Partitioner;
import org.springframework.batch.item.ExecutionContext;
import org.springframework.core.io.Resource;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.lang.NonNull;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class MultiFilePartitioner implements Partitioner {

    private final String filePattern;

    public MultiFilePartitioner(String filePattern) {
        this.filePattern = filePattern;
    }

    @Override
    @NonNull
    public Map<String, ExecutionContext> partition(int gridSize) {
        try {
            Resource[] resources = new PathMatchingResourcePatternResolver()
                .getResources(filePattern);
            Map<String, ExecutionContext> partitions = new HashMap<>();
            for (int i = 0; i < resources.length; i++) {
                ExecutionContext ctx = new ExecutionContext();
                ctx.putString("fileName", resources[i].getURI().toString());
                partitions.put("partition-" + i, ctx);
            }
            return partitions;
        } catch (IOException e) {
            throw new IllegalStateException("Could not resolve file pattern: " + filePattern, e);
        }
    }
}

Partitioned Job Configuration

// src/main/java/com/vimleshpandey/demo/batch/PartitionedJobConfig.java
package com.vimleshpandey.demo.batch;

import com.vimleshpandey.demo.model.EmployeeRecord;
import com.vimleshpandey.demo.model.ProcessedSalary;
import com.vimleshpandey.demo.repository.ProcessedSalaryRepository;
import org.springframework.batch.core.*;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.partition.PartitionHandler;
import org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.FlatFileParseException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
public class PartitionedJobConfig {

    private final JobRepository jobRepository;
    private final PlatformTransactionManager transactionManager;
    private final ProcessedSalaryRepository salaryRepository;

    public PartitionedJobConfig(JobRepository jobRepository,
                                PlatformTransactionManager transactionManager,
                                ProcessedSalaryRepository salaryRepository) {
        this.jobRepository      = jobRepository;
        this.transactionManager = transactionManager;
        this.salaryRepository   = salaryRepository;
    }

    @Bean
    @StepScope
    public FlatFileItemReader<EmployeeRecord> partitionedReader(
            @Value("#{stepExecutionContext['fileName']}") String fileName) {
        return SalaryItemReader.create(new FileSystemResource(fileName));
    }

    @Bean
    public Step workerStep(FlatFileItemReader<EmployeeRecord> partitionedReader) {
        return new StepBuilder("workerStep", jobRepository)
            .<EmployeeRecord, ProcessedSalary>chunk(100, transactionManager)
            .reader(partitionedReader)
            .processor(new SalaryItemProcessor())
            .writer(new SalaryItemWriter(salaryRepository))
            .faultTolerant()
                .skipLimit(10)
                .skip(FlatFileParseException.class)
            .build();
    }

    @Bean
    public PartitionHandler partitionHandler(Step workerStep) {
        TaskExecutorPartitionHandler handler = new TaskExecutorPartitionHandler();
        handler.setTaskExecutor(new SimpleAsyncTaskExecutor("batch-worker-"));
        handler.setStep(workerStep);
        handler.setGridSize(4);
        return handler;
    }

    @Bean
    public Step partitionedManagerStep(PartitionHandler partitionHandler) {
        return new StepBuilder("partitionedManagerStep", jobRepository)
            .partitioner("workerStep",
                new MultiFilePartitioner("file:data/employees-*.csv"))
            .partitionHandler(partitionHandler)
            .build();
    }

    @Bean
    public Job partitionedSalaryJob(Step partitionedManagerStep) {
        return new JobBuilder("partitionedSalaryJob", jobRepository)
            .start(partitionedManagerStep)
            .build();
    }
}

The @StepScope annotation makes the reader a step-scoped bean. Spring creates a new instance for each worker step execution and injects the partition-specific fileName from stepExecutionContext. Without @StepScope, all worker threads would share the same reader instance, which is not thread-safe and would corrupt restartability state.

Restartability with partitioning: a partitioned step is restartable, unlike a multi-threaded step. Each partition is a fully independent StepExecution with its own cursor in BATCH_STEP_EXECUTION_CONTEXT. If partition-2 fails, only that worker's step execution is marked FAILED; the others remain COMPLETED. On restart, Spring Batch re-runs only the failed partitions.

Triggering a Job Programmatically

For a service that launches jobs on-demand — from a REST endpoint, a Kafka message, or a scheduler — inject JobLauncher and call it directly.

// src/main/java/com/vimleshpandey/demo/service/BatchJobService.java
package com.vimleshpandey.demo.service;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.*;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException;
import org.springframework.stereotype.Service;

import java.time.Instant;

@Service
public class BatchJobService {

    private static final Logger log = LoggerFactory.getLogger(BatchJobService.class);

    private final JobLauncher jobLauncher;
    private final Job partitionedSalaryJob;

    public BatchJobService(JobLauncher jobLauncher, Job partitionedSalaryJob) {
        this.jobLauncher          = jobLauncher;
        this.partitionedSalaryJob = partitionedSalaryJob;
    }

    public JobExecution runSalaryJob() throws Exception {
        JobParameters params = new JobParametersBuilder()
            .addLong("run.id", Instant.now().toEpochMilli())
            .toJobParameters();
        try {
            return jobLauncher.run(partitionedSalaryJob, params);
        } catch (JobInstanceAlreadyCompleteException e) {
            log.info("Job already completed for parameters: {}", params);
            throw e;
        }
    }
}

Adding a unique run.id parameter ensures each invocation creates a distinct JobInstance. Without it, Spring Batch will refuse to re-run a job that has already completed with the same parameters.

Testing

Spring Batch provides @SpringBatchTest to load the full batch testing infrastructure, including JobLauncherTestUtils and JobRepositoryTestUtils.

// src/test/java/com/vimleshpandey/demo/batch/SalaryJobTest.java
package com.vimleshpandey.demo.batch;

import com.vimleshpandey.demo.SpringBatchDemoApplication;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.batch.core.*;
import org.springframework.batch.test.JobLauncherTestUtils;
import org.springframework.batch.test.JobRepositoryTestUtils;
import org.springframework.batch.test.context.SpringBatchTest;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;

import static org.assertj.core.api.Assertions.assertThat;

@SpringBatchTest
@SpringBootTest(classes = SpringBatchDemoApplication.class)
@ActiveProfiles("test")
class SalaryJobTest {

    @Autowired
    private JobLauncherTestUtils jobLauncherTestUtils;

    @Autowired
    private JobRepositoryTestUtils jobRepositoryTestUtils;

    @BeforeEach
    void setUp() {
        jobRepositoryTestUtils.removeJobExecutions();
    }

    @Test
    void salaryJob_completesSuccessfully() throws Exception {
        JobExecution execution = jobLauncherTestUtils.launchJob(
            new JobParametersBuilder()
                .addLong("run.id", System.currentTimeMillis())
                .toJobParameters()
        );
        assertThat(execution.getStatus()).isEqualTo(BatchStatus.COMPLETED);
    }

    @Test
    void salaryStep_processesItemsWithoutSkips() throws Exception {
        JobExecution execution = jobLauncherTestUtils.launchStep(
            "salaryProcessingStep",
            new JobParametersBuilder()
                .addLong("run.id", System.currentTimeMillis())
                .toJobParameters()
        );
        StepExecution step = execution.getStepExecutions().iterator().next();
        assertThat(step.getStatus()).isEqualTo(BatchStatus.COMPLETED);
        assertThat(step.getWriteCount()).isGreaterThan(0);
        assertThat(step.getSkipCount()).isZero();
    }
}

# src/test/resources/application-test.yml
spring:
  datasource:
    url: jdbc:h2:mem:batchtest;DB_CLOSE_DELAY=-1;MODE=MySQL
    driver-class-name: org.h2.Driver
    username: sa
    password:
  jpa:
    hibernate:
      ddl-auto: create-drop
  batch:
    jdbc:
      initialize-schema: always

JobRepositoryTestUtils.removeJobExecutions() in @BeforeEach cleans the BATCH_* tables between tests, preventing state leakage between test runs.

Running the Application

Create sample CSV files in data/:

employeeId,name,department,grossSalary
E001,Alice Johnson,Engineering,85000
E002,Bob Smith,Marketing,72000
E003,Charlie Brown,Engineering,91000
E004,Diana Prince,HR,68000

Then run the main class. You should see output like:

INFO  - Job: [partitionedSalaryJob] launched with the following parameters: ...
INFO  - Executing step: [partitionedManagerStep]
INFO  - Executing step: [workerStep:partition-0]
INFO  - Step: [workerStep:partition-0] executed in 142ms
INFO  - Job: [partitionedSalaryJob] completed with exit-code COMPLETED

Inspect the metadata tables to understand what Spring Batch recorded:

SELECT job_instance_id, job_name, start_time, status, exit_code
  FROM BATCH_JOB_EXECUTION
 ORDER BY start_time DESC LIMIT 5;

SELECT step_name, start_time, status, read_count, write_count, skip_count
  FROM BATCH_STEP_EXECUTION
 WHERE job_execution_id = <id>;

SELECT short_context
  FROM BATCH_STEP_EXECUTION_CONTEXT
 WHERE step_execution_id = <id>;

The short_context column shows the serialized reader state — the JSON that makes restartability possible.

Common Pitfalls and Tips

1. Multi-threaded steps are not restartable
A step configured with .taskExecutor() on the StepBuilder processes chunks concurrently, which means the framework cannot reliably record an ordered cursor position. If your step fails mid-run, there is no guaranteed restart point. Use partitioning instead — each partition is a fully independent step execution with its own cursor.

2. Chunk size and page size must align
If your JdbcPagingItemReader or JpaPagingItemReader uses a page size of 100 and your chunk size is 10, the reader fetches 100 rows from the database to satisfy 10 items. On a restart, the page-based reader re-fetches its entire first page and must skip forward. Set page size equal to chunk size to avoid unnecessary re-fetching.

3. The JobInstance vs. JobExecution distinction matters
A JobInstance is identified by job name plus parameters. A JobExecution is one attempt to run that instance. A completed JobInstance cannot be re-run with the same parameters by default — this is by design. For a nightly job that always uses today's date as a parameter, that is exactly the right behaviour. For a manual reprocessing scenario, add a unique run.id timestamp parameter to force a new instance.

4. Write idempotently
Spring Batch guarantees at-least-once delivery of each chunk on a restart: the last committed chunk's items may be re-processed and re-written if the commit happened but the job crashed before updating the step execution cursor. Use ON DUPLICATE KEY UPDATE or upsert semantics at the writer level, not just a blind INSERT.

5. Watch the BATCH_* table row count
In a long-running service with frequent jobs, the metadata tables grow indefinitely. Add a periodic cleanup task to prune BATCH_JOB_EXECUTION rows and their related records older than your retention period (e.g., 30 days).

6. @StepScope is mandatory for partitioned readers
If you forget @StepScope on a reader that injects stepExecutionContext values, all worker threads share the same reader bean. The first thread's context is used for all, every partition reads the same file, and your data is processed incorrectly. Spring will not warn you — the job simply produces wrong results.

Conclusion

Spring Batch 5's chunk model gives you production-grade batch processing with very little ceremony. The restartability guarantee — resume from the last committed chunk — means a 10-million-row job that crashes at row 7 million does not restart from zero. Partitioning multiplies throughput by running independent slices in parallel while keeping the fault-tolerance and restart guarantees intact for each slice.

The patterns in this article scale well beyond CSV files: swap FlatFileItemReader for JdbcPagingItemReader or a Kafka KafkaItemReader, and the same chunk step, skip/retry configuration, and partitioner logic applies. When a single JVM is not enough, the same Partitioner SPI works with remote workers over RabbitMQ or Kafka — but that is a topic for another article.

The key lesson from Spring Batch is architectural: separate the reading of data, the transformation of data, and the writing of data into three independent, composable concerns. That separation is what makes the framework's fault-tolerance and restart guarantees possible, and it is a design principle that applies well beyond batch processing.