Building a Production-Grade RAG System with Spring AI 1.1 and PGVector

Introduction

Large Language Models like GPT-4o are extraordinary reasoners, but they have two hard limits: their knowledge is frozen at a training cutoff date, and they know nothing about your private data. Retrieval-Augmented Generation (RAG) solves both problems by dynamically fetching relevant context from your own knowledge base and injecting it into the LLM prompt at query time — before the model writes a single word of its answer.

In this tutorial we'll build a fully functional, production-grade RAG system using Spring AI 1.1 (the GA release from November 2025) backed by PostgreSQL with the pgvector extension for semantic vector search. By the end you'll have a REST API that can ingest any document — PDF, DOCX, or plain text — and answer natural-language questions about its content with AI-generated, grounded responses.

What We'll Build

• A document ingestion pipeline that chunks, embeds, and stores documents as vectors in PostgreSQL
• A semantic search layer using pgvector's cosine-similarity HNSW index
• An AI query service powered by Spring AI's RetrievalAugmentationAdvisor
• A clean REST API: POST /api/ingest and POST /api/chat

How RAG Works (in 60 seconds)

Ingestion time:
  Document → extract text → split into chunks → embed each chunk → store (text + vector) in pgvector

Query time:
  User question → embed question → cosine-similarity search → retrieve top-K chunks
                → inject chunks as context into LLM prompt → LLM answers

The key insight is that LLMs don't see raw vectors — they receive plain text chunks that scored highest in similarity to the question. Vectors are only used for retrieval speed; the reasoning is pure language.

Prerequisites

Before we begin, ensure you have:

• Java 21 (LTS, required by Spring Boot 3.4+)
• Docker & Docker Compose (to run PostgreSQL + pgvector locally)
• Maven 3.9+
• An OpenAI API key (set via environment variable OPENAI_API_KEY)
• Basic familiarity with Spring Boot and REST APIs

Project Setup

pom.xml

Create a new Spring Boot project and use this pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
             https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.4.5</version>
        <relativePath/>
    </parent>

    <groupId>com.vimleshpandey</groupId>
    <artifactId>rag-demo</artifactId>
    <version>1.0.0</version>
    <name>rag-demo</name>
    <description>Production-grade RAG with Spring AI 1.1 and PGVector</description>

    <properties>
        <java.version>21</java.version>
        <spring-ai.version>1.1.0</spring-ai.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-bom</artifactId>
                <version>${spring-ai.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- Spring AI: OpenAI (chat + embeddings in one starter) -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-model-openai</artifactId>
        </dependency>

        <!-- Spring AI: PGVector vector store -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-starter-vector-store-pgvector</artifactId>
        </dependency>

        <!-- Document parsing: PDF, DOCX, TXT via Apache Tika -->
        <dependency>
            <groupId>org.apache.tika</groupId>
            <artifactId>tika-core</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.tika</groupId>
            <artifactId>tika-parsers-standard-package</artifactId>
            <version>2.9.2</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

docker-compose.yml

Start pgvector locally with Docker:

version: '3.8'
services:
  postgres:
    image: pgvector/pgvector:pg16
    container_name: rag_postgres
    environment:
      POSTGRES_USER: raguser
      POSTGRES_PASSWORD: ragpass
      POSTGRES_DB: ragdb
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

docker compose up -d

src/main/resources/application.yml

spring:
  application:
    name: rag-demo
  datasource:
    url: jdbc:postgresql://localhost:5432/ragdb
    username: raguser
    password: ragpass
    driver-class-name: org.postgresql.Driver

  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o-mini
          temperature: 0.2
      embedding:
        options:
          model: text-embedding-3-small

    vectorstore:
      pgvector:
        initialize-schema: true   # auto-creates vector_store table
        index-type: HNSW
        distance-type: COSINE_DISTANCE
        dimensions: 1536          # must match text-embedding-3-small output size

server:
  port: 8080

logging:
  level:
    org.springframework.ai: DEBUG

Important: initialize-schema: true drops and recreates the vector_store table on startup — perfect for development. In production, set it to false and manage the schema with Flyway or Liquibase.

Application Entry Point

package com.vimleshpandey.demo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class RagApplication {
    public static void main(String[] args) {
        SpringApplication.run(RagApplication.class, args);
    }
}

Building the Document Ingestion Pipeline

RAG has two distinct phases: ingestion (batch, happens once per document) and retrieval (real-time, happens on every query). Let's build ingestion first.

Spring AI models this as an ETL pipeline:

DocumentIngestionService

package com.vimleshpandey.demo.service;

import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.document.Document;
import org.springframework.ai.reader.tika.TikaDocumentReader;
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;

import java.util.List;

@Slf4j
@Service
@RequiredArgsConstructor
public class DocumentIngestionService {

    private final VectorStore vectorStore;

    // 512-token chunks, 100-token overlap, minimum 5 tokens per chunk
    private final TokenTextSplitter splitter =
        new TokenTextSplitter(512, 100, 5, 10_000, true);

    public int ingest(Resource resource, String sourceId) {
        log.info("Ingesting: {}", sourceId);

        // Extract: Apache Tika handles PDF, DOCX, ODT, TXT transparently
        List<Document> rawDocs = new TikaDocumentReader(resource).get();

        // Tag every document with a source key for later filtering/deletion
        rawDocs.forEach(doc -> doc.getMetadata().put("source", sourceId));

        // Transform: chunk by token count, not character count
        List<Document> chunks = splitter.apply(rawDocs);
        log.info("Split '{}' into {} chunks", sourceId, chunks.size());

        // Load: Spring AI calls OpenAI /embeddings, then writes to pgvector
        vectorStore.add(chunks);

        log.info("Ingestion complete for '{}'", sourceId);
        return chunks.size();
    }
}

Why TokenTextSplitter over character splitting?

OpenAI's context windows are measured in tokens, not characters. A 512-token chunk is always a valid input to the embedding model. Character-based splitters can create chunks that silently overflow the model's input limit, producing truncated — and therefore inaccurate — embeddings.

Why 100-token overlap?

Key information often straddles paragraph boundaries. Overlapping consecutive chunks ensures that a sentence split across a chunk boundary is fully represented in at least one chunk that gets retrieved.

Building the RAG Query Service

Spring AI 1.1 introduced RetrievalAugmentationAdvisor — a first-class Advisor that transparently intercepts your ChatClient call, runs a vector similarity search, and injects the retrieved chunks as structured context into the prompt before the model ever sees it.

RagQueryService

package com.vimleshpandey.demo.service;

import lombok.RequiredArgsConstructor;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.RetrievalAugmentationAdvisor;
import org.springframework.ai.rag.retrieval.search.VectorStoreDocumentRetriever;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.stereotype.Service;

@Service
@RequiredArgsConstructor
public class RagQueryService {

    private final ChatClient.Builder chatClientBuilder;
    private final VectorStore vectorStore;

    private static final String SYSTEM_PROMPT = """
        You are a helpful assistant that answers questions based strictly on the
        provided context documents. If the context does not contain sufficient
        information to answer the question, clearly state that — never fabricate facts.
        Cite relevant details from the context in your answer.
        """;

    public String query(String question) {
        RetrievalAugmentationAdvisor ragAdvisor = RetrievalAugmentationAdvisor.builder()
            .documentRetriever(
                VectorStoreDocumentRetriever.builder()
                    .vectorStore(vectorStore)
                    .similarityThreshold(0.45)   // ignore chunks with similarity < 0.45
                    .topK(5)                      // retrieve the 5 most similar chunks
                    .build()
            )
            .build();

        return chatClientBuilder.build()
            .prompt()
            .system(SYSTEM_PROMPT)
            .user(question)
            .advisors(ragAdvisor)
            .call()
            .content();
    }
}

What happens internally when .call() executes:

1. The advisor intercepts the call before it reaches OpenAI
2. It embeds question using text-embedding-3-small
3. It runs SELECT ... ORDER BY embedding <=> $1 LIMIT 5 in pgvector (cosine distance)
4. It formats the 5 retrieved text chunks into a structured Context: block
5. It prepends that block to the prompt and forwards to OpenAI gpt-4o-mini
6. The model replies using only information grounded in your documents

REST API Layer

package com.vimleshpandey.demo.controller;

import com.vimleshpandey.demo.service.DocumentIngestionService;
import com.vimleshpandey.demo.service.RagQueryService;
import lombok.RequiredArgsConstructor;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;

import java.io.IOException;
import java.util.Map;

@RestController
@RequestMapping("/api")
@RequiredArgsConstructor
public class RagController {

    private final DocumentIngestionService ingestionService;
    private final RagQueryService ragQueryService;

    @PostMapping("/ingest")
    public ResponseEntity<Map<String, Object>> ingest(
            @RequestParam("file") MultipartFile file) throws IOException {

        Resource resource = new ByteArrayResource(file.getBytes()) {
            @Override public String getFilename() {
                return file.getOriginalFilename();
            }
        };

        int chunks = ingestionService.ingest(resource, file.getOriginalFilename());

        return ResponseEntity.ok(Map.of(
            "status", "ingested",
            "filename", file.getOriginalFilename(),
            "chunks", chunks
        ));
    }

    @PostMapping("/chat")
    public ResponseEntity<Map<String, String>> chat(
            @RequestBody Map<String, String> body) {

        String question = body.get("question");
        if (question == null || question.isBlank()) {
            return ResponseEntity.badRequest()
                .body(Map.of("error", "field 'question' is required"));
        }

        String answer = ragQueryService.query(question);
        return ResponseEntity.ok(Map.of("answer", answer));
    }
}

Testing

Unit Test

Place this in src/test/java/com/vimleshpandey/demo/RagApplicationTests.java:

package com.vimleshpandey.demo;

import com.vimleshpandey.demo.service.DocumentIngestionService;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.core.io.ClassPathResource;
import org.springframework.test.context.TestPropertySource;

import static org.assertj.core.api.Assertions.assertThat;

@SpringBootTest
@TestPropertySource(properties = {
    "spring.ai.openai.api-key=test-key",
    "spring.ai.vectorstore.pgvector.initialize-schema=true"
})
class RagApplicationTests {

    @Autowired
    private DocumentIngestionService ingestionService;

    @Test
    void contextLoads() {
        // Verifies the Spring context assembles without wiring errors
    }

    @Test
    void ingestionSplitsDocumentIntoMultipleChunks() throws Exception {
        var resource = new ClassPathResource("test-document.txt");
        int chunks = ingestionService.ingest(resource, "test-document.txt");
        assertThat(chunks).isGreaterThan(0);
    }
}

Create src/test/resources/test-document.txt with at least a few paragraphs of sample text to exercise the chunking logic. The test will make real calls to pgvector but mocks the OpenAI key — if you want a fully offline test, mock the VectorStore and EmbeddingModel beans.

Integration Test with MockMvc

@WebMvcTest(RagController.class)
class RagControllerTest {

    @Autowired MockMvc mockMvc;

    @MockBean DocumentIngestionService ingestionService;
    @MockBean RagQueryService ragQueryService;

    @Test
    void chatReturnsBadRequestForMissingQuestion() throws Exception {
        mockMvc.perform(post("/api/chat")
                .contentType(MediaType.APPLICATION_JSON)
                .content("{}"))
            .andExpect(status().isBadRequest());
    }

    @Test
    void chatReturnsAnswer() throws Exception {
        when(ragQueryService.query("What is Spring AI?"))
            .thenReturn("Spring AI is a framework for building AI-powered applications in Java.");

        mockMvc.perform(post("/api/chat")
                .contentType(MediaType.APPLICATION_JSON)
                .content("{\"question\": \"What is Spring AI?\"}"))
            .andExpect(status().isOk())
            .andExpect(jsonPath("$.answer").isNotEmpty());
    }
}

Running the Application

# 1. Start PostgreSQL with pgvector
docker compose up -d

# 2. Export your OpenAI key
export OPENAI_API_KEY=sk-proj-...

# 3. Start the application
mvn spring-boot:run

# 4. Ingest a document
curl -X POST http://localhost:8080/api/ingest \
  -F "file=@/path/to/my-report.pdf"

# Expected response:
# {"status":"ingested","filename":"my-report.pdf","chunks":47}

# 5. Ask a question
curl -X POST http://localhost:8080/api/chat \
  -H "Content-Type: application/json" \
  -d '{"question": "What are the key findings in the report?"}'

# Expected response:
# {"answer":"Based on the document, the key findings are..."}

Common Pitfalls and Tips

1. Dimension mismatch errors on startup

text-embedding-3-small produces 1536-dimensional vectors. If you switch to text-embedding-3-large (3072 dims), you must update spring.ai.vectorstore.pgvector.dimensions: 3072 and drop/recreate the vector_store table — mismatched dimensions cause a PGException at insert time.

2. Never leave initialize-schema: true in production

This flag recreates the table on every restart, wiping all stored embeddings. Use Flyway:

-- V1__create_vector_store.sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE IF NOT EXISTS vector_store (
    id uuid DEFAULT gen_random_uuid() PRIMARY KEY,
    content text,
    metadata jsonb,
    embedding vector(1536)
);
CREATE INDEX ON vector_store USING hnsw (embedding vector_cosine_ops);

3. Tune the similarity threshold per use case

0.45 is a safe default, but high-precision knowledge-base Q&A often needs 0.60+ to filter out loosely related chunks that confuse the model. Log retrieved documents at DEBUG level and examine them manually to calibrate.

4. Chunk overlap matters for dense technical documents

For contracts, technical specs, or academic papers where critical information spans paragraphs, increase overlap from 100 to 150–200 tokens. For narrative text (news articles, blog posts), 50 tokens is usually sufficient.

5. HNSW vs IVFFlat index

HNSW (our choice) offers excellent recall without needing to know the dataset size upfront, and handles incremental inserts well. IVFFlat can be faster for very large datasets (>1M vectors) but needs periodic VACUUM ANALYZE and tuning the lists parameter. Start with HNSW.

6. Rate limiting during bulk ingestion

OpenAI's Embeddings API has per-minute token limits. For bulk ingestion of hundreds of documents, Spring AI's BatchingStrategy sends multiple texts in a single /embeddings call — enable it by implementing BatchingStrategy or throttling with a fixed delay between batches.

7. Use metadata filters for multi-tenant scenarios

If you're building a product where different users upload different documents, add a userId metadata field and pass a FilterExpressionBuilder to VectorStoreDocumentRetriever so users only retrieve their own content:

Filter.Expression userFilter = new FilterExpressionBuilder()
    .eq("userId", currentUserId)
    .build();

VectorStoreDocumentRetriever.builder()
    .filterExpression(userFilter)
    ...

Conclusion

You now have a fully functional RAG pipeline built on Spring AI 1.1 — in under 200 lines of Java. With RetrievalAugmentationAdvisor, the wiring between vector search and the LLM is handled by the framework, letting you focus on the domain logic that actually matters.

The architecture is deliberately open to extension. Swap OpenAI for Ollama (run models locally), replace pgvector with Weaviate or Chroma, add a re-ranking step with DocumentPostProcessor, or wire in RewriteQueryTransformer to rewrite ambiguous questions before embedding them — all without changing your service or controller code.

For production, the next steps are: Flyway migrations for the schema, an authentication layer on /api/ingest, rate limiting on /api/chat, token-usage tracking per user, and structured logging for retrieval quality monitoring. But the RAG loop you've built here is solid and battle-tested — ready to power your next AI-driven product.