BraveNexas Technologies
AI Engineering14 min read•Updated 2026-11-01

Production RAG Architectures: Hybrid Search, Reranking & PGVector in Python

A battle-tested blueprint for moving beyond fragile prototype tutorials to resilient, enterprise-grade retrieval pipelines.

BraveNexas Engineering FacultyPractitioners at Samsung, KFintech, Equipo Health
Executive Summary

Key Architectural Takeaways

  • Why naive vector retrieval fails on dense enterprise documents (loss of structural hierarchy and chunk fragmentation).
  • Designing semantic boundaries: Recursive character splitting with metadata inheritance (parent-child retrieval).
  • Building hybrid search combining dense embeddings (pgvector HNSW) and sparse lexical search (PostgreSQL tsvector).
  • Applying Reciprocal Rank Fusion (RRF) and Cross-Encoder re-ranking to achieve >92% top-3 retrieval accuracy.
  • End-to-end evaluation using RAGAS for context recall, context precision, and hallucination reduction.

Guide Contents

  1. 01.The Naive RAG Trap: Why 5-Line Tutorials Break in Production
  2. 02.Ingestion Pipeline: Semantic Boundaries & Parent-Child Retrieval
  3. 03.Implementing Hybrid Search with PostgreSQL pgvector and Full-Text Search
  4. 04.Cross-Encoder Re-Ranking & RAGAS Evaluation
  5. 05.Frequently Asked Questions

1. The Naive RAG Trap: Why 5-Line Tutorials Break in Production

Every beginner tutorial on Retrieval-Augmented Generation follows the identical naive formula: load a PDF with PyPDFLoader, split text into arbitrary 500-token chunks with 50-token overlap, embed them using an OpenAI API call, dump them into an in-memory vector database, and perform cosine similarity search against the user prompt.

While this setup yields an impressive demo on clean 2-page blog posts, it disintegrates when deployed against real-world enterprise documents such as quarterly financial filings, technical specifications, and legal agreements. Real documents contain hierarchical headers, structured tables, and dispersed contextual dependencies.

When an arbitrary 500-token boundary cuts through a table or severs a clause from its governing condition, the embedding vector captures incomplete semantics. In production, naive cosine search frequently returns irrelevant fragments, causing large language models to hallucinate plausible yet incorrect answers.

Production RealityPure vector similarity measures semantic proximity, not factual precision. A question like 'What was the Q3 revenue in Tamil Nadu?' will retrieve Q2 revenue paragraphs because the vocabulary and topics are mathematically close.

2. Ingestion Pipeline: Semantic Boundaries & Parent-Child Retrieval

Production RAG begins with intelligent document parsing. Instead of naive fixed-size character splitting, high-precision ingestion relies on two structural techniques: Semantic Markdown Boundary Splitting and Parent-Child (Hierarchical) Chunking.

In Parent-Child retrieval, we separate the retrieval unit from the synthesis unit. Documents are indexed at two granularities: small child chunks (150–200 tokens) optimized for vector matching, and larger parent chunks (800–1200 tokens) that preserve complete paragraphs and surrounding context.

When a vector match fires on a child chunk, the ingestion pipeline resolves the parent chunk ID and injects the comprehensive parent context into the LLM prompt. This ensures high search precision without sacrificing contextual continuity.

fastapi_parent_child_indexer.py — Storing parent and child embeddings in PostgreSQLpython
import uuid
from typing import List, Dict
from pydantic import BaseModel

class ChildChunk(BaseModel):
    id: str
    parent_id: str
    content: str
    embedding: List[float]

class ParentChunk(BaseModel):
    id: str
    document_id: str
    content: str
    section_title: str

def create_hierarchical_chunks(document_id: str, text: str) -> tuple[List[ParentChunk], List[ChildChunk]]:
    # 1. Parse document into sections by Markdown headers
    sections = split_by_headers(text)
    parents: List[ParentChunk] = []
    children: List[ChildChunk] = []

    for sec in sections:
        p_id = str(uuid.uuid4())
        parents.append(ParentChunk(id=p_id, document_id=document_id, content=sec.body, section_title=sec.title))
        
        # 2. Slice parent section into compact child passages for dense indexing
        sub_passages = recursive_char_splitter(sec.body, chunk_size=200, chunk_overlap=30)
        for passage in sub_passages:
            emb = compute_dense_embedding(passage)
            children.append(ChildChunk(id=str(uuid.uuid4()), parent_id=p_id, content=passage, embedding=emb))

    return parents, children

4. Cross-Encoder Re-Ranking & RAGAS Evaluation

Hybrid search narrows 100,000 document chunks down to the top 20 candidates in ~15ms. However, passing 20 chunks directly to your LLM introduces 'lost-in-the-middle' attention degradation and wastes token budget.

The final step before prompt synthesis is Cross-Encoder Re-ranking. Unlike bi-encoders (which encode prompt and passage separately into single vectors), a cross-encoder evaluates the prompt and candidate passage together through full transformer cross-attention layers, outputting a precise relevance score between 0.0 and 1.0.

We rerank the top 20 candidates with `bge-reranker-large` or Cohere Rerank v3, retaining only the top 3–5 highest-scoring contexts for the final LLM completion.

Finally, verify your pipeline using the RAGAS framework. Track Faithfulness (ensuring the model output does not contradict retrieved passages) and Context Recall (ensuring all ground-truth facts were retrieved). A production RAG system should sustain >90% faithfulness prior to release.

Common Questions

Frequently Asked Questions

Why use pgvector instead of dedicated vector databases like Pinecone?

pgvector keeps your relational data, user authorization tables, and vector embeddings in a single ACID-compliant database. You avoid cross-system synchronization bugs, separate security perimeters, and extra subscription bills.

What is Reciprocal Rank Fusion (RRF)?

RRF is an algorithm that combines rankings from different search mechanisms (such as dense vector similarity and keyword BM25 search) by scoring each item as 1 / (k + rank). It normalizes scale differences without requiring complex hyperparameter tuning.

Flagship AI Track

Master this architecture with real code reviews

Build, evaluate, and deploy multi-modal RAG systems and autonomous agent workflows in our live online batch.