For years, document management has largely been about storing files and making them searchable. Businesses have become increasingly good at keeping contracts, reports, policies, invoices, research papers and technical documentation in cloud repositories. Yet finding a file is only one part of the problem. The more difficult question is finding the right information inside thousands of files and understanding what that information actually means.
This is where AI-powered document intelligence is changing the architecture of modern information systems. Instead of treating a PDF or DOCX file as a static object, document intelligence systems can extract its contents, understand its structure, convert its information into searchable representations and retrieve relevant passages when a user asks a question in natural language.
Intellodocs is an example of this broader shift. The interesting engineering challenge, however, is not the interface through which someone asks a question. It is everything happening behind that interface: document ingestion, parsing, OCR, chunking, embeddings, vector search, reranking, access control and finally the language model that turns retrieved information into a useful response.
The Architecture Behind an AI Document System
A practical document-intelligence architecture can be viewed as a pipeline rather than a single AI model. A document first enters an ingestion service, where it is stored and assigned an identifier. A processing layer extracts text and metadata, while OCR handles documents that do not contain a machine-readable text layer. The extracted material is then divided into meaningful chunks and indexed using both semantic and traditional search.
At query time, the user's question follows a similar pipeline in reverse. The system interprets the query, searches the relevant indexes, combines the results, reranks the most promising passages and sends the strongest context to the language model. The generated response is then returned along with information about the original document and location of the supporting material.
A simplified architecture looks like this:
User Query
|
v
API / Authentication
|
v
Query Normalization
|
+------------+------------+
| |
v v
Keyword Search Vector Search
(BM25) (Embeddings)
| |
+------------+------------+
|
v
Reranking
|
v
Context Builder
|
v
LLM Layer
|
v
Answer + Citations
On the ingestion side, the flow works independently:
PDF / DOCX / XLSX / Image
|
v
Document Parser
|
+----+----+
| |
v v
Text Layer OCR
| |
+----+----+
|
v
Structure-Aware Chunks
|
+----+----+
| |
v v
Text Index Vector Index
Keeping these responsibilities separate is important. The parser should not be responsible for communicating with an LLM, and the vector database should not be responsible for enforcing application-level permissions. Each layer should have a clear job.
Choosing a Technology Stack
There is no single mandatory stack for document intelligence, but a Python-based architecture provides a practical starting point. FastAPI can expose ingestion and search APIs, PyMuPDF can handle PDF extraction, python-docx can process Word documents and openpyxl can deal with spreadsheets. For scanned documents, OCR engines such as Tesseract or PaddleOCR can provide the missing text layer.
For semantic search, Sentence Transformers can generate embeddings, while FAISS, Qdrant, Milvus or PostgreSQL with pgvector can provide vector search. Elasticsearch or OpenSearch can handle conventional full-text and BM25 search. A cross-encoder can be introduced as a reranking layer, and an LLM can sit at the final answer-generation stage.
PostgreSQL is useful for application metadata, document ownership, processing states, permissions and audit events, while an S3-compatible object store can retain the original files. For larger workloads, Redis and Celery can move expensive document-processing tasks into asynchronous workers. Docker then provides a consistent deployment environment.
The specific technology choices can change. The architectural separation is what matters.
Document Ingestion Is the First Engineering Problem
An AI system cannot retrieve information it never successfully extracted.
A basic FastAPI endpoint can receive an uploaded document and assign it an identifier:
from fastapi import FastAPI, UploadFile, File
from uuid import uuid4
app = FastAPI()
@app.post("/documents")
async def upload_document(file: UploadFile = File(...)):
document_id = str(uuid4())
contents = await file.read()
path = f"/data/{document_id}_{file.filename}"
with open(path, "wb") as output:
output.write(contents)
return {
"document_id": document_id,
"filename": file.filename,
"status": "queued"
}
A production implementation would normally store the file in object storage rather than writing it directly to the API container. The database can maintain the document identifier, original filename, MIME type, storage location, checksum, owner and processing status.
The checksum is particularly useful when organizations repeatedly upload revised copies of the same document. It can help identify duplicates and avoid unnecessarily repeating expensive processing.
The first major trap in document AI is assuming that every document is simply a string.
A PDF may contain paragraphs, tables, headings, footnotes, page numbers, columns, signatures and images. Two documents may look similar to a human while being completely different from a machine-processing perspective.
A simple PDF extraction routine using PyMuPDF might look like this:
import fitz
def extract_pages(pdf_path):
document = fitz.open(pdf_path)
pages = []
for page_number, page in enumerate(document):
pages.append({
"page": page_number + 1,
"text": page.get_text("text")
})
return pages
This works well when a PDF contains an actual text layer. It does not solve the problem of scanned documents.
For an image-based PDF, the pipeline needs to detect that native extraction has returned little or no useful text and route the page through OCR. The resulting text should ideally retain its relationship with the original page so that later retrieval can identify exactly where the information came from.
This is important because document intelligence is not simply about extracting words. It is about preserving enough context to reconstruct meaning.
Chunking Should Follow Document Structure
Once text has been extracted, the next challenge is deciding how much of it should be stored as an individual retrieval unit.
The common RAG implementation starts with a fixed chunk size and overlap. For example, an application might divide text into 1,000-character blocks with 200 characters of overlap. This is easy to implement, but it can produce poor results with structured business documents.
Imagine a policy containing a heading called "International Travel Approval", followed by several paragraphs explaining eligibility and exceptions. A character-based splitter may separate the heading from the content that explains it.
A structure-aware approach instead keeps sections, subsections, paragraphs and tables together wherever practical.
A chunk might therefore be represented as:
chunk = {
"document_id": "doc_102",
"page": 17,
"section": "International Travel Approval",
"content": "...",
"chunk_index": 14
}
That metadata is not incidental. It becomes useful later for filtering, ranking, access control and citations.
Tables require particular attention. A row such as "Europe | Q1 | Q2 | Change" contains relationships that can disappear if the table is converted into a flat sequence of words. A document-intelligence pipeline should therefore preserve table structure whenever possible rather than assuming that ordinary paragraph extraction is sufficient.
Embeddings Turn Text Into Searchable Meaning
After the document has been divided into useful chunks, each chunk can be converted into an embedding.
An embedding model represents text as a numerical vector. Similar meanings tend to occupy nearby positions in the resulting vector space. This makes it possible to retrieve information even when the user's wording differs from the wording used in the source document.
Using Sentence Transformers, a basic implementation can look like this:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"sentence-transformers/all-MiniLM-L6-v2"
)
texts = [
"Employees require approval before submitting travel expenses.",
"Travel reimbursement requires manager authorization."
]
vectors = model.encode(
texts,
normalize_embeddings=True
)
Those vectors can then be stored in a vector index. FAISS is a reasonable option for smaller or self-managed deployments:
import faiss
import numpy as np
dimension = vectors.shape[1]
index = faiss.IndexFlatIP(dimension)
index.add(np.asarray(vectors, dtype="float32"))
At query time, the user's question is embedded using the same model and compared with the stored vectors.
However, semantic search should not necessarily replace conventional search.
Why Hybrid Retrieval Matters
Consider a user searching for a specific policy identifier such as "POL-4827". A keyword search engine can match that identifier precisely. A semantic search system may have little understanding of why the exact character sequence is important.
Now consider a much more natural question such as, "What rules apply to employees travelling internationally?" The relevant document may never contain that exact sentence. It might instead use the phrase "overseas business travel requirements."
These are two different retrieval problems.
Keyword search is strong at exact terminology, identifiers and unusual names. Semantic search is strong at understanding conceptual similarity. Combining both can therefore provide a more robust retrieval layer.
A simplified implementation might retrieve candidates from both systems and merge them:
keyword_results = keyword_search(query, top_k=10)
semantic_results = vector_search(
query,
top_k=10
)
candidates = merge_results(
keyword_results,
semantic_results
)
The combined candidate set can then move into a reranking stage.
Reranking Improves Retrieval Precision
Retrieving twenty potentially relevant passages does not mean that all twenty deserve to be sent to an LLM.
A reranker can evaluate the relationship between the original question and each retrieved passage more carefully. Cross-encoder models are commonly used for this purpose.
from sentence_transformers import CrossEncoder
reranker = CrossEncoder(
"cross-encoder/ms-marco-MiniLM-L-6-v2"
)
pairs = [
(query, result["content"])
for result in candidates
]
scores = reranker.predict(pairs)
ranked = sorted(
zip(candidates, scores),
key=lambda x: x[1],
reverse=True
)
context = [
item[0]
for item in ranked[:5]
]
This two-stage architecture is important because retrieval and reranking solve different problems. Retrieval needs to produce a sufficiently broad candidate set, while reranking focuses on precision.
Simply increasing the number of retrieved documents is not necessarily an improvement. It can actually introduce more irrelevant context into the LLM prompt.
The LLM Should Come After Retrieval
Once the system has identified the most relevant passages, the language model can finally be introduced.
A simplified prompt might look like this:
prompt = f"""
Answer the question using only the supplied context.
If the context does not contain enough information,
say that the information is not available.
Question:
{query}
Context:
{context}
"""
This architecture is commonly described as Retrieval-Augmented Generation, or RAG.
The important architectural principle is that the LLM is not being asked to search the entire document repository from scratch. It receives a carefully selected context retrieved from the organization's own information.
This distinction also helps reduce hallucination. It does not eliminate hallucinations entirely, but it gives the model a defined evidence base against which its response can be evaluated.
In some situations, the most reliable answer is simply that the available documents do not contain enough information.
Citations Should Be Designed Into the Data Model
Source attribution is often treated as a user-interface feature, but it needs to begin much earlier in the pipeline.
Each chunk should retain enough metadata to trace it back to the original document:
{
"content": "International travel requires prior approval.",
"document_id": "doc_102",
"filename": "Travel_Policy.pdf",
"page": 17,
"section": "International Travel"
}
The answer service can then return both the generated response and its sources:
{
"answer": "International travel requires prior approval.",
"sources": [
{
"document": "Travel_Policy.pdf",
"page": 17,
"section": "International Travel"
}
]
}
This makes the answer verifiable.
For enterprise document systems, that matters because users may need to validate a response before acting on it. A system that can answer quickly but cannot explain where the answer came from may be less useful than a slightly slower system with reliable source attribution.
Permissions Must Be Enforced Before Retrieval
Security creates another important architectural requirement.
Imagine a repository containing public policies alongside confidential HR documents. A user asking a broad question should never receive information from a document they are not authorized to access.
The correct flow is therefore closer to:
User
|
v
Authentication
|
v
Permission Filtering
|
v
Retrieval
|
v
Reranking
|
v
LLM
Not:
User
|
v
Retrieve Everything
|
v
LLM
|
v
Remove Restricted Information
Filtering after the LLM has already received sensitive content is too late.
Permission-aware retrieval should therefore be treated as part of the retrieval architecture itself.
Asynchronous Processing Keeps the API Responsive
Processing a short text file may take milliseconds. Processing hundreds of scanned pages can involve OCR, parsing, chunking, embedding generation and indexing.
Those operations should not block the user's HTTP request.
A message queue can separate document upload from document processing:
POST /documents
|
v
API Service
|
v
Redis Queue
|
+---------+---------+
| | |
v v v
Parser OCR Embedding
Worker Worker Worker
|
v
Indexing
Celery provides one possible implementation:
from celery import Celery
celery = Celery(
"document_pipeline",
broker="redis://redis:6379/0"
)
@celery.task
def process_document(document_id):
document = load_document(document_id)
text = extract_text(document)
chunks = create_chunks(text)
embeddings = generate_embeddings(chunks)
store_vectors(
document_id,
chunks,
embeddings
)
mark_as_processed(document_id)
The upload endpoint can return immediately with a processing status while workers handle the expensive operations in the background.
This also creates a natural place to retry failed jobs without requiring the user to upload the document again.
Designing for Production
A document-intelligence prototype can work with a few Python scripts and a local vector index. Production introduces a much larger set of concerns.
PostgreSQL can maintain document metadata, users, permissions and processing jobs. Object storage can retain original files. A vector database can handle semantic retrieval, while Elasticsearch or OpenSearch can provide full-text search.
The architecture can therefore be separated into three storage layers:
Document Intelligence Platform
|
+----------------+----------------+
| | |
v v v
Object Storage PostgreSQL Vector Store
Original Files Metadata Embeddings
Permissions Semantic Search
Audit Logs
This separation allows each component to scale according to its workload.
It also reduces coupling. Changing the embedding model should not require redesigning the document-storage layer. Moving from FAISS to a managed vector database should not require rewriting the application metadata model.
Measuring More Than Answer Quality
One of the biggest mistakes in AI application development is evaluating a RAG system by reading a handful of answers and deciding whether they "look right."
There are several separate things to measure.
Retrieval quality can be evaluated through metrics such as Recall@K, Precision@K, Mean Reciprocal Rank and NDCG. These help determine whether the system is actually finding the relevant evidence.
The generated answer needs a different evaluation layer. Faithfulness, answer relevance, context relevance and citation accuracy can help determine whether the model is using the retrieved information correctly.
Then there are traditional software-engineering metrics: ingestion latency, embedding throughput, search latency, reranking latency, LLM latency, queue depth and end-to-end response time.
For example, reporting p50 and p95 query latency can be considerably more informative than quoting only an average:
p50 query latency: 1.2 seconds
p95 query latency: 3.8 seconds
p99 query latency: 7.1 seconds
The system may feel fast for most users while still having a significant tail-latency problem.
The Hard Problems Are Usually Outside the Demo
The impressive part of an AI document application is often the final question-and-answer interface. The difficult engineering work is hidden underneath it.
Real documents contain rotated tables, multiple columns, broken OCR, repeated headers, handwritten annotations, footnotes and contradictory versions. Users ask questions that require comparison across documents, historical context or precise identification of a particular clause.
Those problems cannot be solved simply by switching to a larger language model.
They require better document processing, richer metadata, structure-aware chunking, hybrid retrieval, reranking and strong access controls.
This is why platforms such as Intellodocs are better understood in the context of a much broader engineering movement: the transformation of documents from passive files into searchable knowledge systems.
The Bigger Engineering Picture
AI document intelligence is sometimes reduced to the phrase "chat with your PDFs." Technically, that description misses most of the interesting work.
A reliable system involves a chain of interconnected components:
Document
|
v
Extraction
|
v
Structure Preservation
|
v
Chunking
|
v
Embeddings
|
v
Hybrid Retrieval
|
v
Reranking
|
v
Context Selection
|
v
LLM
|
v
Citations
A failure at any stage can affect everything downstream. If OCR misses important text, retrieval cannot find it. If chunking destroys the relationship between a table and its heading, the embedding may represent incomplete meaning. If retrieval returns irrelevant passages, the language model has poor evidence to work with. If permissions are applied too late, confidential information may enter the model context.
The central engineering lesson is therefore simple: the intelligence of an AI document system does not come from the LLM alone.
It comes from the infrastructure that surrounds it.
The real goal is not merely to make documents searchable. It is to build a system capable of delivering the right information, from the right source, with the right context, to the right user.
That is the foundation on which the next generation of document-management systems is being built.
