Architecting a Personal Health Intelligence System: RAG-Based Retrieval for Longitudinal Medical Data
Introduction
Managing personal health data presents a significant technical challenge due to the fragmented nature of medical records, often distributed across unstructured PDF lab reports, scanned diagnostic images, and handwritten clinical notes. To transform these static documents into an actionable "Health Brain," we leverage a Retrieval-Augmented Generation (RAG) architecture. This system employs Optical Character Recognition (OCR) for data extraction, Hypothetical Document Embeddings (HyDE) for query expansion, and high-dimensional vector search to provide context-aware responses. This article outlines the implementation of such a system, focusing on data ingestion, semantic indexing, and retrieval optimization.
System Architecture
The following diagram illustrates the data pipeline, from raw document ingestion to the generation of health insights.
graph TD
subgraph Ingestion_Layer
A[Medical PDF/Scans] --> B(Unstructured.io OCR)
B --> C(Recursive Text Splitting)
C --> D[Text Embeddings]
end
subgraph Storage_Layer
D --> E[(Pinecone Vector Database)]
end
subgraph Retrieval_Layer
F[User Query] --> G{HyDE Strategy}
G --> H(Hypothetical Answer)
H --> I(Vector Similarity Search)
E -.-> I
end
subgraph Generation_Layer
I --> J[Contextual Augmentation]
J --> K(Claude 3.5 Sonnet)
K --> L[Structured Health Insight]
end
Prerequisites & Stack
- Language: Python 3.10+
- Orchestration: LangChain
- Data Extraction: Unstructured.io (for PDF and table parsing)
- Vector Database: Pinecone (Serverless)
- LLM: Claude 3.5 Sonnet (Anthropic API)
- Embeddings:
text-embedding-3-small (OpenAI)
Implementation Details
1. Data Ingestion and Document Parsing
Medical reports are often stored in PDF format with complex layouts. We use Unstructured.io to partition the documents, ensuring that tables and hierarchical headers are preserved.
from unstructured.partition.pdf import partition_pdf
def extract_medical_data(file_path: str):
# Partition PDF into elements (text, tables, titles)
elements = partition_pdf(
filename=file_path,
strategy="hi_res",
infer_table_structure=True,
chunking_strategy="by_title",
max_characters=1000,
combine_text_under_n_chars=200
)
return elements
# Convert elements to LangChain Document format
from langchain_core.documents import Document
def prepare_documents(elements):
docs = []
for element in elements:
metadata = element.metadata.to_dict()
docs.append(Document(page_content=str(element), metadata=metadata))
return docs
2. Vector Indexing
Extracted documents are embedded into a high-dimensional vector space. Using Pinecone allows for efficient similarity searches across longitudinal data (e.g., comparing blood glucose levels across five years).
from langchain_pinecone import PineconeVectorStore
from langchain_openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
index_name = "health-brain-index"
def index_documents(documents):
vectorstore = PineconeVectorStore.from_documents(
documents=documents,
embedding=embeddings,
index_name=index_name
)
return vectorstore
3. Retrieval Optimization via HyDE
Medical queries are often brief (e.g., "What does my ALT level mean?"), whereas medical records contain dense technical data. Hypothetical Document Embeddings (HyDE) generates a synthetic answer to bridge the semantic gap between the user's query and the technical content of the records.
from langchain.chains import HypotheticalDocumentEmbedder
from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
# Initialize HyDE with a base embedding model
hyde_embeddings = HypotheticalDocumentEmbedder.from_llm(
llm=llm,
base_embeddings=embeddings,
prompt_key="web_search" # Custom prompts can be injected here
)
def search_health_records(query, vectorstore):
# Perform similarity search using the hypothetical embedding
results = vectorstore.similarity_search(
query,
k=5,
embedding=hyde_embeddings
)
return results
4. Contextual Generation
The final stage involves passing the retrieved context and the original query to Claude 3.5 Sonnet to generate a medically grounded summary.
from langchain_core.prompts import ChatPromptTemplate
template = """
You are a medical data assistant. Answer the question based on the provided context,
including historical lab results and clinical notes.
If the information is missing, state that clearly.
Context: {context}
Query: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
def generate_response(query, context_docs):
context_text = "\n\n".join([doc.page_content for doc in context_docs])
chain = prompt | llm
response = chain.invoke({"question": query, "context": context_text})
return response.content
Advanced Resources
For further technical reading on designing resilient data pipelines and sophisticated architectural patterns for AI-driven healthcare applications, refer to the production guidelines available at WellAlly Technical Blog. These resources offer deeper insights into managing data privacy and scaling RAG systems in HIPAA-compliant environments.
Conclusion
By combining Unstructured.io for precise document parsing and the HyDE strategy for semantic retrieval, we can effectively bridge the gap between human language and complex medical documentation. This RAG architecture ensures that the LLM acts not as a general-purpose model, but as a specialized interface for a user's unique longitudinal health history. The use of Pinecone provides the necessary scalability for managing years of diagnostic data, while Claude 3.5 Sonnet ensures highly accurate and contextually relevant synthesis.
