Hey CoderLegion community,
When connecting LLMs or AI agents directly to production databases or third-party APIs (like HubSpot, Stripe, or GitHub), two major problems usually pop up:
Unsanitized PII/PHI leaks right into the model's context.
Fragmented identities when events come from multiple webhooks and CRMs.
To solve this, I’ve built and released etl-pipeline-mcp — a local-first Model Context Protocol (MCP) server designed to handle high-throughput extraction, transformation, identity resolution, and data warehouse loading without relying on third-party SaaS middle layers.
⚙️ Key Architecture & Features
Decoupled Hot/Cold Storage:
Hot Tier (RAM): High-speed in-memory SQLite buffer to handle high-velocity, real-time webhooks with backpressure throttling.
Cold Tier (Disk): Persistent, on-disk SQLite archive for long-term storage and analytical queries.
Automated PII Redaction & Compliance:
Hashes sensitive fields (emails, IP addresses, customer IDs) using SHA-256 before saving to disk or exporting.
Built-in alignment for GDPR, CCPA, and HIPAA requirements.
Identity Resolution (ID Stitching):
Stitches 5+ platform-specific IDs (Stripe Customer ID, GitHub User ID, HubSpot Lead ID, Social Handles, Email) into a single, unified mcp_root_id.
Multi-Warehouse & Data Lake Exports:
Formats and exports clean records directly to Snowflake, ClickHouse, or BigQuery.
Generates columnar JSON-Parquet buffers for local DuckDB or S3 Data Lake ingestion.
🛠️ Installation & Usage
You can install it directly via PyPI or run it with uv:
Install via PyPI
pip install etl-pipeline-mcp
Or connect it to Claude Desktop or Cursor in claude_desktop_config.json:
{
"mcpServers": {
"seosiri-etl-pipeline": {
"command": "uv",
"args": [
"run",
"--github",
"SEOSiri-Official/etl-pipeline-mcp",
"src/main_server.py"
]
}
}
}
🔗 Links & Resources
PyPI Package: https://pypi.org/project/etl-pipeline-mcp/
GitHub Repository: https://github.com/SEOSiri-Official/etl-pipeline-mcp
Full Architecture Guide: https://www.seosiri.com/2026/07/etl-pipeline-mcp.html
Would love to hear your feedback or ideas for extending the transformation layer!
