Meet Token Saver: An Open-Source MCP Extension Using Local Hybrid RAG to Cut Claude PDF Token Costs 90-99%

AI builders, researchers, and professionals often hit a irritating wall when analyzing giant paperwork with LLMs: the hidden, compounding price of context home windows. Pasting a 200-page PDF right into a chat isn’t a one-time cost. Because the dialog historical past is re-sent to the mannequin on each single flip, that large doc is paid for once more with each follow-up query.

Marktechpost AI group simply launched Token Saver, an open-source Model Context Protocol (MCP) extension for Claude Desktop (MIT licensed, presently at v1.0). It was developed at Marktechpost AI Media Inc by Arnav Rai (CS scholar at Rochester Institute of Technology) throughout his internship at Marktechpost, supervised by Jean-marc Mommessin and Asif Razzaq. Token Saver basically modifications how Claude interacts with native paperwork. By implementing a Local Hybrid RAG (system instantly in your machine, it permits you to ask questions on large PDFs with out ever importing the precise file to the mannequin.

Token consumption is slashed by 92% to 99%, privateness is assured, and setup requires precisely zero Python environments or terminal configurations.

How it appears!

The Hidden Token Drain of Large PDFs

Most customers assume that after they drop a PDF into Claude, it merely extracts the textual content. In actuality, Claude’s default habits converts every web page into a picture to protect charts and layouts, whereas individually extracting textual content. Before a single picture token is even counted, the textual content alone can run 1,500 to 3,000 tokens per web page.

While Prompt Caching and Claude Projects assist soften this blow, they don’t resolve the core subject: your complete doc nonetheless crosses the boundary to the supplier’s servers. Furthermore, should you solely want two related paragraphs from a 1,000-page textbook, forcing the LLM to search your complete 1,000 pages itself is each inefficient and susceptible to hallucination.

How Local Hybrid RAG Solves the Problem

Marktechpost’s Token Saver acts as a neighborhood MCP server : a light-weight background program in your machine that Claude can name as a instrument. The PDF by no means leaves your exhausting drive.

When you ask a query, Token Saver makes use of Local Hybrid RAG to discover the reply. Hybrid RAG is the gold customary for doc retrieval as a result of it combines two highly effective search strategies:

1. Keyword Matching (BM25): Runs over SQLite’s built-in FTS5 search (weighted at 0.4) to discover precise terminology.
2. Semantic Search (Cosine Similarity): Uses a neighborhood all-MiniLM-L6-v2 embedding mannequin (weighted at 0.6) to perceive the which means of your query, fairly than simply matching precise phrases.

By mixing these two approaches regionally, the server searches the file and arms Claude solely the extremely related passages. Claude then synthesizes the reply, citing the precise web page numbers and offering a operating tally of the tokens you simply saved.

The 8-Stage Processing Pipeline

Token Saver runs solely inside a single, long-running native course of. Before any textual content reaches Claude, the native Hybrid RAG pipeline executes eight fast steps:

1. Extract: Uses pypdfium2 (with pypdf as a fallback) to pull textual content.
2. Chunk: Splits textual content into 180-word passages, overlapping by 40 phrases so context isn’t blindly minimize.
3. Score (Hybrid RAG): Evaluates chunks utilizing the blended BM25 and native embedding mannequin.
4. Gate: Enforces a top quality threshold. Passages sharing no precise key phrases should go a semantic similarity flooring of 0.25 to qualify.
5. Deduplicate: Drops near-identical passages.
6. Trim: Narrows the passage down strictly to the sentences that reply the consumer’s immediate.
7. Budget: Caps the overall payload despatched to Claude at 8,000 characters.
8. Envelope: Wraps the retrieved textual content in a document-chunk aspect containing the supply file and exact web page quantity.

(Note: The embedding mannequin is non-compulsory however really helpful. If it fails to load, the system gracefully falls again to keyword-only matching).

The Numbers: 99% Savings at Scale

Because the Hybrid RAG pipeline caps the returned textual content slice, the token financial savings develop exponentially with the dimensions of the doc. Here is how Token Saver carried out in benchmarking (measured by way of tiktoken, assuming one query per doc):

Document	Pages	Whole Doc Tokens	Returned Tokens	Tokens Saved
FDA drug label	33	23,959	1,021	95.7%
GDPR (EU 2016/679)	88	70,260	996	98.6%
SFFA v. Harvard	233	133,349	740	99.4%

Disclaimer: Token counts use cl100k_base as a stand-in, and baselines assume the entire doc is charged on each search.

For professionals working repeatedly with court docket opinions, technical requirements, monetary stories, or dense textbooks, Token Saver eliminates the repetitive token tax.

Security and Local Privacy

For enterprise customers and authorized professionals, knowledge privateness is non-negotiable. Token Saver’s safety mannequin rests on three pillars:

• Zero Uploads: The doc by no means leaves your machine.
• Folder Allowlisting: You configure a selected folder for Token Saver. The server will actively refuse to learn recordsdata outdoors of this designated listing.
• Network Isolation: The server communicates with Claude Desktop over customary I/O (stdio) solely. There are not any listening community ports, drastically lowering the assault floor.

Model Performance: Sonnet vs. Opus

Marktechpost’s Token Saver works throughout all three Claude 3.5 tiers, however testing revealed distinct behaviors:

• Claude 3.5 Sonnet (Recommended): The smart default. It handles unfastened file references completely, asks clarifying questions if two recordsdata have related names, and appropriately applies the retrieved web page numbers.
• Claude 3 Opus: Excels at advanced paperwork with a number of voices (e.g., authorized rulings with majority opinions and dissents). Opus is wise sufficient to flag when it infers an attribution based mostly on context fairly than express textual content.

Getting Started in Minutes

Unlike many open-source AI instruments that require advanced Python environments and JSON configurations, Token Saver is packaged as a single .mcpb bundle.

1. Download token-saver-ccr.mcpb from the venture’s GitHub Releases web page.
2. Open Claude Desktop -> Settings -> Extensions -> Install extension.
3. Enable the extension, click on Configure, and choose a devoted folder the place you retain your reference PDFs.
4. At the primary immediate, select Always permit.

The extension won’t run till a folder is chosen, as a result of that single selection serves three functions directly.

That folder is value a second’s thought. It units the boundary of what could be learn, it’s what allows you to ask for a file by identify as an alternative of typing a path, and it’s the checklist the server reveals Claude when a dialog begins. A small folder holding solely what you propose to ask about works significantly higher than pointing it in any respect of Documents.

Key Takeaways

• Massive Cost Reduction: By utilizing Local Hybrid RAG to go solely related paragraphs to the LLM, token prices drop by up to 99%. The longer the chat, the extra you save.
• Verifiable Accuracy: Because Token Saver attaches precise web page numbers to each retrieved chunk, you possibly can immediately confirm Claude’s claims by opening your native PDF.
• Absolute Privacy: The boundary is your native machine. Your proprietary knowledge isn’t uploaded to an AI supplier’s server.
• Frictionless Setup: No Python required. A easy .mcpb file set up will get you operating in Claude Desktop immediately.

---

Check out the GitHub Repo.

References: MCP Bundle format | all-MiniLM-L6-v2 | pdfkb-mcp | wesleygriffin/pdfrag | Corpora: Dobbs v. Jackson Women’s Health Organization; Berkshire Hathaway 2023 Annual Report

The submit Meet Token Saver: An Open-Source MCP Extension Using Local Hybrid RAG to Cut Claude PDF Token Costs 90-99% appeared first on MarkTechPost.