Tools and libraries for Large Language Model optimization — built to make AI interactions faster, cheaper, and more efficient.
These projects focus on optimizing the input to Large Language Models before they process your request. While most tools concentrate on what the model outputs, these work on what goes into the model — and that can have a dramatic impact on cost, speed, and quality.
Combined workflow: User request → PCM (compress instruction) → COE (optimize context) → LLM
Compress natural language instructions into a compact, machine-readable format.
Instead of sending long, verbose prompts to your LLM, PCM transforms them into a concise format that conveys the same intent with far fewer tokens.
| Before | After |
|---|---|
Review this Python code carefully for race conditions, memory leaks, and optimization opportunities. Return a Markdown report ordered by severity. |
TASK=review INPUT=python CHECK=race,leak,perf FORMAT=markdown ORDER=severity |
Token savings: 51-81% on instruction tokens
Compact your context without losing facts.
When you provide RAG chunks, chat history, code snippets, or logs to an LLM, COE reorganizes and deduplicates the information, merging related facts and removing redundancy while preserving all essential information.
Token savings: 32-46% on context tokens (benchmarked with factual_recall ≥ 0.95)
Every token saved is money saved. For high-volume LLM applications, these optimizations can reduce infrastructure costs by 30-50% or more.
Smaller inputs mean faster responses and the ability to fit more context within model limits.
By structuring information more efficiently, the model can focus on what truly matters rather than parsing through redundancy.
| Scenario | PCM | COE | Combined |
|---|---|---|---|
| Long prompts with detailed instructions | ✅ High savings | ❌ Not applicable | ✅ Ideal |
| RAG with overlapping documents | ❌ Not applicable | ✅ High savings | ✅ Ideal |
| Chat with long conversation history | ❌ Not applicable | ✅ High savings | ✅ Ideal |
| Code review workflows | ✅ Good savings | ✅ Good savings | ✅ Best |
| Simple questions | ❌ Skip (too short) | ❌ Skip (too short) | ❌ Not needed |
Both projects offer multiple integration paths:
| Project | Language | Dependencies | License |
|---|---|---|---|
| PCM | Python 3.11+ | FastAPI, Ollama, Pydantic | Research prototype |
| COE | Python 3.10+ | Pydantic, NumPy, scikit-learn | MIT |
Both projects are fully open source on GitHub:
Contributions, feedback, and stars are welcome!
These are research prototypes. Use at your own risk. Not production-ready.