In 2026, the AI open source scene is no longer just "another ChatGPT wrapper." What is actually climbing GitHub stars is infrastructure that turns messy PDFs into clean Markdown, gives agents memory across sessions, and even lets a 4GB GPU run 70B. If you are building RAG, an enterprise knowledge base, or a multi-agent product, picking the right open components often beats swapping to a bigger closed model.
Drawing on recent Kvmzen deployments and community signals, this guide highlights ten AI open source projects worth watching in 2026 — organized across PDF processing, agent memory, ultra-low-VRAM inference, orchestration, and local deployment — with selection criteria, stack combinations, common pitfalls, and how cloud Mac fits in.
Selection criteria: what counts as "trending"
We did not sort by star count alone. Each project had to pass four checks:
- Active maintenance in the last 12 months — steady releases, responsive issues, updated docs
- Solves a real pain point — garbled PDFs, agents that forget, not enough VRAM, fragmented API vendors
- Plugs into existing stacks — Python SDK or OpenAI-compatible API, not a closed black box
- Observable in production — logging, metrics, or a clear experimental vs production positioning
If you are learning agent development systematically, start with our AI Agent Learning Roadmap (2026): Essential Books, Courses, and Open Source Projects, then work through this list hands-on.
PDF and document processing trio
In RAG, garbage in, garbage out usually starts with PDF parsing. The 2026 consensus: no single parser wins every corpus — but these three cover about 80% of cases.
1. Docling (IBM)
MIT-licensed and maintained by IBM Research. Handles PDF, DOCX, PPTX, HTML, and more. Strengths: structured recovery of tables, headers/footers, and multi-column layouts, outputting JSON or Markdown for downstream indexing. Fits enterprise knowledge pipelines — Docker-friendly, with official LangChain and LlamaIndex examples.
2. Marker
Focused on high-quality PDF and EPUB to Markdown conversion. Handles academic papers and technical whitepapers — equations and footnotes — better than most general OCR. On GPU, throughput can reach dozens of pages per minute. If your corpus is mostly English PDFs headed for clean Markdown chunks, Marker is often the best first step.
3. MinerU
An open source option with strong community results on Chinese scanned documents, complex tables, and textbook layouts. Pairs well with Docling: MinerU for OCR-heavy pages, Docling for unified schema output.
Agent memory: Mem0 and Zep
An agent without memory treats every conversation like a first meeting. In 2026, the memory layer has moved beyond "stuff the full history into context" toward retrievable, updatable, auditable services.
4. Mem0
Marketed as a "memory layer for AI apps." User- and session-level long-term memory APIs extract facts from dialogue and deduplicate them. Integration docs for LangGraph, CrewAI, and AutoGen are solid — fastest path to "remember user preferences." Good for personal assistants, coding agents, and SaaS prototypes.
5. Zep
Through 2025–2026 Zep has pushed Graphiti temporal knowledge graphs: not just "user prefers dark mode," but when a fact became true and when it was overturned — with graph queries and compliance-friendly audit trails. Fits support, CRM, and healthcare-style workflows that need traceable memory chains. Heavier to deploy and model than Mem0.
70B on 4GB VRAM: AirLLM
6. AirLLM
The "4GB GPU running 70B" headline in this guide mostly points to AirLLM. Layer-wise inference keeps weights on SSD and loads one Transformer layer at a time on the GPU. Llama 3.1 70B peaks around 4GB VRAM, with Apple Silicon + MLX support. Trade-offs: typically 0.5–3 tokens/s and a first-run download of roughly 130GB of sharded weights.
We published a full benchmark the same day — see Can a 4GB GPU Run a 70B Model? AirLLM Benchmarks and Setup Guide. Short version: AirLLM is a feasibility tool, not a production API stack.
Agent orchestration: LangGraph and CrewAI
7. LangGraph
From the LangChain team: stateful graph orchestration for agents. Model flows as nodes and edges with native loops, human-in-the-loop interrupts, and persistent checkpoints. By 2026 it is a de facto standard for backend agent services that need fine-grained control over tool order and retries.
8. CrewAI
Abstracts multi-agent work as roles and delegated tasks — define Researcher, Writer, Reviewer in YAML. Gentler learning curve than LangGraph, excellent for prototypes and demos. Production setups often mix both: CrewAI for role split, LangGraph for state machines and observability.
Local deployment: Ollama and LiteLLM
9. Ollama
One command to pull and run open weights on macOS, Linux, and Windows. In 2026 it supports a wide range of quantized and multimodal variants — the default first machine for local development. Pair with Open WebUI for an internal team chat surface.
10. LiteLLM
A unified OpenAI-compatible gateway across Ollama, Anthropic, Azure, Bedrock, and more — one API surface with routing, retries, and cost tracking. When Cursor, CLI tools, and in-house agents share models, LiteLLM avoids configuring keys per client. Conceptually similar to OmniRoute-style gateways on this blog, but focused on the LLM inference layer.
Side-by-side comparison and recommended stacks
| Project | Category | Best for | Main trade-off |
|---|---|---|---|
| Docling | Enterprise multi-format pipelines | Complex scans need upstream OCR | |
| Marker | English academic PDF → Markdown | Weaker on Chinese and handwriting | |
| MinerU | Chinese scans, textbooks | Fewer English-ecosystem integrations | |
| Mem0 | Memory | Quick user long-term memory | Limited complex graph reasoning |
| Zep | Memory | Temporal facts and audit | Heavier ops burden |
| AirLLM | Inference | 4GB VRAM 70B validation | Very slow, large disk |
| LangGraph | Orchestration | Stateful production agents | Steeper learning curve |
| CrewAI | Orchestration | Multi-role prototypes | Less control than LangGraph |
| Ollama | Deploy | Local model pulls for dev | Large models still need RAM/disk |
| LiteLLM | Gateway | Unified multi-vendor API | You own high availability |
Recommended minimum viable stack: Ollama for models → LiteLLM for an OpenAI-compatible endpoint → LangGraph for tools → Mem0 for memory → Marker for uploaded PDFs. A two-week end-to-end demo is realistic.
How this ties to cloud Mac and Apple Silicon
Most of these projects run natively on macOS, but disk, memory, and long-running inference still bite:
- Batch PDF jobs — Marker and MinerU are faster on GPU; Mac mini can run CPU paths, but large batches are better on an overnight cloud node
- 70B experiments — AirLLM works on Apple Silicon via MLX; 24GB unified memory beats a 4GB discrete card by an order of magnitude
- Long agent tasks — LangGraph checkpoints and Ollama model files can consume tens of gigabytes; a cloud Mac gives fixed environment and SSD headroom
iOS and Flutter developers on Windows can move Ollama + LangGraph experiments to a cloud Mac for Unix tooling and Apple Silicon inference without maintaining WSL and driver compatibility.
Cost, performance, and risk
The hidden bill on a "free" open stack is engineer time, power, and disk. A 4GB card plus AirLLM looks like zero API spend, but 130GB models and sub-1-token speeds stretch validation cycles. A monthly cloud Mac often pays back in two to three weeks of saved time. Pay-per-token APIs suit uncertain early traffic; run Ollama locally for dev and route production through LiteLLM.
FAQ
For PDF parsing in 2026, Docling or Marker?
Docling for enterprise multi-format pipelines; Marker for English academic PDF to Markdown; MinerU first for Chinese scans. A/B on 20 pages beats reading star counts.
Mem0 or Zep?
Mem0 for fast user memory; Zep for temporal facts, relationship reasoning, and audit. They can coexist — Mem0 for preferences, Zep for business fact graphs.
Can a 4GB GPU run 70B?
Yes — AirLLM layer-wise inference peaks around 4GB VRAM at 0.5–3 tokens/s with large SSD needs. See our AirLLM benchmark article for full numbers.
Simplest local agent stack?
Ollama + LiteLLM + LangGraph + Mem0 + any PDF parser; macOS has the fewest dependencies, Windows users should consider cloud Mac or WSL.
Assemble this open stack on Mac mini with less friction
Most of the ten projects here run natively on macOS: Homebrew for Python, Ollama for one-command model pulls, MLX to accelerate AirLLM and local inference. Apple Silicon unified memory lets a 24GB Mac mini handle PDF batch jobs and quantized 70B without wrestling CUDA and WSL on Windows. M4 idle power is about 4W — practical for long agent runs and overnight indexing.
Against similarly priced PCs, macOS's low crash rate plus Gatekeeper and FileVault make a better shared remote dev node; the small, fanless chassis also lowers always-on costs. If you develop iOS or Flutter on Windows, moving this AI lab to a cloud Mac often beats buying another GPU.
From PDF parsing to agent memory — from 4GB validation to production orchestration — hardware should not cap your experiments — view our plans and run the full 2026 open stack on a cloud Mac mini in one pass.