ranxianglei/billion-context
A context-compression plugin for month-long single sessions (billions of tokens) and small context windows (a 100K context is enough), billion-context is all you need .
About ranxianglei/billion-context
ranxianglei/billion-context is an open-source project on GitHub, mainly written in TypeScript. A context-compression plugin for month-long single sessions (billions of tokens) and small context windows (a 100K context is enough) It currently holds 7,332 stars and 606 forks with 1,141 open issues, and was last pushed on 2026-11-14 (repository created 2026-08-06).
Project Overview
Git Homed tracks it on the Today's Trending board.
GitHub Repository Details
README
billion-context
Context-compression plugin — billion-context is all you need.
small context windows (100K is enough) · 5× fewer tokens · month-long single sessions (billions of tokens) · high compression quality
npm install -g billion-context --prefix=~/.local
---
Cache health at a glance: a healthy session keeps a 95–97% prefix-cache hit rate — compression itself costs ≤2%. Sustained lower? Check attribution with/acpor/acp-cache(see FAQ); usual causes, in order: upstream cache TTL expiry · model switch · a bili bug (please report) · other/unknown.
Community
QQ Group: 1056132097 (full) 1108730198 (full) 1129868472 (open)
---
📄 Paper / Preprint
- Model-Driven Incremental Hierarchical Compression: Training-Free Multi-Generational Context Management for Long-Lived Coding Agents (English, v0.2)
📝 The paper itself is open-sourced under the MIT License as part of the codebase (paper/). It is a living document — anyone may edit it; improvements are welcome via pull request.
A production-scale longitudinal study: 4.5 months, three hosts, 174,327 model calls, 18.76B cumulative input tokens (~24.7B across all hosts), zero window violations on 204,800-token models, marathon sessions of 8,584–12,049 calls.
📊 Independent Third-Party Benchmarks
- 1,500-Item Cross-Benchmark Evaluation (
LongMemEval-S·ContextBench·MemoryAgentBench) (中文版) — head-to-head comparison ofbillion-context(v0.1.202) againstfullcontext and 3 RAG arms (bm25,embedding,embedding-rerank) withDeepSeek V4.1 Flash,Qwen3-Embedding-8B,Qwen3-Reranker-4B, andPi Agent: #1 on ContextBench (0.3097Utilized F1,-58.3%cost vsfull), +16.6% over best RAG on LongMemEval-S (79.20%vs62.60%,-71.1%independent-session cost vsfull), and 100% survival on 1.48M-token MemoryAgentBench TTL (30×steady-state compression wherefullfails199/199).
billion-context sits between any agent and its model API, rewriting Anthropic/OpenAI streams with acp-kernel compression. The model decides when and what to compress into high-fidelity summaries — not a hard truncation limit.
Why
Long coding sessions blow up context. Each provider charges per token, and once you pass the context window the session degrades or dies. billion-context compresses consumed conversation into layered summaries so you can run a single session for days — billions of tokens through one context window.
Unlike a host's built-in summarizer, compression here is incremental, reversible, and prefix-cache friendly: summaries are written in small ranges, can be decompressed on demand, and the cache prefix stays intact.
How it works
Context-management tools
The proxy injects four context-management tools into the conversation; the model calls them itself as context grows, and the proxy executes compress server-side so folded ranges stay summarized in history until restored:
compress— fold a message range into a detailed summary.decompress— restore a compressed range when exact details are needed again.search_context— keyword search over compressed summaries and visible messages.acp_status— context-usage overview plus which ranges are still compressible.
Which do I need?
Pick by your client:
| Client | Use |
|---|---|
| pi | billion-context — bili pi (launcher) or bili plugin install pi (native); standalone billion-context-pi remains usable — details: docs/clients.md |
| opencode (1.x / 2.x) | billion-context — bili opencode (launcher) or bili plugin install opencode (native); standalone opencode-acp remains usable on 1.x — full guide: OpenCode |
| omp | billion-context via bili omp (built-in plugin) or bili plugin install omp (self-spawning native plugin, no launcher) |
| dsh | bili dsh (launcher — full native plugin via --patch) or bili plugin install dsh ≡ dsh plugin --profile add billion-context (one unified lane) — details: docs/clients.md |
| kimi | bili plugin install kimi (self-spawning native, Kimi Code ≥ 2.0.0) or bili kimi (cert-MITM) or /bili/ prefix — details: docs/clients.md |
| hermes | bili plugin install hermes (self-spawning native, Python plugin #958) or bili hermes (cert-MITM) |
| zcode (Z.ai / bigmodel coding plan) | bili plugin install zcode (self-spawning native, #1145) or cert-MITM via the GUI's Settings → Network or /bili/ prefix — details: docs/clients.md |
| claude | bili claude (launcher) or bili plugin install claude (native posture, #964 — managed settings block + session-owned proxy; see the notes below) |
| codex | bili codex (launcher — the full zero-config posture) or bili plugin install codex (MCP-shell tools companion: start bili first — the shell never spawns a proxy and never routes codex's own traffic) — details: docs/clients.md |
| jcode | billion-context via bili jcode (cert-MITM) or /bili/ prefix — no native mode (compiled Rust binary, no plugin seam, #962) |
| gemini (Gemini CLI) | bili gemini (launcher, GOOGLE_GEMINI_BASE_URL /bili/ rewrite) or /bili/ prefix — launcher-only (no in-loop tool seam, #1043) |
| iflow (iFlow CLI) | bili iflow (launcher, IFLOW_BASE_URL /bili/ rewrite) or /bili/ prefix |
| qwen (Qwen Code) | bili qwen (launcher, cert-MITM) or /bili/ prefix |
| antigravity (Antigravity CLI / agy, Google) | bili antigravity (launcher, CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — Antigravity's real default upstream, #3129) or /bili/ prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: docs/clients.md |
| mcode (MiniMax Code) | billion-context via bili mcode (cert-MITM) or /bili/ prefix — no native mode (event hooks only, no model-request seam, #1050) |
| aider | billion-context via bili aider (cert-MITM) or /bili/ prefix — no native mode (shell-command-only hooks, no tool-injection seam, #1048) |
| copilot (GitHub Copilot CLI) | bili copilot (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) |
| amp (Amp CLI) | bili amp (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) |
| kiro (Kiro CLI, AWS) | bili kiro (launcher, cert-MITM) — Rust shell + Node agent server, no plugin seam; model traffic is the Responses wire on runtime..kiro.dev, gateway quirks handled built-in — details: docs/clients.md |
| crush (Charm Crush) | bili crush (launcher, cert-MITM) — open-source Go binary, no plugin seam; built-in provider hosts whitelisted, custom base_urls auto-discovered from crush.json (#2340) |
| zed (Zed editor) | bili zed (launcher, cert-MITM) — open-source Rust editor, no plugin seam; reqwest honors HTTPS_PROXY, CA via SSL_CERT_FILE (Linux env probing); built-in provider hosts whitelisted, custom api_urls auto-discovered from settings.json; loopback providers (ollama/lmstudio) stay direct via NO_PROXY (#2340) |
| goose (Goose CLI) | bili goose (launcher) — rustls trusts no CA file, so no cert-MITM: openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (#1049) |
| everything else (no context hook) | billion-context — bili (launcher, preferred) or /bili/ prefix |
Native mode vs standalone extensions. The host-native plugins (bili plugin install …) and the standalone in-process extensions (billion-context-pi, opencode-acp) are mutually exclusive — both active means double compression. The installer makes the switch: it replaces the legacy entries (bare name, npm: alias, versioned, path form; array or object shape) and snapshots the original config to .bili-bak once; a project-local install is not touched — remove that one by hand. As a runtime safety net for manual installs, the native entries set BILLION_CONTEXT_NATIVE= synchronously at load so a standalone extension can stand down at action time. On the pi side the marker needs billion-context-pi 0.1.72+, and the pi-native entry scans both pi settings files once its proxy is up and warns loudly when it spots a co-resident legacy entry the installer never saw — that warning is the only visible signal while an old billion-context-pi silently double-compresses.
Install
Linux / macOS — install with a user-level prefix (no sudo, no npm config
changes, and bili's self-update never hits permission errors):
npm install -g billion-context --prefix=~/.local
The bili command lands in ~/.local/bin — already on PATH in most distros;
if not, add export PATH="$HOME/.local/bin:$PATH" to ~/.bashrc or
~/.zshrc. On nvm or Homebrew Node the default prefix is already user-owned —
a plain npm install -g billion-context works as-is. On Windows the default
prefix (%APPDATA%\npm) is also user-writable — plain npm install -g billion-context.
This installs the bili command (bili-proxy is kept as an alias). Hitting
EACCES with an old root-owned prefix? Reinstall with --prefix=~/.local
(pass the flag again on any future npm reinstall of bili) — that is the
permanent fix; avoid sudo.
Prefer containers? A Dockerfile ships at the repository root — see
Running in Docker below.
Quickstart
Three ways to use it — pick one:
- Native plugin (most native):
bili plugin install— bili
- Launcher (config-free): one
bilicommand brings up the proxy and
- URL change (most universal): prefix your client's baseURL with the proxy
/bili/.
Mechanism details behind these three options (plugin lifecycle, runtime-info protocol, injection priority) live in docs/technical-notes.md.
Ports, briefly (#1660): bili start (manual) owns 8787. Everything a lane
spawns for you (native hooks, launcher lanes) lives in a separate
self-managed zone starting at 18787 — collisions hop +1 and each lane
remembers its drift, so zero-config installs never fight you for a port,
and a deliberate bili start daemon is attached by default. Attach is
lane-aware (#2905): a hook rides its own lane's instance first, NEVER
another declared lane's — except lanes you list in
native.attachLanePriority — and the 8787 user-zone daemon stays the
LAST-resort fallback. An
upgrade-restart that finds the previous build still draining on the lane's
port waits for it to release (up to 5s) and rebinds the SAME port instead of
drifting (#1723); only a genuinely occupied port hops +1 — and that hop is
now logged loudly.
Option 1 — Native plugin (bili plugin install pi / omp / opencode / dsh / kimi / hermes / zcode)
The proxy lives inside the client: install once, then start the client exactly as you always do — no launcher command, no env vars, no fixed port, no URL edits. Supported today for pi, omp, opencode (1.x and 2.x), dsh, kimi, hermes and zcode:
bili plugin install pi # registers a "billion-context" entry in pi's settings (npm form when bili itself was npm-installed)
bili plugin install omp # registers an extensions entry in omp's config.yml (~/.omp/agent/config.yml)
bili plugin install opencode # registers the plugin in opencode's real config + disables native auto-compaction
bili plugin install dsh # runs 'dsh plugin --profile add billion-context' for every existing profile
bili plugin install kimi # writes $KIMI_CODE_HOME/plugins/managed/billion-context/kimi.plugin.json (+ installed.json record); per-session routing block lands in config.toml on first start (Kimi Code >= 2.0.0)
bili plugin install hermes # copies the Python plugin into ~/.hermes/plugins/billion-context/ (+ machine-owned bili.json sidecar) and enables it via hermes plugins enable billion-context
bili plugin install zcode # writes hooks.enabled + a SessionStart hook + mcp.servers.bili into ~/.zcode/cli/config.json; per-session routing lands in the bigmodel provider store on first start
bili plugin remove # undo (dsh removes through the same channel; config snapshots go to .bili-bak)
Where a client has its own plugin channel you can also install natively, skipping bili commands entirely:
- dsh:
dsh plugin --profile add billion-contextis the very
bili plugin install dsh drives per profile (npm form only; a git
checkout has no published entry — pasting a GitHub address installs source
without dist/ and the bundle never loads) — same end state either way
(pnpm into the profile, bundled patch layer mounted by dsh itself); remove
through the same channel. See the dsh section below.
- opencode: easiest is
bili plugin install opencode(it picks the
- OpenCode 1.x — add the bare npm name to your real config's plugin
"plugin": ["billion-context"]. The package publishes
exports["./server"] → dist/agent/opencode-native.js, so opencode
loads it through its own Npm.add machinery and the plugin self-spawns
exactly like the bili-installed form.
- OpenCode 2.x — the bare name does NOT work: 2.x resolves it through
main field (dist/index.js, the CLI — no plugin
definition), so the module load fails (#3976). Instead, put a one-line
re-export in a directory and point the config's plugins list at that
directory:
// /index.js
export { default } from "/dist/agent/opencode-native.js";
{ "plugins": [""] }
Either generation: do the two things the bili installer would have done
for you too — set "compaction": { "auto": false } in the same config
(otherwise OpenCode's native auto-compaction double-compresses) and keep a
manual backup of the file first.
- claude: this repository doubles as a Claude Code plugin marketplace —
/plugin marketplace add ranxianglei/billion-context, then
/plugin install billion-context@billion-context, then run
/billion-context:bili-setup (it drives bili plugin install claude for
you and tells you to restart). Same end state as the bili installer; the
plugin ships no hooks or MCP entries of its own, so nothing
double-registers.
For pi / omp / kimi / claude there is no client-side channel — `bili plugin
install ` writes their config entries for you (kimi's declarative
kimi.plugin.json + registry record, claude's managed settings block, …).
Notes:
- Native mode is mutually exclusive with the standalone in-process extensions (
billion-context-pi,opencode-acp) — the installer swaps the entries and snapshots the original config (.bili-bak). - pi: native installs wire in built-in subagents by default (
acp_delegate/acp_delegate_wait/acp_delegate_cancel) — how to disable them while keeping context compression: Pi built-in subagents. - OpenCode legacy sessions, V1/V2 shapes and caveats: OpenCode.
kimireports runtime-info at bootstrap only (static headers can't carry per-request window/model values); subagent tool calls are routed by the proxy's outbound tool-use witness ring (#1685) — no model-visible conversation id.hermes's native plugin is Python: it points hermes' httpx stack at the proxy via env vars after a health check and stamps per-request headers through anllm_requestmiddleware.codexis the one client a plugin install cannot make self-sufficient: codex routes model traffic via env only (no config-file routing seam for the default ChatGPT-login provider — a managedmodel_providersblock would force API-key auth and drop subscription login), and an MCP server cannot inject env into its parent.bili plugin install codexwrites a single[mcp_servers.bili]block into~/.codex/config.toml(command = node, args = dist/mcp.js) exposing the four ACP tools; at session start the shell resolves a proxy — envBILI_MCP_PROXY> the live-instance record (any lane's proxy or abili startdaemon) > the 8787 user-zone default (#1660 removed the install-time origin bake, #403) — nothing reachable →tools/listfails with -32003. So: start bili first (bili startor any client's lane proxy), export HTTPS_PROXY yourself if you also want compression, or usebili codexfor the zero-config full posture. Mechanics: docs/clients.md.claudehas a native posture (#964): managed settings block +SessionStarthook + MCP shell; the hook rides the self-managed port zone (#1660) and re-pins the managed URL to the live origin each session, so port drift self-heals. Since #1821 the MCP shell also watches the proxy mid-session (30 s tick × 3 failures) and respawns a dead lane proxy itself, so model requests recover within ~90 s instead of a new session (same semantics as the zcode lane). Opt out withBILI_NATIVE_CLAUDE=0(passthrough). Mechanics: docs/technical-notes.md. **Known li


