hydra-db/hydradb

▲ 10,023 stars today★ 13,046⑂ 5,483

HydraDB - fast graph database on object storage

About hydra-db/hydradb

hydra-db/hydradb is an open-source project on GitHub, mainly written in Rust. HydraDB - fast graph database on object storage It currently holds 13,046 stars and 5,483 forks with 152 open issues, and was last pushed on 2026-09-30 (repository created 2026-07-03).

Project Overview

Git Homed tracks it on the Today's Trending board.

GitHub Repository Details

Repository hydra-db/hydradb · default branch main · size 1611 KB · watchers 9 · source: GitHub REST API and repository README

README

HydraDB

License: AGPL-3.0 Rust 1.91+

HydraDB is an object-store-native distributed graph database written in Rust. It combines durable graph storage on SlateDB with snapshot-consistent OpenCypher queries, GraphBLAS traversal, Neo4j-compatible Bolt connectivity, and an HTTPS query API.

S3-compatible object storage is the durable source of truth. Query nodes and indexers keep only disposable state in memory and on local SSD or NVMe, so they can be replaced or scaled without moving the graph itself.

Why HydraDB

traversal indexes live in S3-compatible storage. rebuild their local caches from durable state. each cell, while SlateDB writer epochs fence stale writers. Indexed traversal combines a compiled CSC generation with its visible WAL overlay. adjacency, sparse traversal, and SuiteSparse GraphBLAS where appropriate. typed JSON and streaming NDJSON HTTP API. limits, backpressure, cancellation, cache budgets, metrics, and traces are part of the server runtime.

Architecture

flowchart LR
    C["Applications
Neo4j drivers or HTTPS"] SVC["Service or load balancer"]

subgraph Q["Query tier"] Q1["graph-node"] Q2["graph-node"] QN["graph-node"] end

subgraph I["Indexing tier"] IX1["graph-indexer"] IXN["graph-indexer"] end

CACHE["Disposable memory and SSD cache"] STORE["S3-compatible object storage
WAL, SSTs, leases, CSC generations"]

C --> SVC SVC --> Q1 SVC --> Q2 SVC --> QN Q1 <--> CACHE Q2 <--> CACHE QN <--> CACHE Q1 <--> STORE Q2 <--> STORE QN <--> STORE IX1 <--> STORE IXN <--> STORE

Query nodes serve reads and canonical graph mutations. Indexer workers build immutable CSC generations asynchronously and publish them through atomic object-store pointers. Readers remain correct when an index is absent or behind because the visible WAL tail is applied to the indexed base.

See architecture.md for the storage model, query pipeline, writer coordination, index lifecycle, and failure semantics.

Getting Started

Prerequisites

HydraDB requires Rust 1.91 or newer, a C/C++ toolchain, libcypher-parser, and SuiteSparse GraphBLAS.

Ubuntu or WSL:

sudo apt-get update
sudo apt-get install -y \
  build-essential clang libclang-dev cmake pkg-config \
  libcypher-parser-dev libgraphblas-dev \
  curl git python3 python3-venv

The last line is not needed to build, but the steps below use it: curl for the Rust installer and the readiness checks, git to clone, and python3-venv for the Neo4j driver used by scripts/runtime_smoke.sh.

macOS with Homebrew:

xcode-select --install
brew install just cmake pkg-config llvm suite-sparse
brew install cleishm/neo4j/libcypher-parser

Rust, only if rustup toolchain list does not already show a stable toolchain:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

libcypher-parser is not in homebrew-core; the fully-qualified cleishm/neo4j/... name adds the tap automatically. A plain brew install libcypher-parser fails with No available formula.

Rust comes from the official installer rather than Homebrew because the rustup formula is keg-only and no longer ships a rustup-init binary, so brew install rustup leaves nothing named rustup on PATH. rust-toolchain.toml pins channel = "stable", so any rustup-managed stable toolchain works.

No PKG_CONFIG_PATH export is needed: libcypher-parser is not keg-only, so Homebrew links cypher-parser.pc into the default pkg-config search path.

just is the supported command runner for the repository. Install it with cargo install just --locked when your package manager does not provide it. Docker is optional and is used only by MinIO, Neo4j comparison, image-build, and Kubernetes harnesses.

Clone and verify

git clone https://github.com/hydra-db/hydradb.git
cd hydradb

just native-check just smoke

The smoke example creates a local graph, writes and deletes edges, runs a sparse traversal, closes the database, reopens it, and verifies the durable result. The recipe creates and removes an isolated local object-store directory. Use just smoke-graphblas to pin the traversal kernel to SuiteSparse GraphBLAS.

To exercise the same flow against an ephemeral MinIO instance:

just minio-smoke

Run a local server

The following starts a single plaintext development node backed by a local directory. TLS is required by default in deployed environments; plaintext must be enabled explicitly.

mkdir -p .hydradb/store .hydradb/cache
printf '%s\n' 'local-development-token-32-bytes' > .hydradb/auth-token

export CLOUD_PROVIDER=local export LOCAL_PATH="$PWD/.hydradb/store" export GRAPH_NAMESPACE=default export GRAPH_ID=default export GRAPH_CELL_ID=cell-0 export GRAPH_CELLS=cell-0 export GRAPH_NODE_ID=node-0 export GRAPH_BOLT_NODE_ADDRESSES=node-0=127.0.0.1:7687 export GRAPH_ADVERTISED_BOLT_ADDR=127.0.0.1:7687 export GRAPH_DATA_CACHE_DIR="$PWD/.hydradb/cache"

Cache chunks default to 4 MiB. Smaller parts reduce cold probe overfetch,

but can increase later cache misses; benchmark before changing this value.

The value must be a positive multiple of 1024.

export GRAPH_DATA_CACHE_PART_BYTES=4194304

Process-wide budget retained by SlateDB disk-cache file caches. Scoped

runtimes divide it across every possible cell reader and writer.

export GRAPH_DATA_CACHE_MAX_OPEN_FILES=512 export GRAPH_AUTH_TOKEN_FILE="$PWD/.hydradb/auth-token" export GRAPH_ALLOW_PLAINTEXT=true

graph-node's async query futures exceed the default thread stack. Without

this the node builds, serves /readyz, and then aborts on the first query.

export RUST_MIN_STACK=33554432

macOS: cargo is invoked directly here, so it does not inherit what the

justfile exports. Linux installs these on default search paths already.

if command -v brew >/dev/null; then export BINDGEN_EXTRA_CLANG_ARGS="-I$(brew --prefix)/include" export LIBRARY_PATH="$(brew --prefix)/lib" fi

cargo run --locked --features server-runtime --bin graph-node

The node runs in the foreground and does not return; that is it working, not hanging. Run the checks below from a second shell.

The node listens on:

| Endpoint | Address | Purpose | |---|---|---| | Bolt | 127.0.0.1:7687 | Neo4j-driver-compatible queries | | HTTP | 127.0.0.1:8443 | JSON and NDJSON query API | | Admin | 127.0.0.1:9090 | readiness and Prometheus metrics |

In another terminal, write and read a small graph through HTTP:

TOKEN='local-development-token-32-bytes'

curl -sS http://127.0.0.1:8443/v1/graphs/default/query \ -H "Authorization: Bearer $TOKEN" \ -H 'X-Graph-Namespace: default' \ -H 'Content-Type: application/json' \ --data '{"cell_id":"cell-0","query":"CREATE (a {id: 1})-[:FOLLOWS]->(b {id: 2})"}'

curl -sS http://127.0.0.1:8443/v1/graphs/default/query \ -H "Authorization: Bearer $TOKEN" \ -H 'X-Graph-Namespace: default' \ -H 'Content-Type: application/json' \ --data '{"cell_id":"cell-0","query":"MATCH (a {id: 1})-[:FOLLOWS]->(b) RETURN b.id AS id"}'

The second call returns one row containing {"type":"vertex_id","value":2}. A listening port is not proof the node works; a round-tripped write is.

For a scripted Bolt and HTTP round trip, install the Python Neo4j driver and run. Homebrew's and Debian's Python both refuse a bare pip install under PEP 668, so use a virtualenv (apt-get install -y python3-venv on Debian/Ubuntu):

python3 -m venv /tmp/hydradb-venv && /tmp/hydradb-venv/bin/pip install neo4j

macOS: this script calls cargo directly, so it does not inherit what the

justfile exports. Without this it fails at bindgen with

'cypher-parser.h' file not found. Linux needs neither.

if command -v brew >/dev/null; then export BINDGEN_EXTRA_CLANG_ARGS="-I$(brew --prefix)/include" export LIBRARY_PATH="$(brew --prefix)/lib" fi

PYTHON=/tmp/hydradb-venv/bin/python bash scripts/runtime_smoke.sh

Prints runtime-smoke-ok. The node's log is at /tmp/sgk-runtime-smoke/node.log; read it first if the script fails.

Troubleshooting local runs

| Symptom | Cause and fix | |---|---| | No available formula with the name "libcypher-parser" | Use the tap: brew install cleishm/neo4j/libcypher-parser | | command not found: rustup-init | Homebrew's rustup is keg-only and no longer ships it; use the official installer above | | invalid environment variable CLOUD_PROVIDER value \null\` | CLOUD_PROVIDER is unset — null means absent, not the string. local also needs LOCAL_PATH`, pointing at a directory that already exists | | wrapper.h:4:10: fatal error: 'cypher-parser.h' file not found | BINDGEN_EXTRA_CLANG_ARGS unset while invoking cargo directly on macOS. Prefer just, which exports it | | Node answers /readyz, then aborts with has overflowed its stack on the first query | RUST_MIN_STACK unset; export 33554432 | | curl: (7) Failed to connect ... port 9090 | The node is not running. graph-node holds the foreground, so start it in its own shell |

Querying

HydraDB supports a practical OpenCypher subset for graph reads and mutations, including typed relationships, bounded variable-length paths, property and label predicates, ordering, pagination, aggregation, OPTIONAL MATCH, UNION, and batched UNWIND writes.

Applications can connect with a Neo4j driver using a routed URI:

neo4j://127.0.0.1:7687

Use neo4j+s:// with a publicly trusted certificate or neo4j+ssc:// for a self-signed development certificate. Direct bolt:// node addresses are for diagnostics and targeted failure tests; write-capable clustered clients should use routing.

Native path procedures

HydraDB includes native snapshot-scoped path procedures:

them together, avoiding client-side query fan-out.

CALL algo.MSpaths({
  sourceLabel: 'Entity',
  sourceProperty: 'name',
  sourceValues: ['alpha', 'beta', 'gamma'],
  targetValues: ['alpha', 'beta', 'gamma'],
  pairwise: true,
  relTypes: ['RELATES'],
  relDirection: 'both',
  maxLen: 3,
  pathCount: 5,
  fairRelationshipVariants: true,
  resultLimit: 100
})
YIELD path
RETURN path

The procedures use one pinned storage snapshot, compiled GraphBLAS topology when available, the visible WAL overlay, and bounded metadata hydration.

Read Consistency

HydraDB exposes two read modes:

| Mode | Behavior | |---|---| | causal | Uses the node's current durable reader view and refreshes when a supplied bookmark requires a newer sequence. This is the default hot path. | | strong | Refreshes the SlateDB reader from object storage before pinning the query snapshot. This pays the object-store freshness cost. |

HTTPS requests set "consistency": "causal" or "strong" in the request body. Bolt clients set consistency in RUN metadata or hydradb.consistency in transaction metadata. The former turbolay.consistency key remains accepted as a migration alias.

Kubernetes

The Helm chart deploys query nodes, indexer workers, services, cache volumes, network policies, disruption budgets, TLS resources, authentication, and optional Prometheus integration.

helm upgrade --install hydradb charts/hydradb \
  --namespace hydradb \
  --create-namespace \
  --values charts/hydradb/examples/values-eks.yaml \
  --atomic \
  --timeout 15m

Copy and edit the example values before deploying. Object-store credentials, bucket names, image references, TLS, advertised Bolt addresses, and workload identity are environment-specific. See the values file comments and the chart templates under charts/hydradb/ for configuration and rollout details.

Observability

The public HTTP server exposes GET /healthz. The graph-node and indexer admin servers expose:

GET /readyz
GET /metrics

The runtime emits structured tracing fields for query fingerprints, access paths, cache outcomes, consistency mode, scope, cell, storage sequence, and planner decisions. Build with --features server-runtime,otlp or --features indexer-runtime,otlp to export OpenTelemetry data.

Prometheus duration histograms have deliberately different units. Check each metric's unit before building latency dashboards or alerts.

Development

Run just or just help to list the command surface. Recipes use Bash and run from the repository root. The full native suite requires libcypher-parser and SuiteSparse GraphBLAS.

Verification Recipes

| Recipe | Coverage | |---|---| | just native-check | Verifies that cypher-parser and GraphBLAS are discoverable | | just fmt, just fmt-check | Formats Rust or checks formatting without modifying files | | just clippy | Default-feature lint used by CI | | just clippy-chaos, just clippy-opencypher | Chaos-harness and OpenCypher lint configurations | | just clippy-native, just clippy-client-protocols, just clippy-runtime | Full native, public protocol, and production runtime lint configurations | | just check | Checks every default-feature target | | just check-all-features | Checks every target with every Cargo feature | | just check-client-api, just check-bolt-server | Checks shared client code and standalone Bolt independently | | just check-examples, just check-examples-native, just check-examples-chaos | Checks example targets under their supported feature sets | | just test [cargo test args] | Runs default library tests and forwards optional arguments | | just test-opencypher, just test-native, just test-client-protocols, just test-chaos | Runs the major library and public-protocol test matrices | | just test-server-runtime, just test-indexer, just test-node-otlp | Runs graph-node, indexer, and OTLP binary tests | | just test-placement, just test-telemetry | Lints and tests the two workspace crates | | just ci | Runs the complete local CI-equivalent sequence; a clean feature-matrix run can take tens of minutes (25m 41s in the verification run for this README) |

scripts/ci_local.sh is a compatibility entry point that sets a shared Cargo target directory and delegates to just ci, so the script and Justfile cannot silently drift into different test matrices.

Local Harnesses

| Recipe | What it runs | Output or side effect | |---|---|---| | just smoke | Isolated local object-store write, traversal, reopen, and verification | Temporary directory removed on exit | | just smoke-graphblas | The same smoke flow with SuiteSparse selected explicitly | Temporary directory removed on exit | | just query-correctness | Exact OpenCypher result checks | bench-results/query_correctness.csv and .log | | just stress | Multiprocess writes, restart recovery, compaction, GC, and verification | Temporary local stores removed on exit | | just fence | Hard SlateDB writer-takeover proof | Temporary local stores removed on exit |

The benchmark recipes intentionally use production-sized defaults and can run for a long time. Override their documented GRAPH_QUERY_* environment variables for a small development sample.

Docker And MinIO Harnesses

| Recipe | Purpose | |---|---| | just minio-smoke | Runs the object-store smoke flow against an ephemeral MinIO container | | just minio-query-correctness | Runs exact query checks against MinIO | | just minio-query-bench | Runs the query benchmark against MinIO | | just minio-chaos | Pauses, restarts, and recovers MinIO during graph operations | | just minio-fence | Runs writer takeover against MinIO | | just minio-mbt | Replays the formal MBT adapters against MinIO; unavailable in this checkout because the referenced tests/formal_mbt*.rs targets are absent |

The MinIO recipes create isolated containers, networks, buckets, and temporary configuration files. Their cleanup traps remove those resources unless a recipe-specific keep flag is set. Docker may pull pinned images on the first run.

Standalone Scripts

The scripts below are not all exposed as Just recipes because several operate external infrastructure or incur cloud cost.

| Script | Requirements and behavior | |---|---| | scripts/runtime_smoke.sh | Builds graph-node, checks readiness and metrics, then exercises Bolt, scoped databases, HTTP, and graceful shutdown; requires Python neo4j | | scripts/bolt_neo4j_driver_smoke.sh | Exercises direct and routing Bolt URIs through the official Python Neo4j driver | | scripts/query_correctness.sh | Implements the corresponding local Just recipe | | scripts/multiprocess_stress.sh, scripts/fence_takeover.sh | Implement the local stress and fencing recipes | | scripts/minio_*.sh | MinIO smoke, correctness, chaos, fencing, MBT, and write-profile harnesses; require Docker | | scripts/multinode_k3s.sh | Creates a disposable multi-node K3d cluster and performs disruptive failover tests; requires Docker, K3d, kubectl, and Helm | | scripts/deploy_single_node_k3s.sh | Builds and deploys to an existing K3s host using an S3 bucket; changes Kubernetes and AWS resources | | scripts/multinode_k3s_client.py | Runs inside the disposable K3d client Pod created by multinode_k3s.sh |

just update-slatedb is a maintenance command, not a verification command. It updates the pinned SlateDB dependency in Cargo.lock; review and test the resulting lockfile diff before committing it.

Repository layout

src/core/           configuration, graph model, cache policy, errors
src/shard/          storage lifecycle, reads, writes, queries, path procedures
src/engine/         routing, placement, immutable indexes, index GC
src/query/          OpenCypher parsing, algebra, planning, transport types
src/client/         Bolt, HTTP, authentication, quotas, cursors
src/sparse_kernel/  Rust sparse and SuiteSparse GraphBLAS execution
crates/             placement and telemetry workspace crates
charts/hydradb/     Kubernetes Helm chart
examples/           smoke, import, benchmark, and correctness programs
scripts/            local, MinIO, stress, fencing, and deployment harnesses

Benchmarks

Published latency and throughput results are available on the HydraDB benchmark site. To reproduce measurements locally or against S3, use the benchmark commands and scripts documented above.

Documentation

| Document | Contents | |---|---| | Architecture | End-to-end design, snapshots, writer ownership, query execution, and indexing | | Using HydraDB | Getting started, querying, and operating HydraDB |

Contributing

Issues and pull requests are welcome. Keep changes focused, add regression coverage for behavioral changes, and run just ci before opening a pull request. Changes to storage, fencing, snapshots, routing, or index publication should state the invariant they preserve and include a failure-oriented test.

License

HydraDB is licensed under the GNU Affero General Public License v3.0.

GitHub Stars & Activity

13,046Stars
5,483Forks
152Open issues
RustLanguage

GitHub Popularity

GitHub stars13,046
Forks5,483
Open issues152
Primary languageRust
LicenseAGPL-3.0
Stars gained today10,023
Created2026-07-03
Last pushed2026-09-30

Trending History

Monthly boardrank #17 · ▲ 10,023 stars

Related GitHub Projects

1

nautechsystems / nautilus_trader

Rust★ 29,897⑂ 3,992▲ 36 stars
→
2

storytold / artcraft

Rust★ 22,456⑂ 3,614▲ 4,095 stars
→
3

1jehuang / jcode

Rust★ 20,620⑂ 2,400▲ 161 stars
→
4

microsoft / litebox

Rust★ 3,126⑂ 165▲ 44 stars
→
5

HakanSeven12 / OpenCADStudio

Rust★ 2,903⑂ 318▲ 96 stars
→
6

public-apis / public-apis

Python★ 488,041⑂ 54,101▲ 432 stars
→
7

openclaw / openclaw

TypeScript★ 391,810⑂ 82,380▲ 181 stars
→
8

obra / superpowers

Shell★ 298,251⑂ 26,656▲ 576 stars
→

More Trending Repositories