glyph

# Glyph Subnet

[![CI](https://github.com/glyph-research/glyph-subnet/actions/workflows/ci.yml/badge.svg)](https://github.com/glyph-research/glyph-subnet/actions/workflows/ci.yml)

Glyph is a lossless **neural text-compression** benchmark subnet — a perpetual,
decentralized Hutter Prize. Miners commit one permanent codec per hotkey. Validators
sample fresh, never-before-seen text, run each codec's compress→decompress round-trip on
local Docker on an RTX 4090 (Chutes (SN64) serverless GPU is available as an optional
secondary path — see [docs/VALIDATING.md](docs/VALIDATING.md)), and set weights with a
king-of-the-hill policy:

- **emission follows improvement, not slot** (issue #177): each winner earns
  `15%` of the pot per `1%` of ratio improvement it achieved over the winner it dethroned,
  plus a one-off `25%` base for whoever is currently top of the ladder. Shares are assigned
  newest-first until the pot runs out, so a `5%` jump takes the whole pot and a run of small
  wins spreads it across several winners; if the shares under-subscribe the pot they scale
  up to fill it. Improvement is recorded when the crown changes and never recomputed.
- plus a `10%` **temporal burn** (one unpredictable tempo per 10-tempo window → UID 0) that
  makes copy-cat validation strictly losing. **Currently enabled** network-wide
  (`core.constants.BURN_ENABLED = True`, issue #88) — see
  [docs/reign-and-burn.md](docs/reign-and-burn.md) for the current state and how to disable.
- **Miner Conviction** (issues #141/#156): a winner slot must keep its cumulative alpha
  earnings **chain-locked** to its hotkey (`btcli lock add --netuid 117 ...`) — the free
  (unlocked-allowed) amount is `max(20% of earned, 1000 α)`. A slot below its required
  conviction earns nothing that tempo: the pot is allocated among the **compliant**
  winners in the retained history (depth `20`), walking past anyone gated or no longer
  eligible, and burns only when none of them qualifies (issues #170/#177). So staying
  locked keeps paying you after you lose the crown — and a winner that locks is paid again
  at the very next weight-setting. Measured in
  alpha on both sides; the crown itself is never affected. Plain staked-but-unlocked
  alpha does **not** count (issue #162: enforcement is live — the announced
  `CONVICTION_LOCK_CHECK_START_BLOCK` is behind the chain head): locked mass is
  chain-enforced unstakeable, so exit follows the lock's decay schedule (or a deliberate
  perpetual→decaying switch), never a cliff. Either lock mode satisfies the gate —
  perpetual is the low-maintenance choice; a decaying lock gates again as it decays below
  the line until re-locked.

Score = compression ratio (compressed ÷ raw, lower is better) with a hard **bit-exact
round-trip** gate. A challenger takes the crown only by beating the incumbent by `ε`
(default `1%`, lowered from `5%` in issue #177 — a marginal win now earns a marginal
share rather than being refused outright). When several challengers land in the same round they run as a **sequential
gauntlet in commit order** (issue #136): the earliest commit challenges first, and each
later challenger must beat the *current* — possibly just-crowned — winner by the full `ε`,
so a later-committed marginal tweak of someone else's round can't steal it; identical
commit blocks tie-break by hotkey; losers are excluded forever (one shot).

See [`docs/`](docs) for guides.

## Repository layout

```
src/
  core/        shared: constants, commitments, artifact, state, weights, burn_schedule, dotenv
  chain/       chain adapter + commitment reader        (glyph-chain-reader)
  validation/  codec artifact precheck + checks
  eval/        runners (local + Chutes), chute_app, evaluator, scoring, streams, corpus,
               live_corpus (per-round beacon-seeded HF corpus, issue #71), deploy
  weight_setter/     temporal-burn weights                    (glyph-weight-setter)
  reign_worker/      king-of-the-hill round                   (glyph-reign-worker)
  validator/   all-in-one orchestrator + offline demo   (glyph-validator)
miner/               commit | check | publish | register      (glyph-miner)
scripts/             install, genesis king, deploy chute, auto-update (run_auto_validator.sh)
reference_codec/     minimal zstd codec (artifact contract example)
samples/             bundled corpus + demo codec for the offline demo
docs/  tests/
```

## Install

```bash
./scripts/install_deps.sh           # venv + package + pm2
# or: pip install -e ".[dev]"
cp .env.example .env                # miners: set HF_TOKEN (write); validators: recommended too (read-only, avoids anonymous HF CDN 403s) + BLOCKMACHINE_API_KEY (Standard plan, fast conviction backfill) -- see docs/VALIDATING.md
pytest -q
```

## Miner

```bash
glyph-miner check   --local-path ./reference_codec        # self-benchmark locally
glyph-miner publish --path ./my-codec --repo you/your-codec
glyph-miner register --netuid 117 --wallet-name w --hotkey-name h
glyph-miner commit  --netuid 117 --wallet-name w --hotkey-name h --model-repo you/your-codec
```

See [docs/MINING.md](docs/MINING.md). Commitments are permanent per hotkey.

## Validator

**Default eval path is local Docker on an RTX 4090** — every validator running GPU codecs must
use identical hardware, or compress/decompress throughput isn't comparable across validators
(same-system determinism). This is a network-wide requirement, not a suggestion: a
validator without Docker + `nvidia-container-toolkit` + a matching GPU **fails closed by
design** (`DockerRunner` checks the GPU model via `nvidia-smi` and refuses to run on anything
else). See [docs/VALIDATING.md](docs/VALIDATING.md) for the full requirement and CPU-only opt-out.

Every validator builds its own copy of the evaluation corpus live from HuggingFace
(FineWeb-Edu + Pile, 2x/1x scored mix, plus two benchmark-only display windows: enwik9 and
a per-round live-data snapshot of recently-changed Wikipedia text, fetched in the
betwe