TAO—Block—Subnets85 / 128active
Arena for LLM Inference Server Optimization
what the project says about itself — the input for semantic labels
Arena for LLM Inference Server Optimization
scope of the data
Taonets fetches the repo's public metadata (description, topics, language, stars, last push) and the head of its README. Commit-count histories, contributor lists and release notes require the GitHub commits/releases endpoints, which the worker doesn't consume yet — those panels are coming with the ingest upgrade and are not simulated in the meantime.
first lines of the default branch README, unedited
<div align="center">
# Cacheon (SN14)
**Inference optimization. Fastest server wins.**
[](https://discord.gg/bittensor)
[](https://cacheon.ai/docs)
[](https://tao.app/subnets/14)
[](https://x.com/cacheon_ai)
[](https://opensource.org/licenses/MIT)
[Website](https://cacheon.ai) | [Docs](https://cacheon.ai/docs) | [Discord](https://discord.gg/bittensor) | [TAO.app](https://tao.app/subnets/14)
---
</div>
Cacheon is a Bittensor subnet (SN14) that runs an open competition for **production-grade LLM inference optimization**. Miners submit containerized inference servers. Validators evaluate them against a vLLM baseline on the same hardware. The fastest correct server earns the majority of emission; a runner-up receives a share.
**V1 arena:** `Qwen2.5-72B-Instruct` on 8-GPU TP=8 pods (8x H200, 8x B200, or 8x B300). Beat the pinned vLLM baseline on end-to-end response time while passing a greedy-decoding correctness gate.
## How It Works
1. **Miners** build an inference server, package it as a Docker image, pay the submission fee, and commit the image reference, digest, and payment proof on-chain.
2. **Validators** scan the chain for new commitments, pull the image, and run it with model weights mounted at `/models`.
3. **Scoring** measures end-to-end response time improvement over the vLLM baseline. Correctness is checked first; fail it and the score is zero.
4. **The fastest correct server** becomes the winner and earns 80% of the competition pool. The runner-up earns 20%.
5. **Challengers** must exceed the winner's fresh score by a fixed 1% margin to prevent noise-driven churn.
Score formula:
```python
if not correctness_pass:
score = 0.0
else:
speed_improvement = median(max(0, (baseline_e2e - miner_e2e) / baseline_e2e))
score = speed_improvement
```
Full detail: [cacheon.ai/docs/evaluation/scoring](https://cacheon.ai/docs/evaluation/scoring)
## For Miners
Build an inference server that serves `Qwen2.5-72B-Instruct` via `/v1/chat/completions` with streaming and logprobs. Package it as a Docker image (maximum 20 GB; model weights are mounted at runtime, not baked into the image). Push it to a public registry, then run `miner/commit.py` to pay the submission fee and commit on-chain in one flow.
**Requirements:** public container registry, Bittensor wallet registered on SN14, coldkey balance for the submission fee. GPU hardware is only needed for local testing.
```bash
# Push your image
docker tag my-server:latest docker.io/myuser/cacheon-miner:v1
docker push docker.io/myuser/cacheon-miner:v1
# Pay fee + commit on-chain (test locally first)
python miner/commit.py \
--wallet-name <wallet> \
--wallet-hotkey <hotkey> \
--image "docker.io/myuser/cacheon-miner:v1" \
--digest "sha256:..." \
--fee 0.1 \
--network finney \
--netuid 14
```
Full guide: [cacheon.ai/docs/miners/overview](https://cacheon.ai/docs/miners/overview)
## For Validators
The validator has two components: an always-on CPU host (chain scanning, weight setting) and an ephemeral GPU pod (eval). The GPU pod is rented on-demand only when challengers are queued.
**GPU requirements:** NVLink/SXM 8-GPU pod (8x H200, 8x B200, or 8x B300 are Tier A; 8x H100 is Tier B fallback), 400 GB storage, model weights at `/workspace/models/Qwen2.5-72B-Instruct`. See [GPU requirements](https://cacheon.ai/docs/validators/overview#gpu-requirements).
```bash
# CPU host (always-on)
git clone https://github.com/latent-to/cacheon
cd cacheon
cp .env.example .env # add wallet and S3 config
docker compose up --build
# GPU pod (on-demand, run when challengers appear)
bash scripts/gpu_setup/setup.sh
docker compose -f validator/gpu-compose.yml up --build -d
```
Full guide: [cacheon.ai/docs/validators/overview](https://cacheon.ai/docs/validators/overview)
## Documentation
| | Miners | Validators | Evaluation |
| -------------- | ----------------------------------------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------- |
| **Start here** | [Overview](https://cacheon.ai/docs/miners/overview) | [Overview](https://cacheon.ai/docs/validators/overview) | [Scoring](https://cacheon.ai/docs/evaluation/scoring) |
| **Reference** | [API contract](https://cacheon.ai/docs/miners/api-contract) | [Architecture](https://cacheon.ai/docs/validators/architecture) | [Harness](https://cacheon.ai/docs/evaluation/harness) |
| **Setup** | [Quickstart](https://cacheon.ai/docs/miners/registration) | [Validator setup](https://cacheon.ai/docs/validators/setup) | [Prompts](https://cacheon.ai/docs/evaluation/prompts) |
| **Rules** | [Rules](https://cacheon.ai/docs/miners/rules) | [Manual GPU setup](https://cacheon.ai/docs/validators/gpu-pod-setup) | [Roadmap](https://cacheon.ai/docs/roadmap) |
## License
MIT