Refinery

# Refinery

Accelerating Open Intelligence through incentivized research.

Refinery is **Bittensor subnet 125**: an incentivized research network for
improving the algorithms behind language-model training. It connects a global
pool of contributors to a shared objective: make open models learn more from
the compute available to them.

Miners submit optimizer code. Validators train and independently score the
resulting checkpoints under a fixed compute budget. Improvements that pass
confirmation earn rewards.

**The first research target is gradient descent.** The aim is practical: discover
update rules that reach lower validation loss with the resources a training run
already has, then make those discoveries available for others to build on.

[Start mining](MINING.md) · [Instructions for research agents](MINING_AGENTS.MD) · [Optimizer template](sn125/miner_template.py)

## Why Refinery exists

Model performance is downstream of algorithms and data. The difference between
training systems need not come from one breakthrough: improvements in update
rules, architectures, pretraining data and reinforcement-learning environments
can compound. We believe a sustained, open search across these components can
help narrow the capability and efficiency gap between open and closed models.

We see that the Bittensor network is a bastion of capable researchers who we believe
are well suited to advancing the open frontier.
We also see that the capabilities of Frontier language models have reached a point
where semi automated research appears plausible, enabling these researchers to 
work more effectively and explore idea spaces more thoroughly.


Refinery starts with one measurable part of that ambition. Instead of asking
contributors to build an entire frontier model, it asks them to improve how
one learns—and provides a common evaluation and a reward for verified progress.

Refinery brings that research to Bittensor for three reasons:

- **Advance Open Intelligence.** Produce methods, code and evidence that help
  the open-source AI community improve its training systems.
- **Make Bittensor's research contribution visible.** Use its global talent pool
  to produce results that researchers can inspect, reproduce and challenge.
- **Bring researchers into the network.** Give optimizer specialists, ML engineers
  and research-agent teams a concrete route into mining through useful discoveries.

## Why start with optimizers?

An optimizer determines how a model learns from each batch of data. A better
update rule can reach lower validation loss with the same compute budget—or
reach a useful loss sooner. That makes optimizer research a concrete first
target: contributors can change one component and measure its effect under a
shared training contract.

Our two starting beliefs are that bounded training runs can identify methods
worth testing at larger scales, and that Bittensor's global talent pool can
discover those methods. LLM research agents may broaden that pool when paired
with useful feedback, enough compute and the right constraints. The subnet's
results will test those beliefs.

We believe this benchmark is our best practical way to measure optimizer
performance within the resources of an evaluation run. Transfer to other
models, datasets and training scales still needs to be measured. Data selection
and other training components are future research directions, not current
mining tasks.

## What miners do

1. Implement an optimizer in Python using the supplied interface.
2. Test the candidate against matched controls on their own research budget.
3. Serve its source from a registered miner hotkey and fund evaluation credits.
4. Commit to the source hash, then reveal the same source when requested.
5. Earn rewards if the candidate clears the improvement threshold and passes
   a validator-funded confirmation run.

Mining is **algorithm research**, not supplying GPUs to the validator. The miner
service serves source code; the validator runs the scored training jobs. A GPU
is not required simply to serve a submission, but meaningful local training
experiments require suitable compute.

The intended output is reusable research: optimizer source, training evidence
and results that other researchers can investigate. A subnet benchmark win is
a starting point for broader evaluation, not a claim of universal superiority.

## The initial evaluation

| Component | Production contract in this release |
| --- | --- |
| Model | In-repository Llama-style approximately 360M-parameter model |
| Data | Manifest-pinned FineWeb-Edu; SmolLM2 tokenizer |
| Training shape | Sequence length 2,048; batch size 32 |
| Compute | One B200; at most 20 hours |
| Step limit | 188,000 steps, or the compute budget, whichever comes first |
| Objective | Lower validation loss from clean checkpoint scoring |
| Qualification | Improvement strictly exceeding the adaptive bar; floor 0.03 nats |
| Confirmation | Both the initial and confirmation run must qualify |

The submitted optimizer owns its learning-rate schedule. The confirmed frontier
uses the worse loss of the qualifying pair. Read the authenticated round contract
before spending: historical reference numbers are not a live target.

## Rewards follow progress

There is no reward merely for submitting, remaining online, or matching an
existing optimizer. Confirmed improvements receive credit proportional to their
size, with a 14-day half-life. Credit determines how rewards are shared, while
recent network progress determines how much miner emission is paid rather than
burned. The launch burn floor is 0% from day one—not a guaranteed payout.

The initial optimizer search is planned for at least six weeks, followed by a
review of the results and a discussion of whether to continue, revise the task,
or pause for research. The code does not schedule an automatic pause.
See [mining economics](MINING.md#costs-and-expected-value) for costs and
the distinction between a calculated weig