Anyone can predict the future.
Almost nobody keeps the receipt.
Seal a probability before the answer exists. Settle it from a source. Then find out what your confidence was actually worth. Every forecast is hash-chained the moment it is given, and the head is anchored on Robinhood Chain — edit one digit later and the chain names the line.
npx brier demo && npx brier score
npm install -g github:Noisyxl/brier
Four seconds, no API key, no network. Then
doctor ·
ask ·
calibrate ·
ledger --verify.
Thirty-one points of overconfidence, and a Brier score worse than a coin — so on this question set, saying nothing at all would have scored better. That distance is the entire product.
| the problem | what brier does | command |
|---|---|---|
| “the model called it” — after the fact, from memory | the probability is hash-chained the moment it is given; one edited digit breaks the chain and names the line | ledger --verify |
| questions graded by whoever wrote them | a question carries its own resolver and test, fixed at ask time and hashed into its id — rewording it makes a different question with no forecasts | ask |
| accuracy, which rewards only ever saying 1% and 99% | proper scoring rules, where your best score comes from stating what you believe | score |
| a good score on a set where most things happen | always-yes is on every scoreboard automatically, and it will be brilliant on that set |
score |
A forecast cannot be sealed once the question is settled — the obvious cheat.
A question with no forecasts cannot be settled — the quieter one, settling only what went well.
The resolver cannot see any forecast, and the scorer cannot see the resolver. Settlement is a pure function of source and test, fixed before any answer existed.
Each one has a test named after it. Everywhere else in this space, the person who wrote the question also decides afterwards whether it came true.
Every question on the belt is asked, refused or sealed, held while nothing happens, settled from one number and then graded. The four offline forecasters answer on the left. Yellow is the only colour in the room, and it only ever marks a miss.
They need no key and they are not models. They exist so a calibration diagram produced with no API budget still shows a real shape — and so the demo is reproducible by anyone, byte for byte, from the same seed.
XAI_API_KEY
docs.x.ai ↗
claude-sonnet-4-5
Anthropic · ANTHROPIC_API_KEY
platform.claude.com ↗
claude-opus-4-5
Anthropic · ANTHROPIC_API_KEY
platform.claude.com ↗
With no key set the offline panel still runs, every record says so, and every command works — that is why npm test needs no network. Put the keys in .env and brier demo --with-models prints the same table with your panel on it. Any OpenAI-compatible endpoint works, the prompt lives in prompts/forecaster.md where you can read it, and the model name plus the round-trip time goes into every sealed record — so the model half of a claim is at least pinned to a named model on a named day.
| panelist | n | brier | log | said | right | gap | skill |
|---|
The best forecaster is the one with no opinion. parrot
says the base rate and nothing else. It ties the base baseline exactly —
which is also a self-check that the scoring is right — and it beats every panelist that had something to say.
always-yes and always-no land
identically because this set is a coin flip overall; on a set where most things happen, one of them would look
like a genius. That is exactly what they are there to catch.
│ stated · ● observed · a dot to the left of the line is a claim the world did not pay. expected calibration error 31.2%.
BRIER_BINS grows, and most write-ups drop it silently.
Drag it. This is a proper scoring rule: the score you can expect is lowest when the number you say is the number you believe. Overclaim and the penalty is quadratic — which is why 91% said, 60% right costs what it costs.
So ask refuses anything without a future
resolution date, a source, and a test that turns a reading into true or false — and it names which one is
missing, at ask time, in front of the person who can fix it.
# accepted brier ask "US 10-year yield closes at or below 4.00 on 2026-12-31" \ --on 2026-12-31 --kind manual \ --source "https://fred.stlouisfed.org/series/DGS10" \ --test "lte 4.00" --base-rate 0.45 # the id is a hash of the parts that decide the answer # reword it afterwards and you have a different question, # with no forecasts against it. visible immediately.
brier ledger --verifyhashₙ = sha256( seq | at | kind | body | hashₙ₋₁ ) # what it buys, and it is enough: a single edited line is obvious — change one probability after a bad result and every hash after it stops matching. the head is one short string you can publish the day you seal. The timestamp comes from wherever you published it; the chain only proves the file has not moved since. # it does not prove when anything happened, and it # says so in three places.
ledger --verify walks the file and names the first line that breaks.pending as a real view: what is sealed, what it said, and how long until the world answers.n should say “we do not know yet”.The only thing that turns this into evidence about a model is real questions, a real resolver, and waiting — and the waiting cannot be shipped. Everything above is ordered by how much it helps with that, not by how quickly it demos.
NOT DEPLOYED YET · PASTE YOUR CA IN THE CONFIG
The mark is a community token on Robinhood Chain, and the same chain is where a ledger head gets its timestamp. The repository itself holds nothing, trades nothing and has no market in it — that is stated in the code, in the docs and in SAFETY.md. Nothing here is financial advice.
Paste the contract address once, in the config at the bottom of this file. The pool, the chart, the price and every link resolve themselves from it.
Straight from the pool with nothing smoothed over. If the frame stays dark, your network is blocking the embed — use one of the links underneath instead.
Paste the token contract address in the config at the bottom of this file and the chart wires itself up. If an address is already set and this is still dark, the embed is being blocked here — open the pool with one of the links below.
Powered by GeckoTerminal
No. It grades what somebody else predicted. There is no signal here, no trading in it, and nothing to act on. Every number it prints is about a forecaster, never about the future.
No — the demo panel is four rules, and the almanac is a lognormal random walk with a label on it. A score against it says the scoring works. For evidence about a model, ask real questions with a real resolver and wait; the waiting is the method.
Because it usually does, and that is the finding. Beating the base rate is hard, most confident forecasting does not, and a tool that never showed you the parrot would let you believe otherwise.
It is recorded as having failed to answer. It is not turned into 0.5, and it is not dropped from the denominator either.