Independent forecasting research
We measure how well AI models forecast official economic data.
Blink Labs LLC is an independent research company. We evaluate AI forecasting models on scheduled U.S. releases such as the monthly jobs report, the Consumer Price Index, Federal Reserve rate decisions and weekly jobless claims. Each forecast is locked before the release, then scored against the market-implied probability and the official result.
Research
Scheduled U.S. data releases
We start with official releases whose dates are announced in advance and whose outcomes are objective. Many of them are also priced in prediction markets, which gives every forecast a public point of comparison.
| Release | Publisher | Schedule |
|---|---|---|
| Employment SituationNonfarm payrolls, unemployment rate | Bureau of Labor Statistics | Monthly, usually first Friday, 8:30 a.m. ET |
| Consumer Price IndexHeadline and core inflation | Bureau of Labor Statistics | Monthly, 8:30 a.m. ET |
| FOMC statementFederal funds target range | Federal Reserve | Eight scheduled meetings a year, 2:00 p.m. ET |
| Weekly jobless claimsInitial unemployment insurance claims | U.S. Department of Labor | Weekly, usually Thursday, 8:30 a.m. ET |
Later studies may apply the same method to other event categories with clear, verifiable outcomes, such as sports.
Method
A protocol fixed before the data arrive
Every study follows the same four steps. They are written down before the first forecast is made, so results cannot be shaped after the fact.
-
Register the rules
We record which events are included, the forecast cut-off for each, and how scores and their uncertainty will be computed. The rules stay fixed once the study starts.
-
Lock the forecast
Each model forecast is saved before its cut-off together with a SHA-256 fingerprint of the record, so any later change would show.
-
Keep prices out
The models under evaluation receive no market prices. The market-implied probability is read only after the forecast is locked.
-
Score against the official result
Model forecasts and market-implied probabilities are scored the same way, with the Brier score, against the outcome published by the issuing agency.
The Brier score
BS = ∑ (pi − oi)²
p is the forecast probability. o is 1 if the event happened and 0 if it did not. A perfect forecast scores 0, and lower is better.
Because the market-implied probability is scored the same way, the difference between the two scores shows which one was closer to what actually happened.
| Forecast | Squared error | Score |
|---|---|---|
| Model, locked | (0.70 − 1)² | 0.0900 |
| Market-implied | (0.62 − 1)² | 0.1444 |
| Market minus model | 0.0544 |
Markets
Market prices as the benchmark
Prediction markets attach a probability to many of these releases, and traders put money behind it. That makes the market price a demanding benchmark. AI forecasting benchmarks increasingly use market prices as the reference to beat, and our studies use them the same way.
In the future, Blink Labs may also trade event contracts for its own account on regulated U.S. exchanges, including as a liquidity provider, under each exchange’s rules.
What we do not do
- Manage money for others or take outside investment.
- Give investment advice or sell forecasts or trading signals.
- Resell or republish third-party market data.
We are building and testing our evaluation pipeline. In testing, all positions are simulated and no orders are placed. Blink Labs does not currently trade and has not yet published results.
Company
Blink Labs LLC
Blink Labs LLC is a limited liability company organized in New Mexico, United States. We keep records that can be checked: written rules, fingerprinted forecasts, and scores that can be recomputed from the stored data.
Our first public project was agent-startup-kit, an open-source set of workflows for AI coding agents, released under the MIT license.
Contact
Get in touch
For research, data or business inquiries, write to us.
- Mailing address
- 9169 W State St #3336
Garden City, ID 83714
United States - Organized in
- New Mexico, United States