TabFM vs. the UFC betting market

Benchmarking Google's TabFM — a tabular foundation model that predicts via in-context learning with zero training — on UFC fight prediction, then testing whether it produces a betting edge against real closing lines.

Four findings, and the model is only the first one:

1. TabFM beats a tuned, 5-model stacking ensemble on the same data, out of the box, with no hyperparameter search and no calibration step.
2. It still loses decisively to the moneyline market. A frontier model extracting the maximum signal from public fight statistics is not sharp enough to beat the closing line — a clean result about the ceiling of public-data sports modeling.
3. The props market — the "softer" one — is closed too. Method-of-victory props carry a 22.8% vig (5× the moneyline). The market out-predicts the model there as well, and every historical "edge" turns negative once tested against real prices.
4. As a pure regressor, TabFM shows real skill — strike-volume error 34% below a naive baseline and 54% below the project's old bespoke model — which sharpens the moral: model capability and market edge are different quantities. The skill is real; the edge still isn't.

Reproduce the headline results in a few seconds, no model weights or private data required:

pip install pandas numpy
python src/reproduce_results.py   # Result 2: moneyline
python src/reproduce_props.py     # Result 3: method props
python src/make_figures.py        # regenerate all README figures (needs matplotlib)

There's also a data-science dashboard (dashboard/index.html) — a self-contained fight-lab console rendering the full dataset, model diagnostics, and this verdict.

---

Result 1 — TabFM is the best model the feature set has ever had

554-fight chronological held-out test (never shuffled — betting evaluation must respect time). Trained on ~2,200 fights, ~46 numeric features, RTX 5060 Ti (16 GB).

Model	Accuracy	Brier ↓	Log loss ↓	AUC
TabFM raw (full context)	0.6372	0.2220	0.6347	0.6932
TabFM raw (70% context)	0.6318	0.2235	0.6383	0.6893
TabFM + isotonic calibration	0.6047	0.2306	0.7726	0.6789
Stacking ensemble (same split)	0.6191	0.2267	0.6451	0.6782

The stacking baseline is XGBoost + LightGBM + RandomForest + neural net + logistic regression with a logistic meta-learner — retrained faithfully on the identical temporal split. TabFM beats it on every metric.

Two things worth noting:

• fit() is instant. TabFM does no training — your rows become in-context examples and it predicts in a single forward pass. All the cost is at predict time (~97 s for 831 fights at n_estimators=32 on GPU; peak VRAM 8.2 GB).
• Calibrating it made it worse. Isotonic regression on a validation fold hurt Brier and log loss — TabFM's raw probabilities are already well-calibrated. Use them raw.

Result 2 — the market is sharper, and TabFM adds nothing to it

TabFM's 554 predictions were joined to a 21-book consensus closing line (median across books, including Pinnacle) for 332 matched fights.

Sharpness (Brier, lower = sharper):

	Brier	Accuracy
Consensus closing line	0.1969	0.684
Pinnacle close (n=180)	0.2110	—
TabFM	0.2270	0.605

The gap between TabFM and the market is larger than the gap between TabFM and the ensemble it beat. On the 94 fights where TabFM disagreed with the market on the winner, the market was right 64% of the time.

The kill shot — the blend sweep. If TabFM held any information the market lacked, mixing some of it into the market probability would improve the combined Brier. It doesn't. The optimum is 0% model weight:

w=0.00   0.1969   <- best
w=0.05   0.1971
w=0.10   0.1974
w=0.20   0.1985
w=0.50   0.2051
w=1.00   0.2270

Value betting (flat stakes vs. consensus close):

Edge threshold	Bets	Win %	ROI	CLV (line moved to us)
> 2%	291	32.0%	−19.1%	15.8%
> 5%	271	31.0%	−20.0%	14.0%
> 8%	246	30.1%	−19.0%	13.8%

Every "positive-EV" bucket loses ~19% — worse than the ~5% you'd lose betting randomly into the vig. That's the winner's-curse signature: a noisier-than-market model's disagreements cluster exactly where it's most wrong. And closing-line value is an anti-signal — the line moved toward our bets only ~15% of the time, meaning sharp money was on the other side.

Result 3 — the props door is closed too

The obvious rebuttal to Result 2: moneylines are the sharpest MMA market — try the softer props. So TabFM was retrained as a 6-class model (winner × KO / submission / decision) and tested on 472 fights carrying a full 6-way method market (2023-07 to 2024-12).

The market wins again, and this time the reason is structural, not just informational:

	Value
Method-market vig (6-way overround)	22.8% — vs ~5% on the moneyline
Sharpness — market Brier	0.7373
Sharpness — TabFM Brier	0.7857
Top-1 method accuracy — market / TabFM	0.403 / 0.305
Blend sweep — optimal model weight	~0%
Value-bet ROI (edge > 3%) — KO / SUB	−25% / −47%

"Props are softer" is true about pricing effort — but the book knows it, so it charges a 5× wider margin. You'd have to out-predict the market by 20+ points of probability mass just to break even. One bucket (decision bets) showed +4.4% ROI, but at t = 0.33 — a few longshots landing, not signal.

The historical "edges," retested on real prices. The old profitable angles were scored against implied rates, never real odds. Betting each blindly on the actual 6-way market, 2020–2024:

Angle	Once claimed	Measured on real odds
Women's decision props	+71%	−13.0%
Any submission (all divisions)	(sub specialists +127%)	−38.4%
Heavyweight KO	—	−29.8%
Any KO / TKO (all divisions)	—	−27.2%
Flyweight decision	—	−11.1%

All five negative. The edges were in-sample artifacts.

Result 4 — TabFM as a props regressor: real skill, still nothing bettable

The first three results use TabFM as a classifier. The newest run (2026-07-12) uses its regression head on continuous fight-prop targets: per-minute significant-strike rate, takedowns, and fight duration — the ingredients of strike-total and over/under props. Train: 9,000 fighter-rows; test: 726 fighter-rows, strictly chronological (all fights since 2025-07-01).

target	TabFM MAE	naive MAE	old bespoke model
Sig-strike rate (per min)	1.98	2.32	—
Sig-strike totals (composed: rate × predicted duration)	26.6	40.3	58.2
Sig-strike totals (oracle duration)	17.9	21.5	—
Takedowns	1.14	1.09	—
Fight duration (seconds)	295	309	—

Three honest readings:

• The skill is real. A 34% strike-volume error reduction over naive (54% over the old model) on a chronological test is the largest baseline gap in this repo. The oracle-vs-composed rows decompose the error: about a third of the deployable error comes from predicting duration, not striking.

• It is far from oracular. Predicted-vs-actual correlation is r = 0.36 — the model narrows the error, it does not call fights:

  

• Skill against a naive baseline is not a market edge. On a synthetic 2.5/3.5-round over/under line the model's lean is right 66.9% of the time vs 65.0% for the naive lean — a +1.9pp sliver, measured against no real prices. Takedowns don't beat naive at all. Given Results 2–3 (a sharper market plus 5–22% vig), the prior that this sliver survives real over/under odds is poor. Nothing here changes the repo's verdict; it only makes it more precise about why the verdict holds: the model is good, and the market is better.

Reproduce: src/tabfm_props_regression.py (needs the public scrape CSVs and TabFM regression weights; predictions it produced are committed at data/tabfm_props_test_preds.csv).

A falsifiable forward test

The same run produced projections for a card that had not happened yet: UFC Fight Night: Du Plessis vs Usman (2026-07-18). They are committed here (generated 2026-07-12, pushed before the event) precisely so they can be scored against reality afterward — a small prospective test, the same discipline as the chronological splits above. These are model projections of fight statistics, not betting advice; no odds are attached, and Results 2-3 explain why attaching odds would be a losing idea.

Raw values (including per-fighter strike rates and the 8 fighters skipped for insufficient history): data/tabfm_props_card_20260718.json. After the event, scoring these against the official stats will be Result 4's out-of-sample epilogue.

Why this is a useful result, not a failure

Because TabFM needs no tuning, this removes the usual escape hatch ("maybe my model just wasn't good enough"). It's a statement about the data: everything public UFCStats-derived features know is already priced into both markets, plus information they have and the features don't (injuries, camp reports, sharp order flow). MMA markets are among the softer major markets — if the edge isn't here, across moneylines and props, it isn't in public fight data.

The only market left genuinely untested against real prices is round totals (over/under). Result 4 now supplies the model half of that test (a duration regressor with a +1.9pp lean over naive), but no O/U odds exist in the data used here — and on the same prop sheet that holds 22.8%, the prior is not encouraging. Caveats: the method odds are single-source and single-snapshot (no line shopping, not certified closers), and in-fight live betting — plausibly the last real MMA edge — was never in scope.

Repo layout

src/reproduce_results.py   self-contained — moneyline verdict from committed CSV
src/reproduce_props.py     self-contained — method-props verdict from committed CSV
src/tabfm_benchmark.py     Result 1: load TabFM from safetensors, benchmark vs stacking
src/roi_backtest.py        Result 2: join predictions to moneyline odds, test for edge
src/tabfm_method_props.py  Result 3: 6-class method model vs the 6-way method market
data/tabfm_test_predictions.csv   554 held-out moneyline predictions (model output)
data/tabfm_roi_matched.csv        332 fights joined to consensus/Pinnacle odds
data/tabfm_method_prop_test.csv   472 fights: 6-class preds + de-vigged fair probs + odds
dashboard/index.html              fight-lab console (self-contained; open in a browser)

The two reproduce_*.py scripts run on the committed data. The three upstream scripts require the TabFM weights (~6 GB) and your own UFC feature matrix + odds table — schemas are documented in each script's header.

Setup notes for the TabFM weights

• No TabFM_HF.from_pretrained in the pip build, and load() expects a .bin; the weights ship as .safetensors. Working load: build the module and load the state dict directly (see src/tabfm_benchmark.py).
• On a Blackwell GPU (RTX 50-series, sm_120), pip install tabfm[pytorch] pulls a CPU-only torch. Pin a CUDA build: torch==2.11.0+cu128.

License & honesty

Code: MIT. The TabFM weights are under Google's non-commercial license — this repo does not redistribute them. Not betting advice; this documents a negative result. Sample sizes are modest, so weight the calibration, sharpness, and blend results over raw ROI — though every measure agrees.