Patents against v4: result

Run 2026-11-05 under docs/prospective/patents-preregistration.md (commits af150f9 and 29cb155). Result file: data/processed/patents_result.json. Retrospective, on cutoffs already seen when v4 was chosen. Not a skill claim. v4 stays the ranker.

Short version

• The pre-registered test passed. NIH-linked patents added 0.0085 AUC to the v4 logit at S0 (95 percent CI 0.0027 to 0.0151). All three conditions held.
• It is a weak signal. The gain does not improve the top of the list, and it fades when only older patents are used. An adversarial audit rated it WEAK.
• Leakage is not ruled out. The link table is 2026 state. Only a forward-looking read, or dated full patent data, can settle it.

Pre-registered read

Per-fold paired intervals: 0.0011 to 0.0160, and -0.0001 to 0.0186. Company-clustered: 0.0013 to 0.0166, and -0.0011 to 0.0214. So fold 2 alone does not exclude zero.

Secondaries: patent total +0.0083, per project +0.0084, recent five years +0.0064, without SBIR and STTR +0.0087. Baselines on the same rows are in the result file (fold 1 and fold 2): rubric 0.692 and 0.700, most papers 0.653 and 0.667, NIH dollars 0.621 and 0.618, random 0.534 and 0.513. The citation-rank and plain-LLM baselines are in the result file but not in this list: citation rank did not run on fold 2, and the stored plain-LLM read is on a case-control sample, so it is not comparable with full-fold AUC.

Adversarial audit (post hoc, cannot change the verdict)

backtest-auditor rebuilt the features from raw files and matched the gain exactly.

• Survives: within-institution gain +0.0088 and +0.0103; a project-count and institution-rate control leaves +0.0080; the within-institution shuffle's maximum over 150 draws was +0.0031.
• No direct encoding found: no hit's patent assignee matches the company name, though patents were not read one by one for shared technology.
• Where the gain sits: low-ranked hits that hold patents moved up. Hits below the top quarter of v4 give about 90 percent of it; hits in the top 5 percent contribute slightly negatively. Top 500 hits fell from 36 to 30 in fold 2 and rose from 34 to 35 in fold 1.
• Old patents: coverage roughly halves at 5 years, so fading there fits reporting lag and plain information loss equally well.

What this does and does not mean

• May be said: NIH-linked patents added about 0.008 AUC to the v4 logit on two seen folds. The gain was in the middle of the ranking, did not improve precision at the top, and did not hold on patents at least 5 years old.
• May not be said: that lab selection improved, that top-of-list precision improved, that patents are a validated signal, or that the live ranker improved. The live S0 model is a logit-plus-GBM ensemble, and this test used the v4 logit.
• By hard rule 5 this is not yet a decision improvement. A VC reads the top of the list, and that did not get better.

Limits

1. The NIH file covers NIH-supported patents, is incomplete by NIH's own statement, and is 2026 state.
2. 25 of 30 sampled links have the PI as a named inventor. The rest are supported by the grant but invented by others.
3. Labs only; two folds; labels already seen. Agent-checked throughout: no human read the links or the audit.
4. The white-noise check was weak (dense random columns against sparse real ones); the lab shuffle is the real null.
5. Dates: the raw folder and these documents say 2026-11-05 (UTC); the commits are stamped 2026-11-04 US Eastern. Same night.

Prospective freeze

Frozen 2026-11-04 US Eastern, from clean committed code, under addenda 2 and 3 of the pre-registration: 27,379 labs, 2,754 with a patent, sha256 554a9f5047fd0a85cb3a279d2d10c8a3dd215bc51e93428dd73c03448c981e23. Rank correlation with frozen v4 is 0.962. A same-environment refit of v4 without patents correlates 0.99999999 with frozen v4, so the environment explains almost none of that difference. First read 2028-06-30; final 2030-06-30. This ranking has no retrospective read (it is the ensemble, and the test above used the logit), and it does not change any live list.