deal-v0.1 results (IDEA-014 M5-M7)
Run 2026-11-03 under the spec committed at 7d8d61d, after a pre-result audit with two re-checks (docs/audits/idea014-hindcast-preresult-audit.md). Files: data/processed/acquire/r2.json, r3.json, noise.json. All on past data; nothing here is a skill claim.
Short version
- The acquisition check runs from public data for companies in the database: evidence, dated comparables, a verdict and what would change it, with source ids on evidence and comparables. With
--target, an as-of date after the company's own deal is accepted and labelled, not refused;--dealis the hindcast path. - The verdict is not validated. Whether past deals worked cannot be tested on local data.
- The rubric shows no measurable edge over counting trials at separating acquired from not-acquired listed companies. The interval cannot exclude a difference of about 0.07 AUC either way, and this is descriptive, not a test.
R2: which deals worked (not which deals to do). Result: cannot test
| horizon | deals | positives | censored | minimum detectable AUC |
|---|---|---|---|---|
| 36 months | 69 | 4 | 55 | 0.920 |
| 60 months | 48 | 2 | 76 | 1.000 |
With 4 positives, any AUC is noise, so they are in r2.json as description only and support no statement. Positives are probably undercounted, because acquired programmes can move to the acquirer's name in the registry; that was not measured. A usable label needs intervention names (R4, a free public fetch on the owner queue).
R3: does public evidence separate acquired from not-acquired? Descriptive; label noise unmeasured
Among trial-bearing companies listed at T, acquired by an SEC-registered acquirer in a parsed filing within 36 months. T = 2020-12-31 (33 of 406) and 2022-12-31 (52 of 490). This measures acquisition, not value: being acquired is not the same as being worth buying.
| scorer | within-date mean AUC | 95% CI, company-clustered | T=2020 | T=2022 |
|---|---|---|---|---|
| deal-v0.1 rubric (identical without economics) | 0.578 | 0.503 to 0.655 | 0.596 | 0.560 |
| trial count (ruled citation-rank substitute) | 0.583 | 0.505 to 0.657 | 0.609 | 0.557 |
| stage only | 0.559 | 0.488 to 0.625 | 0.596 | 0.521 |
| random | 0.489 | 0.422 to 0.558 | 0.443 | 0.535 |
| blind LLM on anonymised packs | 0.586 | not computed | 0.612 | 0.560 |
Rubric minus trial count: -0.005 (95% CI -0.079 to 0.069). No measurable edge over a trial count; the interval cannot exclude about 0.07 AUC either way. By the plan's rule this is the finding; deal-v0.1 is not tuned, and any change is deal-v0.2.
Secondaries look stronger and should not be read that way: trial-bearing any 0.714, all companies with a dated trace 0.715. Those pools hold private companies that the ledger cannot label as acquired. A plausible, untested explanation for the higher figures is that they partly measure "was public".
Limits that bind every number above
- Trials link to companies by 2026 sponsor name, not point in time. The direction of the bias is not guaranteed. Real fix: R4.
- Label noise was unmeasured at this run; **update 2026-11-04: D3 has since run, and an estimated 44 percent of true acquisitions are missing (95 percent interval 9 to 72), see
deal-r4-d3-results.md. R3 stays descriptive.** At this run, the 30-company check (D3) had not been done, which alone kept R3 descriptive. Negatives with a delisted-acquired or delisted-unknown row in the window: 8 at 2020 and 31 at 2022, against 33 and 52 positives, so missed acquisitions could be a large share of the true events. exit_dateis deal completion, so some positives were already announced at T.- There are no declined deals in public data. Nothing here observes a deal someone chose not to do.
- Blind LLM, the row in the table is not a clean comparator. Route: path-mode subagent, blind by instruction, not by tool restriction. The answer key sat in its folder during the run; its tool log shows 16 reads of the pack file, and its 3 shell commands were read and neither list nor glob that folder. The model's training data postdates these deals. It reported 6 of 965 packs as recognisable; that is its own report, not a measured rate, and unrecognised packs may still be recalled. All 6 are in R2 (AUC 0.681 with them, 0.644 without; description only). The spec required a 20-pack check for sponsor or product strings; that was not done. An automated check ran over all packs for words of 5+ letters from the company's own name only (3 hits, all condition words), so the spec's condition was relaxed, not met: product and acquirer names were not tested. R2 and R3 packs were scored in one batch and 26 companies appear in both. It is reported because hard rule 6 asks for this baseline; its 0.586 is not evidence that it beats the rubric.
- Comparable values are filed lower bounds. Stage-cell floors span orders of magnitude, and the frozen bands call phase 2 plus one recent trial "strong", so "buy" fires for most typed prices in a populated cell. Treat the verdict as a framing device for now.
Deviations from the spec, all logged in decisions.md
Registry termination proxy not run. Holm not applied, since descriptive mode runs no test. All-companies secondary narrowed to a dated trace. Headline moved to a within-date mean before any number was computed. Leak check covered all packs by own-name words rather than a 20-pack sample.