How it was tested

The method, what each result is compared with, and the known limits.

The method

• As of a date. Every fact carries the date it became public. A test set at a past date sees only what was public that day; one rule in the code separates what was knowable from what came later.
• Written down first. Each test, its pass mark and its comparisons were fixed and fingerprinted before the result was read.
• Always compared. A result is shown beside chance, simple rules such as paper count, a hand-set scorecard, and where one exists an AI given the same facts.
• Who graded. Answer keys are built by code from dated public events. Where a judgement was needed, such as whether a claim matches its source, a separate AI made it. No person graded anything.
• Public data only. Papers, grants, patents, trial records and SEC filings.

Known limits

• The seed-stage result is the one fresh test that passed its mark. Over the whole list the model and an AI given the same facts are level.
• The two lab results on the Results page come from dates the model was designed with in view. They are a best case, not a fresh test. The fresh lab test passed its mark, but did not hold up when a second audit corrected the labels.
• "A lab formed a company" is read from SEC filings that name the lab head. An audit found this misses most spinouts, so every lab number is measured against an incomplete answer key.
• The report check measures whether a claim matches its source. It does not measure whether the source is true or whether the read was right. The wrong claims slipped in were easy ones.
• In the worked runs the AI was given the company's name, so it may know what happened later. Long filings were read in part. Only public companies with an annual report can be run.
• The valuation range is context from earlier deals, not a valuation. No stage gives a verdict.
• Reports and briefs on this site are examples. None has been verified by a person or published as a memo.

Not on the Results page

9 of 16 registered tests are not shown as results, because they did not pass, were too small to judge, or are not finished:

• How early a founding lab was surfaced before its field crowded: not finished yet.
• Find the founding lab from the company's first description: too small to judge.
• How good are the company-forming labels: did not pass.
• Do past company-forming rates predict an institution's later spinouts: could not tell either way.
• Name the competitors a later 10-K names: not finished yet.
• Does the next financing fall inside the runway band: not finished yet.
• Does the science reader doubt programs that later failed: not finished yet.
• Does the science reader doubt papers later retracted: did not pass.
• Was the deal later written down: not finished yet.

The detail