Score test #2: the plan, written before the data
Written November 11, 2026. Published November 14, 2026, before any result of this test exists. The analysis runs once, on March 1, 2027.
Why a second test
Our AI reads each earnings report and gives it a score from 0 to 100. Test #1 looked at 492 S&P 500 reports from July to September 2026. Reports scored 88 or higher beat reports scored below 75 by 7.1 points over the next 40 market days.
Then a Reddit reader, u/QuanTradin, pointed out what that number could be hiding, and our recheck showed most of it came from two things: the high-scored group had more tech stocks, and a handful of big winners. Comparing stocks only with others in the same sector, the gap shrank to 4.2 points. Against each stock's own sector fund it was 1.7 points, and the luck range crossed zero. So the honest answer is: we don't know yet whether the score helps.
Test #2 asks the question once, with the stricter checks as the main test, and with every rule below decided before any of the data exists.
The question
Within the same sector, do reports our AI scores 88 or higher beat reports it scores below 75 over the next 40 market days?
The setup (frozen)
| Companies | Every company on the S&P 500 list we saved on Oct 15, 2026 (file fingerprint, SHA-1: 74ec814111602ede1b7de51a92eae0786ea1223a). A company that leaves the index later stays in. |
|---|---|
| Reports | Every earnings release (SEC form 8-K, Item 2.02) filed from Oct 1 to Dec 31, 2026, dated by the SEC's New York acceptance time. One report per company per quarter: a second earnings filing within 25 days is skipped. Filings that aren't earnings reports (dividend notices, stock offerings, mergers, "we'll report on this date" notices) are left out by the same rule the site already uses. |
| The score | The site's own score, made by the same scoring program the site uses (fingerprint, SHA-1: 7dadff9f5b2508c8074bd68f2b496a19a59f6744) with the AI model claude-sonnet-5-5, whose knowledge ends in June 2026, so every report is newer than anything it learned. A report the site already scored live keeps that score; it is never scored again. The rest are scored once, in one batch, with the same instructions. The AI reads only the report text, never prices. |
| No filter | The site's price screen is NOT used here. Every report from the companies above counts. |
| Buy day | The closing price of the first market day the report was public. A report out before the market opens uses that day's close; a report during or after market hours uses the next market day's close. |
| Result | The stock's return minus the return of its own sector fund (the SPDR sector funds: XLK, XLF, XLV, XLY, XLP, XLE, XLI, XLB, XLU, XLRE, XLC; sector from our saved S&P 500 list), over 40 market days. Prices include dividends, for both. |
| Groups | High: score 88 or more. Middle: 75 to 87. Low: below 75. The same groups as test #1. |
The main number
The within-sector gap. In each sector with at least 3 high and 3 low reports, take the average result of the high reports minus the average of the low reports. Then average those sector gaps, giving each sector a weight equal to the size of its smaller group.
The luck range
How much the gap could move by luck alone, as a 95% range. We group reports by week and sector and resample those groups 10,000 times (a block bootstrap, random seed 11). We also show the range by week only and by sector only, but those two don't count toward the verdict.
What counts as a pass (decided now)
PASS means all five of these hold:
- The within-sector gap is at least +2.0 points.
- The low end of the luck range (week × sector) is above 0.
- With the biggest 5% of winners and the biggest 5% of losers removed (out of all reports), the gap is still above 0.
- With the 10 reports that help the gap most removed, it is still above 0.
- The gap is positive in more than half of the sectors that count.
PROMISING means checks 1 and 3 hold, but 2, 4 or 5 fail. We keep watching for one more season and claim nothing.
FAIL means the within-sector gap is 0 or lower, or the trimmed gap (check 3) is 0 or lower. The score does not beat sector peers at 40 days, and we will say so publicly.
Also shown, but not part of the verdict: results at 20 and 60 market days, results against the S&P 500 instead of the sector fund, middle vs low and high vs middle, and the rank correlation between score and result.
Timing
- Reports filed through Dec 31, 2026. The last 40-day window closes around Feb 26, 2027.
- The analysis runs once, on Monday, March 1, 2027 (or the first day after every window has closed). Nobody runs the main number before then. Simple counts, like how many reports have been scored so far, are fine.
- The reports the site didn't score live are scored once, in one batch, in early January 2027.
- The result goes on this site and in a video, whether it's PASS, PROMISING or FAIL, with a link to this page.
Known limits (said up front)
- It's one season again, about 450 to 500 reports. A pass would be evidence, not proof.
- The Oct 15 list is close to the index on each report day, but not exactly the same.
- Sector funds are mostly the big names in each sector, so a small company can lag its fund for reasons that have nothing to do with its report.
- Prices come from Yahoo Finance (closing prices adjusted for dividends and splits).
- Reports from Oct 1 to Nov 11 were already public when this plan was written. We'll also show the result for reports filed after Nov 11 on their own, as the cleanest check. (Added November 14, 2026.)
Changes to this plan
If anything in this plan changes before March 1, 2027, the change, its date and the reason are listed here. The rules above are never edited quietly.
- November 14, 2026: Added to Known limits, after a second review: reports from Oct 1 to Nov 11 were already public when this plan was written, so we will also show the result for reports filed after Nov 11 on their own, as the cleanest check. This adds a check; the main test and the pass bar are unchanged.
A paper-money experiment for learning and fun, not financial advice. The score rates a report; it is not a prediction. About Bots4Stocks · How it works