Same checklist as the e-commerce, MovieLens, and ACNH case studies, extended to forecasting. The rules for judging the forecast were written down before the data was opened, so they could not bend to fit the results.
A sales analysis and revenue forecast of Online Retail II (a public dataset: 1,067,371 order lines from a UK online gift retailer, 2009–2011), with the data quality checked first, by Alex Kwon. Verdict in one line: the simplest honest method, “predict the same week as last year,” beat every model we tried, and the report says so.
Weekly revenue from real product sales, after cleaning. The shaded band is the test period: the final 13 weeks (Sep–Nov 2011), hidden from every forecasting method and used only to grade them. The dashed line is the winning method, “predict this week will look like the same week last year” (the textbook name is seasonal-naive). It tracks the Christmas ramp because last year had one; every method without a memory of last year missed the ramp by twice as much.
| Week | Actual | Seasonal-naive | Error |
|---|---|---|---|
| 2011-09-05 | 186,085 | 137,799 | +48,286 |
| 2011-09-12 | 222,719 | 128,263 | +94,456 |
| 2011-09-19 | 320,224 | 161,372 | +158,852 |
| 2011-09-26 | 202,648 | 236,765 | -34,117 |
| 2011-10-03 | 301,209 | 288,014 | +13,194 |
| 2011-10-10 | 202,641 | 254,859 | -52,218 |
| 2011-10-17 | 259,712 | 306,144 | -46,431 |
| 2011-10-24 | 235,086 | 233,486 | +1,600 |
| 2011-10-31 | 289,610 | 202,945 | +86,664 |
| 2011-11-07 | 351,159 | 275,742 | +75,417 |
| 2011-11-14 | 370,687 | 360,749 | +9,939 |
| 2011-11-21 | 296,531 | 344,289 | -47,757 |
| 2011-11-28 | 307,087 | 287,699 | +19,388 |
The test date was written down before any model existed, so no method could be tuned to look good on the answer key. One planned model (a seasonal smoothing model) turned out to be impossible to fit honestly: it needs at least two full years of training data and only 89 weeks were available. Its slot went to a simpler trend model, and a built-in check confirms no method ever saw the test weeks.
| Method | Avg weekly miss (£) | Avg miss (%) |
|---|---|---|
| seasonal-naive (this week last year) | 52,948 | 20.0% |
| moving average (last 4 weeks) | 114,202 | 39.1% |
| Holt, damped trend | 128,101 | 44.4% |
| naive (last week) | 135,199 | 47.2% |
The obvious next idea, deliberately not taken: the winner still under-predicted the test period by 9.2% (£3.22M vs £3.55M actual), because the business grew year over year and “same as last year” cannot see growth. Adding a growth adjustment would probably win, but it was not part of the plan written before the test, so it is listed as future work instead of being quietly added after peeking at the answers.
“Weekly revenue” meant nothing until the cleanup decided what counts. Every problem found, and what was done about it, is logged in FINDINGS.md with the row ids that prove it:
| Problem found | What was done |
|---|---|
| The file's two sheets repeat the same nine days of data, as exact copies (22,523 rows, £377k) | removed the repeat; nothing lost |
| 11,812 identical rows inside the data (£54k, 0.29% of revenue): real repeat purchases or a glitch, the data cannot say | kept, with the numbers also published without them |
| Six accounting entries for unpaid debts (−£147,614) mixed in with sales | excluded: bookkeeping, not sales |
| 3,393 warehouse notes (“damages”, “found”, “thrown away”) filed as if they were orders | excluded from unit counts |
| 14 fee and adjustment codes posing as products (Amazon fees, bank charges, even a literal test product) | excluded, by a published list |
| The two largest “orders” ever (74,215 and 80,995 units) | typos, cancelled 12–16 minutes later; the purchase and the cancellation cancel out |
| Gift voucher names looked corrupted on screen | checked the raw data: it was fine; the display was wrong, not the file |
The UK is 85.5% of product revenue (Ireland 3.2%, Netherlands 2.9%). Returns run 3.65% of sales. The catalog is a long tail: the top 10 of 5,291 products carry only 7.6% of revenue, so no single product's forecast matters much; the total is the story. November–December carries 21–24% of the year, and the biggest week (Nov 14, 2011) took in £370,687.
⚠ limits
The data holds only two Christmases, so the holiday pattern's shape is trustworthy but its exact size is not. The last few weeks look slightly better than they really were: any cancellations that arrived after the file ends are missing. Whether the 11,812 identical rows are real purchases or a glitch cannot be settled (either way the difference is 0.29% of revenue). And none of this says anything about any other business.
Analysis: Alex Kwon. Full workings, findings with ids, runnable SQL and scripts, the frozen series, and the per-week backtest: online-retail-forecast (FINDINGS.md, REPORT.md, sql/01–03, scripts/01–05, series/). Every figure independently recomputed before publication; the chart geometry on this page is generated from the frozen CSVs at build time.
Cite: Kwon, A. (2026). The Bar Was Seasonal-Naive: an audit-first sales analysis and demand forecast of Online Retail II. github.com/collapseindex/online-retail-forecast (machine-readable: CITATION.cff in the repository). License: report CC BY 4.0; code Apache-2.0. Disclosure: AI-assisted implementation; methodology and validation framework developed in prior research; every figure recomputed from raw data by the runnable code in the repository.
Alex Kwon · ask@collapseindex.org · more case studies · github.com/collapseindex