Power Law Performance Subscribe

ResourcesForecasting: Revenue

Back Testing

The whole book four ways, then every method made again from each close since 2021, against what billed.

Chapter 3 of 3 · 10 min read · A guide by Broad Eye Solutions · Version 1.0 · Updated November 14, 2026

Get the whole guide as a PDF

Chapters 1 and 2 built each half of the book four ways. Add them, method by method, and you have the whole book: a full forecast for the business from any of the four. This chapter adds them, then asks which to trust, and how far ahead.

The whole book, four ways

Each method’s existing book plus the same method’s net new in year.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: Straight Line, Seasonal, Cohort · made by the engine

Next year comes to $38.15M by Straight Line, $38.15M by Seasonal, $38.32M by Cohort and $38.38M by Power Law. The four land within 0.6% of each other on the total.

Sample data: an invented payments portfolio, no client data. In the repository: Excel, tab 3.1 Whole book · CSV: Straight Line, Seasonal, Cohort

Close on the total doesn’t make them equally good. They differ in the months, and nothing so far says how far any of them can be trusted. A back test measures that.

How the back test works

Stand at every month-end close from December 2021 to September 2026, 58 closes in all. Make each method’s forecast for the whole book with only the statements on hand that day, then set every month of it against what the book went on to bill. Go as far ahead as each method reaches.

What You’ll Need

Nothing new: the statements you already have, and the discipline to use only what had closed at each close.

The Five Methods, As They Were Run

  • Straight Line. Chapters 1 and 2. Net new: for the rest of the year, the months closed divided by last year’s share of them; next year, this year times 1.10, on a straight line from this year’s January share. The existing book: Chapter 2’s Straight Line, the same way: this year closed out by last year’s shares, next year that total grown by how much bigger the book comes in (this year’s December plus this year’s new accounts), on a straight line from this year’s January share. It reaches December of the next year.
  • Seasonal. Net new: the same numbers, with every open month at its average share of the year. The existing book: Straight Line’s numbers, with every month at its average share of the year.
  • Cohort. The existing book: Chapter 2’s cohort layer. Net new: an engine carries this year’s new accounts forward on the path older cohorts followed at the same age, and brings in the accounts not approved yet at the plan’s count, each billing what an account billed at the same age across the last 24 approval months. Both run on the whole book’s own time of year: each month read against the twelve around it. Past next December, the layer runs a second year from its own December, with next year’s new accounts joining in January. It reaches 24 months.
  • Power Law. The same, with Chapter 2’s tier layer and each tier on its own grid in the engine. It reaches 24 months.
  • Trend line. The usual first cut through the year, added here so it can be judged the same way: a straight line through the whole book’s last twelve monthly totals, carried forward. It reaches 24 months.

A forecast counts only where its method had history to read at that distance. At the December 2021 close the engine had nothing to read 24 months ahead, so that one is left out.

What The Engine Does

Net new in year for Cohort and Power Law is too many cells for a spreadsheet, so an engine runs it.

  • It reads each account’s approval month, what it billed every month, and its tier.
  • It assumes each cohort keeps following the path older cohorts followed at the same age, and that the whole book’s time of year repeats. Accounts not approved yet arrive at the plan’s count, each billing what an account billed at the same age across the last 24 approval months.
  • It puts out each new account’s revenue for every month up to 24 months ahead, added up by approval month and by tier.

Every projection against what billed

From each close, every method’s forecast runs into months that hadn’t happened yet. Here’s how each one looked against what the book went on to bill. Switch between the closes under the title.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: December 2021, October 2022, October 2023, October 2024, October 2025 · made by the engine

The strength of each method, by months ahead

Every close gives one forecast for each month ahead. Line up the misses by how far ahead they were made, and each distance gets a typical size of miss.

Sample data: an invented payments portfolio, no client data. In the repository: CSV · made by the engine

One month ahead, the typical miss is 3.6% for Straight Line, 1.4% for Seasonal, 0.6% for Cohort, 0.6% for Power Law and 6.2% for the trend line. At twelve months it’s 6.0%, 5.0%, 4.0%, 4.1% and 7.5%. At every distance it reaches, Straight Line misses more than Seasonal, since a line can’t see the time of year, and less than the trend line. At twenty-four months the trend line misses by 10.9%, against 2.4% for Power Law. Past twelve months the typical miss stops climbing. The forecasts that reach that far land in only three calendar years, and there the accounts already on the book run about 1.5 points of the whole book low while the plan’s count runs about 3.4 points high, so the two partly cancel.

Each method also has a band: its average miss plus or minus one standard deviation. Switch between the methods under the title. The scale stays put, so the bands can be compared.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: Straight Line, Seasonal, Cohort, Trend line · made by the engine

Twelve months ahead, Straight Line’s band is about plus or minus 7.1 points, Seasonal’s 6.1, the trend line’s 9.3 and Power Law’s 4.7. At every distance Straight Line and Seasonal reach, Seasonal lands closer than Straight Line and holds a tighter band, and Cohort and Power Law land closer and tighter still. Out to two years the builds made from the accounts hold the tightest band of the five.

The year and the quarter

A budget is held to the year and reviewed by the quarter. From each planning close, here’s each method on both.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: The whole book, full year; Net new in year and the existing book, full year; The whole book by quarter · made by the engine

On the whole book’s year, Straight Line and Seasonal missed by 2.0% on average (they share a year), Cohort by 2.0% and Power Law by 2.1%: level. The trend line missed by 4.8%, high every year.

The quarters are where the methods part. Power Law’s quarters missed by as much as 4.3% and Seasonal’s 4.8%. Straight Line’s missed by as much as 9.0%: a straight line runs under the spring and summer and over the fall. The trend line’s first quarter ran 9% to 15% high in every year: the book bills less in January and February than in December, and the trend line can’t see it.

The split says where the year is won. On the existing book’s year the four are level, each missing by 1.3% to 1.5% on average. On net new in year, no method wins: Straight Line and Seasonal missed by 7.4% on average, Cohort and Power Law by 8.1%. Every one of them rests on the plan’s count of accounts nobody has signed yet.

Where the miss comes from

The same split, month by month: each part’s miss as points of the whole book’s month.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: Net new in year, The existing book · made by the engine

Twelve months ahead, Straight Line’s typical miss on net new is 1.5 points of the whole book and 4.9 on the existing book. Seasonal’s is 1.5 and 3.7, and Cohort’s 1.2 and 3.1. On the existing book each layer takes a little more off: the time of year takes Straight Line’s 4.9 to Seasonal’s 3.7, and the cohorts take it to 3.1. On net new, the straight line keeps up with Seasonal, since new accounts add up through the year, close to a straight climb, and the builds made from the accounts are closer.

2026 so far

Chapters 1 and 2 made each method’s Reforecast at the 3+9, the 6+6 and the 9+3, and stopped before what happened. Here’s what happened, through October.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: 3+9, 6+6, 9+3 · made by the engine
Sample data: an invented payments portfolio, no client data. In the repository: CSV: Net new in year, The existing book · made by the engine

On net new, from the March close to October, Cohort’s and Power Law’s forecasts ran +3.4% and +3.4% against what billed. Straight Line’s ran −19.3% and Seasonal’s −20.0%. Neither looks close on the year: both put it at $3.17M at the March close, and January to October alone went on to bill $2.68M. That leaves $0.49M for November and December, last year’s two biggest months for new accounts. On the existing book every layer stayed within about two points, Straight Line included: −1.0% from the March close, +0.2% for Seasonal.

Every method has limits

Straight Line: A Line, Not A Shape

A straight line from January’s share fits new accounts, which add up through the year. On the existing book it gets the year about right but not the months, which move with the time of year: one month ahead it misses by 3.6%, against 1.4% for Seasonal. Its band is wider than Seasonal’s and the account builds’ at every distance.

Seasonal: The Right Shape, A Wide Band

Seasonal fixes the shape. Its average miss stays within 2.0 points of zero at every distance, +2.0% at twelve months, and on the year it’s level with the account builds. But it reads the whole existing book at one rate and net new from the averages, so any one forecast can land 6.1 points either side of that, twelve months ahead.

Trend Line: No Shape, No Accounts

The trend line knows only the whole book’s last twelve totals. It can’t see the time of year, the accounts or the plan. Past a year, its band is far wider than the account builds’.

Cohort And Power Law: The Same Total

The two lines sit on top of each other out to eighteen months: 4.0% against 4.1% at twelve. At twenty-four months Power Law is closer, 2.4% against 2.5%. Splitting the book by tier barely moves the total. What the tiers add is where the year comes from: on this book, 8% of the existing book’s accounts bill 53% of its December revenue.

Both lean a little high further out: Cohort by +2.2% at twelve months and +2.0% at twenty-four, Power Law by +2.1% and +1.9%. The plan’s count adds 10% a year. On this book the new accounts that came in beat it by 5.5% in 2023, then fell short of it by 4.4% in 2024, 9.3% in 2025 and 8.6% so far in 2026.

The Engine: Only As Good As The Count It’s Fed

Every build forecasts what it can see, and the accounts already on the book can be seen. What none of them can see is the accounts not approved yet. How many, and when, is an input: here, the plan’s count. That’s why no method wins on net new’s year. Run the same engine with every account approved after the close at the latest cohorts’ pace instead, each month still to come billing what the average approval month of the last 24 billed at the same age, and it drifts low the further out it looks.

Sample data: an invented payments portfolio, no client data.

At twelve months the typical miss is 4.1% on the plan’s count and 3.4% at the latest pace. At twenty-four it’s 2.4% against 7.3%. The accounts already approved are forecast the same way in both, and at twenty-four months they run 1.5 points of the whole book low. The accounts approved after the close make the difference: 3.4 points high on the plan’s count, 5.8 points low at the latest pace. The latest pace doesn’t grow, and this book did.

The plan’s count moves the result too. Run Power Law with the plan at +5% and +15% instead of +10%:

Sample data: an invented payments portfolio, no client data.

Twelve months ahead its typical miss is 3.8% at +5% and 4.3% at +15%, both closer than Seasonal’s 5.0%. Two years out the plan decides more, 1.6% against 4.2%. On net new’s year it decides most: 6.8% at +5% and 10.7% at +15%. The method holds its edge on the months; the count you feed it sets how far out that edge reaches.

This Back Test: One Book, Overlapping Closes

Everything here is one invented payments portfolio, with sample data and no client data. The 58 closes overlap: past twelve months, each band rests on three or four calendar years of outcomes, not on 58 separate tests. Read a band as how this book’s forecasts landed, not as a promise for yours. The methods were also designed with the whole history in view, even though every forecast uses only what had closed. Run the same test on your own book before you lean on these numbers.

Here’s the band chart again.

Sample data: an invented payments portfolio, no client data. In the repository: CSV: Straight Line, Seasonal, Cohort, Trend line · made by the engine

Get the whole guide as a PDF

Every chapter, version 1.0, in one file, sent to your inbox. The newsletter follows with the articles that build on it.

One email a week after the PDF.

Your choice

Consent Preferences

Choose what this site can store in your browser. Your choice is kept in this browser for 6 months.

Strictly necessaryAlways on

Your cookie choice and the bot check on the sign-up forms. The site can’t work without them.

Cookie Policy Privacy Policy