How RoutingCat works

RoutingCat shows each visitor one variant of each slot on the pages you choose. It learns from your conversions which variants work, and for whom, and it measures what routing adds. This page explains enough to predict what it does and to read its numbers.

The words

page: a page of your site that RoutingCat can vary.
slot: a place on a page that can change, such as the headline.
variant: one alternative for a slot.
default: the variant your site shows today. Every slot has one.
version: one state of a variant’s content. Each edit makes a new one, and every version is kept.
control: the 10% of visitors who always see the defaults.
visit: a visitor’s 7-day window, below. RoutingCat counts in visits.

From code to visitors

Variants are markup: each slot is written in your templates, every variant in the page, and routingcat.config.json names the pages. Slots vary independently, so every combination of a page’s variants must read well.

npx routingcat push registers your templates for production, in your CI or your host’s deploy script after each deploy, or from your machine with --env production once you confirm its plan. A change is live within about 2 minutes of both the push and the deploy. npx routingcat plan shows beforehand what will go live.
A push in CI on any other branch registers a preview, which changes nothing live. A host your config lists under environments.preview, such as a staging server, records to that preview, out of your results.
On a host your config lists in no environment, such as your own machine, your pages run in development mode: they show the defaults and record nothing.
On Hobby, routing starts once a card is on file, with nothing charged, and your sites serve at most 10 live variants together, the defaults not counted: a variant past them is held until there is room. See Plans, billing and emails.

Acts need no deploy. Pausing, resuming, promoting or retiring a variant, reverting a page, or changing a routing setting with the command is live within about 2 minutes.

A production push applies only what your templates changed since the previous one. So an act holds until a push changes the same thing in your templates, and a pause holds until you resume.

After an act, update the code to match: Change what’s live shows the edit for each act, and npx routingcat status lists where what’s live still differs from your code.

What happens on a visit

RoutingCat’s script picks the visitor’s variants in the browser, before the page paints, from routing data the browser fetches from RoutingCat and keeps for a day. A visitor’s first view of a page, with none kept, waits for it: the page’s routed slots stay hidden until it arrives, at most 1,000 ms by default, which wait_ms sets, and past that show the defaults (Run it in production).
The visitor keeps the same variants for the whole visit.
Your page carries every variant, the same for every visitor, and RoutingCat shows the visitor’s own before the page paints. The browser then reports what it showed.
Your code reports conversions and links visitors to accounts, from your server through the HTTP events API, or from the page with window.routingcat.track() and window.routingcat.identify().
A conversion is credited through the visitor or their account, so it counts even on another device.

RoutingCat also notes where each visit came from, on every page it runs on, routed or not, which is how sources shows where conversions come from.

Visits and windows

A visit starts the first time a visitor reaches a routed page, and lasts 7 days by default.

On your site, RoutingCat’s script writes the visitor id, which Safari deletes after 7 days without a visit: one person can then count as several visitors over a month, as in browser analytics. The server helpers set it from your server instead, for 400 days.

A conversion counts for a page when it happens within the objective’s window after the visitor first saw that page, 24 hours by default. It counts at most once per visit.

Maturity. Once the window has closed, the visit’s outcome is known, and the visit has matured.

Figures over matured visits are settled.
Figures over all visits so far are marked incomplete, and read low, since recent visits can still convert.

See Conversions and visits.

How routing learns

Routing learns to get more of each page’s objective: the main conversion you declare, such as signup. Other outcomes you track, such as purchase, are reported beside it but never steer routing.

Routing is a contextual bandit:

A Bayesian model of each page estimates how likely each variant is to convert, and for whom.
Thompson sampling gives each variant traffic in line with the probability that it is the best.

In practice:

It learns once a day on Hobby, and on Pro as conversions mature, at most hourly, from visits whose outcome is known. With the default window, a visit starts to shape routing a day or two later on Hobby, about a day later on Pro.
It favors what is likely best, and keeps exploring. An uncertain variant keeps being shown until the evidence settles, and a small random slice of visits is always split evenly.
It learns for whom, not only what. When some visitors respond differently from others, routing can learn to show them different variants. Each such difference must earn its place with evidence; until then, everyone is routed alike.
New variants start cautiously, from what the page’s other variants taught it, with a limited share of traffic until evidence builds.
An edit is not a new idea. An edit under the same key builds on what the variant already showed. A new idea goes under a new key and is compared with the default at the same time.

Two settings change this per slot:

A fixed share splits traffic evenly and keeps each visitor on the same variant across visits. At 1, it is a classic A/B test, the right choice for prices, since routing toward conversions would favor the cheaper price.
Personalization off routes every visitor the same way.

On your site, a variant your server honors, such as a price or a discount, is a trade-off: each visitor’s browser picks their variants from routing data anyone can read, so a determined visitor could pick the better one, while most visitors browse normally. Your agent explains this before it builds such a variant, then does as you decide: see Weigh a variant your server honors.

See How routing learns and Routing settings.

The control and the measured gain

The control is 10% of visitors, chosen at random, who always see the defaults, on every page and every visit. Because the choice is random, comparing routed visits with the control measures what routing changed, without relying on the model.

A page’s measured gain is the extra conversions its routed visits produced, compared with showing everyone the defaults, with a 95% range.

A range that spans zero means the direction isn’t known yet. “+23 sign-ups, between −95 and +141” is too early to call.
A promote keeps it. A promoted variant becomes the default, which the control sees too, and measurement continues against the new defaults.
Each page’s gain stands alone. Visitors see several pages, so gains don’t add up to a site total.
The range is not a stopping rule. Checking daily and acting on the first good day overstates gains.

Three kinds of number

Kind	Examples	Read it as
Counted	Visits, conversions, rates, revenue, the share each variant was shown	What happened. Exact, but routing shows variants to different visitors, so raw rates aren’t a fair comparison.
Predicted	A variant’s predicted rate, its difference from the default, signal, labels	The model’s current belief, with a 95% range. It moves as evidence arrives, and holds only as far as the model fits your visitors.
Measured	The measured gain, and a fixed share’s comparison of its variants	From a comparison set up by chance, without the model. The measured gain is the only figure that counts as a gain.

Decisions your visitors’ browsers make are replayed by RoutingCat from what each browser recorded, and one that doesn’t replay is left out of every result. Results that rest on them carry verification: unverified in the command’s JSON: replay shows that each decision follows from its inputs, never that a person saw it.

Signal says how much evidence backs a variant’s predicted rate, from 0 to 100: New, Early, Building, Strong. It is not a probability of winning.

Labels say what routing does now:

favored: the variant it intends to show most;
learning or prior: whether the estimate comes from your own results yet.

None is a verdict: what to keep is a judgment on the evidence, yours or your agent’s. See What the numbers mean.

What updates when

What	When
A deploy or an act reaching visitors	Within about 2 minutes
Live activity	Within a minute
Counts, and most warnings	Every hour; every 15 minutes on Pro
Learning	Once a day; on Pro, as conversions mature, at most hourly
The measured gain	Once a day
Predictions	After each fit of the model, and soon after each deploy or act

On your site, each visitor’s browser decides from the routing data it kept, up to a day old, and fetches fresher data as a page shows once its copy is more than a minute old. So a returning visitor’s first page after a change or an act can still show what was live before, and their next page follows it. Turning routing off reaches open pages too: a page whose routing data is more than 5 minutes old checks as it shows, and switches to its defaults.

When something fails

Visitors always get a page:

If RoutingCat is unreachable, a browser that kept routing data decides from it, for up to a day; a first view, with none kept, shows the defaults once its wait runs out.
A push that can’t reach RoutingCat registers nothing and exits with code 4. Your deploy is unaffected, but where it changed your templates, your pages show the defaults and record no decision until a push gets through.
A browser that doesn’t run RoutingCat’s script, with JavaScript off or the script blocked, shows the defaults, and its visit isn’t recorded.
A browser without the style rules RoutingCat uses, such as Safari before 15.4, Chrome before 105 or Firefox before 121, shows the defaults.
A page whose routing data passes 128 KB serves its defaults, unrouted, until a push brings it under.
window.routingcat.track() and window.routingcat.identify() never throw, so they never break a form.