A/B Testing Your Landing Pages with Short Links

Traditional landing-page A/B testing means installing a JavaScript snippet on your site, configuring a tool that splits traffic client-side, and dealing with the inevitable flash-of-original-content problem when the test variant loads slightly later than the control. Google Optimize handled this for years and was free, until Google sunset it in September 2023. The replacements (VWO, Convert, Optimizely, AB Tasty) start at $200/month and climb fast.

Meanwhile, the simpler approach has been hiding in your link shortener the whole time. Split traffic at the redirect layer, before the visitor reaches either variant, and you sidestep nearly every traditional A/B-testing pain point.

What redirect-layer A/B testing actually does

You create a short link pointing to two destination URLs and set a traffic split. When someone clicks the short link, Flyn's redirect server flips a weighted coin and sends them to one URL or the other. The visitor never sees the alternative variant; the variants don't need to share a server or framework; no JavaScript runs on either page to detect or coordinate the test.

When traditional A/B testing is still the right choice

Redirect-layer testing has trade-offs. If you want to test variants that share a single URL (e.g., two button colours on the same page), JavaScript-based tools are still required because the visitor needs to land on the same URL and see different content. If you want to test interaction-level changes (form steps, modal timing), you need DOM-level control. For pure landing-page tests where the variants are distinct pages, redirect-layer testing is faster, cheaper, and statistically cleaner.

The implementation is intentionally simple, one screen in the dashboard, one optional API field, no separate "experiments" surface to manage.

Setting up a test in the dashboard

1. Open the link drawer for an existing short link, or create a new one.
2. Open the A/B Testing tab.
3. Enter your Variant A URL (the original or control page).
4. Enter your Variant B URL (the test variant).
5. Type a weight into each destination's number field. Weights are relative whole numbers from 0 to 999, not percentages, so 50 and 50 and 1 and 1 both mean an even split. As you type, the editor shows each destination's resulting share as a percentage next to a bar, so you can see exactly what you are about to save.
6. Add more destinations if you want them. One link splits between 2 and 12 of them, and because the weights are relative, any ratio is expressible: 1 and 99 for a cautious canary, 3 and 1 for a 75/25 ramp.
7. Save.

The short URL stays the same. Share it as you would any link, in email, social, paid ads, QR codes. Flyn handles the split at the redirect server.

Setting up a test via the API

For programmatic test creation, send the required url field plus a rotation array. Each entry carries a url and a relative weight, with an optional label, and the primary url is listed as one of the entries:

POST /api/links { "url": "https://yoursite.com/hero-v1", "rotation": [{ "url": "https://yoursite.com/hero-v1", "weight": 50, "label": "A benefits" }, { "url": "https://yoursite.com/hero-v2", "weight": 50, "label": "B problem" }] }

The response echoes the rotation with a stable id minted on every destination. That id is what each click is recorded against, so when you later PATCH the list, send back every destination you are keeping with the id it already has and leave id off only for genuinely new ones. A rotation holds a maximum of 12 destinations; send more and the API returns a 400 rather than quietly dropping the tail. The older two-URL shape, abUrlB with abSplit (the percentage of traffic kept on the primary url), is still accepted on create and still returned on every link, so existing integrations keep working. On update it is refused for a link whose rotation holds more than two destinations, which is a 409 asking you to send the rotation array instead, but rotation is the only one of the two that goes past two destinations. The API documentation covers every field.

How the split actually decides

On every click, Flyn adds up the weights, draws a random point across that total, and serves whichever destination the point lands in. With weights of 60 and 40, roughly 60% of clicks go to A and 40% to B, and the ratio settles as volume grows rather than following a repeating order. The roll is independent on every single click: Flyn sets no cookie and stores no visitor assignment, so the same person clicking twice can land on different variants. This is the right model for top-of-funnel tests, where each click is treated as an independent traffic event. If you need one person pinned to one experience for the length of a session, that has to happen on your own pages, not at the redirect layer.

Bots and crawlers are the one deliberate exception. They are never sampled into the split and always receive the primary destination. That is an anti-cloaking guard rather than an oversight: a security scanner or a link preview fetcher must never be shown a different page than a human gets. It also keeps automated traffic out of your variant counts.

What gets recorded per click

The Flyn analytics dashboard tags every click with the destination that served it, the timestamp, geographic data, device, OS, and referrer. The link's own analytics page carries a Destination Performance panel: each destination listed with its click count, the share of traffic it actually took, and the share its weight intended, so any drift between the two is visible at a glance. A destination you remove from the rotation keeps its history and is marked as removed, and older two-destination A/B links get the same report including their past clicks. Below that, the recent-clicks log names the serving destination on every row, and the CSV export carries a Variant column, so you can pivot the raw clicks yourself. For conversion data, you'll cross-reference with your downstream analytics tool (covered in detail below).

Concrete example. You suspect your hero copy could be sharper. Currently you have a long, benefits-focused headline; you want to test a punchy, problem-focused alternative.

Step 1, Define the hypothesis

Write it down before the test starts. "I believe replacing the benefits headline with a problem statement will increase conversion rate by >10% because problem-first copy reduces the time visitors spend understanding what we sell." Store this in the link's description field in Flyn so you can read it back when the test concludes.

Step 2, Build the variants

Create two distinct landing page URLs. Don't deploy a JavaScript-toggled variant, for a redirect-layer test, you want genuinely separate pages so the test stays clean. URLs:

• Variant A (control): https://yoursite.com/hero-benefits
• Variant B (test): https://yoursite.com/hero-problem

Step 3, Add UTMs to each variant URL

So conversion data downstream can attribute back to the variant. Build them with the UTM Builder:

• Variant A: ?utm_source=ab-test&utm_medium=redirect&utm_campaign=hero-q2-2026&utm_content=variant-a-benefits
• Variant B: ?utm_source=ab-test&utm_medium=redirect&utm_campaign=hero-q2-2026&utm_content=variant-b-problem

Same campaign, different content. GA4 will record sessions and conversions by content value, letting you compare conversion rates between the variants.

Step 4, Create the Flyn link with A/B testing enabled

Set both URLs in the A/B Testing tab. Use a 50/50 split for first-test maximum data speed. Name the link with a clear slug (ab-hero-q2-2026) and tag it with ab-test for filtering.

Step 5, Drive matched traffic to the link

Use the same short link everywhere you'd normally use the landing page URL. Email campaign, paid ad, social post, QR code. Don't use different traffic sources for different variants, that confounds the test.

Step 6, Wait for statistical significance

At least 7 days and at least 1,000 total clicks before reading the result. Don't peek and call winners mid-test. The temptation to call results early is the most common A/B-testing mistake, and it produces statistically meaningless conclusions.

The hardest part of A/B testing isn't setting up the test, it's reading the results honestly. Most teams call winners too early and act on noise. The discipline below keeps you out of that trap.

The minimum sample size by effect size

Smaller expected lifts require larger samples to detect reliably. As a rough table:

If your link gets 500 clicks/week and you're looking for a 10% lift, plan a 10-week test. If you can't commit that long, accept that you're only powered to detect 20%+ lifts and frame the test accordingly.

The 1,000-click rule of thumb

For most marketing teams, "1,000 total clicks and at least 7 days" is the operational threshold that catches the obvious wins without obsessing over statistical significance calculators. The 7-day minimum protects against weekday/weekend traffic mix effects.

Reading the conversion comparison

Filter your downstream analytics tool (GA4, Mixpanel, PostHog) by the variant's UTM content value. Compare conversion rates: total conversions ÷ total sessions for each variant. The variant with the higher conversion rate is the directional winner. Use a statistical significance calculator (many free ones online, or your tool may have one built in) to confirm the p-value is below 0.05 before calling the win.

Handling close-but-not-significant results

A 4% lift with p=0.18 is inconclusive, the data doesn't support a confident decision. Three options: extend the test until you have enough data, accept the result as inconclusive (and update the page based on other signal, not this test), or run a follow-up test with a sharper hypothesis.

The teams that improve fastest from A/B testing don't run the most tests, they run tests with the strongest hypotheses, accept inconclusive results gracefully, and never argue with the data when their preferred variant loses.

The rules below come from running thousands of redirect-layer tests across customer accounts. They're unglamorous but they're what separates teams that learn from their tests from teams that just run them.

1. Test one variable at a time

If you change the headline and the CTA copy and the hero image simultaneously, you won't know which change drove the result. For multi-variable tests, use a structured approach (factorial design), but for the first 80% of marketing teams, single-variable tests are enough and far easier to interpret.

2. Start at 50/50

The split that produces the most data fastest. Skewed splits (90/10 to protect a working page from an untested variant) take 5x longer to reach significance. Once you have a clear winner from the 50/50 test, you can ramp gradually (70/30, then 90/10) before fully committing.

3. Document hypothesis before the test starts

"I expect Variant B to lift CVR by 15% because shorter copy reduces cognitive load on mobile users." Stored in the link description before traffic starts. Without this, you'll inevitably retrofit a narrative to whatever the result is, and you won't learn anything generalisable.

4. Run tests for at least 7 days

Traffic composition differs by weekday vs weekend. A test that runs Monday-Wednesday systematically over-samples weekday visitor patterns. 7 days minimum, ideally 14 if traffic is sufficient.

5. Don't test on tiny traffic volumes

Below 500 clicks per variant, your confidence interval is too wide to draw conclusions. If your link gets 30 clicks/week, it's not a candidate for A/B testing, pick a higher-traffic link or wait for more traffic.

6. Don't run multiple tests on the same audience

If you're testing hero copy on the same audience that's seeing a pricing-page test, the two tests confound each other. Either segregate the audiences or run tests sequentially.

7. Record losses, not just wins

A failed test ("the snappy headline didn't lift conversion") is as valuable as a win, it deprioritises a hypothesis that competitors might still be wasting time on. Maintain a test log: hypothesis, result, action taken. Six months in, you'll have a knowledge base of what works for your audience.

Once basic A/B testing is working, the next question is how the split interacts with the rest of Flyn's routing. The answer is a strict order of operations, and knowing it is the difference between a test you can read and one you cannot.

Routing rules run first, the split takes the remainder

Smart routing (country and device rules) is evaluated before the split. If a rule matches a click, that rule decides the destination and the split never runs for that click at all. Only the clicks no rule claimed reach the weighted draw. Read the two features as "rule first, split for the remainder".

The practical consequence is worth stating plainly, because it is the opposite of what most people assume: you cannot nest a split inside a rule. There is no "US visitors get their own two-variant test while EU visitors get a different one" from a single short link, because the moment the US rule matches, that click is already placed and the rotation is skipped. If you want an independent experiment per market or per device class, give each segment its own short link and put the rotation on that link.

Using the order deliberately

Turned around, the order is genuinely useful. Send one segment somewhere fixed with a rule (a localised page for Germany, an app store URL for iOS) and let everyone else fall through into the test. The rule carves out the traffic you did not want in the experiment, and the split gets a cleaner remainder to measure. That is a layering traditional JavaScript tools handle badly, because they see the visitor only after the page has already loaded.

Bots and crawlers always get the primary destination

Automated clients are never sampled into the split, they always receive the link's primary URL. This is a deliberate anti-cloaking guard: a security scanner, a Safe Browsing crawler, or a chat app preview fetcher must never be shown a different page than a human gets. It also means a preview fetcher never reveals which destination a human would have got, and that your variant counts are made of human clicks rather than crawler noise.

Pausing a destination without deleting it

Set a destination's weight to 0 and it stops receiving traffic while staying in the list, keeping its identifier and therefore its click history in the Destination Performance panel. That is the clean way to end a test: drop the loser to 0, leave it in place for the record, and raise the winner. Destinations and weights can be edited at any time, and the short URL never has to be reissued, so nothing you already printed or scheduled breaks.

Tagging variants for your own analytics

Flyn does not inject per-variant UTMs for you. To tell variants apart in GA4 or Mixpanel, put a distinct utm_content in each destination URL yourself when you add it to the rotation, exactly as in the worked example above. If parameter forwarding is left on, parameters the visitor arrived with on the short link are carried through to the destination as well, so click-side campaign tagging survives the redirect.

Combining with QR code experiments

The QR codes printed in your campaign assets can themselves drive A/B test traffic, split scans 50/50 across two destinations to test the post-scan lander. The QR codes marketing playbook covers offline A/B testing in depth. For dynamic codes that route into different test variants over time, see the dynamic QR codes guide.

Flyn tracks which variant was clicked. Your downstream analytics tracks what happens after the click. Stitching these together is what turns A/B testing from a click-rate exercise into a revenue-optimisation discipline.

The two-system pattern

Flyn captures the click (which destination served it, geographic context, device, timestamp). The visitor lands on the variant page. Your analytics tool (GA4, Mixpanel, PostHog, Amplitude) tracks sessions, events, and conversions. Use UTM utm_content values to identify variants in the analytics tool.

GA4 setup for variant tracking

In GA4, build an Exploration: rows by session source filtered to your test's campaign value; columns showing sessions and key events. The conversion rate per variant is the comparison number. For sharper analysis, build a custom dimension for the variant identifier and use it across multiple GA4 reports.

Mixpanel and PostHog setup

Capture the variant assignment as a user property on the first event in the session (use utm_content in the page-load handler to set it). Funnel analyses can then segment by variant property to show conversion rate at each step. PostHog's built-in experimentation features integrate cleanly with Flyn's A/B testing, see the integration docs.

Server-side conversion attribution

For purchases or signup events that happen on a server (not in the browser), pass the variant identifier from the click through your application as a cookie or hidden form field, then include it in the server-side conversion event. This catches conversions that bypass front-end analytics (e.g., PII-stripped server-only events for privacy compliance).

After watching teams run A/B tests for years, the same mistakes recur. Avoiding these alone gets you most of the way to a working testing programme.

1. Calling winners too early

The single most common mistake. The lead at day 2 of a 14-day test is statistical noise 60% of the time. Set a stopping rule before the test starts (e.g., 1,000 clicks total, 7 days minimum) and don't check daily.

2. Running tests on too-small traffic

Below 500 clicks per variant, the confidence interval is wider than any plausible lift. Pick higher-traffic links for testing, or batch multiple low-traffic surfaces behind a single short link to aggregate the traffic.

3. Changing multiple variables at once

If variant B has a new headline, new CTA, new hero image, and new colour scheme, you won't know which change drove the result. Single-variable tests are the only kind worth interpreting.

4. Forgetting to add UTMs for downstream tracking

Flyn knows which variant was served. Your conversion analytics doesn't, unless you pass that information through UTM utm_content. Without UTMs, you'll see total conversions but can't segment them by variant.

5. Letting the test run forever

"We've had this test live for 3 months, and the result is still inconclusive." The right interpretation isn't "more data needed", it's "the effect size is too small to matter." Move on to a sharper hypothesis.

6. Using a test to validate a decision already made

If the team has already committed to launching variant B regardless of what the test shows, the test is theatre. Save the budget and the analytics noise for tests where you genuinely don't know which variant will win.

7. Skipping the post-test analysis

The test ended, B won, you launched B. Now write up what happened, what surprised you, what to test next. Without this step, the team learns the same lesson three times in a row across three tests. Maintain a test log, even a simple Notion or Google Sheet, and review quarterly. If you want the test to settle on sales rather than clicks, this guide covers the sample sizes and lag windows that make the verdict real.