A/B Testing Your Landing Pages with Short Links

Traditional landing-page A/B testing means installing a JavaScript snippet on your site, configuring a tool that splits traffic client-side, and dealing with the inevitable flash-of-original-content problem when the test variant loads slightly later than the control. Google Optimize handled this for years and was free, until Google sunset it in September 2023. The replacements (VWO, Convert, Optimizely, AB Tasty) start at $200/month and climb fast.

Meanwhile, the simpler approach has been hiding in your link shortener the whole time. Split traffic at the redirect layer, before the visitor reaches either variant, and you sidestep nearly every traditional A/B-testing pain point.

What redirect-layer A/B testing actually does

You create a short link pointing to two destination URLs and set a traffic split. When someone clicks the short link, Flyn's redirect server flips a weighted coin and sends them to one URL or the other. The visitor never sees the alternative variant; the variants don't need to share a server or framework; no JavaScript runs on either page to detect or coordinate the test.

When traditional A/B testing is still the right choice

Redirect-layer testing has trade-offs. If you want to test variants that share a single URL (e.g., two button colours on the same page), JavaScript-based tools are still required because the visitor needs to land on the same URL and see different content. If you want to test interaction-level changes (form steps, modal timing), you need DOM-level control. For pure landing-page tests where the variants are distinct pages, redirect-layer testing is faster, cheaper, and statistically cleaner.

The implementation is intentionally simple, one screen in the dashboard, one optional API field, no separate "experiments" surface to manage.

Setting up a test in the dashboard

1. Open the link drawer for an existing short link, or create a new one.
2. Open the A/B Testing tab.
3. Enter your Variant A URL (the original or control page).
4. Enter your Variant B URL (the test variant).
5. Use the split slider to set the percentage (default 50/50).
6. Save.

The short URL stays the same. Share it as you would any link, in email, social, paid ads, QR codes. Flyn handles the split at the redirect server.

Setting up a test via the API

For programmatic test creation, pass variantBUrl and variantBPercentage as fields on the link object:

POST /api/links { destination: 'https://yoursite.com/hero-v1', variantBUrl: 'https://yoursite.com/hero-v2', variantBPercentage: 50 }

The API documentation covers all the testing parameters, including support for tests with 3+ variants (where supported) and per-variant UTM injection.

How the split actually decides

On every click, Flyn generates a random number between 0 and 100. If the split is set to 60/40 favouring A, visitors with a generated value below 60 see A; the rest see B. The randomisation is independent per click, there's no sticky cookie that remembers the visitor's assignment by default, so the same visitor clicking twice may see different variants. This is the right default for top-of-funnel tests where each click is treated as an independent traffic event. For sticky assignment (where the same visitor always sees the same variant), enable cookie persistence in the link settings.

What gets recorded per click

The Flyn analytics dashboard tags every click with the variant served (A or B), the timestamp, geographic data, device, OS, and referrer. Filter analytics by variant to compare top-of-funnel metrics. For conversion data, you'll cross-reference with your downstream analytics tool (covered in detail below).

Concrete example. You suspect your hero copy could be sharper. Currently you have a long, benefits-focused headline; you want to test a punchy, problem-focused alternative.

Step 1, Define the hypothesis

Write it down before the test starts. "I believe replacing the benefits headline with a problem statement will increase conversion rate by >10% because problem-first copy reduces the time visitors spend understanding what we sell." Store this in the link's description field in Flyn so you can read it back when the test concludes.

Step 2, Build the variants

Create two distinct landing page URLs. Don't deploy a JavaScript-toggled variant, for a redirect-layer test, you want genuinely separate pages so the test stays clean. URLs:

• Variant A (control): https://yoursite.com/hero-benefits
• Variant B (test): https://yoursite.com/hero-problem

Step 3, Add UTMs to each variant URL

So conversion data downstream can attribute back to the variant. Build them with the UTM Builder:

• Variant A: ?utm_source=ab-test&utm_medium=redirect&utm_campaign=hero-q2-2026&utm_content=variant-a-benefits
• Variant B: ?utm_source=ab-test&utm_medium=redirect&utm_campaign=hero-q2-2026&utm_content=variant-b-problem

Same campaign, different content. GA4 will record sessions and conversions by content value, letting you compare conversion rates between the variants.

Step 4, Create the Flyn link with A/B testing enabled

Set both URLs in the A/B Testing tab. Use a 50/50 split for first-test maximum data speed. Name the link with a clear slug (ab-hero-q2-2026) and tag it with ab-test for filtering.

Step 5, Drive matched traffic to the link

Use the same short link everywhere you'd normally use the landing page URL. Email campaign, paid ad, social post, QR code. Don't use different traffic sources for different variants, that confounds the test.

Step 6, Wait for statistical significance

At least 7 days and at least 1,000 total clicks before reading the result. Don't peek and call winners mid-test. The temptation to call results early is the most common A/B-testing mistake, and it produces statistically meaningless conclusions.

The hardest part of A/B testing isn't setting up the test, it's reading the results honestly. Most teams call winners too early and act on noise. The discipline below keeps you out of that trap.

The minimum sample size by effect size

Smaller expected lifts require larger samples to detect reliably. As a rough table:

If your link gets 500 clicks/week and you're looking for a 10% lift, plan a 10-week test. If you can't commit that long, accept that you're only powered to detect 20%+ lifts and frame the test accordingly.

The 1,000-click rule of thumb

For most marketing teams, "1,000 total clicks and at least 7 days" is the operational threshold that catches the obvious wins without obsessing over statistical significance calculators. The 7-day minimum protects against weekday/weekend traffic mix effects.

Reading the conversion comparison

Filter your downstream analytics tool (GA4, Mixpanel, PostHog) by the variant's UTM content value. Compare conversion rates: total conversions ÷ total sessions for each variant. The variant with the higher conversion rate is the directional winner. Use a statistical significance calculator (many free ones online, or your tool may have one built in) to confirm the p-value is below 0.05 before calling the win.

Handling close-but-not-significant results

A 4% lift with p=0.18 is inconclusive, the data doesn't support a confident decision. Three options: extend the test until you have enough data, accept the result as inconclusive (and update the page based on other signal, not this test), or run a follow-up test with a sharper hypothesis.

The teams that improve fastest from A/B testing don't run the most tests, they run tests with the strongest hypotheses, accept inconclusive results gracefully, and never argue with the data when their preferred variant loses.

The rules below come from running thousands of redirect-layer tests across customer accounts. They're unglamorous but they're what separates teams that learn from their tests from teams that just run them.

1. Test one variable at a time

If you change the headline and the CTA copy and the hero image simultaneously, you won't know which change drove the result. For multi-variable tests, use a structured approach (factorial design), but for the first 80% of marketing teams, single-variable tests are enough and far easier to interpret.

2. Start at 50/50

The split that produces the most data fastest. Skewed splits (90/10 to protect a working page from an untested variant) take 5x longer to reach significance. Once you have a clear winner from the 50/50 test, you can ramp gradually (70/30, then 90/10) before fully committing.

3. Document hypothesis before the test starts

"I expect Variant B to lift CVR by 15% because shorter copy reduces cognitive load on mobile users." Stored in the link description before traffic starts. Without this, you'll inevitably retrofit a narrative to whatever the result is, and you won't learn anything generalisable.

4. Run tests for at least 7 days

Traffic composition differs by weekday vs weekend. A test that runs Monday-Wednesday systematically over-samples weekday visitor patterns. 7 days minimum, ideally 14 if traffic is sufficient.

5. Don't test on tiny traffic volumes

Below 500 clicks per variant, your confidence interval is too wide to draw conclusions. If your link gets 30 clicks/week, it's not a candidate for A/B testing, pick a higher-traffic link or wait for more traffic.

6. Don't run multiple tests on the same audience

If you're testing hero copy on the same audience that's seeing a pricing-page test, the two tests confound each other. Either segregate the audiences or run tests sequentially.

7. Record losses, not just wins

A failed test ("the snappy headline didn't lift conversion") is as valuable as a win, it deprioritises a hypothesis that competitors might still be wasting time on. Maintain a test log: hypothesis, result, action taken. Six months in, you'll have a knowledge base of what works for your audience.

Once basic A/B testing is working, Flyn's routing engine supports layered experiments that traditional JavaScript-based tools can't handle.

Geo-routed A/B tests

Different tests in different markets from a single short link. US visitors see a test between hero variants A and B; EU visitors see a test between localised variants. The short link routes by IP-derived country, then applies the within-country A/B split. Useful for regional teams running independent experiments under a unified link strategy.

Device-routed A/B tests

Different variants for mobile vs desktop. Layer this on top of A/B testing to run desktop A/B and mobile A/B independently from one short link. Useful when you expect device-specific behaviours (e.g., the long-form copy that wins on desktop loses on mobile).

Time-windowed tests

Different variants for different time windows. Useful for promo expiration handling, Variant A while the promo is live, Variant B (with adjusted copy) after the promo ends. Configurable per-link without code changes.

UTM-passthrough for variant tracking

Pre-flight UTM injection, Flyn appends per-variant UTMs to the destination URL automatically based on which variant was selected. The visitor never sees raw UTMs in their address bar in some configurations; analytics still receives the full attribution chain.

Bandit-style optimisation (Pro)

For long-running tests where you want traffic to gradually shift toward the higher-converting variant, enable bandit mode. Instead of static 50/50, the split adjusts dynamically based on observed conversion rate. Trades pure statistical clarity for cumulative revenue capture during the test window. Available on Pro.

Combining with QR code experiments

The QR codes printed in your campaign assets can themselves drive A/B test traffic, split scans 50/50 across two destinations to test the post-scan lander. The QR codes marketing playbook covers offline A/B testing in depth. For dynamic codes that route into different test variants over time, see the dynamic QR codes guide.

Flyn tracks which variant was clicked. Your downstream analytics tracks what happens after the click. Stitching these together is what turns A/B testing from a click-rate exercise into a revenue-optimisation discipline.

The two-system pattern

Flyn captures the click (variant assignment, geographic context, device, timestamp). The visitor lands on the variant page. Your analytics tool (GA4, Mixpanel, PostHog, Amplitude) tracks sessions, events, and conversions. Use UTM utm_content values to identify variants in the analytics tool.

GA4 setup for variant tracking

In GA4, build an Exploration: rows by session source filtered to your test's campaign value; columns showing sessions and key events. The conversion rate per variant is the comparison number. For sharper analysis, build a custom dimension for the variant identifier and use it across multiple GA4 reports.

Mixpanel and PostHog setup

Capture the variant assignment as a user property on the first event in the session (use utm_content in the page-load handler to set it). Funnel analyses can then segment by variant property to show conversion rate at each step. PostHog's built-in experimentation features integrate cleanly with Flyn's A/B testing, see the integration docs.

Server-side conversion attribution

For purchases or signup events that happen on a server (not in the browser), pass the variant identifier from the click through your application as a cookie or hidden form field, then include it in the server-side conversion event. This catches conversions that bypass front-end analytics (e.g., PII-stripped server-only events for privacy compliance).

After watching teams run A/B tests for years, the same mistakes recur. Avoiding these alone gets you most of the way to a working testing programme.

1. Calling winners too early

The single most common mistake. The lead at day 2 of a 14-day test is statistical noise 60% of the time. Set a stopping rule before the test starts (e.g., 1,000 clicks total, 7 days minimum) and don't check daily.

2. Running tests on too-small traffic

Below 500 clicks per variant, the confidence interval is wider than any plausible lift. Pick higher-traffic links for testing, or batch multiple low-traffic surfaces behind a single short link to aggregate the traffic.

3. Changing multiple variables at once

If variant B has a new headline, new CTA, new hero image, and new colour scheme, you won't know which change drove the result. Single-variable tests are the only kind worth interpreting.

4. Forgetting to add UTMs for downstream tracking

Flyn knows which variant was served. Your conversion analytics doesn't, unless you pass that information through UTM utm_content. Without UTMs, you'll see total conversions but can't segment them by variant.

5. Letting the test run forever

"We've had this test live for 3 months, and the result is still inconclusive." The right interpretation isn't "more data needed", it's "the effect size is too small to matter." Move on to a sharper hypothesis.

6. Using a test to validate a decision already made

If the team has already committed to launching variant B regardless of what the test shows, the test is theatre. Save the budget and the analytics noise for tests where you genuinely don't know which variant will win.

7. Skipping the post-test analysis

The test ended, B won, you launched B. Now write up what happened, what surprised you, what to test next. Without this step, the team learns the same lesson three times in a row across three tests. Maintain a test log, even a simple Notion or Google Sheet, and review quarterly. If you want the test to settle on sales rather than clicks, this guide covers the sample sizes and lag windows that make the verdict real.