A/B testing a landing page hero without a heavy analytics stack
TL;DR — You don't need Amplitude or Mixpanel for a credible A/B test on your hero CTA. Pendro's built-in primitive — deterministic visitor-bucket cookie, render-time variant selection, per-variant pageview beacon — gives the same signal in 8 hours that a heavyweight stack gives in 8 weeks. A real test, end-to-end.
Most A/B testing advice on the internet assumes you already have Amplitude, Mixpanel, Heap, or Optimizely set up. That assumption costs early-stage teams months of setup before they've run a single experiment.
The reality: for a landing-page A/B test, you need exactly three things — a way to bucket visitors deterministically, a way to render different variants based on the bucket, and a way to count which variant got more conversion events. Everything else is shopping for trouble.
This piece walks through running a real A/B test on the Pendro marketing site's hero CTA, using only the primitives Pendro ships out of the box.
The experiment
We want to know whether changing the hero CTA from "Start free" (current) to "Try the AI" moves the click-through rate to /pricing.
The setup: variant A is the existing CTA. Variant B is the new one. Visitors get assigned to one or the other deterministically (same visitor always sees the same variant on repeat visits) and we measure CTA click-through rate per variant after enough traffic to declare a winner.
Step 1: Deterministic visitor bucketing
When a visitor first lands on the site, the renderer sets a cookie called `pendro_ab` with a UUID. On every subsequent page-load, the renderer reads that UUID, runs it through a stable hash function, and uses the result modulo 100 to assign the visitor a bucket from 0-99.
Each running experiment has a percentage split (50/50, 20/80, etc.) and a bucket range it claims. Visitors with bucket < 50 see variant A; bucket >= 50 see variant B. The hash is keyed on the experiment id + the visitor UUID, so the same visitor sees the same variant across all visits, while a different visitor with the same UUID would see a different variant if they happened to be in a different experiment.
Step 2: Defining the experiment
In Pendro, an A/B experiment is a row in the `ab_experiments` table with:
A unique slug (used as the cache-tag for invalidation).
A target section ID on the project (e.g. the hero section).
A 50/50 split (or custom percentages).
A "variant B" patch — a partial copy of the section's content that overrides variant A's fields.
Start and end dates.
The variant B patch for our experiment is one line: `{ primaryCta: { label: 'Try the AI' } }`. The renderer takes the hero section's content from variant A, applies the patch, and serves the modified hero to bucket-B visitors.
Defining the experiment via the dashboard (Project → Experiments → New) writes the row + invalidates the `ab-experiments` cache tag, so the next page-load picks it up. No deploy required.
Step 3: Render-time variant selection
On every tenant page request, the renderer:
Reads the visitor's `pendro_ab` cookie. If absent (first visit), generates a new UUID and sets the cookie.
Loads the project's active experiments via a tag-cached query.
For each running experiment, computes the visitor's bucket via the hash function.
Applies variant-B patches to any sections in experiments where the visitor lands in bucket B.
Renders the page.
The whole process adds about 2-4ms to the render time — most of it the bucket computation. The experiment configuration is cache-tagged on the project, so the per-request cost is one cookie read + one O(experiments × sections) merge in memory.
Step 4: The pageview beacon
Every rendered tenant page sends a single beacon to `/api/__pv` (Pendro's pageview endpoint) with:
project_id
pathname (e.g. /)
variant assignment for the page's primary experiment ('a' or 'b')
session UUID (the same `pendro_ab` value)
The beacon is fire-and-forget (no awaited response) so it never blocks the render. The server writes a row to `pageviews` with all of the above. That table is the entire dataset the experiment readout reads from.
Step 5: The conversion event
We care about click-through rate on the primary CTA. The CTA itself is a regular `<a>` tag pointing at `/pricing`. To track clicks, we attach a tiny inline script that fires a beacon to `/api/__cv` (conversion endpoint) on click, with the experiment id, variant assignment, and conversion type (`cta_click`).
Step 6: Reading the result
After running the experiment for two weeks (or until we hit enough traffic for statistical significance), open the dashboard's Experiments page. For each experiment, the readout shows:
Total visitors per variant.
CTA click-throughs per variant.
Conversion rate per variant.
Confidence interval (95% by default).
A simple recommendation: "ship variant B" / "ship variant A" / "keep running — not significant yet".
The math is a basic two-proportion z-test. We deliberately don't support Bayesian methods, sequential testing, or any of the more sophisticated approaches — for an early-stage A/B test, frequentist + a clean readout is enough. When you graduate to needing those methods, you'll know, and you'll graduate to a heavier tool.
What we found
In our real experiment running the above setup for 14 days: variant B ("Try the AI") outperformed variant A ("Start free") by 27% on CTA click-through, with 95% confidence. We shipped variant B as the new permanent hero CTA and the bump held over the following two months.
Total cost of the experiment: 30 minutes to define it in the dashboard, two weeks of waiting. Total infrastructure: the same infra that was already running the site. No new vendor, no new bill.
When you do need a heavier tool
The lightweight stack gets you through the first 6-12 months of A/B testing. You'll outgrow it when:
You start wanting to slice conversion by visitor properties (country, device, referrer) — Pendro's beacon table captures the basics but doesn't auto-segment.
You start running 5+ experiments at once and need a way to detect interaction effects between them.
You need server-side experiment exposure (mobile-app integrations, email links, etc.).
You start needing per-feature feature flags that aren't tied to a single page section.
At that point, look at Statsig, GrowthBook, or LaunchDarkly. Until then, the simple primitives are enough.
Build your site in Pendro.
Pick a template, edit live, hit publish. No code, no hosting setup, no surprises.
Stay close
More posts like this in your inbox.
Occasional updates — new posts, product changes, and notes on what we're learning. One email every couple of weeks, never more.
No spam, ever. Unsubscribe in one click.