FAQ

Why is my experiment not receiving impressions?

Work down this list:

• The experiment isn't live. The SDK only assigns variants and sends impressions for a live experiment. Launch it from the detail page.
• Wrong apiKey or experimentId. A bad key or id makes the config request fail, and the SDK falls back to control silently — by design, so your page never breaks. Double-check both against Settings and the experiment page.
• init() isn't running on that page, or runs after the code that reads getVariant(). It must be called on the client, early.
• You're refreshing the same browser. Impressions are deduplicated per session, so reloads don't add to the count. Use a fresh session (new private window, cleared cookies) to test.
• An ad blocker or privacy extension is blocking the request to your API origin.
• Your account is past its trial or unpaid. Events from inactive accounts are accepted by the API and then discarded — see below.

How much traffic do I need?

There is no minimum. Absolutely Butter has no detectable-effect floor and won't stop you launching a low-traffic test. Low traffic just means wider credible intervals and a longer — or unavailable — forward projection. The result stays honest the whole way; it just takes longer to sharpen. Bigger true differences need far fewer visitors to show up than small ones.

Why does the confidence seem low after two weeks?

Usually one of two things:

• Not enough data yet. With small samples the two distributions still overlap, so P(variant > control) sits near 50%. Check the forward projection for the estimated time to 95%.
• The real difference is small. If relative lift is only a percent or two, the probability will climb slowly no matter how long you wait. Decide whether a change that size is worth shipping at all.

A low number after two weeks is information, not a malfunction.

What happens when my trial expires?

The 30-day trial ends and, until you add a card, the account is inactive. While inactive:

• You can't create, launch, or reactivate experiments.
• Impression and conversion events are never rejected with an error — the API accepts them and silently discards them, so a forgotten SDK integration on a live site never starts throwing.
• Your experiments and all their historical counts stay exactly as they were.

Start the $19/month subscription from Settings and data collection resumes immediately — no reconfiguration.

Can I run multiple experiments at the same time?

Not in v1. The SDK holds one experiment's assignment in memory and init() is meant to be called once per page. You can conclude one experiment and launch another, but concurrent experiments on the same page aren't supported yet. It's on the roadmap.

What does "may not reach 95% confidence within 90 days" mean?

That's the forward projection telling you it couldn't produce a useful estimate. Extrapolating from your traffic rate since launch and the conversion rates so far, the experiment isn't on track to reach 95% confidence inside a 90-day window — so rather than show a number you can't plan around, it says so plainly. The usual causes are low traffic or a very small difference between the arms. Options: send more traffic to the tested surface, accept a directional read, or conclude on the relative lift if it's already big enough to act on.

Is any of this personally identifiable?

No. The SDK sets two cookies — the assigned variant and a random session id — and neither is tied to a person. Events carry only the experiment id, the arm, the event type, a timestamp, and that random id. There is no user identifier, and the per-session rows are deleted when an experiment is archived.

Can I change the control or variant after launching?

The descriptions and the conversion goal lock when you launch, so the data can't drift against a moving definition. Name and hypothesis stay editable. If you need to change what you're testing, conclude the experiment and start a new one. See Experiment lifecycle.