zintara.io

Reply measurement · Practical guide

Cold email sample size planning: start with the decision and detectable change

There is no useful universal number of emails for every test. Plan around the outcome rate, the smallest change worth acting on and the uncertainty you are willing to accept.

Reviewed · Examples are illustrative

Who this helps: Campaign owners and analysts evaluating attribution, comparisons and experiment evidence.

Define the decision

Rare outcomes require different planning from common ones. Sending several follow-ups to the same person does not create several independent prospects. Before asking for a sample size, define the unit of assignment and the commercial decision the experiment should inform.

Work through the procedure

  1. Choose one primary outcome, such as unique positive respondents per contacted person.
  2. Estimate a relevant baseline from a comparable cohort and state uncertainty in that estimate.
  3. Set the smallest practically meaningful change, desired power and error criterion before collecting outcomes.
  4. Have a suitable calculation account for the design, including clustering, then check whether the eligible audience and observation time can support it.

Worked example

The following is a synthetic example for this procedure, not a customer result or performance benchmark.

Planning brief
Unit: account, with contacts kept in the same arm
Outcome: at least one qualified response per account
Baseline assumption: 3%
Meaningful alternative: 5%
Observation window: chosen before launch
Required sample: calculate for this design; do not substitute a generic 100-email rule.

Read the result

The brief exposes decisions that a sample-size calculator cannot make for you. An uncertain baseline or account-level clustering can change the planning result. If the available audience is too small, narrow the decision, collect qualitative evidence or acknowledge that the experiment may remain inconclusive.

Check before moving on

  1. Distinguish relative lift from percentage-point change.
  2. Budget for records that will be excluded under rules set in advance.
  3. Do not replace unavailable sample size with repeated peeking for a favorable result.

Limits and next action

This is a planning checklist, not a sample-size calculation or a statistical guarantee. Use an appropriate validated method and qualified review for consequential decisions. Zintara does not provide a native power-analysis feature through this guide.

Source: Zintara product context; analytical methods and examples are defined in the guide

Source references

Worked examples are illustrative. Editorial procedures are suggested methods, not measured performance claims or promises of additional product features.

Related guides

Explore the Zintara workflow