AirPulse AI-Search Visibility Measurement Methodology

This methodology is the calculation and reporting contract behind AirPulse research pages. It should be used with the 30,504-response stability study and the prompt-selection methodology. The AI-search measurement research hub keeps the current pages, public downloads and update record in one place.

1. Define the measurement question before collecting data

Every study must state the decision it is meant to support. “Are we visible in AI?” is too broad. A usable question is: “How often is our brand mentioned for these evaluation-stage prompts on ChatGPT and Perplexity during this four-week window?”

Lock these fields before the baseline:

• target brand and approved aliases;
• exact prompt text and locale;
• prompt intent and buyer stage;
• answer engine and surface;
• run schedule and minimum valid runs;
• mention and citation matching rules;
• start and end dates;
• parser, roster and measurement version.

2. Use a prompt-engine cell as the unit

A prompt-engine cellis one exact prompt paired with one answer engine. “Best AI visibility tools for B2B SaaS” on ChatGPT is one cell. The same prompt on Perplexity is another.

Do not pool engines before calculating the primary rates. In AirPulse's frozen cohort, consecutive mention flip rates ranged from 6.2% on Gemini to 14.7% on Google AI. The difference may reflect the studied cohort as well as engine behaviour, but it is large enough that a hidden average can mislead.

3. Freeze the prompt roster

Every prompt must have a stable identifier. Fixing spelling after the first run creates a new version. Adding a product qualifier changes the question and should create a new prompt or a declared roster revision.

Store:

The selection process belongs in Which AI search prompts should your brand track? Generated suggestions should be labelled as suggestions; observed questions from customers, sales calls or Search Console should remain distinguishable.

4. Schedule repeated runs

Start with at least seven valid runs for brand detection and eight for source coverage per prompt-engine cell, spread across two to four weeks. These are practical floors from external research. They are not a universal theorem.

Store failed calls separately. Do not count a timeout, refusal, empty answer or parser failure as “brand not mentioned.” Report valid runs and excluded runs.

5. Preserve one evidence unit per run

Every valid run should retain:

• prompt ID and roster version;
• engine, surface and available model identifier;
• request and response timestamps;
• raw answer text;
• parsed brand mention and matched span;
• citation URLs and normalised domains;
• available search or retrieval results;
• generated search queries where the engine exposes them;
• parser version and validation flags.

Generated queries, search results and citations are different fields. A search result shows what may have been retrieved. A citation shows what the answer exposed. A generated query shows how the system searched, when that trace is available.

6. Calculate the primary outcomes

Mention rate

Mention rate equals valid runs containing a valid brand mention divided by valid runs in the same prompt-engine-window scope.

mention rate = runs with brand mention / valid runs

Matching rules must cover approved brand names and aliases while rejecting unrelated words. If a reviewer changes a match, record the override and reason.

Own-domain citation rate

Own-domain citation rate equals valid runs containing at least one cited URL whose registrable domain belongs to the tracked brand, divided by valid runs.

own-domain citation rate = runs citing the brand's domain / valid runs

Normalise scheme, www, fragments and tracking parameters. Preserve the original URL as evidence.

Third-party citation rate

Third-party citation rate measures runs that cite a separate domain which mentions, validates, compares or competes with the brand. Classification should be transparent; the same source can play different roles for different questions.

Source coverage and stability

Source coverage is the distinct set of cited domains or URLs observed across runs. For comparable consecutive sets, AirPulse uses Jaccard similarity:

Jaccard similarity = intersection / union

Report the distribution or at least the median and mean. AirPulse's frozen study found a mean of 0.396 and a median of 0.333. Do not turn one minus Jaccard into a claim that each URL has that probability of disappearing.

7. Show uncertainty

For a binary rate, show the numerator, denominator and a binomial confidence interval. A Wilson interval is preferable to a simple normal approximation when samples are small or rates are near zero or one.

Do not claim that overlapping intervals prove “no difference,” or that non-overlap alone proves a causal effect. For a content trial, compare a preregistered target set with matched control prompts and preserve the time series.

The paper Quantifying Uncertainty in AI Visibility explains why single-run point estimates can give a false sense of precision.

8. Separate the four doors

An AI outcome can fail at four different stages:

1. Access: can the relevant crawler or user-requested fetcher reach the page?
2. Discovery/indexing: is the URL known to search and retrieval systems?
3. Retrieval: is the page selected as evidence for this question?
4. Answer selection: does the final answer mention the brand or cite the page?

Record these stages separately. OpenAI distinguishes OAI-SearchBot from GPTBot in its publisher guidance. Perplexity distinguishes PerplexityBot from Perplexity-User in its crawler documentation. Google also separates crawling, indexing and serving in its Search documentation. Allowing access does not guarantee a mention or citation.

9. Version every material change

Create a new measurement version when any of these changes:

• prompt roster or wording;
• engine or surface definition;
• model identifier where exposed;
• mention or citation parser;
• brand alias list;
• source URL normalisation;
• retry or failure handling;
• run cadence;
• page intervention being tested.

Do not draw a before/after conclusion across an undeclared measurement boundary. If a boundary cannot be reconstructed, label the series as non-comparable.

10. Use controls for intervention claims

To test a page change, freeze target prompts and matched control prompts before publication. Collect a baseline, record the intervention date, then continue the same schedule. Do not rewrite the prompts after launch to make the result appear stronger.

The result can be:

• positive: the target moved more than controls under the registered rule;
• null: the page became retrievable but target outcomes did not materially improve;
• inconclusive: access, sample size or contamination prevents a decision.

A null result is publishable evidence. A single citation is an observation until it repeats under the plan.

11. Minimum weekly report

The executive summary can be shorter, but the underlying table must remain available.

12. Privacy and tenant isolation

Public research exports must be aggregate-only unless a customer has explicitly approved a case study. Remove names, tenant IDs, private prompts, raw answers, API keys, account-state details and URLs that could identify a private programme.

Analytical code should enforce organisation filters at query time. Public downloads should be generated from a reviewed allow-list of fields rather than by deleting sensitive columns after export.