Methodology

1. Observation boundary

An observed query enters the local trace only when a provider adapter finds it inside an explicitly search-scoped interface element. Generic message containers are ineligible. Chat messages, titles, account identity, conversation URLs and conversation IDs do not exist in the event schema.

observed_model_searchA search string surfaced by ChatGPT or Claude tool UI.

observed_expanded_queryA query expansion explicitly exposed by the provider.

google_user_searchThe disclosed Google Search seed-query exception.

If an adapter can no longer recognize that boundary, it fails closed. Estimation is a separate user-triggered action and never mutates an observed event into an estimate.

2. Controlled generation experiment

For a seed query $x$ , provider model $m$ and minimal versioned prompt $p_v$ , the model produces one structured vector of exactly 12 candidate queries:

Y=(q_1,\ldots,q_{12})\sim P_m(\,\cdot\mid x,p_v\,)

The prompt asks only for distinct web-search queries in the seed language. It contains no categories, rationales, scoring rubric or rank instructions.

Candidates are normalized with NFKC, whitespace normalization and exact case-folded deduplication. Unsafe strings are removed. No model is allowed to score another provider’s candidate set.

3. Native token log probabilities where exposed

GPT-5.6 Luna returns token log probabilities for the same structured output that contains the candidates. For a query $q$ , let $T(q)$ be the set of output tokens whose UTF-8 byte interval overlaps that query’s JSON string content. Every token is included once; token length does not create extra weight.

\bar{\ell}(q)=\frac{1}{|T(q)|}\sum_{t_k\in T(q)}\log P_m(t_k\mid t_{<k},x,p_v)

\operatorname{PP}(q)=\exp\!\left(-\bar{\ell}(q)\right)

s(q)=\operatorname{PP}(q)^{-1}=\exp\!\left(\bar{\ell}(q)\right)

Lower perplexity implies greater compatibility with that provider’s own decoding distribution in this run. The API exposes the mean log probability, perplexity, inverse perplexity and token count.

The system fails closed if fewer than six candidates can be mapped to finite native token logprobs. It does not substitute another provider or an ordinal fallback. The inverse-perplexity score is a ranking statistic for one conditional generation—not an independent probability that the original assistant searched that query.

4. Google and Anthropic: empirical native sampling

The current Anthropic endpoint does not return output token logprobs for Claude. The configured Gemini 3.1 Flash-Lite Developer API endpoint rejects responseLogprobs. These are measured endpoint capabilities, not assumptions about every historic model or SDK. Open Queries therefore reports a different provider-native estimator instead of silently using GPT. Each provider model receives the same versioned prompt in 16 independent structured-output calls. At least 12 must succeed.

K(q)=\sum_{i=1}^{n}\mathbf{1}\!\left[q\in Y_i\right]

\hat p(q)=\frac{K(q)}{n}

\operatorname{CI}_{95}(q)=\frac{\hat p+\frac{z^2}{2n}\;\pm\;z\sqrt{\frac{\hat p(1-\hat p)}{n}+\frac{z^2}{4n^2}}}{1+\frac{z^2}{n}},\qquad z=1.95996

Here z = 1.95996 and n is the number of valid samples from the named provider. Queries rank by inclusion frequency, then mean first position, then lexical order.

Exact normalized matches are counted once per sample. The score object exposes K, n and its Wilson 95% confidence interval, making the wider uncertainty at n ≤ 16 explicit.

5. Provenance and aggregation

Every estimate carries its provider model, method and prompt version. The side panel leads with ordinal rank and keeps the mathematical evidence behind an expandable detail control. It never formats these values as demand percentages.

Query contribution starts off and is controllable during onboarding and in Settings. Raw events expire after 13 months. Durable daily aggregates require at least five distinct anonymous donor tags. Estimated fan-outs never enter observed-query aggregates.

6. Limits and interpretation

Visible tool activity can be incomplete.
Provider UI changes can temporarily disable an adapter.
Generation and sampling are stochastic and model-version specific.
Tokenization makes scores comparable mainly within a provider run.
Empirical Gemini and Claude intervals are deliberately wide at 12–16 samples.
Observed query frequency is not population search volume.

For assumptions, derivations and alternatives, read the technical note: Estimating fan-out queries with log probabilities.