
Method · Questions
Questions of Method
The hard questions about governed AI/search readings — query panels, instruments, single runs, AI reading AI — answered on the record.
This page answers the questions a careful reader should ask of any observation system. It is descriptive, not promotional.
Quick answer
Any serious observation system must answer for its choices: which queries it asks, which instrument it reads with, what a single observation proves, and what role AI plays in its own reading. The Observatory publishes those answers rather than waiting for the questions. Each one is scoped, versioned, and verifiable in the published readings themselves.
“Your query panel defines your results.” It does — and the claim is scoped accordingly. Panel-based search readings use market query panels that are governed, researched and reviewed. Each reading family follows its governed calculation rules, with Reading Basis stating the applicable scope and coverage. Such a reading describes what was observed for those queries, on that date, on that surface; it does not establish the absolute truth of a market. A different panel would read differently — which is why search panels are versioned and their actual query and observation coverage belongs in Reading Basis. Narrative families use their own source, mode or surface and capture-window basis.
“You depend on a single capture instrument.” The observed source is the search or answer surface; the instrument records what that surface serves. Matching the engine, product, mode, geography, language and device improves comparability. It does not force an external system to return the same output: answers can vary between requests or as the service changes. Capture and parsing can also introduce artefacts. Instrument corrections and relevant version context belong in the audit trail so readers can distinguish a change in observation conditions from a change in the observed surface.
“A single-run card is anecdotal.” A single-run card is a dated photograph and says so on its face. A reading states whether it concerns one observation or a period window, its date and its applicable evidence basis. Query-panel coverage applies to measured search readings; narrative readings identify their source, mode or surface and capture window. A single is one observation, never a trend. Comparable period readings can reveal recurring appearances and changes; confidence depends on compatible conditions and actual coverage.
Can these two numbers actually be compared? Two figures called “visibility” or “share of voice” may count different things. Start with the measured object and denominator: appearances, cited URLs, opening roles, queries, answers or runs are not interchangeable. Then check entity-resolution rules, the query panel and its version, engine, product and mode, geography, language and device. Compare the observation dates and repeated observations behind each result, including missing captures or unavailable outputs and how they enter the denominator. Finally, identify changes in the capture instrument or the transformations used to produce the figure. A matching label is not a matching measure. Where these conditions differ, state what remains comparable and qualify the conclusion; where they cannot be reconciled, do not present the figures as a like-for-like change. Panel & Market Methodology explains the panel basis; Benchmark Governance governs stability and disclosed changes.
“AI judging AI.” In the narrative readings, the AI's own view is precisely the material being observed: capturing verbatim what an answer engine says about a market is the product, not a bias. Where a language model assists production — composing card prose, structuring captured readings — it operates under canonical, versioned prompts and family contracts, and deterministic controls check numerical consistency, actor identity, dates and permitted claims. These controls provide evidence about what they test. Human oversight governs methodology and correction.
Public reading text and illustrations are produced using automated workflows, including AI models. The published checks are described here. These checks do not mean that a person has reviewed every item.
Intelligence by AAI Kernel. Published by The Observatory.
These positions are part of the governed method: scoped claims, versioned panels, documented instruments, and deterministic validation. A reading holds only inside its stated scope and window.
Related references
- Panel & Market Methodology — because the full panel-building methodology behind the first answer is published there.
- Method & Standards — because the standards these answers rest on are stated there.
- Attestation Standard — because every figure travels with its scope and window.
- Benchmark Governance — because panels are governed at the market level.
- Public Reading Boundaries — because what a reading never claims is stated there.