Updating 50 Pages for AI Search: A Controlled Experiment Protocol

What happens when 50 existing pages are updated for AI search? A trustworthy answer requires a completed experiment with controls, repeated measurements and mixed results—not a retrospective assembled from successful examples. TotalAuthority has not completed that 50-page experiment, so this article publishes the full protocol instead of manufacturing a case study.

The protocol is designed to produce a future report that can withstand scrutiny. Teams can also use it to test their own content programme now.

The Research Question

For existing pages with measurable search eligibility, does a defined package of evidence, structure, entity and technical improvements change:

• citation rate;
• citation absorption;
• brand mention rate;
• factual accuracy;
• qualified recommendation rate;
• generative-search impressions;
• AI referral traffic;
• conventional organic performance?

The study should not ask whether vague “GEO optimization” works. Every intervention must be observable and reproducible.

Pre-Register the Hypotheses

Before editing, record directional hypotheses.

Primary hypothesis

Updated pages will show a larger change in citation rate across their matched prompt set than comparable control pages.

Secondary hypotheses

• Pages receiving stronger primary evidence will show greater citation absorption.
• Entity and authorship corrections will reduce material factual errors.
• Technical fixes will improve eligibility but will not guarantee citations.
• Changes will vary by platform and prompt class.

Pre-registration prevents the team from selecting only positive outcomes after seeing the data.

Select the 50 Pages

Create an inventory of at least 100 plausible candidates, then match 50 treatment pages with 50 controls.

Eligibility criteria could include:

• stable canonical URL;
• indexable and publicly accessible;
• a defined audience task;
• enough historical data for context;
• no planned migration or deletion;
• an identifiable prompt set;
• no high-risk claims that cannot be reviewed.

Exclude pages already scheduled for major redesign, product retirement or unrelated campaigns.

Match the Control Group

Match each treatment page to a control on:

• topic type;
• existing impressions and links;
• page age;
• brand prominence;
• commercial versus informational intent;
• market and language;
• historical update frequency.

Perfect matching is unlikely. Record the differences and consider stratified analysis. Do not secretly improve the controls during the observation period.

Define the Intervention Package

Every treatment page should receive the same core audit, while edits remain appropriate to the topic.

1. Direct answer and task completion

State the main answer early, then provide the detail needed to act. Remove introductions that delay the task.

2. Verifiable evidence

Replace unsupported generalities with primary sources, calculations, real examples, subject-matter input, dates and limitations. Keep claim-to-source distance short.

3. Information architecture

Use descriptive headings, genuine comparison tables, sequential steps and concise definitions where they clarify the task. Do not mechanically force every page into identical “AI chunks.”

4. Entity clarity

Correct organization, author, product, location and credential facts. Link official profiles consistently. Use valid structured data that matches visible content.

5. Technical eligibility

Resolve crawler blocks, render failures, duplicate URLs, poor canonicals, broken internal links and stale sitemap signals.

6. Conversion path

Connect the answer to a relevant next step without turning the page into an advert. Preserve analytics events.

The citation-ready content guide and technical GEO audit define the acceptance criteria.

Build the Prompt Set

Assign five to ten natural prompts to each page. Include the primary task, two variants, a comparison or validation question and a related follow-up. Avoid using the brand name in discovery prompts.

With eight prompts per page, five repeated runs and three platforms, 50 treatment pages produce 6,000 treatment observations per collection wave. The control group doubles the total. If that volume is impractical, reduce platform or prompt scope before abandoning repetition.

Establish the Baseline

Collect at least two pre-intervention waves. Record:

• exact prompt and answer;
• platform, mode, date, market and account state;
• visible sources;
• mention and recommendation outcomes;
• answer accuracy;
• page search and analytics metrics;
• external events.

A single pre-period is vulnerable to normal variance.

Implement and QA

Use a documented brief for each page. Record every material change in a structured change log:

Quality assurance should cover rendered HTML, mobile layout, schema, citations, internal links, accessibility, factual accuracy and analytics.

Allow Discovery Time

Platforms need time to recrawl, index and refresh sources. Use a fixed waiting rule based on observed recrawl, not a convenient date chosen after results appear. Record when updated pages are first fetched where server logs or search tools allow.

Measure the Outcome

Collect repeated post-intervention waves on the unchanged benchmark prompts. Compare the change in treatment pages with the change in controls.

A simple difference-in-differences structure is:

(treatment after - treatment before) - (control after - control before)

Apply it to predefined outcomes and report confidence intervals. The design improves causal interpretation, but concurrent platform and market changes can still confound results.

Report Failures

The finished study must publish:

• pages that lost visibility;
• outcomes that did not change;
• technical fixes that improved access but not citations;
• pages with contradictory platform results;
• prompts with unstable answers;
• implementation mistakes;
• missing and invalid observations.

A “50 pages updated, 50 wins” story is less credible than a mixed result.

Suggested Result Table

These fields are intentionally unpopulated until the experiment is completed.

How to Use the Protocol on Ten Pages

Smaller teams can run a pilot:

1. Choose ten valuable pages and ten controls.
2. assign five prompts per page.
3. collect repeated baseline observations.
4. apply a fixed intervention checklist.
5. wait for verified discovery.
6. collect two or more post waves.
7. compare treatment and control changes.
8. publish negative results internally.

Frequently Asked Questions

Why update existing pages instead of publishing 50 new ones?

Existing pages provide historical context and reduce topic overlap. They also test whether evidence and technical improvements can strengthen assets already serving users.

Can organic clicks be the primary outcome?

They can be a supporting outcome, but AI citations and mentions may not produce a click. Keep visibility, referral and commercial metrics separate.

Should publication dates be changed on every page?

Only when the page has been substantially updated. Google advises using dates that accurately describe publication or significant modification.

When will TotalAuthority publish results?

After the 50 treatment pages, controls, repeated observations and reproducible analysis are complete. Until then, this remains the public protocol.

Sources

• Don't Measure Once: Measuring Visibility in AI Search
• Quantifying Uncertainty in AI Visibility
• From Citation Selection to Citation Absorption
• Google: Creating helpful, reliable, people-first content
• Google: Influence byline dates