How the CTR Optimizer Works

1. Discovery — Finding Underperforming Pages

Every Monday at 3 AM, the auto-pilot fetches your page data from Google Search Console (last 28 days) and applies 5 safety gates to find the best candidates for optimization:

Pages are ranked by wasted potential = impressions × CTR gap %. The top 3 per week are selected for optimization.

2. Tiered Readiness Gate — When Is a Page Ready for a New Test?

Instead of a fixed 30-day cooldown, the system uses a data-driven approach based on traffic volume:

All tiers have a hard minimum of 14 days — GSC data has a ~3-day delay and uses a rolling 28-day window, so measurements before 14 days would be unreliable.

3. AI Generation — How Suggestions Are Created

For each candidate page, the system:

1. Scrapes the live page — extracts current title, description, and page body content (first ~1,500 chars of meaningful text, stripped of nav/footer)
2. Fetches top 5 GSC queries — provides the AI with the full search landscape (keywords, impressions, CTR per query)
3. Detects search intent — classifies the page as informational, commercial, comparison, or consideration based on URL patterns and query analysis
4. Injects learnings — feeds the AI real win/loss data from all past experiments on this site, plus page-specific history of what was already tried
5. Generates 4 structurally different variations using distinct frameworks:

Additional quality rules enforced:

• Title: 50–60 characters, keyword front-loaded within first 30 chars
• Description: 140–150 characters, must end with a micro-CTA, must include a specific detail from the page
• Fake statistics are banned ("Boost ARR by 30%", "10x growth", etc.) — confirmed losers from past experiments
• Generic openers banned ("Many founders...", "In today's...", "Welcome to...")
• SERP differentiation: each title must contain an element no competitor would have (specific number, named framework, provocative qualifier)

4. CTR-Adjusted Scoring — How Suggestions Are Evaluated

The system uses two different scoring approaches:

This prevents the system from protecting bad titles that "look good on paper" but don't earn clicks. Example: a title scoring 98 mechanically but getting 0 clicks from 2,000 impressions gets an autopilot score of ~24, making it easy for any reasonable alternative to beat it.

5. WordPress Push Pipeline

When you approve a suggestion (Semi-Auto mode), the system:

1. Validates staleness — re-scrapes the page to ensure the title hasn't changed since the suggestion was created
2. Finds the post — searches WordPress REST API (posts + pages) to match the URL
3. Writes via Rank Math API — POST /wp-json/rankmath/v1/updateMeta with the new title and description
4. Busts the cache — triggers a save_post hook by POSTing {"status":"publish"}, which clears WP Rocket/W3TC/Cloudflare caches
5. Verifies the write — re-fetches the post with ?context=edit and checks rankMath.assessor.serpData
6. Checks sitemap — HEAD request to /sitemap_index.xml to confirm it's accessible
7. Inspects in GSC — calls the URL Inspection API to check index status and last crawl date

Creates an experiment + iteration record for tracking, and schedules a verification scrape after 10 minutes.

6. Measurement & Auto-Rollback

Two measurement systems run in parallel:

Before measuring, the system checks:

• Google crawl date — has Google actually re-crawled since the push? If not, the measurement would be stale.
• Position stability — if position shifted more than 3 spots, the CTR change is likely due to ranking movement, not the copy change. Result is marked inconclusive.
• Manual override detection — if you manually created a newer iteration, auto-rollback is skipped.

7. Learning Loop

Every title change becomes training data for future suggestions:

• Winners (CTR increased) — their patterns are highlighted in the AI prompt as "confirmed winners"
• Losers (CTR decreased) — their patterns are flagged as "confirmed losers" and specific patterns like fake statistics are hard-coded into a rejection filter
• Page history — for each page, the AI sees all previous attempts and their results, ensuring it never repeats a failed approach

This creates a compounding advantage: the more experiments you run, the smarter the system gets at predicting what works for your specific audience.

8. Notifications

After each weekly run, the system sends a Telegram notification with:

• Number of new suggestions ready for review
• Page slugs and suggested titles
• CTR gap and impression count for each
• Run summary (candidates scanned, suggestions created, rollbacks performed)

Configure the Telegram bot token and chat ID in the Auto-Pilot Settings tab.