thetimes.co.uk

Détail des vérifications

• Accès des bots IA (robots.txt)Problème
  • 4/13 known AI bots are allowed by robots.txt.
  • Bloccato: ClaudeBot (Anthropic (Claude) — training/crawling).
  • Bloccato: anthropic-ai (Anthropic — crawling generico).
  • Bloccato: Claude-User (Anthropic — browsing live in Claude).
  • Bloccato: PerplexityBot (Perplexity — crawling/citazioni).
  • Bloccato: Perplexity-User (Perplexity — browsing live).
  • Bloccato: Google-Extended (Google — training Gemini/AI Overviews).
  • Bloccato: CCBot (Common Crawl (used to train many LLMs)).
  • Bloccato: Bytespider (ByteDance (also used for AI training)).
  • Bloccato: Meta-ExternalAgent (Meta AI).

  Comment le corriger : Remove the Disallow rules for: ClaudeBot, anthropic-ai, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, CCBot, Bytespider, Meta-ExternalAgent.

• Présence et qualité du llms.txtProblème
  • /llms.txt found (1334 characters).
  • Missing an H1 title at the top of the file.
  • It contains no markdown links to specific pages/resources.

  Comment le corriger : Improve llms.txt: an H1 with the site name, sections per topic area, markdown links to key pages.

• Données structurées Schema.org (JSON-LD)Problème
  • No HTML page available for analysis (homepage unreachable).
• Meta title/description et Open GraphProblème
  • No HTML page available for analysis.
• Sitemap XMLÀ améliorer
  • Sitemap reachable with 0 URLs.
  • The sitemap contains no URLs (it may be an unfollowed sitemap index, or empty).
  • It is not declared in robots.txt (crawlers have to guess its location).

  Comment le corriger : Make sure the sitemap contains valid URLs and is declared in robots.txt.

• Fraîcheur du contenu (lastmod du sitemap)Non applicable
  • The sitemap has no <lastmod> tags: freshness cannot be assessed.

  Comment le corriger : Add <lastmod> to every sitemap URL to signal the update date.

• Structure du contenu (answer-first)Problème
  • No HTML page available for analysis.
• Cohérence de l'identité de marqueNon applicable
  • At least 2 crawled pages with content are needed to assess cross-page consistency.