# aiseocheck.hu

> Fast AI SEO check for crawler access, indexing, entity data and citability. Learn how ChatGPT and Perplexity reconstruct brands.

This site provides a fast check of crawler access, indexing, entity data and citability for AI models. AI visibility is stochastic; a single number serves only as a proxy for complex model behavior.

## Author

- Zoltán Varga — AI visibility researcher and AI ecosystem architect, Budapest: https://vargazoltan.ai/
- English profile: https://zoltanvarga.ai/en/about/
- ORCID: https://orcid.org/0009-0003-1020-834X
- arXiv author: https://arxiv.org/a/varga_z_1.html
- Zenodo: https://zenodo.org/search?q=owners%3A1653886

## Publications

- Per-Entity Bias Mapping for AI Visibility: Why Brand Mentions Require Entity-Specific Calibration (arXiv:2606.21595): https://arxiv.org/abs/2606.21595
- AI Visibility in the Generative Era: A Manifesto for Entity-Level Measurement (DOI 10.5281/zenodo.20285424): https://doi.org/10.5281/zenodo.20285424
- Quality Challenges Do Not Escalate Source Fabrication in Large Language Models (RICE-P1) (DOI 10.5281/zenodo.20474050): https://doi.org/10.5281/zenodo.20474050
- Rejection-Induced Confabulation Escalation — Pilot Study 2 (RICE-P2) (DOI 10.5281/zenodo.20509755): https://doi.org/10.5281/zenodo.20509755

## Key topics

- AI SEO check
- AI visibility audit
- crawler access test
- entity data optimization
- AI citability score
- hallucination detection
- LLMO strategy
- GEO optimization
- AI search indexing
- brand reconstruction
- AI SEO Hungary
- crawling check
- indexing issues
- AI citation errors
- perplexity audit
- chatgpt sources
- AI visibility index
- machine readable entity

## Questions and answers

### Why do AI models invent sources for my brand?

Models reconstruct entities based on available training data rather than live verification. High-awareness brands often trigger more confident but false attributions due to the brand hallucination paradox. This occurs when familiarity provides material for plausible completions without factual support.

### What is the difference between SEO and AI visibility?

SEO targets deterministic ranking algorithms while AI visibility addresses stochastic answer generation. AI systems do not log impressions or clicks, making traditional metrics unreliable. Optimization focuses on making content easy to quote and structurally sound for retrieval.

### How do I check if AI crawlers can access my site?

Verify that all major AI bots receive HTTP 200 responses on your key pages. Indexing remains a separate bottleneck where engines may know zero pages despite successful crawling. Use tools that simulate specific AI agent behaviors to confirm access.

### Can I improve my AI visibility score permanently?

No single number guarantees permanent visibility because model outputs are probabilistic distributions. Improvements involve fixing infrastructure gaps like entity anchoring and reducing source fabrication risks. Scores fluctuate based on model updates and query contexts.

### What is the PEBM study regarding AI hallucinations?

PEBM analyzed 1,400 runs across 100 Hungarian B2B companies to measure source fabrication rates. It found that 45.9% of cited sources were fabricated overall, with higher rates for well-known firms. The study highlights the correlation between brand awareness and hallucination frequency.

### Does paid advertising guarantee better AI citations?

The pay-to-appear hypothesis suggests a correlation but not causation between spending and citation quality. Models prioritize machine-readable entity data over commercial signals when constructing answers. Structural clarity matters more than ad spend for accurate reconstruction.

## Related

- AI visibility expert page: https://vargazoltan.ai/ai-lathatosagi-szakerto/
- AI visibility audit: https://vargazoltan.ai/audit/
- Research: https://vargazoltan.ai/kutatas/

## Articles

- [AI Bots in robots.txt: GPTBot, ClaudeBot, PerplexityBot and More](https://aiseocheck.hu/articles/ai-bots-in-robots-txt/): To allow AI crawlers like GPTBot and ClaudeBot, add specific User-agent directives followed by Allow: / in your robots.txt file. You must list each bot explicitly (e.g., GPTBot, ClaudeBot, PerplexityBot) before the wildcard rule to override default restrictions. Ensure no conflicting HTTP headers like X-Content-Signal override these file-level permissions.
- [Training Crawlers vs Search Fetchers: Why the Difference Matters](https://aiseocheck.hu/articles/training-crawlers-vs-search-fetchers/): You should generally allow bots that fetch content for user queries while carefully managing those that harvest data for model training. Blocking GPTBot prevents your text from becoming part of future training sets, whereas allowing OAI-SearchBot enables your site to appear in real-time search answers. The decision hinges on whether you want your content to influence the model's knowledge base or simply serve live user requests.
- [Testing Real Bot Access: Why robots.txt Text Is Not Enough](https://aiseocheck.hu/articles/test-real-bot-access/): To test if AI crawlers can access your site, you must send HTTP requests using specific bot user-agents like GPTBot or ClaudeBot and record the returned status codes. A robots.txt file stating "allow" does not guarantee access if server headers or firewall rules return a 403 Forbidden error. Manual verification of these responses is the only way to confirm true crawlability.
- [Silent 403s: How Firewalls and CDNs Block AI Crawlers](https://aiseocheck.hu/articles/silent-403s-firewalls-cdns/): AI bots receive 403 Forbidden errors because firewall rules, CDN default settings, or conflicting HTTP headers block their specific User-Agent strings. These blocks occur silently because standard SEO tools only read the robots.txt file and do not simulate actual requests from GPTBot or ClaudeBot. The result is a site that appears accessible in reports but remains invisible to answer engines.
- [When HTTP Headers Contradict robots.txt: An Audit Pattern](https://aiseocheck.hu/articles/headers-contradicting-robots-txt/): Yes, an HTTP header can block AI crawlers even when robots.txt explicitly allows them. In cases where a server or Content Delivery Network injects headers like X-Content-Signal with values such as noai or notraining, these directives override the permissions granted in the robots.txt file. Sophisticated crawlers prioritize these response headers, effectively preventing indexing regardless of the text file's instructions.
- [JavaScript Rendering and AI Crawlers: SSR, SSG or Client-Only?](https://aiseocheck.hu/articles/javascript-rendering-ai-crawlers/): AI crawlers can read JavaScript-rendered content only if the server sends a complete HTML document or if the crawler successfully executes the script. Client-only single-page applications frequently appear empty to these systems because the initial response lacks visible text. Server-side rendering, static generation, or pre-rendering ensures the content exists in the raw HTML source for immediate indexing.
- [Bing Indexing and IndexNow: The AI Answer Engine Layer](https://aiseocheck.hu/articles/bing-indexing-and-indexnow/): Bing indexing matters for AI assistants like Copilot because Microsoft's answer engine relies on Bing's index as its primary source for real-time web citations. If a website is missing from Bing's index, it remains invisible to the AI models that power these conversational search tools, regardless of its quality or relevance.
- [Sitemaps, lastmod and hreflang: Freshness Signals for AI Crawlers](https://aiseocheck.hu/articles/sitemaps-lastmod-hreflang/): Yes, sitemaps and lastmod dates are critical for AI search crawlers to prioritize fresh content and discover new URLs efficiently. Without explicit lastmod tags and consistent hreflang signals, AI models may skip updated pages or fail to recognize language versions, reducing citation likelihood.
- [Technical AI-SEO Check: Access, Fetch, Parse, Index](https://aiseocheck.hu/articles/technical-ai-seo-check/): To ensure AI visibility, you must verify that your site explicitly allows AI crawlers in robots.txt, returns clean HTTP status codes without conflicting headers, renders content via server-side processing for easy parsing, and provides structured sitemaps with schema markup. These four stages—access, fetch, parse, and index—form the baseline for any AI-readiness audit.
- [AI Crawler Traffic Trends: What the Bot Share Numbers Say](https://aiseocheck.hu/articles/ai-crawler-traffic-trends/): AI crawler traffic is rising rapidly, with specific agents like ClaudeBot nearly doubling their desktop share between 2024 and 2025. While these bots now account for a significant portion of requests, they generate far less referral traffic than traditional search engines, creating a paradox where server load increases while human click-through rates decline.
- [Crawler Access Glossary: User-Agents, Status Codes, Directives](https://aiseocheck.hu/articles/crawler-access-glossary/): User-agent strings identify the specific bot requesting your page, while directives in robots.txt and HTTP headers explicitly grant or deny access to AI systems. Status codes confirm whether the server successfully delivered the content or blocked the request, serving as the final gatekeeper for AI visibility.
- [Technical AI-SEO FAQ: Blocking, Rendering and Indexing](https://aiseocheck.hu/articles/technical-ai-seo-faq/): You should generally not block AI crawlers if you want your content cited in AI answers, as blocking prevents access to your information entirely. Modern AI systems rely on open access to public web pages to build their knowledge bases, and blocking via robots.txt stops this process. The critical factor is ensuring your server correctly handles bot requests and renders content for both humans and machines.