# AI site audit: can AI crawl and quote you? · LLMagnet

> Can AI crawl, read, and quote your site? A grade, six categories and the gaps worst-first, each finding tagged as evidence or judgment, with a fix attached.

URL: https://llmagnet.com/site-audit

## Frequently asked questions

### How many pages do you crawl?

The homepage and a sample of pages on the same host, along with your robots.txt, your sitemap and your llms.txt if there is one.

### Do I need an llms.txt file?

The evidence says AI systems are barely reading them: in Ahrefs’ data, 97% of the llms.txt files it found got no requests in a month, and Google has said it doesn’t support the file. It costs nothing to publish one. It isn’t a ranking factor, and we don’t score it as one.

### Should I block AI crawlers?

That’s your call, and it’s mostly about how your content may be used. Worth knowing first: blocking OAI-SearchBot keeps you out of ChatGPT search, OpenAI’s crawlers are set independently of each other, and Google-Extended doesn’t affect Google Search but does decide whether Gemini can use your content.

More about LLMagnet: https://llmagnet.com/llms.txt