Free tip · comment word

SCAN

If AI can't read your website, you don't stand a chance.

Here's the free tip first. The checklist is below.

Try this first · free

Check your robots.txt file

Every website can have a small file that tells bots where they can and can't go. Some sites block AI bots without the owner knowing, often from a security plugin or an old setting.

Do it todayType yourwebsite.com/robots.txt into your browser (with your real web address). If you see an AI bot's name, like GPTBot or PerplexityBot, followed by Disallow: /, that bot is told to stay out. Ask whoever runs your site to remove it.
Your bonus

The 10 things AI needs to read your site

  1. Your robots.txt isn't blocking AI. Open yourwebsite.com/robots.txt. Look for AI bot names like GPTBot, OAI-SearchBot, PerplexityBot or ClaudeBot next to "Disallow." That line tells the bot to stay out.
  2. Your words are in the page itself. Right-click your page and pick "View Page Source." Search for a sentence from your page. If it isn't there, it only shows up after code runs, and many AI bots don't run that code.
  3. No "noindex" by mistake. In that same page source, search for "noindex." If your important pages have it, you're telling search engines to skip them.
  4. You have a sitemap, and it's submitted. Check yourwebsite.com/sitemap.xml. Submit it in Google Search Console and Bing Webmaster Tools.
  5. Pages load fast and don't break. No error pages, no login walls in front of the info people need, and it works on a phone.
  6. One clear page per service. Put a plain answer near the top: what you do, who it's for, and where you work.
  7. Headings that sound like real questions. "How much does a drain cleaning cost?" is easier for AI to match than "Our Solutions."
  8. Schema markup. A small block of code that tells machines your business name, address, phone, hours and services. Most website builders have a setting or plugin for it.
  9. The same name, address and phone everywhere. Your site, your Google Business Profile and directories should all match, word for word.
  10. An llms.txt file. A short, plain-text map of your site, written for AI, at yourwebsite.com/llms.txt. It's a newer idea and not every AI tool reads it yet, but it takes 10 minutes. There's a sample below.
Sample

What an llms.txt file looks like

Swap in your own business, then save it as llms.txt at the top level of your website.

# Your Business Name
> Plumbing in Long Beach, CA. Open 7 days. Call (555) 555-5555.

## Services
- [Drain cleaning](https://yourwebsite.com/drains): clogs, slow drains, camera checks
- [Water heaters](https://yourwebsite.com/water-heaters): repair and replacement

## About
- [Reviews](https://yourwebsite.com/reviews)
- [Contact and booking](https://yourwebsite.com/contact)
Want a free look?

I'll check your site for you

Send me 3 things: your business, your website, and the one thing you most want more of (calls, sales or trust). I'll look and get back to you.

  • This one isn't a tool, and there's no affiliate link on this page.
  • Fixing these helps AI read your site. It doesn't guarantee AI will recommend you.