AI Crawler Policy Matrix
Inspect robots.txt decisions for officially documented AI search, training and user-request agents.
How to use the AI Crawler Policy Matrix
Inspect robots.txt decisions for officially documented AI search, training and user-request agents.
Interpret the results
Compare search access and model-training preferences separately; a user-request fetch does not always behave like a crawler.
Example
Enter https://example.org/ and inspect public source evidence.
Limits and methodology
Real public HTTP(S) evidence is scoped to a page or robots file. Not a live ranking report.
Use this tool together with related checks for a fuller picture. Recommendations are based on the available observed evidence, not on secret search engine ranking data.
Tool workflow and verification
Enter https://example.org/ and inspect public source evidence.
Compare search access and model-training preferences separately; a user-request fetch does not always behave like a crawler.
Real public HTTP(S) evidence is scoped to a page or robots file. Not a live ranking report.
This tool page is an educational guide as well as a functional utility. Demand trends and real search impressions must come from verified search data, not keyword guesses.
Frequently asked questions
Does this check real information?
Yes. The output is based on published public HTML/robots data or the content supplied to this form.
Does it guarantee Google ranking or AI citations?
No. Source checks, imported measurements, and editorial heuristics do not guarantee placement.
What should I verify after the result?
Compare search access and model-training preferences separately; a user-request fetch does not always behave like a crawler.