Troubleshooting by crawl status

Crawler status messages inform you of successful or failed crawl attempts. Use the Monitoring tool to examine a crawler’s status. URLs can have one of three crawl statuses:

• Success.
• Ignored (skipped). URLs didn’t match any of your configuration’s pathsToMatch or did match one of your exclusionPatterns
• Failed. Each failed URL fits into one of four categories:
  • Fetching. URLs that failed when accessing your site
  • Extraction. URLs that failed when attempting to extract data from your site.
  • Indexing. URLs that weren’t sent to Algolia.
• Internal. An internal crawler error

Failed URL details

To get more details about a failed URL:

1. Open the URL inspector
2. Click Test URL to open the URL Tester.
3. Click Run Test.

In the following example, the URL failed because the record was too big: