Tools
/
Crawler
/
Troubleshooting
Nov 18, 2024
Crawler error messages
Error messages and warnings generated by the Algolia Crawler.
| Error | Category | Status | Solution |
|---|---|---|---|
| Algolia error: Record too big | Indexing | Failed | Records exceed the maximum for your Algolia plan |
| Algolia error: Timeout | Indexing | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Algolia’s record quota exceeded | Indexing | Failed | Records exceed the maximum for your Algolia plan |
| Canonical URL | Extraction | Skipped | Canonical URL errors |
| Canonical URL - Not processed | Extraction | Skipped | Canonical URL errors |
| Content-Type not supported | Fetch | Skipped | HTTP status errors |
| Crawler internal error (extraction) | Extraction | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Crawler internal error (fetch) | Fetch | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Crawler internal error (indexing) | Indexing | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Crawler internal error (isolated extraction) | Extraction | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Crawler internal error (processJob) | Global | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Crawler internal error (work) | Global | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| DNS error | Fetch | Failed | DNS error |
| Document extraction timeout reached | Fetch | Failed | Document extraction timeout |
| Document extraction: unprocessable document | Fetch | Failed | Tika errors |
| Extractor exceeded memory limit | Extraction | Failed | Page is too big |
| Extractor execution timed out | Extraction | Failed | Extractor timed out |
| Extractor record duplicate error | Extraction | Failed | Crawler data limitations |
| Extractor record error | Extraction | Failed | Crawler data limitations |
| Extractor record merge error | Extraction | Failed | Page is too big |
| Extractor record missing ID error | Extraction | Failed | Crawler data limitations |
| Extractor returned too many links | Extraction | Failed | Extractor returned too many links |
| Extractors returned too many records | Extraction | Failed | Records exceed the maximum for your Algolia plan |
| Fetch timeout | Fetch | Failed | Network errors |
| File type didn’t match any action | Fetch | Skipped | Document type actions |
| Forbidden by robots.txt | Fetch | Skipped | Forbidden by robots.txt |
| HTTP Bad Request (400) | Fetch | Skipped | HTTP status errors |
| HTTP Connection Error (502,503,504) | Fetch | Failed | HTTP status errors |
| HTTP Forbidden (403) | Fetch | Skipped | HTTP status errors |
| HTTP Gone (410) | Fetch | Skipped | HTTP status errors |
| HTTP Internal Server Error (500) | Fetch | Failed | HTTP status errors |
| HTTP Not Found (404) | Fetch | Skipped | HTTP status errors |
| HTTP Not Implemented (501) | Fetch | Skipped | HTTP status errors |
| HTTP redirect (301, 302) | Fetch | Skipped | Redirect issues |
| HTTP redirect (301, 302) - Not followed | Fetch | Skipped | Redirect issues |
| HTTP Status Code not supported | Fetch | Failed | HTTP status errors |
| HTTP Too Many Requests (429) | Fetch | Failed | HTTP status errors |
| HTTP Unauthorized (401) | Fetch | Skipped | HTTP status errors |
| JavaScript redirect - Not followed | Fetch | Skipped | Redirect issues |
| JavaScript renderer backend unavailable | Fetch | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| JavaScript renderer error | Fetch | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Link Extractors return an invalid payload | Extraction | Failed | linkExtractor should return an array of strings but returns an unexpected type |
| Maximum amount of retries reached | Global | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| Network error | Fetch | Failed | Network errors |
| Record objectID too long | Extraction | Failed | Data issues |
| Records extracted are too big | Extraction | Failed | Records exceed the maximum for your Algolia plan |
| Runtime extractor error | Extraction | Failed | Error in your crawler configuration |
| SSL error | Fetch | Failed | SSL issues |
| Tika Error | Fetch | Failed | Tika errors |
| Too many failed URLs | Fetch | Failed | Safety check breaches |
| Too many missing records | Fetch | Failed | Safety check breaches |
| Unsupported Algolia error | Indexing | Failed | Re-try the crawl. Contact the Algolia support team if the error persists |
| url_too_deep | Extraction | Failed | Crawler data limitations |
See also
Did you find this page helpful?
