Tools / Crawler / APIs / Configuration

Crawler: IgnoreNoFollowTo

Type: boolean
Default: false
Parameter syntax
ignoreNoFollowTo: true|false

About this parameter

Whether the Crawler should follow links with the rel=”nofollow” tag and extract links from a page whose robots meta tag contains nofollow or none.

When true, the Crawler will consider links as if they weren’t marked by a nofollow directive.

This setting applies to both:

The crawler always ignores links that don’t match the patterns of your configuration.

While ignoreNoFollowTo only accepts a boolean, it might eventually accept an array of patterns, similar the ones in exclusionPatterns or ignoreCanonicalTo. Setting ignoreNoFollowTo to an array of patterns would ignore the noFollow directives for only links matching those patterns. Contact your customer success manager or the Algolia support team if you need such fine-grained control.


  ignoreNoFollowTo: true,
Did you find this page helpful?