Tools / Crawler / ignoreNoFollowTo

ignoreNoFollowTo

Type: boolean
Default: false
Parameter syntax
ignoreNoFollowTo: true|false

About this parameter

Whether the Crawler should follow links marked as nofollow (i.e., with the rel=”nofollow” tag) and extract links from a page whose robots meta tag contains nofollow or none.

When true, the Crawler will consider links as if they weren’t marked by a nofollow directive.

This setting applies to both:

The crawler always ignores links that don’t match the patterns of your configuration.

While ignoreNoFollowTo only accepts a boolean, it might eventually accept an array of patterns, similar the ones in exclusionPatterns or ignoreCanonicalTo. Setting ignoreNoFollowTo to an array of patterns would ignore the noFollow directives for only links matching those patterns. Please reach out to your Customer Success Manager or support@algolia.com if you need such fine-grained control.

Examples

1
2
3
{
  ignoreNoFollowTo: true,
}

Did you find this page helpful?