API Reference / Crawler Configuration API / ignoreCanonicalTo

ignoreCanonicalTo

Type: boolean
Default: false
Parameter syntax
ignoreCanonicalTo: true|false|[
  'http://www.example.com',
  ...
]

About this parameter

Whether the Crawler should extract records from a page that has a canonical URL specified.

When true all canonical will be ignored and page crawled as if there was none specified.

When using an array of patterns, the crawler will ignore the value of the canonical only if it matches one or more patterns.

Examples

1
2
3
{
  ignoreCanonicalTo: true,
}
1
2
3
4
5
{
  ignoreCanonicalTo: [
    'http://www.mysite.com/my-path/**',
  ],
}

Did you find this page helpful?