Tools
/
Crawler
/
APIs
/
Configuration
Jan 17, 2022
Crawler: MaxDepth
Type:
number
Parameter syntax
maxDepth: max_depth
About this parameter
Limits the processing of URLs to the specified depth, inclusively.
This parameter can’t be higher than 100
.
URLs added manually (startUrls
, sitemaps
) are not checked against this limit.
How depth is calculated
Copy
1
2
3
4
5
6
7
http://example.com => 1
http://example.com/ => 1
http://example.com/foo => 1
http://example.com/foo/ => 2
http://example.com/foo/bar => 2
http://example.com/foo/bar/ => 3
...
Examples
Copy
1
2
3
{
maxDepth: 10,
}
Did you find this page helpful?