login

About this parameter

This property defines how the crawler acquires a session cookie.

The crawler extracts the Set-Cookie response header from the login page and sends that cookie when crawling all pages of the website defined in the configuration.

This cookie is only fetched at the beginning of each complete crawl. If it expires, we won’t renew it automatically.

Examples

1
2
3
4
5
6
7
8
9
10
11
12
13
{
  login: {
    fetchRequest: {
      url: `http://example.com/secure/login-with-post`,
      requestOptions: {
        method: 'POST',
        headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
        body: 'id=my-id&password=my-password',
        timeout: 5000 // in milliseconds
      }
    }
  }
}