> ## Documentation Index
> Fetch the complete documentation index at: https://algolia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# linkExtractor

> Function for finding links to crawl.

* **Type**: `function`

By default, the crawler adds URLs to the queue based on
[`pathsToMatch`](/doc/tools/crawler/apis/configuration/actions#param-paths-to-match),
[`fileTypesToMatch`](/doc/tools/crawler/apis/configuration/actions#param-file-types-to-match),
and [`exclusionPatterns`](/doc/tools/crawler/apis/configuration/exclusion-patterns).

To customize this behavior, define a `linkExtractor` function.
This function runs on every crawled page and returns a list of URLs as strings.

## Examples

```js JavaScript icon=code theme={"system"}
  {
    linkExtractor: ({ $, url, defaultExtractor }) => {
      if (/example.com\/doc\//.test(url.href)) {
        // For all pages under /doc, only queue the first found link
        return defaultExtractor().slice(0,1);
      }
      // Otherwise, use the default logic (queue all found links)
      return defaultExtractor();
    },
  }
```

```js JavaScript icon=code theme={"system"}
{
  linkExtractor: ({ $, url, defaultExtractor }) => {
    // This turns off link discovery, except for URLs listed in sitemap.xml
    return /sitemap.xml/.test(url.href) ? defaultExtractor() : [];
  },
}
```

```js JavaScript icon=code theme={"system"}
{
  linkExtractor: ({ $ }) => {
    // Access the DOM and extract what you specify
    return [$('.my-link').attr('href')]
  },
}
```

## Parameters

<ParamField body="$" type="object">
  A [Cheerio](https://cheerio.js.org/) instance with the HTML of the crawled page.
</ParamField>

<ParamField body="defaultExtractor" type="function">
  The default extractor function.
</ParamField>

<ParamField body="url" type="URL">
  URL object of the crawled page.
</ParamField>