Tools / Crawler / Enriching data

If you use Adobe Analytics to track metrics like popularity, you can use this data to boost relevant search results and improve your site’s search experience.

With some configuration changes, your crawler can automatically fetch Adobe Analytics metrics.

  1. Add developers and enable the correct permissions in Adobe Admin Console.
  2. Generate a service account on Adobe Developer Console.
  3. Set up the analytics data
  4. Create a new external data source in the Crawler dashboard.
  5. Edit your crawler’s recordExtractor to add Adobe Analytics metrics to the output records.

Set up Adobe Analytics access

Get your company and report suite IDs

The Crawler needs your company and report suite IDs to access your Adobe Analytics data:

  1. Go to your Adobe Analytics interface
  2. Go to Admin > All Admin > Company settings home > API Access.
  3. Note down your Company ID and Report Suite ID for when you create a service account later.

For more information about report suites, refer to Adobe’s documentation.

Add developer permissions to your product profile

The Crawler’s connection to Adobe Analytics requires Developer permissions:

  1. Go to Adobe Admin Console and select Adobe Analytics in the Products list.
  2. Select or create a Product Profile and click the Developers tab.
  3. Click Add Developer.

Adobe admin console

For more information about developer management, read Adobe’s documentation.

Grant permissions for the report suite

In the Product Profile of the Adobe Admin Console:

  1. Go to the Permissions tab.
  2. Click Report Suites and add your report suite to the Included Permission Items.
  3. Do the same for the Metrics and Dimensions you want to use with the crawler.

Adobe profile permissions

Create a service account

  1. Go to Adobe Developer Console and sign in with the developer account you added.
  2. In the Projects tab, click Create new project or use an existing one.
  3. In the project, click + Add to Project > API, select the Adobe Analytics API, and click Next.
  4. When asked for the authentication type, select OAuth Server-to-Server.
  5. Use your Company ID and Report Suite ID when setting up the service account.

Adobe dev console

Push page URLs into an eVar

The Crawler uses Adobe’s Page dimension to map analytics data to each visited URL. By default, this dimension collects data from the pageName variable and falls back to the pageURL variable (the pageURL dimension isn’t available in Adobe Analysis Workspace).

To ensure reliable URL population of named pages, create an eVar and push the page URLs into it:

  1. Enable a new eVar.

    Adobe create evar

  2. Use a processing rule to copy the page URL into the eVar.

    Adobe create processing rule

Create a new external data source

  1. From the Crawler page, select the External Data tab.
  2. Click Add External Data.
  3. As External Data type, select Adobe Analytics**, and fill in the form:
  • Unique name: references this external data in your crawler configuration.
  • Client ID and Client Secret: the credentials obtained when you created your service account
  • Scopes: a comma-separated list of the scopes you require. To see the scopes available for your account, go to your project on the Adobe Developer Console.

Add the external data source to your crawler configuration

Edit the crawler configuration to use the external data you just created:

  1. Select a crawler and click the Editor tab.
  2. Add the external data’s Unique name to your externalData property. For example:

    externalData: ['myAdobeAnalyticsData']

  3. Save your changes.

The Crawler reports errors encountered when fetching analytics less than one minute after the crawl begins.

Integrate analytics into records

Edit your recordExtractor so that it integrates metrics from Adobe Analytics into the records it produces:

  1. Select a crawler and click the Editor tab.
  2. Read metric values from the external data source you added and store them as attributes for your resulting records. If the current page has Adobe Analytics data, the associated metrics are present in the dataSources parameter of your recordExtractor:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    
    recordExtractor: ({ url, dataSources }) => {
      // 'myAdobeAnalyticsData' must match one of the unique names defined in 'externalData'
      const pageviews = dataSources.myAdobeAnalyticsData['metrics/pageviews'];
      return [
        {
          objectID: url.href,
          pageviews,
        },
      ];
    },
    
  3. In the URL Tester, enter the URL of a page with Adobe Analytics data.
  4. Click Run test.
  5. When the test completes, click External data. You should see the data extracted from Adobe Analytics for that page.

If this doesn’t work as expected, try adding a trailing / to your URL or test with another URL.

Did you find this page helpful?