Tools / Crawler / Enriching data

Linking Google Analytics

By using Google Analytics metrics to enrich records your crawler extracts from a site, you can make your search more relevant.

The following steps show how to configure your crawler to integrate Google Analytics data automatically:

  1. Provide read access to Google Analytics data
  2. Create a service account
  3. Create a new external data source
  4. Test your external data
  5. Update your crawler configuration
  6. Integrate Google Analytics metrics into your records.

Provide read access to Google Analytics data

The administrator of your Google Analytics account must provide read access to the email address of the Google Account you want to use (a personal account or service account).

  1. Sign in to Google Analytics.
  2. Select Account, Property, and then the View that contains the analytics for the site you’re crawling.
  3. Go to the Admin tab.
  4. In the Properties panel (on the left side of the screen), click Properties Access Management.
  5. Click the + button, then click Add users.
  6. Paste the email address of the Google Account you want to add.
  7. In the Permissions panel, ensure that only the “Read & Analyze” permission is enabled.
  8. Click Add to confirm.

Add Read & Analyze permissions to Google account

Create a service account

  1. Create (or select an existing) project from the Google Cloud developer console.
  2. Activate the Google Analytics Data API and the Google Analytics Admin API in that project.
  3. In the Credentials section, create a new service account. Skip all the optional steps.
  4. Open the service account and click Add key -> Create new key. Select JSON and click Create to download the resulting private key file.

Create a private key file for the Google Analytics connector

Create a new external data source

  1. From the Crawler page, select the External Data tab.
  2. Click Add External Data.
  3. As External Data type, select Google Analytics**, and fill in the form:
  • Unique name: the name used to reference this external data in your crawler’s configuration.
  • Email: your Google service account email
  • Private key: paste the text from your service account’s downloaded private key file.
  • Properties: select properties to fetch data from. By default, all available properties are selected.
  • Metrics: select one or more metrics to fetch.
  • Collection period: the number of days, starting from any point before yesterday, to use for gathering data.

Create a Google Analytics external data source

Test your external data

Verify that your crawler can connect to your Google Analytics property:

  1. From the Crawler page, select the External Data tab.
  2. Click the icon to the right of your external data source to go to the Explore Data page.
  3. Trigger a manual refresh with the Refresh Data button and verify that the extraction is successful by checking the Status on the rightmost column. If it’s green, it’s all good.

If you see errors, check your credentials.

Update your crawler configuration

Edit the crawler configuration to use the external data you created:

  1. Go to the Crawler editor.
  2. Add the unique name of the external data to your externalData property:
  • externalData: ['myGoogleAnalyticsData']
  1. Save your changes.

After saving your crawler configuration, your Google Analytics metrics will be ready whenever you crawl your site.

If an error occurs while fetching your analytics, it will appear less than one minute after the crawling process starts.

You must crawl your site at least once before the metrics will appear.

Integrate Google Analytics metrics into your records

Edit your recordExtractor so that it integrates metrics from Google Analytics into the records it produces:

  1. Go to the Crawler editor.
  2. Read metric values from the external data source you’ve added beforehand, and store them as attributes for your resulting records. If Google Analytics has data for the current page, the associated metrics should be present in the dataSources parameter of your recordExtractor:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    
    {
      // ...
      recordExtractor: ({ url, dataSources }) => {
        // "myAnalytics" must match one of the unique name defined in `externalData`
        const screenPageViews = dataSources.myAnalytics['screenPageViews'];
    
        return [
          {
            objectID: url.href,
            screenPageViews,
          },
        ];
      },
    };
    

For a complete example, check the recordExtractor of the sample Google Analytics crawler configuration.

  1. In the Test a URL field of the configuration editor (which you can find in the Admin > Settings tab), type the URL of a page with analytics attached to it.
  2. Click Run test.
  3. When the test finishes, go to the External Data tab. You should see the data extracted from Google Analytics for that page.

If this doesn’t work as expected, try adding a trailing / to your URL or test with another URL.

Did you find this page helpful?