Agent classification enrichment

CloudPrivate Managed CloudSelf-HostedAddon

Availability

This enrichment is available since version 6.12.0 of Enrich.

The agent classification enrichment identifies automated agents — AI crawlers, bots, and other non-human clients — by matching the agent name parsed by the YAUAA enrichment against a lookup of known agents. For each match, it adds an entity to the event identifying the operator (e.g. OpenAI) and, where known, the agent's purpose (e.g. AI_TRAINING).

Prerequisites

To use this enrichment, you need to enable the YAUAA enrichment. The agent classification enrichment reads the agentName field from the YAUAA entity. If YAUAA is not active, no agent classification entity is attached.

Configuration

The enrichment takes these parameters:

Parameter	Required	Description
classificationFile	❌	Location of a JSON file mapping YAUAA agent names to classification records. Already provided for CDI customers.
classification	❌	Inline classification records, keyed by YAUAA agent name. Takes precedence over matching entries in classificationFile.

At least one of classificationFile or classification must be present.

Console

Configure the parameters in the Console enrichment editor. Keep the Console defaults for the database and uri fields. For example:

json

{
  "classificationFile": {
    "uri": "<use default value from Console>",
    "database": "<use default value from Console>"
  },
  "classification": {
    "SomeBot": {"operator": "Acme Corp", "purpose": "Monitoring"}
  }
}

The agent categories in the default configuration are:

• AI_SEARCH_INDEX — agents that power AI-based search features, e.g. OAI-SearchBot
• AI_TRAINING — agents used for crawling content and training AI models, e.g. GPTBot
• AI_USER_FETCH — agents that fetch content upon user request, e.g. ChatGPT-User
• LINK_PREVIEW — agents that visit links in order to provide a preview of the content, e.g. BingPreview
• SEARCH_INDEX — agents powering traditional search, e.g. Bingbot

Self-Hosted

For Self-Hosted, provide a complete JSON. For example:

json

{
  "schema": "iglu:com.snowplowanalytics.snowplow.enrichments/agent_classification_enrichment_config/jsonschema/1-0-0",
  "data": {
    "name": "agent_classification_enrichment_config",
    "vendor": "com.snowplowanalytics.snowplow.enrichments",
    "enabled": true,
    "parameters": {
      "classificationFile": {
        "uri": "<your file location>",
        "database": "agent-classifications.json"
      },
      "classification": {
        "SomeBot": {"operator": "Acme Corp", "purpose": "Monitoring"}
      }
    }
  }
}

Testing with Micro

Unsure if your enrichment configuration is correct or works as expected? You can easily test it using Snowplow Micro, either through Console or on your machine.

classificationFile

Snowplow CDI

If you're using Snowplow CDI, you don't need to configure this. Use the default values provided in Console.

Points to a JSON file containing the agent-to-classification mapping. The file must be a JSON object where the keys are YAUAA agentName values and the values are objects with operator (required) and purpose (optional). For example:

json

{
  "ChatGPT-User": {"operator": "OpenAI", "purpose": "AI_USER_FETCH"},
  "GPTBot": {"operator": "OpenAI", "purpose": "AI_TRAINING"},
  "ClaudeBot": {"operator": "Anthropic"}
}

Matching logic

The matching is not case-sensitive, i.e., both GPTBot and GPTbot (lowercase b) will match GPTBot.

Field	Type	Description
uri	string	Base URI where the file is hosted. Supports http:, s3:, and gs: schemes. Must not end with a trailing slash.
database	string	The JSON filename.

classification

An inline classification using the same format as the file. Records defined here take precedence over any matching entry in classificationFile.

Output

This enrichment adds an agent_classification entity to events where the YAUAA agentName matches a known agent.

This enrichment won't produce any output if:

• The YAUAA enrichment is not enabled
• The YAUAA entity does not contain an agentName field
• The agentName is not found in the classification lookup

agent_classification

Entity

Schema for agent classification context generated by agent classification enrichment

Schema URIiglu:com.snowplowanalytics.snowplow/agent_classification/jsonschema/1-0-0

Example data

json

{
  "operator": "OpenAI",
  "purpose": "AI_TRAINING"
}

Properties and schema

Table

Property	Description
operator string	Required. The vendor operating the agent, e.g. OpenAI
purpose string	Optional. The purpose of the agent, e.g. AI crawler, if known

Complete JSON schema

json

{
  "$schema": "http://iglucentral.com/schemas/com.snowplowanalytics.self-desc/schema/jsonschema/1-0-0#",
  "description": "Schema for agent classification context generated by agent classification enrichment",
  "self": {
    "vendor": "com.snowplowanalytics.snowplow",
    "name": "agent_classification",
    "format": "jsonschema",
    "version": "1-0-0"
  },
  "type": "object",
  "properties": {
    "operator": {
      "type": "string",
      "description": "The vendor operating the agent, e.g. OpenAI"
    },
    "purpose": {
      "type": [
        "string",
        "null"
      ],
      "description": "The purpose of the agent, e.g. AI crawler, if known"
    }
  },
  "required": [
    "operator"
  ],
  "additionalProperties": false
}