Sync warehouse tables to Signals

To sync existing, pre-calculated attributes from your warehouse to Signals, configure an attribute group with a warehouse (batch) source. No additional modeling is required — the batch engine reads rows from your warehouse table at a fixed interval and sends them to the Profiles Store.

Use stream attributes with backfill when possible

If your source data comes from Snowplow events, consider using a stream attribute group with backfill enabled instead. It gives you real-time updates from your event stream alongside historical backfill, without needing to maintain a separate warehouse table. Attributes that are fetched from a warehouse are best suited for pre-calculated values from non-Snowplow sources, or tables that already exist independently of your event pipeline.

Before you begin

You need:

• Signals with a Snowflake, BigQuery, or Databricks warehouse connection. To add one, see Set up Signals.
• Read access to the table you want to sync.
• A column in that table that records, in UTC, when each row was last written or updated, as described in Configure a warehouse source.

Configure a warehouse source

Console

When creating an attribute group, select Warehouse as the data source.

Provide the basic configuration

Select the Attribute key you have data for. The attribute key for the group must correspond to a column in the table that contains the key values.

Set the TTL to control how long the data persists in the Profiles Store.

Updates run hourly by default. Use the Sync frequency selector to sync less often (6h, 12h, or 24h).

You must also specify a Backfill start date: the earliest date from which Signals should read rows on the first sync.

Define which fields to sync

Select which Table to sync from.

Choose the Modified date column: the column that tells when a particular row was last written or updated in the table (for example, a load_tstamp or updated_at column populated by your ETL process on every write). The Modified date column must be in UTC. A column in local time causes rows to be silently matched against the wrong sync window, with no error raised. If multiple rows exist for the same attribute key within a sync period, the engine uses the row with the latest Modified date value.

Use a last-modified timestamp

Don't use a business or event timestamp that doesn't change when a row is reloaded or updated. The batch engine won't be able to tell the row has changed, and the data may be missed.

Finally, select which Table fields you want to send to Signals.

Python SDK

Start by configuring which table to sync by specifying a BatchSource object.

python

from snowplow_signals import BatchSource

data_source = BatchSource(
    name="ecommerce_transaction_interactions_source",
    database="SNOWPLOW_DEV1",
    schema="SIGNALS",
    table="SNOWPLOW_ECOMMERCE_TRANSACTION_INTERACTIONS_FEATURES",
    timestamp_field="UPDATED_AT",
    owner="user@company.com",
)

This table lists all available arguments for a BatchSource:

Argument	Description	Type	Required?
name	The name of the source	string	✅
description	A description of the source	string	❌
database	The database where the attributes are stored	string	✅
schema	The schema for the table of interest	string	✅
table	The table where the attributes are stored	string	✅
timestamp_field	Primary timestamp of the attribute value, indicating data freshness	string	✅
owner	The owner of the source	string	❌

The batch engine uses timestamp_field as the change-watermark for incremental syncs — it must reflect when a row was actually last written or updated in the table (for example, a load_tstamp or updated_at column populated by your ETL process on every write), not a business or event timestamp that stays fixed once set. timestamp_field must be in UTC. A column in local time causes rows to be silently matched against the wrong sync window, with no error raised.

If multiple rows exist for the same attribute key within a sync period, the engine uses the row with the greatest timestamp_field value.

Use a last-modified timestamp

Don't use a business or event timestamp that doesn't change when a row is reloaded or updated. The batch engine won't be able to tell the row has changed, and the data may be missed.

Define which fields to sync

Use ExternalBatchAttributeGroup to define which source table and fields to use. Instead of attributes, this class uses fields — abstractions over the warehouse columns.

You must set backfill_since_tstamp to tell the batch engine the earliest timestamp from which to read rows on the first sync. Without this, no historical data will be loaded.

The attribute key must correspond to a column in the source table. The example below uses the built-in domain_userid key, so the table must have a domain_userid column. To key on a different column, define a custom attribute key with external_column set to the column name.

python

from datetime import datetime, timedelta, timezone
from snowplow_signals import ExternalBatchAttributeGroup, domain_userid, Field

# data_source: the BatchSource from the previous block

attribute_group = ExternalBatchAttributeGroup(
    name="ecommerce_transaction_interactions_attributes",
    version=1,
    attribute_key=domain_userid,
    owner="user@company.com",
    batch_source=data_source,
    backfill_since_tstamp=datetime(2026, 6, 1, tzinfo=timezone.utc),
    refresh_rate=timedelta(hours=6),
    fields=[
        Field(name="TOTAL_TRANSACTIONS", type="int64"),
        Field(name="TOTAL_REVENUE", type="double"),
        Field(name="AVG_TRANSACTION_REVENUE", type="double"),
    ],
)

This table lists all available arguments for ExternalBatchAttributeGroup:

Argument	Description	Type	Required?
name	The name of the attribute group	string	✅
version	The version of the attribute group	int	✅
attribute_key	The key used to identify profiles. Its name must match a column in the source table	AttributeKey	✅
batch_source	The BatchSource defining the warehouse table to sync from	BatchSource	✅
backfill_since_tstamp	The earliest timestamp from which to read rows on the first sync. Accepts tz-aware or naive UTC datetime; tz-aware is recommended.	datetime	✅
refresh_rate	How frequently the attribute group should be synced (e.g. timedelta(hours=1), timedelta(hours=6), timedelta(days=1)). Defaults to hourly.	timedelta	❌
fields	The list of Field objects defining which columns to sync	list	✅
description	A description of the attribute group	string	❌
owner	The owner of the attribute group	string	❌

This table lists all available arguments for a Field:

Argument	Description	Type	Required?
name	The name of the field	string	✅
description	A description of the field	string	❌
type	The type of the field	one of: bytes, string, int32, int64, double, float, bool, unix_timestamp, bytes_list, string_list, int32_list, int64_list, double_list, float_list, bool_list, unix_timestamp_list	✅

Publish and retrieve

Signals starts syncing the table once you publish the attribute group. In Console, click Publish on the attribute group. With the Python SDK, use publish(), as described in Publish attribute groups:

python

# sp_signals: the client from Connect to Signals
sp_signals.publish([attribute_group])

To read the synced values in your application, add the attribute group to a service, then retrieve the attributes.

To understand why the same definition runs in both the streaming and batch engines, see Signals vs a feature store on the Signals site.