Base Snapshot Ingestion & Schema Reference

Base is a customer intelligence engine that turns customer data into deep audience insights for marketers and AI agents. Customer data is ingested via point-in-time Snapshots, which contain individual Customer records.

Core Entities

1. `BaseSnapshot` (Snapshot Container Header)

A BaseSnapshot represents a single batch upload of customer data (e.g. quarterly customer list, CRM sync, or marketing export).

Field	Type	Description
`snapshot_id`	`STRING`	1–32 char lowercase slug key (e.g. `q3-customers`, `black-friday`)
`pid`	`STRING`	Partner ID (channel host)
`_id`	`STRING`	User / Account ID
`domain`	`STRING`	Target domain associated with the snapshot
`batch_type`	`ENUM`	`CSV_UPLOAD` \| `API_STREAM` \| `INTEGRATION` \| `MANUAL`
`value_type`	`ENUM`	`LTV` \| `AOV` \| `Total_Revenue` \| `ARPU`
`jtbd`	`STRING`	Snapshot-specific marketing intent / Job-To-Be-Done (e.g. `Q3 Holiday Lookalike Campaign`)
`status`	`ENUM`	`ACTIVE` \| `EXPEDITE` \| `COMPLETED` \| `DELETED`
`uploaded_at`	`TIMESTAMP`	Ingestion timestamp (`CURRENT_TIMESTAMP()`)

Snapshot ID Format & Restrictions

Length: Must be between 1 and 32 characters.
Format: Lowercase alphanumeric characters with hyphens and underscores (/^[a-z0-9_-]+$/).
Auto-Sanitization: Incoming values (e.g. "Q3 Customer List 2026") are automatically lowercased, trimmed, and slugified (e.g. "q3-customer-list-2026") by the schema parser.
Usability: Optimized for node IDs in Force Graphs, Webflow CMS slugs, and clean REST API paths.

2. `BaseCustomerRaw` (Raw Customer Rows)

Each row in a snapshot is stored in the base_customers_raw table linked by snapshot_id, _id, and domain.

Field	Type	Description
`uuid`	`STRING`	Unique row UUID (`GENERATE_UUID()`)
`snapshot_id`	`STRING`	1–32 char lowercase slug key linking to `BaseSnapshot`
`_id`	`STRING`	User / Account ID
`domain`	`STRING`	Target domain
`fkID`	`STRING`	External CRM / Customer ID
`email_hash`	`STRING`	SHA-256 hash of customer email address
`zip`	`STRING`	5-digit US ZCTA zip code
`value`	`FLOAT64`	Customer monetary value (defined by `value_type`)
`segment`	`STRING`	Spatial persona segment (e.g., `A01`, `B04`)
`persona_version`	`STRING`	Persona taxonomy version (e.g., `26.01`)
`status`	`ENUM`	`ACTIVE` \| `ARCHIVED` \| `DELETED`
`snapshot_at`	`TIMESTAMP`	Temporal cutoff date of customer event

3. `BaseCustomer` (Enriched View)

The base_customers View joins base_customers_raw and base_snapshots via a compound tenant key (snapshot_id, _id, domain) to guarantee zero multitenant data pollution.

CREATE OR REPLACE VIEW `socialsignal.beaconv2.base_customers` AS
SELECT 
  c.uuid,
  c.snapshot_id,
  c._id,
  s.pid,
  c.domain,
  s.batch_type,
  s.value_type,
  c.fkID,
  c.email_hash,
  c.zip,
  c.value,
  COALESCE(c.segment, e.segment) AS segment,
  COALESCE(c.persona_version, e.persona_version, '26.01') AS persona_version,
  CASE 
    WHEN c.segment IS NOT NULL THEN 'DIRECT'
    WHEN e.segment IS NOT NULL THEN 'EMAIL'
    ELSE 'UNMATCHED'
  END AS response_type,
  c.status,
  c.snapshot_at,
  s.uploaded_at
FROM `socialsignal.beaconv2.base_customers_raw` c
JOIN `socialsignal.beaconv2.base_snapshots` s 
  ON c.snapshot_id = s.snapshot_id 
 AND c._id = s._id 
 AND c.domain = s.domain
LEFT JOIN `socialsignal.beaconv2.lk_email_segments` e 
  ON c.email_hash = e.email_hash
WHERE (s.status IS NULL OR s.status = 'ACTIVE') 
  AND (c.status IS NULL OR c.status = 'ACTIVE');

API Payload Mockups

1. Flow Template API Invocation (`base.snapshot.create`)

{
  "snapshot_id": "snap_9a8b7c6d-5e4f-3a2b-1c0d-9e8f7a6b5c4d",
  "pid": "GRP_44901",
  "_id": "usr_acme_marketing",
  "domain": "acmestore.com",
  "batch_type": "CSV_UPLOAD",
  "value_type": "LTV",
  "customers": [
    {
      "uuid": "cust_1001_a1b2c3d4",
      "fkID": "CRM_90812",
      "email_hash": "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824",
      "zip": "90210",
      "value": 1450.00,
      "segment": null,
      "persona_version": "26.01",
      "snapshot_at": "2026-08-01T00:00:00Z"
    },
    {
      "uuid": "cust_1002_e5f6g7h8",
      "fkID": "CRM_90813",
      "email_hash": "4813494d137e1631bba301d5acab6e7bb7aa74ce1185d456565ef51d737677b2",
      "zip": "10001",
      "value": 820.50,
      "segment": "A01",
      "persona_version": "26.01",
      "snapshot_at": "2026-08-01T00:00:00Z"
    },
    {
      "uuid": "cust_1003_i9j0k1l2",
      "fkID": "CRM_90814",
      "email_hash": "8f4e5d6c7b8a90123456789abcdef0123456789abcdef0123456789abcdef012",
      "zip": "30301",
      "value": 310.00,
      "segment": null,
      "persona_version": "26.01",
      "snapshot_at": "2026-08-01T00:00:00Z"
    }
  ]
}

2. GCP Streaming Payload (`batch-publish-to-bq`)

Internal payload formatted by the flow runner for parallel BigQuery streaming:

{
  "mode": "batch",
  "payloads": {
    "base_snapshots": [
      {
        "snapshot_id": "snap_9a8b7c6d-5e4f-3a2b-1c0d-9e8f7a6b5c4d",
        "pid": "GRP_44901",
        "_id": "usr_acme_marketing",
        "domain": "acmestore.com",
        "batch_type": "CSV_UPLOAD",
        "value_type": "LTV",
        "status": "ACTIVE",
        "uploaded_at": "2026-08-04T08:41:00Z"
      }
    ],
    "base_customers_raw": [
      {
        "uuid": "cust_1001_a1b2c3d4",
        "snapshot_id": "snap_9a8b7c6d-5e4f-3a2b-1c0d-9e8f7a6b5c4d",
        "fkID": "CRM_90812",
        "email_hash": "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824",
        "zip": "90210",
        "value": 1450.00,
        "segment": null,
        "persona_version": "26.01",
        "status": "ACTIVE",
        "snapshot_at": "2026-08-01T00:00:00Z"
      },
      {
        "uuid": "cust_1002_e5f6g7h8",
        "snapshot_id": "snap_9a8b7c6d-5e4f-3a2b-1c0d-9e8f7a6b5c4d",
        "fkID": "CRM_90813",
        "email_hash": "4813494d137e1631bba301d5acab6e7bb7aa74ce1185d456565ef51d737677b2",
        "zip": "10001",
        "value": 820.50,
        "segment": "A01",
        "persona_version": "26.01",
        "status": "ACTIVE",
        "snapshot_at": "2026-08-01T00:00:00Z"
      },
      {
        "uuid": "cust_1003_i9j0k1l2",
        "snapshot_id": "snap_9a8b7c6d-5e4f-3a2b-1c0d-9e8f7a6b5c4d",
        "fkID": "CRM_90814",
        "email_hash": "8f4e5d6c7b8a90123456789abcdef0123456789abcdef0123456789abcdef012",
        "zip": "30301",
        "value": 310.00,
        "segment": null,
        "persona_version": "26.01",
        "status": "ACTIVE",
        "snapshot_at": "2026-08-01T00:00:00Z"
      }
    ]
  }
}

Flow Template Operations

Flow ID	Method	Description
`base.snapshot.create`	`POST`	Register new snapshot container & stream customer rows
`base.snapshot.list`	`GET`	List active snapshots for account/domain with pagination
`base.snapshot.get`	`GET`	Get snapshot header, record counts, email match rate %, and mean value
`base.snapshot.delete`	`DELETE`	Soft-delete a snapshot (`status = 'DELETED'`)

Base Snapshot Ingestion & Schema Reference

Core Entities

1. BaseSnapshot (Snapshot Container Header)

2. BaseCustomerRaw (Raw Customer Rows)

3. BaseCustomer (Enriched View)