Data sources

A data source is one tenant-owned connection to an external system you want to pull records from. Sources are managed through a real CRUD surface at /v1/data/sources (backed by the data_sources table, alembic 0097_data_sources) and surfaced in the app under Data → Sources.

Every row carries its operational state inline — status, last_synced_at, last_error, and last_records_count — while per-provider connection parameters live in a free-form config_json blob so the row shape never changes when a new connector kind is added.

Only Salesforce and HubSpot sources sync today

CRUD (create, read, update, delete) is fully real and tenant-isolated. Sync never writes a number nothing measured (core/data_source_sync.py):

salesforce and hubspot sources run Pact's real read connector on your workspace's connection (the same engine as Settings → Integrations → Sync now). The count comes from that connector's run log, integration_sync_logs — imported plus updated — and every read of the source reports it with the run it came from (sync.measured_by).
Every other kind has no connector yet. Sync answers unsupported with a gap naming what is missing and what to do instead (usually the import wizard under Admin → Import), and the source row is not changed.

Until this change, sync on any kind wrote a random per-kind record count. Those figures are still on the rows, but no surface shows them as a count: the API reports sync.unmeasured_history: true instead. Seeded sample sources (config.seeded) show their figures labeled Sample.

Supported connector kinds

The kind field is validated against a fixed allowlist (ALLOWED_KINDS):

Kind	Description
`s3`	Amazon S3 bucket / prefix
`postgres`	External Postgres database
`http_csv`	CSV fetched over HTTP
`salesforce`	Salesforce org
`hubspot`	HubSpot portal
`webhook`	Inbound webhook feed
`manual_csv`	Operator-uploaded CSV
`snowflake`	Snowflake warehouse
`bigquery`	Google BigQuery

A source's status is one of idle, syncing, error, or disabled. Only idle and disabled are user-settable via PATCH. Run state for connector kinds lives in the connector's run log and is reported as sync.state (measured, running, stalled, never_run, sample, unsupported, read_failed).

Endpoints

code

POST   /v1/data/sources            create
GET    /v1/data/sources            list (filter by kind, status; paginated)
GET    /v1/data/sources/{id}       fetch one
PATCH  /v1/data/sources/{id}       update name / config / status
DELETE /v1/data/sources/{id}       delete
POST   /v1/data/sources/{id}/sync  run the kind's connector, or refuse with a gap

The list endpoint accepts kind, status, limit (1–200, default 50), and offset query parameters. Every read and write is scoped to your tenant, so a {id} from another tenant resolves to 404.

Path param is the int id today

The source {id} path param is the internal integer id — the routes in api/routes/data_sources.py type it as int, so a UUID public_id is not yet accepted here. The public_id column exists on the table (alembic 0099) for the platform-wide IDOR migration, but this route has not been switched to the dual-accept resolver yet.

1
Create a source
POST /v1/data/sources with name, kind, and a config object. The tenant is derived from your auth context — never sent in the body.
2
Configure connection params
Provider credentials and settings go into config_json. Patch them any time with PATCH /v1/data/sources/{id}.
3
Trigger a sync
POST /v1/data/sources/{id}/sync answers queued (the connector is running; records_ingested is null until the run finishes), not_connected, unsupported, or read_failed. The source detail page polls every 3s while a connector run is in flight, and the count appears when the run finishes.

bash

curl -X POST https://api.pact.place/v1/data/sources \
  -H "Authorization: Bearer $PACT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Prod Postgres","kind":"postgres","config":{"host":"…"}}'

Data source detailConnection status, sync history, and per-source controls.PipelinesTransform and load source data into Pact objects.Data vaultExport Pact data outbound to your own warehouse or object store.