Integrations and data sync

Integrations (per project) connects Guidely to the tools a team already uses:

Connecting or changing an integration needs the Developer settings permission (admins by default); everyone in the workspace can see what's connected and how syncs went.

How syncs work

Each CRM, CDP, knowledge source and warehouse is a connection (one per provider per project). Credentials are checked with the vendor before they're saved, stored encrypted (AES-256-GCM with a key derived from SESSION_SECRET, like Linear and Jira tokens) and never returned; the dashboard shows who they belong to and their last four characters.

Connections sync hourly, daily or only when asked (Sync now). Every run — scheduled, manual or a backfill — is kept in the connection's History with what it did and any error (the last 100 per connection). The worker runs them (pnpm worker): a run that hits a rate limit or an outage is retried up to 3 times; a run whose credentials stopped working marks the connection Reconnect and stops scheduling it until someone reconnects. Each provider keeps a watermark (the newest change it has synced), so a run only reads what changed since the last good one.

AI assistants can read sync status with the MCP tools list_integration_syncs and get_integration_sync_history (mcp.md).

Salesforce

Connect: Connect with Salesforce (tick Sandbox for a sandbox org) and approve access. Guidely uses an OAuth connected app with the scopes api, refresh_token and offline_access; the access token is refreshed automatically. Calls go to the org's own API host (REST API v60.0).

Field mapping (per Guidely entity):

• Pull: Salesforce fields → traits (you name the trait, e.g. Industry → industry). Records changed since the last sync (SystemModstamp) are read; matching accounts / visitors get the values merged into their traits, so they can be used in audiences and guide targeting. Records with no match are counted, never created. Trait names that look sensitive (password, token…) are refused.
• Push: usage metrics → Salesforce fields (create the custom fields first; formula and read-only fields can't be picked). Accounts and visitors active in the last 30 days, plus any pushed before, are written with sObject Collections (200 per call, partial success allowed); values that didn't change since the last push aren't written again. The CRM record id is remembered after the first match.

Usage metrics (from the environment you pick, production by default):

The list is a registry (registerUsageMetric in apps/web/src/server/services/sync/metrics.ts): metrics added there (for example a Product Engagement Score) appear in the mapping UI and in pushes automatically.

Operator setup: in Salesforce Setup → App Manager → New Connected App: enable OAuth, callback URL ${APP_URL}/api/v1/sync/oauth/salesforce/callback, scopes Manage user data via APIs (api) and Perform requests at any time (refresh_token, offline_access), require secret for the web server flow. Put the consumer key and secret in SALESFORCE_CLIENT_ID / SALESFORCE_CLIENT_SECRET (on AWS: the salesforceOAuthSecretArn config). Connected apps from one org work for every customer org once installed or allowed by their admins.

HubSpot

Same shape as Salesforce: companies ↔ accounts, contacts ↔ visitors (match on e.g. a guidely_account_id company property, and email). Pull reads records changed since the last sync (hs_lastmodifieddate, CRM search API); push uses batch updates of 100. Datetime properties get epoch milliseconds.

Connect with Connect with HubSpot (OAuth; scopes crm.objects.companies.read/write, crm.objects.contacts.read/write, crm.schemas.companies.read, crm.schemas.contacts.read), or paste a private app access token with the same scopes (HubSpot → Settings → Integrations → Private apps), which works without any server setup.

Operator setup (for the OAuth button): create a public app in a HubSpot developer account with redirect URL ${APP_URL}/api/v1/sync/oauth/hubspot/callback and the scopes above; set HUBSPOT_CLIENT_ID / HUBSPOT_CLIENT_SECRET (on AWS: hubspotOAuthSecretArn).

Segment and other CDPs

Creating the Segment connection shows its inbound key (gly_cdp_…) once; New key replaces it.

Into Guidely (Guidely as a destination): send Segment-spec calls to

POST ${NEXT_PUBLIC_GUIDELY_API_URL}/api/v1/cdp/v1/batch      (also /v1/identify, /v1/track, /v1/page, /v1/screen, /v1/group, /v1/alias, or /api/v1/cdp)
Authorization: Basic base64("<inbound key>:")                (or: Bearer <inbound key>, ?key=, or "writeKey" in the body)

That is Segment's HTTP Tracking API, so Segment's Webhooks destination, RudderStack's webhook destination, any Segment library with its API host pointed here, and mParticle event batches (webhook output) all work. Bodies up to 512 KB (gzip allowed), 1,000 messages per request.

messageId deduplicates retries. Calls with only a userId reuse that visitor's known anonymous id. Data goes to the environment you choose (production by default) and counts towards the analytics allowance like SDK events. Property names that look sensitive are dropped, as for the SDK.

Out of Guidely (Guidely as a source): add the write key of a Segment HTTP API source, turn on Send Guidely events to Segment and pick the events (guide seen / completed / dismissed, checklist completed, survey answered, feature flag exposure, …). They're sent within about a minute as track calls named like Guide Completed, with guide_id, guide_key, guide_name, step_id and the event's properties, the user id, anonymous id and context.groupId (account). The base URL is SEGMENT_API_URL (https://events.eu1.segmentapis.com for EU workspaces).

Webhook (analytics event stream)

Integrations → Webhook streams the events you pick (any event name: guide_viewed = guide seen, guide_completed, survey_submitted = survey answered, experiment_viewed = feature flag exposure, …) from the environments you pick:

POST <your URL>
X-Guidely-Event: events
X-Guidely-Signature: sha256=<hex HMAC-SHA256 of the raw body with the signing secret>
X-Guidely-Delivery: <delivery id>

{ "type": "events", "projectId": "…", "webhookId": "…", "deliveryId": "…", "sentAt": "…",
  "events": [{ "id": "…", "name": "guide_completed", "customName": null, "occurredAt": "…", "receivedAt": "…",
               "environment": "production", "userId": "u_42", "anonymousId": "…", "accountId": "acme",
               "sessionId": "…", "guide": { "id": "…", "key": "welcome", "name": "Welcome tour", "version": 3 },
               "stepId": null, "stepIndex": null, "url": "…", "properties": {} }] }

Batches hold up to 100 events, in the order Guidely received them, about 15 seconds behind real time. A non-2xx answer or timeout (10 s) is retried 5 times (30 s, 1, 2, 4 minutes). The signing secret is shown once when the webhook is created; Deliveries → New signing secret replaces it. A new webhook starts from the moment it's created (no history). URLs must be public http(s) on ports 80/443 (requests are SSRF-checked).

Knowledge sources (Zendesk Guide, Intercom, Salesforce Knowledge)

Each knowledge source can, independently:

• Search live (on by default): Scout's article search and the help widget's search also query the source when someone asks, with up to 3 of Scout's 8 results kept for it. Results open on the source's site; Scout reads and cites them like its own articles. Lookups time out after 4 seconds (the source is then skipped) and are cached for 5 minutes.
• Copy articles into the resource center on a schedule: new and changed published articles are imported (into a collection you pick, published or as drafts); an article changed at the source gets a new published version. Imported articles are matched by source id (zendesk:<id>, intercom:<id>, sfkb:<id>), so settings changed in Guidely are kept and articles from the earlier one-time Intercom import are adopted.

New sources implement KnowledgeSource (search, get, list) in apps/web/src/server/services/sync/knowledge-sources.ts and register with knowledgeProvider.

Tickets in Zendesk and Intercom

Under workspace Settings → Issue trackers, Zendesk (subdomain, agent email, API token) and Intercom (access token with Read and write users and Read and write conversations) sit next to Linear and Jira. Create ticket in the Inbox, on a complaint, error or recording opens one there; Zendesk tickets have the person who asked as requester (by their email trait or the complaint's contact email), Intercom gets a conversation from that person (found by email, or created as a lead). With Open a ticket when Scout hands a conversation to the team on, every handoff in the workspace queues one automatically (created by the worker within seconds, retried on failure).

Data sync (S3, BigQuery, Snowflake)

Pick the datasets, the environment (production by default) and hourly or daily. Each run exports what changed since the last run, up to two minutes ago, as gzipped JSON Lines (.jsonl.gz, at most 20,000 rows per file), one folder per table and day: <table>/dt=YYYY-MM-DD/<run id>-<part>.jsonl.gz. The first run starts a day before the connection was made; use Backfill (a date range, up to 400 days per backfill) for older data. Backfills don't change what the next scheduled run exports. A failed run is retried with the same cut-off and file names, so files are replaced rather than duplicated.

Datasets

events / guidely_events — every product event (heatmap batches excluded); incremental on received_at, partitioned by occurred_at; key event_id.

guide_events / guidely_guide_events — guide, checklist and survey lifecycle events: the event columns without url/path/sdk, plus guide_key and guide_name.

visitors / guidely_visitors — id, user_id, anonymous_id, traits (JSON), account_ids (JSON array of your account ids), first_seen_at, last_seen_at, sessions_count, synced_at; incremental on last_seen_at; one row per export (BigQuery and S3: take the latest synced_at per id; Snowflake: upserted on id).

accounts / guidely_accounts — id, account_id, name, traits, created_at, updated_at, synced_at; incremental on updated_at; same as visitors.

survey_responses / guidely_survey_responses — response_id, guide_id, guide_key, guide_name, guide_version, visitor_id, user_id, account_id, environment, answers (JSON by question id), submitted_at.

pages / guidely_pages — page views per path per finished UTC day: date, path, views, visitors, sessions; key date + path.

Amazon S3

Bucket, key prefix (default guidely) and region. Cross-account role (recommended): create a role in your account allowing s3:PutObject on arn:aws:s3:::<bucket>/<prefix>/*, whose trust policy allows sts:AssumeRole for the principal shown in the dashboard (DATA_SYNC_AWS_PRINCIPAL_ARN, Guidely's AWS account) with the condition sts:ExternalId = the project's external id (guidely-<project id>, shown in the dashboard). Or use an access key pair of a user with that permission. Saving writes <prefix>/_guidely/connection-check.json to check access.

Google BigQuery

Google Cloud project id, dataset (must exist) and location, plus a service account key (JSON) with BigQuery Data Editor on the dataset and BigQuery Job User on the project. Tables guidely_<dataset> are created on the first load with the schema above, partitioned by day. Each file is staged in Guidely's bucket (exports/…, kept 7 days) and loaded with one load job whose id is derived from the file, so a retried run never loads a file twice.

Snowflake

Account identifier (orgname-account or xy12345.eu-west-1), user, warehouse, database, schema (created if missing) and optional role. Authentication is key-pair: generate a key (openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt), set the public key on the user (ALTER USER … SET RSA_PUBLIC_KEY='…') and paste the private key. Guidely uses the SQL API: tables are created on first use and each staged file is loaded with MERGE statements keyed on the dataset's key columns, so reloads never duplicate rows and visitors / accounts / pages are upserted.

Testing locally without vendor accounts

Every vendor base URL is configurable (SALESFORCE_LOGIN_URL, HUBSPOT_API_URL, SEGMENT_API_URL, ZENDESK_API_URL, INTERCOM_API_URL, BIGQUERY_API_URL, GOOGLE_TOKEN_URL, SNOWFLAKE_API_URL, DATA_SYNC_S3_ENDPOINT, DATA_SYNC_STS_ENDPOINT; see environment.md). The integration tests (apps/web/src/server/__tests__/sync-*.integration.test.ts) run each integration against a local mock server (src/test/mock-vendor.ts).