
Preparing Datacloud
Prepare Salesforce Data Cloud ingestion—connectors, schema upsert, and example sends—before streaming badge-scan or custom object data into a tenant.
Overview
Preparing-datacloud is an agent skill most often used in Build (also Operate) that walks through Data Cloud Ingestion API connector prep, schema upsert, and example record sends.
Install
npx skills add https://github.com/forcedotcom/sf-skills --skill preparing-datacloudWhat is this skill?
- Documents Ingestion API prerequisites: connector, schema-upsert via sf data360, and optional data stream UI step.
- Public-safe send-data.py example with env vars for consumer key, tenant URL, and connector/object names.
- Links to ingest-api-connection.json and ingest-api-schema.json in connecting-datacloud examples.
- Part of the forcedotcom *-datacloud skill family with shared CREDITS and UPSTREAM maintenance docs.
- Assumes External Client App credentials and private key file path for server-side ingest.
- Example documents 7 named environment variables for ingest (CONSUMER_KEY through OBJECT_NAME).
- References two example JSON artifacts: ingest-api-connection and ingest-api-schema.
Adoption & trust: 602 installs on skills.sh; 513 GitHub stars; 3/3 security scanners passed (skills.sh audits).
What problem does it solve?
You have a Data Cloud tenant and a source system but lack the ordered connector, schema, and credential setup required before ingestion scripts can run.
Who is it for?
Builders implementing Salesforce Data Cloud ingest for custom objects or badge-style event streams with CLI and Python.
Skip if: Generic warehouse ETL without Data Cloud, Marketing Cloud-only integrations, or teams without External Client App and tenant access.
When should I use this skill?
User is setting up Salesforce Data Cloud ingestion, Ingestion API connectors, schema upsert for a data stream, or running send-data examples after connect/prepare steps.
What do I get? / Deliverables
You complete connector and schema preparation aligned with connecting-datacloud examples and can run the documented send-data flow with the required environment variables.
- Configured ingestion connector and schema upsert checklist
- Runnable send-data.py example wired to env-based credentials
- Cross-links to connection and schema JSON examples
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Data Cloud prep happens in Build when you wire external systems and ingestion APIs into the product data model ahead of go-live. Integrations subphase matches connector creation, Ingestion API credentials, and schema alignment with upstream connect skills.
Where it fits
Create Badge_Scanner connector and upsert schema before enabling event ingest from a scanner app.
Validate tenant URL and connector names when cloning ingest setup to another org.
Align server.key and OAuth consumer credentials for a Python ingest worker.
How it compares
Preparation checklist for Data Cloud Ingestion API—not a full analytics stack design or non-Salesforce database migration guide.
Common Questions / FAQ
Who is preparing-datacloud for?
Solo developers and integrators wiring Salesforce Data Cloud ingestion who need step order for connectors, schema upsert, and a minimal Python send example.
When should I use preparing-datacloud?
Use it in Build → integrations while standing up ingest connectors and schemas; revisit in Operate → infra when extending streams or rotating ingestion credentials.
Is preparing-datacloud safe to install?
Examples use placeholder secrets and private key paths—never commit real credentials; review the Security Audits panel on this Prism page before running ingest scripts in production orgs.
Workflow Chain
Requires first: connecting datacloud
Then invoke: orchestrating datacloud
SKILL.md
READMESKILL.md - Preparing Datacloud
# Credits & Acknowledgments Primary contributor: **Gnanasekaran Thoppae** This skill is part of the `*-datacloud` family. Shared attribution, upstream source mapping, and maintenance notes live in: - [../orchestrating-datacloud/CREDITS.md](../orchestrating-datacloud/CREDITS.md) - [../orchestrating-datacloud/UPSTREAM.md](../orchestrating-datacloud/UPSTREAM.md) CONSUMER_KEY=<your consumer key from External Client App> CONSUMER_SECRET=<your consumer secret> SF_USERNAME=<your salesforce username> SF_LOGIN_URL=https://login.salesforce.com TENANT_URL=https://<tenant-id>.c360a.salesforce.com PRIVATE_KEY_FILE=/path/to/server.key CONNECTOR_NAME=Badge_Scanner OBJECT_NAME=Badge_Scan # Ingestion API example This folder contains a minimal, public-safe example for sending records into Salesforce Data Cloud through the Ingestion API. ## What this example assumes Before running `send-data.py`, complete the connect/prepare setup steps: 1. create an Ingestion API connector 2. upload the schema with `sf data360 connection schema-upsert` 3. create the corresponding data stream in the UI if your org requires that step Related connector definitions live in: - [../../../connecting-datacloud/examples/connections/ingest-api-connection.json](../../../connecting-datacloud/examples/connections/ingest-api-connection.json) - [../../../connecting-datacloud/examples/connections/ingest-api-schema.json](../../../connecting-datacloud/examples/connections/ingest-api-schema.json) ## Prerequisites ```bash pip install PyJWT cryptography requests ``` ## Setup ```bash cd skills/preparing-datacloud/examples/ingestion-api cp .env.example .env # edit .env with your values python3 send-data.py ``` ## Environment variables - `CONSUMER_KEY` — external client app consumer key - `CONSUMER_SECRET` — external client app consumer secret if your auth flow needs it - `SF_USERNAME` — Salesforce username used for JWT auth - `SF_LOGIN_URL` — login host such as `https://login.salesforce.com` - `TENANT_URL` — Data Cloud tenant URL such as `https://<tenant>.c360a.salesforce.com` - `PRIVATE_KEY_FILE` — path to the JWT private key - `CONNECTOR_NAME` — Ingestion API connector name - `OBJECT_NAME` — uploaded schema object name ## Notes - auth is a staged flow: JWT → Salesforce token → Data Cloud token - the ingestion endpoint uses the Data Cloud tenant URL, not the Salesforce instance URL - `202` means the payload was accepted for processing - validation failures often appear in the Problem Records DLO family #!/usr/bin/env python3 """ Send data to Data Cloud through the Ingestion API. Prerequisites: pip install PyJWT cryptography requests Usage: 1. Copy .env.example to .env and fill in your values 2. python3 send-data.py See README.md in this folder for setup notes. """ from __future__ import annotations import os import time import uuid from datetime import datetime, timezone from pathlib import Path import jwt import requests def load_env_file() -> None: env_file = Path(__file__).parent / ".env" if not env_file.exists(): return for line in env_file.read_text().splitlines(): if "=" in line and not line.startswith("#"): key, val = line.split("=", 1) os.environ.setdefault(key.strip(), val.strip()) load_env_file() CONSUMER_KEY = os.environ["CONSUMER_KEY"] SF_USERNAME = os.environ["SF_USERNAME"] SF_LOGIN_URL = os.environ.get("SF_LOGIN_URL", "https://login.salesforce.com") TENANT_URL = os.environ["TENANT_URL"] PRIVATE_KEY_FILE = os.environ["PRIVATE_KEY_FILE"] CONNECTOR_NAME = os.environ["CONNECTOR_NAME"] OBJECT_NAME = os.environ["OBJECT_NAME"] def get_cdp_token() -> str: """Authenticate: JWT -> Salesforce access token -> Data Cloud token.""" private_key = Path(PRIVATE_KEY_FILE).read_text() claim = { "iss": CONSUMER_KEY, "sub": SF_USERNAME, "aud": SF_LOGIN_URL, "exp": int(time.time()) + 300, } assertion = jwt.encode(claim, private_key, algorithm="RS2