
Sap Cloud Sdk Ai Python
- 205 installs
- 399 repo stars
- Updated August 4, 2026
- secondsky/sap-skills
Helps with python tasks.
About
sap-cloud-sdk-ai-python is a Claude Code skill for python. It helps solo builders move faster with AI-assisted development.
- sap-cloud-sdk-ai-python
- Python
- AI-coding skill
Sap Cloud Sdk Ai Python by the numbers
- 205 all-time installs (skills.sh)
- +30 installs in the week ending Jul 27, 2026 (Skillselion tracking)
- Ranked #62 of 290 Python skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/secondsky/sap-skills --skill sap-cloud-sdk-ai-pythonAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 205 |
|---|---|
| repo stars | ★ 399 |
| Last updated | August 4, 2026 |
| Repository | secondsky/sap-skills ↗ |
What it does
Helps with python tasks.
Files
SAP Cloud SDK for AI (Python)
Package rename: The PyPI package generative-ai-hub-sdk is deprecated (v4.12.4 is the last release).Its successor is `sap-ai-sdk-gen` (currently v6.10.0 per public PyPI registry evidence from 2026-06-15). Code and tutorials referencing
generative-ai-hub-sdkshould migrate tosap-ai-sdk-gen; the import name remainsgen_ai_hub.
The official Python SDK for SAP Generative AI Hub and Orchestration Service. It wraps the native SDKs of model providers (OpenAI, Amazon Bedrock, Google GenAI) and offers a harmonised LangChain integration and a full Orchestration client — all routed through SAP AI Core with unified authentication. Package freshness is registry-verified; AI Core runtime behavior and exact model availability still require target-tenant validation.
Related Skills
- sap-ai-core: Platform setup, deployments, resource groups, and model management in SAP AI Core
- sap-cloud-sdk-ai: JavaScript/TypeScript and Java equivalents of this SDK
- sap-hana-ml: HANA-side machine learning in Python
- sap-dependency-security: Pip dependency hygiene and upgrade patterns
Related external skills
If your task involves working inside Databricks (notebooks, Unity Catalog, Spark, SAP Databricks in SAP Business Data Cloud), consider installing the Databricks agent skills plugin. Ask whether you would like help installing it — never install unprompted.
When to Use This Skill
Use this skill when:
- Building Python applications that call LLMs through SAP AI Core / Generative AI Hub
- Using the
gen_ai_hubPython package (installed assap-ai-sdk-gen) - Integrating OpenAI, Amazon Bedrock, or Google GenAI models via SAP's proxy
- Implementing LangChain chains with SAP AI Core as the backend
- Using the Orchestration Service from Python (templating, filtering, masking, grounding)
- Migrating code from the deprecated
generative-ai-hub-sdktosap-ai-sdk-gen - Generating embeddings through SAP AI Core
- Working with SAP RPT-1 (Relational Pretrained Transformer) for tabular predictions
Table of Contents
- Quick Start
- Installation
- Authentication
- Available Modules
- Supported Models
- Core Features
- Bundled Resources
Quick Start
Native OpenAI Chat Completion
from gen_ai_hub.proxy.native.openai import chat
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is SAP BTP?"}
]
response = chat.completions.create(
model_name="gpt-4o-mini",
messages=messages
)
print(response.choices[0].message.content)Orchestration Service
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService,
ModuleConfig, PromptTemplatingModuleConfig,
Template, UserMessage, LLMModelDetails
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(
template=[UserMessage(role="user", content="{{?question}}")]
),
model=LLMModelDetails(name="gpt-4o-mini")
)
)
)
service = OrchestrationService(config=config)
response = service.run(placeholder_values={"question": "What is SAP?"})
print(response.final_result.choices[0].message.content)Installation
# All providers + LangChain support
pip install "sap-ai-sdk-gen[all]"
# Default (OpenAI only, no LangChain)
pip install sap-ai-sdk-gen
# Specific providers (without LangChain)
pip install "sap-ai-sdk-gen[google, amazon]"Authentication
The SDK reads credentials via AICoreV2Client.from_env(), which resolves credentials in this order:
1. Keyword arguments passed to GenAIHubProxyClient(...) 2. Environment variables — AICORE_CLIENT_ID, AICORE_CLIENT_SECRET, AICORE_AUTH_URL, AICORE_BASE_URL, AICORE_RESOURCE_GROUP 3. Config file — $AICORE_HOME/config.json (or path set by AICORE_CONFIG); use AICORE_PROFILE to select a named profile 4. VCAP_SERVICES — automatic on Cloud Foundry/Kyma when the AI Core service is bound
Local Development (Environment Variables)
export AICORE_CLIENT_ID="sb-..."
export AICORE_CLIENT_SECRET="..."
export AICORE_AUTH_URL="https://<tenant>.authentication.sap.hana.ondemand.com/oauth/token"
export AICORE_BASE_URL="https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com/v2"
export AICORE_RESOURCE_GROUP="default"Config File Profile
# ~/.aicore/config.json
{
"AICORE_CLIENT_ID": "sb-...",
"AICORE_CLIENT_SECRET": "...",
"AICORE_AUTH_URL": "https://<tenant>.authentication.sap.hana.ondemand.com/oauth/token",
"AICORE_BASE_URL": "https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com/v2",
"AICORE_RESOURCE_GROUP": "default"
}For detailed auth setup and troubleshooting, see references/getting-started-auth.md.
Available Modules
| Module | Import Path | Purpose |
|---|---|---|
| Proxy (native clients) | gen_ai_hub.proxy.native.* | Direct model access per provider |
| LangChain integration | gen_ai_hub.proxy.langchain | init_llm, init_embedding_model, ChatOpenAI, etc. |
| Orchestration | gen_ai_hub.orchestration_v2 | Templating, filtering, masking, grounding |
| Document Grounding | gen_ai_hub.document_grounding | Pipeline, Vector, Retrieval APIs |
| Prompt Registry | gen_ai_hub.prompt_registry | Template management and config storage |
| Evaluations | gen_ai_hub.evaluations | Model evaluation runs and metrics |
| SAP RPT-1 | gen_ai_hub.proxy.native.sap | Tabular prediction (classification, regression) |
Native Clients by Provider
| Provider | Import | Key Classes |
|---|---|---|
| OpenAI | gen_ai_hub.proxy.native.openai | OpenAI, completions, chat, embeddings, responses |
| Amazon Bedrock | gen_ai_hub.proxy.native.amazon | Session, ClientWrapper |
| Google GenAI | gen_ai_hub.proxy.native.google_genai | Client |
| SAP RPT-1 | gen_ai_hub.proxy.native.sap | RPTClient, RPTRequest |
Supported Models
The Generative AI Hub catalog includes models from multiple providers. Check SAP's model catalog and the target tenant catalog for the authoritative model IDs. Example families:
| Provider | Example Families |
|---|---|
| OpenAI | GPT-family chat, multimodal, reasoning, and embedding models |
| Anthropic (via Bedrock) | Claude-family models |
| Amazon | Nova/Titan-family models |
| Gemini-family models | |
| Mistral | Mistral-family models |
| SAP | RPT-family tabular prediction models where enabled |
Core Features
Chat Completion with OpenAI Client
from gen_ai_hub.proxy.native.openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain CAP in one paragraph."}]
)
print(response.choices[0].message.content)Streaming
from gen_ai_hub.proxy.native.openai import OpenAI
client = OpenAI()
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain SAP CAP."}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")Embeddings
from gen_ai_hub.proxy.native.openai import embeddings
response = embeddings.create(
input="Every decoding is another encoding.",
model_name="text-embedding-3-small"
)
print(response.data[0].embedding)LangChain Integration
from gen_ai_hub.proxy.langchain import init_llm, init_embedding_model
llm = init_llm("gpt-4o-mini", max_tokens=300)
result = llm.invoke("What is SAP BTP?")
print(result.content)
embeddings = init_embedding_model("text-embedding-3-small")
vector = embeddings.embed_query("SAP Business Technology Platform")Content Filtering (via Orchestration)
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService,
ModuleConfig, PromptTemplatingModuleConfig,
Template, UserMessage, LLMModelDetails,
FilteringModuleConfig, InputFiltering, OutputFiltering,
AzureContentSafetyInput, AzureContentSafetyOutput, AzureThreshold
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(template=[UserMessage(role="user", content="{{?question}}")]),
model=LLMModelDetails(name="gpt-4o-mini")
),
filtering=FilteringModuleConfig(
input=InputFiltering(filters=[
AzureContentSafetyInput(hate=AzureThreshold.ALLOW_SAFE, violence=AzureThreshold.ALLOW_SAFE)
]),
output=OutputFiltering(filters=[
AzureContentSafetyOutput(hate=AzureThreshold.ALLOW_SAFE, violence=AzureThreshold.ALLOW_SAFE)
])
)
)
)
service = OrchestrationService(config=config)
response = service.run(placeholder_values={"question": "Explain SAP."})Data Masking (via Orchestration)
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService,
ModuleConfig, PromptTemplatingModuleConfig,
Template, UserMessage, LLMModelDetails,
MaskingModuleConfig, MaskingProviderConfig,
DPIStandardEntity, MaskingMethod, DataMaskingProviderName
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(template=[UserMessage(role="user", content="{{?text}}")]),
model=LLMModelDetails(name="gpt-4o-mini")
),
masking=MaskingModuleConfig(
masking_providers=[
MaskingProviderConfig(
type=DataMaskingProviderName.SAP_DATA_PRIVACY_INTEGRATION,
method=MaskingMethod.ANONYMIZATION,
entities=[
DPIStandardEntity(type="profile-email"),
DPIStandardEntity(type="profile-person")
]
)
]
)
)
)
service = OrchestrationService(config=config)
response = service.run(placeholder_values={"text": "Contact john@example.com for details."})Document Grounding (via Orchestration)
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService,
ModuleConfig, PromptTemplatingModuleConfig,
Template, UserMessage, LLMModelDetails,
GroundingModuleConfig, DocumentGroundingConfig,
DocumentGroundingFilter, DocumentGroundingPlaceholders,
GroundingSearchConfig, DataRepositoryType, GroundingType
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(template=[UserMessage(role="user", content="{{?question}}")]),
model=LLMModelDetails(name="gpt-4o-mini")
),
grounding=GroundingModuleConfig(
type=GroundingType.DOCUMENT_GROUNDING_SERVICE,
config=DocumentGroundingConfig(
placeholders=DocumentGroundingPlaceholders(
input=["{{?question}}"],
output="{{?context}}"
),
filters=[
DocumentGroundingFilter(
id="my-vector-repo-id",
data_repository_type=DataRepositoryType.VECTOR,
search_config=GroundingSearchConfig(max_chunk_count=5)
)
]
)
)
)
)
service = OrchestrationService(config=config)
response = service.run(placeholder_values={"question": "What is the refund policy?"})Common Errors
| Error | Cause | Solution |
|---|---|---|
No credentials found in any source | Missing AI Core service key/env vars | Set all AICORE_* environment variables or create a config file profile |
No deployment found | Model not deployed in AI Core | Deploy the model in your resource group, or use deployment_id directly |
AICORE_RESOURCE_GROUP not set | Missing resource group | Set AICORE_RESOURCE_GROUP env var or pass resource_group to the client |
ModuleNotFoundError: No module named 'gen_ai_hub' | Wrong package installed | Install sap-ai-sdk-gen (not generative-ai-hub-sdk) |
Import from generative_ai_hub_sdk fails | Using deprecated package name | The package was renamed; import from gen_ai_hub (installed via sap-ai-sdk-gen) |
ValidationError on proxy client init | Incomplete credentials | Verify all four required env vars: AICORE_CLIENT_ID, AICORE_CLIENT_SECRET, AICORE_AUTH_URL, AICORE_BASE_URL |
Bundled Resources
Reference Documentation
1. references/getting-started-auth.md - Installation, authentication, and config setup 2. references/native-clients-guide.md - Native client usage for OpenAI, Amazon, Google, and SAP RPT-1 3. references/orchestration-guide.md - Orchestration service: templating, filtering, masking, grounding, embeddings 4. references/langchain-guide.md - LangChain integration: LLM/embedding init, chains, structured outputs 5. references/troubleshooting.md - Common errors, version compatibility, migration from generative-ai-hub-sdk
Documentation Sources
Keep this skill updated using these sources:
- PyPI: https://pypi.org/pypi/sap-ai-sdk-gen/json — package metadata and README
- SDK Reference: https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/gen_ai_hub.html
- SAP Samples: https://github.com/SAP-samples/btp-gen-ai-hub-sdk-samples
- AI Core Help: https://help.sap.com/docs/sap-ai-core
- Deprecated Package: https://pypi.org/pypi/generative-ai-hub-sdk/json (for migration notes)
SAP Cloud SDK for AI (Python)
Python SDK for SAP Generative AI Hub and Orchestration Service. Package sap-ai-sdk-gen (formerly generative-ai-hub-sdk, now deprecated). Provides native client integrations for OpenAI, Amazon Bedrock, Google GenAI, LangChain support, and full Orchestration Service access including content filtering, data masking, and document grounding. Package freshness is registry-verified; AI Core tenant execution and exact model availability still require target-tenant validation.
Capability Index
| Capability | Status |
|---|---|
| Commands | 1: /cloud-sdk-ai-python-chat-template |
| Agents | 0 |
| Hooks | No |
| MCP | No |
| LSP | No |
| Source Freshness | last_verified: 2026-06-15; package registry evidence captured in docs/project/package-evidence/2026-06-15.json. |
| Verification | npm run validate; AI Core tenant execution and exact model availability remain pending. |
Quick Start
pip install "sap-ai-sdk-gen[all]"export AICORE_CLIENT_ID="sb-..."
export AICORE_CLIENT_SECRET="..."
export AICORE_AUTH_URL="https://<tenant>.authentication.sap.hana.ondemand.com/oauth/token"
export AICORE_BASE_URL="https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com/v2"
export AICORE_RESOURCE_GROUP="default"from gen_ai_hub.proxy.native.openai import chat
response = chat.completions.create(
model_name="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello, SAP!"}]
)
print(response.choices[0].message.content)Installation Variants
| Command | Includes |
|---|---|
pip install sap-ai-sdk-gen | OpenAI only |
pip install "sap-ai-sdk-gen[google, amazon]" | OpenAI + Google + Amazon (no LangChain) |
pip install "sap-ai-sdk-gen[all]" | All providers + LangChain |
Key Modules
- `gen_ai_hub.proxy.native` — Direct model access (OpenAI, Amazon, Google, SAP RPT-1)
- `gen_ai_hub.proxy.langchain` — LangChain integration (
init_llm,ChatOpenAI, etc.) - `gen_ai_hub.orchestration_v2` — Orchestration Service client
- `gen_ai_hub.document_grounding` — Vector and Retrieval APIs
- `gen_ai_hub.prompt_registry` — Prompt template management
- `gen_ai_hub.evaluations` — Model evaluation framework
License
GPL-3.0
Getting Started and Authentication
Installation
The SAP Cloud SDK for AI for Python is distributed on PyPI as sap-ai-sdk-gen.
# Full installation (all providers + LangChain)
pip install "sap-ai-sdk-gen[all]"
# Minimal installation (OpenAI provider only, no LangChain)
pip install sap-ai-sdk-gen
# Select specific providers without LangChain
pip install "sap-ai-sdk-gen[google, amazon]"Package Rename
The previous package generative-ai-hub-sdk (v4.12.4) is deprecated and no longer maintained. Its PyPI page states to use sap-ai-sdk-gen instead. The Python import name is still gen_ai_hub:
import gen_ai_hub # installed via: pip install sap-ai-sdk-genIf you have existing code importing from generative_ai_hub_sdk, update your dependencies:
# Remove old package
pip uninstall generative-ai-hub-sdk
# Install new package
pip install "sap-ai-sdk-gen[all]"The import paths under gen_ai_hub remain the same between the old and new packages.
Python Version
The SDK requires Python 3.9+.
Prerequisites
- SAP AI Core service instance on SAP BTP (Extended or
sap-internalplan) - Resource group in AI Core with deployed models (or use the default resource group)
- Orchestration deployment (required only for orchestration features)
Authentication
The SDK authenticates through AICoreV2Client from the ai-core-sdk package. Credentials are resolved in this precedence order:
1. Keyword arguments passed directly to GenAIHubProxyClient(...) 2. Environment variables 3. Config file ($AICORE_HOME/config.json or path from AICORE_CONFIG) 4. VCAP_SERVICES (automatic on Cloud Foundry/Kyma)
Environment Variables
Set these for local development:
export AICORE_CLIENT_ID="sb-abc123-..."
export AICORE_CLIENT_SECRET="abc123-..."
export AICORE_AUTH_URL="https://<subdomain>.authentication.<region>.hana.ondemand.com/oauth/token"
export AICORE_BASE_URL="https://api.ai.prod.<region>.aws.ml.hana.ondemand.com/v2"
export AICORE_RESOURCE_GROUP="default"The four required credentials (AICORE_CLIENT_ID, AICORE_CLIENT_SECRET, AICORE_AUTH_URL, AICORE_BASE_URL) are the service key values from your AI Core service instance on SAP BTP. You can retrieve them from the SAP BTP cockpit or via the CF CLI:
cf service-key <AICORE_INSTANCE> <KEY_NAME>Config File Profile
Create a config file at ~/.aicore/config.json (default location; AICORE_HOME controls the directory, AICORE_CONFIG can set an explicit file path):
{
"AICORE_CLIENT_ID": "sb-abc123-...",
"AICORE_CLIENT_SECRET": "abc123-...",
"AICORE_AUTH_URL": "https://<subdomain>.authentication.<region>.hana.ondemand.com/oauth/token",
"AICORE_BASE_URL": "https://api.ai.prod.<region>.aws.ml.hana.ondemand.com/v2",
"AICORE_RESOURCE_GROUP": "default"
}For named profiles, create ~/.aicore/config_<profile>.json and select it:
export AICORE_PROFILE="production"Cloud Foundry / Kyma Binding
On BTP runtimes, bind the AI Core service instance to your application. The SDK auto-detects credentials from VCAP_SERVICES (Cloud Foundry) or mounted secrets (Kyma). No manual env var setup is needed.
X.509 Certificate Authentication
For environments requiring certificate-based auth, provide certificate paths or strings as keyword arguments to AICoreV2Client:
from ai_core_sdk.ai_core_v2_client import AICoreV2Client
client = AICoreV2Client(
base_url="https://api.ai.prod.<region>.aws.ml.hana.ondemand.com/v2",
cert_file_path="/path/to/client.crt",
key_file_path="/path/to/client.key"
)Or via environment variables by setting AICORE_CERT_FILE_PATH and AICORE_KEY_FILE_PATH in your config profile.
Resource Groups
Resource groups isolate AI Core resources (deployments, models, configurations). Set the default resource group via AICORE_RESOURCE_GROUP or pass it explicitly:
from gen_ai_hub.proxy import GenAIHubProxyClient
proxy_client = GenAIHubProxyClient(resource_group="my-project")Verifying the Setup
Test that credentials are configured correctly:
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
print(f"Proxy client created successfully: {proxy_client.base_url}")If this raises ValidationError: No credentials found in any source, review your environment variables or config file.
Installing Extras
The SDK supports optional dependency groups:
| Extra | Includes |
|---|---|
[all] | All providers + LangChain |
[google] | Google GenAI provider |
[amazon] | Amazon Bedrock provider |
LangChain dependencies are included with [all] but not with individual provider extras.
Resources
- PyPI: https://pypi.org/project/sap-ai-sdk-gen/
- SDK Reference: https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/gen_ai_hub.html
- AI Core Setup: https://help.sap.com/docs/sap-ai-core
LangChain Integration Guide
The SDK provides harmonised LangChain integration through gen_ai_hub.proxy.langchain, enabling LangChain chains and agents to use models via SAP AI Core.
Module Structure
gen_ai_hub.proxy.langchain/
├── __init__.py # Exports: init_llm, init_embedding_model, ChatOpenAI, OpenAI, ...
├── openai/ # OpenAI LangChain classes
├── amazon/ # Amazon Bedrock LangChain classes
├── google_genai/ # Google GenAI LangChain classes
└── base/ # Shared base classesKey Exports
| Class / Function | Description |
|---|---|
init_llm | Harmonised LLM initialisation (any provider) |
init_embedding_model | Harmonised embedding model initialisation |
ChatOpenAI | LangChain ChatOpenAI via AI Core proxy |
OpenAI | LangChain OpenAI (completions) via AI Core proxy |
OpenAIEmbeddings | LangChain OpenAI embeddings via AI Core proxy |
ChatBedrock | LangChain Bedrock (Invoke API) via AI Core proxy |
ChatBedrockConverse | LangChain Bedrock (Converse API) via AI Core proxy |
ChatGoogleGenerativeAI | LangChain Google GenAI via AI Core proxy |
OpenAIClient / AsyncOpenAIClient | Synchronous/async OpenAI clients for LangChain |
Harmonised Model Initialisation
init_llm
The init_llm function is the recommended way to create LangChain LLM objects. It automatically selects the correct provider class based on the model name:
from gen_ai_hub.proxy.langchain import init_llm
llm = init_llm("gpt-4o-mini", max_tokens=300, temperature=0.0)
result = llm.invoke("What is SAP BTP?")
print(result.content)Signature:
init_llm(
model_name, # Positional: model name (e.g. "gpt-4o-mini")
/,
*, # All following are keyword-only
proxy_client=None, # Optional: custom GenAIHubProxyClient
temperature=0.0, # Generation temperature
max_tokens=256, # Maximum output tokens
top_k=None, # Top-K sampling
top_p=1.0, # Top-P (nucleus) sampling
init_func=None, # Optional: override provider init function
model_id="", # Optional: explicit model ID (for Bedrock)
**kwargs # Passed to underlying LangChain class
) -> BaseLanguageModelinit_embedding_model
from gen_ai_hub.proxy.langchain import init_embedding_model
embeddings = init_embedding_model("text-embedding-3-small")
vector = embeddings.embed_query("SAP Business Technology Platform")
print(len(vector))Signature:
init_embedding_model(
model_name, # Positional: model name (e.g. "text-embedding-3-small")
/,
*, # All following are keyword-only
proxy_client=None, # Optional: custom GenAIHubProxyClient
init_func=None, # Optional: override provider init function
model_id="", # Optional: explicit model ID
**kwargs # Passed to underlying LangChain class
) -> EmbeddingsLCEL Chains (LangChain Expression Language)
Simple Chain
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from gen_ai_hub.proxy.langchain import init_llm
template = """Question: {question}
Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm = init_llm("gpt-4o-mini", max_tokens=300)
chain = prompt | llm | StrOutputParser()
response = chain.invoke({"question": "What is a supernova?"})
print(response)Chat Chain
from langchain.prompts.chat import (
ChatPromptTemplate,
SystemMessagePromptTemplate,
HumanMessagePromptTemplate,
)
from gen_ai_hub.proxy.langchain import ChatOpenAI
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
chat_llm = ChatOpenAI(proxy_model_name="gpt-4o-mini", proxy_client=proxy_client)
chat_prompt = ChatPromptTemplate.from_messages([
SystemMessagePromptTemplate.from_template("You are a helpful assistant that translates English to pirate."),
HumanMessagePromptTemplate.from_template("{text}")
])
chain = chat_prompt | chat_llm
response = chain.invoke({"text": "I love programming."})
print(response.content)Structured Outputs
from pydantic import BaseModel
from gen_ai_hub.proxy.langchain import ChatOpenAI
from gen_ai_hub.proxy import get_proxy_client
from langchain.schema import HumanMessage
class Person(BaseModel):
name: str
age: int
chat_model = ChatOpenAI(proxy_model_name="gpt-4o-mini", proxy_client=get_proxy_client())
chat_model = chat_model.with_structured_output(method="json_schema", schema=Person, strict=True)
message = HumanMessage(content="Tell me about a person named John who is 30")
result = chat_model.invoke([message])
print(result) # Person(name="John", age=30)Provider-Specific Classes
OpenAI
from gen_ai_hub.proxy.langchain import ChatOpenAI, OpenAI, OpenAIEmbeddings
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
# Chat
chat = ChatOpenAI(proxy_model_name="gpt-4o-mini", proxy_client=proxy_client)
# Completions (legacy)
llm = OpenAI(proxy_model_name="gpt-4o-mini", proxy_client=proxy_client)
# Embeddings
embeddings = OpenAIEmbeddings(proxy_model_name="text-embedding-3-small", proxy_client=proxy_client)Amazon Bedrock
from gen_ai_hub.proxy.langchain import ChatBedrock, ChatBedrockConverse, BedrockEmbeddings
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
# Invoke API
chat_invoke = ChatBedrock(
proxy_model_name="amazon--nova-pro",
model_id="amazon.nova-pro-v1:0",
proxy_client=proxy_client
)
# Converse API (recommended for newer models)
chat_converse = ChatBedrockConverse(
proxy_model_name="anthropic--claude-4-sonnet",
model_id="anthropic.claude-sonnet-4-20250514-v1:0",
proxy_client=proxy_client
)
# Embeddings
embeddings = BedrockEmbeddings(
proxy_model_name="amazon--nova-premier",
proxy_client=proxy_client
)Google GenAI
from gen_ai_hub.proxy.langchain import ChatGoogleGenerativeAI, GoogleGenerativeAIEmbeddings
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
# Chat
chat = ChatGoogleGenerativeAI(
proxy_model_name="gemini-2.5-flash",
proxy_client=proxy_client
)
# Embeddings
embeddings = GoogleGenerativeAIEmbeddings(
proxy_model_name="text-embedding-004",
proxy_client=proxy_client
)Using New Models Before Official Support
For models not yet auto-detected by init_llm, pass the init_func parameter:
from gen_ai_hub.proxy.langchain import init_llm
from gen_ai_hub.proxy.langchain.amazon import (
init_chat_model as amazon_init_invoke,
init_chat_converse_model as amazon_init_converse
)
from gen_ai_hub.proxy.langchain.google_genai import init_chat_model as google_init
# New Bedrock model with Converse API
llm = init_llm(
"anthropic--claude-newer-model",
model_id="anthropic.claude-newer-v1:0",
init_func=amazon_init_converse
)
# New Google model
llm = init_llm("gemini-newer-version", init_func=google_init)Implicit Proxy Client
If you don't pass proxy_client, the SDK creates one from the environment:
from gen_ai_hub.proxy.langchain import ChatOpenAI
# Works without explicit proxy_client if AICORE_* env vars are set
chat = ChatOpenAI(proxy_model_name="gpt-4o-mini")
response = chat.invoke("Hello!")Resources
- SDK Reference: https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/gen_ai_hub.html
- LangChain Docs: https://python.langchain.com/docs/
- PyPI: https://pypi.org/project/sap-ai-sdk-gen/
Native Clients Guide
The SDK provides provider-specific native client integrations that act as drop-in replacements for the original provider SDKs. All requests are routed through SAP AI Core's proxy, adding authentication, logging, and resource group isolation transparently.
Module Structure
gen_ai_hub.proxy.native/
├── openai/ # OpenAI-compatible (GPT, embeddings, responses API)
├── amazon/ # Amazon Bedrock (invoke model, converse)
├── google_genai/ # Google GenAI (generate content)
└── sap/ # SAP RPT-1 (tabular predictions)OpenAI Client
The OpenAI integration provides the most comprehensive coverage: completions, chat completions, embeddings, structured outputs, and the Responses API.
Module-Level Convenience Functions
These use a global client instance and are the simplest way to get started:
from gen_ai_hub.proxy.native.openai import chat, completions, embeddings
# Chat completion
response = chat.completions.create(
model_name="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is SAP?"}
]
)
print(response.choices[0].message.content)
# Legacy completions
response = completions.create(
model_name="gpt-4o-mini",
prompt="The capital of France is",
max_tokens=20,
temperature=0
)
# Embeddings
response = embeddings.create(
input="Every decoding is another encoding.",
model_name="text-embedding-3-small"
)OpenAI Client Instance
For more control, instantiate OpenAI directly:
from gen_ai_hub.proxy.native.openai import OpenAI
client = OpenAI()
# Chat completion
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}]
)
# With explicit deployment_id instead of model name
response = client.chat.completions.create(
deployment_id="dcef02e219ae4916",
messages=[{"role": "user", "content": "Hello!"}]
)The OpenAI constructor accepts:
OpenAI(
proxy_client=None, # Optional: custom GenAIHubProxyClient
api_version="2025-03-01-preview", # OpenAI API version
**kwargs # Passed to underlying openai.OpenAI
)Streaming
from gen_ai_hub.proxy.native.openai import OpenAI
client = OpenAI()
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain SAP CAP."}],
stream=True
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")Responses API
The SDK supports OpenAI's Responses API (agentic-style calls):
Replace the example model with a model ID returned by the target SAP AI Core tenant catalog.
from gen_ai_hub.proxy.native.openai import responses
response = responses.create(
model="gpt-4o-mini",
instructions="You are a helpful assistant.",
input="What is the capital of France?"
)
print(response.output_text)Structured Outputs
from pydantic import BaseModel
from gen_ai_hub.proxy.native.openai import chat, responses
class Person(BaseModel):
name: str
age: int
# Via chat completions
response = chat.completions.parse(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Tell me about John Doe, aged 30."}],
response_format=Person
)
person = response.choices[0].message.parsed
print(person)
# Via responses API
response = responses.parse(
model="gpt-4o-mini",
input="Tell me about John Doe aged 30.",
text_format=Person
)
print(response.output_parsed)Model Selection
Use model_name for convenience functions (auto-discovers deployment) or model for the client instance. You can also use model_version="latest":
response = chat.completions.create(
model_name="gpt-4o-mini",
model_version="latest",
messages=[{"role": "user", "content": "Hello!"}]
)Using a Specific Deployment
When you know the deployment ID (e.g., from AI Launchpad), skip model discovery:
response = chat.completions.create(
deployment_id="dcef02e219ae4916",
messages=[{"role": "user", "content": "Hello!"}]
)Amazon Bedrock Client
Invoke Model (Raw)
import json
from gen_ai_hub.proxy.native.amazon import Session
bedrock = Session().client(model_name="amazon--nova-premier")
body = json.dumps({
"inputText": "Explain black holes in astrophysics.",
"textGenerationConfig": {
"maxTokenCount": 3072,
"temperature": 0.7,
"topP": 0.9
}
})
response = bedrock.invoke_model(body=body)
response_body = json.loads(response.get("body").read())
print(response_body)Converse (High-Level)
from gen_ai_hub.proxy.native.amazon import Session
bedrock = Session().client(model_name="anthropic--claude-4-sonnet")
conversation = [
{"role": "user", "content": [{"text": "Describe the purpose of a hello world program."}]}
]
response = bedrock.converse(
messages=conversation,
inferenceConfig={"maxTokens": 512, "temperature": 0.5, "topP": 0.9}
)
print(response)Session and Client Options
from gen_ai_hub.proxy.native.amazon import Session
# The Session creates a client for a specific model
session = Session()
client = session.client(model_name="amazon--nova-pro")
# With explicit deployment
client = session.client(deployment_id="abc123")Google GenAI Client
Generate Content
from gen_ai_hub.proxy.native.google_genai import Client
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
client = Client(proxy_client=proxy_client)
response = client.models.generate_content(
model="gemini-2.5-flash",
contents="How many paws does a dog have?"
)
print(response)Streaming
from gen_ai_hub.proxy.native.google_genai import Client
from gen_ai_hub.proxy import get_proxy_client
proxy_client = get_proxy_client()
client = Client(proxy_client=proxy_client)
response_stream = client.models.generate_content_stream(
model="gemini-2.5-flash",
contents="Explain quantum computing in simple terms."
)
for chunk in response_stream:
print("Chunk:", chunk.text)Function Calling
from google.genai import types
from gen_ai_hub.proxy.native.google_genai import Client
from gen_ai_hub.proxy import get_proxy_client
def get_current_weather(location: str) -> str:
"""Returns the current weather."""
return "sunny"
proxy_client = get_proxy_client()
client = Client(proxy_client=proxy_client)
response = client.models.generate_content(
model="gemini-2.5-flash",
contents="What is the weather like in Boston?",
config=types.GenerateContentConfig(tools=[get_current_weather])
)SAP RPT-1 (Relational Pretrained Transformer)
SAP RPT-1 performs classification and regression on tabular data without training or fine-tuning. It uses in-context learning.
Regression Example
from gen_ai_hub.proxy.native.sap import (
RPTClient, RPTRequest, PredictionConfig, TargetColumn
)
rows = [
{"PRODUCT": "Couch", "PRICE": 999.99, "ORDERDATE": "28-11-2025", "ID": "35", "DISCOUNT_RATE": "[PREDICT]"},
{"PRODUCT": "Office Chair", "PRICE": 150.80, "ORDERDATE": "02-11-2025", "ID": "44", "DISCOUNT_RATE": 0.12},
{"PRODUCT": "Server Rack", "PRICE": 2200.00, "ORDERDATE": "01-11-2025", "ID": "104", "DISCOUNT_RATE": 0.05},
{"PRODUCT": "Standing Desk", "PRICE": 640.00, "ORDERDATE": "05-11-2025", "ID": "205", "DISCOUNT_RATE": 0.10},
{"PRODUCT": "Monitor 27 inch", "PRICE": 289.99, "ORDERDATE": "08-11-2025", "ID": "306", "DISCOUNT_RATE": "[PREDICT]"},
]
client = RPTClient()
body = RPTRequest(
prediction_config=PredictionConfig(
target_columns=[TargetColumn(name="DISCOUNT_RATE", task_type="regression")]
),
rows=rows
)
response = client.predict(body=body, model_name="sap-rpt-1-small")
print(response.predictions)Classification Example
from gen_ai_hub.proxy.native.sap import RPTClient
request_dict = {
"prediction_config": {
"target_columns": [
{"name": "COSTCENTER", "prediction_placeholder": "[PREDICT]", "task_type": "classification"}
]
},
"columns": {
"PRODUCT": ["Couch", "Office Chair", "Server Rack"],
"PRICE": [999.99, 150.8, 2200.00],
"ORDERDATE": ["28-11-2025", "02-11-2025", "01-11-2025"],
"ID": ["35", "44", "104"],
"COSTCENTER": ["[PREDICT]", "Office Furniture", "Data Infrastructure"]
},
"data_schema": {
"PRODUCT": {"dtype": "string"},
"PRICE": {"dtype": "numeric"},
"ORDERDATE": {"dtype": "date"},
"ID": {"dtype": "string"},
"COSTCENTER": {"dtype": "string"}
}
}
client = RPTClient()
response = client.predict(body=request_dict, model_name="sap-rpt-1-small")
print(response.predictions)Async RPT-1
response = await client.apredict(body=request_dict, model_name="sap-rpt-1-small")Using New Models Before Official SDK Support
You can use new models via Gen AI Hub before they are officially listed, provided their provider family is supported:
1. Native SDK Clients: Use the provider's native SDK through the proxy with the new model name directly. 2. LangChain: Pass init_func to init_llm to select the correct provider:
from gen_ai_hub.proxy.langchain import init_llm
from gen_ai_hub.proxy.langchain.amazon import (
init_chat_model as amazon_init_invoke,
init_chat_converse_model as amazon_init_converse
)
# Force Converse API for a new Bedrock model
llm = init_llm(
"anthropic--claude-newer-version",
model_id="anthropic.claude-newer-version-v1:0",
init_func=amazon_init_converse
)Proxy Version Management
The SDK supports multiple proxy versions. Use set_proxy_version to switch:
from gen_ai_hub.proxy import set_proxy_version, get_proxy_version
set_proxy_version("gen-ai-hub")
print(get_proxy_version()) # "gen-ai-hub"Resources
- SDK Reference: https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/gen_ai_hub.html
- PyPI: https://pypi.org/project/sap-ai-sdk-gen/
- AI Core Models: https://help.sap.com/docs/sap-ai-core/generative-ai-hub/available-models
Orchestration Service Guide
The Orchestration Service provides a unified API for combining LLM calls with pre/post-processing modules: prompt templating, content filtering, data masking, document grounding, and translation. This guide covers the gen_ai_hub.orchestration_v2 module (the current module; gen_ai_hub.orchestration is the legacy v1).
Module Structure
gen_ai_hub.orchestration_v2/
├── OrchestrationConfig # Pipeline configuration
├── OrchestrationService # Client for running completions
├── models/
│ ├── config/ # ModuleConfig, PromptTemplating, Filtering, Masking, Grounding
│ ├── message/ # SystemMessage, UserMessage, AssistantMessage, ToolChatMessage
│ ├── response/ # CompletionPostResponse, StreamCompletionPostResponse
│ ├── embeddings/ # EmbeddingsOrchestrationConfig
│ ├── template/ # Template, TemplateRef
│ └── ...
└── exceptions/Core Concepts
OrchestrationConfig
The OrchestrationConfig defines the pipeline. It accepts either a single ModuleConfig or a list (for fallback configurations):
from gen_ai_hub.orchestration_v2 import OrchestrationConfig, ModuleConfig
config = OrchestrationConfig(
modules=ModuleConfig(...) # or [ModuleConfig(...), ModuleConfig(...)]
)ModuleConfig
Each ModuleConfig can include:
| Field | Type | Purpose |
|---|---|---|
prompt_templating | PromptTemplatingModuleConfig | Required: template + model config |
filtering | FilteringModuleConfig | Optional: content safety filters |
masking | MaskingModuleConfig | Optional: PII/data masking |
grounding | GroundingModuleConfig | Optional: document grounding (RAG) |
translation | TranslationModuleConfig | Optional: input/output translation |
Quick Start
Simple Chat Completion
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService,
ModuleConfig, PromptTemplatingModuleConfig,
Template, UserMessage, LLMModelDetails
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(
template=[UserMessage(role="user", content="{{?question}}")]
),
model=LLMModelDetails(name="gpt-4o-mini")
)
)
)
service = OrchestrationService(config=config)
response = service.run(placeholder_values={"question": "What is SAP?"})
print(response.final_result.choices[0].message.content)With Conversation History
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService,
ModuleConfig, PromptTemplatingModuleConfig,
Template, UserMessage, AssistantMessage, LLMModelDetails
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(
template=[UserMessage(role="user", content="{{?question}}")]
),
model=LLMModelDetails(name="gpt-4o-mini")
)
)
)
service = OrchestrationService(config=config)
history = [
UserMessage(role="user", content="What is SAP BTP?"),
AssistantMessage(role="assistant", content="SAP BTP is a platform..."),
]
response = service.run(
placeholder_values={"question": "What services does it offer?"},
history=history
)Streaming
service = OrchestrationService(config=config)
for chunk in service.stream(placeholder_values={"question": "Explain SAP CAP."}):
if chunk.final_result and chunk.final_result.choices:
delta = chunk.final_result.choices[0].delta
if delta and delta.content:
print(delta.content, end="")Async Execution
response = await service.arun(placeholder_values={"question": "What is SAP?"})
async for chunk in service.astream(placeholder_values={"question": "Explain SAP CAP."}):
print(chunk)Prompt Templating
Template Messages
Templates use {{?variable}} placeholders in message content:
from gen_ai_hub.orchestration_v2 import Template, SystemMessage, UserMessage
template = Template(
template=[
SystemMessage(role="system", content="You are a {{?role}}."),
UserMessage(role="user", content="{{?question}}")
],
defaults={"role": "helpful assistant"} # Optional defaults for placeholders
)Model Configuration
from gen_ai_hub.orchestration_v2 import LLMModelDetails
model = LLMModelDetails(
name="gpt-4o-mini", # Model name in AI Core
version="latest", # Optional: model version
params={ # Optional: generation parameters
"max_tokens": 500,
"temperature": 0.7
},
timeout=60, # Optional: request timeout in seconds
max_retries=2 # Optional: retry count on failure
)Template References
Instead of inline templates, reference templates stored in the Prompt Registry:
from gen_ai_hub.orchestration_v2 import TemplateRefByID, TemplateRefByScenarioNameVersion
# By ID
ref = TemplateRefByID(id="template-uuid-here")
# By scenario name and version
ref = TemplateRefByScenarioNameVersion(scenario_name="my-scenario", version="1")Response Format
Control the output format:
from gen_ai_hub.orchestration_v2 import (
Template, UserMessage,
ResponseFormatText, ResponseFormatJsonObject, ResponseFormatJsonSchema
)
# Plain text (default)
template = Template(
template=[UserMessage(role="user", content="{{?question}}")],
response_format=ResponseFormatText(type="text")
)
# JSON object
template = Template(
template=[UserMessage(role="user", content="{{?question}}")],
response_format=ResponseFormatJsonObject(type="json_object")
)
# JSON schema
template = Template(
template=[UserMessage(role="user", content="{{?question}}")],
response_format=ResponseFormatJsonSchema(
type="json_schema",
json_schema=JSONResponseSchema(
name="result",
schema_={"type": "object", "properties": {"answer": {"type": "string"}}}
)
)
)Tool / Function Calling
from gen_ai_hub.orchestration_v2 import FunctionTool, FunctionObject
tools = [
FunctionTool(
type="function",
function=FunctionObject(
name="get_weather",
description="Get current weather for a city",
parameters={
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
)
)
]
template = Template(
template=[UserMessage(role="user", content="What is the weather in Berlin?")],
tools=tools
)Content Filtering
Apply Azure Content Safety filters to input and/or output:
from gen_ai_hub.orchestration_v2 import (
FilteringModuleConfig, InputFiltering, OutputFiltering,
AzureContentSafetyInput, AzureContentSafetyOutput, AzureThreshold
)
filtering = FilteringModuleConfig(
input=InputFiltering(filters=[
AzureContentSafetyInput(
hate=AzureThreshold.ALLOW_SAFE,
sexual=AzureThreshold.ALLOW_SAFE,
violence=AzureThreshold.ALLOW_SAFE,
self_harm=AzureThreshold.ALLOW_SAFE
)
]),
output=OutputFiltering(filters=[
AzureContentSafetyOutput(
hate=AzureThreshold.ALLOW_SAFE,
violence=AzureThreshold.ALLOW_SAFE
)
])
)Threshold Values
Thresholds are set using the AzureThreshold enum or equivalent integer values:
| Enum | Int | Meaning |
|---|---|---|
AzureThreshold.ALLOW_ALL | 0 | Allow all content (no filtering) |
AzureThreshold.ALLOW_SAFE | 2 | Allow safe content only |
AzureThreshold.ALLOW_SAFE_LOW | 4 | Allow safe and low-risk content |
AzureThreshold.ALLOW_SAFE_LOW_MEDIUM | 6 | Allow safe, low, and medium-risk content |
from gen_ai_hub.orchestration_v2 import AzureContentSafetyInput, AzureThreshold
# Using enum (recommended)
AzureContentSafetyInput(hate=AzureThreshold.ALLOW_SAFE)
# Using integer
AzureContentSafetyInput(hate=2)Llama Guard Filter
from gen_ai_hub.orchestration_v2 import LlamaGuard38bFilter, LlamaGuard38bFilterConfig
filtering = FilteringModuleConfig(
input=InputFiltering(filters=[
LlamaGuard38bFilter(config=LlamaGuard38bFilterConfig(self_harm=True, violence=True))
])
)Data Masking
Anonymize or pseudonymize PII before sending to the LLM:
from gen_ai_hub.orchestration_v2 import (
MaskingModuleConfig, MaskingProviderConfig,
DPIStandardEntity, MaskingMethod, DataMaskingProviderName
)
masking = MaskingModuleConfig(
masking_providers=[
MaskingProviderConfig(
type=DataMaskingProviderName.SAP_DATA_PRIVACY_INTEGRATION,
method=MaskingMethod.ANONYMIZATION,
entities=[
DPIStandardEntity(type="profile-email"),
DPIStandardEntity(type="profile-person"),
DPIStandardEntity(type="profile-phone")
]
)
]
)Masking Methods
| Method | Description |
|---|---|
ANONYMIZATION | Replace PII with generic placeholders |
PSEUDONYMIZATION | Replace PII with consistent pseudonyms (same input → same output) |
Standard Entity Types
Common entity types for DPIStandardEntity:
| Type | Description |
|---|---|
profile-person | Person names |
profile-email | Email addresses |
profile-phone | Phone numbers |
profile-address | Physical addresses |
profile-org | Organization names |
profile-location | Geographic locations |
profile-url | URLs |
profile-ssn | Social security numbers |
profile-iban | IBAN numbers |
profile-credit-card-number | Credit card numbers |
profile-passport | Passport numbers |
profile-nationalid | National IDs |
profile-username-password | Usernames and passwords |
Custom Entities
from gen_ai_hub.orchestration_v2 import DPICustomEntity
entities = [
DPICustomEntity(regex=r"\b\d{3}-\d{2}-\d{4}\b")
]Document Grounding
Ground LLM responses in your data via vector repositories or other data sources:
from gen_ai_hub.orchestration_v2 import (
GroundingModuleConfig, DocumentGroundingConfig,
DocumentGroundingFilter, DocumentGroundingPlaceholders,
GroundingSearchConfig, DataRepositoryType, GroundingType
)
grounding = GroundingModuleConfig(
type=GroundingType.DOCUMENT_GROUNDING_SERVICE,
config=DocumentGroundingConfig(
placeholders=DocumentGroundingPlaceholders(
input=["{{?question}}"],
output="{{?context}}"
),
filters=[
DocumentGroundingFilter(
id="my-vector-repo-id",
data_repository_type=DataRepositoryType.VECTOR,
search_config=GroundingSearchConfig(
max_chunk_count=5,
max_document_count=3
)
)
]
)
)Data Repository Types
| Type | Description |
|---|---|
VECTOR | Vector-based similarity search |
URL | URL-based document retrieval |
Embeddings via Orchestration
from gen_ai_hub.orchestration_v2 import (
OrchestrationService,
EmbeddingsOrchestrationConfig, EmbeddingsModuleConfigs,
EmbeddingsModelConfig, EmbeddingsModelDetails,
EmbeddingsInput, EmbeddingsInputType
)
embed_config = EmbeddingsOrchestrationConfig(
modules=EmbeddingsModuleConfigs(
model=EmbeddingsModelConfig(
model=EmbeddingsModelDetails(name="text-embedding-3-small")
)
)
)
service = OrchestrationService()
response = service.embed(
config=embed_config,
input=EmbeddingsInput(input="Text to embed", input_type=EmbeddingsInputType.query)
)
print(response.data[0].embedding)Fallback Configurations
Provide multiple module configs; the service tries each in order until one succeeds:
config = OrchestrationConfig(
modules=[
ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(template=[UserMessage(role="user", content="{{?q}}")]),
model=LLMModelDetails(name="gpt-4o")
)
),
ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(template=[UserMessage(role="user", content="{{?q}}")]),
model=LLMModelDetails(name="gpt-4o-mini")
)
)
]
)Configuration References
Store orchestration configs in the Prompt Registry and reference them by ID or name:
from gen_ai_hub.orchestration_v2 import OrchestrationService
service = OrchestrationService(
config_ref=CompletionRequestConfigurationReferenceByIdConfigRef(
config_id="my-config-id"
)
)
response = service.run(placeholder_values={"question": "Hello"})Response Structure
CompletionPostResponse
response.request_id # Unique request ID
response.final_result # LLMModuleResult
response.final_result.choices # List[LLMChoice]
response.final_result.choices[0].message.content # Model output
response.final_result.choices[0].finish_reason # "stop", "length", etc.
response.final_result.model # Model name used
response.final_result.usage # TokenUsage (prompt_tokens, completion_tokens, total_tokens)
response.intermediate_results # ModuleResults from all pipeline stages
response.intermediate_failures # Errors from failed modules (if any)Streaming Response
for chunk in service.stream(placeholder_values={"question": "..."}):
chunk.final_result # StreamLLMModuleResult with delta content
chunk.final_result.choices[0].delta.content # Incremental textRetries
response = service.run_with_retries(
placeholder_values={"question": "What is SAP?"},
max_retries=5,
base_delay=2.0 # Seconds between retries (exponential backoff)
)Full Pipeline Example
from gen_ai_hub.orchestration_v2 import (
OrchestrationConfig, OrchestrationService, ModuleConfig,
PromptTemplatingModuleConfig, Template, SystemMessage, UserMessage,
LLMModelDetails, FilteringModuleConfig, InputFiltering, OutputFiltering,
AzureContentSafetyInput, AzureContentSafetyOutput, AzureThreshold,
MaskingModuleConfig, MaskingProviderConfig,
DPIStandardEntity, MaskingMethod, DataMaskingProviderName,
GroundingModuleConfig, DocumentGroundingConfig,
DocumentGroundingFilter, DocumentGroundingPlaceholders,
GroundingSearchConfig, DataRepositoryType, GroundingType
)
config = OrchestrationConfig(
modules=ModuleConfig(
prompt_templating=PromptTemplatingModuleConfig(
prompt=Template(template=[
SystemMessage(role="system", content="Answer based on the provided context."),
UserMessage(role="user", content="Context: {{?context}}\n\nQuestion: {{?question}}")
]),
model=LLMModelDetails(name="gpt-4o-mini", params={"temperature": 0.3})
),
filtering=FilteringModuleConfig(
input=InputFiltering(filters=[AzureContentSafetyInput(hate=AzureThreshold.ALLOW_SAFE)]),
output=OutputFiltering(filters=[AzureContentSafetyOutput(hate=AzureThreshold.ALLOW_SAFE)])
),
masking=MaskingModuleConfig(
masking_providers=[MaskingProviderConfig(
type=DataMaskingProviderName.SAP_DATA_PRIVACY_INTEGRATION,
method=MaskingMethod.ANONYMIZATION,
entities=[DPIStandardEntity(type="profile-email"), DPIStandardEntity(type="profile-person")]
)]
),
grounding=GroundingModuleConfig(
type=GroundingType.DOCUMENT_GROUNDING_SERVICE,
config=DocumentGroundingConfig(
placeholders=DocumentGroundingPlaceholders(
input=["{{?question}}"],
output="{{?context}}"
),
filters=[DocumentGroundingFilter(
id="knowledge-base-repo",
data_repository_type=DataRepositoryType.VECTOR,
search_config=GroundingSearchConfig(max_chunk_count=5)
)]
)
)
)
)
service = OrchestrationService(config=config)
response = service.run(placeholder_values={
"question": "What is the company travel policy?",
"context": ""
})
print(response.final_result.choices[0].message.content)Resources
- SDK Reference: https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/gen_ai_hub.html
- AI Core Orchestration: https://help.sap.com/docs/sap-ai-core/orchestration
- SAP Samples: https://github.com/SAP-samples/btp-gen-ai-hub-sdk-samples
Troubleshooting
Common Errors and Solutions
Authentication Errors
ValidationError: No credentials found in any source
Cause: The SDK cannot find AI Core credentials in any of the four resolution sources (keyword arguments, environment variables, config file, VCAP_SERVICES).
Solution: 1. Verify environment variables are set: AICORE_CLIENT_ID, AICORE_CLIENT_SECRET, AICORE_AUTH_URL, AICORE_BASE_URL 2. Check for typos in variable names (all prefixed with AICORE_, not AI_CORE_) 3. If using a config file, verify the path: echo $AICORE_CONFIG or check ~/.aicore/config.json 4. On Cloud Foundry, verify the service is bound: cf env <APP_NAME> | grep AICORE
# Quick verification
env | grep AICORE401 Unauthorized or 403 Forbidden
Cause: Expired or invalid credentials.
Solution: 1. Regenerate the service key in SAP BTP cockpit 2. Verify the AICORE_AUTH_URL includes /oauth/token as the token endpoint 3. Check that the service instance is on the correct plan (Extended plan required for GenAI Hub)
AICORE_RESOURCE_GROUP not set (warning)
Cause: No resource group specified.
Solution: Set AICORE_RESOURCE_GROUP or pass resource_group to the client constructor.
Model and Deployment Errors
No deployment found with: deployment.model_name == 'gpt-4o-mini'
Cause: No model deployment matching the requested model name exists in the specified resource group.
Solution: 1. Check available deployments in SAP AI Launchpad under your resource group 2. Verify the model name matches exactly (including provider prefix for some models) 3. Try deploying the model if it's not already deployed 4. Use deployment_id directly if you know it:
from gen_ai_hub.proxy.native.openai import chat
response = chat.completions.create(
deployment_id="dcef02e219ae4916",
messages=[{"role": "user", "content": "Hello!"}]
)Model not found or 404 Not Found
Cause: The model is not available in your AI Core region or plan.
Solution: 1. Check SAP's model catalog for regional availability 2. Verify your AI Core service plan supports the model 3. Some models require specific resource group configurations
Package and Import Errors
ModuleNotFoundError: No module named 'gen_ai_hub'
Cause: The SDK is not installed, or the wrong package was installed.
Solution:
pip uninstall generative-ai-hub-sdk # Remove deprecated package if present
pip install "sap-ai-sdk-gen[all]"ImportError from generative_ai_hub_sdk
Cause: Code was written for the deprecated generative-ai-hub-sdk package.
Solution: The import name is gen_ai_hub for both old and new packages. Update your import statements:
# Old (may work with deprecated package, but should be updated)
import gen_ai_hub # This import path is correct for both packages
# Ensure you have the new package installed
# pip install sap-ai-sdk-genOrchestration Errors
Orchestration deployment not found
Cause: No orchestration deployment exists in the specified resource group.
Solution: 1. Deploy orchestration in your AI Core resource group (default resource group typically has this) 2. Specify a deployment_id when creating OrchestrationService:
service = OrchestrationService(deployment_id="your-orchestration-deployment-id")Content filter violation
Cause: Input or output was blocked by the configured content safety filter.
Solution: 1. Review the filter thresholds (e.g., AzureThreshold.ALLOW_SAFE is the strictest useful setting) 2. Relax thresholds if appropriate: AzureThreshold.ALLOW_SAFE_LOW_MEDIUM allows more content 3. Modify the input to avoid triggering the filter 4. Check response.intermediate_failures for details on which filter blocked the request
Token limit exceeded
Cause: The response exceeds the model's maximum token limit.
Solution: 1. Set max_tokens in the model parameters:
LLMModelDetails(name="gpt-4o-mini", params={"max_tokens": 500})2. Use a model with a larger context window 3. Reduce the prompt length
Network and Timeout Errors
ConnectionError or timeout
Cause: Network connectivity issues between your environment and AI Core.
Solution: 1. Verify AICORE_BASE_URL is correct and accessible 2. Check firewall/proxy settings in your network 3. Increase timeout:
service = OrchestrationService(config=config, timeout=120)4. For BTP environments, ensure the AI Core service instance is in the same region
LangChain-Specific Errors
init_llm fails for a new model
Cause: The model is not yet in init_llm's auto-detection catalog.
Solution: Pass an explicit init_func:
from gen_ai_hub.proxy.langchain import init_llm
from gen_ai_hub.proxy.langchain.amazon import init_chat_converse_model
llm = init_llm(
"anthropic--claude-new",
model_id="anthropic.claude-new-v1:0",
init_func=init_chat_converse_model
)Bedrock model requires model_id
Cause: Amazon Bedrock models need both a model name and a Bedrock model ID.
Solution: Always provide model_id for Bedrock models:
llm = init_llm("anthropic--claude-4-sonnet", model_id="anthropic.claude-sonnet-4-20250514-v1:0")Migration from generative-ai-hub-sdk
Package Rename
| Old | New |
|---|---|
pip install generative-ai-hub-sdk | pip install sap-ai-sdk-gen |
pip install "generative-ai-hub-sdk[all]" | pip install "sap-ai-sdk-gen[all]" |
PyPI: generative-ai-hub-sdk | PyPI: sap-ai-sdk-gen |
Import Paths
The import name gen_ai_hub is unchanged. All import paths remain compatible:
# These imports work with both old and new packages
from gen_ai_hub.proxy.native.openai import chat
from gen_ai_hub.proxy.langchain import init_llm
from gen_ai_hub.orchestration_v2 import OrchestrationServiceVersion Differences
The generative-ai-hub-sdk stopped at v4.12.4. The sap-ai-sdk-gen starts at v5.0.0 and is currently at v6.10.0. Key additions since the rename include:
- Responses API support (
gen_ai_hub.proxy.native.openai.responses) - SAP RPT-1 client (
gen_ai_hub.proxy.native.sap.RPTClient) - Orchestration v2 with enhanced configuration model
- Prompt Registry integration for orchestration config references
- Evaluations module (
gen_ai_hub.evaluations) - Google GenAI native client (using
google-genaiSDK) - Structured outputs (
.parse()methods,response_formatin orchestration)
Migration Steps
1. Update requirements.txt or pyproject.toml:
# Before
generative-ai-hub-sdk[all]>=4.0
# After
sap-ai-sdk-gen[all]>=6.02. Install the new package:
pip uninstall generative-ai-hub-sdk
pip install "sap-ai-sdk-gen[all]"3. Test existing code — import paths under gen_ai_hub are compatible.
4. Update any direct references to the package name in CI/CD configs or Dockerfiles.
5. Check for deprecated model names (e.g., gpt-35-turbo → gpt-4o-mini).
Version Compatibility
| sap-ai-sdk-gen | Python | ai-core-sdk | Key Feature |
|---|---|---|---|
| 6.10.0 | 3.9+ | Bundled | Responses API, RPT-1, Orchestration v2 |
| 6.x | 3.9+ | Bundled | Google GenAI native client |
| 5.x | 3.8+ | Bundled | Initial rename from generative-ai-hub-sdk |
The ai-core-sdk (AICoreV2Client) is included as a dependency of sap-ai-sdk-gen.
Resources
- PyPI: https://pypi.org/project/sap-ai-sdk-gen/
- SDK Reference: https://help.sap.com/doc/generative-ai-hub-sdk/CLOUD/en-US/_reference/gen_ai_hub.html
- Deprecated Package: https://pypi.org/project/generative-ai-hub-sdk/
- SAP Community: https://community.sap.com/