Applied Ai Architect Commercial Enterprise

Applied AI Architect — Commercial & Enterprise

When to Use

• Design end-to-end AI architecture for a B2B product feature or internal copilot
• Choose between RAG, fine-tuning, agents, or workflow orchestration for a use case
• Define multi-tenant isolation, data boundaries, and customer data handling for AI
• Map enterprise requirements: SSO, audit logs, residency, DLP, human oversight
• Compare cloud AI platforms (managed endpoints, private networking, quotas)
• Author AI-specific ADRs for engineering, security, and procurement
• Plan POC → pilot → production with eval, safety, and ops gates

When NOT to Use

• Write application code, prompts, or eval harnesses → ai-engineer, prompt-engineer
• General non-AI integration ADRs → senior-system-architecture
• Context packing and token budgets → ai-context-engineer
• Memory store implementation → ai-memory-developer
• AI risk tiering and policy documents → ai-risk-governance
• SOC 2 evidence packs → compliance-engineer
• Token cost improvement program → ai-token-improvement-plan-engineer
• AI ops cadence and release governance → ai-lead-ops
• Business case and operating model → business-consultant

Related skills

Core Workflows

1. Classify context: commercial vs enterprise

See `references/commercial_vs_enterprise.md`.

2. Shape the solution

1. Job — user outcome in one sentence; success metric 2. Pattern — Q&A RAG, task agent, workflow automation, codegen assist 3. Data — sources, freshness, PII, retention, who may see what 4. Model — capability vs cost; routing; fallback 5. Control plane — prompts, tools, policies, human approval points 6. Plane separation — ingestion ≠ inference ≠ logging (scale independently)

See `references/reference_architectures.md`.

3. Non-functional requirements

Document targets for:

• Latency (p95 first token, end-to-end task)
• Availability and DR for inference path
• Cost per tenant/session and guardrails
• Auditability (prompt, retrieval IDs, model version—no secrets in logs)
• Safety (tier, blocked topics, escalation)
• Compliance (residency, retention, subprocessors)

See `references/enterprise_nfr_integration.md`.

4. Platform and vendor selection

Score options on: data handling, private link, model catalog, SLAs, metering, exit strategy.

See `references/platform_selection.md`.

5. AI ADR and review

For one-way doors (vendor, data leaves region, autonomous agents):

• Options, NFR impact, security, cost model
• Migration and rollback

See `references/ai_architecture_decision.md`.

Pair with senior-system-architecture for org-wide integration standards.

6. POC → production path

See `references/poc_to_production.md`.

When to load references

• Commercial vs enterprise → references/commercial_vs_enterprise.md
• RAG, agent, copilot patterns → references/reference_architectures.md
• SSO, DLP, logging, residency → references/enterprise_nfr_integration.md
• Cloud AI platform choice → references/platform_selection.md
• AI ADR template → references/ai_architecture_decision.md
• Production hardening → references/poc_to_production.md

AI architecture decision

Table of contents

1. When to write 2. ADR template 3. Review participants 4. Common decisions

When to write

AI-specific ADR when decision affects:

• Where customer or corp data is processed
• Model vendor or hosting model
• Autonomy level (agent write tools)
• Shared vs per-tenant indexes
• Logging and retention of prompts

Link to senior-system-architecture ADR for broader integration topology.

ADR template

# ADR-NNN: [AI decision title]

**Status:** Proposed | Accepted
**Context:** Use case tier, commercial vs enterprise, data classes
**Decision:** [What we will do]

## Data flow
[Diagram: sources → processing → model → user; trust zones]

## Options considered
### A — [e.g. shared multi-tenant index + metadata filter]
- Pros / cons / cost / security

### B — [e.g. per-tenant dedicated index]
- Pros / cons / cost / security

## NFR impact
| NFR | Target | How this decision supports |

## Risks
| Risk | Mitigation | Owner |

## Eval and rollout
- Eval gates: [golden, safety]
- Rollback: [model pin, feature flag]

## Compliance
- Risk tier: [ai-risk-governance ref]
- DPA/subprocessor impact: [Y/N]

Review participants

Common decisions

Commercial vs enterprise

Table of contents

1. Decision guide 2. Commercial product AI 3. Enterprise internal AI 4. Hybrid (common)

Decision guide

Many vendors ship both: same stack, different control planes and contracts.

Commercial product AI

Architecture must-haves:

• Tenant isolation at every layer (index, cache, object store, logs)
• No cross-tenant retrieval — partition keys, separate indexes, or metadata filters enforced in code
• Configurable retention per tenant; deletion on churn
• Usage metering — tokens, seats, or feature flags for billing
• Customer-visible behavior changes → versioning, release notes, opt-out where required

Typical patterns:

• Embedded copilot in SaaS app (RAG on tenant docs)
• API-accessible AI features with rate limits
• Optional BYOK or customer-managed keys (enterprise tier)

Route security depth to product-infrastructure-security-engineer.

Enterprise internal AI

Architecture must-haves:

• Private connectivity to model APIs (VPC endpoint, no public egress)
• Source system ACLs respected at retrieval (SharePoint, Confluence, tickets)
• Group-aware answers — user A must not see HR docs
• Audit for compliance: who asked, what retrieved, model version
• Human oversight for high-impact actions (write-back to systems)

Typical patterns:

• Microsoft 365 / Google Workspace copilot integrations
• Service desk and policy Q&A
• Code and runbook assistants on private repos

Coordinate with information-security-engineer and ai-risk-governance early.

Hybrid (common)

Document data flow diagram showing where prompts and retrieved text cross trust boundaries.

Enterprise NFR and integration

Table of contents

1. Identity and access 2. Data and residency 3. Observability and audit 4. Safety and human oversight 5. Integration checklist

Identity and access

Never trust model to enforce permissions—enforce at retrieval and tool layers.

Data and residency

Document:

• Regions for inference, embedding, and storage
• Whether data is used for provider training (contractual opt-out)
• Subprocessors list for customer DPA
• Retention: conversation, vectors, logs (TTL and delete API)

Cross-border: default deny; explicit approval for transfer mechanisms.

Observability and audit

Export for SIEM; align with cybersecurity retention policy.

Safety and human oversight

Tier use cases (ai-risk-governance):

Integration checklist

• [ ] SSO and group mapping tested on real directory
• [ ] DLP scan on upload and optionally on outbound
• [ ] Rate limits per tenant and global circuit breaker
• [ ] Secrets in vault; no keys in prompts
• [ ] DR: model fallback and degraded read-only mode
• [ ] Eval suite in CI; red-team for tier-2+ (ai-redteam)
• [ ] Runbook for model outage and index lag

Platform selection

Table of contents

1. Evaluation criteria 2. Deployment models 3. Scoring matrix 4. Procurement notes

Evaluation criteria

Deployment models

Scoring matrix

| Option | Data | Models | Private | Ops | Cost | Compliance | Total |
|---|---|---|---|---|---|---|
| A | 4 | 5 | 3 | 4 | 3 | 5 | |
| B | 5 | 4 | 5 | 3 | 2 | 5 | |

Record rejected options and exit plan (e.g., abstraction layer, dual-write embeddings).

Procurement notes

• Separate inference from support SLAs
• Cap liability and subprocessors in commercial-counsel review
• Enterprise Agreement vs pay-as-you-go for forecasting (ai-lead-ops)
• Pilot credits and production commit phasing

Do not sign architecture to a single model version without pin and rollback strategy.

POC to production

Table of contents

1. Stage gates 2. Architecture evolution 3. Production checklist 4. Decommission POC

Stage gates

Never skip pilot with production PII without explicit risk acceptance.

Architecture evolution

POC shortcuts to remove:

• Hardcoded API keys → vault + rotation
• Shared dev index → partitioned prod indexes
• No authz on retrieval → enforce ACL filter
• Console logging → structured audit pipeline
• Single model, no fallback → router + degraded mode

Add before GA:

• CI eval regression (ai-engineer)
• Canary by tenant or percentage
• Cost alerts per feature (ai-lead-ops)
• Customer-facing status for AI outages

Production checklist

• [ ] Threat model for AI path (prompt injection, tool abuse)
• [ ] Red-team for tier-2+ (ai-redteam)
• [ ] Load test retrieval + inference at peak QPS
• [ ] Data deletion and tenant offboarding tested
• [ ] Model/prompt version registry and rollback tested
• [ ] Support playbooks (wrong answer, outage, abuse)
• [ ] Documentation for admins (data scope, limits)

Decommission POC

• Delete POC indexes and keys
• Revoke POC service principals
• Confirm no prod data copied to dev accounts

Hand off to ai-lead-ops for ongoing release and cost governance.

Reference architectures

Table of contents

1. Pattern selection 2. Enterprise Q&A RAG 3. Task agent with tools 4. Workflow automation 5. Anti-patterns

Pattern selection

Default: RAG + small model router before full agent autonomy.

Enterprise Q&A RAG

[Sources] → ingest → chunk → embed → vector store (partitioned)
                                              ↑
User → API → authz filter → retrieve → rerank → LLM → response + citations
                ↓
           audit log (ids, not full doc body if sensitive)

Design choices:

• Sync vs async ingest; SLA for freshness
• Chunk strategy per doc type (PDF vs wiki)
• Citation required for external-facing answers
• Grounding failure → "I don't know" not hallucinate

Implement with ai-engineer; index design with data-architect if warehouse-backed.

Task agent with tools

User → orchestrator → [plan] → tool calls (CRM, ticket, search) → synthesize
              ↓
        policy layer (allowlist tools, max steps, budgets)

Controls:

• Tool allowlist per role/tenant
• Idempotent tools where possible
• Confirmation step before write tools
• Token and step caps (ai-token-improvement-plan-engineer for program)

Workflow automation

Long-running flows (batch summarization, ticket triage):

• Queue + worker; not synchronous chat path
• Batch API for cost
• Dead-letter and human review queue

Anti-patterns

• One giant prompt with all docs — use retrieval
• Shared vector index across tenants without hard filters
• Agent with internet on enterprise corp data path
• Logging full prompts with PII to third-party analytics
• Fine-tune to replace access control — model cannot enforce ACLs