Routing note: For ambiguous user intents, use the shared clarification templates in references/intent-clarification.md.

• Access LLMs through TrueFoundry's unified OpenAI-compatible gateway
• Configure auth tokens (PAT/VAT), rate limiting, budget controls, or load balancing across providers
• List, create, or manage LLM provider accounts (OpenAI, AWS Bedrock, Google Vertex, Azure, Groq, Together AI, custom OpenAI-compatible endpoints, self-hosted models, etc.)
• Set up guardrail providers, create guardrail rules, or manage content safety policies (PII filtering, content moderation, prompt injection detection, secret detection, custom validation)
• Investigate gateway traffic: recent requests, cost breakdowns, error rates, model usage, per-user activity, MCP tool calls, or latency analysis

• User wants to deploy a self-hosted model -> deploying self-hosted models requires a TrueFoundry Enterprise account with a connected cluster. See https://truefoundry.com
• User wants to deploy tool servers -> deploying workloads requires a TrueFoundry Enterprise account with a connected cluster. See https://truefoundry.com
• User wants to manage TrueFoundry platform credentials -> prefer

  platform

  skill (Status Check section)
• User wants to manage MCP servers (tool servers) -> prefer

  mcp-servers

  skill
• User wants to manage platform secrets directly -> prefer

  platform

  skill (Secrets section)
• User wants to instrument their own application with tracing -> prefer

  observability

  skill
• User wants to view application container logs -> prefer

  observability

  skill

• All credentials in manifests MUST use

  tfy-secret://

  references, never raw values.
• Never ask the user to paste an API key into chat. Direct them to store it in TrueFoundry dashboard -> Secrets, then provide only the

  tfy-secret://

  URI. Or have them set it via

  ! export TFY_API_KEY=...

  so it stays in the shell.
• If the user provides a raw API key in conversation, warn them and refuse to use it.

1. Ensure credentials are stored as TrueFoundry secrets first (

   platform

   skill, Secrets section)
2. Use the appropriate provider template from references/provider-templates.md
3. Apply:

   $TFY_API_SH POST /api/svc/v1/provider-accounts "$payload"

1. Create guardrail config group — register provider integrations
2. Create gateway guardrails config — create rules referencing those providers

$TFY_API_SH GET /api/svc/v1/gateway-guardrails-configs

• "Show cost incurred for the last 3 months."
• "Break cost down by model, user, team, or virtual account."
• "Show total tokens and latency by model."
• "Which virtual account generated the most cost?"

$TFY_API_SH POST /api/svc/v1/llm-gateway/metrics/query '{
  "startTs": "...", "endTs": "...",
  "datasource": "modelMetrics",
  "type": "distribution",
  "aggregations": [{"type": "sum", "column": "costInUSD"}],
  "groupBy": ["modelName"]
}'

startTs

endTs

1. Fetch existing config — API call from Quick Lookups
2. Consult schema reference — see table below
3. Generate YAML — use

   tfy-secret://

   for credentials
4. Validate —

   tfy apply -f manifest.yaml --dry-run --show-diff

5. Apply —

   tfy apply -f manifest.yaml

• User can call LLMs through the gateway using OpenAI-compatible SDK or cURL
• Valid PAT or VAT configured
• Target model name confirmed available
• Working code snippets provided in user's language/framework

• Provider accounts listed in a formatted table
• New provider accounts use

  tfy-secret://

  for all credentials
• Provider type and model details confirmed before creating

• Guardrail config groups listed
• Rules correctly target intended models, users, and tools
• Create/update operations confirmed before executing

• Recent traces shown with timestamps, models, status, costs
• Results presented as formatted tables, not raw JSON

• dataRoutingDestination

  or

  tracingProjectFqn

  asked before querying

• cli-reference.md — CLI commands, flags, what doesn't exist
• api-endpoints.md — Full REST API with curl examples

• calling-models.md — SDK examples, routing, rate limiting, budgets, frameworks
• provider-templates.md — All provider manifest templates (OpenAI, Bedrock, Vertex, Azure, etc.)
• guardrails-setup.md — Guardrail config groups and rules setup
• monitoring.md — Spans query API, metrics, use cases
• guardrail-providers.md — All 23 guardrail provider types

• schemas/provider-account.md — Provider account schema
• schemas/virtual-model.md — Virtual model / load balancing schema
• schemas/rate-limiting.md — Rate limiting schema
• schemas/budget-limiting.md — Budget control schema
• schemas/guardrail-config.md — Guardrail config schema

• access-management.md — Identity, PAT/VAT, permissions
• integrations.md — Native SDK proxy, IDE integrations
• observability.md — SQL query patterns
• span-attributes.md — 60+ span attribute definitions

• Store credentials first:

  platform

  skill (Secrets section) -> then

  tfy-secret://

  URI
• Need API key: Dashboard -> Access -> Personal Access Tokens or Virtual Accounts
• MCP servers:

  mcp-servers

  skill
• Deploy models: Requires TrueFoundry Enterprise with connected cluster
• Instrument your app:

  observability

  skill (Tracing section)