ai-gateway

Purpose

This skill manages an AI gateway for routing, securing, and monitoring AI service requests in ML operations, ensuring efficient traffic handling, API security, and performance tracking within the aimlops cluster.

When to Use

Use this skill when building ML pipelines that require centralized routing of AI requests, such as in production environments with multiple AI models, to enforce security policies, monitor traffic, or scale API endpoints. Apply it in scenarios involving microservices for AI inference or when integrating with tools like Kubernetes for aimlops workflows.

Key Capabilities

Routing: Dynamically route requests to AI services based on rules, using path-based or header-based matching.
Security: Enforce authentication, rate limiting, and encryption via JWT or API keys.
Monitoring: Track metrics like request latency and error rates through integrated logging and Prometheus exporters.
Configuration: Support YAML-based configs for defining routes, e.g., specifying source and destination endpoints.
Scalability: Handle load balancing across multiple AI backends with automatic failover.

Usage Patterns

To use this skill, first set up the AI gateway via CLI or API, then define routes and security rules. Always authenticate requests using the

$AI_GATEWAY_API_KEY

environment variable. For CLI usage, initialize with

ai-gateway-cli init --config path/to/config.yaml

, then apply changes with

ai-gateway-cli apply

. In code, import the SDK and call methods like

createRoute()

for programmatic setup. Monitor usage by querying metrics endpoints periodically.

Common Commands/API

CLI Commands:

Initialize gateway:

ai-gateway-cli init --cluster aimlops --key $AI_GATEWAY_API_KEY

Add a route:

ai-gateway-cli add-route --path /predict --target http://ai-service:8080 --method POST

Secure an endpoint:

ai-gateway-cli secure --endpoint /predict --auth jwt --rate-limit 100/min

View metrics:
```
ai-gateway-cli metrics --format json
```

API Endpoints:
- Create route: POST /api/v1/routes with body
```
{ "path": "/predict", "target": "http://ai-service:8080", "method": "POST" }
```
- Update security: PUT /api/v1/security/{endpoint} with body
```
{ "authType": "jwt", "rateLimit": 100 }
```
- Get metrics: GET /api/v1/metrics?type=latency

Code Snippets:

python

import requests
headers = {'Authorization': f'Bearer {os.environ.get("AI_GATEWAY_API_KEY")}'}
response = requests.post('http://gateway:8080/api/v1/routes', json={"path": "/predict", "target": "http://ai-service:8080"}, headers=headers)

bash

export AI_GATEWAY_API_KEY=your_api_key_here
ai-gateway-cli add-route --path /chat --target http://llm-service:5000

Config Formats: Use YAML for configurations, e.g.:

routes:
  - path: /predict
    target: http://ai-service:8080
    methods: [POST]
security:
  - endpoint: /predict
    auth: jwt
    rateLimit: 100

Integration Notes

Integrate with aimlops by deploying the gateway as a sidecar or standalone service in your cluster. For Kubernetes, add annotations to pods, e.g.,

kubectl annotate pod ai-pod aimlops/gateway=true

. Use the SDK to link with other AI tools: import and initialize with

AI_Gateway(api_key=os.environ['AI_GATEWAY_API_KEY']).connect(cluster='aimlops')

. Ensure compatibility by matching tags like "ai" and "mlops". For external services, set up webhooks by configuring the gateway's callback URL in your config, e.g., add

callback: http://external-service/webhook

in YAML.

Error Handling

Handle errors by checking HTTP status codes from API responses; for example, 401 indicates authentication failure, so retry with

headers['Authorization'] = f'Bearer {new_key}'

. For CLI, parse output errors like "Error: Invalid route path" and correct inputs. Common issues include missing API keys—always verify

if not os.environ.get('AI_GATEWAY_API_KEY'): raise ValueError('API key required')

. Log errors using the gateway's built-in logger: enable with

ai-gateway-cli config --log-level debug

, then monitor for patterns like rate limit exceedances and implement retries with exponential backoff in code.

Concrete Usage Examples

Route AI Requests: To route prediction requests to an ML model, first export your API key, then use the CLI:

export AI_GATEWAY_API_KEY=abc123; ai-gateway-cli add-route --path /ml-predict --target http://model-service:8000

. Verify with a curl request:

curl -H "Authorization: Bearer abc123" http://gateway:8080/ml-predict -d '{"input": "data"}'

Secure and Monitor API: Secure an endpoint and monitor traffic by running:
```
ai-gateway-cli secure --endpoint /chat --auth api-key --rate-limit 50/min
```
. Then, query metrics:
```
ai-gateway-cli metrics --endpoint /chat
```
. In code, handle it as: ```python import requests; headers = {'Authorization': f'Bearer {os.environ["AI_GATEWAY_API_KEY"]}'}; requests.get('http://gateway:8080/api/v1/metrics', headers=headers)
```
undefined
```

Graph Relationships

Related to: aimlops (cluster), ai (tag), mlops (tag)
Depends on: authentication services for security
Used by: AI services for routing and monitoring
Integrates with: Kubernetes for deployment, Prometheus for metrics collection

ai-gateway

NPX Install

Tags

SKILL.md Content

ai-gateway

Purpose

When to Use

Key Capabilities

Usage Patterns

Common Commands/API

Integration Notes

Error Handling

Concrete Usage Examples

Graph Relationships