integrate

Original：🇺🇸 English

Translated

Add Olakai monitoring to existing AI code — wrap your LLM client, configure custom KPIs, and validate the integration end-to-end

26installs

Sourceandrewyng/context-hub

Added on2026-03-21

NPX Install

npx skill4agent add andrewyng/context-hub integrate

SKILL.md Content

View Translation Comparison →

Integrate Olakai into Existing AI Code

This skill guides you through adding Olakai monitoring to an existing AI agent or LLM-powered application with minimal code changes.

For full SDK documentation, see: https://app.olakai.ai/llms.txt

Prerequisites

Existing working AI agent/application using OpenAI, Anthropic, or other LLM

Olakai CLI installed and authenticated (

npm install -g olakai-cli && olakai login

)

Olakai API key for your agent (get via CLI:

olakai agents get AGENT_ID --json | jq '.apiKey'

)

Node.js 18+ (for TypeScript) or Python 3.7+ (for Python)

Note: Each agent can have its own API key. Create one with
olakai agents create --name "Name" --with-api-key

Why Custom KPIs Are Essential

Adding monitoring is only the first step. The real value of Olakai comes from tracking custom KPIs specific to your agent's business purpose.

Without KPIs configured:

Only basic token counts and request data
No aggregated business KPIs on dashboard
No alerting capabilities
No ROI tracking

With KPIs configured:

Custom KPIs (items processed, success rates, quality scores)
Trend analysis and performance dashboards
Threshold-based alerting
Business value calculations

Plan to configure at least 2-4 KPIs that answer: "How do I know this agent is performing well?"

KPIs are unique per agent. If adding monitoring to an agent that needs the same KPIs as another already-configured agent, you must still create new KPI definitions for this agent. KPIs cannot be shared or reused across agents.

Understanding the customData to KPI Pipeline

Before adding monitoring, understand how custom data flows through Olakai:

SDK customData → CustomDataConfig (Schema) → Context Variable → KPI Formula → kpiData

Critical Rules

Rule	Consequence
Only CustomDataConfig fields become variables	Unregistered customData fields are NOT usable in KPIs
Formula evaluation is case-insensitive	`stepCount` , `STEPCOUNT` , `StepCount` all work in formulas
NUMBER configs need numeric values	Don't send `"5"` (string), send `5` (number)

IMPORTANT: The SDK accepts any JSON in
customData
, but only fields registered as CustomDataConfigs are processed. Unregistered fields are stored but cannot be used in KPIs.

Quick Start (5-Minute Integration)

For TypeScript/JavaScript

1. Install the SDK:

bash

npm install @olakai/sdk

2. Add tracking after your LLM call:

Before:

typescript

import OpenAI from "openai";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userMessage }],
});

After:

typescript

import OpenAI from "openai";
import { olakaiConfig, olakai } from "@olakai/sdk";

olakaiConfig({ apiKey: process.env.OLAKAI_API_KEY });

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userMessage }],
});

// Track the interaction (fire-and-forget)
olakai("event", "ai_activity", {
  prompt: userMessage,
  response: response.choices[0].message.content,
  tokens: response.usage?.total_tokens,
  userEmail: user.email,
  task: "Customer Experience",
});

For Python

1. Install the SDK:

bash

pip install olakai-sdk

2. Add tracking after your LLM call:

Before:

python

from openai import OpenAI
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": user_message}],
)

After:

python

from openai import OpenAI
from olakaisdk import olakai_config, olakai, OlakaiEventParams

olakai_config(os.getenv("OLAKAI_API_KEY"))
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": user_message}],
)

# Track the interaction
olakai("event", "ai_activity", OlakaiEventParams(
    prompt=user_message,
    response=response.choices[0].message.content,
    tokens=response.usage.total_tokens,
    userEmail=user.email,
    task="Customer Experience",
))

Detailed Integration Guide

Step 1: Identify Your Integration Pattern

Pattern A: Single LLM Client You have one OpenAI/Anthropic client used throughout your app. Use the fire-and-forget

olakai()

call after each completion.

Pattern B: Multiple LLM Calls per Request Your agent makes several LLM calls to complete one task. Use manual event tracking to aggregate calls into a single event.

Pattern C: Streaming Responses You stream LLM responses to users. Track after the stream completes with the full accumulated response.

Pattern D: Third-Party LLM (not OpenAI/Anthropic) You use Perplexity, Groq, local models, etc. Use manual event tracking via

olakai()

olakai_event()

Step 2: Install and Configure

TypeScript Setup

typescript

// lib/olakai.ts - Initialize once at app startup
import { olakaiConfig } from "@olakai/sdk";

olakaiConfig({
  apiKey: process.env.OLAKAI_API_KEY!,
  debug: process.env.NODE_ENV === "development",
});

Python Setup

python

# lib/olakai.py - Initialize once at app startup
import os
from olakaisdk import olakai_config

olakai_config(
    api_key=os.getenv("OLAKAI_API_KEY"),
    debug=os.getenv("DEBUG") == "true"
)

Step 3: Add Context to Calls

Adding User Information

TypeScript:

typescript

olakai("event", "ai_activity", {
  prompt: userMessage,
  response: aiResponse,
  userEmail: user.email,
  task: "Customer Experience",
});

Python:

python

olakai("event", "ai_activity", OlakaiEventParams(
    prompt=user_message,
    response=ai_response,
    userEmail=user.email,
    task="Customer Experience",
))

Grouping Events by Conversation (chatId)

For assistive AI (chatbots/copilots), use

chatId

to group multiple turns of a conversation together. This is required for CHAT-scoped KPIs that analyze the full conversation.

typescript

olakai("event", "ai_activity", {
  prompt: userMessage,
  response: aiResponse,
  chatId: conversationId,  // groups turns in the same conversation
  userEmail: user.email,
});

When to use
chatId
: If your agent handles multi-turn conversations and you want KPIs that evaluate the entire conversation (e.g., sentiment scoring, satisfaction), pass a consistent
chatId
across all turns.

Adding Custom Data

IMPORTANT: Only send fields you've registered as CustomDataConfigs (Step 5.3). Unregistered fields are stored but cannot be used in KPIs.

Only send data you'll use in KPIs or for filtering. Don't duplicate fields already tracked by the platform (session ID, agent ID, user email, timestamps, token count, model, provider — all tracked automatically).

TypeScript:

typescript

olakai("event", "ai_activity", {
  prompt: userMessage,
  response: aiResponse,
  userEmail: user.email,
  customData: {
    // Only include fields registered as CustomDataConfigs
    Department: user.department,
    ProjectId: currentProject.id,
    Priority: ticket.priority,
  },
});

Step 4: Handle Agentic Workflows

If your agent makes multiple LLM calls per task, aggregate them into a single event.

taskExecutionId
— Critical for multi-agent workflows. If multiple agents collaborate on the same task, the orchestrator must generate ONE
taskExecutionId
and pass it to all agents. This is how Olakai correlates cross-agent work as a single logical task.

typescript

async function processDocument(doc: Document): Promise<string> {
  const startTime = Date.now();
  let totalTokens = 0;

  // Step 1: Extract
  const extraction = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: `Extract from: ${doc.content}` }],
  });
  totalTokens += extraction.usage?.total_tokens ?? 0;

  // Step 2: Analyze
  const analysis = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: `Analyze: ${extraction.choices[0].message.content}` }],
  });
  totalTokens += analysis.usage?.total_tokens ?? 0;

  const result = analysis.choices[0].message.content ?? "";

  // Track the complete workflow as ONE event
  olakai("event", "ai_activity", {
    prompt: `Process document: ${doc.title}`,
    response: result,
    tokens: totalTokens,
    requestTime: Date.now() - startTime,
    taskExecutionId: crypto.randomUUID(),
    task: "Data Processing & Analysis",
    customData: {
      DocumentType: doc.type,
      StepCount: 2,
      Success: 1,
    },
  });

  return result;
}

Step 5: Configure Custom KPIs (Essential for Value)

This step is required to get real value from Olakai. Without KPIs, you're only tracking events — not gaining actionable insights.

5.1 Install CLI (if not already)

bash

npm install -g olakai-cli
olakai login

5.2 Register Your Agent

bash

olakai agents create \
  --name "Document Processor" \
  --description "Processes and summarizes documents" \
  --workflow WORKFLOW_ID \
  --with-api-key

5.2.1 Ensure Agent Has a Workflow

Every agent MUST belong to a workflow, even if it's the only agent.

bash

# Check if agent has a workflow
olakai agents get YOUR_AGENT_ID --json | jq '.workflowId'

# If null, create a workflow and associate:
olakai workflows create --name "Your Workflow Name" --json
olakai agents update YOUR_AGENT_ID --workflow WORKFLOW_ID

5.3 Create Custom Data Configs FIRST

IMPORTANT: Create configs for ALL fields you send in
customData
. Only registered fields can be used in KPIs. CustomDataConfigs are agent-scoped.

bash

olakai custom-data create --agent-id YOUR_AGENT_ID --name "DocumentType" --type STRING
olakai custom-data create --agent-id YOUR_AGENT_ID --name "StepCount" --type NUMBER
olakai custom-data create --agent-id YOUR_AGENT_ID --name "Success" --type NUMBER

# Verify all configs exist for this agent
olakai custom-data list --agent-id YOUR_AGENT_ID

5.4 Create KPIs

bash

olakai kpis create \
  --name "Documents Processed" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "IF(Success = 1, 1, 0)" \
  --aggregation SUM

olakai kpis create \
  --name "Avg Steps per Document" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "StepCount" \
  --aggregation AVERAGE

5.5 Update SDK Code to Match

After creating configs, ensure your SDK code sends exactly those field names:

typescript

customData: {
  DocumentType: doc.type,     // Matches CustomDataConfig "DocumentType"
  StepCount: 2,               // Matches CustomDataConfig "StepCount"
  Success: true ? 1 : 0,      // Matches CustomDataConfig "Success"
}

Framework-Specific Integrations

Next.js API Routes

typescript

// app/api/chat/route.ts
import { NextRequest, NextResponse } from "next/server";
import { olakai } from "@olakai/sdk";
import { auth } from "@/auth";

export async function POST(req: NextRequest) {
  const session = await auth();
  if (!session?.user) {
    return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
  }

  const { message } = await req.json();

  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: message }],
  });

  olakai("event", "ai_activity", {
    prompt: message,
    response: response.choices[0].message.content,
    userEmail: session.user.email!,
    task: "Customer Experience",
  });

  return NextResponse.json({ reply: response.choices[0].message.content });
}

FastAPI (Python)

python

from fastapi import FastAPI, Depends
from olakaisdk import olakai_config, olakai, OlakaiEventParams

app = FastAPI()

@app.on_event("startup")
async def startup():
    olakai_config(os.getenv("OLAKAI_API_KEY"))

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

@app.post("/chat")
async def chat(message: str, user: User = Depends(get_current_user)):
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": message}]
    )

    olakai("event", "ai_activity", OlakaiEventParams(
        prompt=message,
        response=response.choices[0].message.content,
        userEmail=user.email,
        task="Customer Experience",
    ))

    return {"reply": response.choices[0].message.content}

Handling Edge Cases

Streaming Responses

Track after the stream completes with the full response:

typescript

const stream = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userMessage }],
  stream: true,
});

let fullResponse = "";
for await (const chunk of stream) {
  fullResponse += chunk.choices[0]?.delta?.content ?? "";
  res.write(chunk.choices[0]?.delta?.content ?? "");
}

// Track after stream completes
olakai("event", "ai_activity", {
  prompt: userMessage,
  response: fullResponse,
  userEmail: user.email,
});

Error Handling

typescript

try {
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages,
  });
  return response.choices[0].message.content;
} catch (error) {
  // Track the failed attempt
  olakai("event", "ai_activity", {
    prompt: messages[messages.length - 1].content,
    response: `Error: ${error instanceof Error ? error.message : "Unknown"}`,
    task: "Software Development",
    customData: { Success: 0 },
  });
  throw error;
}

Non-OpenAI Providers

For Anthropic, Perplexity, or other providers, use manual tracking:

typescript

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

async function callClaude(prompt: string): Promise<string> {
  const startTime = Date.now();

  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });

  const content = response.content[0].type === "text" ? response.content[0].text : "";

  olakai("event", "ai_activity", {
    prompt,
    response: content,
    tokens: response.usage.input_tokens + response.usage.output_tokens,
    requestTime: Date.now() - startTime,
    task: "Content Development",
  });

  return content;
}

Test-Validate-Iterate Cycle

Never assume your integration is working. Always validate by generating a test event and inspecting the actual data.

Step 1: Generate a Test Event

Run your application to trigger at least one LLM call.

Step 2: Fetch and Inspect the Event

bash

olakai activity list --limit 1 --json
olakai activity get EVENT_ID --json

Step 3: Validate Each Component

Check customData is present:

bash

olakai activity get EVENT_ID --json | jq '.customData'

Check KPIs are numeric (not strings or null):

bash

olakai activity get EVENT_ID --json | jq '.kpiData'

CORRECT:

json

{ "My KPI": 42 }

WRONG (formula stored as string):

json

{ "My KPI": "MyVariable" }

Fix:

olakai kpis update KPI_ID --formula "MyVariable"

WRONG (null value):

json

{ "My KPI": null }

Fix by ensuring:

CustomDataConfig exists:

olakai custom-data create --agent-id ID --name "MyVariable" --type NUMBER

Field name case matches exactly (case-sensitive)
SDK actually sends the field in customData

Validation Flow

1. Trigger LLM call (generate event)
           ↓
2. Fetch: olakai activity get ID --json
           ↓
3. Event exists? NO → Check API key, SDK init, debug mode
           ↓
4. customData correct? NO → Fix SDK customData parameter
           ↓
5. kpiData numeric? NO → olakai kpis update ID --formula "X"
           ↓
6. kpiData not null? NO → Create CustomDataConfig, check field name case
           ↓
✅ Integration validated

KPI Formula Reference

Supported Operators

Category	Operators
Arithmetic	`+` , `-` , `*` , `/`
Comparison	`<` , `<=` , `=` , `<>` , `>=` , `>`
Logical	`AND` , `OR` , `NOT`
Conditional	`IF(condition, true_val, false_val)`
Null handling	`ISNA(value)` , `ISDEFINED(value)`

Common Formula Patterns

bash

--formula "StepCount"                          # passthrough
--formula "SuccessRate * 100"                  # percentage conversion
--formula "IF(Success = 1, 1, 0)"              # conditional counting
--formula "IF(PII detected, 1, 0)"             # built-in variable
--formula "IF(ISDEFINED(MyField), MyField, 0)" # null-safe

Aggregation Types

Aggregation	Use For
`SUM`	Totals, counts
`AVERAGE`	Rates, percentages

Quick Reference

typescript

// TypeScript — initialize once
import { olakaiConfig, olakai } from "@olakai/sdk";
olakaiConfig({ apiKey: process.env.OLAKAI_API_KEY });

// Track any interaction
olakai("event", "ai_activity", {
  prompt: "input",
  response: "output",
  tokens: 1500,
  requestTime: 5000,
  userEmail: "user@example.com",
  chatId: "conversation-id",
  taskExecutionId: "uuid-shared-across-agents",
  task: "Data Processing & Analysis",
  customData: { StepCount: 3, Success: 1 },
});

python

# Python — initialize once
from olakaisdk import olakai_config, olakai, OlakaiEventParams
olakai_config(os.getenv("OLAKAI_API_KEY"))

# Track any interaction
olakai("event", "ai_activity", OlakaiEventParams(
    prompt="input",
    response="output",
    tokens=1500,
    requestTime=5000,
    userEmail="user@example.com",
    chatId="conversation-id",
    taskExecutionId="uuid-shared-across-agents",
    task="Data Processing & Analysis",
    customData={"StepCount": 3, "Success": 1},
))

integrate

NPX Install

Tags

SKILL.md Content

Integrate Olakai into Existing AI Code

Prerequisites

Why Custom KPIs Are Essential

Understanding the customData to KPI Pipeline

Critical Rules

Quick Start (5-Minute Integration)

For TypeScript/JavaScript

For Python

Detailed Integration Guide

Step 1: Identify Your Integration Pattern

Step 2: Install and Configure

TypeScript Setup

Python Setup

Step 3: Add Context to Calls

Adding User Information

Grouping Events by Conversation (chatId)

Adding Custom Data

Step 4: Handle Agentic Workflows

Step 5: Configure Custom KPIs (Essential for Value)

5.1 Install CLI (if not already)

5.2 Register Your Agent

5.2.1 Ensure Agent Has a Workflow

5.3 Create Custom Data Configs FIRST

5.4 Create KPIs

5.5 Update SDK Code to Match

Framework-Specific Integrations

Next.js API Routes

FastAPI (Python)

Handling Edge Cases

Streaming Responses

Error Handling

Non-OpenAI Providers

Test-Validate-Iterate Cycle

Step 1: Generate a Test Event

Step 2: Fetch and Inspect the Event

Step 3: Validate Each Component

Validation Flow

KPI Formula Reference

Supported Operators

Common Formula Patterns

Aggregation Types

Quick Reference