groq-performance-tuning

Original：🇺🇸 English
Translated

Optimize Groq API performance with caching, batching, and connection pooling. Use when experiencing slow API responses, implementing caching strategies, or optimizing request throughput for Groq integrations. Trigger with phrases like "groq performance", "optimize groq", "groq latency", "groq caching", "groq slow", "groq batch".

3installs
Sourcejeremylongshore/claude-code-plugins-plus-skills
Added on

NPX Install

npx skill4agent add jeremylongshore/claude-code-plugins-plus-skills groq-performance-tuning

Tags

Translated version includes tags in frontmatter
groq-api-optimizationcaching-strategyrequest-batchingconnection-poolingperformance-tuning

SKILL.md Content

View Translation Comparison →

Groq Performance Tuning

Overview

Optimize Groq API performance with caching, batching, and connection pooling.

Prerequisites

  • Groq SDK installed
  • Understanding of async patterns
  • Redis or in-memory cache available (optional)
  • Performance monitoring in place

Latency Benchmarks

OperationP50P95P99
Read50ms150ms300ms
Write100ms250ms500ms
List75ms200ms400ms

Caching Strategy

Response Caching

typescript
import { LRUCache } from 'lru-cache';

const cache = new LRUCache<string, any>({
  max: 1000,
  ttl: 60000, // 1 minute
  updateAgeOnGet: true,
});

async function cachedGroqRequest<T>(
  key: string,
  fetcher: () => Promise<T>,
  ttl?: number
): Promise<T> {
  const cached = cache.get(key);
  if (cached) return cached as T;

  const result = await fetcher();
  cache.set(key, result, { ttl });
  return result;
}

Redis Caching (Distributed)

typescript
import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

async function cachedWithRedis<T>(
  key: string,
  fetcher: () => Promise<T>,
  ttlSeconds = 60
): Promise<T> {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const result = await fetcher();
  await redis.setex(key, ttlSeconds, JSON.stringify(result));
  return result;
}

Request Batching

typescript
import DataLoader from 'dataloader';

const groqLoader = new DataLoader<string, any>(
  async (ids) => {
    // Batch fetch from Groq
    const results = await groqClient.batchGet(ids);
    return ids.map(id => results.find(r => r.id === id) || null);
  },
  {
    maxBatchSize: 100,
    batchScheduleFn: callback => setTimeout(callback, 10),
  }
);

// Usage - automatically batched
const [item1, item2, item3] = await Promise.all([
  groqLoader.load('id-1'),
  groqLoader.load('id-2'),
  groqLoader.load('id-3'),
]);

Connection Optimization

typescript
import { Agent } from 'https';

// Keep-alive connection pooling
const agent = new Agent({
  keepAlive: true,
  maxSockets: 10,
  maxFreeSockets: 5,
  timeout: 30000,
});

const client = new GroqClient({
  apiKey: process.env.GROQ_API_KEY!,
  httpAgent: agent,
});

Pagination Optimization

typescript
async function* paginatedGroqList<T>(
  fetcher: (cursor?: string) => Promise<{ data: T[]; nextCursor?: string }>
): AsyncGenerator<T> {
  let cursor: string | undefined;

  do {
    const { data, nextCursor } = await fetcher(cursor);
    for (const item of data) {
      yield item;
    }
    cursor = nextCursor;
  } while (cursor);
}

// Usage
for await (const item of paginatedGroqList(cursor =>
  groqClient.list({ cursor, limit: 100 })
)) {
  await process(item);
}

Performance Monitoring

typescript
async function measuredGroqCall<T>(
  operation: string,
  fn: () => Promise<T>
): Promise<T> {
  const start = performance.now();
  try {
    const result = await fn();
    const duration = performance.now() - start;
    console.log({ operation, duration, status: 'success' });
    return result;
  } catch (error) {
    const duration = performance.now() - start;
    console.error({ operation, duration, status: 'error', error });
    throw error;
  }
}

Instructions

Step 1: Establish Baseline

Measure current latency for critical Groq operations.

Step 2: Implement Caching

Add response caching for frequently accessed data.

Step 3: Enable Batching

Use DataLoader or similar for automatic request batching.

Step 4: Optimize Connections

Configure connection pooling with keep-alive.

Output

  • Reduced API latency
  • Caching layer implemented
  • Request batching enabled
  • Connection pooling configured

Error Handling

IssueCauseSolution
Cache miss stormTTL expiredUse stale-while-revalidate
Batch timeoutToo many itemsReduce batch size
Connection exhaustedNo poolingConfigure max sockets
Memory pressureCache too largeSet max cache entries

Examples

Quick Performance Wrapper

typescript
const withPerformance = <T>(name: string, fn: () => Promise<T>) =>
  measuredGroqCall(name, () =>
    cachedGroqRequest(`cache:${name}`, fn)
  );

Resources

Next Steps

For cost optimization, see 
groq-cost-tuning
.