Workflow Management Skill

apps/api/src/lib/workflows/

When to Use This Skill

• Adding new scheduled workflows for data fetching
• Debugging workflow execution errors
• Modifying existing workflow schedules or logic
• Integrating new data sources into the update pipeline
• Adding new social media posting workflows

Workflow Architecture

apps/api/src/lib/workflows/
├── cars/                 # Car registration data workflows
│   └── update.ts        # Scheduled car data updates
├── coe/                 # COE bidding data workflows
│   └── update.ts        # Scheduled COE data updates
└── social/              # Social media posting workflows
    ├── discord.ts
    ├── linkedin.ts
    ├── telegram.ts
    └── twitter.ts

Key Patterns

1. Workflow Definition

import { serve } from "@upstash/workflow";

export const POST = serve(async (context) => {
  // Step 1: Fetch data
  await context.run("fetch-data", async () => {
    // Fetching logic
  });

  // Step 2: Process data
  const processed = await context.run("process-data", async () => {
    // Processing logic
  });

  // Step 3: Store results
  await context.run("store-results", async () => {
    // Storage logic
  });
});

2. Scheduling Workflows

• SST infrastructure (

  infra/

  )
• QStash console
• Manual API calls to workflow endpoints

3. Error Handling

await context.run("step-name", async () => {
  try {
    // Logic here
  } catch (error) {
    console.error("Step failed:", error);
    // Log to monitoring service
    throw error; // Re-throw for workflow retry
  }
});

Common Tasks

Adding a New Workflow

1. Create workflow file in appropriate directory
2. Define workflow steps using

   context.run()

3. Add route handler in

   apps/api/src/routes/

4. Configure scheduling (if needed)
5. Add tests for workflow logic

Debugging Workflow Failures

1. Check QStash dashboard for execution logs
2. Review CloudWatch logs for Lambda errors
3. Verify environment variables are set correctly
4. Test workflow locally using development server
5. Check database connectivity and Redis availability

Modifying Existing Workflows

1. Read existing workflow implementation
2. Identify which step needs modification
3. Update step logic while maintaining error handling
4. Test changes locally
5. Deploy and monitor execution

Environment Variables

• DATABASE_URL

  - PostgreSQL connection

• UPSTASH_REDIS_REST_URL

  /

  UPSTASH_REDIS_REST_TOKEN

  - Redis

• QSTASH_TOKEN

  - QStash authentication
• Service-specific tokens (Discord webhook, Twitter API, etc.)

Testing Workflows

pnpm -F @sgcarstrends/api test -- src/lib/workflows

# Start dev server
pnpm dev

# Trigger workflow via HTTP
curl -X POST http://localhost:3000/api/workflows/cars/update

References

• QStash Workflows: Check Context7 for Upstash QStash documentation
• Related files:

  • apps/api/src/routes/workflows.ts

    - Workflow route handlers

  • apps/api/src/config/qstash.ts

    - QStash configuration

  • apps/api/CLAUDE.md

    - API service documentation

Best Practices

1. Idempotency: Ensure workflows can safely retry without duplicating data
2. Step Granularity: Break workflows into small, focused steps
3. Logging: Add comprehensive logging for debugging
4. Timeouts: Configure appropriate timeouts for long-running operations
5. Testing: Write unit tests for workflow logic
6. Monitoring: Track workflow execution metrics