Effect-TS Concurrency

Overview

When to Use

• Processing large arrays of effects (e.g.,

  Effect.all

  ,

  Effect.forEach

  )
• Rate limiting external API calls or database connections
• Coordinating work between background processes (fibers)
• Signaling completion or state changes across different parts of the app

• Simple sequential operations
• When standard

  Promise.all

  is sufficient (though Effect is usually preferred for consistency)

Core Pattern: Bounded Parallelism

Effect.all(effects)

Effect.all(effects, { concurrency: 10 })

Quick Reference

Effect.fork

Fiber.join

semaphore.withPermits(n)

Deferred.await

Deferred.succeed

all

forEach

mapEffect

Implementation

1. Bounded Parallelism (Critical)

import { Effect } from 'effect';

// Process 1000 items, max 10 concurrent
const results = yield* Effect.all(
  items.map(processItem),
  { concurrency: 10 }
);

2. Semaphore for Rate Limiting

import { Effect } from 'effect';

const program = Effect.gen(function* () {
  const semaphore = yield* Effect.makeSemaphore(5); // Max 5 concurrent
  
  yield* Effect.all(
    requests.map((req) => 
      semaphore.withPermits(1)(handleRequest(req))
    ),
    { concurrency: 'unbounded' } // Semaphore controls actual concurrency
  );
});

3. Deferred for Signaling

import { Deferred, Effect, Fiber } from 'effect';

const program = Effect.gen(function* () {
  const signal = yield* Deferred.make<void>();
  
  const worker = yield* Effect.fork(
    Effect.gen(function* () {
      yield* Deferred.await(signal); // Wait for signal
      yield* doWork();
    })
  );
  
  yield* setup();
  yield* Deferred.succeed(signal, undefined); // Trigger worker
  yield* Fiber.join(worker);
});

Common Mistakes

• Unbounded parallelism: Forgetting

  { concurrency: n }

  in

  Effect.all

  .
• Leaking Fibers: Forking fibers without joining or interrupting them (use

  Effect.scoped

  for safety).
• Deadlocks: Circular dependencies between semaphores or deferreds.

Red Flags - STOP and Start Over

• Using

  Effect.all

  on a large array without

  { concurrency: n }

  .
• Using

  Promise.all

  inside an Effect-TS codebase.
• Manual

  setTimeout

  for rate limiting instead of

  Effect.makeRateLimiter

  or

  Semaphore

  .

Rationalization Table