Multimedia Backend Integrator

generate_media

Architecture Overview

_base.py          -- Registration: API keys, default models, priority lists
_selector.py      -- Auto-selection logic: picks best backend by key + priority
_image.py         -- Image backends: OpenAI, Google (Gemini/Imagen), Grok, OpenRouter
_video.py         -- Video backends: Grok, Google Veo, OpenAI Sora
_audio.py         -- Audio backends: ElevenLabs, OpenAI TTS
generate_media.py -- Entry point: routing, validation, batch mode, image-to-image

Complete Checklist: Adding a New Backend

1. Registration (

_base.py

)

• Add to

  BACKEND_API_KEYS

  : map backend name to env var(s)
• Add to

  DEFAULT_MODELS

  : map backend name to

  {MediaType: model_name}

  for each supported type
• Add to

  BACKEND_PRIORITY

  : insert at correct position per media type

2. Implementation (

_image.py

/

_video.py

/

_audio.py

)

• Add

  import

  for SDK at module top
• Implement

  _generate_{media}_{backend}(config) -> GenerationResult

• Check API key first, return error result if missing
• Create SDK client with API key
• Map

  config.*

  fields to SDK parameters
• Handle continuation (if applicable) — see Continuation Store Patterns
• Write output bytes to

  config.output_path

• Return

  GenerationResult

  with metadata
• Wrap in try/except, log errors

3. Dispatcher Update

• Add

  elif backend == "new_backend":

  in the media type's

  generate_{media}()

  function

4. Image-to-Image Support (

generate_media.py

)

• Add backend name to the

  selected_backend not in (...)

  check in

  _generate_single_with_input_images

• Add fallback:

  elif has_api_key("new_backend"):

  in the auto-selection chain
• Update error message to mention new backend + env var

5. Documentation

• TOOL.md

  : Add env var to frontmatter, backend to tables, keywords

• generate_media.py

  docstring: Update

  backend_type

  list and

  Supported Backends

6. Tests

• Backend registration tests (API keys, default models, priority order)
• Auto-selection tests (with only this backend's key, with multiple keys)
• SDK call verification (correct params passed through)
• Output file written correctly
• Continuation flow (if applicable)
• Error handling (missing key, API errors)
• Parameter mapping (aspect_ratio, size, duration)
• Update existing tests that assert priority list length/contents

Continuation Store Patterns

response.id

previous_response_id

_GeminiChatStore

gemini_chat_{uuid12}

send_message()

_GrokImageStore

grok_img_{uuid12}

image_url

Store Pattern Template

class _NewBackendStore:
    def __init__(self, max_items: int = 50):
        self._store: OrderedDict[str, Any] = OrderedDict()
        self._max = max_items

    def save(self, data: Any) -> str:
        store_id = f"prefix_{uuid.uuid4().hex[:12]}"
        if len(self._store) >= self._max:
            self._store.popitem(last=False)  # LRU eviction
        self._store[store_id] = data
        return store_id

    def get(self, store_id: str) -> Any | None:
        return self._store.get(store_id)

_store = _NewBackendStore()

Common Pitfalls

1. Missing from priority list — Backend works when explicitly specified but never auto-selected
2. Sync vs async — Some SDKs are sync-only; wrap in

   asyncio.to_thread()

   if needed
3. Ephemeral URLs — Some APIs return temporary URLs; always prefer base64 or download immediately
4. Falsy duration —

   duration or default

   treats

   0

   as falsy; use

   if duration is not None

5. Existing test breakage — Adding to priority list changes auto-selection; update existing tests that clear env vars
6. Image-to-image gating — The

   _generate_single_with_input_images

   function has a backend allowlist

Reference Files

massgen/tool/_multimodal_tools/generation/_base.py

massgen/tool/_multimodal_tools/generation/_selector.py

massgen/tool/_multimodal_tools/generation/_image.py

massgen/tool/_multimodal_tools/generation/_video.py

massgen/tool/_multimodal_tools/generation/_audio.py

massgen/tool/_multimodal_tools/generation/generate_media.py

massgen/tool/_multimodal_tools/TOOL.md

massgen/tests/test_grok_multimedia_generation.py

massgen/tests/test_grok_multimedia_backend_selection.py

massgen/tests/test_multimodal_image_backend_selection.py