Modular Code Organization

File Size Guidelines

When to Split

• File exceeds 500 lines
• Multiple unrelated concerns in same file
• Scroll fatigue finding functions
• Tests for the file are hard to organize
• AI tools truncate or miss context

How to Split

Natural Split Points

1. By domain concept:

   auth.py

   →

   auth/login.py

   ,

   auth/tokens.py

   ,

   auth/permissions.py

2. By abstraction layer: Separate interface from implementation
3. By data type: Group operations on related data structures
4. By I/O boundary: Isolate database, API, file operations

Package Structure

feature/
├── __init__.py      # Keep minimal, just exports
├── core.py          # Main logic (under 500 lines)
├── models.py        # Data structures
├── handlers.py      # I/O and side effects
└── utils.py         # Pure helper functions

DO

• Use meaningful module names (

  data_storage.py

  not

  utils2.py

  )
• Keep

  __init__.py

  files minimal or empty
• Group related functions together
• Isolate pure functions from side effects
• Use snake_case for module names

DON'T

• Split files arbitrarily by line count alone
• Create single-function modules
• Over-modularize into "package hell"
• Use dots or special characters in module names
• Hide dependencies with "magic" imports

Refactoring Large Files

1. Identify clusters: Find groups of related functions
2. Extract incrementally: Move one cluster at a time
3. Update imports: Fix all import statements
4. Run tests: Verify nothing broke after each move
5. Document: Update any references to old locations

Current Codebase Candidates

• Math compute modules (scipy, mpmath, numpy) - domain-specific, may be acceptable
• patterns.py - consider splitting by pattern type
• memory_backfill.py - consider splitting by operation type

Sources

• The Hitchhiker's Guide to Python
• Python Project Best Practices - Dagster
• Right-Sizing Python Files for AI Editors
• PEP 8 Style Guide