Modernize Scientific Computing Stack

When to Use This Skill

• Migrating Python 2 scientific scripts to Python 3
• Updating legacy data processing code using outdated patterns
• Modernizing scripts that use deprecated file handling, string encoding, or numerical libraries
• Converting scripts from csv module to pandas for data analysis
• Replacing os.path with pathlib for path manipulation

Approach

Phase 1: Complete Code Discovery

1. Read all source files completely - If a file read is truncated, request the full content before proceeding. Never assume file contents based on partial reads.
2. Identify all dependencies - Check for:
   • Import statements (standard library and third-party)
   • Configuration files (JSON, YAML, INI)
   • Data files (CSV, Excel, pickle)
   • Environment requirements
3. Map the data flow - Understand:
   • Input file formats and encodings
   • Data transformations applied
   • Output format requirements
   • Any intermediate files or caches

Phase 2: Identify Migration Requirements

print "text"

print("text")

unicode()

str()

str()

open(file)

open(file, encoding='utf-8')

os.path.join()

pathlib.Path()

csv

pandas.read_csv()

for key in dict.keys()

for key in dict

dict.has_key(x)

x in dict

with

xrange()

range()

/

//

Phase 3: Implementation Strategy

1. Create the modernized script with these priorities:
   • UTF-8 encoding for all file operations
   • pathlib.Path for all file path manipulations
   • pandas for CSV/data processing
   • Type hints where beneficial
   • Context managers for resource handling
2. Handle configuration files - Check for file existence before reading:
   python

   config_path = Path("config.json")
   if config_path.exists():
       config = json.loads(config_path.read_text(encoding='utf-8'))

3. Create requirements.txt - Include all dependencies with version constraints

Phase 4: Verification Protocol

• The complete content was written (not truncated)
• The syntax is valid
• All imports are present

1. Syntax validation - Run Python syntax check:
   bash

   python -m py_compile script.py

2. Import verification - Test all imports resolve:
   bash

   python -c "from script import *"

3. Functional test - Run the script and compare output to expected results
4. Output validation - Verify output format matches requirements exactly

Common Pitfalls to Avoid

1. Truncated file content - Never proceed with partial file reads. If a response shows

   ... [truncated]

   or incomplete content, request the full file before continuing.
2. Unverified writes - After using a write operation, always read the file back to confirm the complete content was written correctly.
3. Encoding issues - Always specify

   encoding='utf-8'

   explicitly in file operations. Legacy scripts often have implicit ASCII assumptions.
4. Path string concatenation - Replace all

   os.path.join()

   and string concatenation for paths with

   pathlib.Path

   operations.
5. Missing edge case handling:
   • Empty data files or datasets
   • Missing required files
   • Invalid data types in CSV columns
   • Stations/entities with no matching data
6. Environment setup repetition - When setting up environments (venv, PATH), verify the setup persists rather than repeating in each command.

Verification Checklist

• All source files were read completely (no truncation)
• Written files were verified by reading back
• All Python 2 patterns have been converted
• File encodings are explicitly specified
• pathlib is used for all path operations
• pandas is used for data processing (where appropriate)
• requirements.txt includes all dependencies
• Script runs without errors
• Output matches expected format exactly
• Edge cases are handled (empty data, missing files)

Output Validation

1. Run the modernized script
2. Capture the output
3. Compare against expected format character-by-character if needed
4. Pay attention to:
   • Decimal precision in numerical output
   • Whitespace and formatting
   • Order of output items
   • Units and labels