caffe-cifar-10

Original：🇺🇸 English
Translated

Guidance for building Caffe from source and training CIFAR-10 models. This skill applies when tasks involve compiling Caffe deep learning framework, configuring Makefile.config, preparing CIFAR-10 dataset, or training CNN models with Caffe solvers. Use for legacy ML framework installation, LMDB dataset preparation, and CPU-only deep learning training tasks.

4installs
Sourceletta-ai/skills
Added on

NPX Install

npx skill4agent add letta-ai/skills caffe-cifar-10

Tags

Translated version includes tags in frontmatter
caffe-buildcifar10-trainingdeep-learning-frameworklmdb-datasetcpu-only-training

SKILL.md Content

View Translation Comparison →

Caffe CIFAR-10 Build and Training

This skill provides procedural guidance for building the Caffe deep learning framework from source and training models on the CIFAR-10 dataset.

When to Use This Skill

  • Building Caffe from source on Ubuntu/Debian systems
  • Training CIFAR-10 or similar image classification models with Caffe
  • Configuring Caffe for CPU-only execution
  • Troubleshooting Caffe build and dependency issues

Critical Requirements Checklist

Before starting, identify ALL requirements from the task specification:
  1. Execution mode: CPU-only vs GPU (affects solver configuration)
  2. Iteration count: Specific number of training iterations required
  3. Output files: Where training logs and models should be saved
  4. Model checkpoints: Which iteration's model file is expected

Phase 1: Dependency Installation

System Dependencies

Install required packages before attempting to build:
bash
apt-get update && apt-get install -y \
    build-essential cmake git \
    libprotobuf-dev libleveldb-dev libsnappy-dev \
    libhdf5-serial-dev protobuf-compiler \
    libatlas-base-dev libgflags-dev libgoogle-glog-dev liblmdb-dev \
    libopencv-dev libboost-all-dev \
    python3-dev python3-numpy python3-pip

Verification Step

Confirm critical libraries are installed:
bash
dpkg -l | grep -E "libhdf5|libopencv|libboost"

Phase 2: Caffe Source Acquisition

Clone and Checkout

bash
git clone https://github.com/BVLC/caffe.git
cd caffe
git checkout 1.0  # Note: Tag is "1.0", not "1.0.0"

Common Mistake

The release tag is 
1.0
, not 
1.0.0
. Verify with 
git tag -l
if uncertain.

Phase 3: Makefile.config Configuration

Create Configuration File

bash
cp Makefile.config.example Makefile.config

Essential Configuration Changes

Apply these modifications to 
Makefile.config
:
  1. CPU-Only Mode (if no GPU available):
    CPU_ONLY := 1
  2. OpenCV Version (for OpenCV 3.x or 4.x):
    OPENCV_VERSION := 3
    Note: OpenCV 4 may require additional compatibility patches.
  3. HDF5 Paths (Ubuntu-specific):
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial
  4. Python Configuration (Python 3):
    PYTHON_LIBRARIES := boost_python3 python3.8
PYTHON_INCLUDE := /usr/include/python3.8 /usr/lib/python3/dist-packages/numpy/core/include
    Adjust version numbers based on installed Python version.

Configuration Verification

After editing, verify no duplicate definitions exist:
bash
grep -n "PYTHON_INCLUDE\|PYTHON_LIB\|CPU_ONLY" Makefile.config
Ensure each setting appears only once in an uncommented form.

Phase 4: Building Caffe

Memory-Aware Compilation

Avoid using all CPU cores on memory-constrained systems:
bash
# For systems with limited RAM (< 8GB)
make all -j2

# For systems with adequate RAM
make all -j$(nproc)

Build Failure Recovery

If the build fails or is killed (often due to memory):
  1. Clean the build:
    bash
    make clean
  2. Rebuild with reduced parallelism:
    bash
    make all -j1

Build Verification

Confirm the binary exists after build:
bash
ls -la .build_release/tools/caffe.bin
# or for CPU-only builds:
ls -la .build_release/tools/caffe

Phase 5: Dataset Preparation

Download CIFAR-10

bash
./data/cifar10/get_cifar10.sh

Convert to LMDB Format

bash
./examples/cifar10/create_cifar10.sh

Verification

Confirm LMDB directories exist:
bash
ls -la examples/cifar10/cifar10_train_lmdb
ls -la examples/cifar10/cifar10_test_lmdb

Phase 6: Solver Configuration

Modify Solver for Requirements

Edit 
examples/cifar10/cifar10_quick_solver.prototxt
:
  1. Set iteration count:
    max_iter: 500  # Or as specified in task
  2. Set execution mode:
    solver_mode: CPU  # Change from GPU if required

Verification

bash
grep -E "max_iter|solver_mode" examples/cifar10/cifar10_quick_solver.prototxt

Phase 7: Training Execution

Run Training with Output Capture

bash
./build/tools/caffe train \
    --solver=examples/cifar10/cifar10_quick_solver.prototxt \
    2>&1 | tee training_output.txt

Alternative Binary Paths

Depending on build configuration, the binary may be at:
  • .build_release/tools/caffe
  • build/tools/caffe
  • .build_release/tools/caffe.bin

Phase 8: Verification

Required Outputs Checklist

  1. Caffe binary exists:
    bash
    test -f .build_release/tools/caffe && echo "OK" || echo "MISSING"
  2. Model file exists (iteration-specific):
    bash
    ls -la examples/cifar10/cifar10_quick_iter_*.caffemodel
  3. Training output captured:
    bash
    test -f training_output.txt && echo "OK" || echo "MISSING"
  4. Solver configured correctly:
    bash
    grep "solver_mode: CPU" examples/cifar10/cifar10_quick_solver.prototxt

Common Pitfalls

1. Premature Termination

Never stop after 
make clean
or intermediate steps. Complete the full workflow: Dependencies -> Build -> Dataset -> Configure -> Train -> Verify

2. Missing Solver Configuration

The solver file must be modified for:
  • CPU vs GPU execution mode
  • Specific iteration count requirements

3. Skipping Dataset Preparation

Training will fail without LMDB data. Always run both:
  • get_cifar10.sh
    (download)
  • create_cifar10.sh
    (convert)

4. Build Parallelism Issues

High parallelism (
-j$(nproc)
) can exhaust memory. Start with 
-j2
on constrained systems.

5. Duplicate Configuration Entries

Multiple edits to 
Makefile.config
can create duplicate definitions. Always verify single definitions for each setting.

6. Wrong Git Tag

Use 
1.0
not 
1.0.0
for the stable release.

Decision Framework

When encountering issues:
  1. Build killed: Reduce parallelism, run 
    make clean
    , rebuild with 
    -j1
  2. Missing headers: Check HDF5 and OpenCV include paths in Makefile.config
  3. Python errors: Verify Python version matches configuration
  4. Training fails immediately: Check dataset preparation completed
  5. Wrong output location: Verify solver paths and output file redirection