Image Benchmark Results 🔗

Image Augmentation Benchmarks 🔗

This directory contains benchmark results for image augmentation libraries.

Overview 🔗

The image benchmarks measure the performance of various image augmentation libraries on standard image transformations. The benchmarks are run on a single CPU thread to ensure consistent and comparable results.

Methodology 🔗

1. Image Loading: Images are loaded using library-specific loaders to ensure optimal format compatibility:

   • OpenCV (BGR → RGB) for Albumentations and imgaug
   • torchvision for PyTorch-based operations
   • PIL for augly
   • Normalized tensors for Kornia

2. Warmup Phase:

   • Performs adaptive warmup until performance variance stabilizes
   • Uses configurable parameters for stability detection
   • Implements early stopping for slow transforms
   • Maximum time limits prevent hanging on problematic transforms

3. Measurement Phase:

   • Multiple runs of each transform
   • Measures throughput (images/second)
   • Calculates statistical metrics (median, standard deviation)

4. Environment Control:

   • Forces single-threaded execution across libraries
   • Captures detailed system information and library versions
   • Monitors thread settings for various numerical libraries

Running the Benchmarks 🔗

To run the image benchmarks:

./run_single.sh -l albumentations -d /path/to/images -o /path/to/output

To run all libraries and generate a comparison:

./run_all.sh -d /path/to/images -o /path/to/output

Benchmark Results 🔗

Image Benchmark Results 🔗

System Information 🔗

• Platform: macOS-15.1-arm64-arm-64bit
• Processor: arm
• CPU Count: 16
• Python Version: 3.12.8

Benchmark Parameters 🔗

• Number of images: 2000
• Runs per transform: 5
• Max warmup iterations: 1000

Library Versions 🔗

• albumentations: 2.0.7
• augly: 1.0.0
• imgaug: 0.4.0
• kornia: 0.8.0
• torchvision: 0.20.1

Performance Comparison 🔗

Number shows how many uint8 images per second can be processed on one CPU thread. Larger is better. The Speedup column shows how many times faster Albumentations is compared to the fastest other library for each transform.

Analysis 🔗

The benchmark results show that Albumentations is generally the fastest library for most image transformations. This is due to its optimized implementation and use of OpenCV for many operations.

Some key observations:

• Albumentations is particularly fast for geometric transformations like resize, rotate, and affine
• For some specialized transformations, other libraries may be faster
• The performance gap is most significant for complex transformations

Recommendations 🔗

Based on the benchmark results, we recommend:

1. Use Albumentations for production workloads where performance is critical
2. Consider the specific transformations you need and check their relative performance
3. For GPU acceleration, consider Kornia, especially for batch processing