Image Benchmark Results 🔗

Image Augmentation Benchmarks 🔗

This directory contains benchmark results for image augmentation libraries.

Overview 🔗

The image benchmarks measure the performance of various image augmentation libraries on standard image transformations. The benchmarks are run on a single CPU thread to ensure consistent and comparable results.

Methodology 🔗

1. Image Loading: Images are loaded using library-specific loaders to ensure optimal format compatibility:

   • OpenCV (BGR → RGB) for Albumentationsx and imgaug
   • torchvision for PyTorch-based operations
   • PIL for augly
   • Normalized tensors for Kornia

2. Warmup Phase:

   • Performs adaptive warmup until performance variance stabilizes
   • Uses configurable parameters for stability detection
   • Implements early stopping for slow transforms
   • Maximum time limits prevent hanging on problematic transforms

3. Measurement Phase:

   • Multiple runs of each transform
   • Measures throughput (images/second)
   • Calculates statistical metrics (median, standard deviation)

4. Environment Control:

   • Forces single-threaded execution across libraries
   • Captures detailed system information and library versions
   • Monitors thread settings for various numerical libraries

Running the Benchmarks 🔗

To run the image benchmarks:

./run_single.sh -l albumentationsx -d /path/to/images -o /path/to/output

Number shows how many uint8 images per second can be processed on one CPU thread. Larger is better. The Speedup column shows how many times faster Albumentationsx is compared to the fastest other library for each transform.

Analysis 🔗

The benchmark results show that Albumentationsx is generally the fastest library for most image transformations. This is due to its optimized implementation and use of OpenCV for many operations.

Some key observations:

• Albumentationsx is particularly fast for geometric transformations like resize, rotate, and affine
• For some specialized transformations, other libraries may be faster
• The performance gap is most significant for complex transformations

Recommendations 🔗

Based on the benchmark results, we recommend:

1. Use Albumentationsx for production workloads where performance is critical
2. Consider the specific transformations you need and check their relative performance
3. For GPU acceleration, consider Kornia, especially for batch processing