Skip to content
Recently [shraiwi] decided to have a bit of good nerdy fun and implement the FAST machine vision algorithm on an ESP32-S3 using its SIMD ISA. This involves using the assembly instructions for the ISA, making for an exciting opportunity to become familiar with this part of this Xtensa Lx7-based twin-core MCU.
The Features from Accelerated Segment Test (FAST) algorithm is a corner detection method that can determine with reasonable accuracy and in real-time whether a given input frame contains said corners which thus may correspond to objects and other characteristics worthy of note. This is essential for machine vision, with machine learning able to bump up its recognition accuracy even more.
Implementing this algorithm on the ESP32-S3 proved to be a bit of a challenge due to somewhat limited SIMD ISA. Missing instructions included things like comparisons between unsigned byte values, and there is no support for direct misaligned reads.
All of these required some creative coding to work around, but in the end the ESP32-S3 running at 240 MHz was able to process a VGA stream in real-time at 30 FPS. This means about a 2x speed-up over a naive implementation using only non-SIMD instructions.
If you want to play along at home, the Platform IO project is made available on GitHub.
Instagram Engineering
Shopify Engineering
