Emerging Tech For Autonomous Vehicles: Thermal Stereo Sensing Posted on : Sep 30 - 2021

Over the past five years, there have been a series of high-profile cases of vehicles misinterpreting the driving environment while using Advanced Driver-Assistance Systems (ADAS). Most recently in July, algorithms inside of Tesla’s TSLA +0.3% Full Self-Driving (FSD) system-initiated slowdowns for the yellowish-moon since sensors were accidentally interpreting the distant orb as a traffic signal. Another blockbuster-yet-slightly-humorous story was when a second Tesla vehicle crashed within a week, but the latter accident was with a parked police vehicle (March, 2021). Or sometimes, as in the cases of Tesla vehicles in Delray Beach (2019) and Williston (2016), the vision system’s miss of a semi-trailer resulted in two separate, unfunny crashes killing both drivers. In fact, in response to multiple such cases, the National Highway and Traffic Safety Administration (NHTSA) reported in March that is was launching a Special Crash Investigation. And, yes, all of the incidents are rare and Tesla does get more press than other such incidents (e.g. Uber’s 2018 fatal crash in Tempe), but these news stories quietly highlight that the standard fidelity of vision-based systems (e.g., cameras) might warrant additional improvements to meet the Safety of the Intended Functionality (SOTIF) and support the predicted self-driving market of $173B by 2023.

With that as the backdrop, long ago (circa the late twentieth century), infrared (IR) sensors were considered and occasionally included for augmented ADAS systems or autonomous research vehicles because they helped with scenarios where vision-systems struggled: night-time darkness, extreme weather (e.g. fog, snow) and/or poor reflectivity. Adding the heat sensing to the long-vision, forward-looking vision would allow greater perception. However, a reoccurring design-slash-operational issue made such sensors less desirable: significant calibration set-up. Believe it or not, for a long while such systems required a separate manufacturing board decorated with a checkerboard pattern and either light bulbs or heat-emitting wires such that both the color cameras and infrared sensors could adjust its base calibration. So not only was the original design expensive operationally, but subsequent recalibration was occasionally needed regardless of how rigidly the platform was designed. Therein, many manufacturers moved away from IR augmentations. In fact, earlier this year Musk announced that Telsa would move to a “pure vision” approach and “… believes a vision-only system is ultimately all that is needed for full autonomy.” View More