New sensing technologies complement standard vision system capabilities, enhancing occupant safety, enabling increased personalization, advancing today's in-cabin experience to tomorrow’s third space experience DTS® , a global leader in next-generation audio, imaging and sensing technology, and...
investor.xperi.com
Xperi Develops World-first In-cabin Monitoring Technologies on Neuromorphic Vision Systems
April 21, 2021
Download PDFPDF Format (opens in new window)
New sensing technologies complement standard vision system capabilities, enhancing occupant safety, enabling increased personalization, advancing today's in-cabin experience to tomorrow’s third space experience
CALABASAS, Calif.--(BUSINESS WIRE)--
DTS®, a global leader in next-generation audio, imaging and sensing technology, and a wholly owned subsidiary of
Xperi Holding Corporation (NASDAQ: XPER) (“Xperi”), today announced a world-first neuromorphic driver monitoring solution (DMS), powered by Prophesee Metavision® - Event-Based Vision sensor.
Using the raw feed from the Metavision® - Event-Based Vision sensor, the DTS AutoSense* team developed driver-centric monitoring features such as gaze tracking, head pose, identification, and eyelid opening.
With more than 20 years of world-leading experience in neural solutions and computer vision, and with billions of products powered by their solutions, the Xperi team maps and adapts in-cabin monitoring (ICM) technologies to existing and future sensor centric trends.
“Intelligent sensing is required by the next-generation in-cabin experience. The critical path to this is a sensor fusion-focused advanced research strategy,” said Petronel Bigioi, chief technology officer, Xperi. “DTS AutoSense’s advanced research team is centered on state-of-the-art sensing solutions and neuromorphic sensing is one of the technologies that can shape the future of the industry.”
The neuromorphic sensors capture information at an equivalent frame rate of 10,000 frames per second without requiring active illumination. This enables better low light performance for driver monitoring features as well as never-before-seen capabilities such as saccadic eye movement or micro-expressions, crucial next steps in the next-generation in-cabin experiences.
Xperi relies on an end-to-end data generation and training system built to address the particular needs of the sensor. The training data set was generated based on Xperi’s extensive computer vision infrastructure, reusing ground-truth from the visible and near infrared spectrums, synthesizing a completely novel approach in neuromorphic based sensing.
“Being able to instantaneously detect the subtlest, almost imperceptible, face and eye motions can be lifesaving,” said Petronel Bigioi. “The low light powerful vision processing capabilities, fast response times and low power operation that this pioneering neuromorphic-based technology application enables makes it ideal for the safety-critical applications of driver monitoring systems, while also broadening the ability to further enhance and personalize the vehicle cabin.”
In addition to excellent low light performance and robustness to various illumination conditions, neuromorphic sensing directly addresses the privacy aspect specific to traditional imaging-based sensing (always-on cameras).
“DTS AutoSense’s innovative AI-enabled approach underscores the ability of Prophesee’s sensor solution to provide OEMs with an effective safety enhancement for vehicles of all types,” said Marc Rousseau, VP Products and Business Development at Prophesee. “We are delighted to collaborate on this first-of-its-kind technology with Xperi and its forward-thinking, high caliber team.”
*DTS AutoSense comprises a Driver Monitoring Solution (DMS) and an Occupancy Monitoring Solution (OMS), the first to be designed into passenger vehicles projected to be on the road in 2021. Working together they provide actionable insights into activity inside the vehicle, including the driver, passengers, pets and objects, to create a better, safer experience.