Evermont
Stealth Mode
Howdy All,
This is showing as having been published 3 hours ago. I don't think it's been posted, but if so, I'll delete.
Neuromorphic vision sensors are coming to smartphones
By Dan O'SheaFeb 28, 2023 03:45pm
Prophesee SAQualcomm Snapdragonneuromorphicimage sensors
![]()
A new partnership teaming Prophesee and Qualcomm will optimize event-based neuromorphic vision sensors for use in mobile devices, resulting in better images from device cameras. (Prophesee).
Prophesee, a provider of event-based neuromorphic vision sensor technology, announced a partnership with Qualcomm Technologies at the massive Mobile World Congress event in Barcelona, Spain, this week, a collaboration that will see Propheseeâs Metavision sensors optimized for use with Qualcommâs Snapdragon platform to bring neuromorphic-enabled image capture capabilities to mobile devices.
Event-based neuromorphic vision sensor technology could be a game changer for camera performance, with neuromorphic capabilities processing movement and moments closer to the way the human brain processes themâwith less blurring and more clarity than a frame-based camera can manage. Specifically, the technology allows cameras to perform better while capturing fast movements and scenes in low lighting in their photos and videos. For the most part, the consumer marketplace is still waiting to get their hands on devices with these capabilities, but the new partnership means that wait is growing shorter.
Later this year, the Prophesee plans to release a development kit to support the integration of the Metavision sensor technology for use with devices that contain next-generation Snapdragon platforms. After that, it will not be too much longer before consumers can experience the benefits of event-based neuromorphic vision sensor technology themselves.
Luca Verre, co-founder and CEO of Prophesee, told Fierce Electronics, âWe expect phones with this feature/capability to be in the market by 2024. It will likely appear in âflagshipâ models first.â
When the technology arrives, it is not expected to replace traditional frame-based sensors, but instead work in tandem with them, with much of the camera performance improvement coming through Propheseeâs event-based continuous and asynchronous pixel sensing approach which will help in the âde-blurring of imagesâ and the highlighting of focal points that otherwise might become lost where low lighting intrudes on a captured moment.
That could make consumers much happier about the quality of the pictures they take on their mobile devices, although there is a good chance they may not even know they will have neuromorphic sensors to thank for the improvement, as they probably will not have to think about switching into a different photo capture mode to take advantage of neuromorphic sensing.
âItâs unlikely that smartphones would have a âneuromorphic mode,â but instead would work seamlessly with the existing image capture capabilities - but, in theory, that could be something the OEM could consider,â Verre said. âNote that using an event based camera actually reduces the amount of data processed, capturing only things in a scene that move, which are often âinvisibleâ to traditional cameras, so it is largely an augmentation of traditional frame based cameras (and other sensors, such as lidar in a car), not a replacement, especially in consumer applications.â
These sensors already are used in other kinds of applications, including business and industrial use cases such as security cameras, surveillance, preventative maintenance, vibration monitoring, high speed counting, and others where event cameras can work âas a standalone machine vision option,â Verre said, adding, âThere is vast potential in the idea of sensor fusion, combining event-based sensors with other types of sensors, like frame-based cameras.â
The Qualcomm partnership comes almost a year after Prophesee announced a partnership with Sony that revolved around enabling improved integration of event-based sensing technology into devices, and Verre said the migration of the technology to mobile phones likely will be smoother as a result of the earlier partnership. Working with Sony, a leading CMOS sensor provider for the mobile industry, helped make the sensors âmore applicable for mobile (smaller size, lower power, etc.) with 3D stacking manufacturing techniques,â he said.
Prophesee also sees the technology as having potential in other mobile and wearable device applications, such as augmented reality headsets. Prophesee already is talking to OEMs about moving in that direction, and Verre said the company believes that âby enabling a paradigm shift in sensing modalities with this approach, there are countless applications that can benefit.â
Nice pick-up @Bravo
Last paragraph below has synergies with one of the AGM slides last year. Can't wait to see where we go from here.
Prophesee also sees the technology as having potential in other mobile and wearable device applications, such as augmented reality headsets. Prophesee already is talking to OEMs about moving in that direction, and Verre said the company believes that âby enabling a paradigm shift in sensing modalities with this approach, there are countless applications that can benefit.â