Fullmoonfever
Top 20
Some snippets of info from within the patent.@Diogenese
Thoughts, if you would be kind enough?
Appears to be a division of Valeo obviously.
Haven't dug into the patent as most of it probs over my head but I have some suspicions
METHOD OF RECONSTRUCTING A PROPERTY OF AN IMAGE AND COMPUTING DEVICE
Abstract
Application Number EP2024052574 Publication Number 2024/160994 Status In Force Filing Date 2024-02-02 Publication Date 2024-08-08 Owner VALEO SCHALTER UND SENSOREN GMBH (Germany) Inventor Nagiub, Mena
The present application relates to a Method of reconstructing a property of an image in a sequence of images with a spiking neural network. The method comprises: obtaining raw data of the image (RDt), calculating a similarity measure using raw data of the image (RDt) and raw data of a previous image (RDt-1) preceding the image in the sequence, if the similarity measure exceeds a predefined threshold: - generating a spike train (ST) for the spiking neural network, the spike train (ST) comprising data encoding differential raw data, wherein the differential raw data (DRD) has been calculated using raw data of the im-age and raw data of the previous image (RDt, RDt-1), - inputting the spike train (ST) to the spiking neural network, - reconstructing the property of the image with the spiking neural network, - outputting the property of the image. The application further relates to a computing device (34), a lidar system (30) comprising the computing device (34) and a vehicle (100) comprising the lidar system (30).
IPC Classes ?
- G01S 17/89 - Lidar systems, specially adapted for specific applications for mapping or imaging
- G01S 17/931 - Lidar systems, specially adapted for specific applications for anti-collision purposes of land vehicles
- G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
- G06V 20/56 - Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G01S 7/4865 - Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak
- G06N 3/049 - Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
Performing the property reconstruction based on the difference between image raw data at one instance and a previous instance has the advantage that it may reduce the required processing time, since the calculations will be done only on the differential data rather than full data input.
Furthermore, the property reconstruction is done on the spikes representation of the data rather than the original data. This enables the use of spiking neural networks, which may benefit from the low power properties of the spiking neural networks and the advantageous use of neuromorphic processors.
The advantage of the described method is that event-based processing may be applied and that the raw data is processed if there is a change in the scene. This allows to lower the processing power and the temperature and power consumption of the system. If the sequence of images depicts a scene with a low rate of change, the number of processed images may drop down, and that the rate of processing new frames may drop down. Additionally, the lifetime of the components of the computing device implementing the method may be increased.
Also, it is possible to apply modern processor architectures based on neuromorphic computing, which may be optimized for event based processing and low power consumption.
The present application also relates to a computing device for image property reconstruction using a spiking neural network. The computing device comprises circuitry to execute any one of the previously described methods. Due to the event based processing of spiking neural networks, the lifetime of the components of the computing device may be increased. Also, it is possible to apply modern processor architectures based on neuromorphic computing, which may be optimized for event based processing and low power consumption.
The present application also relates to a lidar system comprising such a computing device and an image sensor. The image sensor is coupled to the computing device and the raw data of the image and/or the raw data of the preceding image comprises data provided by the image sensor.