BRN Discussion Ongoing

[Diogenese]

Went live when?

If you don't have dreams, you can't have dreams come true!

29 February

 

[Diogenese]

iMedia

Home
Tech
Application of SNN in Vehicle Field

2023-02-03 22:41

iMedia

Home
Tech
Application of SNN in Vehicle Field

2023-02-03 22:41 HKT

Depots are using neuromorphic technologies to implement AI functions such as keyword recognition, driver attention, and passenger behavior monitoring.

Mimicking biological brain processes is tantalizing because it promises to enable advanced functionality without a significant increase in power consumption, which is EV-friendly. Neuromorphic computing and perception are also expected to bring these advantages, such as extremely low latency, enabling real-time decision-making in some cases. This combination of low latency and high energy efficiency is very attractive.

Spike Network


The truth is, there's still something we don't know about how the human brain works. However, cutting-edge research has shown that neurons communicate by sending each other electrical signals called spikes, and that the sequence and timing of the spikes (rather than their size) are key factors. Mathematical models of how neurons respond to these spikes are still being studied. But many scientists agree that if multiple spikes arrive at adjacent neurons at the same time (or in very rapid succession), it means that the information those spikes represent are correlated, thus causing the neuron to fire a spike.

This is in contrast to artificial neural networks based on deep learning (the mainstream AI today), where information travels through the network in a regular rhythm; that is, the information entering each neuron is represented as a numerical value, rather than based on time.

Making a spike-based artificial system is not easy. Besides we don't know how neurons work, there is no consensus on the best way to train spike neural networks. Backpropagation requires computing derivatives, which is not possible with spikes. Some companies approximate the derivative of the spike in order to use backpropagation (like SynSense), and some use another technique called STDP (spike timing dependent plasticity), which is closer to how biological brains function. However, STDP is not yet mature as a technique (BrainChip uses this method for one-shot learning at the edge). It is also possible to take a deep learning CNN, trained by backpropagation in the normal way, and convert it to run in the spike domain (another technique used by BrainChip).


SynSense Speck

SynSense is working with BMW to advance the integration of neuromorphic chips in smart cockpits and explore related areas together. BMW will evaluate SynSense's Speck SoC, which combines SynSense's neuromorphic vision processor and Inivation's 128x128-pixel event camera. Can be used to capture visual information in real-time, identify and detect objects, and perform other vision-based detection and interaction functions.

Dylan Muir, vice president of global research operations at SynSense, said: "When BMW replaces RGB cameras with Speck modules for visual perception, they can replace not only sensors, but also a lot of GPU or CPU computing required to process standard RGB visual streams."


Using event-based cameras provides a higher dynamic range than standard cameras, which is beneficial for use in extreme lighting conditions inside and outside the vehicle.

BMW will explore the use of neuromorphic technology in cars, including monitoring driver attention and passenger behavior through the Speck module.

"In the coming months, we will explore more applications inside and outside the vehicle," Muir said.


SynSense's neuromorphic vision processors have a fully asynchronous digital architecture. Each neuron uses integer logic with 8-bit synaptic weights, 16-bit neuron states, 16-bit thresholds, and unit input-output spikes. Neurons use a simple integrate-and-fire model, where when the neuron fires a simple 1-bit spike, the input spike is combined with the neuron's synaptic weights until a threshold is reached. Overall, the design is a balance between complexity and computational efficiency, Muir said.

Application of SNN in Vehicle Field

SynSense's electronic neurons are based on the integrate-and-fire model


SynSense's digital chips are designed to process event-based CNNs, with each layer processed by a different core. The kernel runs asynchronously and independently; the entire processing pipeline is event-driven.

"Our Speck modules run in real-time with low latency," Muir said. "We can manage effective inference rates above 20Hz at less than 5mW power consumption. This is much faster than using traditional low-power computing on standard RGB video streams. ."

While SynSense and BMW will initially explore neuromorphic use cases in smart cockpits, it has potential for other automotive applications as well.


"First, we'll explore non-safety-critical use cases, and we're planning future versions of Speck with higher resolution, as well as improvements to our DynapCNN vision processor, which will interface with high-resolution sensors," Muir said. We plan for these future technologies It will support advanced automotive applications such as autonomous driving, emergency braking, etc."

Application of SNN in Vehicle Field

SynSense and Inivation Speck module, an event camera-based module containing sensors and processors


BrainChip Akida

Mercedes-Benz's EQXX concept car, which debuted at CES earlier this year, uses BrainChip's Akida neuromorphic processor for in-vehicle keyword recognition. Billed as "the most efficient car Mercedes has ever made," the car utilizes neuromorphic technology that consumes less power than a deep learning-based keyword spotting system. That's crucial for a car with a range of 620 miles, or 167 miles more than Mercedes' flagship electric car, the EQS.

Mercedes said at the time that BrainChip's solution was five to 10 times more efficient than traditional voice controls at recognizing the wake word "Hey Mercedes."


Application of SNN in Vehicle Field

Mercedes said, “Although neuromorphic computing is still in its infancy, such systems will soon be on the market within a few years. When applied at scale throughout vehicles, they have the potential to radically reduce the amount of effort required to run the latest AI technologies. power consumption."

BrainChip's CMO Jerome Nadel said: "Mercedes is focused on big issues like battery management and transmission, but every milliwatt counts, and when you think about energy efficiency, even the most basic reasoning, like finding keywords, matters. important."


A typical car could have as many as 70 different sensors by 2022, Nadel said. For cockpit applications, these sensors can enable face detection, gaze assessment, emotion classification, and more.

He said: “From a system architecture perspective, we can do a 1:1 approach where there is a sensor that will do some preprocessing and then the data will be forwarded. The AI will do inference near the sensor...it will Instead of the full array of data from sensors, the inference metadata is passed forward.”

The idea is to minimize the size and complexity of packets sent to AI accelerators, while reducing latency and minimizing power consumption. Each vehicle will likely have 70 Akida chips or sensors with Akida technology, each of which will be "low-cost parts that won't notice them at all," Nadel said. He noted that attention needs to be paid to the BOM of all these sensors.


Application of SNN in Vehicle Field

BrainChip expects to have its neuromorphic processor next to every sensor on the vehicle

Going forward, Nadel said, neuromorphic processing will also be used in ADAS and autonomous driving systems. This has the potential to reduce the need for other types of power-hungry AI accelerators.


"If every sensor could have Akida configured on one or two nodes, it would do adequate inference, and the data passed would be an order of magnitude less, because that would be inference metadata...that would affect the servers you need," he said. power."

BrainChip's Akida chip accelerates SNNs (spike neural networks) and CNNs (by converting to SNNs). It's not tailored for any specific use case or sensor, so it can be paired with visual sensing for face recognition or people detection, or other audio applications like speaker ID. BrainChip also demonstrated Akida's smell and taste sensors, although it's hard to imagine how these could be used in cars (perhaps to detect air pollution or fuel quality through smell and taste).

Akida is set up to handle SNNs or deep learning CNNs that have been transformed into SNNs. Unlike the native spike network, the transformed CNN preserves some spike-level information, so it may require 2 or 4 bits of computation. However, this approach allows exploiting the properties of CNNs, including their ability to extract features from large datasets. Both types of networks can be updated at the edge using STDP. In the case of Mercedes-Benz, this might mean retraining the network after deployment to discover more or different keywords.


Application of SNN in Vehicle Field

According to Autocar, Mercedes-Benz confirmed that "many innovations" from the EQXX concept car, including "specific components and technologies," will be used in the production model. There's no word yet on whether new Mercedes-Benz models will feature artificial brains.






Depots are using neuromorphic technologies to implement AI functions such as keyword recognition, driver attention, and passenger behavior monitoring.

Mimicking biological brain processes is tantalizing because it promises to enable advanced functionality without a significant increase in power consumption, which is EV-friendly. Neuromorphic computing and perception are also expected to bring these advantages, such as extremely low latency, enabling real-time decision-making in some cases. This combination of low latency and high energy efficiency is very attractive.

Spike Network


The truth is, there's still something we don't know about how the human brain works. However, cutting-edge research has shown that neurons communicate by sending each other electrical signals called spikes, and that the sequence and timing of the spikes (rather than their size) are key factors. Mathematical models of how neurons respond to these spikes are still being studied. But many scientists agree that if multiple spikes arrive at adjacent neurons at the same time (or in very rapid succession), it means that the information those spikes represent are correlated, thus causing the neuron to fire a spike.

This is in contrast to artificial neural networks based on deep learning (the mainstream AI today), where information travels through the network in a regular rhythm; that is, the information entering each neuron is represented as a numerical value, rather than based on time.

Making a spike-based artificial system is not easy. Besides we don't know how neurons work, there is no consensus on the best way to train spike neural networks. Backpropagation requires computing derivatives, which is not possible with spikes. Some companies approximate the derivative of the spike in order to use backpropagation (like SynSense), and some use another technique called STDP (spike timing dependent plasticity), which is closer to how biological brains function. However, STDP is not yet mature as a technique (BrainChip uses this method for one-shot learning at the edge). It is also possible to take a deep learning CNN, trained by backpropagation in the normal way, and convert it to run in the spike domain (another technique used by BrainChip).


SynSense Speck

SynSense is working with BMW to advance the integration of neuromorphic chips in smart cockpits and explore related areas together. BMW will evaluate SynSense's Speck SoC, which combines SynSense's neuromorphic vision processor and Inivation's 128x128-pixel event camera. Can be used to capture visual information in real-time, identify and detect objects, and perform other vision-based detection and interaction functions.

Dylan Muir, vice president of global research operations at SynSense, said: "When BMW replaces RGB cameras with Speck modules for visual perception, they can replace not only sensors, but also a lot of GPU or CPU computing required to process standard RGB visual streams."


Using event-based cameras provides a higher dynamic range than standard cameras, which is beneficial for use in extreme lighting conditions inside and outside the vehicle.

BMW will explore the use of neuromorphic technology in cars, including monitoring driver attention and passenger behavior through the Speck module.

"In the coming months, we will explore more applications inside and outside the vehicle," Muir said.


SynSense's neuromorphic vision processors have a fully asynchronous digital architecture. Each neuron uses integer logic with 8-bit synaptic weights, 16-bit neuron states, 16-bit thresholds, and unit input-output spikes. Neurons use a simple integrate-and-fire model, where when the neuron fires a simple 1-bit spike, the input spike is combined with the neuron's synaptic weights until a threshold is reached. Overall, the design is a balance between complexity and computational efficiency, Muir said.

Application of SNN in Vehicle Field

SynSense's electronic neurons are based on the integrate-and-fire model


SynSense's digital chips are designed to process event-based CNNs, with each layer processed by a different core. The kernel runs asynchronously and independently; the entire processing pipeline is event-driven.

"Our Speck modules run in real-time with low latency," Muir said. "We can manage effective inference rates above 20Hz at less than 5mW power consumption. This is much faster than using traditional low-power computing on standard RGB video streams. ."

While SynSense and BMW will initially explore neuromorphic use cases in smart cockpits, it has potential for other automotive applications as well.


"First, we'll explore non-safety-critical use cases, and we're planning future versions of Speck with higher resolution, as well as improvements to our DynapCNN vision processor, which will interface with high-resolution sensors," Muir said. We plan for these future technologies It will support advanced automotive applications such as autonomous driving, emergency braking, etc."

Application of SNN in Vehicle Field

SynSense and Inivation Speck module, an event camera-based module containing sensors and processors


BrainChip Akida

Mercedes-Benz's EQXX concept car, which debuted at CES earlier this year, uses BrainChip's Akida neuromorphic processor for in-vehicle keyword recognition. Billed as "the most efficient car Mercedes has ever made," the car utilizes neuromorphic technology that consumes less power than a deep learning-based keyword spotting system. That's crucial for a car with a range of 620 miles, or 167 miles more than Mercedes' flagship electric car, the EQS.

Mercedes said at the time that BrainChip's solution was five to 10 times more efficient than traditional voice controls at recognizing the wake word "Hey Mercedes."


Application of SNN in Vehicle Field

Mercedes said, “Although neuromorphic computing is still in its infancy, such systems will soon be on the market within a few years. When applied at scale throughout vehicles, they have the potential to radically reduce the amount of effort required to run the latest AI technologies. power consumption."

BrainChip's CMO Jerome Nadel said: "Mercedes is focused on big issues like battery management and transmission, but every milliwatt counts, and when you think about energy efficiency, even the most basic reasoning, like finding keywords, matters. important."


A typical car could have as many as 70 different sensors by 2022, Nadel said. For cockpit applications, these sensors can enable face detection, gaze assessment, emotion classification, and more.

He said: “From a system architecture perspective, we can do a 1:1 approach where there is a sensor that will do some preprocessing and then the data will be forwarded. The AI will do inference near the sensor...it will Instead of the full array of data from sensors, the inference metadata is passed forward.”

The idea is to minimize the size and complexity of packets sent to AI accelerators, while reducing latency and minimizing power consumption. Each vehicle will likely have 70 Akida chips or sensors with Akida technology, each of which will be "low-cost parts that won't notice them at all," Nadel said. He noted that attention needs to be paid to the BOM of all these sensors.


Application of SNN in Vehicle Field

BrainChip expects to have its neuromorphic processor next to every sensor on the vehicle

Going forward, Nadel said, neuromorphic processing will also be used in ADAS and autonomous driving systems. This has the potential to reduce the need for other types of power-hungry AI accelerators.


"If every sensor could have Akida configured on one or two nodes, it would do adequate inference, and the data passed would be an order of magnitude less, because that would be inference metadata...that would affect the servers you need," he said. power."

BrainChip's Akida chip accelerates SNNs (spike neural networks) and CNNs (by converting to SNNs). It's not tailored for any specific use case or sensor, so it can be paired with visual sensing for face recognition or people detection, or other audio applications like speaker ID. BrainChip also demonstrated Akida's smell and taste sensors, although it's hard to imagine how these could be used in cars (perhaps to detect air pollution or fuel quality through smell and taste).

Akida is set up to handle SNNs or deep learning CNNs that have been transformed into SNNs. Unlike the native spike network, the transformed CNN preserves some spike-level information, so it may require 2 or 4 bits of computation. However, this approach allows exploiting the properties of CNNs, including their ability to extract features from large datasets. Both types of networks can be updated at the edge using STDP. In the case of Mercedes-Benz, this might mean retraining the network after deployment to discover more or different keywords.


Application of SNN in Vehicle Field

According to Autocar, Mercedes-Benz confirmed that "many innovations" from the EQXX concept car, including "specific components and technologies," will be used in the production model. There's no word yet on whether new Mercedes-Benz models will feature artificial brains.

SynSense have their chips without sauce:

SynSense's neuromorphic vision processors have a fully asynchronous digital architecture. Each neuron uses integer logic with 8-bit synaptic weights, 16-bit neuron states, 16-bit thresholds, and unit input-output spikes.

 

[Wickedwolf]

As I mentioned recently, Intel has made a very smart play in offering/accepting Brainchip Inc into their ecosystem.

Loihi maybe fantastic research in the making, but nothing will ever beat owning revolutionary technology that is not only 100%
proven in silicon, but is also available immediately as a commercial offering, whether as in IP block/s or SoC.

I personally believe that they have conceded (finally) that our technology is superior, much more advanced, ready now, cheaper, and moving
further ahead, it will be very interesting over the next 5 years how Intel plays this, I'd watch this space rather closely, but that's just my own view.

Please see the link below..........Tech

Brutal Earnings From Intel Signal Changing of Guard in Chips

(Bloomberg) -- From Intel Corp. to SK Hynix Inc., some of the world’s largest semiconductor makers stunned investors with brutal losses heading into 2023. But two Asian companies — Taiwan Semiconductor Manufacturing Co. and Samsung Electronics Co. — navigated the turmoil with greater agility...

www.yahoo.com

This is huge, and hasn’t got the attention it deserves

 

[Tothemoon24]

SynSense have their chips without sauce:

SynSense's neuromorphic vision processors have a fully asynchronous digital architecture. Each neuron uses integer logic with 8-bit synaptic weights, 16-bit neuron states, 16-bit thresholds, and unit input-output spikes.

Hi Dio , could the secret sauce ingredient be added here

 

[wilzy123]

Went live when?

If you don't have dreams, you can't have dreams come true!

No

But it is accessible via cache

BrainChip Partners with AI Labs on Next-Generation Application Development

BrainChip Partners with AI Labs on Next-Generation Application Development Laguna Hills, Calif. – DATE, 2023 – BrainChip Holdings Ltd (ASX: BRN, OTCQX: BRCHF, ADR: BCHPY), the world’s first commercial producer of ultra-low power, fully digital, event-based, neuromorphic AI IP, today announced...

web.archive.org

 

[DaBenjamins]

AI labs Inc have one asset which is the Minsky Ai Engine, and it is very impressive. I haven't found much about the company other than their LinkedIn info page.
It has 2-10 employees, founded in 2021 and is supposedly a public company ??. I haven't found it on the main exchanges so I don't know the company financials.

Now for my Loooong Bow.
The Minsky Ai Engine is a perfect fit for Akida and can potentially provide an extra fast pathway into markets.
Ai Labs seem small enough for a takeover, possibly to tune of hmmm, how much was the LDA Capital call worth again???

Sweet dreams

Love the way you think VG. Def a possibility considering the terms we picked up JAST for in 2022.

 

[Taproot]

I can’t find anything about the Founder of AI Labs Inc Bhaskar Rao. There is nothing on the AI Labs Inc website about the management team either.

There are others with the same name and I have found one who is an entrepreneur who is possibly the same man as for AI Labs Inc.

He is also Founder and CEO of another company named Technosphere (in some areas referred to as Technosphere Labs so similiar to AI Labs) https://www.technosphere.io/about-us/

https://www.linkedin.com/in/bhaskar-rao-2490087/

View attachment 28691


Anyway I find it really unusual that it is very difficult to find anything on Bhaskar Rao and AI Labs Inc when I search.

Does anyone else have more luck?

OK,
Deleted last three attempts.
We got the wrong guy !

It's this bloke

Bhasker Rao

None

www.enzen.com

With more than 20 years experience of developing and applying emerging technologies to industry, Bhasker is a results-focused business leader with an in-depth understanding of deploying IoT, AI and Blockchain in the utilities and energy sector.

During his career, Bhasker has founded, led and advised companies focused on delivering innovative solutions to their customers, especially in the manufacturing sector. He brings a strong combination of technology, client relationship and commercial skills to any business challenge, plus a passion for innovation and creativity that has seen him develop five patents.

In his current role as CEO of Fortech Energy, he leads the development of enterprise automation and IoT solutions for the power, gas, water and manufacturing industry. Bhasker has a Physics degree from the Indian Institute of Technology in Bombay, plus a PhD in Electrical Engineering from the University at Buffalo in New York.

Here is link to Rob Telson post about Edge impulse official support for Brainchip
Bao Bhasker ( Spelt with an "e" ) responds with a congrats etc.

See Bhasker Rao, Ph.D’s activity on LinkedIn

Sign in or join now to see posts like this one and more.

www.linkedin.com

 

[Taproot]

OK,
Deleted last three attempts.
We got the wrong guy !

It's this bloke

Bhasker Rao

None

www.enzen.com

With more than 20 years experience of developing and applying emerging technologies to industry, Bhasker is a results-focused business leader with an in-depth understanding of deploying IoT, AI and Blockchain in the utilities and energy sector.

During his career, Bhasker has founded, led and advised companies focused on delivering innovative solutions to their customers, especially in the manufacturing sector. He brings a strong combination of technology, client relationship and commercial skills to any business challenge, plus a passion for innovation and creativity that has seen him develop five patents.

In his current role as CEO of Fortech Energy, he leads the development of enterprise automation and IoT solutions for the power, gas, water and manufacturing industry. Bhasker has a Physics degree from the Indian Institute of Technology in Bombay, plus a PhD in Electrical Engineering from the University at Buffalo in New York.

Here is link to Rob Telson post about Edge impulse official support for Brainchip
Bao Bhasker ( Spelt with an "e" ) responds with a congrats etc.

See Bhasker Rao, Ph.D’s activity on LinkedIn

Sign in or join now to see posts like this one and more.

www.linkedin.com

https://www.linkedin.com/in/bhaskerrao/

 

[Terroni2105]

OK,
Deleted last three attempts.
We got the wrong guy !

It's this bloke

Bhasker Rao

None

www.enzen.com

With more than 20 years experience of developing and applying emerging technologies to industry, Bhasker is a results-focused business leader with an in-depth understanding of deploying IoT, AI and Blockchain in the utilities and energy sector.

During his career, Bhasker has founded, led and advised companies focused on delivering innovative solutions to their customers, especially in the manufacturing sector. He brings a strong combination of technology, client relationship and commercial skills to any business challenge, plus a passion for innovation and creativity that has seen him develop five patents.

In his current role as CEO of Fortech Energy, he leads the development of enterprise automation and IoT solutions for the power, gas, water and manufacturing industry. Bhasker has a Physics degree from the Indian Institute of Technology in Bombay, plus a PhD in Electrical Engineering from the University at Buffalo in New York.

Here is link to Rob Telson post about Edge impulse official support for Brainchip
Bao Bhasker ( Spelt with an "e" ) responds with a congrats etc.

See Bhasker Rao, Ph.D’s activity on LinkedIn

Sign in or join now to see posts like this one and more.

www.linkedin.com

Well done Taproot! Fantastic finding, and he looks quite impressive too.

 

[Diogenese]

Hi Dio , could the secret sauce ingredient be added here

Hi Ttm,

As I understand it, the secret sauce was all about using the Spikenet JAST rules and N-of-M coding. Initially this allowed 1-bit activations and weights, and was expanded to 4-bit activations/weights for CNN2SNN.

SpikeNet will be using MACs from last millennium to perform MAC (Multiply Accumulate) calculations using their 8-bit and 16-bit weights and measures. This could use several times more power and be considerably slower than 4-bit Akida ... and that's without accounting for the N-of-M coding. I like to think of MACs as mini-von Neumann bottlenecks.

The Prophesee/SynSense article is from 20211015. Our affair with Prophesee was announced 8 months later.

Luca Verra spoke of the Prophesee/Synsense cooperation as having the potential for further development. He was ebullient about the Akida.

I have speculated that this would be due to the speed of response of SynSense not being able to fully match the response time of the Prophesee DVS pixels (equivalent to 10000 fps?), whereas Akida can handle this.

 

[Tothemoon24]

Hi Ttm,

As I understand it, the secret sauce was all about using the Spikenet JAST rules and N-of-M coding. Initially this allowed 1-bit activations and weights, and was expanded to 4-bit activations/weights for CNN2SNN.

SpikeNet will be using MACs from last millennium to perform MAC (Multiply Accumulate) calculations using their 8-bit and 16-bit weights and measures. This could use several times more power and be considerably slower than 4-bit Akida ... and that's without accounting for the N-of-M coding. I like to think of MACs as mini-von Neumann bottlenecks.

The Prophesee/SynSense article is from 20211015. Our affair with Prophesee was announced 8 months later.

Luca Verra spoke of the Prophesee/Synsense cooperation as having the potential for further development. He was ebullient about the Akida.

I have speculated that this would be due to the speed of response of SynSense not being able to fully match the response time of the Prophesee DVS pixels (equivalent to 10000 fps?), whereas Akida can handle this.

Thank you for the reply Dio , much appreciated

 

[Taproot]

Well done Taproot! Fantastic finding, and he looks quite impressive too.

@Terroni2105
You were absolutely spot on chasing down Bhasker Rao, it's all about the people behind these company names.
This is looking pretty exciting.
Check this out, something to really get your teeth into if you have some spare time.
Bhasker Rao was and still is the founder of Fortech Energy which comes under the umbrella of Enzen Global Limited.
https://www.enzen.com/global/about-enzen/enzen-group-companies/

 

[Boab]

@Terroni2105
You were absolutely spot on chasing down Bhasker Rao, it's all about the people behind these company names.
This is looking pretty exciting.
Check this out, something to really get your teeth into if you have some spare time.
Bhasker Rao was and still is the founder of Fortech Energy which comes under the umbrella of Enzen Global Limited.
https://www.enzen.com/global/about-enzen/enzen-group-companies/

Yes, and the home page for the Australia webpage shows a picture of the Swan River and the city of Perth. Lets catch up for coffee PvdM

 

[buena suerte :-)]

Yes, and the home page for the Australia webpage shows a picture of the Swan River and the city of Perth. Lets catch up for coffee PvdM

Your long lost brother .. .... come and join us on the next get together Boab!

 

[TopCat]

iMedia

Home
Tech
Application of SNN in Vehicle Field

2023-02-03 22:41

iMedia

Home
Tech
Application of SNN in Vehicle Field

2023-02-03 22:41 HKT

Depots are using neuromorphic technologies to implement AI functions such as keyword recognition, driver attention, and passenger behavior monitoring.

Mimicking biological brain processes is tantalizing because it promises to enable advanced functionality without a significant increase in power consumption, which is EV-friendly. Neuromorphic computing and perception are also expected to bring these advantages, such as extremely low latency, enabling real-time decision-making in some cases. This combination of low latency and high energy efficiency is very attractive.

Spike Network


The truth is, there's still something we don't know about how the human brain works. However, cutting-edge research has shown that neurons communicate by sending each other electrical signals called spikes, and that the sequence and timing of the spikes (rather than their size) are key factors. Mathematical models of how neurons respond to these spikes are still being studied. But many scientists agree that if multiple spikes arrive at adjacent neurons at the same time (or in very rapid succession), it means that the information those spikes represent are correlated, thus causing the neuron to fire a spike.

This is in contrast to artificial neural networks based on deep learning (the mainstream AI today), where information travels through the network in a regular rhythm; that is, the information entering each neuron is represented as a numerical value, rather than based on time.

Making a spike-based artificial system is not easy. Besides we don't know how neurons work, there is no consensus on the best way to train spike neural networks. Backpropagation requires computing derivatives, which is not possible with spikes. Some companies approximate the derivative of the spike in order to use backpropagation (like SynSense), and some use another technique called STDP (spike timing dependent plasticity), which is closer to how biological brains function. However, STDP is not yet mature as a technique (BrainChip uses this method for one-shot learning at the edge). It is also possible to take a deep learning CNN, trained by backpropagation in the normal way, and convert it to run in the spike domain (another technique used by BrainChip).


SynSense Speck

SynSense is working with BMW to advance the integration of neuromorphic chips in smart cockpits and explore related areas together. BMW will evaluate SynSense's Speck SoC, which combines SynSense's neuromorphic vision processor and Inivation's 128x128-pixel event camera. Can be used to capture visual information in real-time, identify and detect objects, and perform other vision-based detection and interaction functions.

Dylan Muir, vice president of global research operations at SynSense, said: "When BMW replaces RGB cameras with Speck modules for visual perception, they can replace not only sensors, but also a lot of GPU or CPU computing required to process standard RGB visual streams."


Using event-based cameras provides a higher dynamic range than standard cameras, which is beneficial for use in extreme lighting conditions inside and outside the vehicle.

BMW will explore the use of neuromorphic technology in cars, including monitoring driver attention and passenger behavior through the Speck module.

"In the coming months, we will explore more applications inside and outside the vehicle," Muir said.


SynSense's neuromorphic vision processors have a fully asynchronous digital architecture. Each neuron uses integer logic with 8-bit synaptic weights, 16-bit neuron states, 16-bit thresholds, and unit input-output spikes. Neurons use a simple integrate-and-fire model, where when the neuron fires a simple 1-bit spike, the input spike is combined with the neuron's synaptic weights until a threshold is reached. Overall, the design is a balance between complexity and computational efficiency, Muir said.

Application of SNN in Vehicle Field

SynSense's electronic neurons are based on the integrate-and-fire model


SynSense's digital chips are designed to process event-based CNNs, with each layer processed by a different core. The kernel runs asynchronously and independently; the entire processing pipeline is event-driven.

"Our Speck modules run in real-time with low latency," Muir said. "We can manage effective inference rates above 20Hz at less than 5mW power consumption. This is much faster than using traditional low-power computing on standard RGB video streams. ."

While SynSense and BMW will initially explore neuromorphic use cases in smart cockpits, it has potential for other automotive applications as well.


"First, we'll explore non-safety-critical use cases, and we're planning future versions of Speck with higher resolution, as well as improvements to our DynapCNN vision processor, which will interface with high-resolution sensors," Muir said. We plan for these future technologies It will support advanced automotive applications such as autonomous driving, emergency braking, etc."

Application of SNN in Vehicle Field

SynSense and Inivation Speck module, an event camera-based module containing sensors and processors


BrainChip Akida

Mercedes-Benz's EQXX concept car, which debuted at CES earlier this year, uses BrainChip's Akida neuromorphic processor for in-vehicle keyword recognition. Billed as "the most efficient car Mercedes has ever made," the car utilizes neuromorphic technology that consumes less power than a deep learning-based keyword spotting system. That's crucial for a car with a range of 620 miles, or 167 miles more than Mercedes' flagship electric car, the EQS.

Mercedes said at the time that BrainChip's solution was five to 10 times more efficient than traditional voice controls at recognizing the wake word "Hey Mercedes."


Application of SNN in Vehicle Field

Mercedes said, “Although neuromorphic computing is still in its infancy, such systems will soon be on the market within a few years. When applied at scale throughout vehicles, they have the potential to radically reduce the amount of effort required to run the latest AI technologies. power consumption."

BrainChip's CMO Jerome Nadel said: "Mercedes is focused on big issues like battery management and transmission, but every milliwatt counts, and when you think about energy efficiency, even the most basic reasoning, like finding keywords, matters. important."


A typical car could have as many as 70 different sensors by 2022, Nadel said. For cockpit applications, these sensors can enable face detection, gaze assessment, emotion classification, and more.

He said: “From a system architecture perspective, we can do a 1:1 approach where there is a sensor that will do some preprocessing and then the data will be forwarded. The AI will do inference near the sensor...it will Instead of the full array of data from sensors, the inference metadata is passed forward.”

The idea is to minimize the size and complexity of packets sent to AI accelerators, while reducing latency and minimizing power consumption. Each vehicle will likely have 70 Akida chips or sensors with Akida technology, each of which will be "low-cost parts that won't notice them at all," Nadel said. He noted that attention needs to be paid to the BOM of all these sensors.


Application of SNN in Vehicle Field

BrainChip expects to have its neuromorphic processor next to every sensor on the vehicle

Going forward, Nadel said, neuromorphic processing will also be used in ADAS and autonomous driving systems. This has the potential to reduce the need for other types of power-hungry AI accelerators.


"If every sensor could have Akida configured on one or two nodes, it would do adequate inference, and the data passed would be an order of magnitude less, because that would be inference metadata...that would affect the servers you need," he said. power."

BrainChip's Akida chip accelerates SNNs (spike neural networks) and CNNs (by converting to SNNs). It's not tailored for any specific use case or sensor, so it can be paired with visual sensing for face recognition or people detection, or other audio applications like speaker ID. BrainChip also demonstrated Akida's smell and taste sensors, although it's hard to imagine how these could be used in cars (perhaps to detect air pollution or fuel quality through smell and taste).

Akida is set up to handle SNNs or deep learning CNNs that have been transformed into SNNs. Unlike the native spike network, the transformed CNN preserves some spike-level information, so it may require 2 or 4 bits of computation. However, this approach allows exploiting the properties of CNNs, including their ability to extract features from large datasets. Both types of networks can be updated at the edge using STDP. In the case of Mercedes-Benz, this might mean retraining the network after deployment to discover more or different keywords.


Application of SNN in Vehicle Field

According to Autocar, Mercedes-Benz confirmed that "many innovations" from the EQXX concept car, including "specific components and technologies," will be used in the production model. There's no word yet on whether new Mercedes-Benz models will feature artificial brains.






Depots are using neuromorphic technologies to implement AI functions such as keyword recognition, driver attention, and passenger behavior monitoring.

Mimicking biological brain processes is tantalizing because it promises to enable advanced functionality without a significant increase in power consumption, which is EV-friendly. Neuromorphic computing and perception are also expected to bring these advantages, such as extremely low latency, enabling real-time decision-making in some cases. This combination of low latency and high energy efficiency is very attractive.

Spike Network


The truth is, there's still something we don't know about how the human brain works. However, cutting-edge research has shown that neurons communicate by sending each other electrical signals called spikes, and that the sequence and timing of the spikes (rather than their size) are key factors. Mathematical models of how neurons respond to these spikes are still being studied. But many scientists agree that if multiple spikes arrive at adjacent neurons at the same time (or in very rapid succession), it means that the information those spikes represent are correlated, thus causing the neuron to fire a spike.

This is in contrast to artificial neural networks based on deep learning (the mainstream AI today), where information travels through the network in a regular rhythm; that is, the information entering each neuron is represented as a numerical value, rather than based on time.

Making a spike-based artificial system is not easy. Besides we don't know how neurons work, there is no consensus on the best way to train spike neural networks. Backpropagation requires computing derivatives, which is not possible with spikes. Some companies approximate the derivative of the spike in order to use backpropagation (like SynSense), and some use another technique called STDP (spike timing dependent plasticity), which is closer to how biological brains function. However, STDP is not yet mature as a technique (BrainChip uses this method for one-shot learning at the edge). It is also possible to take a deep learning CNN, trained by backpropagation in the normal way, and convert it to run in the spike domain (another technique used by BrainChip).


SynSense Speck

SynSense is working with BMW to advance the integration of neuromorphic chips in smart cockpits and explore related areas together. BMW will evaluate SynSense's Speck SoC, which combines SynSense's neuromorphic vision processor and Inivation's 128x128-pixel event camera. Can be used to capture visual information in real-time, identify and detect objects, and perform other vision-based detection and interaction functions.

Dylan Muir, vice president of global research operations at SynSense, said: "When BMW replaces RGB cameras with Speck modules for visual perception, they can replace not only sensors, but also a lot of GPU or CPU computing required to process standard RGB visual streams."


Using event-based cameras provides a higher dynamic range than standard cameras, which is beneficial for use in extreme lighting conditions inside and outside the vehicle.

BMW will explore the use of neuromorphic technology in cars, including monitoring driver attention and passenger behavior through the Speck module.

"In the coming months, we will explore more applications inside and outside the vehicle," Muir said.


SynSense's neuromorphic vision processors have a fully asynchronous digital architecture. Each neuron uses integer logic with 8-bit synaptic weights, 16-bit neuron states, 16-bit thresholds, and unit input-output spikes. Neurons use a simple integrate-and-fire model, where when the neuron fires a simple 1-bit spike, the input spike is combined with the neuron's synaptic weights until a threshold is reached. Overall, the design is a balance between complexity and computational efficiency, Muir said.

Application of SNN in Vehicle Field

SynSense's electronic neurons are based on the integrate-and-fire model


SynSense's digital chips are designed to process event-based CNNs, with each layer processed by a different core. The kernel runs asynchronously and independently; the entire processing pipeline is event-driven.

"Our Speck modules run in real-time with low latency," Muir said. "We can manage effective inference rates above 20Hz at less than 5mW power consumption. This is much faster than using traditional low-power computing on standard RGB video streams. ."

While SynSense and BMW will initially explore neuromorphic use cases in smart cockpits, it has potential for other automotive applications as well.


"First, we'll explore non-safety-critical use cases, and we're planning future versions of Speck with higher resolution, as well as improvements to our DynapCNN vision processor, which will interface with high-resolution sensors," Muir said. We plan for these future technologies It will support advanced automotive applications such as autonomous driving, emergency braking, etc."

Application of SNN in Vehicle Field

SynSense and Inivation Speck module, an event camera-based module containing sensors and processors


BrainChip Akida

Mercedes-Benz's EQXX concept car, which debuted at CES earlier this year, uses BrainChip's Akida neuromorphic processor for in-vehicle keyword recognition. Billed as "the most efficient car Mercedes has ever made," the car utilizes neuromorphic technology that consumes less power than a deep learning-based keyword spotting system. That's crucial for a car with a range of 620 miles, or 167 miles more than Mercedes' flagship electric car, the EQS.

Mercedes said at the time that BrainChip's solution was five to 10 times more efficient than traditional voice controls at recognizing the wake word "Hey Mercedes."


Application of SNN in Vehicle Field

Mercedes said, “Although neuromorphic computing is still in its infancy, such systems will soon be on the market within a few years. When applied at scale throughout vehicles, they have the potential to radically reduce the amount of effort required to run the latest AI technologies. power consumption."

BrainChip's CMO Jerome Nadel said: "Mercedes is focused on big issues like battery management and transmission, but every milliwatt counts, and when you think about energy efficiency, even the most basic reasoning, like finding keywords, matters. important."


A typical car could have as many as 70 different sensors by 2022, Nadel said. For cockpit applications, these sensors can enable face detection, gaze assessment, emotion classification, and more.

He said: “From a system architecture perspective, we can do a 1:1 approach where there is a sensor that will do some preprocessing and then the data will be forwarded. The AI will do inference near the sensor...it will Instead of the full array of data from sensors, the inference metadata is passed forward.”

The idea is to minimize the size and complexity of packets sent to AI accelerators, while reducing latency and minimizing power consumption. Each vehicle will likely have 70 Akida chips or sensors with Akida technology, each of which will be "low-cost parts that won't notice them at all," Nadel said. He noted that attention needs to be paid to the BOM of all these sensors.


Application of SNN in Vehicle Field

BrainChip expects to have its neuromorphic processor next to every sensor on the vehicle

Going forward, Nadel said, neuromorphic processing will also be used in ADAS and autonomous driving systems. This has the potential to reduce the need for other types of power-hungry AI accelerators.


"If every sensor could have Akida configured on one or two nodes, it would do adequate inference, and the data passed would be an order of magnitude less, because that would be inference metadata...that would affect the servers you need," he said. power."

BrainChip's Akida chip accelerates SNNs (spike neural networks) and CNNs (by converting to SNNs). It's not tailored for any specific use case or sensor, so it can be paired with visual sensing for face recognition or people detection, or other audio applications like speaker ID. BrainChip also demonstrated Akida's smell and taste sensors, although it's hard to imagine how these could be used in cars (perhaps to detect air pollution or fuel quality through smell and taste).

Akida is set up to handle SNNs or deep learning CNNs that have been transformed into SNNs. Unlike the native spike network, the transformed CNN preserves some spike-level information, so it may require 2 or 4 bits of computation. However, this approach allows exploiting the properties of CNNs, including their ability to extract features from large datasets. Both types of networks can be updated at the edge using STDP. In the case of Mercedes-Benz, this might mean retraining the network after deployment to discover more or different keywords.


Application of SNN in Vehicle Field

According to Autocar, Mercedes-Benz confirmed that "many innovations" from the EQXX concept car, including "specific components and technologies," will be used in the production model. There's no word yet on whether new Mercedes-Benz models will feature artificial brains.

“SynSense is working with BMW to advance the integration of neuromorphic chips in smart cockpits and explore related areas together. BMW will evaluate SynSense's Speck SoC, which combines SynSense's neuromorphic vision processor and Inivation's 128x128-pixel event camera. Can be used to capture visual information in real-time, identify and detect objects, and perform other vision-based detection and interaction functions.”

I wonder what the details of the partnership are? Who is working with BMW the most…SynSense or IniVation…or both equally? If IniVation were to release a better SoC, say their Aeveon SoC, and SynSense isn’t involved with that technology, what would BMW do?

 

[Learning]

Learning 🏖

 

[Steve10]

Interesting that Edge Impulse is a Nvidia Inception Premier member.

What Is NVIDIA Inception?​

NVIDIA Inception is a free program designed to help startups evolve faster through access to cutting-edge technology and NVIDIA experts, opportunities to connect with venture capitalists, and co-marketing support to heighten your company’s visibility.

Program Benefits​

Unlike traditional accelerators, NVIDIA Inception supports all stages of a startup’s life cycle. We work closely with members to provide the best technical tools, latest resources, and opportunities to connect with investors. As your startup matures, your program benefits also evolve to further your growth. Premier members receive increased NVIDIA marketing support, access to Premier-only member events, and a dedicated NVIDIA relationship manager.

Get Started​

You’re encouraged to apply to NVIDIA Inception no matter what your current funding stage is. There are no application deadlines, cohorts, or term limits.


NVIDIA Inception Premier Members

North America

Edge Impulse​

Enabling the ultimate development experience for machine learning on embedded devices for sensors, audio, and computer vision at scale.

 

[alwaysgreen]

Sure….

News has been out for 5 mins and we’re already picking them apart. Get a grip.

In terms of lightening up - I’m a retired 34yo.

Eat a dick hahahahaha

I made a joke about a tech company and you reacted like I was talking about your mum.

Despite nobody asking, please keep telling me how amazing you are though. I'm all ears.

 

[MrNick]

I see they’ve released Andrew Tate.

 

[FJ-215]

@FJ-215 Well?

Very, thanks for asking.

How about you? Good weekend I trust.