BRN Discussion Ongoing

Diogenese

Top 20

Realistic Retinas Make Better Bionic Eyes​

Following nature’s example more closely could lead to better visual sensors​

EDD GENT
23 MAR 2022
shape of eye with different colored dots
ISTOCKPHOTO
New visual sensors inspired by the human eye could help the blind see again and provide powerful new ways for machines to sense the world around them. Recent research shows that more faithfully copying nature’s hardware could be the key to replicating its powerful capabilities.
Efforts to build bionic eyes have been underway for several decades, with much of the early research focused on creating visual prostheses that could replace damaged retinas in humans. But in recent years, there’s been growing recognition that the efficient and adaptable way in which the eye processes information could prove useful in applications where speed, flexibility, and power constraints are major concerns. Among these are robotics and the Internet of things.
Now, a pair of new research papers describe significant strides toward replicating some of the eye's capabilities by more closely imitating the function of the retina—the collection of photoreceptors and neurons at the back of the eye responsible for converting light into visual signals for the brain. “It’s a very exciting extension from where we were before,” says Hongrui Jiang, a professor of engineering at the University of Wisconsin–Madison. “These two papers [explain research aimed at] trying to mimic the natural visual system’s performance, but on the retina level right at the signal-filtering and signal-processing stage.”
One of the most compelling reasons to do this is the retina’s efficiency. Most image sensors rely on components called photodiodes, which convert light into electricity, says Khaled Salama, professor of electrical and computer engineering at King Abdullah University of Science and Technology (KAUST), but photodiodes constantly consume electricity, even when they’re on standby, which leads to high energy use. In contrast, the photoreceptors in the retina are passive devices that convert incoming light into electrical signals that are then sent to the brain. In an effort to recreate this kind of passive light-sensing capability, Salama’s KAUST team turned to an electrical component that doesn’t need a constant source of power—the capacitor.
“The problem is that capacitors are not sensitive to light,” says Salama. “So we decided to embed a light-sensitive material inside the capacitor.” The team sandwiched a layer of perovskite—a material prized for its electrical and optical properties—between two electrodes to create a capacitor whose ability to store energy, or capacitance, changed in proportion to the intensity of the light to which it is exposed. The researchers found that the resulting device mimicked the characteristics of the rod-cell photoreceptors found in the retina.
To see if the devices they created could be used to make a practical image sensor, the team fabricated a 100-by-100 array of them, then wired them up to simple circuits that converted the sensors’ change in capacitance into a string of electrical pulses, similar to the spikes of neural activity that rod cells use to transmit visual information to the brain. In a paper published in the February issue of Light: Science & Applications, they showed that a special kind of artificial neural network could learn how to process these spikes and recognize handwritten numbers with an accuracy of roughly 70 percent.

An array of the bio-inspired sensors produces strings of electrical pulses in response to light, which are then processed by a spiking neural network.

An array of the bio-inspired sensors produces strings of electrical pulses in response to light, which are then processed by a spiking neural network.DR. MANI TEJA VIJJAPU/KING ABDULLAH UNIVERSITY OF SCIENCE AND TECHNOLOGY

Thanks to its incredibly low energy requirements, Salama says future versions of the KAUST team’s bionic eye could be a promising solution for power-constrained applications like drones or remote camera systems. “The best application for something like this is security, because often nothing is happening,” he says. “You are wasting a lot of power to take images and take videos and process them to figure out that there is nothing happening.”
Another powerful capability of our eyes is the ability to rapidly adapt to changing light conditions. Image sensors can typically operate only within a limited range of illuminations, says Yang Chai, an associate professor of materials science at the Hong Kong Polytechnic University. Because of this, they require complex workarounds like optical apertures, adjustable exposure times, or complex postprocessing to deal with varying real-world light conditions. By contrast (pun intended), when you transition from a dark cinema hall to a brightly lit lobby, it takes only a short while for your eyes to adjust automatically. That’s thanks to a mechanism known as visual adaptation, in which the sensitivity of photoreceptors changes automatically depending on the level of illumination.
In an effort to mimic that adaptability, Chai and his colleagues designed a new kind of image sensor whose light sensitivity can be modulated by applying different voltages to it. In a paper in Nature Electronics, his team showed that an array of these sensors could operate over an even broader range of illuminations than the human eye. They also paired the array with a neural network and showed that the system’s ability to recognize handwritten numbers improved drastically as the sensors adapted, going from 9.5 percent to 96.1 percent accuracy as it adjusted to bright light and 38.6 percent to 96.9 percent as it adjusted to darkness. These capabilities could be very useful for machines that have to operate in a wide range of lighting conditions. One application for which it will be quite helpful, says Yang, is in a self-driving car, which has to keep track of its position with respect to other objects on the road as it enters and exits a dark tunnel.
While there’s still a long way before bionic eyes approach the capabilities of their biological cousins, Jiang says the kinds of in-sensor adaptation and signal processing achieved in these papers show why researchers should be paying more attention to the finer detail of how the retina achieves its impressive capabilities. “The retina is an amazing organ,” he says. “We’re only just scratching the surface.”
Thanks FF,

The capacitive perovskite light sensor is brilliant, but, despite citing Masqualier & Thorpe (69), by using spike rate coding they failed to take full advantage of STDP with N-out-of-M coding. Mind you, they only had a small sensor array and a similarly sparse (single layer) SNN.

https://www.nature.com/articles/s41377-021-00686-4#Sec9
Herein, we demonstrate the light intensity capacitive photoreceptor (CPR) that mimics the retina’s rod cells. The capacitance of CPRs is dependent on visible light illumination and can lead to the development of the artificial retina by integrating with peripheral electronics9. To fabricate CPRs with excellent light tunable properties, we require materials that are photosensitive and materials that have tendencies to tune the dielectric properties. In order to obtain the combination of exceptional optoelectronic and ferroelectric properties, we prepared a hybrid composite of methylammonium lead bromide perovskite (MAPbBr3) and the terpolymer polyvinylidene fluoride trifluoroethylene-chlorofluoroethylene (PVDF-TrFE-CFE). We demonstrated the fabrication and characterization of flexible CPR, which has a frequency-dependent capacitance within the range of 1−100 kHz.

The hybrid perovskites as fillers in the ferroelectric polymer attribute to modulate the dielectric properties proportional to the intensity and wavelength of the incident light. The capacitive change with respect to the wavelength of the incident light mimics the spectral sensitivity curve of human photopic vision with the maximum response in the greenish-yellow regime. The photoresponse of these CPRs is reproducible with negligible hysteresis. Furthermore, the fabricated device is resistive to humidity and oxygen due to the encapsulation of the hybrid perovskites in the hydrophobic ferroelectric terpolymer (FP). To the best of our knowledge, we report the longest stability measurement of hybrid perovskites (~129 weeks) owing to their PVDF-TrFE-CFE encapsulation.

The proposed device is modeled with an RC network and integrated with a novel low power spike oscillator to generate the spike train with a firing rate proportional to the incident light intensity and wavelength (color). Then, the functionality of the proposed CPR and sensing circuit is demonstrated through simulation to recognize the handwritten digits (MNIST) dataset using an unsupervised trained spiking neural network
.
...
This SNN is a single-layer network with 100 output neurons employing a winner-take-all (WTA) mechanism followed by a statistical output classifier. The network was trained with simplified spike-timing-dependent plasticity, STDP, with a leaky integrate-and-fire neuron model69,70.

1650204982157.png
 
  • Like
  • Love
Reactions: 14 users

Slade

Top 20
Don't know if its been discussed but I think we may be connected to a mob called Autobrains. The connection being that a big part of their tech relies on Renesas' R-Car.


The video on the home page of autobrains is worth watching.

I find this PDF interesting as on the second page it lists the patents pending that autobrains have:

 
  • Like
  • Fire
Reactions: 18 users

Slade

Top 20
Autobrains Cartex 4H introduces self-learning AI and will be available Q3 2022.
Remember the Cartex 4H is based on a Renesas V3H architecture
No date given for the release of the Catex 5M that promises to
bridge the gap towards a fully autonomous vehicle

1650216712141.png


1650216840182.png

 
  • Like
  • Love
  • Fire
Reactions: 27 users

Slade

Top 20
  • Like
  • Fire
  • Love
Reactions: 29 users
D

Deleted member 118

Guest
Rocket has really been putting you to work lately FF 😂

Yes and I’m so glad he can speed read

 
  • Haha
  • Like
Reactions: 11 users
For me everything is pointing to Akida IP being in Renesas's R-Car V4H.

Samples of the R-Car V4H SoC are available now, with mass production scheduled for the second quarter of 2024.​

Great research Slade.

Reading their website you provided they certainly reads like they could have Akida inside.

Based in Israel is interesting as I recall there was a company there supplying Brainchip products. I assumed it was because of Nanose.

Here’s their “listed” partners, and I recall Continental being mentioned before (LDN) and likely have an EAP.

Toyota were involved with the Denso link with AM.

Looks promising.

1650232811434.jpeg


Cheers
 
Last edited:
  • Like
  • Fire
Reactions: 22 users

Justchilln

Regular
Has anyone heard if Akida 500 is still on track for this year?
 
  • Like
  • Fire
Reactions: 5 users
Has anyone heard if Akida 500 is still on track for this year?
Hi JC
Somebody said so if I had a guess it would be Ken Scarince in the recent presentation he did filling in for Sean Hehir.

Also be sure to check out @Fullmoonfever’s thread ‘Hit List Leads’.

@Stable Genius that BMW logo just keeps turning up - I am sure it is just a coincidence that it was a BMW in the Brainchip video for Hey AKIDA to throw us off the Hey Mercedes track.

Then again if we finally accept that AKIDA is the one and only first in the world commercially available SCNN chip and the only one offering chip or IP who the bloody else could it actually be???

You really would have to be a WANCA to propose it was not AKIDA if it has ultra low power, on chip convolution, on chip incremental and one shot learning with and without connection it is now just so ……. obvious.

My opinion only DYOR
FF

AKIDA BALLISTA
 
  • Like
  • Fire
  • Love
Reactions: 38 users

Learning

Learning to the Top 🕵‍♂️
  • Like
  • Fire
Reactions: 8 users

Sirod69

bavarian girl ;-)
that would be nice i come from bavaria, but till now i can´t find something about BMW and Brainchip
 
  • Like
  • Love
  • Fire
Reactions: 10 users
that would be nice i come from bavaria, but till now i can´t find something about BMW and Brainchip
Agree no one has found anything but BMW just keeps popping up. We will just keep looking and maybe we will eventually find the link.

Maybe you could rent an apartment near the factory and buy an AKIDA board and set up a camera trained to recognise Brainchip employees and send an alert when one is detected entering the BMW factory. 😂🤣

My opinion only DYOR
FF

AKIDA BALLISTA
 
  • Haha
  • Like
  • Fire
Reactions: 27 users
For me everything is pointing to Akida IP being in Renesas's R-Car V4H.

Samples of the R-Car V4H SoC are available now, with mass production scheduled for the second quarter of 2024.​

I cannot find anything in the documents that jumps out at me as being AKIDA IP nodes.

We are told Renesas are bringing out MCU’s with AKIDA IP on board. The documentation in your links states this eliminates the need for MCU’s.

I personally think we are looking in the wrong direction here but it is just one opinion.

Renesas did not licence the full AKIDA IP stack.

My opinion only DYOR
FF

AKIDA BALLISTA
 
  • Like
Reactions: 12 users

Sirod69

bavarian girl ;-)
Stimmen Sie zu, dass niemand etwas gefunden hat, aber BMW taucht einfach immer wieder auf. Wir werden einfach weiter suchen und vielleicht finden wir irgendwann den Link.

Vielleicht könnten Sie eine Wohnung in der Nähe der Fabrik mieten und ein AKIDA-Board kaufen und eine Kamera aufstellen, die darauf trainiert ist, Brainchip-Mitarbeiter zu erkennen und eine Warnung zu senden, wenn einer beim Betreten der BMW-Fabrik entdeckt wird.😂🤣

Meine Meinung nur DYOR
FF

AKIDA-BALLISTA
so nice really but i hope i finde anotherway to find news i don´t want to live in munic, we will see, i live next to the bavarian lake
 
  • Like
  • Love
  • Haha
Reactions: 9 users
so nice really but i hope i finde anotherway to find news i don´t want to live in munic, we will see, i live next to the bavarian lake
That is the marvel of AKIDA you can keep your water view and let AKIDA remotely monitor at virtually no power cost. You could even sublet the apartment to someone else on the condition they do not touch or move the camera. 😂🤣😂 FF

You might even get some pictures of the latest BMW that you could sell to a motoring magazine. 😂🤣😂 FF
 
  • Like
  • Haha
Reactions: 9 users

Slade

Top 20
I cannot find anything in the documents that jumps out at me as being AKIDA IP nodes.

We are told Renesas are bringing out MCU’s with AKIDA IP on board. The documentation in your links states this eliminates the need for MCU’s.

I personally think we are looking in the wrong direction here but it is just one opinion.

Renesas did not licence the full AKIDA IP stack.

My opinion only DYOR
FF

AKIDA BALLISTA
Time will tell. Meanwhile I have woken with something else that is puzzling me. We see that Ford is no longer appearing on BrainChip presentations. I think that this is because they are the one company that Sean recently acknowledged as not pursuing Akida directly via BrainChip Inc anymore (Pure speculation). I say directly because I think it may actually makes more sense for Ford to purchase Akida through Valeo or Renesas. This way they cannot only be secretive about their use of Akida but they also end up acquirIng Akida from those that expertise in the automotive industry, supported of course by BrainChip engineers.
It is difficult trying to figure how the different players fit together. I have certainly developed an appreciatIon, or more a sense of awe, for the many products and forms in which Akida could be used. I guess when it comes to BrainChip there will always be dots to try and join.
 
  • Like
  • Thinking
  • Love
Reactions: 28 users

Potato

Regular


Not sure if this has been posted but its quite cool to watch. Enjoy & Happy Easter Monday.

10x tomorrow when the market opens.
 
  • Like
  • Fire
  • Love
Reactions: 25 users

Justchilln

Regular
Hi JC
Somebody said so if I had a guess it would be Ken Scarince in the recent presentation he did filling in for Sean Hehir.

Also be sure to check out @Fullmoonfever’s thread ‘Hit List Leads’.

@Stable Genius that BMW logo just keeps turning up - I am sure it is just a coincidence that it was a BMW in the Brainchip video for Hey AKIDA to throw us off the Hey Mercedes track.

Then again if we finally accept that AKIDA is the one and only first in the world commercially available SCNN chip and the only one offering chip or IP who the bloody else could it actually be???

You really would have to be a WANCA to propose it was not AKIDA if it has ultra low power, on chip convolution, on chip incremental and one shot learning with and without connection it is now just so ……. obvious.

My opinion only DYOR
FF

AKIDA BALLISTA
Thanks for the tip!
Everyone needs to chk out full moon fevers thread Hit list leads!
 
  • Like
  • Love
Reactions: 5 users
I cannot find anything in the documents that jumps out at me as being AKIDA IP nodes.

We are told Renesas are bringing out MCU’s with AKIDA IP on board. The documentation in your links states this eliminates the need for MCU’s.

I personally think we are looking in the wrong direction here but it is just one opinion.

Renesas did not licence the full AKIDA IP stack.

My opinion only DYOR
FF

AKIDA BALLISTA
Lest anyone think I have decided to downramp I have brought over my post from FMF’s thread just now:

Something that occurred to me just a moment ago. Of the confirmed engagements Ford is the only one he takes credit for and does not take credit for any of the following:

1. Mercedes Benz
2. Valeo
3. Socionext
4. Renesas
5. MegaChip
6. NaNose
7. Noisey Gut Belt
8. NASA
9. Vorago
10. ISL
11. Intellisense, or
12. Biotome

The former CEO Mr. Dinardo stated publicly that they had signed “well North of 100 NDA’s”.

Some of the phrases used to describe the Brainchip opportunity:

“Insatiable demand” “disrupting industries that do not exist” “explosive sales” “saturate the market” “ ubiquitous” “become the standard” “two to three year lead” “that’s your margin of victory” “5 to 10 times more efficient than conventional solutions” “It’s not enough to be 30% better, it needs to be five to 10 [times] better,” see FU Bacon “stay at the forefront”

Just three of the public endorsements of Brainchip by Global Industrial players:

SOCIONEXT:

“As a leading provider of ASICs worldwide, we are pleased to offer our customers advanced technologies driving new innovations,” said Noriaki Kubo, Corporate Executive Vice President of Socionext Inc. “The Akida family of products allows us to stay at the forefront of the burgeoning AI market. BrainChip and Socionext have successfully collaborated on the Akida IC development and together, we aim to commercialize this product family and support our increasingly diverse customer base.”​

MEGACHIPS:
“As a trusted and loyal partner to market leaders, we deliver the technology and expertise they need to ensure products are uniquely designed for their customers and engineered for ultimate performance,” said Tetsuo Hikawa, President and CEO of MegaChips. “Working with BrainChip and incorporating their Akida technology into our ASIC solutions service, we are better able to handle the development and support processes needed to design and manufacture integrated circuits and systems on chips that can take advantage of AI at the Edge.”

MERCEDES BENZ:
“Working with California-based artificial intelligence experts BrainChip, Mercedes-Benz engineers developed systems based on BrainChip’s Akida hardware and software,” Mercedes noted in a statement describing the Vision EQXX. “The example in the Vision EQXX is the “Hey Mercedes” hot-word detection. Structured along neuromorphic principles, it is five to ten times more efficient than conventional voice control,” the carmaker claimed…

My opinion only DYOR
FF

AKIDA BALLISTA in 2022
 
Last edited:
  • Like
  • Fire
  • Love
Reactions: 43 users
I cant help but keep coming back to the Brainchip - Valeo (lidar) connection.



These are old articles but no matter how many times I read them and how critical a position take I always come away convinced that it is only a matter of time. tick, tick tick.... Boom
 
  • Like
  • Fire
  • Love
Reactions: 19 users
Top Bottom