I suggested Northrop Grumman as one of those that provided a letter. The following link and the video supports this proposition particularly the video when the second last presenter speaks about the speed of operation when operating autonomously:
The future of life in space starts right here—with the Lunar Terrain Vehicle, a mobile capable of traversing the harsh environments of planets and their moons while preserving human life.
www.northropgrumman.com
My opinion and speculation only so DYOR
FF
AKIDA BALLISTA
Nice one FF, Thanks
I was just on the Northrop Grumman website & I put keyword "Akida" in the search bar & I got a hit but then when I read the article there was no Akida to be found, still an interesting article though talking about ML & Neuromorphic Cameras, changing pixels etc
From enhanced battlefield protection systems to maintaining aerial drone delivery fleets, neuromorphic cameras hold promise for the future.
www.northropgrumman.com
Neuromorphic Cameras Provide a Vision of the Future
From enhanced battlefield protection systems to maintaining aerial drone delivery fleets, neuromorphic cameras hold promise for the future.
- Home
- What We Do
- Aerospace Digital Transformation
- Neuromorphic Cameras Provide a Vision of the Future
Tweet this article
Post on Reddit
Share on LinkedInShare
By Scott Gourley
From enhanced
battlefield protection systems to maintaining aerial drone delivery fleets, neuromorphic cameras have the potential to enhance many future defense, commercial and industrial tasks. When coupled with machine learning, this technology may soon set the stage for dramatic enhancements in how systems operate and how we perceive and understand surrounding environments.
Like the Human Eye
“Cameras we use today have an array of pixels: 1024 by 768,” explained Isidoros Doxas, an AI Systems Architect at Northrop Grumman. “And each pixel essentially measures the amount of light or number of photons falling on it. That number is called the flux. Now, if you display the same numbers on a screen, you will see the same image that fell on your camera.”
By contrast, neuromorphic cameras only report changes in flux. If the rate of photons falling on a pixel doesn’t change, they report nothing.
“If a constant 1,000 photons per second is falling on a pixel, it basically says, ‘I’m good, nothing happened.’ But, if at some point there are now 1,100 photons per second falling on the pixel, it will report that change in flux,” Doxas said.
“Surprisingly, this is exactly how the human eye works,” he added. “You may think that your eye reports the image that you see. But it doesn’t. All that stuff is in your head. All the eye reports are little blips saying ‘up’ or ‘down.’ The image we perceive is built by our brains.”
You need a lot of energy to send a number from pixel to computer or from eye to brain. And you don’t want to spend all that energy. In fact, that’s why people started thinking about building neuromorphic cameras. They require much less energy because the pixels just report changes and not actual values.”
Isidoros Doxas
Northrop Grumman AI Systems Architect
Advantages of Neuromorphic Cameras
Doxas identified several advantages in neuromorphic imaging, beginning with reduced power requirements.
“You need a lot of energy to send a number from pixel to computer or from eye to brain,” he said. “And you don’t want to spend all that energy. In fact, that’s why people started thinking about building neuromorphic cameras. They require much less energy because the pixels just report changes and not actual values.”
He continued, “Another important advantage is speed. If you have a million pixels, and you have to send the computer a thousand frames per second, that’s one billion numbers per second. However, usually nothing changes in a scene from one millisecond to the next, so you don’t need to report that entire image.”
Doxas likened the process to compression methods for video entertainment, noting that a 4K-resolution movie represents 8 million pixels, times three colors, times 30 frames per second.
“That’s over a gigabyte per second,” he said. “Yet, you can watch that over an internet connection that’s only a few megabits per second. That’s because little changes from one frame to the next. They leverage that fact and compress frames in the same way. The difference here is that neuromorphic cameras do the compression.”
Decompression is accomplished by computers, where reporting speeds accelerating from one to tens of thousands of frames per second allow for millisecond reaction times. This paves the way for a huge range of different applications — from active combat systems detecting and defeating a bullet, to self-driving cars interpreting dangerous situations almost instantly.
Doxas said that Northrop Grumman has been involved with the technology for more than a decade, highlighting a collaboration with Johns Hopkins University that resulted in the recent design of a readout integrated circuit as well as “the brains that go behind that circuitry.”
Future efforts will include increasing the number of pixels, further lowering power and increasing resolution.
Coupling with Machine Learning
Optimizing the
new camera technology involves the application of
machine learning methods that can work directly with the photon plus and minus signals. With self-driving cars, for example, machine learning can construct an image of a cat or dog with just a few pixel pluses or minuses, resulting in much quicker decisions compared with images built by convolutional neural network sensors.
“
In the same vein, Northrop Grumman can use this system for non-invasive diagnostics for high-speed parts. This technology will dramatically change power requirements and time to decision across any number of applications,” Doxas concluded.
