Yes. I saw that and wondered, as you did, if it referred to Thorpe's N-of-M coding.
N-of-M coding is directly tied to the arrival time of spikes from different "pixel" neurons although I would guess it also applies to auditory and other sensory neurons.
To get to N-of-M coding, there was a two step process. The first step was in realizing that the then "orthodox" spike rate coding was inefficient and error-prone. When a nerve fires, it transmits a burst of decreasing pulses, the pulse rate being a measure of the strength of the signal. Thorpe realized that measuring the rate was inefficient mainly because it involved using a redundant secondary source of the spike information, and it was error-prone because neuron firing at rates of less than 1/10th of a second is unreliable. As I've mentioned before, Thorpe found that the initial spike of a spike burst from an optical nerve carried all the necessary information.
Thorpe also noticed that the firing time was inverse to the amplitude of the excitatory energy - the stronger the signal, the sooner the nerve fired.
Hence the switch from rate coding to spike time coding.
So that leaves all M spikes from M neurons still being processed, which brings us to the second step.
The next step was the realization that the later-arriving spikes added little to the accuracy of the visual detection. In other words, accurate detection could be carried out from the first N pulses to arrive - N-of-M coding.
As you can see, Thorpe's N-of-M coding is all about spikes which arrive asynchronously.
I could find nothing in the Renesas article or in their patents suggesting the use of spikes.
This recent Renesas patent application uses MACs.
US2024054083A1 SEMICONDUCTOR DEVICE 20220808
View attachment 58151
A
semiconductor device capable of shortening processing time of a neural network is provided. The memory stores a compressed weight parameter. A plurality of multiply accumulators perform a multiply-accumulation operation to a plurality of pixel data and a plurality of weight parameters. A decompressor restores the compressed weight parameter stored in the memory to a plurality of weight parameters. A memory for weight parameter stores the plurality of weight parameters restored by the decompressor. The DMA controller transfers the plurality of weight parameters from the memory to the memory for weight parameter via the decompressor. A sequence controller writes down the plurality of weight parameters stored in the memory for weight parameter to a weight parameter buffer at write timing.
That said, I haven't found out what their N:M refers to, and there is still 18 months of unpublished patent applications.
And the latest, dunno if they’ve perhaps upgraded the DRP with some extra sauce…
Renesas Unveils Powerful Single-Chip RZ/V2H MPU for Next-Gen Robotics with Vision AI and Real-Time Control
New Generation AI Accelerator with 10 TOPS/W Power Efficiency Delivers AI Inference Performance of up to 80 TOPS Without Cooling Fan
February 29, 2024, 8:00 AM Eastern Standard Time
TOKYO--(
BUSINESS WIRE)--Renesas Electronics Corporation (TSE:6723), a premier supplier of advanced semiconductor solutions, has expanded its popular RZ Family of microprocessors (MPUs) with a new device targeting high-performance robotics applications.
This press release features multimedia. View the full release here:
https://www.businesswire.com/news/home/20240229979997/en/
Offering the highest levels of performance within the family, the RZ/V2H enables both vision AI and real-time control capabilities.
The device comes with a new generation of Renesas proprietary AI accelerator, DRP (Dynamically Reconfigurable Processor)-AI3, delivering 10 TOPS/W power efficiency, an impressive 10-fold improvement over previous models. Additionally, pruning technology employed in the DRP-AI3 accelerator significantly improves AI computing efficiency, boosting AI inference performance up to 80 TOPS. This performance boost allows engineers to process vision AI applications directly at edge AI devices without relying on cloud computing platforms. The details of the new
DRP-AI3 acceleration technology were recently presented at the International Solid-State Circuits Conference (ISSCC 2024) in San Francisco.
The RZ/V2H incorporates four Arm® Cortex®-A55 CPU cores with a maximum operating frequency of 1.8 GHz for Linux application processing, two Cortex-R8 cores running at 800 MHz for high-performance real-time processing, and one Cortex-M33 as a sub core. By integrating these cores into a single chip, the device can effectively manage both vision AI and real-time control tasks, making it ideal for demanding robotics applications of the future. Since the RZ/V2H consumes less power, it eliminates the need for cooling fans and other heat-dissipating components. This means engineers can design systems that are smaller in size, less expensive, and more reliable.
“As a market leader in motor control microprocessors, Renesas is ready to take on the next challenge to drive the advancement of the robotics market with AI technology,” said Daryl Khoo, Vice President of the Embedded Processing 1st Business Division at Renesas. “The RZ/V2H will facilitate the development of next-generation autonomous robots with vision AI capabilities, that have the ability to think independently and control movements in real time."
Renesas has applied its proprietary
DRP technology to develop the
OpenCV Accelerator that speeds up the processing of OpenCV, an open-source industry standard library for computer vision processing. The resulting speed improvement is up to 16 times faster compared to CPU processing. The combination of the DRP-AI3 and the OpenCV Accelerator enhances both AI computing and image processing algorithms, enabling the power-efficient, real-time execution of Visual SLAM(Note 1) used in applications such as robot vacuum cleaners.
To accelerate development, Renesas also released AI Applications, a library of pre-trained models for various use cases, and the AI SDK (Software Development Kit) for rapid development of AI applications. By running these software on the RZ/V2H's evaluation board, engineers can evaluate AI applications easily and earlier in the design process, even if they do not have extensive knowledge of AI.
“We are thrilled to be part of the launch of the RZ/V2H, which combines AI technology with real-time control,” says Rolf Segger, founder of SEGGER Microcontroller GmbH. “SEGGER’s J-Link debug probe, widely adopted by numerous embedded development projects globally, will provide the support needed for the RZ/V2H, helping accelerate the development of next-generation robotic innovations. We look forward to this next phase in our multi-decade long partnership with Renesas."
Winning Combinations
Renesas has developed the "
Visual Detection Single Board Computer" that uses camera images to identify its surroundings, and to determine and control its movements in real-time. This solution combines the RZ/V2H with power management ICs and VersaClock programmable clock generators to support power-efficient industrial robots and machinery. Its efficient design eliminates the requirement for an additional cooling fan, keeping the solution BOM and size down. These Winning Combinations are technically vetted system architectures from mutually compatible devices that work together seamlessly to bring an optimized, low-risk design for faster time to market. Renesas offers more than 400 Winning Combinations with a wide range of products from the Renesas portfolio to enable customers to speed up the design process and bring their products to market more quickly. They can be found at
renesas.com/win.
Availability
The RZ/V2H is available today, along with the evaluation board and the AI SDK. More information about the device and development tools are available at:
https://www.renesas.com/rzv2h.
About Renesas Electronics Corporation
Renesas Electronics Corporation (
TSE: 6723) empowers a safer, smarter and more sustainable future where technology helps make our lives easier. The leading global provider of microcontrollers, Renesas combines our expertise in embedded processing, analog, power and connectivity to deliver complete semiconductor solutions. These Winning Combinations accelerate time to market for automotive, industrial, infrastructure and IoT applications, enabling billions of connected, intelligent devices that enhance the way people work and live. Learn more at
renesas.com. Follow us on
LinkedIn,
Facebook,
X,
YouTube, and
Instagram.
(Note 1) Visual SLAM (Simultaneous Localization and Mapping) is a technology that analyzes images captured by on-board cameras on robots and drones and estimates their own position while simultaneously creating detailed maps of their surroundings.
(Remarks) This DRP-AI technology uses a part of the results of work commissioned by the New Energy and Industrial Technology Development Organization (NEDO). Arm, Arm Cortex are trademarks or registered trademarks of Arm Limited in the EU and other countries. All names of products or services mentioned in this press release are trademarks or registered trademarks of their respective owners.
View source version on businesswire.com:
https://www.businesswire.com/news/home/20240229979997/en/
Contacts
Media Contacts:
Americas
Akiko Ishiyama
Renesas Electronics Corporation
+ 1-408-887-9006
akiko.ishiyama.xf@renesas.com
