SW-F put me onto Roger Levinson's analog adventure:
https://www.eetimes.com/blumind-harnesses-analog-for-ultra-low-power-intelligence/
Canadian startup Blumind recently developed an analog computing architecture for ultra-low power AI acceleration of sensor data, Blumind CEO Roger Levinson told EE Times. The company hopes to enable widespread intelligence in Internet of Things (IoT) devices.
Advanced process nodes aren’t cost effective for tiny chips used in tens of hundreds of millions of units in the IoT. Combine this with the fragmentation of the IoT market, the need for application-specific silicon, and the requirement for zero additional power consumption and it’s easy to see why the IoT has been slow to adopt AI, Levinson said.
“The request from our customer was: I need to lower my existing power, add no cost, and add intelligence to my system,” he said. “That isn’t possible, but how close to zero [power consumption] can you get? We’re adding a piece to the system, so we have to add zero to the system power, and cost has to be negligible, and then we have a shot. Otherwise, they’re not going to add intelligence to the devices, they’re going to wait, and that’s what’s happening. People are waiting.”
This is the problem Blumind is taking on. Initial development work on efficient machine learning (ML) at ultra-low power by Blumind cofounder and CTO John Gosson forms the basis for the startup’s architecture today.
“John said, ‘What you need to do is move charge around as the information carrier, and not let it go between power supplies’,” Levinson said. “Energy [consumption] happens when charge moves from the power supply to ground, and heat is generated. So he built an architecture [around that idea] which is elegant in its simplicity and robustness.”
Like some of its competitors in the ultra-low power space, Blumind is focusing on analog computing.
“We’ve solved the system-level always-on problem by making it all analog,” he said. “We look like a compute in memory architecture because we use a single transistor to store coefficients for the network, and that device also does the multiplication.”
The transistor’s output is the product of the input and the stored weight; the signal integrates for a certain amount of time, which generates a charge proportional to that product. This charge is then accumulated on a capacitor. A proprietary scheme measures the resulting charge and generates an output proportional to it which represents the activation.
“Everything is time based, so we are not looking at absolute voltages or currents,” he said. “All our calculations are ratiometric, which makes us insensitive to process, voltage and temperature. To maintain analog dynamic range, we do have to compensate for temperature, so even though the ratios remain stable, the magnitudes of signals can change.”
Levinson said Blumind has chosen to focus on “use cases that are relevant to the world today”—keyword spotting and vision—partly in an effort to prove to the market analog implementations of neural networks are viable in selected use cases.
Blumind test silicon. (Source: Blumind)
“
One of the biggest challenges has been, does it have to be software configurable, or not?” he said. “Our first architecture is not configurable in terms of the network—we build a model in silicon, which happens to robust for the class of applications we’re going after, and is orders of magnitude more power and area efficient.”
Model weights are adjustable, but everything else is fixed. However, this is enough flexibility to cater for a class of problems, Levinson said.
“The layers are fixed, the neurons and synapses are fixed,” he said. “We’re starting with audio because our [customer] wants an always-on voice system. However, our silicon is capable of doing anything that can utilize a recurrent neural network.”
Blumind’s software stack supports customer training of the recurrent neural network (RNN) its silicon is designed for with customers’ own data.
This strategy helps minimize power consumption, but it means a separate tapeout for every new class of application Blumind wants to target. Levinson said that at legacy 22-nm nodes, the cost of an analog/mixed-signal tapeout is a little over $1 million, and requires a team of just five to eight people.
In tinyML today, the performance difference from changing models is minor, he argues.
“There is a hard limit at the edge, especially in sensors,” he said. “I have X amount of memory and X amount of compute power, and a battery. The data scientist has to fit the model within these constraints.”
Blumind has test chips for its first product, the RNN accelerator designed for keyword spotting, voice activity detection and similar time series data applications. This silicon achieves 10 nJ per inference; combined with feature extraction, it consumes a few microwatts during always-on operation. The chip also includes an audio data buffer (required for the Amazon Echo specification) within “single digit microwatts,” Levinson said.
Blumind’s chip connects directly to an analog microphone for input, and sends a wake up signal to an MCU when it detects a keyword. The current generation requires weight storage in external non-volatile memory, but Blumind plans to incorporate that in future devices.
View attachment 57407
Tapeout for the commercial version of the RNN accelerator is underway.
Blumind’s also currently bringing up test silicon of a convolutional neural network (CNN) accelerator it’s designed for vision applications in its lab, which it plans to demonstrate this summer. The target is object detection, such as person detection, at up to 10 fps using 5-20 µW, depending on configuration, Levinson said.
The company’s also working with an academic partner on a software-definable version of its analog architecture for future technology generations.
First samples of Blumind’s RNN accelerator are due in Q3.
Having fixed layers and synapses designed according to each customers data means designing a new tapeout for each customer - a mere bagatelle according to Levinson. $1M a pop.
I wonder about accuracy. This is for low hanging fruit which is not safety-critical, so there may be a market for ultra-low power near-enuf-is-good-enuf NNs.
.PS: Roger's looking pretty ripped, so don't tell him I said this.