And this presumably after the ARM IPO.Nandan Nayampally will be one of the speakers at tinyML Asia 2023 in Seoul:
View attachment 44221
Thursday November 16, 2023
9:00 am to 9:15 am
Opening / Welcome
9:15 am to 10:00 am
Keynote - TBA
10:00 am to 10:20 am
Solutions and Applications
Brainchip’s Specialized Akida Architecture for Low Power Edge Inference with HW Spatiotemporal Acceleration and On-chip Learning
Nandan NAYAMPALLY, CMO, Brainchip
Abstract (English)
Brainchip’s Innovative Akida v2.0 Architecture
- Event-based, neuromorphic processing
- Extremely efficient computation only processes and communicates on events resulting in minimal latency and power consumption. Hardware accelerates event domain convolutions, temporal convolutions, and vision transformers while consuming very low power for battery-operated devices.
- At-memory compute
- Low-latency and low-power computation via parallel execution of events propagating through a mesh-connected array of Neural Processing Units (NPUs) each with a dedicated SRAM memory.
- Quantized parameters and activations
- Supports 8, 4, 2, 1-bit parameters and activations on layer-by-layer basis for model implementations tailored for optimum accuracy versus size, power, and latency.
- Accelerates Traditional AI models
- The MetaTF software stack quantizes and converts models in Keras/TensorFlow and Pytorch/ONNX to Akida event-driven models.
- Fully-digital Implementation
- Portable across process technology with standard synthesis design flow.
- Plus Novel Architectural Features below …
Select Architectural Highlights
Brainchip’s patented algorithm for homeostatic Spike Time Dependent Plasticity (STDP) enables on-device learning. AI applications can implement incremental learning so that models can adapt to changes in the field. On-device learning can add new classes to a model while leveraging features learned in its original, off-line training.
- Temporal Event-based Neural Networks (TENNs)
- Radically reduces number of parameters and operations by an order of magnitude or more for spatiotemporal computations with 2D streaming input such as video and 1D streaming input such as raw audio. Easily trained with back propagation like a CNN and inferences like a RNN. Extracts 2D spatial and temporal 1D kernels for 3D input and extracts temporal 1D kernels for 1D input.
- Vision Transformers (ViTs)
- Multi-head attention included in hardware implementation of Transformer Encoder block with configurations from 2 to 12 nodes. A two-node configuration at 800 MHz executes TinyViT at 30 fps (224x224x3).
- On-device learning
Model Development Methodologies
- No-code Model Development and Deployment with Edge Impulse Users with limited AI expertise can develop and deploy optimized Akida models.
- Build-Your-Own-Model with Brainchip’s MetaTF Experienced AI users can build and optimize custom Akida models.
Results
- Video Benchmarks
- Audio and Medical Vital Signs Benchmarks
- Image Benchmarks
Emphasis on this: (Availability of v2.0)