Wonder if these guys may become a reseller or at least stock the upcoming VVDN Akida Edge AI Box?
Recent blog on their site indicating Akida as a possible neuromorphic option in edge boxes.
They have a small online shop with several Aaeon AI Edge Boxes using Jetson at the mo.
I still feel BRN "engaged" VVDN to create the box for the mkt as a real world POC.
From what I've seen, VVDN create products or partner with companies like Ambarella, NVIDIA, TI, Intel, NXP, Kinara, AMD and even Renesas (camera).
edgeainodes.com
edgeainodes.com
Edge AI Box Inference Options
An important element to research and evaluate in Edge AI projects is inference workload performance, which consists of model size, speed of each inference (usually measured in milliseconds), or frames per second for video applications. There are various model frameworks such as YOLO, MobileNet, and many others that underpin a model, and of course the model parameters and features, number of items being classified, and other factors can increase the model size. But given a static or constant neural network, the speed at which it runs is dependent upon the hardware being used. There is slower, cheaper hardware, and there is faster, more expensive hardware.
However,
where the model is run on the hardware, can vary. Inference can be performed directly on the device’s CPU cores, can be run on a GPU for parallel processing,
or could be offloaded to a custom AI accelerator chip such as a Tensor Processing Unit (TPU) like the one in a
Google Coral,
neuromorphic processors like Brainchip Akida, or other dedicated math and matrix multipliers designed for loading and processing AI models. Most Edge AI boxes have one or more options available, or could be upgraded or configured with added GPU or Accelerator options if there is enough expansion capability.
Here is some general information on each option, and a bit of guidance to help you choose the right solution for your AI project.
CPU – Using the device’s native processor is usually the easiest and simplest form of running an AI model, though it
might also be the slowest. This is useful for beginning to explore Edge AI, as it is typically quick and easy to load a model and begin to understand sensor data, image classification, object detection, or audio classification. If you are using small devices like single board computers (such as the Raspberry Pi or similar), this may be your only option to perform inferencing, as expansion, size and power constraints, or processing power may be your limiting factor. Once deployed and tested, if the performance is adequate and the project’s
use case or needs are being met, there may not be any reason to worry about the increases that can be achieved with GPU’s or custom accelerators. To keep power consumption down, save on costs, and in smaller projects, CPU inferencing could be the best solution.
GPU – Offloading machine learning models to graphics processing units (GPU) will in most cases speed up inferencing of complex algorithms, but exact performance should be tested and benchmarked. GPU’s come in many shapes and sizes. Some are integrated into the CPU, like Iris or Xe graphics cores on Intel processors, or Mali on may Arm SoC’s, while Jetson SoC’s will contain Nvidia GPU cores. Edge AI boxes could also contain a standalone GPU connected via PCIe or MXM, with Nvidia GeForce or A-class GPUs or perhaps AMD Radeon devices. GPUs come in a wide range of cost and performance options, and power needs. Keep in mind that with large GPUs, several hundred watts could be required, which may or may not be possible depending upon the the end destination / location of the device. High power GPUs can also be expensive, so a thorough analysis of cost versus actual performance increases should be considered.
AI Accelerator – Custom silicon solutions dedicated to running machine learning models also exist, and could be used in Edge AI projects. This type of hardware is generally added by plugging a device into a USB port or PCIe slot, and again come in various performance and price points, though in some cases can be integrated directly into a PCB. The accelerators have varying intended model frameworks that they are best suited for loading and running, so you’ll need to make sure your model is suitable and can actually benefit from the device and your inference performance will indeed increase. This might require some testing using a development kit or sample unit. Here again, because the performance options and price points vary, you will need to evaluate the performance gain and do a cost/benefit analysis to determine if it’s worthwhile. However, dedicated AI Accelerators might also enable features or functionality that would not otherwise be possible, such as event-based tracking where objects are tracked through space and time (for example a golf swing or path of a ball), aggregated video streams and camera frames, or other specific capabilities not possible (or with feasible performance) on CPUs and GPUs.
Posted
October 14, 2023
in
Blog
by
admin
. 
