Wickedwolf
Regular
Jim Cramer almost makes the motley fools look professional
BRN Discussion Ongoing
- Thread starter TechGirl
- Start date
Fullmoonfever
Top 20
Was having a look at the recent TinyML conference which we know Anil was at for the Sept one.
This one is the EMEA Oct one and noticed that one of our Engineers was doing a Demo / Poster as below fwiw as not sure if posted before.
www.tinyml.org
Demos & Posters
Exploiting activation sparsity for easy model optimization in the Akida processor
Sébastien CROUZET, R&D Engineer, Brainchip
Abstract (English)
Neuromorphic processors offer high processing efficiency, but at the cost of an apparent technical barrier for the new user. BrainChip’s Akida processor mitigates this obstacle by running standard convolutional neural networks, and by offering transparent and performant quantization and lossless conversion methods. Neuromorphic efficiency is retained because the event–based processor can exploit the sparsity (high number of zero–values) inherently present in CNN activation maps.
Nonetheless, selection and optimization of models to target Akida (or indeed, any dedicated hardware) can seem daunting without expert knowledge. Here we show that i) by applying simple off–the–shelf optimization techniques, such as regularization and pruning, we can significantly accelerate inference at minimal reduction to accuracy; ii) applying these techniques provides a powerful lever to manipulate the speed/accuracy trade–off according to task requirement; iii) this in turn reduces the requirement to maintain an extensive library of pretrained models to cover a range of task complexities – instead, a handful of pretrained models allows a continuum of final model sizes.
[Method] We select a representative and well–known benchmark task, visual wakewords (VWW), starting from a tiny–ML relevant solution, MobileNet V1, alpha=0.25 (as per the tinyML Perf benchmark). We use a transfer learning–based pipeline, with imagenet pretraining, and quantization of both weights and activations to 4–bits (first layer 8–bit weights), 5 epochs of fine tuning on VWW. We test 2 techniques to optimize performance on our hardware: i) activity regularization added during training (mix of L1 and L2 loss), testing model accuracy and sparsity as a function of the amplitude of regularization ii) structured pruning (whole filters), using a well–known metric for filter selection (batch
normalization gamma). We first test the “prunability” of each layer individually by evaluating model accuracy (without re–training) as a function of % filters pruned. We then test several pruning strategies for the whole model (fixed % accuracy loss per layer; targeted pruning of the high–cost layers; …).
[Results] We first introduce a minor network optimization from the original MobileNet based on known properties of Akida: for low filter numbers (<=64), the efficient implementation of standard convolutional layers in Akida means that they run as fast as an equivalently sized separable convolution, while offering a moderately increased representational capacity and thus accuracy. Specifically, the first 3 separable conv layers of MobileNet v1 are swapped for standard conv layers, forming what we will call “AkidaNet”. With quantization at 4–bits, and using the alpha=0.25 network, we obtain a baseline accuracy of 83.3% on the VWW validation set, with single–sample (batch–size = 1) processing rate of 66 FPS (latency 15.15 ms). Applying a combination of regularization and pruning to optimize this model, we can accelerate processing to 133 FPS while maintaining >80% accuracy. We then test optimization from larger starting models (alpha=0.5 or even 1.0) and find that we can reduce models sufficiently to achieve processing rates similar to the baseline, but with higher accuracy (a=0.5, 84.1% acc. at 84 FPS; a=1.0, 84.8% acc. at 69 FPS)
Also see that TDK InvenSense playing with minimal Bit Weight Quantization as well.
Towards Universal 1-bit Weight Quantization of Neural Networks with end-to-end deployment on ultra-low power sensors
Le MINH TRI, Ph.D. student in deep learning, TDK InvenSense
This one is the EMEA Oct one and noticed that one of our Engineers was doing a Demo / Poster as below fwiw as not sure if posted before.
EMEA 2022 | tinyML Foundation
TinyML Summit. The topic is advances in ultra-low power Machine Learning technologies and applications.
Demos & Posters
Exploiting activation sparsity for easy model optimization in the Akida processor
Sébastien CROUZET, R&D Engineer, Brainchip
Abstract (English)
Neuromorphic processors offer high processing efficiency, but at the cost of an apparent technical barrier for the new user. BrainChip’s Akida processor mitigates this obstacle by running standard convolutional neural networks, and by offering transparent and performant quantization and lossless conversion methods. Neuromorphic efficiency is retained because the event–based processor can exploit the sparsity (high number of zero–values) inherently present in CNN activation maps.
Nonetheless, selection and optimization of models to target Akida (or indeed, any dedicated hardware) can seem daunting without expert knowledge. Here we show that i) by applying simple off–the–shelf optimization techniques, such as regularization and pruning, we can significantly accelerate inference at minimal reduction to accuracy; ii) applying these techniques provides a powerful lever to manipulate the speed/accuracy trade–off according to task requirement; iii) this in turn reduces the requirement to maintain an extensive library of pretrained models to cover a range of task complexities – instead, a handful of pretrained models allows a continuum of final model sizes.
[Method] We select a representative and well–known benchmark task, visual wakewords (VWW), starting from a tiny–ML relevant solution, MobileNet V1, alpha=0.25 (as per the tinyML Perf benchmark). We use a transfer learning–based pipeline, with imagenet pretraining, and quantization of both weights and activations to 4–bits (first layer 8–bit weights), 5 epochs of fine tuning on VWW. We test 2 techniques to optimize performance on our hardware: i) activity regularization added during training (mix of L1 and L2 loss), testing model accuracy and sparsity as a function of the amplitude of regularization ii) structured pruning (whole filters), using a well–known metric for filter selection (batch
normalization gamma). We first test the “prunability” of each layer individually by evaluating model accuracy (without re–training) as a function of % filters pruned. We then test several pruning strategies for the whole model (fixed % accuracy loss per layer; targeted pruning of the high–cost layers; …).
[Results] We first introduce a minor network optimization from the original MobileNet based on known properties of Akida: for low filter numbers (<=64), the efficient implementation of standard convolutional layers in Akida means that they run as fast as an equivalently sized separable convolution, while offering a moderately increased representational capacity and thus accuracy. Specifically, the first 3 separable conv layers of MobileNet v1 are swapped for standard conv layers, forming what we will call “AkidaNet”. With quantization at 4–bits, and using the alpha=0.25 network, we obtain a baseline accuracy of 83.3% on the VWW validation set, with single–sample (batch–size = 1) processing rate of 66 FPS (latency 15.15 ms). Applying a combination of regularization and pruning to optimize this model, we can accelerate processing to 133 FPS while maintaining >80% accuracy. We then test optimization from larger starting models (alpha=0.5 or even 1.0) and find that we can reduce models sufficiently to achieve processing rates similar to the baseline, but with higher accuracy (a=0.5, 84.1% acc. at 84 FPS; a=1.0, 84.8% acc. at 69 FPS)
Also see that TDK InvenSense playing with minimal Bit Weight Quantization as well.
Towards Universal 1-bit Weight Quantization of Neural Networks with end-to-end deployment on ultra-low power sensors
Le MINH TRI, Ph.D. student in deep learning, TDK InvenSense
Kachoo
Regular
Sometimes there are project that have lump some payments and the support is free. You have to remember that this tech is really new to many. That said I will say when BRN give employees shares for milestones what merit are they based on I wish revenue was a measure to pay out employees too. But I don't know there contracts either. Disappointed yes.
Fullmoonfever
Top 20
EOD slightly wider view to engage the bottom possible supp zones.
jtardif999
Regular
Don’t be frustrated, think of it as an opportunity the shorters are providing you with to buy at a down down price
Thanks Xhosa,
If I have read your explanation correctly that means on quarter ending Sept there might be new revenue but not reported yet as the 4 C only reports cash receipt and not the revenue. If that is the case I think it is not an issue for me, my only concern is there is no revenue during that quarter.
Deadpool
Did someone say KFC
@Fact Finder is it possible for you to channel the Fourth Estate my way, it would make my life so much easier.
On the other hand, can Brainchip put in the revenue in the 4C if they wish to do so?
Rise from the ashes
Regular
Yak 52 would tear you another asshole for that commentDon’t be frustrated, think of it as an opportunity the shorters are providing you with to buy at a down down price
Anyway hard to stay positive for the time being. But chin up for those who are feeling it hard. Take a look around you and be grateful for what you have, loving family, good health. In the end that's all that matters.
Violin1
Regular
They are getting slower year by year and because Commsec won't provide direct access to account info then you do a little bit more coding now but they are efficient, cost-effective and their help info is really good. Don't expect to talk to anyone though - you need to be able to follow instructions on your own to do the coding. I'll continue using them on experience over a few years so far.
Food4 thought
Regular
Hey it really is only a number at the moment. I am not selling so makes no a great deal of difference, unless the BRN team have stuffed up completely then I will start to get concerned maybe even crap myself , but until that can be confirmed I will hold very tight.Hang in there mate
2025 is looking very positive
Deadpool
Did someone say KFC
Wish I was a zoomer. Have a good night and remember to stay out of trouble
JK200SX
Regular
I remember attending a Peter Thornehill seminar quite a few years ago and a comment he made has stuck in my head. The comment was along the lines of, "the market(share price) reacts rationally in a perfectly irrational way...". The SP drop today was purely based on fear, nothing else. The fundamentals of the company have only improved over time. The same goes when the the SP rockets astronomically over a short period of time, and that is fuelled by greed.
Below I've attached a few of my screenshots of the BRN SP over the years. If you can link all of these daily extreme price changes to "material" changes or price sensitive announcements from the company then you probably should be on this forum - perhaps you should get a job at BRN HQ
Its all good, and we've invested in a great company that will deliver!
Deadpool
Did someone say KFC
Intelligence I reckonSomeone has faith.
wilzy123
Founding Member
I can tell you "get it". Well done to you and your conviction. 100% agree that 2025 is looking positive.
wilzy123
Founding Member
Literally nobody cares. James Mickleboro is a Motley Fool puppet that has very limited career prospects.
ForrestofHope
Regular
Hi @Diogenese and @factfinder @uiux and any other long termer.
Have there been any previous dots connecting Brainchip with Logitech? I don’t recall reading anything.
I was trolling through Rob Telson’s Linkedin account and he ‘liked’ a Logitech product:
Logitech announces Sight, an AI-powered tabletop camera for Hybrid Meetings
I know a ‘random’ like by itself doesn’t mean anything. Who knows, maybe he ‘liked’ it because he wants to buy one for the office to help facilitate zoom meetings.
However, I came across an article from October 2019 co-written by Giorgia Dellaferrera of IBM and Miloš Cerňak of Logitech.
The article discusses the validity of using SNN architecture for VAD (Voice Activity Detection system). It references Loihi on a few occasions but no reference to AKIDA.
This is the conclusion:
4. CONCLUSION We propose a SNN architecture for VAD. We developed a novel method for encoding the filterbank single frame features into spike patterns. We next integrated the new encoding scheme into a simple bilayer architecture which we evaluated on the QUT-NOISE-TIMIT data set. We showed that exploiting the power-efficiency properties of SNNs enables the design of VAD systems working at constant low power and achieving performances comparable to the state-of-the art methods. Our framework exhibits a low latency and relies on relatively small training data.
file:///C:/Users/g_rho/Downloads/A_Bin_Encoding_Training_of_a_Spiking_Neural_Networ.pdf
In another article she writes about SNN and back-propogation, so I don’t think she was using AKIDA. I haven’t found any connection to Brainchip but was curious if Logitech may have seen the light and started using AKIDA in the past 2 years. I’m wondering if there is any significance to Rob’s ‘like’, SNN, and Logitech. Are there any patents from Logitech using SNN?
I would appreciate any insights. Thanks in advance.
Similar threads
