BRN Discussion Ongoing

[Bravo]

Intel NPU Library v1.2 Adds Int4 Support & Performance Optimizations​

Written by Michael Larabel in Intel on 5 June 2024 at 06:20 AM EDT. 4 Comments

Intel released a new version of its NPU Acceleration Library, the user-space Python library for leveraging the Neural Processing Unit (NPU) found within their Core Ultra "Meteor Lake" laptops and upcoming Lunar Lake and Arrow Lake hardware as well.

The Intel NPU Library makes it easy to exploit the potential of the NPU hardware on systems with the IVPU kernel driver present. Thanks to this library, in just a few lines of Python code it's possible to perform matrix multiplication on the NPU, compiling PyTorch models for the NPU, and even run Tiny Llama on the NPU.

With today's v1.2 update, there is now int4 support plumbed into this library. In addition to now working with int4 data types, the Intel NPU Library 1.2 offers new backend performance optimizations, Scalar Dot Production Attention (SDPA) NPU kernel support, and persistent compilation handling.

Downloads and more details on the Intel NPU Library 1.2 release via GitHub.

Intel NPU Library v1.2 Adds Int4 Support & Performance Optimizations - Phoronix

www.phoronix.com

 

[manny100]

I been wondering the same.

It's actually worrying if none of the big companies will try buying brainchip, they don't really consider them a treat if that's the case.

Okay Nvidia isn't only edge but got really many things going on, like working for M.B..

BRN will not receive buyout proposals IMO until there is more certainy regarding the arrival of AI at the Edge and real growth appears.
To many uncertainties until then which is reflected in the SP at the moment.
Once Edge growth becomes exponential eyebrows will be raised and BRN will become a target.
If edge growth was known to be verysoon and known to be exponential very soon then the SP would right now be many, many times its current level.

 

[overpup]

Intel NPU Library v1.2 Adds Int4 Support & Performance Optimizations​

Written by Michael Larabel in Intel on 5 June 2024 at 06:20 AM EDT. 4 Comments

Intel released a new version of its NPU Acceleration Library, the user-space Python library for leveraging the Neural Processing Unit (NPU) found within their Core Ultra "Meteor Lake" laptops and upcoming Lunar Lake and Arrow Lake hardware as well.

The Intel NPU Library makes it easy to exploit the potential of the NPU hardware on systems with the IVPU kernel driver present. Thanks to this library, in just a few lines of Python code it's possible to perform matrix multiplication on the NPU, compiling PyTorch models for the NPU, and even run Tiny Llama on the NPU.

With today's v1.2 update, there is now int4 support plumbed into this library. In addition to now working with int4 data types, the Intel NPU Library 1.2 offers new backend performance optimizations, Scalar Dot Production Attention (SDPA) NPU kernel support, and persistent compilation handling.

Downloads and more details on the Intel NPU Library 1.2 release via GitHub.

Intel NPU Library v1.2 Adds Int4 Support & Performance Optimizations - Phoronix

www.phoronix.com

Maybe they read Peter's article?

 

[7für7]

BRN will not receive buyout proposals IMO until there is more certainy regarding the arrival of AI at the Edge and real growth appears.
To many uncertainties until then which is reflected in the SP at the moment.
Once Edge growth becomes exponential eyebrows will be raised and BRN will become a target.
If edge growth was known to be verysoon and known to be exponential very soon then the SP would right now be many, many times its current level.

I want to thank you and the others here for always providing objective counterarguments and having the patience to stand up to the trolls and bashers in the HC forum. I can't manage it and quickly digress... It's due to my southern European temperament... even if one tries to stay calm. Keep it up!


Best regards,
7

 

[Diogenese]

No, Chris Jones started working for Meta in June 2022, see my post and his LinkedIn profile.




That’s exactly why I wrote the following:



But has there ever been any indication that Meta was indeed an EAP customer at the time? Any announcement as with other EAP customers? If not, we shouldn’t simply assume so.

But for the sake of the discussion, let’s assume for a minute it was indeed the case.


In practice, the overlapping period regarding Chris Jones working for Meta and his introduction to BrainChip and TENNs would have been much shorter, though, given that Chris Jones said on May 23, 2024

:

Saying “about a year ago” on May 23, 2024 could mean July, June, May, April, possibly even March 2023. He certainly wouldn’t have put it that way if he and his colleagues had already been introduced to BrainChip in let’s say October 2022. And since he appears to have been laid off in mid-April, the potential time window shrinks to a maximum of six weeks, I’d say. That’s far from the nine months you claimed.

Your use of the participle “outplaced” instead of “laid off” implies that Meta would have helped him to find his current job? Again, there is no indication of that at all when you read his LinkedIn post, especially the last paragraph:

View attachment 65061


Mind you, I did not say there is no way that Chris Jones could have found out about TENNs while still working for Meta, but to me his words are certainly not conclusive evidence, the way FF presented them. They can very well be interpreted differently, especially with the background knowledge that he was laid off more than a year ago (which he didn’t mention in the video). I had already taken notice of that a while ago, when I had had a look at his LinkedIn profile after learning that he would be the one giving the talk Nandan Nayampally was supposed to have presented. (This was even before we found out from the Quarterly Investor podcast that Nandan and Rob had all of a sudden left the company.)

So no, mine is not a tendentious reading and I am not shooting myself in the foot either, if that is what you meant to say. My argument is well-founded. I don’t exclude the possibility that Chris Jones got introduced to BrainChip while still working for Meta, but I believe it is the unlikelier sequence of events for the reasons stated.

Also: Why would he have asked his LinkedIn network for assistance in finding a new job in his April 2023 LinkedIn post and only started working for BrainChip in October 2023? If he had already been that excited about our company prior to being laid off at Meta, they might even have been able to offer him a new position from August onwards, a smooth transition from Meta to BrainChip without a paycheck missing. Of course I have no idea whether it was possibly a deliberate decision of Chris Jones to pick October as the start date for his new job (maybe he wanted to spend quality time with his family between jobs, go on a long vacation, rest and recharge, renovate the house or perhaps he was suffering from an illness, was taking care of elderly relatives or was grieving for a loved one etc) or whether there was simply no earlier job vacancy for his position at BrainChip, but the two month gap between jobs could just as well signify that he didn’t yet know about BrainChip’s offerings by the time he started looking for a new job and that they were possibly not even his first choice.

Ultimately, everything - and that includes FF’s reading - is speculation, unless we hear it from the horse’s mouth. Can we at least agree on that?

But has there ever been any indication that Meta was indeed an EAP customer at the time? Any announcement as with other EAP customers? If not, we shouldn’t simply assume so.

Absence of evidence is not evidence of absence. There would be an NDA and it is possible that Meta insisted on secrecy about the collaboration.

I did not assume Meta was an EAP. I thought there was a reasonable inference to be made based on the available evidence.


I have trouble understanding your 6 week window. It seems to be unsupported conjecture.

Chris Jones said

“So, about a year ago, uh, I was at Meta, I was in their AI Infrastructure Group, and on an almost daily basis I would see new neural network architectures.

So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first. As I started getting to understand the benchmarks and a little bit more of the math and how it worked, I started to get pretty excited by what they had.”

Your contention is that this means "on an almost daily basis (in the last 6 weeks) of my time at Meta, I would see new NNs."

... and then, ... and then ... and then ... (along came Jones), some time after he left Meta, someone introduced him to TeNNs while he was on gardening leave.

I find that quite implausible.

A possible alternative interpretation is "Up until about a year ago, I was at Meta, and on an almost daily basis I would see new NNs."

" So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first."

Does that sound like someone on gardening leave, or someone with the weary or arrogant disdain of the "not-invented-here" attitude nurtured in a mega-corporation.

Chris jones was introduced to Brainchip, and they told him about TeNNs. So was the Akida marketing group going around talking to random gardeners in the off-chance they wished to lay out a lazy $5M for a licence?

The more we discuss this, the more I am inclined towards the possibility of Meta being an EAP. Mr Ed is galloping this way.

 

[7für7]

Will we see a cockroach move? Let’s see…

 

[Iseki]

But has there ever been any indication that Meta was indeed an EAP customer at the time? Any announcement as with other EAP customers? If not, we shouldn’t simply assume so.

Absence of evidence is not evidence of absence. There would be an NDA and it is possible that Meta insisted on secrecy about the collaboration.

I did not assume Meta was an EAP. I thought there was a reasonable inference to be made based on the available evidence.


I have trouble understanding your 6 week window. It seems to be unsupported conjecture.

Chris Jones said

“So, about a year ago, uh, I was at Meta, I was in their AI Infrastructure Group, and on an almost daily basis I would see new neural network architectures.

So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first. As I started getting to understand the benchmarks and a little bit more of the math and how it worked, I started to get pretty excited by what they had.”

Your contention is that this means "on an almost daily basis (in the last 6 weeks) of my time at Meta, I would see new NNs."

... and then, ... and then ... and then ... (along came Jones), some time after he left Meta, someone introduced him to TeNNs while he was on gardening leave.

I find that quite implausible.

A possible alternative interpretation is "Up until about a year ago, I was at Meta, and on an almost daily basis I would see new NNs."

" So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first."

Does that sound like someone on gardening leave, or someone with the weary or arrogant disdain of the "not-invented-here" attitude nurtured in a mega-corporation.

Chris jones was introduced to Brainchip, and they told him about TeNNs. So was the Akida marketing group going around talking to random gardeners in the off-chance they wished to lay out a lazy $5M for a licence?

The more we discuss this, the more I am inclined towards the possibility of Meta being an EAP. Mr Ed is galloping this way.

Would you even call SNN's a Neural Network architecture?
Isn't the architecture all about how many hidden layers there are, and how many nodes in each layer?

 

[Smoothsailing]

Maybe they read Peter's article?
View attachment 65087

Fingers crossed

 

[IloveLamp]

Kevin O'Donovan on LinkedIn: #aiwithpurpose #industrialai #aiwithpurpose #siemensailab #industrialai…

#AIWithPurpose : "We've seen that in #IndustrialAI, the game is a little different. To start with, we don't have room for hallucinations, and we have a large…

www.linkedin.com

 

[Diogenese]

But has there ever been any indication that Meta was indeed an EAP customer at the time? Any announcement as with other EAP customers? If not, we shouldn’t simply assume so.

Absence of evidence is not evidence of absence. There would be an NDA and it is possible that Meta insisted on secrecy about the collaboration.

I did not assume Meta was an EAP. I thought there was a reasonable inference to be made based on the available evidence.


I have trouble understanding your 6 week window. It seems to be unsupported conjecture.

Chris Jones said

“So, about a year ago, uh, I was at Meta, I was in their AI Infrastructure Group, and on an almost daily basis I would see new neural network architectures.

So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first. As I started getting to understand the benchmarks and a little bit more of the math and how it worked, I started to get pretty excited by what they had.”

Your contention is that this means "on an almost daily basis (in the last 6 weeks) of my time at Meta, I would see new NNs."

... and then, ... and then ... and then ... (along came Jones), some time after he left Meta, someone introduced him to TeNNs while he was on gardening leave.

I find that quite implausible.

A possible alternative interpretation is "Up until about a year ago, I was at Meta, and on an almost daily basis I would see new NNs."

" So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first."

Does that sound like someone on gardening leave, or someone with the weary or arrogant disdain of the "not-invented-here" attitude nurtured in a mega-corporation.

Chris jones was introduced to Brainchip, and they told him about TeNNs. So was the Akida marketing group going around talking to random gardeners in the off-chance they wished to lay out a lazy $5M for a licence?

The more we discuss this, the more I am inclined towards the possibility of Meta being an EAP. Mr Ed is galloping this way.

Meta has over 100 patents referencing NNs, eg:


US2024153239A1 MULTIMODAL IDENTIFICATION OF OBJECTS OF INTEREST IN IMAGES 20221109

Technology for identifying an object of interest includes obtaining object embeddings for a plurality of objects in an image, obtaining text embeddings for text associated with the image, determining, for each of the plurality of objects, a similarity score via a similarity model based on the text embeddings and the object embeddings, while bypassing use of bounding box coordinates, and selecting the object having the highest similarity score as the object of interest. In another example, technology for identifying an object of interest includes obtaining object embeddings for a plurality of objects in an image, obtaining text embeddings and text identifiers for text associated with the image, generating, via a single transformer encoder, a set of CLS embeddings based on the text embeddings and the object embeddings, and determining, via a neural network, the object of interest based on the CLS embeddings.

As of 2022, they thought NNs uses MACs:
US2022308835A1 COUNTER BASED MULTIPLY-AND-ACCUMULATE CIRCUIT FOR NEURAL NETWORK 20220615

a system, a method, and a device for improving computation efficiency of a neural network. In one aspect, adder circuitry is configured to add input data from processing of the neural network and a first number of bits of accumulated data for the neural network to generate summation data. In one aspect, according to a carry value of the adding from the adder circuitry, a multiplexer is configured to select between i) a second number of bits of the accumulated data and ii) incremented data comprising the second number of bits of the accumulated data incremented by a predetermined value. The summation data appended with the selected one of the second number of bits of the accumulated data or the incremented data may form appended data.

This document describes many-shot learning.

It's quite ironic that this Meta patent application was filed using technology from the previous millennium, in the same month as the SotA TeNNs application. You can readily understand Chris' enthusiasm.

 

[DingoBorat]

Akida by Brainchip is an advanced artificial intelligence processor inspired by the neural architecture of the human brain, designed to provide high-performance AI capabilities at the edge with exceptional energy efficiency. Version 1.0 is available for purchase in the form factor of PCIe x1 as shown in Figure 3, and supports convolutional neural network architectures. Version 2.0 adds support for a variety of neural network types including RNNs and transformer architectures, but is currently only available in simulation.

You apparently still can't get your hands on the 2.0 launched March 2023! Hmm, hopefully one day eh!

The 6 month price chart is very depressing looking, a very short spike to 0.49 cent and then slowly bleeding out from then on.

Not sure the TENN's whitepaper will move the needle, doubt it but hopefully people in the industry will be very impressed.

It's over my pay-grade I must admit.

We need a IP deal or two to move the needle, that I know.

https://brainchip.com/wp-content/uploads/2023/06/TENNs_Whitepaper_Final.pdf

"You apparently still can't get your hands on the 2.0 launched March 2023! Hmm, hopefully one day eh!"

AKIDA 2.0 IP is available now.

Saying you can't get your "hands on" the IP is misleading and not physically possible anyway..

They were just saying, there is no AKD2000 chip currently, which Dr Anthony Lewis, does not see as a problem (who for a CTO, has a strong commercialisation focus) as the core technology has been proven, by 2 iterations of AKD1000 and AKD1500, in 2 separate processes.

 

[DingoBorat]

No, Chris Jones started working for Meta in June 2022, see my post and his LinkedIn profile.




That’s exactly why I wrote the following:



But has there ever been any indication that Meta was indeed an EAP customer at the time? Any announcement as with other EAP customers? If not, we shouldn’t simply assume so.

But for the sake of the discussion, let’s assume for a minute it was indeed the case.


In practice, the overlapping period regarding Chris Jones working for Meta and his introduction to BrainChip and TENNs would have been much shorter, though, given that Chris Jones said on May 23, 2024

“So, about a year ago, uh, I was at Meta, I was in their AI Infrastructure Group, and on an almost daily basis I would see new neural network architectures.

So, when I was introduced to BrainChip, I didn’t think I would really be impressed by anything a small team was gonna develop, erm. They told me about TENNs, I was a little bit skeptical to be honest at first. As I started getting to understand the benchmarks and a little bit more of the math and how it worked, I started to get pretty excited by what they had.”


Saying “about a year ago” on May 23, 2024 could mean July, June, May, April, possibly even March 2023. He certainly wouldn’t have put it that way if he and his colleagues had already been introduced to BrainChip in let’s say October 2022. And since he appears to have been laid off in mid-April, the potential time window shrinks to a maximum of six weeks, I’d say. That’s far from the nine months you claimed.

Your use of the participle “outplaced” instead of “laid off” implies that Meta would have helped him to find his current job? Again, there is no indication of that at all when you read his LinkedIn post, especially the last paragraph:

View attachment 65061


Mind you, I did not say there is no way that Chris Jones could have found out about TENNs while still working for Meta, but to me his words are certainly not conclusive evidence, the way FF presented them. They can very well be interpreted differently, especially with the background knowledge that he was laid off more than a year ago (which he didn’t mention in the video). I had already taken notice of that a while ago, when I had had a look at his LinkedIn profile after learning that he would be the one giving the talk Nandan Nayampally was supposed to have presented. (This was even before we found out from the Quarterly Investor podcast that Nandan and Rob had all of a sudden left the company.)

So no, mine is not a tendentious reading and I am not shooting myself in the foot either, if that is what you meant to say. My argument is well-founded. I don’t exclude the possibility that Chris Jones got introduced to BrainChip while still working for Meta, but I believe it is the unlikelier sequence of events for the reasons stated.

Also: Why would he have asked his LinkedIn network for assistance in finding a new job in his April 2023 LinkedIn post and only started working for BrainChip in October 2023? If he had already been that excited about our company prior to being laid off at Meta, they might even have been able to offer him a new position from August onwards, a smooth transition from Meta to BrainChip without a paycheck missing. Of course I have no idea whether it was possibly a deliberate decision of Chris Jones to pick October as the start date for his new job (maybe he wanted to spend quality time with his family between jobs, go on a long vacation, rest and recharge, renovate the house or perhaps he was suffering from an illness, was taking care of elderly relatives or was grieving for a loved one etc) or whether there was simply no earlier job vacancy for his position at BrainChip, but the two month gap between jobs could just as well signify that he didn’t yet know about BrainChip’s offerings by the time he started looking for a new job and that they were possibly not even his first choice.

Ultimately, everything - and that includes FF’s reading - is speculation, unless we hear it from the horse’s mouth. Can we at least agree on that?

"But has there ever been any indication that Meta was indeed an EAP customer at the time? Any announcement as with other EAP customers? If not, we shouldn’t simply assume so".

BrainChip, has said precious little, about who have been their EAP customers.

Which customers, have actually been announced, as Early Access Partners, as I cannot think of any.
I think there are only strong associations?

So I think you saying, that Meta had not been announced as an EAP, is a very weak argument against the possibility of it.

 

[Getupthere]

Computers built like brains could be a 'competition killer'?

Computers built like brains could be much more energy efficient than current designs.

www.bbc.com

 

[Pom down under]

Working student position in the field of Machine Learning & Neuromorphic Computing from August 2024

The AI (Artificial Intelligence) Research Team conducts research into current topics relating to artificial intelligence and applies the results to use cases from various specialist areas within the Group. We are researching the application of neuromorphic computing, an emerging technology that...

jobs.mercedes-benz.com

Tasks
The AI (Artificial Intelligence) Research Team conducts research into current topics relating to artificial intelligence and applies the results to use cases from various specialist areas within the Group. We are researching the application of neuromorphic computing, an emerging technology that draws inspiration from neuroscience and promises particularly fast and energy-efficient machine learning. For this project, we are looking for a working student to support us in building demonstrators and developing novel algorithms.

As pioneers in the field of Innovations & Future Technologies, we overcome boundaries to identify relevant future technologies and innovations and accompany the relevant concepts from prototype to series production. We see ourselves as an innovation partner for all Mercedes-Benz development areas: From components, software & AI, the electric powertrain to the complete vehicle. In the innovation network with colleagues from various departments as well as with external company, start-up and university partners, we provide decisive impetus and steer innovations through to series production.

You will face these challenges:

• Working with new chip technologies
• Setting up demonstrators (streaming sensor data, training machine learning algorithms, deployment on neuromorphic chips, live visualization of processing results)
• Further development and evaluation of novel neuromorphic algorithms

 

[IloveLamp]

 

[itsol4605]

View attachment 65106

Power consumption doesn't matter.
The exponentially increasing power consumption is irrelevant.
As long as it doesn't play a major role, electricity consumption will continue to rise.
It will only be in 2040 that electricity will no longer be sufficient.
As long as things continue like this, nothing will change.

 

[manny100]

I want to thank you and the others here for always providing objective counterarguments and having the patience to stand up to the trolls and bashers in the HC forum. I can't manage it and quickly digress... It's due to my southern European temperament... even if one tries to stay calm. Keep it up!


Best regards,
7

Cheers

 

[Fullmoonfever]

Wonder if we'll get a mention.

Yole appear bullish on neuromorphic.

Edge AI and Vision Insights: June 5, 2024

LETTER FROM THE EDITOR Dear Colleague, On Thursday, July 11, 2024 at 9 am PT, the Yole Group will deliver the free webinar “The Rise of Neuromorphic

www.edge-ai-vision.com

LETTER FROM THE EDITOR

Dear Colleague, ​ On Thursday, July 11, 2024 at 9 am PT, the Yole Group will deliver the free webinar “The Rise of Neuromorphic Sensing and Computing: Technology Innovations, Ecosystem Evolutions and Market Trends” in partnership with the Edge AI and Vision Alliance. The neuromorphic sensing and computing markets are projected to experience substantial growth in the coming years, reaching a combined value of $8.4B by 2034. Mobile applications will drive the neuromorphic sensing market, while data center applications will lead in the neuromorphic computing market. The neuromorphic ecosystem is maturing, with key players and startups adopting diverse strategies to capitalize on market opportunities. Neuromorphic technologies, inspired by biological brains, offer power-efficient solutions for AI tasks, addressing the declining economic feasibility of scaling semiconductor devices. These technologies provide benefits such as low latency, high scalability, and online learning, enabling real-time edge-AI applications and addressing privacy concerns. Neuromorphic sensing options include standalone event-based sensors and hybrid sensors combining RGB and event-based pixels. Neuromorphic computing systems, featuring event-driven processing and spiking neural network algorithms, enable online learning and autonomous robotics. This webinar, co-presented by Adrien Sanchez and Florian Domengie, senior technology and market analysts at the Yole Group, will delve into the latest advancements in neuromorphic sensing and computing technologies and their applications across various industries, offering insights into the future of sustainable and efficient AI processing at the edge. A question-and-answer session will follow the presentation. For more information and to register, please see the event page. Brian Dipert Editor-In-Chief, Edge AI and Vision Alliance

 

[wilzy123]

Yeah mate, busy writing for MF at the moment. Sorry didn't have time to clown you here.
Anyway keep an eye out for me and @Iseki new articles tomorrow about IR that has done SFA and directors laughing all the way to the bank.
Really appreciated your baiting. Do keep going champ!
Seriously got nothing more to say really. Waiting for some positive news before thinking of buying back in. Have some cash ready to go at maybe around 10 or 15c.
Not advice

What a great post. I can hear the embers of all the nuggets in this post crackling away. Thanks again. The entire TSE community values you.

As always though, we feel bad about your public display of insecurities and your need to use more than zero words in a TSE post. Maybe @Iseki can help you.