BRN Discussion Ongoing

[Bravo]

Akida to the rescue methinks!


SYSTEMS & DESIGN
OPINION

Unlocking The Power Of Edge Computing With Large Language Models​

Training and inferencing at the edge enables AI applications with low latency, enhanced privacy, and the ability to function offline.
OCTOBER 30TH, 2023 - BY: PAUL KARAZUBA

In recent years, Large Language Models (LLMs) have revolutionized the field of artificial intelligence, transforming how we interact with devices and the possibilities of what machines can achieve. These models have demonstrated remarkable natural language understanding and generation abilities, making them indispensable for various applications.
However, LLMs are incredibly resource-intensive. Training them on cloud servers with massive GPU clusters is expensive, and the inference of these models on cloud servers can result in substantial latency, poor user experience, and privacy and security risks. Many smartphone, IoT, and automobile makers have set a goal of edge inference deployments of LLMs in future platforms. In this article, we’ll explore the significance of deploying large language models on edge devices, as well as the challenges and future.

Moving LLMs to the edge: Why?​

It would be impossible to discuss every reason for edge deployment of LLMs, which may be industry, OEM, or LLM-specific. For this article, we will address five of the more prevalent reasons we hear.
One of the primary motivations for moving LLM inference to the edge is reduced connectivity dependency. Cloud-based LLMs rely on a stable network connection for inference. Moving LLM inference to the edge means applications can function with limited or no network connectivity. For instance, the LLM could be the interface to your notes, or even your whole phone, regardless of your 5G strength.
Many LLM-based applications depend on low latency for the best user experience. The response time of a cloud-based LLM depends on the stability and speed of the network connection. When inference occurs locally, the response time is significantly reduced, leading to a better user experience.
Edge computing can enhance privacy and data security. Since data processing happens on the local device, attack surfaces are significantly reduced versus a cloud-based system. Sensitive information doesn’t need to be sent over the network to a remote server, minimizing the risk of data breaches and providing users more control over their personal information.
Personalization is another key motivator for edge deployments, not only in inference but in training. An edge-based LLM can learn how the device user speaks, how they write, etc. This allows the device to fine-tune models to cater to the user’s specific personality and habits, providing a more tailored experience. Doing so on the edge can add additional assurance of privacy to the user.
The final motivator we will address in this article is scalability. Edge devices are deployed at scale, making it possible to distribute applications across a wide range of devices without overloading central servers.

Challenges in deploying large language models on edge devices​

While the advantages of deploying LLMs on edge devices are clear, there are several challenges that developers and organizations must address to ensure success. As before, there are more than we will discuss below.
Let’s first address resource constraints. Compared to cloud servers, edge devices have limited processing power, memory, and storage. Adapting LLMs to run efficiently on such devices will be a significant technical challenge. After all, large language models are precisely that—large. Shrinking these models without sacrificing performance is a complex task, requiring optimization and quantization techniques. While many in the AI industry are hard at work at doing this, successfully reducing LLM size is going to be mandatory for successful edge deployment, coupled with use-case tailored NPU (Neural Processing Units) deployments.
Energy efficiency is also a huge challenge. Running resource-intensive models on battery-powered devices can drain the battery quickly. Both developers and chip architects need to optimize their designs for energy efficiency to not create noticeable adverse effects on battery life.
Security requirements of LLMs, and by extension any AI implementation, are different from more traditional processors and code. Device OEMs must adapt to this and ensure privacy and data security is maintained. Even though edge computing may enhance data privacy versus cloud-based implementations, it also brings challenges in terms of securing data stored on edge devices.
A final challenge to consider is compatibility. LLMs may simply not be compatible with all edge devices. Developers must ensure that models are either developed which run on various hardware and software configurations, or that tailored hardware and software will be available to support the custom implementations.

The future of edge-deployed large language models​

The large-scale deployment of LLMs on edge devices is not a question of if, but rather a question of when it happens. This will enable smarter, more responsive, and privacy-focused applications across various industries. Developers, researchers, and organizations are actively working to address the challenges associated with this deployment, and as they do, we can expect more powerful and efficient models that run on a broader range of edge devices.
The synergy of large language models and edge computing opens up a world of possibilities. With low latency, enhanced privacy, and the ability to function offline, edge devices become more useful and versatile than ever before.

Unlocking The Power Of Edge Computing With Large Language Models

Training and inferencing at the edge enables AI applications with low latency, enhanced privacy, and the ability to function offline.

semiengineering.com

 

[McHale]

FYI concerning the vote against Peter's re -election as a director was 109,789,182 Million ~ 27.94% of votes

"Tuesday, 24 May 2022:
In accordance with Listing Rule 3.13.2 and Section 251AA(2) of the Corporations Act, BrainChip Holdings Ltd (ASX:BRN) provides the following information with respect to the results of its Annual General Meeting held today. Resolutions voted on at the meeting If decided by poll Proxies received in advance of the meeting Resolution Result/ Resolution Type Voting method If s250U applies Voted for Voted against Abstained For Against Abstain Discretion No Short description Number % Number % Number Number Number Number Number 1 Adoption of Remuneration Report Carried Ordinary Poll Yes 262,519,106 88.12 35,383,729 11.88 94,770,967 195,936,865 35,042,053 94,166,812 12,950,370 2 Re-election of Peter Van Der Made as Director Carried Ordinary Poll N/A 283,100,194 72.06 109,789,182 27.94 172,514,611 223,140,439 101,974,925 172,511,110 12,554,811 3 Elect

Thanks for your response @Xray1, so my recollection of that vote against Peters' re-election was correct, and as you have put it, it was in 2022.

So IMO there was no way retail holders would have supported that vote against PvDM. I didn't say it in my previous post on this subject, but I wouldn't be surprised if these votes at the last 2 AGMs were the result of institutions playing games, and yes also exercising their rights as shareholders.

It is pure speculation on my part, but that is what I have felt since the 2022 AGM, and I feel the same about what took place at this years AGM, malevolent actors working at undermining confidence in BRN and undermining price appreciation.

I welcome any feedback, or other research to give more flesh to the bones of this matter.

 

[Gemmax]

Thanks for your response @Xray1, so my recollection of that vote against Peters' re-election was correct, and as you have put it, it was in 2022.

So IMO there was no way retail holders would have supported that vote against PvDM. I didn't say it in my previous post on this subject, but I wouldn't be surprised if these votes at the last 2 AGMs were the result of institutions playing games, and yes also exercising their rights as shareholders.

It is pure speculation on my part, but that is what I have felt since the 2022 AGM, and I feel the same about what took place at this years AGM, malevolent actors working at undermining confidence in BRN and undermining price appreciation.

I welcome any feedback, or other research to give more flesh to the bones of this matter.

I have always wondered about this McHale. Not putting words into Techs mouth, but he has referred to agendas at play in the past.
FWIW. Between 12:08pm and 12:33 pm today, one share was traded in 25 minutes to knock it down.

 

[HopalongPetrovski]

Thanks for your response @Xray1, so my recollection of that vote against Peters' re-election was correct, and as you have put it, it was in 2022.

So IMO there was no way retail holders would have supported that vote against PvDM. I didn't say it in my previous post on this subject, but I wouldn't be surprised if these votes at the last 2 AGMs were the result of institutions playing games, and yes also exercising their rights as shareholders.

It is pure speculation on my part, but that is what I have felt since the 2022 AGM, and I feel the same about what took place at this years AGM, malevolent actors working at undermining confidence in BRN and undermining price appreciation.

I welcome any feedback, or other research to give more flesh to the bones of this matter.

Yes.
I too have no proof to offer but there was the definite feel of a coordinated campaign leading up to the last AGM in a bid to dissolve the BOD's standing.
At the time my impression was that it was shorter led to try and sow FUD in the minds of holders and provide further downside pressure.
The prior attack on PVDM was an attack on a pillar of the company and was just muscle being exercised and tested in my opinion.
The people/organisations behind these assaults have no interest in anything other than the manipulation of the share price and in extracting every grubby last cent they can wring from their victims.
The financial equivalent to a parasite.
GLTAH

 

[Space Cadet]

I have always wondered about this McHale. Not putting words into Techs mouth, but he has referred to agendas at play in the past.
FWIW. Between 12:08pm and 12:33 pm today, one share was traded in 25 minutes to knock it down.

Well, they are going to have to do that a few more times because that 22,226 buy is me at 16.5c. It started at 25,000 and going down in bits. Very hard to watch, It's like pulling teeth.

SC

 

[McHale]

Hi McH,

There was a time under the ebullient Lou "Hockey Sticks" Dinardo when reference was made to "a major North American vehicle maker" which drew a "please explain" from ASX and caused a possible breach of an NDA when we were forced to disclose Ford as an EAP.

Not only that, but it was a blot on our escutcheon, which has the potential to turn around and bite us on the hindquarters when we seek NYSE registration - our first yellow card.

Because the ASX, being gun-shy of tech companies since getting their fingers burnt in the dotcom bubble, is red hot on tech startups. (Don't you just love a metaphor salad?)

The company is well aware of the disgruntlement of the roiling mass of commentators on TSEx in relation to announcements, but they have chosen to be like Caesar's wife in this regard.

Being a bit of a share market ingenue back in 2018 when I first came across BRN, my first purchase was in FOMO mode because I read that PvdM was about to make a presentation to a gaggle of geeks in the US in the following week, and I expected that would launch us to the moon and beyond.* Not a sausage ...

So I think that, as far as management's concern for shareholders goes, they have their eye on the main prize, NYSE registration, and doing anything to pump the ASX price would earn a second yellow card.

*(I still expect that every day now.)

Yes @Diogenese, I have heard the story before regarding the Ford stuff up, but I don't subscribe to the rest of the story re gun shy ASX etc.

The fact is there are quite a number of tech start ups on ASX which are making "Non Price Sensitive" Anns every day. I would be interested to hear what the ASX Announcement platform has to say about yellow cards or even the dreaded red card, because if they do, then there should be an official explanation to outline their Announcement policies.

Is it a real thing or imagined, I do understand that current BRN management seem to have an inordinate fear about it, and for myself I would be much happier if BRN could find their way toward actually initiating a dialogue with ASX about a reasonable explanation on how the ASX Ann platform works; and then some kind of communication from BRN about what the facts of this matter are - to their long patient retail share holders.

In the meantime BRNs listing on the Nasdaq is but a far distant mirage at this point, while I remain much more concerned with gathering not so distant black swans.

 

[SharesForBrekky]

Yes.
I too have no proof to offer but there was the definite feel of a coordinated campaign leading up to the last AGM in a bid to dissolve the BOD's standing.
At the time my impression was that it was shorter led to try and sow FUD in the minds of holders and provide further downside pressure.
The prior attack on PVDM was an attack on a pillar of the company and was just muscle being exercised and tested in my opinion.
The people/organisations behind these assaults have no interest in anything other than the manipulation of the share price and in extracting every grubby last cent they can wring from their victims.
The financial equivalent to a parasite.
GLTAH

Just a thought, could be way off the mark though.

Maybe trying to throw PVDM off the Board and Brainchip getting a 1st strike could be related in some way.

Maybe an institution and their client/s who are heavily invested in Brainchip are trying to cause some turmoil, so as to create an opportunity for a takeover situation.

Seeing that Brainchip's management and retail shareholders have control of a takeover decision, an institution maybe looking at other avenues of creating a takeover situation.

Not sure how this could eventuate, just a thought that came into my head.

 

[Diogenese]

Yes @Diogenese, I have heard the story before regarding the Ford stuff up, but I don't subscribe to the rest of the story re gun shy ASX etc.

The fact is there are quite a number of tech start ups on ASX which are making "Non Price Sensitive" Anns every day. I would be interested to hear what the ASX Announcement platform has to say about yellow cards or even the dreaded red card, because if they do, then there should be an official explanation to outline their Announcement policies.

Is it a real thing or imagined, I do understand that current BRN management seem to have an inordinate fear about it, and for myself I would be much happier if BRN could find their way toward actually initiating a dialogue with ASX about a reasonable explanation on how the ASX Ann platform works; and then some kind of communication from BRN about what the facts of this matter are - to their long patient retail share holders.

In the meantime BRNs listing on the Nasdaq is but a far distant mirage at this point, while I remain much more concerned with gathering not so distant black swans.

Hi McH,

The yellow card is of relevance to the NYSE. It may count against us when we have to prove we are a fit and proper entity, or whatever character test they apply. Out here in the wild ASX badlands, the rustlers are the sheriff.

One of the NYSE listing requirements is a copy of good standing certificate from jurisdiction of incorporation.

If memory serves. some time ago someone posted a statement from a company spokesperson to the effect that the company does liaise with ASX about announcements.

 

[Esq.111]

Afternoon Chippers ,

Eyeballing the five day chart at one min duration..... getting that feeling should see some decent volume punch through in one hours
time .

Also on the two month chart at one day volumes .... could very well see some substantial volumes....thinking around the 20 to 30odd million volume size.

All just me gut feel .... Not financial advice.

Regards,
Esq

 

[Esq.111]

Well i've cracked the first beer on the above .... the anticipation got me .... guessing ten minutes to go....

Not Financial Advice.

Esq.

 

[wilzy123]

Just proofs what this place is becoming Have a great day.

Proofs?

 

[Esq.111]

Second beverage cracked ...... it wants to.....

Akiba Ballista....

Esq,

 

[buena suerte :-)]

Second beverage cracked ...... it wants to.....

Akiba Ballista....

Esq,

 

[Esq.111]

Afternoon Buena suerte : } ,

Tizz not a science but a gut feel / intuition honed over many moons looking at market manipulation in too much detail.

.

Regards,
Esq

 

[buena suerte :-)]

Afteronno Buena suerte : } ,

Tizz not a science but a gut feel / intuition honed over many moons looking at market manipulation in too much detail.

.

Regards,
Esq

Afternoon Esqy ... Love ya work mate ..Cheers

 

[TECH]

Thanks for your response @Xray1, so my recollection of that vote against Peters' re-election was correct, and as you have put it, it was in 2022.

So IMO there was no way retail holders would have supported that vote against PvDM. I didn't say it in my previous post on this subject, but I wouldn't be surprised if these votes at the last 2 AGMs were the result of institutions playing games, and yes also exercising their rights as shareholders.

It is pure speculation on my part, but that is what I have felt since the 2022 AGM, and I feel the same about what took place at this years AGM, malevolent actors working at undermining confidence in BRN and undermining price appreciation.

I welcome any feedback, or other research to give more flesh to the bones of this matter.

Hello,

After the AGM in 2022 I spoke with three individuals whom I can't name in regards to the volume of votes to try and unsettle Peter
from our Board.

I do believe it is known which entity/individual was attempting to make a statement, was it a former disgruntled employee ? I reserve
my opinion as it's now water under the bridge.

Any shareholder whatever their category attempting to remove our Founder would be committing suicide, both Peter and Anil will
retire in their own good time or earlier if illness effects their ability to function at the high levels that they both set for themselves, in my
opinion for what it's worth.

Nvidia has always been the company that I've believed may make a play for us over the next 5 years, yes it's a guess, but a good one !

Regards....Tech

 

[Cirat]

Hello,

After the AGM in 2022 I spoke with three individuals whom I can't name in regards to the volume of votes to try and unsettle Peter
from our Board.

I do believe it is known which entity/individual was attempting to make a statement, was it a former disgruntled employee ? I reserve
my opinion as it's now water under the bridge.

Any shareholder whatever their category attempting to remove our Founder would be committing suicide, both Peter and Anil will
retire in their own good time or earlier if illness effects their ability to function at the high levels that they both set for themselves, in my
opinion for what it's worth.

Nvidia has always been the company that I've believed may make a play for us over the next 5 years, yes it's a guess, but a good one !

Regards....Tech

Absolutely agree Tech!

Peter & Anil are critical to the ongoing value and ultimate success of the company. If a particular person or people are trying to unsettle Peter then I believe they are very petty and misled in their intentions as they also will lose - stupid business decision and petulant.

I also believe once Brainchip signs 2 or more large IP deals then no doubt takeover offers will be formally presented (maybe even earlier who knows). I have no idea if that will be positive or negative for me personally when the time to vote on the offer arrives. At this point my preference is for Brainchip to grow to the behemoth I know it can and enjoy the dividends.

Just my musings on Brainchip's possible future that may still be a long way out.

This is not Financial Advice, DYOR, GLTA

 

[IloveLamp]

https://www.extremetech.com/computing/amds-monstrous-mi300-finally-arrives-carrying-the-companys-ai-hopes-and?utm_source=flipboard&utm_content=ExtremeTech/magazine/Extreme+Computing&es_id=10d579b451

 

[Tony Coles]

Well, they are going to have to do that a few more times because that 22,226 buy is me at 16.5c. It started at 25,000 and going down in bits. Very hard to watch, It's like pulling teeth.

SC

Well done on your purchase Space Cadet, hope you 100x

 

[McHale]

Hello,

After the AGM in 2022 I spoke with three individuals whom I can't name in regards to the volume of votes to try and unsettle Peter
from our Board.

I do believe it is known which entity/individual was attempting to make a statement, was it a former disgruntled employee ? I reserve
my opinion as it's now water under the bridge.

Any shareholder whatever their category attempting to remove our Founder would be committing suicide, both Peter and Anil will
retire in their own good time or earlier if illness effects their ability to function at the high levels that they both set for themselves, in my
opinion for what it's worth.

Nvidia has always been the company that I've believed may make a play for us over the next 5 years, yes it's a guess, but a good one !

Regards....Tech

Well that's interesting @TECH, because the entity or individuals you allude to had 109 million votes they used to vote to unseat Peter, but as you said "I reserve my opinion as it's now water under the bridge".

What I have been speculating on really is; could there be a relationship between the actor(s) who voted against PVDM at AGM 2022 and possibly the actor(s) who voted down the remuneration package at AGM 2023.

Both actions could be construed as "not a good look for the company" for market participants or share holders or investors who might be considering buying into BRN.

At current Market Cap BRN is very cheap, but I wasn't speculating about takeovers, and quite frankly that would be one of the last things I want to see at the moment, having lost so much paper money on this particular investment since early last year.