Kachoo
Regular
It's innthe half year reportSorry if I missed something but people are referring to forecast revenue. Can someone please show me the link to where we have this forecast? Thanks in advance
BRN Discussion Ongoing
- Thread starter TechGirl
- Start date
Because of Akida's 4-bit capability, it can have "home-made" models which are more compact than the standard 8-bit models, and, of course, Akida can handle the 8-bit models as well, with the accompanying reduction in efficiency advantage. And let's not forget Akida's 1 and 2-bit capability for super low power consumption.Isn't this what Tony Lewis is focussed on? I remember he posted a reply on his Linkedin saying something like we are the first he's aware of to be able to run SOTA SML's on edge devices or something like that.
Does anyone have his Linkedin post handy?
IBM & NTT Explain How AI Works on Edge Computing
by Senior Technology Journalist
Ray Fernandez
Fact Checked byEddie Wrenn
Updated on 3 September 2024
One trillion plus AI parameter models are already banging hard at our front door. These new artificial intelligence models, with unprecedented computing power, will become the norm in the coming months and years ahead.
While the technological advancements of new generative AI models are promising and expected to benefit most sectors and industries, the world still has a big AI-size-edge-infrastructure problem (we will break this down in a moment).
How will giant AI models operate on the edge of networks while offering low latency and real-time services? It’s a ‘shrinking giant’ problem.
In this report, Techopedia talks with NTT and IBM experts to understand an approach to solving fast, resource-intensive artificial intelligence without over-burdening a network.
Key Takeaways
Table of Contents
- Large AI models are often too resource-intensive for edge devices.
- Smaller, more efficient AI models offer a practical solution for deploying AI at the edge.
- Industries such as manufacturing need AI solutions that can operate effectively in distributed environments.
- Successful edge AI deployment requires collaboration between silos of information, and it is beginning to happen today.
Is the Solution an Edge Infrastructure Combined with Smaller AI Models?
Compute power is rapidly moving from the data center to the edge. Organizations expect edge computing to significantly impact all aspects of operations. Meanwhile, worldwide spending on edge computing is expected to be $232 billion in 2024, an increase of 15.4% over 2023 — largely driven by AI.
Techopedia spoke to Paul Bloudoff, Senior Director, Edge Services, NTT DATA, which focuses on edge AI — particularly smaller, more efficient AI models. These models are use-case specific and sized to be simple to deploy and run.
Advertisements
Bloudoff explained how IT and Operational Technology (OT) teams can benefit from small AI models running on lightweight edge.
For example, in factory floors, these AI solutions can enhance maintenance by breaking down siloes and bringing together a suite of information provided by the Internet of Things (IoT) to, for instance, track data on vibration, temperature, machine wear, and more.
Industrial predictive maintenance AI can save organizations thousands of dollars by reducing downtime.
Shrinking AI Giants into TinyAI to Drive Sustainable Development
Scentistific researchers already advocate for TinyAI models as the solution to the complex transition of data center AI models into edge computing.
Scientists argue that TinyAI models can be deployed in healthcare, agriculture, and urban development and contribute to the development of the United Nations Sustainable Development Goals. Tiny models are designed for specific use cases and, therefore, are more cost-efficient, and consume less power, driving sustainability targets.
Nick Fuller, Vice President of AI and Automation at IBM Research, told Techopedia that for edge workloads requiring low latency, inferencing on devices is more than highly desirable; it is essential.
“Models, of course, can still be trained on-premise or on the cloud. To this end, lightweight AI models are very appealing to the edge market and especially to specific workloads where latency requirements are essential.”
Speaking the Many Languages of Edge Computing
Many developers fear that edge infrastructure is constrained in terms of computing power processing, data storage, and memory. Tehcopedi asked Bloudoff from NTT how the company approached this problem.
Bloudoff explained that one of the challenges organizations face when deploying solutions at the edge is the silos between machines and devices across their network — machines and devices from different manufacturers do not always communicate well with each other.
“Think of it as different individuals speaking different languages who are all providing data that needs to be collected, analyzed, processed, and transformed into action,” Bloudoff said.
“What’s powerful about an Edge AI platform is that its software layer automatically discovers devices and machines across an IT or OT environment through its smaller, more efficient language learning model,” Bloudoff added.
NTT’s Edge AI platform runs auto-discovery and unifies and processes the data to provide a comprehensive diagnostic report that can be used for AI-powered solutions.
Speaking to Techopedia, IBM Fellow Catherine Crawford added that the challenge is not just in how to move AI to the edge.
“There are multiple existing use cases that can leverage edge using smaller AI models where the technical challenges still exist and assets can be developed for distributed, secure Edge to Cloud continuous development AIOps.”
Crawford explained that there is ongoing interest and research in understanding how task-tuned, smaller genAI and foundational models can be developed and leveraged for edge use cases considering constraints like compute, memory, storage, and power (battery life).
An AI Built for Your Edge
The 2023 Edge Advantage Report — which surveyed 600 enterprises across multiple industries — found that approximately 70% of organizations use edge solutions to solve business challenges. Still, nearly 40% worry that their current infrastructure will not support advanced solutions.
Bloudoff from NTT said that the company is well aware of the market´s concern and focuses on smaller, more efficient language models. These are easier to deploy and can run in real-time without demanding advanced edge hardware updates.
Smaller AI models represent enormous savings in edge hardware costs for industries and businesses across the world. By going tiny, AI on the edge can maximize the edge computing power already in play through optimization.
The company is also moving forward by implementing dozens of proofs of concepts with customers across manufacturing, automotive, healthcare, power generation, logistics, and other industries.
The Bottom Line
Supercomputers, quantum computers, and state-of-the-art AI racks hosted in massive data centers, are undoubtedly the tip of the spear of modern innovation. However, the real world works at the edge level.
From the smartphone in your pocket to the automation of machines in industrial environments, healthcare, and agriculture, modern edge networks connect us all.
As bigger, faster, harder, and stronger AI models roll out, NTT and IBM invest in small and tiny models. They believe it is the solution to the future of giant AIs in our edge world.
I think that developing Akida-specific models for Valeo, Mercedes and others is where a lot of our effort will be focussed. These will not be universal models. They will be more functionally biassed.
Afternoon Gazzafish ,Sorry if I missed something but people are referring to forecast revenue. Can someone please show me the link to where we have this forecast? Thanks in advance
Info from the latest APPENDIX 4D Half Year Financial Report , page 13.
Regards,
Esq.
Attachments
DingoBorat
Slim
I concur Diogenese
And he said we "will" be the first @Bravo.
However, Kaist claimed to be running ChatGPT2 in full, on their neuromorphic research chip, back in March this year, which post date's TL's comments.
KAIST develops human brain-like AI chip
Researchers at the Korea Advanced Institute of Science and Technology (KAIST) have developed an AI semiconductor capable of processing large language model (LLM) data at ultra-high speeds while significantly reducing power consumption, according to the Ministry of Science and ICT.
m.koreatimes.co.kr
"Neuromorphic computing is a technology that even companies like IBM and Intel have not been able to implement, and we are proud to be the first in the world to run the LLM with a low-power neuromorphic accelerator," Yoo said.
Last edited:
Come to think about it, maintaining and updating models could be a nice little ongoing earner.
wilzy123
Founding Member
Forget th edge boxes guys, we will start to get traction in th next 3 quarters from auto, industrial and euro space industry!!!
..... Yeah! ....
Caution, delusional up ramper... Who is creeping into top holder territory on pure faith and speculation....
God save us.
Autonomous Vehicles, Military and Defense, Unmanned Aerial Vehicles, Robotics
Boab
I wish I could paint like Vincent
Me too.Just picked up some more...couldn't resist ..
Nice update to our website.
brainchip.com
Radar Lidar Applications Whitepaper
The Advantages of Event-Based Computing for Radar/LiDAR Applications In the rapidly evolving field of radar technology, the demand for more efficient, responsive, and adaptable systems is ever-growing. Traditional radar systems have served us well for decades, but the advent of event-based...
brainchip.com
SERA2g
Founding Member
Pom down under
Top 20
Wow we never ended up red like the rest of the asx.
CHIPS
Regular
Very cute
... the baby I mean !Congratulations to the parents and grandparents
.Have a great time with her (it's a girl?) but don't give her too many kisses
I think you might find there is a fair bit of interest in Gen 2 and engagements.
Chips on that basis not necessary for GEN 2.
AKIDA 1000 and 1500 chips were produced to get the industries 'head around, this new fangled contraption'.
rgupta
Regular
So they said they can do transformers as well on their SNN.I concur Diogenese
And he said we "will" be the first @Bravo.
However, Kaist claimed to be running ChatGPT2 in full, on their neuromorphic research chip, back in March this year, which post date's TL's comments.
KAIST develops human brain-like AI chip
Researchers at the Korea Advanced Institute of Science and Technology (KAIST) have developed an AI semiconductor capable of processing large language model (LLM) data at ultra-high speeds while significantly reducing power consumption, according to the Ministry of Science and ICT.
"Neuromorphic computing is a technology that even companies like IBM and Intel have not been able to implement, and we are proud to be the first in the world to run the LLM with a low-power neuromorphic accelerator," Yoo said.
As per brainchip we can also take transformer load with akida 2000 but the same is do able because of TENNs.
On top brainchip also said earlier the results are very encouraging when comparing akida with gpt 2.
Is there a co incidence??
On top the processor used is samsung on 28 nm.
DingoBorat
Slim
I don't think Kaist, has anything to do with us, personally.
Process size, doesn't mean anything, it's just a good proven one, that doesn't cost as much as the smaller ones (nobody, is going to produce "research chips" in 7nm for example).
I don't think their chip is pure digital either (I think @Diogenesed looked into it?)..
No surprise, that Samsung is involved and I believe they have a history with us, but that is one of their main foundries.
Any input from BrainChip, is inspired in my opinion.
And I'd Love for Samsung, to be onboard.
Yes. KAIST are into analog. The term "in-memory compute" is usually used in relation to analog, in that the calculations are performed by the memory circuits by accumulating a voltage whose amplitude is proportional to the number of input signals.
Hi Diogenese, I believe you are best person for this question.. Are the VVDN edge box use for research and development purposes or it can be used by end user? Let's say I want to set up a face recognition security system in my office, can I just buy an edge box and plug it into my camera system and it is ready to go or I still need to go and develop some software system to interact with the edge box?
Thank you in advance if you can provide me with the answer..
Hi KK,
The Edge Boxes are definitely suitable for end use. Of course they can also be used for R&D, but they are the real thing.
They come with pre-made models, but you can also develop your own model library or adapt the pre-made ones using on-chip learning.
rgupta
Regular
The reason I try to relate is not only 28nm chip, but also a report by brainchip where they compare akida with chat gpt2. But the biggest shock to me is they said their chip can do transformers though 1000 was not able to perform that function but akida 2 can do the same.
Anyway at the end a lot of wires are entangled and only time will have all the answers. But one thing for sure there is a lot of happening behind the scenes and still no concrete news.
DingoBorat
Slim
It definitely is, a tangled web out there Rgupta..
That's what makes the dot joining Fun, Frustrating and potentially Fruitless, all at the same time..
DingoBorat
Slim
Sounds like a bit of stretch now, saying it's running ChatGPT2, when it's actually a compressed model of it, with just over a quarter of the parameters...
유회준 교수 연구팀, 뉴로모픽 컴퓨팅 기반의 초저전력 거대 언어 모델 인공지능반도체 핵심기술 ‘상보형-트랜스포머’ 개발 | KAIST 반도체시스템공학과
KAIST 반도체시스템공학과sse.kaist.ac.kr
Used Google translator from Korean to English
Their SNN also compairing on GPT-2
Through this, it was possible to reduce the parameters of the GPT-2 giant model from 708 million to 191 million and the parameters of the T5 model used for translation from 402 million to 76 million. As a result of this compression work, we succeeded in reducing the power consumed by loading language model parameters from external memory by 70%. According to the researchers, the complementary transformer consumes 1/625th the power of the NVIDIA A100 GPU, while enabling high-speed operation of 0.4 seconds for language generation using the GPT-2 model and 0.2 seconds for language translation using the T5 model. In the case of language generation due to parameter lightweighting, the accuracy decreased by 1.2 branching coefficient (lower means that the language model was learned better), but the researchers explained that it is at a level where people will not feel awkward when reading the generated sentences. In the future, the research team plans to expand the scope of neuromorphic computing to various application fields rather than limiting it to language models.
Would be more accurate I think, to say it was running a SLM "derived" from ChatGPT2..
Similar threads
