Isn't this what Tony Lewis is focussed on? I remember he posted a reply on his Linkedin saying something like we are the first he's aware of to be able to run SOTA SML's on edge devices or something like that.
Does anyone have his Linkedin post handy?
IBM & NTT Explain How AI Works on Edge Computing
by Senior Technology Journalist
Ray Fernandez
Fact Checked by
Eddie Wrenn
Updated on 3 September 2024
One trillion plus AI
parameter models are already banging hard at our front door. These new
artificial intelligence models, with unprecedented computing power, will become the norm in the coming months and years ahead.
While the technological advancements of new
generative AI models are promising and expected to benefit most sectors and industries, the world still has a big AI-size-edge-infrastructure problem (we will break this down in a moment).
How will giant AI models operate on the edge of networks while offering low latency and real-time services? Itâs a âshrinking giantâ problem.
In this report, Techopedia talks with NTT and IBM experts to understand an approach to solving fast, resource-intensive
artificial intelligence without over-burdening a network.
Key Takeaways
- Large AI models are often too resource-intensive for edge devices.
- Smaller, more efficient AI models offer a practical solution for deploying AI at the edge.
- Industries such as manufacturing need AI solutions that can operate effectively in distributed environments.
- Successful edge AI deployment requires collaboration between silos of information, and it is beginning to happen today.
Table of Contents
Is the Solution an Edge Infrastructure Combined with Smaller AI Models?
Compute power is rapidly moving from the
data center to the edge. Organizations expect edge computing
to significantly impact all aspects of operations. Meanwhile, worldwide spending on edge computing is expected to be
$232 billion in 2024, an increase of 15.4% over 2023 â largely driven by AI.
Techopedia spoke to
Paul Bloudoff, Senior Director, Edge Services, NTT DATA, which focuses on edge AI â particularly smaller, more efficient
AI models. These models are use-case specific and sized to be simple to deploy and run.
Advertisements
âWhen we look at bringing AI to the edge, we need to understand the business use case and what organizations are trying to accomplish. Not every deployment will need monolithic AI models to support real-life use cases.â
Bloudoff explained how IT and
Operational Technology (OT) teams can benefit from small AI models running on lightweight edge.
For example, in factory floors, these AI solutions can enhance maintenance by breaking down siloes and bringing together a suite of information provided by the
Internet of Things (IoT) to, for instance, track data on vibration, temperature, machine wear, and more.
Industrial predictive maintenance AI
can save organizations thousands of dollars by reducing downtime.
âTo accomplish this (use case), organizations will not need a million-dollar AI platform to process this kind of information.
âFor many organizations, the resource investment to bring monolithic AI solutions to the edge is too complex and costly.â
Shrinking AI Giants into TinyAI to Drive Sustainable Development
Scentistific researchers already
advocate for TinyAI models as the solution to the complex transition of data center AI models into edge computing.
Scientists argue that TinyAI models can be deployed in healthcare, agriculture, and urban development and contribute to the development of the United Nations Sustainable Development Goals. Tiny models are designed for specific use cases and, therefore, are more cost-efficient, and consume less power, driving sustainability targets.
Nick Fuller, Vice President of AI and Automation at IBM Research, told Techopedia that for edge workloads requiring low latency, inferencing on devices is more than highly desirable; it is essential.
âTo this end, serving âsmallâ foundation models on such devices (robotic arms, mobile devices, cameras, etc.) facilitates on-device inferencing within the budget (memory, compute, accelerator) constraints of such devices.â
âModels, of course, can still be trained on-premise or on the
cloud. To this end, lightweight AI models are very appealing to the edge market and especially to specific workloads where latency requirements are essential.â
Speaking the Many Languages of Edge Computing
Many developers fear that edge infrastructure is constrained in terms of computing power processing, data storage, and memory. Tehcopedi asked Bloudoff from NTT how the company approached this problem.
Bloudoff explained that one of the challenges organizations face when deploying solutions at the edge is the silos between machines and devices across their network â machines and devices from different manufacturers do not always communicate well with each other.
âThink of it as different individuals speaking different languages who are all providing data that needs to be collected, analyzed, processed, and transformed into action,â Bloudoff said.
âWhatâs powerful about an Edge AI platform is that its software layer automatically discovers devices and machines across an IT or OT environment through its smaller, more efficient language learning model,â Bloudoff added.
NTTâs Edge AI platform runs auto-discovery and unifies and processes the data to provide a comprehensive diagnostic report that can be used for AI-powered solutions.
âOnce you break down silos and connect with stakeholders across different teams, you will have a better understanding of your processing power and memory needs.
âBigger isnât always better.â
Speaking to Techopedia,
IBM Fellow Catherine Crawford added that the challenge is not just in how to move AI to the edge.
âThere are multiple existing use cases that can leverage edge using smaller AI models where the technical challenges still exist and assets can be developed for distributed, secure Edge to Cloud continuous development AIOps.â
Crawford explained that there is ongoing interest and research in understanding how task-tuned, smaller genAI and foundational models can be developed and leveraged for edge use cases considering constraints like compute, memory, storage, and power (battery life).
âFor instance, perhaps having multiple, tuned smaller models on-board devices with hierarchical inferencing algorithms will be sufficient for edge use cases.
âAI algorithms and corresponding models will continue to evolve and, with the focus in our industry on sustainability, these models will quickly evolve to take less resources, becoming more appropriate for edge systems.â
An AI Built for Your Edge
The 2023 Edge Advantage Report â which surveyed 600 enterprises across multiple industries â found that approximately
70% of organizations use edge solutions to solve business challenges. Still, nearly
40% worry that their current infrastructure will not support advanced solutions.
Bloudoff from NTT said that the company is well aware of the market´s concern and focuses on smaller, more efficient language models. These are easier to deploy and can run in real-time without demanding advanced edge hardware updates.
Smaller AI models represent enormous savings in edge hardware costs for industries and businesses across the world. By going tiny, AI on the edge can maximize the edge computing power already in play through optimization.
The company is also moving forward by implementing dozens of proofs of concepts with customers across manufacturing, automotive, healthcare, power generation, logistics, and other industries.
The Bottom Line
Supercomputers, quantum computers, and state-of-the-art AI racks hosted in massive data centers, are undoubtedly the tip of the spear of modern innovation. However, the real world works at the edge level.
From the
smartphone in your pocket to the
automation of machines in industrial environments, healthcare, and agriculture, modern edge networks connect us all.
As bigger, faster, harder, and stronger AI models roll out, NTT and IBM invest in small and tiny models. They believe it is the solution to the future of giant AIs in our edge world.
www.vvdntech.com
..... Yeah! ....
