Environmental impact of Generative AI products:
AI, with the recent arrival of ChatGPT, Bard, etc., has sparked massive interest in the public and industries alike. These NLP tools built on large language models assist in customer support, help in developing and debugging code, and serve as conversational agents, aiming to boost our productivity and hence boost the economic value.
But would these benefits justify the harm done to our planet?
1. Electricity consumption for training humungous language models is beyond the roof, for example, training a 13B parameter LLM on 390B text tokens on 200 GPUs for 7 days costs $151,744 (Source: HuggingFace new training cluster service page -
https://lnkd.in/g6Vc5cz3). And even larger models with 100+B parameters cost $10+M just to train. Then pay for inferencing every time a new prompt request arrives.
2. Water consumption for cooling, researchers at the University of California, Riverside estimated the environmental impact of ChatGPT-like service, and say it gulps up 500 milliliters of water (close to what’s in a 16-ounce water bottle) every time you ask it a series of between 5 to 50 prompts or questions. The range varies depending on where its servers are located and the season. The estimate includes indirect water usage that the companies don’t measure — such as to cool power plants that supply the data centers with electricity. (Source:
https://lnkd.in/gybcxX8C)
Given the benefits of AI/ GenAI, its demand is only bound to go up, and so will its side effects on our planet. How can we reduce or neutralize the side effects of AI on our planet? Carbon capture and nuclear power are in the right direction. But we need to fundamentally rethink the way we do AI, is it the wrong way to do tonnes of matrix multiplications?
Our brain can learn and do many tasks in parallel, in and under 10W, but why do these AI systems consume 10s of megawatts to train models?
Perhaps the future holds energy-efficient architectures such as neuromorphic architectures and spiking neural network-based transformers that are closest to the human brain, which might consume 100-1000x lower energy, hence reducing the cost of using AI, thereby democratizing it and saving our planet.
kudos to the others too that jumped on board below 100k. but I bet the top 20 will happy soon 
