Seeing a few people casually playing around with LLM chatbots from the usual big tech suspects, so if you're just trying to get a handle on this stuff, here's a document by and for machine learning / "AI" users and developers about their environmental impact.
"There is currently little transparency on the energy demands of specific AI applications, although a recent estimate has put the amount of energy used for a ChatGPT query to be anywhere between 6-10 times more than a traditional Web search (0.3 Wh vs. 2.9 Wh)"
"Overall, the growing energy demand for AI is significantly outpacing the increase in renewable energies – entailing substantial new GHG emissions and squeezing an already tight renewable energy market."
Source: https://huggingface.co/blog/sasha/ai-environment-primer
Other research indicates that a ChatGPT query uses somewhere in the region of 60x the energy of a conventional web search.
Source: https://arxiv.org/abs/2407.16894
It's worth remembering that the limitations and structural characteristics of LLMs make their results less useful for most purposes than those of a conventional search, as well/
If you're interested in understanding how machine learning works and what results it produces, I would strongly recommend using local tools and models, which work effectively and in privacy on devices including Raspberry Pi 5s, old laptops and servers, and your PC.
You're not going to reverse the environmental impact of model training, but power consumption as a user is in line with general purpose PC use.
For LLMs:
Ollama - https://ollama.com/
Llama.CPP - https://github.com/ggerganov/llama.cpp
For voice synthesis and transcription:
SpeechNote (Piper + Whisper) - https://flathub.org/apps/net.mkiol.SpeechNote
Whisper.cpp - https://github.com/ggerganov/whisper.cpp
For non-copyright infringing text to image diffusion:
Mitsua Diffusion One: https://huggingface.co/Mitsua/mitsua-diffusion-one
Here's a script for it: https://codeberg.org/MightyOwlbear/Raspberry-Pi-5-Diffusion
Image description:
Blip - https://huggingface.co/Salesforce/blip-image-captioning-base
Captioner script: https://codeberg.org/MightyOwlbear/blip-image-description-tool
You'll find my Strongly Worded Opinions on AI trends, plus step-by-step guides on using some of these tools at:
https://magpi.raspberrypi.com/issues/141
The PDF is free to download. No idea what the "roll your own models" cover line is in aid of, we don't do that in this.
I hesitate to call any of this harm reduction, because the training costs of most models is horrific in environmental terms, and to use them is to encourage their development, but at very least, you don't have to use live service LLMs like Copilot, Gemini, and ChatGPT if you're looking to understand what's going on with this stuff.
#AI #chatbots #LLM
