Acknowledgments: This article was made possible by contributions and collaboration from the entire Machine Learning team at eyeo. Thank you for running the benchmarks, crunching the numbers and reviewing the content.
Artificial intelligence (AI), an undeniable force in the modern world, can lead to positive developments or be used for more sinister motives. At eyeo, we are committed to using AI and machine learning (ML) models ethically and sustainably as we pioneer the automation of commercial ad-filtering to protect user choice while providing solutions for publishers to monetize their content and advertisers to connect with consumers on mutually agreed terms. Our series “eyeo and Ethical AI” focuses on how we integrate AI and machine learning with our engineering work, adhering to fundamental values like individual rights, privacy and non-manipulation, while minimizing the negative impact this technology can have on the environment.
Sustainable AI is defined bytwo branches–“AI for sustainability” and “sustainability of AI”. (1)AI for sustainabilityis using AI to help promote environmental sustainability, e.g. using AI to decarbonize media by optimizing the supply chain of ads (supply-path-optimization) in the advertising industry. (2) Thesustainability of AIis developing AI with minimal negative costs to the environment.
While much of the ML community is moving towards bigger datasets and multi-billion parameter models, we are moving in the opposite direction— making our models as lean and as energy efficient as practically possible. This blog will illustrate our approach to ensuring that our entire machine-learning infrastructure is just as effective as any other modelbut alsoalignswith our purpose of transforming the web into a safe, sustainable and accessible place.
Industry-wide emission standards
If we’re talking about using AI ethically, the environmental impact of the technology cannot be ignored. The energy consumption of popular ML models is staggering and growing exponentially. Here are some figures relating to the training and inference of Machine Language Models according to a2019 study by the University of Massachusetts Amherst:
GPT-3(175 billionparameter model), on which the eponymousChatGPTis based, was trained over the course of six months.4,789different versions of the model were trained, requiring9,998 total days’ worth of GPUtime (more than27 years). Taking all these runs into account, the researchers estimated that building this model generated over78,000total poundsof CO2emissions.
BERT,which forms the de-facto base of most LLMs, has a carbon footprint of roughly1,400 pounds of CO2during one training phase– equivalent to a round-trip trans-America flight for one person.
The inference stage consumes even more energy than training.According to this 2019Forbes article, Nvidia estimates that80 to 90 percentof the resources consumed in a neural network happen when deploying a trained model to make predictions (in inference) rather than in training.
eyeo’s approach: Lean and green
To protect user privacy, it is imperative for our models to run in the browser, which has limited computing power and memory thus necessitating the need for leaner models. To focus on keeping our carbon footprint close to zero without compromising effectiveness, we have consciouslyadopted the following best practicesto optimize our ML models and support infrastructure.
Sparse modelarchitectures: reduce the computation complexity by3x-10xwhile upholding the ML quality and prediction performance. We carefully selected the feature set and trained thegraph convolutional neural networks (GCNs)to handle sparse graph structures efficiently.
Model Weight Quantization: selecting32-bit floating point numbersover 64-bit floats for our weights reduces the model binary size to~700 KB(BERT base model weighs440 MBin comparison).
Limiting the number of layers reduces the total trainable parameters to ~145K(BERT base has110Mparameters).
Cloud-based computationrather than on-premise computation reduces energy usage and therefore emissions by1.4x-2x. Cloud-based data centers are new,custom-designedwarehouses equipped for energy efficiency for 50,000 servers, resulting in high-qualitypower usage effectiveness(PUE).
We choseGoogle Cloud Platformas our cloud service provider since it runs57 percentof its services on renewable energy and specifically selected regions in the EU that support renewable energy, further reducing the gross carbon footprint by5x–10x.
Using processors (GPU/TPU) and systems optimized for ML training, versus general-purpose processors, can improve performance and energy efficiency by2x–5x.
By the numbers
In theory, all those best practices sound helpful but are they really effective? We plugged our dataset into theMachine Learning Emissions Calculatorto estimate our carbon footprint. Here are the findings:
Training our general-purpose ad-filtering model experiments were conducted:
Our model saves more than1000xthe energy than the standard Large Language Models for training:
Inference energy consumption
On average our energy consumption is~0.1 kWh per user session.Much less than the energyconsumed by a fluorescent light bulb and an incandescent lamp:
TheEPAestimate that 1 kilowatt-hour of energy consumption generates 0.43 kg of carbon emissions. Our model, which only consumes 0.1 kilowatt-hour of energy per user session, emits 0.043 kg of carbon dioxide.
Optimizing the sustainability of AI
If we are going to create something to promote sustainability, we need to consider the ethics and synchronicity of doing it in a sustainable way.
eyeo Director of Engineering, Dr. Humera Noor Minhas states,
“Through our commitment to sustainability and ethical AI, we harness the power of machine learning responsibly, ensuring our technological advancements align with a greener future.” It is a clear sign of our imperatives to sustainability that all signposts and stations of the process are approached with an eye toward optimization.
We encourage others to look at their own models to see where they can reduce their carbon emissions. In this challenge, you may just find an opportunity.
Come back for the next blog in the series “eyeo and Ethical AI” where we discuss using AI to protect user choice.
Machine Learning and AI will be one of thefocus topics of the upcoming 2023 Ad-Filtering Dev Summit in Amsterdam and online from 4-5 October 2023.Registration is now open, save your spot here.