One morning in 2019, Adebayo Alonge was preparing to enter a hotel room in Cape Town, South Africa, to present his startup's AI-based solution to a critical problem in African healthcare: detecting counterfeit drugs, which claim the lives of thousands of people across the continent every year.

"RxScanner" device for detecting counterfeit drugs.

AI-based counterfeit drug detection device.

The "RxScanner" is a handheld spectrometer that scans a pill using infrared light, then sends the molecular profile (molecular composition) of the drug to an AI model equipped with a pharmaceutical database. Within seconds, the AI identifies the drug through its molecular profile or flags it as counterfeit.

Pharmacies were using the system in more than a dozen countries, including Ghana, Kenya, Myanmar, and Nigeria, Alonge's home country. But on that morning in South Africa, the system did not work. "I was shocked," says Alonge. The spectrometer was connected to the AI model, but the data center was 14,000 kilometers away, and internet speeds were limited. The IEEE Spectrum website quoted Alonge as saying, "Our server was in the United States, and it was taking me over 5 minutes to get a single scan result."

So, Alonge immediately asked his engineers to shrink the AI model into a smaller, low-power, offline version that could run entirely on his Android phone. They accomplished this in two hours, which saved the demonstration. More importantly, this work resulted in a new version of his device capable of verifying the authenticity of a pill in places lacking high-speed internet, computers, or even reliable electricity. Alonge also became an advocate for this type of "Tiny AI."

Tiny AI for global healthcare differs significantly from Large Language Models (LLMs), hyperscale data centers, and multi-billion-dollar investments, and is far removed from the debates surrounding AI consciousness in wealthy nations. But for millions of people around the world, the only AI that matters—and often the only kind available—is "Tiny AI."

According to a report released by the World Bank in November, only 0.7 percent of internet users in the world's poorest countries used ChatGPT, compared to a quarter of internet users in more developed nations.

Ajay Banga, President of the World Bank, said at the World Economic Forum in Davos last January: "Most people discuss AI from the perspective of machine learning/generative AI. But that requires massive computing power, electricity, huge data, and skilled personnel to manage it. Outside the developed world, with the exception of perhaps India and China, very few countries have these components."

In contrast, small AI systems can offer useful, and even life-saving, services to residents of areas that lack all these resources, as Banga explained.

In India, where government AI plans call for the development of more small-scale AI systems, many of these systems work for the benefit of farmers.

For example, a drone-based system developed by Bala Murugan and his colleagues at the Vellore Institute of Technology in India captures images of cashew plants and quickly identifies those showing spots indicative of disease. All processing is done on the drone itself, so there is no need for an on-site computer or a connection to a central server.

Using small language models trained to solve a specific problem, sometimes running on cheap, low-power hardware, other Tiny AI applications have been developed to identify ant infestations in vineyards in Uruguay, detect the presence of malaria-carrying mosquitoes in several countries, and conduct electrocardiograms from an Arduino device in parts of Brazil that lack more sophisticated equipment.

"This is the most important field in AI today. It is growing very fast," says Marcelo José Rufai, a professor at the Institute of Engineering and Information Systems at the Federal University of Itajubá in Brazil, who participated in all three projects.

For Alonge, Rufai, and other proponents of these trends, Tiny AI is not just a "promising trend," as described by the World Bank report; in the long run, it may be the optimal form of AI that will impact the lives of the most people and remain sustainable now that some large models have become too expensive for most users.

"I think the future of AI does not lie in a single massive model at the center of the system, but in millions of small, precise models widely distributed, where each one solves a specific problem in a specific context," says Alonge. This is partly because a large segment of humanity—including residents of some wealthy and developing nations—cannot access the latest cutting-edge models. But he adds that it is also because these models are unsustainable.

There is no precise definition of the term "Tiny AI," but the term is usually used to describe language models that do not exceed a few billion parameters. (Compare this to state-of-the-art models that may contain more than a trillion parameters.) This small size is enough to run them directly on a phone or a Raspberry Pi computer. This allows these applications to work on devices without needing a connection to a data center, consuming only a few watts of power, which is often derived from a battery or a solar panel.

Rufai says that despite their small size, these models are not fundamentally different from large AI models. Many small language models have been created by "pruning" large models, or removing parameters unnecessary for the task. The result is a system that is generally less capable, but still excellent at the specific task for which it was pruned, says Rufai.

Other small models are created through "distillation." Rufai says these systems are trained to mimic a large model until their performance approaches that of the "teacher." In other cases, the precision of a larger model is reduced; for example, a model operating on a 32-bit architecture can be run on 8-bit designs. In cases where a machine learning application is used for data classification or pattern prediction (like an ant invasion), it is trained from scratch on a small device, not derived from a larger model at all.