Sakana: Advantages of Smaller AI Models
Two brilliant experts walked away from promising careers at Google to carve out a new frontier in artificial intelligence. Their inspiration? The natural world, specifically the behavior of animals like fish and bees.
The Philosophy Behind Sakana's New Approach to AI
Within the new system, each individual module doesn't quite match the prowess of models developed by renowned industry frontrunners. However, imagine the synergy when these digital mimics of marine creatures and insects are united in schools or hives.
David Ha and Llion Jones have dubbed their forward-thinking AI venture “Sakana.” In Japanese, “Sakana” primarily means “fish”. Yet, it's also colloquially used to describe elaborate fish dishes, such as sushi, and products related to bee farming. The brand captures their inventive ethos, drawing inspiration from the communal behaviors of these species.
Sakana: An AI Meld of Bees and Fish Source: Sakana's account on X
While the AI behemoths are engrossed in creating expansive, sprawling systems, Sakana takes a contrasting route, exploring the nuances of smaller AI models. Collaboratively, they create a 'swarm' of programs, aspiring to match the prowess of their more sizable counterparts.
Ants move around and dynamically form a bridge by themselves, which might not be the strongest bridge, but they can do it right away and adapt to the environments,” Ha said. “I think this sort of adaptation is one of the very powerful concepts that we see in natural algorithms,explains David Ha.
Advantages of Sakana's AI Model
Large language models present their own challenges. For instance, an issue in even the smallest portion of the software can potentially crash the entire system, especially since many don't have backup mechanisms to temporarily compensate for malfunctioning segments.
The human brain still works better than our best AI. So, clearly the human brain is doing something right that we haven't quite caught onto yet,Jones notes.
The pioneering vision of David Ha and Llion Jones could pave the way for both economical and highly adaptable AI solutions. Their ambition doesn't stop there; they're gearing up to craft a generative artificial intelligence that can produce content from user prompts.
Sakana's Genesis
David Ha and Llion Jones aren't just enthusiasts jumping on the AI bandwagon. They're respected powerhouses in the artificial intelligence domain. Jones, for instance, played a crucial role in penning the influential paper titled "Attention is All You Need," under the auspices of Google. His work has since shaped countless AI innovations. On the other hand, Ha's credentials boast significant contributions to generative AI during his tenure at Google Brain.
In the buzzing halls of Google's Tokyo branch, the duo realized a stark truth: innovation in sprawling corporations often becomes a victim of its own bureaucratic mire, stifling pace and creativity. Outside the behemoth's shadow, the brains behind Sakana have found a swifter, more agile path to innovation.
