AI and the Year of the Dragon

Welcome to Improbable Automata !

2024 was the Year of the Dragon, which occurs every 12 years in the Chinese Zodiac and symbolize wisdom, power, and divinity.

The previous Year of the Dragon was in 2012. Apple had just launched Siri. 12 years before that, Nokia was releasing new cellphones. Now its 2024, humans spent billions of dollars training on trillions of tokens, and there still isn’t a better Siri?

I understand technological progress can be perplexing. Even more challenging when you’re bad at interpreting exponentials, as most humans are.

Moore's Law, which started as a simple model of the doubling of transistor density every two years, is now the heartbeat of technological progress.

At our current point on the graph, a breakthrough technology—generative artificial intelligence—is starting to mature. Organizations are adopting new technological paradigms that enable them to work in ways that were unimaginable in the prior era of technology. We've taught computers to think, and now, humanity must contend with what that means in a competitive market where all incentives lead to hyper-automation.

This journey will be challenging yet exciting. Generative AI offers much more than previous technological advancements. Perhaps most importantly is its capacity to empower individuals to do the work of teams—not just by doing more of what they could do already, but of what they previously could not.

The window of opportunity to build the next Siri won’t be open for long. Apple may have lost its footing, but many said the same about Google, whose barrage of product launches this year showed the world otherwise.

2024 felt like the year AI stopped being merely impressive and started being practical. Token costs have fallen by orders of magnitude, and the breakthroughs didn’t stop either. We saw everything from multimodal models and autonomous agents solving mathematical problems that were supposed to resist AI "for years" to decentralized networks threatening the cloud computing status quo. It's been one hell of a ride.

As our genesis post for Improbable Automata, we’re reviewing the biggest highlights from 2024 to set the stage for what is to come for AI in 2025.

Multimodal had its moment

Remember when AI was just autocompleting your texts? Haha. Simpler times. Now that OpenAI’s finally released Sora we can all generate videos. The age of infinite media is upon us!

No, but seriously. After almost a year of waiting, and watching both Google’s Gemini and Meta’s Llama get big multimodal upgrades, no one is impressed anymore. Literally anyone with access to the internet can generate videos of practically anything, for free (assuming it passes the Great Content Filter).

Small models got smarter

Bigger wasn’t always better in 2024. Microsoft's Phi-3 family along with Gemini Flash models showed that small models have utility.

Not only are they cost effective, but they are fast, scalable, and resource efficient. In the wake of inference-time-compute, or ‘thinking’ models like OpenAI’s o1, developers can trade tokens for intelligence. But when spending more tokens on inference, we typically sacrifice speed. With small models, we may be able to make domain specific trade-offs that enable high cost savings with similar or on par performance with state-of-the-art models.

We remembered the power of personalization

Among the drama of silicon valley this year was Google’s surprising acquisition of Character AI. Character AI has created a platform where millions of users create custom character agents that they interact and grow with over time. Noam Brown, the founder, was previously a Google AI researcher himself. But not just any researcher—one of the core authors of the Attention is All You Need Paper that kicked of the generative AI revolution.

Noam’s success with CharacterAI reminded the world of the value personalization can bring to a product. Now, new AI personality research and emerging open source frameworks support the production of hyper-personalized software.

Google finally made a move

After spending 2023 looking like they might have missed the AI revolution, Google decided to remind everyone why they're Google. The rebranding of Bard to Gemini came with actual substance: Gemini Advanced powered by Ultra 1.0, a premium subscription that people actually wanted, and the Gemma open-source models that democratized access to serious AI capabilities.

But Google didn’t just release models, they released a whole new suit of products, including a few good ones like:

• NotebookLM: turn any content into a podcast episode

• Veo 2: state of the art video generation that makes OpenAI’s Sora look like a research preview

• Gemini Flash: a small multi-model LLM for the agentic era

It turns out the company that essentially invented modern AI wasn't going to sit this one out. Who knew?

AI agents gained their footing

2024 was the year AI agents went from unreliable demos to real world experiments. From building 5-minute sales research agents to autonomous AI trading memecoins, we saw the beginning of a new class of technologies beginning to take shape.

AI Agents like the Terminal of Truths have taken grants from VC’s like Marc Andreessen and hold memecoins with market caps in the hundreds of millions. The crypto industry rallied behind as the decentralized AI movement continues to grow. AI agent frameworks like Eliza and its DAO, ai16z, currently has a market cap of over $2.5B and growing. As if we didn’t need more validation for personalized agents, Meta revealed its plans for AI characters on its own platforms.

I’ve seen predictions suggesting that half of business social media content will be AI-generated by 2026. This might be missing the point.

Robots became reality

Waymo expanded its robotaxi services into more cities this year, serving over 150,000 trips each week across fleets. It’s main competitor Tesla unveiled its own robotaxi model this year, while it’s Optimus robots seem to be improving rapidly, demonstrating their ability to perform a range of tasks like bartending, laundry, and dancing.

AI started speedrunning science

Sakana AI's "AI Scientist" demonstrated another unprecedented breakthrough for the year 2024: end-to-end machine learning research automation. If extended to other domains, this could entirely reinvent how scientific research is conducted.

As if Google had something to prove this year, they shipped multiple scientific AI products including AlphaFold 3 expanded beyond protein folding into modeling complex molecular interactions, while GraphCast achieved unprecedented accuracy in medium-range weather forecasting.

The worlds biggest computer was built

Musk’s OpenAI competitor, xAI, did something probably only an Elon company could do: built a state-of-the-art data center in just 19 days. It’s nicknamed “Colossus” and its the most powerful computer ever built, orchestrating 100,000 NVIDIA H100 GPUs. This thing is so massive the city of Memphis can barley keep up.

Sounds like we’re going to get a few more of those nuclear reactors back up and running.

Decentralized compute found a market

With rising compute demands, there is a naturally need for alternatives. It is possible that access to compute becomes as important to freedom as access to the internet is today.

Luckily, decentralized AI infrastructure market started to take shape this year, with the DePin market cap reaching $35B. Startups such as Internet Computer, Akash Network, Ocean Protocol, Lilypad, and Rare Compute are just some examples of many startups building some much needed infrastructure for open source technology to thrive in the AI age.

AI solved math (well, not quite.)

In the 12th day of 12 days of OpenAI, we got a surprise announcement: o3—the most advance model of the omni series from OpenAI—scored a massive 25% on the FrontierMath benchmark.

These problems are at a difficulty level that Fields Medalists like Terrence Tao—a literal mathematical geniuses—thought would resist AI for years. For context, previous generations of models could only score 2%.

This is the kind of leap in performance where you really need to be paying attention. AlphaGeometry matching human gold medalist performance almost feels like a footnote in comparison. Almost.

2025: Year of the Snake

2025 is the year of the Snake, which symbolizes wisdom, growth, and transformation.

It’s clear that generative AI technology is entering the early stages of maturity. Engineering frameworks for building chatbots, agents, and systems for retrieval augmented generation (RAG) have become established open source companies. If builders were heads down in 2024 we’re about to see a lot of launches.

We should expect to see:

1. Useful Agents: With good enough models and established engineering frameworks, theres nothing stoping agents from taking off in 2025. I’m sure we’ll see lots of weird behavior and have reliability issues to work through, but we’ve crossed a point where if you throw enough engineering hours at a problem, you can probably solve it if the laws of physics don’t get in the way. Businesses will be able to run a lot leaner with teams of agents, so expect to see startups and SMBs sprinting towards adoption.

2. Game of Chips: 2025 being the year of rapid adoption means the ever-growing need for compute will not be satiated. Decentralized compute alternatives will experience rapid adoption. The biggest AI companies will be bidding for a spot on NVIDIA’s waitlist. The incentives for reduced compute cost will lead the adoption of efficient small models like Gemini Flash and the Phi series of 1-3B parameter models.

3. Hyper-personalization: The future isn't one model to rule them all; it's millions of specialized models evolving with their users. This has implications beyond business software and personal productivity tools—this is the creation of ‘living’ software agents that grow and evolve within digital environments and social structures.

2024 wasn't just another year of AI development - it was the year AI started solving problems we thought it wouldn't touch for years. The question isn't whether AI will transform businesses and industries; it's whether we're ready for how quickly it's happening.

This newsletter will continue tracking these developments, but with a focus on what actually matters for builders and decision-makers. No hype, no fear-mongering - just analysis of what's happening and what it means for those of us building in the augmented age.

Until next time,

Alfie