Expanding Horizons

Welcome back to the AI Geekly, by Brodie Woods, bringing you yet another week of fast-paced AI developments packaged neatly in a 5 minute(ish) read.

TL;DR Open Source Domination; Portability Primacy; Hardware Hustle

This week, the AI landscape is abuzz with activity, showcasing the rapid evolution and increasing accessibility of AI technology. The three themes this week are Accessibility (open source) Portability, and Throughput. Open-source models continue to gain traction, challenging established players like OpenAI and Google with highly performant models available to all. Meanwhile, researchers are continuing to develop models that are both more powerful than prior iterations, but also much smaller —small enough they can run on a smartphone. The last ingredient here is speed —you can have the smartest model in the world, but if it’s too slow it’s DOA. Chip start-up Groq (not to be confused with the xAI Grok model) shows us the meaning of being lonely (at the top) as its chip delivers blazing-fast responses using the newly released Llama 3 model.

Like the Cream of the Crop
Open source models rising to the top

Snowflake Arctic: Snowflake's open-source LLM, Arctic, is making waves with its unique Dense-MoE Hybrid transformer architecture. Boasting 408B (that’s big) parameters and 128 experts (that’s a lot), it offers a powerful and flexible solution for enterprise use cases.

Apple OpenELM: Apple gets in on the open-source love with its OpenELM family of language models, designed for on-device performance (more on that below). These models prioritize privacy and efficiency, catering to the growing demand for local AI processing. Apple’s angle here is to open its models-up to the OS community in order to benefit from the free labor contributed.

Meta Llama 3's Success: Meta's Llama 3 models continue to gain popularity, surpassing 1.2 million downloads and spawning over 600 derivative models on Hugging Face in only one week. The open access and impressive performance of Llama 3 solidify its position as a leading contender in the LLM space, leaving many to wonder whether Llama 3 should replace GPT 3.5 as the default model for low-cost use cases.

Takeaway: It was a great week for Open Source. The community is thriving, and corporate partners seem to be engaging in mutually beneficial behaviors, which is a nice change from the anti-competitive practices we often see from many large tech companies.

Have AI, Will Travel
Smaller models getting smarter

What it is: Along with Apple’s open source models mentioned above, we draw attention to Microsoft’s recently announced phi-3 family of models, particularly phi-3-mini, which demonstrates the potential for powerful AI on mobile devices. Its compact size and competitive performance make it suitable for a range of enterprise applications on smartphones.

What it means: We’re seeing smaller models become more performant in recent releases, almost across the board, often approaching the capabilities of much larger models released only a year ago. We suspect that as AI matures, and society continues to feel around for the right modality of use, that we will ultimately settle on a Bring Your Own AI (BYOAI) model of highly bespoke, individualized models.

Takeaway: Wider adoption of AI requires AI that is open, accessible, and usable across a multitude of use cases business and personal. This is easier said than done as many use cases are precluded from the use of cloud AI solutions due to connectivity, privacy, or regulatory considerations. Thus, there is a need for performant AI on so-called edge devices (including smartphones). Researchers have been catering to this very need with their latest series of model releases.

Hardware Race Heats Up
It’s not just the software that counts

What it is: Two noteworthy hardware updates from Groq and xAI this week serve as a good reminder that it’s not just the models that matter, it’s what hardware you run them on (Groq) and train them on (xAI).

Groq's Breakthrough Chip: Startup chip company Groq's innovative AI chip achieves an impressive 800 tokens per second on LLaMA 3 (a token is roughly equivalent to a word). This development underscores the ongoing efforts to improve hardware efficiency and performance for AI workloads. At 800 tokens a second it takes mere seconds to write an entire book. The potential of such a pace of content creation is worth considering.

xAI Raises $6 Bn: Yes, you read that right, $6 Bn. The richest man in the world has been able to raise this eye-watering sum as he looks to dramatically ramp-up the compute capabilities of his company, xAI as part of a now very public rivalry with Open AI.

Takeaway: We remain of the view that outsized investments like this demand outsized returns. As our readers know, we are bullish on the space, but we are also practical. We agree that these large sums are needed in order to progress the technology, but we question how long shareholders in these various ventures are willing to hold the bag. That said, Microsoft just noted a 17% increase in quarterly revenue last week which it partially attributed to its AI copilot business, so at least that’s a promising green shoot.

About the Author: Brodie Woods

With over 18 years of capital markets experience as a publishing equities analyst, an investment banker, a CTO, and an AI Strategist leading North American banks and boutiques, I bring a unique perspective to the AI Geekly. This viewpoint is informed by participation in two decades of capital market cycles from the front lines; publication of in-depth research for institutional audiences based on proprietary financial models; execution of hundreds of M&A and financing transactions; leadership roles in planning, implementing, and maintaining of the tech stack for a broker dealer; and, most recently, heading the AI strategy for the Capital Markets division of the eighth-largest commercial bank in North America.