AI Geekly - The Chipset Jetset

Welcome back to the AI Geekly, by Brodie Woods, bringing you yet another week of fast-paced AI developments packaged neatly in a 5 minute(ish) read.

TL;DR Hopping to Blackwell; AI Apple Core; Hidden Signals

How can we describe Nvidia’s AI status in the tech world and equity markets? Only one word will suffice: ‘Imperial’. The full strength of that empire and its throngs of adoring subjects were on display at Nvidia’s GTC conference this week as NVDA showcased its latest advances in hardware and software. Apple was in the news as well, probably feeling a little jealous of Nvidia’s rockstar status given the relative perceptions of the two companies (consensus would view AAPL as behind, but we disagree). To its credit, Apple announced a critical AI partnership and released a new suite of its own AI models. Lastly, mark this under the “there’s still hope for us yet” column: two breakthroughs this week in healthcare show incredible promise for those with neurological conditions. First, Neuralink live-streamed a demonstration of its prototype being used by a quadriplegic patient to perform tasks, and even play chess, using thoughts alone. Second, Stability AI and peers showed-off an AI model that can read fMRIs and reproduce images and text prompts being visualized by the human brain.

Step Aside Hopper, it’s Time for Blackwell
Nvidia announces new flagship B100 and GB100 chips

What it is: At its annual GTC conference last week, Nvidia unveiled its next generation ‘Blackwell’ AI-focused chips, the B100 (GPU-only), B200 (2x GPU dies on a single card) and the Grace Blackwell GB200 (2x GPU+ 1x CPU).

What it means: These cards raise the bar considerably. Nvidia was already the top when it came to AI cards with its prior generation ‘Hopper’ cards (H100, GH100, H200, GH200). Nvidia’s top of the line B200 offering boasts 25x the energy efficiency, 5x the horsepower (20 petaflops at FP4) and 7-30x the inference performance. In layman’s terms, this is like moving from a Honda Civic hatchback to a Corvette Stingray.

Why it matters: The rate of growth in compute is shocking even to us. It’s hard to fathom these levels of performance enhancement generation over generation, especially given that these chips are produced at the 4nm scale (due to 3nm production issues at chip fabricator TSMC), the same as the prior Hopper chips. To squeeze such massive performance gains out of these chips at the same process scale is truly impressive. The next logical consideration is what this does to upcoming AI models. These chips will allow the training of models 10x the estimated size of OpenAI’s GPT-4 model which underlies ChatGPT.

Consider the scale: We want to impress on you the overall massive scale of compute that is being delivered in the next year. We expect roughly a 1000x increase in global compute dedicated to AI over the course of the next 12 months. Keep in mind that all of the GPUs that were used to train GPT-3, GPT-4, Gemini, Claude, Llama, Mistral, Falcon, etc. still exist and are still available to churn away training new AI models,. We’re only talking about incremental compute being added with next generation cards to the already massive current global compute supply. 

The AIpple Doesn’t Fall Far
Updates on Apple’s AI ambitions

What it is: Apple has been burnishing its AI bona fides as critics have derided its relative silence on AI compared to other tech giants. Management recently canceled its secretive Apple Car project, redirecting the resources to its GenAI programs. Rumors emerged that it will partner with Google to bring its AI models to Apple devices. Finally, it released its new MM1 multi-modal models just this week. 

What it means: Looking at major trends in technology over the past decades, it’s hard to argue that any of them really took off until Apple eventually came along and democratized them to the masses. Were there really smartphones before the iPhone? (Shh! Canadians and physical keyboard lovers, it’s done), were there really MP3 players before the iPod (Microsoft Zune FTW!)? It’s the same story with music streaming, spatial computing, and more. For whatever reason, time and again Apple bests its competitors in one simple area others ignore: user experience and clean design.

Why it matters: We think Apple has a clear strategic vision with regard to its AI plans (recall it has had dedicated AI chiplets in its hardware for longer than any of its peers). With the release of its MM1 models, Apple is demonstrating its own capabilities in the LLM space. The models are state of the art multi-modal models including both dense models up to 30B parameters and sparse MoE (Mixture of Experts) models up to 68B parameters. While these models aren’t game changers, they demonstrate the company’s par-with-industry GenAI chops. Just remember: Apple is never the first, but they’re often the best.

The ‘System’ in Nervous System
Neuralink and Stability AI interface with the human brain at new depths

What it is: Two impressive breakthroughs this week as Neuralink live-streamed a paralyzed patient using his cybernetic implant to perform everyday tasks and play chess using only his thoughts. The second was a demonstration of MindEye2, technology co-developed by Stability AI which can process and reproduce images being imagined by the human brain by analyzing fMRIs.

What it means: Despite all of the AI fear-mongering about job loss, energy use, and apocalyptic events, there are many AI benefits that go underreported. We’d even go so far as to suspect that much of the reason the Neuralink story is getting so much press is less because of the life-changing promise this technology carries, and more to do with morbid curiosity. Either way, we’ll take it if it helps get the word out that these AI models may hold the key to solving a great many of the world’s seemingly insurmountable challenges.

Why it matters: We’ve mentioned in prior Geeklies, but it bears repeating. There is a cost to delaying, slowing, and overregulating AI and it is a very human cost. The longer we take to introduce widespread self-driving vehicles, the longer thousands will perish in avoidable road accidents. The more we restrict, limit and tie-down healthcare innovation, the fewer people’s lives will be forever improved through these life-changing technological breakthroughs.

What can be done: It’s important to take risks now. Societies, companies and individuals who avoid dabbling in AI today are taking a major risk, it’s the risk of becoming irrelevant. We need to be brave enough to take the chances now, the alternatives are far too costly.

About the Author: Brodie Woods

With over 18 years of capital markets experience as a publishing equities analyst, an investment banker, a CTO, and an AI Strategist leading North American banks and boutiques, I bring a unique perspective to the AI Geekly. This viewpoint is informed by participation in two decades of capital market cycles from the front lines; publication of in-depth research for institutional audiences based on proprietary financial models; execution of hundreds of M&A and financing transactions; leadership roles in planning, implementing, and maintaining of the tech stack for a broker dealer; and, most recently, heading the AI strategy for the Capital Markets division of the eighth-largest commercial bank in North America.