Skip to main content
November 22, 202618:47

Don't Buy 128GB MacBook For Local AI: Here's Why

By Samuel Gregory

About this video

Your 128gb MacBook Pro is officially too small for the newest AI era. In this video, I test the M5 Max with 128gb memory against the latest "open" models from Thinking Machines and Mistral to see if local AI is still viable for developers. Key takeaways: - Why Inkling 2 Small and Mistral Large 4 physically cannot fit on any current MacBook. - The simple maths to calculate if a model will fit on your machine. - The massive difference between total parameters and active parameters in MoE models. - Benchmarking the 32gb vs 128gb vs Cloud experience for web development. - Why the cloud is 10x faster and costs only $0.04 for complex coding tasks.

The 128gb MacBook Pro is officially too small for the best AI

Buying the most expensive MacBook Pro available today will still leave you unable to run the world's best open weights AI models. We have reached a point where the hardware in our pockets and laps simply cannot keep up with the sheer scale of modern machine learning.

The math of memory

To understand why a 128gb machine fails, you only need one simple sum: Size in bytes equals parameters multiplied by bits per weight, divided by 8. When you apply this to the new Mistral Large 4 with its 1tn parameters, the numbers become terrifying. Even at a heavily compressed 1 bit precision, it requires roughly 233gb of memory. Your 128gb laptop is not just slightly short: it is 105gb under the minimum requirement.

Total vs Active: The MoE trap

The rise of Mixture of Experts (MoE) models like Inkling 2 Small has confused many buyers. These models might only use 17bn active parameters for a single word, making them fast to run, but they require all 594bn parameters to be loaded into memory simultaneously.

For the 128gb user, this means:

  • Inkling 2 Small (594bn) requires 172gb minimum.
  • Mistral Large 4 (1tn) requires 233gb minimum.
  • Inkling Small (276bn) fits at 3 bit precision (91gb).

The performance reality check

In my side by side testing building a pricing page, the results were stark. The cloud version of Inkling 2 Small finished the task in 2.23 mins for a cost of just $0.04. The best model that would fit locally on a 128gb machine took 4.57 mins. Most shockingly, a 27bn model running on a 32gb tier took a massive 25.5 mins to complete the same work.

Is local AI dead?

Local AI is not dead, but the dream of running "the best" models on a laptop is over. A 128gb machine is the baseline for a usable local development experience, offering a middle ground between the snail's pace of 32gb machines and the high cost of enterprise cloud hardware. However, if your goal is purely quality and speed, the cloud wins every single time for less than the price of a postage stamp.

Transcript▾

This is a MacBook Pro with an M5 Max chip and 128gb of memory. That is the most memory you can buy in a MacBook today. And if any laptop should be able to run the newest AI open models, it is this one. In the last 2 weeks, 2 big open models came out. Thinking Machines released Inkling 2 Small, and Mistral released the weights for Mistral Large 4. Neither of them fit on this laptop. So in this video, I am going to explain why using simple maths you can do yourself. Then I am going to find out what does fit and test it. I will give the same coding task to a giant model running in the cloud, to the best model I can fit on this laptop, and to a smaller model that would run on a 32gb machine. Then we will see how big the difference really is.

Here is the plan for this video. First, I will tell you about the 2 new models and what makes them interesting. Second, I will show you why they do not fit in 128gb. It comes down to one sum. Third, I will go through the memory sizes Macs come in, 16, 32, and 40gb and up, and tell you which models make sense at each size. Fourth, I will run a test building a pricing page. And finally, I will tell you which size of Mac is good enough for real web development work.

Open means the company publishes the model weights so anyone can download it. It does not mean it will run on your computer. Inkling 2 Small has 594bn parameters. It is a mixture of expert model, or MoE. It has 384 experts, but only 8 are used per word. So although it has 594bn total, only 17bn are active. Mistral Large 4 is even bigger at roughly 1tn parameters with about 50bn active. The smallest version is about 233gb. You would need a Mac Studio with 512gb of memory for that.

The sum is: Size in bytes equals parameters times bits per weight divided by 8. At 16 bits, Inkling 2 Small is 1,100gb. That is more than a terabyte. Quantisation helps, but even a 2 bit version is 173gb. Still too big for a 128gb laptop. Remember that MacOS and apps need memory too. I set aside 16gb for that, leaving 112gb for the weights.

At 16gb, you can run 9bn parameter models. At 32gb, you can run 27bn. At 128gb, the biggest thing I can fit is Inkling Small at 3 bits, which is 91gb. In my test building a pricing page: The cloud model took 2.23 mins and cost $0.04. Inkling Small on this laptop took 4.57 mins. The 27bn model on the 32gb tier took 25.5 mins.

If you are buying 128gb to run the newest models, it will not work. These models want 2 to 4 times that. 128gb is the first point where local work feels usable, but the cloud is still faster and costs pennies. Keep on vibing.