Über dieses Video
Apple is secretly building a way to turn your M4 Mac into an AI server farm through a new app called Harbor. By leveraging Thunderbolt 5, you can now cluster multiple Macs together to pool memory and run massive local LLMs without ever touching the cloud.\n\nKey Takeaways:\n- Harbor is an Apple-owned local LLM server app developed via Claris.\n- It enables hardware clustering over Thunderbolt 5 to increase memory and concurrency.\n- Models are downloaded directly from Hugging Face for 100% private use.\n- The app uses private APIs for better system integration than third-party tools.\n- It currently lacks KV caching, which is a missed opportunity for workflow persistence.
Apple is building a wall around your data, and it is the best thing they have done for AI in years.
While the world focuses on cloud subscriptions and privacy trade-offs, Apple has been quietly developing a tool that could revolutionise how we interact with large language models (LLMs). The app, titled Harbor, emerged via Apple-owned sister brand Claris. Although it was swiftly removed from the App Store, the implications of its existence are massive.
Local AI Goodness
Harbor is designed to run open-weight models locally on Apple Silicon. This means your data never leaves your hardware. In an era where data leaks are common, the promise of local AI that never reaches the cloud is a significant shift. The app allows users to download models directly from Hugging Face and chat with them in a sandboxed environment.
The Power of the Cluster
The most intriguing feature is the ability to connect multiple Mac computers over Thunderbolt 5. By daisy-chaining machines, such as an M5 Max and several M4 Pro Mac Minis, users can pool unified memory. This enables the hosting of much larger models and higher concurrency than a single computer could ever handle alone.
Missing Pieces
Despite the sleek, Apple-like interface, Harbor is currently a basic offering. It lacks advanced features found in competitors like OMLX, specifically KV storage caching. Caching is vital because it allows your work and conversations to survive application restarts. Without it, Harbor feels more like a polished prototype than a finished professional tool.
Apple clearly realises that the future of AI is local. By using private APIs and deep system integration, they are creating a platform that 3rd party apps will struggle to match in efficiency. Whether you use LM Studio, Olama, or wait for the official release of Harbor, the message is clear: your Mac is becoming a personal AI server.
Transkript▾
Why is no one talking about the fact that Apple is working on an LLM server for the Mac? Now, I do not want this to turn into an AR reading channel or anything like that, but unfortunately, they have taken down the app from the App Store and so I cannot test it and show you kind of what it is capable of. It is called Harbor. They are working on it with one of their sister brands, Claris. It is Apple owned.
Essentially, it runs open-weight models locally on Apple Silicon, and your data never leaves your hardware and nothing reaches the cloud. All of that local AI goodness. You can connect multiple Mac computers over Thunderbolt 5 to pool unified memory enabling large models and higher concurrency than a single computer can handle.
Now, we have seen this before. There was an app released, I forgot if it was Kiko or something, but it was very nerdy. It was a terminal setup and it had to restart your computer, which the settings would reset after every time you restart your computer. It was a bit of a pig's ear. This app looks clean. It looks Apple-like. And the reason why this is interesting is because when Apple are working on something, you assume they have got access to private APIs or that they just know the system a lot better than third party app builders.
Now, you can look at the UI here and what can we work out from this? You have got a main MacBook Pro that they are working on, an M5 Max with 48 GB of memory, and it is connected to 2 Mac Minis via Thunderbolt 5. And those Mac minis are connected to each other via Thunderbolt 5 too. And they are two M4 Pro, one with 16, one with 24 GB. So, all of this is working together to serve or host a local open weights model and use it, whichever machine chooses to use it.
I guess now we over here we can look at the fact that there is an API endpoint and it is being served over the network as well. It is loaded GPOS 20b using 48 gig total memory. Now I am not sure why it is GPOS loaded there and Llama up here. I am guessing this is just a mockup or whatever, but yeah, as I say, this is serving over the network.
Here is downloading the model GPOS 20b and it is seemingly downloading them directly from Hugging Face. And this looks like it is serving the model just on the machine which you can chat directly with in the model sandbox chat. But of course on this channel we love to connect things like Open Code and Claude Code.
Now, one thing I would really like to see which I get in OMLX is actually caching the KV storage, which means that your conversations, your work, it survives restarts of the application and stuff like that. I have not really seen any significant innovations in terms of LM Studio, Olama, all of this stuff. Really, it is the caching that seems to be quite interesting to me. I know LM Studio has LM link, but I could probably serve this over Tailscale and get LM Studio link functionality there.
So, there are no innovations there. So, yeah, very basic app as we kind of would expect from Apple really: a clean way to do it, but a few more features would be pretty nice. So, as I say, short one from me because unfortunately they have taken it down. Let me know what server you are using, whether it is OMLX, LM Studio, or indeed Olama. Like, subscribe if you haven't already, and if you want to hear more about all things Apple and local AI, till next time, keep on vibing.