About this video
Cloud-based AI is officially a legacy solution for developers who value their privacy and their wallets. In this video, I dive deep into DS4 (Dwarf), the project that allows you to run the massive DeepSeek V4 model on your local hardware. We cover everything from the initial terminal setup to building a fully functional OCR application using local reasoning. Key Takeaways: - DeepSeek V4 Flash is now the fastest-growing model on Alama. - DS4 enables running 1.6T parameter models on 128GB or 512GB Macs. - Step-by-step guide on cloning, building, and serving the model locally. - How to integrate local models into dev tools like Open Code and VS Code. - Real-world testing: Building a document scanner in under 3 hours at 23 tokens per second.
The Cloud is a Golden Cage: Why You Must Run DeepSeek Locally
Your hardware is sitting idle while you funnel money into API subscriptions for models that you do not even own. The release of DeepSeek V4 has changed the landscape of open-source AI, but the real magic happens when you bring that power home using the DS4 (Dwarf) project.
What is DS4?
DS4 is a highly optimised implementation built on Llama.cpp that allows you to run DeepSeek V4 Flash and Pro versions on consumer hardware. Whether you are rocking a 128GB MacBook Pro or a high-end Nvidia rig, this codebase bridges the gap between massive parameter counts and local VRAM limitations.
The Setup Process
Getting started is surprisingly straightforward. You simply clone the repository, run the 'make' command suited to your hardware (Metal for Mac, CUDA for Nvidia), and download the specific quantised model that fits your memory profile. For those with 128GB of RAM, the 2-bit or 4-bit quantised versions of the 1.6 trillion parameter model are finally within reach.
Real-World Performance
In my testing, integrating DS4 with Open Code allowed the model to act as a fully autonomous agent. While it took nearly three hours to implement a complex document scanning application, it did so with an average speed of 23 tokens per second. That is a staggering level of productivity for a system running entirely on a local machine without any external data sent to a third party.
Final Thoughts
The days of being tethered to a server are numbered. By optimising how we handle mixture-of-experts algorithms and quantisation, projects like DS4 are democratising the most powerful AI models on the planet. Stop waiting for a better API and start building with the hardware you already own.
Transcript▾
DeepSeek V4 Flash is currently the fastest growing model on the Alama platform. DeepSeek has broken new ground in terms of the resources required to train a highly capable model. The Flash version features 304 billion parameters, while the Pro version boasts 1.6 trillion parameters with 13 billion activated parameters on its mixture of experts algorithm.
In this video, we explore a project called DS4, or Dwarf, which enables a quantised version of DeepSeek to run on local hardware. I have compared this to MLX on Mac, and DS4 performs significantly better due to clever algorithmic changes at the weight level. It runs on Metal for Mac, Nvidia hardware, and even ROCM for Framework desktops.
To set this up, you need to clone the DS4 GitHub repository and run the 'make' command specific to your system. This bundles the files together using Llama.cpp with specific optimisations. Depending on your RAM, you can choose different quantisation levels, ranging from 2-bit versions for 128GB systems to larger versions for Mac Studios.
Once built, you can run the model via a CLI or use the DS4 server to access it in tools like Open Code or Claude Code. By configuring your local host and port, you can use DeepSeek as a local provider for your development workflow.
In my practical test, I used DeepSeek to implement a document scanning application. Despite the screen recording overhead, it averaged 23 tokens per second. The process took about 2 hours and 44 minutes, but the result was a fully functional application with an improved user interface. This proves that large parameter models can successfully run on accessible consumer hardware.