Skip to main content
August 3, 202610:23

DeepSeek v4 0731 is FINALLY Available for DwarfStar (DS4)!!!

By Samuel Gregory

About this video

Stop sending your proprietary code to the cloud and start running massive 304B parameter models locally on your Mac. This video explores the latest DeepSeek 0731 update and how to use Dwarf Star to quantise it for your own hardware. Key Takeaways: - How to update the DS4 repo to support the latest DeepSeek 0731 model. - The shift from 158B to 304B parameters and what it means for agentic coding. - Realistic performance benchmarks on M5 Max and Mac Studio hardware. - Why increasing your context window to 100k is essential for complex projects. - A comparison of local DeepSeek performance versus cloud based models like Claude.

Model filenamesdownload_model.sh
# download_model.sh
Q2_IMATRIX_FILE="${Q2_IMATRIX_FILE:-DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf}"
Q4_IMATRIX_FILE="DeepSeek-V4-Flash-Q4KExperts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-imatrix.gguf"

REPO="Rednalreden/DeepSeek-V4-Flash-0731-dwarfstar-q2-gguf" Q2_IMATRIX_FILE="DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf" ./download_model.sh q2-imatrix
Start the serverOne model downloaded — host and port are optional
./ds4-server --metal --host 0.0.0.0 --port 8765
Start the server, choosing a modelSeveral models downloaded — name the one you want
./ds4-server --model gguf/DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf --metal --host 0.0.0.0 --port 8765

The era of renting intelligence from Silicon Valley giants is coming to an end.

For too long, founders and CEOs have been forced to trade their most valuable asset, their intellectual property, for the convenience of cloud based AI agents. Every time you prompt a frontier model with your proprietary codebase, you are essentially feeding your company's secret sauce into a black box. But the release of DeepSeek version 0731, combined with the quantization magic of Dwarf Star, has changed the calculus for the modern executive.

Local Intelligence is the New Moat

We are now seeing 304 billion parameter models running locally on consumer hardware. With a high spec MacBook Pro or a Mac Studio, you can host a coding agent that rivals the best in the world without a single packet of data leaving your secure network. DeepSeek 0731 has doubled its parameter count and significantly enhanced its agentic capabilities, making it a viable alternative for high level implementation tasks.

The Performance Reality

In our latest tests, the 0731 version fixed complex document scanning issues in a single shot, a feat the previous version failed to achieve after hours of processing. Whilst it may not yet match the raw speed of Claude 3.5 Sonnet, the trade off for total data sovereignty is one that many founders are now willing to make. We observed speeds of 20 to 30 tokens per second, which, while slowing down as the context window fills, remains highly usable for background development tasks.

Why it Matters for Founders

  1. Data Sovereignty: Your code remains on your hardware.
  2. Cost Predictability: No more escalating API bills as you scale your agentic workflows.
  3. Customisation: The ability to quantise and tweak models specifically for your technical stack.

The move toward personal software is not just a trend; it is a strategic necessity for those who value privacy as much as productivity. It is time to stop asking permission from the cloud and start owning your intelligence.

Transcript▾

DeepSeek recently released version 0731 of their immensely popular DeepSeek Flash and Flash Pro models. Last week I explored Dwarf Star, a project that quantises these massive models to run on Apple Silicon. The new version increases parameters to 304 billion, nearly doubling the previous size.

Whilst Dwarf Star doesn't natively support 0731 yet, you can manually override the repo variables to download community quantisations. By pointing the download script to community versions on Hugging Face, such as those provided by Redden, you can pull the Q2 version, which is around 86GB.

In testing, the 0731 model showed significant improvements in agentic capabilities. It successfully fixed a document scanning application in 57 minutes, a task the previous version couldn't complete. While Claude 3.5 Sonnet remains faster at the same task, the local DeepSeek model provides total privacy. Token speeds averaged 20 to 30 tokens per second but slowed as the 100k context window filled up. This model is best suited for targeted bug fixes rather than massive scope changes, but it marks a significant step forward for personal software development.

Back to All Videos