Zum Hauptinhalt springen
4. August 20267:07

DeepSeek V4 0731 wird in DwarStar (DS4) OFFIZIELL UNTERSTÜTZT

Von Samuel Gregory

Über dieses Video

Ihre lokale KI ist wahrscheinlich besser, als Sie denken – aber nur, wenn Sie aufhören, sie wie einen Chatbot zu behandeln, und anfangen, sie wie einen Ingenieur zu behandeln. In diesem Video gehe ich auf das neueste DS4-Update ein, das offizielle DeepSeek-0731-Unterstützung in das Repo bringt. Wir testen die neue Hybrid-Quantisierung und stellen meine „Big Model Spec“-Theorie auf die Probe. Die wichtigsten Erkenntnisse: - Offizielle DeepSeek-0731-Integration in DS4 mit hybrider Q2/Q4-Quantisierung. - Warum du nach deinem „git pull“ unbedingt „make“ ausführen musst, um Konfigurationsfehler zu vermeiden. - Der Arbeitsablauf bei der Verwendung von Claude Opus für Spezifikationen und DeepSeek für die Implementierung. - Leistungsstatistiken aus der Praxis: 23 Token pro Sekunde auf lokaler Hardware. - Wie man mit einer einzigen Eingabeaufforderung eine funktionsfähige OCR-App erstellt.

Laden Sie einen Quant herunterWählen Sie die Version aus, für die Ihr Rechner über ausreichend Arbeitsspeicher verfügt
./download_model.sh ds4f-q2           # Rechner mit 96/128 GB RAM
./download_model.sh ds4f-q2-q4        # q2 mit den letzten 6 Experten-Schichten auf q4
./download_model.sh ds4f-q4           # Maschinen mit ≥ 256 GB RAM
./download_model.sh ds4f-mxfp4        # native MXFP4-Experten, ca. 156 GB
./download_model.sh pro-q2-imatrix    # Maschinen mit 512 GB RAM, PRO q2 imatrix quant
Starten Sie den ServerEin Modell wurde heruntergeladen – Host und Port sind optional
./ds4-server --metal --host 0.0.0.0 --port 8765
Starten Sie den Server und wählen Sie ein Modell ausEs wurden mehrere Quants heruntergeladen – benennen Sie das gewünschte
./ds4-server --model gguf/DeepSeek-V4-Flash-0731-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf --metal --host 0.0.0.0 --port 8765
OpenCode-Konfiguration~/.config/opencode/opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ds4": {
      "name": "ds4",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:8765/v1",
        "apiKey": "dsv4-local"
      },
      "models": {
        "deepseek-v4-flash": {
          "name": "DeepSeek V4 Flash",
          "limit": {
            "context": 100000,
            "output": 384000
          }
        }
      },
      "variants": {
        "high": {
          "reasoningEffort": "high",
          "textVerbosity": "low",
          "reasoningSummary": "auto"
        },
        "low": {
          "reasoningEffort": "low",
          "textVerbosity": "low",
          "reasoningSummary": "auto"
        }
      }
    }
  }
}

Die lokale KI-Revolution: DeepSeek 0731 auf DS4

Die meisten Entwickler verschwenden nach wie vor Geld für API-Token für Aufgaben, die ihre eigene Hardware präziser und ohne Datenschutzrisiken bewältigen könnte. Mit dem neuesten Update des Dwarf Star (DS4)-Repositorys wurden die DeepSeek-0731-Quantisierung offiziell integriert, und die Ergebnisse sind, offen gesagt, atemberaubend für jeden, der sich ernsthaft mit lokalen LLMs beschäftigt.

Der Vorteil der Hybridtechnologie

Bei der neuen Implementierung handelt es sich nicht einfach nur um eine Standard-Verkleinerung. Wir setzen auf eine ausgeklügelte Hybrid-Quantisierung, bei der der Großteil des Modells mit Q2 läuft, die letzten sechs Schichten jedoch mit Q4 beibehalten werden. So entsteht ein Modell, das in den Bereich von 90 GB bis 100 GB passt und gleichzeitig eine Schlussfolgerungsfähigkeit beibehält, die mit der von wesentlich größeren proprietären Systemen mithalten kann.

Die Strategie „Spezifizieren und Umsetzen“

Einer der effektivsten Arbeitsabläufe, die ich entdeckt habe, ist der „Big Spec“-Ansatz. Ich habe Claude Opus im „High Mode“ verwendet, um eine umfassende technische Spezifikation für eine Dokumentenscan-Anwendung zu entwerfen. Anstatt das lokale Modell zu bitten, gleichzeitig Ideen zu entwickeln und zu programmieren, habe ich diesen hochrangigen Entwurf in das lokale DeepSeek-0731-Modell eingespeist.

Das Ergebnis war eine voll funktionsfähige OCR-Anwendung, die auf der Grundlage einer einzigen Eingabeaufforderung erstellt wurde. Die Verarbeitung des gesamten Befehlssatzes dauerte zwar fast drei Stunden, doch der lokale Rechner erreichte eine durchschnittliche Geschwindigkeit von 23 Tokens pro Sekunde. Das ist mehr als ausreichend für einen Entwicklungsprozess im Hintergrund, der keinerlei Abonnementkosten verursacht.

Wichtige technische Erkenntnisse

Wenn Sie heute Ihre lokale Umgebung aktualisieren, denken Sie daran, dass ein einfacher „git pull“ nicht ausreicht. Sie müssen den Befehl „make“ erneut ausführen, um sicherzustellen, dass alle Dateien korrekt neu gebündelt werden. Sollten Sie außerdem Probleme damit haben, dass das Modell endlos nachdenkt, stellen Sie sicher, dass der „Reasoning Mode“ in Ihrer Konfiguration auf „max“ eingestellt ist.

Lokale KI ist längst kein Hobby für Enthusiasten mehr, sondern ein vollwertiges Werkzeug für die Entwicklung auf Produktionsniveau. Die einzige Frage, die noch offen bleibt, ist, ob Sie mutig genug sind, die Cloud auszuschalten und auf Ihre eigene Hardware zu vertrauen.

Transkript▾

Yesterday, I showed you how you could get Deep Seek version 0731 running inside of DS4 or Dwarf Star. And those quantisations were produced by the community whilst we waited for the Dwarf Star community to introduce this officially to the repo. And I am happy to say we finally have that result. In the latest commit here, we could scroll down and see that the old Q2 matrix file has been replaced here with the 0731. What we get here is the interesting Q2 and the last six layers being quantised four here as well. So we get the full range basically updated to 0731. And here we have the updated scripts. So let's get that all set up and running. So I am going to change directory into my DS4 folder here. And because I have pulled this down, I am just going to go get pull. And I am probably going to get conflicts here because I played with the actual download file, which we do. So, let's go get diff. You will see that I made these changes in order to update the repo and the Q2 I matrix file. So, realistically, we can get stash that. I don't need that anymore. Get pull. And with that downloaded, you are going to want to make sure you run make again. Not do not make the same mistake I did. Think it is all just going to work just because you have get pulled. You want to make again to make sure all of the files are rebundled up. And I am just going to remove RF GGUF and I am going to remove that quantisation which was kindly provided to us by the community. And I am simply going to go download model and I am going to play around with the interesting quantisation because I believe I have the room for it. And we are literally just going to do that and it is going to download the model for us. Now, I have made a video on how to get this all set up, and I will link that above. You are literally just going to be changing the Q2I matrix command that I use there. You are literally just replacing it with that or depending on the version that you want. Bearing in mind, Q2 is available for 96 to 128 gig RAM machines. Similarly, I think we can squeeze onto there. That is what we are going to check today. If you have the room, if you have a bigger machine, then you can go to the Q4. Or you can get the mixture of experts unquantised lossy format here. If you got 156 gig or if you got a big daddy like a Mac studio, you can go for the pro variant, which you can use this script here. Okay, this is coming in at 90.88 gig, whereas the 2bit quant that we downloaded yesterday was coming at 80.7 and the original 2bit quant 0731 was 86.7. So just to give you an idea of the size, it is a little bit bigger, this hybrid quant they have got going on here. Now, what I have gone and done is actually get Claude to write out a plan because what I said in my last video was it is kind of good to have bigger models, more capable models, spec out a project and then get lesser models, local models to actually implement those. It was a theory. I want to put it to the test. And as I say, I asked Opus on high mode to write a very detailed spec in order to build the same app we built yesterday, which is this document scanning application. Now, it asked me a couple of questions. It is built out this plan here. I am not even going to I am not even going to look at this plan. I am just going to get DeepSeek to actually just implement it from the get-go, and we are going to assess at the end. So, let's get this up and running. So, how we do that is we go DS4 server. We are going to do the whole shebang here, and we are going to increase that context to 100,000. This should pick up on the one and only model that's in there. This looks good. It is loading up right now. I am going to whack on my fan, so excuse this for a sec. And I will put in noise cancelling. Now, I have done instructions on how to set up open code on my previous video. However, there are some reports that it thinks forever and that we should put reasoning mode on max. So, I will leave links down below to the configuration that you should need to give you the ability to set reasoning mode as max. All I am going to say is implement the plan and link to that file and let it rest. So, let's check back in however long this takes and have a look at if it even runs first and foremost and exactly how long it took. Right. And we are finally done. And I say that I have already started running. It is running. I haven't tested it yet, but this has been running for 2 hours and 44 minutes minus a few instances where I didn't notice I needed to approve something or or whatever. It took a while and generally it moved quite fast. I mean, it is again we are talking about two hour two and a half hours of logs here, but tokens per second averaging 23 tokens per second. Not too bad. Compacted itself a few times, but as AI always does, it seems quite confident. Let's just give the damn thing a go. So, we got it here. Let's open this up. And it is slightly more improved the user interface. Like, we have got that typical kind of early Opus 4.5 look. Nothing too crazy, but I am not really after any design, to be honest. Now, let's give it our test lease here. So, if we drop that, this should be connecting to this machine which has an OCR model running on it and it should detect what is going on. Now, you can't see this. However, I should be able to see if we're getting some action on this end and it looks like it is generating. It looks like there is something going on. If you remember rightly, this did not work first time with the previous version of of of deep sea. However, that could be a number of reasons. The fact that I asked Opus to give it a very indepth like spec implementation plan. Anything is possible. Okay, I had to refresh the page to get that. So, there is a bug there, but we open it. It is done it. It is a working app. And I should be able to query what is the date on this lease. I think this is using sonnet 4.5 but let's give it a go on open router reading. There we go. So really good result again a one prompt a single prompt result there with it obviously being given direction from Opus a relatively fast result I think just observing it whilst it is doing its thing given the scale of the app. I have no doubt this would have taken half an hour to an hour with a like a Frontier model. So quite good speed there. Overall, I am impressed. So there you go. I will probably record another video kind of from start to finish as I did with the previous version as the scripts have changed. So look out for that one. Again, let me know what you guys are doing with your local AI stuff, what you will be doing with this Deep Seek model. I love hearing what everyone's doing. I am obviously a coder. I use Open Claw and Hermes and stuff like that, but what are you doing? Like, subscribe if you haven't already. Until next time, keep on vibing.