# DeepSeek just opened up the engine behind its AI

> DeepSeek has released DeepInfer, the inference engine behind its production services. The open-source release runs V4.1-Flash offline on supported NVIDIA SM100 or Huawei Ascend 950 hardware. It is not a consumer GPU runtime or a complete hosted API service.

We may earn a commission when you buy through links in this article. [How we make money](/disclosure).

DeepSeek is opening up another part of the AI business: the machinery that keeps a model answering questions without making the GPU bill unbearable.

The company has released [DeepInfer](https://github.com/deepseek-ai/DeepInfer), sharing the inference framework it says powers its large-scale production services. The initial release supports DeepSeek-V4.1-Flash and comes under the Apache 2.0 license. For developers running their own infrastructure, it offers a chance to inspect and adapt the software DeepSeek uses to put its models to work.

There is a substantial catch for anyone reaching for a gaming PC. DeepInfer currently targets NVIDIA SM100 accelerators and Huawei Ascend 950 NPUs. Its public release handles offline inference; important pieces of a complete online service remain outside the repository. The code is available. A miniature DeepSeek data center is not suddenly an afternoon project.

That makes this a different announcement from [V4.1-Flash-1017](https://api-docs.deepseek.com/news/news261017/), the model update DeepSeek released on October 17 with improved project-development capabilities. That model is already available through the company's website, app, and API. This release is about how to serve it yourself.

Looking for a smaller DeepSeek model to run at home? [See three desktop GPU options for smaller DeepSeek-R1 models](#deepseek-r1-gpu-options).

## The expensive part comes after the download

An open-weight release puts the model within reach. Running it efficiently enough to serve paying customers is a separate engineering problem.

When an AI assistant responds, its servers first process the conversation, then generate new tokens. Those stages have different demands. Long prompts require substantial work before the first word appears; generating an answer involves a stream of smaller steps. A service also has to share hardware among requests without leaving users staring at a motionless cursor.

DeepSeek's [deployment notes](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md) describe separating vision processing, prompt processing, and generation, then assigning hardware to each. The production design also retains cached context for more than 72 hours, allowing repeated conversations to reuse earlier work. That cache system is one of the pieces the public release does not fully provide.

This is where the announcement gets interesting. A third-party host can have the same weights as DeepSeek and still deliver a slower, more expensive product. Publishing the engine gives those operators something concrete to study, reproduce, and improve. The potential payoff is more competition among providers that serve open models. Whether it translates into lower customer bills will depend on how well others deploy it.

DeepSeek also released [deepseek-recipe](https://github.com/deepseek-ai/deepseek-recipe), a companion set of Rust libraries and Python bindings. It translates API requests into the prompts and token sequences the model expects, then turns model output back into responses, including reasoning and tool calls. It supports Messages, Chat Completions, and Responses formats. Developers still have to supply inference, tool execution, and the HTTP service, but they get an official reference for the translation layer. Matching a model's name is of limited use if the software around it feeds it the wrong conversation format.

## The hardware caveat is a big one

The word “Blackwell” on a graphics-card box is not enough. [NVIDIA's own compatibility table](https://developer.nvidia.com/cuda/gpus) puts B200 and GB200 accelerators at compute capability 10.0, the SM100 class supported by DeepInfer. A GeForce RTX 5090 is compute capability 12.0. Sharing an architectural family does not make those devices interchangeable software targets.

The model itself is enormous. [DeepSeek's V4.1-Flash announcement](https://api-docs.deepseek.com/news/news260910/) describes a mixture-of-experts model with hundreds of billions of parameters, even though only a fraction are active for each token. “Active” describes the work selected for a token; it does not mean the rest of the model's weights disappear from the deployment. A modest active-parameter count is not a promise that the full model fits on a modest GPU.

DeepInfer's performance plots use [four SM100 GPUs for prompt processing](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/images/prefill_pareto.png) and [64 for generation](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/images/decode_pareto.png). Those are benchmark configurations, not a universal minimum hardware requirement. They do, however, tell you the scale of the problem DeepSeek is tackling. We have not reproduced those results, and they should not be read as a speed forecast for a home workstation.

For a business evaluating the release, the unfinished service layer matters just as much as the chips. DeepSeek's [deployment Q&A](https://github.com/deepseek-ai/DeepInfer/blob/main/doc/inference_deployment_qa.md) explains that cache management and load balancing are tied to its internal infrastructure. A team adopting the engine still has integration work ahead of it.

## How to run DeepInfer

For people with access to supported accelerators, DeepSeek's [build instructions](https://github.com/deepseek-ai/DeepInfer#prerequisites) start with Linux, GCC 13 or newer, and Python 3.12. NVIDIA systems need CUDA 13; Ascend systems need CANN 9.2. Use a fresh Python environment and check the repository's current requirements before provisioning a machine.

On a correctly configured NVIDIA system, this prepares the source and runs the project's documented offline example:

```bash
git clone https://github.com/deepseek-ai/DeepInfer.git
cd DeepInfer
python3.12 -m venv .venv
source .venv/bin/activate

./build.sh
python examples/offline_inference.py --ep 8
```

The final command uses eight-way expert parallelism: model experts are distributed across the participating accelerators. It is the [repository's example configuration](https://github.com/deepseek-ai/DeepInfer/blob/main/examples/README.md), not an instruction for a single GeForce card. The example downloads `deepseek-ai/DeepSeek-V4.1-Flash-1017` into the Hugging Face cache on first use, so plan for a large checkpoint download and sufficient storage. If the weights are already on the machine, set `DEEPINFER_HF_MODEL_DIR` to their directory before running it. Keep `DEEPINFER_ACCELERATOR` exported in the running environment; Ascend users select `ascend` instead of `nvidia` when building and running.

These commands follow the upstream documentation; we have not executed them on an SM100 or Ascend system. Treat the result as an offline inference example. It does not start a public chat API.

If your goal is simply to try the current model, [DeepSeek's chat interface](https://chat.deepseek.com/) is the easier route. Developers with an API key can use the hosted [Chat Completions endpoint](https://api-docs.deepseek.com/api/create-chat-completion/):

```bash
# Set DEEPSEEK_API_KEY in your shell first.
curl https://api.deepseek.com/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${DEEPSEEK_API_KEY}" \
  -d '{
    "model": "deepseek-flash",
    "messages": [{
      "role": "user",
      "content": "Explain speculative decoding in plain English."
    }],
    "thinking": {"type": "disabled"},
    "max_tokens": 512,
    "stream": false
  }'
```

DeepSeek's [current API guide](https://api-docs.deepseek.com/) maps `deepseek-flash` to V4.1-Flash-1017. This sends your prompt to DeepSeek's servers and uses a billed API account. You are using the model through the hosted service, rather than installing DeepInfer.

## What makes sense on a desk

For readers who want to experiment locally, start with the computer you already have. [Ollama's DeepSeek-R1 8B package](https://ollama.com/library/deepseek-r1:8b) provides a much smaller option: DeepSeek-R1-0528-Qwen3-8B, a distilled reasoning model. After [installing Ollama](https://ollama.com/download), run:

```bash
ollama run deepseek-r1:8b
```

That is an older, different model, running through a different engine. Its listed 5.2GB download is far more approachable, though runtime memory also has to accommodate the conversation and software overhead. It is a sensible way to learn about local inference while the new server software remains out of reach for most home setups.

### Hardware for DeepSeek

For distilled R1 models in Ollama. DeepInfer and the full V4.1-Flash model need data-center hardware.

#### ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB

Start with R1 8B · 12GB GDDR6 · ZT-A30600H-10M

Room for the 8B model, plus context and runtime memory.

Model: [DeepSeek-R1 8B in Ollama](https://ollama.com/library/deepseek-r1:8b) · 5.2GB model download. [View ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB at Amazon](https://www.amazon.com/dp/B08W8DGK3X?tag=llamabuilds-20).

#### ASUS Dual GeForce RTX 5060 Ti 16GB GDDR7 OC Edition

For R1 14B · 16GB GDDR7

More memory for R1 14B. Choose the 16GB version of this card.

Model: [DeepSeek-R1 14B in Ollama](https://ollama.com/library/deepseek-r1:14b) · 9GB model download. [View ASUS Dual GeForce RTX 5060 Ti 16GB GDDR7 OC Edition at Amazon](https://www.amazon.com/dp/B0F7WB6LSH?tag=llamabuilds-20).

#### EVGA GeForce RTX 3090 FTW3 Ultra Gaming 24GB

For the larger R1 32B · 24GB GDDR6X · 24G-P5-3987-KR

Fits R1 32B at 4K context. Check used condition, case space and power.

Model: [DeepSeek-R1 32B in Ollama](https://ollama.com/library/deepseek-r1:32b) · 20GB model download. [View EVGA GeForce RTX 3090 FTW3 Ultra Gaming 24GB at Amazon](https://www.amazon.com/dp/B08J5F3G18?tag=llamabuilds-20).

Memory-fit estimates use Q4_K_M at 4K context, not hands-on benchmarks. Download sizes are not total runtime memory; longer context needs more VRAM. Check case clearance, power and runtime support.

[Ollama GPU support](https://docs.ollama.com/gpu).

We may earn a commission when you buy through these links. Check the retailer for current price, availability and the exact variant. [How we make money](/disclosure).

For the 32B option, run `ollama run deepseek-r1:32b`, then enter `/set parameter num_ctx 4096` to use the 4K context assumed here. Ollama can otherwise choose a larger default on a 24GB card. While the model is running, check `ollama ps` in another terminal: `100% GPU` means it has not offloaded to the CPU. See [Ollama’s context settings](https://docs.ollama.com/context-length).

Storage is the less glamorous upgrade. If model downloads keep crowding out everything else, the [Samsung 990 PRO 2TB NVMe SSD](https://www.amazon.com/dp/B0BHJJ9Y77?tag=llamabuilds-20) gives a local model library room to grow. This link is for the bare drive, model MZ-V9P2T0B/AM; it needs a compatible M.2 2280 NVMe slot and suitable cooling. Buy it to solve a storage shortage. An SSD cannot substitute for enough GPU memory to hold the model you want to run.

DeepInfer's immediate audience is the people operating AI servers. Its wider significance is that more of the work behind a usable open model is now available for others to inspect. Downloading the weights was one step. Making them economical to serve is the next contest, and DeepSeek has given its competitors more code to work with.


---
Source: https://www.llamabuilds.ai/guides/deepseek-deepinfer-open-source-release · Published October 20, 2026 · Llama Builds.