Unsloth Desktop (Image © Unsloth)
Using the NPU for Local AI
A key component of this integration is the Unsloth Desktop API, which provides an OpenAI-compliant endpoint. This enables the use of a wide range of compatible software that leverages NPU acceleration. Currently, over 100 agent applications and workflows are available. Notable compatible tools include:
- Development environments: VS Code and Cline.
- Agent frameworks: Hermes Agent, OpenClaw, and OMP.
- User interfaces: Open WebUI.
Energy Efficiency and Hardware Resource Management
Offloading AI workloads to the XDNA 2 NPU is primarily energy-efficient and conserves resources. For example, text-to-speech models can be offloaded to the MPU to better utilize the GPU for large models.
This redistribution of resources offers two key technical advantages:
- 1. Battery conservation: Lower power consumption during inference extends the operating time of mobile devices.
- 2. Resource availability: CPU and GPU cycles remain available for other concurrently running system applications, provided that...
Guide to Using the NPU with Unsloth Desktop
Running Unsloth Desktop on an AMD XDNA 2 NPU requires a specific setup process in Windows. Users must first install Unsloth Desktop, navigate to the Model Hub, and apply the NPU filter to enable the hardware. Memory allocation is a critical limitation for local inference. The amount of available system RAM determines which models can be successfully loaded. For example, the “Qwen 3.6 35B A3B” model requires approximately 28 GB of memory, which is allocated to the NPU. Since Windows typically uses a 50 percent shared memory allocation by default, a system must have at least 64 GB of total system RAM to run a model of this size.
The deployment process involves downloading the NPU-specific model and selecting it in the NPU section of the chat interface, where users can also define the desired context window.
Performance of Local Inference and MoE Models
The current implementation supports the execution of complex model architectures, including Mixture-of-Experts (MoE). Technical demonstrations have confirmed that the Qwen-3.6 model, consisting of 35 billion parameters, can be executed entirely on the local NPU. This process is made possible by the combined orchestration of Unsloth Desktop and the FLM inference engine, enabling high-parameter local AI without reliance on cloud-based infrastructure.
CPUs und APUs mit XDNA 2 NPUs
Folgende Mobil- und Desktop-Modelle sind mit einer XDNA 2 NPU ausgestattet:
- Ryzen AI 9 HX 375, Ryzen AI 9 HX PRO 375, Ryzen AI 9 HX 370, Ryzen AI 9 HX PRO 370, Ryzen AI 9 365
- Ryzen AI 7 PRO 360, Ryzen AI 7 350, Ryzen AI 7 345
- Ryzen AI 5: Ryzen AI 5 340, Ryzen AI 5 330
- Ryzen AI 9 HX 475, Ryzen AI 7 450 und Ryzen AI 5 430
- Ryzen AI 7 450G/GE, Ryzen AI 5 440G/GE und Ryzen AI 5 435G/GE
- Ryzen AI Max+ 395, Ryzen AI Max 390 und Ryzen AI Max 385
- Ryzen AI Max+ PRO 495, Ryzen AI Max PRO 490 und Ryzen AI Max PRO 485
We will be adding some relevant NBU benchmarks to this article shortly.
