Dynamic LoRA on AMD Ryzen AI

Fine-tuning with LoRA has become the default way for local AI developers to turn a general LLM into a specialist. A few million adapter parameters on top of a frozen base can teach a model to write SQL, answer domain questions, or follow a house style, without the cost of full fine-tuning. But on an AI PC, the interesting question is not only how you train the adapter. It is how you deploy several of them efficiently on the NPU.

With Ryzen™ AI Software, you can now compile a base model once and switch LoRA adapters dynamically at inference time. One quantized base model stays resident on the NPU, and small adapters are activated by name. In the current release, dynamic LoRA is supported for selected models: Qwen2.5-3B, Qwen3-4B, Qwen3.5-4B (text). This post explains why that matters and how the end-to-end flow works.

Why Dynamic Beats Static Merge

Until now, the supported way to run a fine-tuned model on Ryzen AI was static merge. You fold the LoRA weights into the base, then compile the merged model into a single NPU binary. That works for one adapter. It scales badly for many.

Each merged model is a full copy of the base. If you have five specialist adapters for the same base model, you ship and load five complete binaries, each one re-duplicating the large base weights, while the part that actually differs, the adapter, is tiny. On a memory-constrained AI PC that is wasteful. It also means switching "personas" inside an application means loading an entirely different model.

Dynamic LoRA removes that duplication. The base is compiled once. Adapters are compiled into small, loadable artifacts. At runtime a single base model serves all of them, and your application selects the active adapter per request. Moving from the SQL specialist to the medical specialist is a cheap adapter switch, not a model reload.

The One Thing You Must Get Right: Train Against the Quantized Base

There is a subtle trap worth calling out. On the NPU, the base model is quantized. In our example it is INT4. If you train a LoRA on the full-precision model and then apply it to the quantized base, the correction the adapter learned no longer lines up with the weights it is added to, and much of its benefit disappears on-device.

The fix is simple. Train the adapter against the already-quantized base. The base stays frozen and quantized during fine-tuning, and only the adapter is trained. The adapter you get is valid for the exact model you deploy.

The End-to-End Flow

The flow has four stages, and only the last one runs on the Ryzen™ AI PC.

• Quantize the base model once with AMD Quark (INT4 AWQ). This base is shared by every adapter.
• Fine-tune each LoRA adapter against that frozen quantized base, and save it unmerged.
• Compile the base and each adapter for the NPU with a single model_generate call per adapter.
• Serve with dynamic switching. Load the base once, then activate any adapter by name.

At runtime, switching is a single call.

The base model never reloads. The same loaded model answers as a different specialist depending on which adapter is active.

Seeing It Work

Our example trains two adapters on the same Qwen3-4B INT4 base: a text-to-SQL adapter (Spider) and a medical multiple-choice adapter (MedMCQA). Running on the Ryzen AI NPU, each adapter measurably lifts task accuracy over the base - both evaluated at n=200 examples.

Task

Metric

Base

LoRA

Delta

Spider text-to-SQL

execution accuracy

47.5%

63.0%

+15.5

MedMCQA medical

exact match

42.0%

53.0%

+11.0

Both outputs below came from the same loaded model in one session, one set_active_adapter call apart:

Try it Yourself

The full, runnable example is published in the Ryzen AI Software examples repository . It walks through every stage above: quantization, data preparation, LoRA fine-tuning against the quantized base, NPU compilation, and the dynamic switching script that produces the outputs shown here. Clone it, train your own adapters on your own data with one of the supported models, and switch them live on your Ryzen AI PC.

Dynamic LoRA turns one resident base model into as many specialists as your application needs, without paying the full model cost for each one. It is a small change to how you deploy fine-tuned models, and a big change to how many of them you can afford to ship on an AI PC.

Dynamic LoRA is supported in Ryzen AI software release 1.9, which also introduces long-context support, importing INT4-quantized models produced through Quantization-Aware Training (QAT), and support for additional models. See the Ryzen AI Release Notes for details.