Run Ollama with Your Old AMD GPU

Unlock the power of your GPU and CPU to explore large language models locally with ease.

AMD GPUs and Their GFX Versions

Here's a list of AMD GPUs with their GFX versions that are earlier than the AMD Radeon RX 6800 XT (based on the RDNA 2 architecture, GFX1030):

Vega Series (Vega 10, Vega 20) - GFX900

Example: Radeon RX Vega 64, Vega 56

Navi 10 (RDNA 1.0) - GFX1010

Example: Radeon RX 5700 XT, RX 5700, RX 5600 XT, RX 5500 XT

Navi 12 (RDNA 1.0) - GFX1011

Example: Radeon RX 5300 XT

Navi 14 (RDNA 1.0) - GFX1012

Example: Radeon RX 5500 XT (mobile)

Renoir APU (Vega-based iGPU) - GFX902

Example: Ryzen 5 4500U (with Vega 6 integrated graphics)

Polaris Series (GCN 4.0) - GFX803

Example: Radeon RX 480, RX 580, RX 570

Fiji (GCN 3.0) - GFX804

Example: Radeon R9 Fury, Fury X

Hawaii (GCN 2.0) - GFX803

Example: Radeon R9 290, 290X

The RX 6800 XT (GFX1030) represents a significant shift to RDNA 2, so these earlier GPUs are from older architectures, starting from GCN (Graphics Core Next) to RDNA 1.0.

Step-by-Step Guide

1. Install Ollama

Download Ollama for Windows from their official site and install it.

2. Download ROCm Libraries

Visit the ROCblas GitHub repository and find libraries for your GPU. For older AMD GPUs, look for the appropriate architecture code. Extract and copy them to the Ollama ROCm folder.

3. Setup ROCm for Ollama

Follow these steps to ensure compatibility:

Unpack the library version with 7-zip or WinRar.
Copy the ROCm folder from the unpacked library to:

C:\Users\%username%\AppData\Local\Programs\Ollama\lib\ollama\rocblas\

4. Set Environment Variables

If you face errors, set these environment variables:

$Env:HCC_AMDGPU_TARGET = "your_gpu_architecture_code"
$Env:HSA_OVERRIDE_GFX_VERSION = "your_gfx_version"

Performance Improvement

Comparison of eval rates on a Ryzen 5 7500F CPU and an AMD RX 5500 XT GPU.