Using AMD GPUs with minikube - Minikube

This tutorial shows how to start minikube with support for AMD GPUs.

Support is provided by the AMD GPU device plugin for Kubernetes.

Prerequisites

Linux
Latest AMD GPU Drivers 6.2.1 or greater
minikube v1.35.0 or later (docker driver only)

Using the docker driver

Ensure you have an AMD driver installed, you can check if one is installed by running rocminfo, if one is not installed follow the Radeon™ Driver Installation Guide
Delete existing minikube (optional)

If you have an existing minikube instance, you may need to delete it if it was built before installing the AMD drivers.
shell
```
minikube delete
```

Start minikube:

shell

minikube start --driver docker --container-runtime docker --gpus amd

Verifying the GPU is available

Test the AMD GPUs are available to the cluster.

Create the following Job:

shell

cat <<'EOF' | kubectl apply -f -
apiVersion: batch/v1
kind: Job
metadata:
  name: amd-gpu-check
  labels:
    purpose: amd-gpu-check
spec:
  ttlSecondsAfterFinished: 100
  template:
    spec:
      restartPolicy: Never
      securityContext:
        supplementalGroups: 
        - 44
        - 110
      containers:
        - name: amd-gpu-checker
          image: rocm/rocm-terminal
          workingDir: /root
          command: ["rocminfo"]
          args: []
          resources:
            limits:
              amd.com/gpu: 1 # requesting a GPU
EOF

Check the Job output kubectl logs jobs/amd-gpu-check looks something like the following:

plain

ROCk module version 6.8.5 is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.14
Runtime Ext Version:     1.6
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
DMAbuf Support:          YES

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD Ryzen 7 7840U w/ Radeon  780M Graphics
  Uuid:                    CPU-XX                             
...

Where can I learn more about GPU passthrough?

See the excellent documentation at https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

Why does minikube not support AMD GPUs on Windows?

minikube supports Windows host through Hyper-V or VirtualBox.

VirtualBox doesn't support PCI passthrough for Windows host.
Hyper-V supports DDA (discrete device assignment) but only for Windows Server 2016

Since the only possibility of supporting GPUs on minikube on Windows is on a server OS where users don't usually run minikube, we haven't invested time in trying to support GPUs on minikube on Windows.