Skip to main content

Vendor Support Matrix

On this page
Vendor/FrameworkDetectionFull-GPU DeploymentdGPUTopology/Notes
NVIDIA / CUDAnvidia-smi, sysfsDocker, runveil, nerdctlHAMi libvgpu.so (x86_64, glibc ≥ 2.17)Supports CUDA/NCCL topology; driver libraries are supplied by the host
Ascend / CANNnpu-smi, driver and CANN factsDocker + Ascend runtime; runveil in some environmentsvcann-rt; capability depends on host device-share permissionsHost and container CANN versions must not be conflated into one conclusion
AMD / ROCmamdgpu sysfs, KFDLinux amd64, whole-card deployment on Docker/nerdctl/runveil/Kubernetes; the upstream vLLM ROCm image has been tested for single-GPU BF16 text inference on MI325X (gfx942) and AMD395 (gfx1151), other combinations require per-profile validationamd-dgpu memory quota (rocm-slicing-shim, user space; tested on MI300X/gfx942 under Docker and runveil and on MI325X under Kubernetes; Docker/nerdctl/runveil/Kubernetes)Multi-node and PP run on the vLLM mp backend (RCCL reads the NCCL_* network variables; PP=2 and cross-node TP=2 tested on two MI325X hosts); vLLM sleep/wake (L1/L2) and PD (vllm-nixl, Ethernet UCX TCP) tested on MI300X; not a compatibility claim for all AMD GPUs or ROCm combinations
PPU / HGGCppu-smiDepends on provider/runtimeppu-slicing-shim memory quota and weighted QoS (Docker, Kubernetes)Sustained 20/80 workloads have demonstrated throughput weighting under load, not hard SM partitioning
Hygon / DTKhy-smiDetermined by the catalog spec and runtimeNative Z100 / K100_AI / BW vdev (experimental, x86_64, Docker or runveil/udocker)Requires the optional native helper, host libraries and a per-GPU behavioral probe; other models remain full-GPU only
Apple / Metalsysctl, ioreg (macOS 15+, Apple silicon)host (the engine is a host process; the image is a darwin environment package)None; a Mac runs multiple instances concurrently using unified memoryOne unified-memory card; GPU memory is the Metal working set; engine sleep, multi-node operation, and the KV disk tier are unsupported; see ADR 0044
Kunlunxin / XRExpu-smi XML, PCI/sysfs device identityLinux amd64, Docker/nerdctl/Kubernetes, P800 PCIe / KL3, whole-card TP / PP / DPUnsupportedTP, PP and engine DP; cross-node communication prerequisites apply; no PD, sleep or runveil; explicit image/model catalog specs

The dGPU capability status in the table above is derived from the server's partition capability assessment:

  • ready: dGPU is available.
  • unavailable: dGPU is unavailable, with a reason explaining why, such as “Missing dGPU assets” or “dGPU probe failed”.

When dGPU is unavailable, the node can still use full-GPU deployment. The platform does not silently start instances that fail dGPU requirements with unlimited GPU memory instead. For tested Ascend combinations, CANN, and runtime details, see the Ascend Matrix.

Kubernetes nodes (see Connect a Kubernetes Cluster) support whole-card deployment for NVIDIA, Ascend, Hygon, PPU, AMD, and Kunlunxin; dGPU is available there for NVIDIA HAMi-core, Ascend vcann-rt, PPU ppu-slicing-shim, and AMD rocm-slicing-shim, while Hygon's native vdev remains unavailable on Kubernetes (whole-card deployment only).

dGPU Capability Comparison

BackendGPU memory limitCompute semanticsMemory quota changesBehavioral probe
NVIDIA HAMi-coreHard memory limitThe platform does not expose a compute cap; CUDA graphs and cuBLAS bypass HAMi launch throttlingSupports safe live adjustment; the engine may still need a restart to use the new quotaUses embedded asset validation and admission checks; exempt from behavioral probes
Ascend vcann-rtMemory quota, subject to host device-share permissionsDepends on the vcann-rt and host combination; does not promise a fixed throughput ratioRequires an instance restartMust pass per GPU
PPU ppu-slicing-shimMemory quotaWeighted QoS under contention, borrowing spare capacity when idle; not hard SM partitioningRequires an instance restartMust pass per GPU; Docker and Kubernetes
Hygon Z100 / K100_AI / BW native vdev (experimental)Native vdev memory quota; one GPU per instanceCUs are allocated in proportion to memory; no independently configurable compute quotaRequires an instance restart, recreating the vdev with the new memory quotaMust pass per GPU; Docker or runveil/udocker on x86_64
AMD rocm-slicing-shim (user space)Hard memory limit interposed at the HIP runtime layerThe platform does not expose a compute cap; ROCm enforces compute quotas in hardware through HSA_CU_MASK, below what a preloaded library can see or verifyRequires an instance restart (the limit is read once, at HIP runtime init)Exempt from behavioral probes (neither the host nor the lightweight probe image is guaranteed a ROCm user-space runtime to probe with); relies on embedded asset validation; Docker/nerdctl/runveil/Kubernetes

For safe live adjustment conditions and reasons a restart is pending, see Adjust dGPU Quotas. These capabilities do not imply that every inference image has been verified; availability still depends on the GPU's status / reason.

AMD full-GPU implementation boundary and validation status

The current AMD host integration in the agent and server is scoped to Linux amd64, Docker, nerdctl, Kubernetes or runveil (the udocker driver), and full-GPU deployment (single host or vLLM multi-node); Kubernetes supports both whole cards and dGPU memory partitioning. The PCI roster recognizes AMD functions only when bound to amdgpu. The agent then reads their amdgpu sysfs/KFD facts, checks inventory by PCI BDF and available unique IDs, and injects only the selected card's render node plus /dev/kfd. dGPU memory partitioning (amd-dgpu, libvrocm, memory only — no compute limit) layers an LD_PRELOAD and quota environment on top of this device injection; see "dGPU Capability Comparison" above. dGPU runs under Docker, nerdctl, Kubernetes and runveil (PRoot binds libvrocm.so and loads it through LD_PRELOAD alike). runveil binds the host's whole /dev and /sys, so HIP enumerates every card; the agent pins the selection with ROCR_VISIBLE_DEVICES=GPU-<unique_id> (ROCm's GPU UUID), and refuses a card without a unique_id under runveil. PRoot runs as the non-root runtime user with the device nodes' owning groups (such as render) added automatically; validated on MI300X. Data parallelism (DP) takes the same path as on NVIDIA. Sleep and PD reuse the vLLM protocol and the vllm-nixl mode; with RDMA the platform names the HCA but not the CUDA-specific UCX_TLS, letting UCX choose its ROCm transports, which is not validated on hardware. Multi-node and PP use the vLLM mp backend: like NVIDIA, RCCL reads NCCL_SOCKET_IFNAME and NCCL_IB_*, the agent fills in the interface from the node address, and IB is disabled without RDMA. This describes GPUWorks' implementation scope; it is not a compatibility claim for every AMD GPU, driver, kernel, or ROCm userspace combination.

The built-in image catalog lists gpuworks/gpuworks-runner:rocm7.2.3-vllm0.30.0-r1 (from the upstream vllm/vllm-openai-rocm, ROCm 7.2.3, vLLM 0.30.0). The built-in inference catalog has BF16 vLLM specs for Qwen3 0.6B–32B and 30B-A3B with arch_id: gfx942 (MI300X/MI325X), and for 0.6B–8B with arch_id: gfx1151 (Ryzen AI Max+ 395); pick a model and deploy, with no image or template import. Tested: MI325X single GPU, PP=2 and cross-node TP=2 on two MI325X hosts, and AMD395 single GPU; this shows short text inference and the deploy/stop flow only, not answer quality, sustained performance, long contexts, or image input. The SGLang ROCm entry remains an unpublished catalog: false candidate. UMA capacity uses the observed KFD/GTT facts and should not be generalized to other APUs or hosts. gpus.arch_id is an exact, case-insensitive match; missing or mismatched architecture facts do not pass. See the AMD Hardware Validation Guide and the ROCm image recipe.

AMD dGPU memory quota

The amd-dgpu backend uses libvrocm.so (rocm-slicing-shim, build/runtime-assets/amd/), which interposes the HIP runtime's allocating and capacity-query entry points. It links only libc, so one build serves every ROCm version. Injection layers on top of AMD's full-GPU device selection (render node + /dev/kfd): LD_PRELOAD points at the mounted /opt/gpuworks/partition/libvrocm.so, VROCM_DEVICE_MEMORY_LIMIT_<N> (MiB) is written for each selected card at its position N in the container's own visible-device list (not its physical ordinal), and VROCM_LEDGER_PATH names a per-instance ledger under the host's /dev/shm (/dev/shm/gpuworks-amd-<instance uuid>), unchanged across Docker, nerdctl and runveil. The container reaches that path under Docker, nerdctl and runveil through --ipc=host, sharing the host's /dev/shm directly. On Kubernetes alone the agent's own /dev/shm is its own pod's private tmpfs rather than the host's: the agent pre-creates an empty placeholder at that same path through the host root (/proc/1/root), mode 0o666 (not 0o777: the container may run as a UID with no entry on the host and must be able to write the file, but needs no execute bit), and hostPath-mounts that exact host path, read-write, into the instance pod at the same path. VROCM_LEDGER_PATH's value is unchanged by this; libvrocm.so opens the (already-existing) file as usual and unlinks it on a clean exit, and the agent's own cleanup removes it again through the host root as a backstop. The limit is read once, at HIP runtime init, so lowering it takes effect only after an instance restart.

There is no compute (SM/CU) quota: ROCm enforces compute quotas in hardware through HSA_CU_MASK, below anything a preloaded library can see or verify, so this backend is memory-only and never exposes a compute limit.

Unlike PPU/Ascend/Hygon, AMD's full-GPU device injection mounts no host ROCm user-space library at all (only /dev/kfd and the selected render node), and the lightweight probe image has no HIP runtime to call into either. This backend is therefore probe-exempt (SkipsBehaviorProbe): readiness rests on the embedded asset's checksum and the Docker/nerdctl/udocker/Kubernetes runtime constraint alone — the same approach as NVIDIA's HAMi-core, unlike PPU/Ascend/Hygon, which require a passing per-GPU probe.

PPU device identity and dGPU probes

PPU queries ppu-smi and verifies identities against device nodes, then injects two ordered lists: numeric device selectors in CUDA_VISIBLE_DEVICES for compute, and full UUIDs in GPUWORKS_PPU_DEVICE_UUIDS for the probe and HGML quota library. Docker/nerdctl selectors use the filtered inventory; udocker/runveil selectors use host indexes. Both lists preserve selection order. vLLM and SGLang share numeric visibility without engine-specific dispatch. HGML reuses UUID-order quota and ledger lookup, falling back to the older agent visibility variable only when the separate UUID variable is absent.

The embedded gpuworks/partition-probe:v4 verifies both the selected UUID and its memory quota. Missing, invalid, duplicate or unqueryable identities fail, as does a quota applied only to another card. Recreate existing workloads after upgrading the agent to activate the new selection and slicing libraries. Device visibility under runveil still does not provide kernel device isolation.

HGGC_LEDGER_PATH names a per-instance ledger under the host's /dev/shm (/dev/shm/gpuworks-ppu-<instance uuid>), reaching the instance pod the same way AMD's does (see "AMD dGPU memory quota" above): unchanged under Docker (--ipc=host), and pre-created through the host root and hostPath-mounted at the same path on Kubernetes. PPU dGPU partitioning currently supports Docker and Kubernetes; udocker remains unverified.

Hygon full-GPU driver libraries

Hygon full-GPU workloads receive the host /opt/hyhal through a structured mount shared by Docker, nerdctl and udocker/runveil. For example, DTK 26.04 PyTorch needs its librocm_smi64.so.2. Device injection fails early if the directory is absent. Containers keep the image's DTK and library search path; recreate workloads after upgrading the agent to apply the mount fix. The mount is declared read-only; PRoot binds use the host runtime account's permissions and do not provide kernel-enforced read-only isolation.

The selection variables HIP_VISIBLE_DEVICES and HYGON_VISIBLE_DEVICES use the numbering HIP sees inside the container. Docker and nerdctl pass only the selected cards' DRM nodes, and HIP numbers them 0..N-1 in the container, so these runtimes get container ordinals: host card 2 is 0 in a one-card container. udocker/runveil binds the host's whole /dev and /sys, HIP sees every card, and the host indexes are kept. Recreate workloads after upgrading the agent for the new numbering to apply.

On hosts whose cards cannot use P2P, Hygon vLLM's custom all-reduce may fail to start at TP > 1 (HIP error: invalid device pointer); add --disable-custom-all-reduce to the extra arguments.

Hygon Z100 / K100_AI / BW dGPU

Z100, K100_AI devices identified precisely as K100AI, and cards reporting the exact model BW use the hygon-dgpu backend; K100_AI and K100 AI are aliases of the same SKU. BW is not rewritten as BW1000, and a report containing only the K100 family does not enable dGPU. The agent reads the selected physical card's CU count through hy-smi virtual -d N --show-device-info, then assigns CUs in proportion to the memory quota, rounding up. Failed or incomplete queries reject vdev creation without falling back to a fixed CU count. Build and install the optional gpuworks-hygon-vdev-helper on the node (see build/runtime-assets/hygon-vdev-helper/README.md in the source tree), and provide the host DTK, HYHAL and DCGM libraries. DTK 26 environments may keep the RSMI library in HYHAL's lib directory. The agent itself remains a static binary. Docker and runveil/udocker on x86_64 are supported; nerdctl and other models do not receive this capability automatically.

Each dGPU instance can use only one supported Hygon GPU. The requested memory quota, rounded up to MiB, must be positive and strictly below the card's physical memory. The CU count is ceil(quota MiB × physical CU count / physical memory MiB); there is no separate compute quota or promise of proportional inference throughput. Quota changes require an instance restart, releasing the old vdev before creating one with the new quota.

The current compatibility profile lists DTK 26.04.x × x86_64, but the profile is not marked as fully verified on real hardware. Version information is advisory and cannot replace the per-GPU probe.

Container configuration uses /etc/vdev/docker/vdev<ID>.conf matching the native ID, exposing exactly one Docker configuration. Hardware checks on Z100 and K100_AI show that renaming a nonzero native ID to vdev0.conf prevents HIP initialization. The native filename has not been revalidated on older HYHAL 1.4 environments; they must still pass the per-GPU probe. The small Docker probe receives a private /var/log/hylog tmpfs and supplementary tinfo5/6 libraries under the SONAMEs actually installed on the host.

runveil/udocker binds an entire private /etc/vdev view for each instance, containing only that instance's native configuration and the Docker configuration with the same filename. Other IDs and the ownership ledger are not exposed through this view. HYHAL logs use a separate writable sticky 1777 directory for each instance. Both launchers reuse the same structured mounts. PRoot provides neither a separate cgroup nor kernel-enforced read-only mounts; workloads use the host runtime account's permissions, so this does not establish security isolation against hostile programs. The BW Notebook experiment verified that two HIP contexts with different quotas coexist in the same outer cgroup, retain their data, reject allocations over quota, and allow one context to continue after the other stops. Each node must still pass its own per-GPU probe.

The per-GPU probe must confirm that HIP reports the requested memory quota, permits a 1 MiB allocation, and rejects an allocation exceeding the quota by 1 MiB. dGPU scheduling requires a passing probe. Inference containers use their own DTK; host DTK and supplementary ABI libraries are used only by the small probe. The built-in Qwen3-0.6B bf16-vllm-hygon-z100 spec uses the SourceFind image tested on Z100; the DTK 26.04 vLLM spec also accepts cards reporting the exact model BW.

Non-root users can use a vdev created beforehand by root. The image must provide a passwd entry for the workload UID and a writable HYHAL log directory. The workload must be able to read the configuration snapshot and DTK/HYHAL libraries and access /dev/kfd, /dev/mkfd, and the selected DRM devices. A host DTK symlink into /root can still block a non-root user through directory permissions. The K100_AI HYHAL 1.5.0 CLI requires root to create and delete vdevs, so non-root workloads do not imply that the agent's entire vdev lifecycle can run without root. Inference containers do not need root to satisfy host lifecycle permissions.

Under a host lock, the helper creates a vdev and saves a configuration snapshot and owner token, preventing stale requests from releasing a later gpuworks allocation that reuses the same ID. The vdev is released only after its container has been removed. Failed cleanup retains the record and the instance stays removing; no replacement container starts for that instance. After an agent restart, orphaned records require manual inspection and are not scanned automatically. Other vdev managers that bypass the helper lock and tokens are outside this ownership protection.

GPU Models and Architectures

When the server receives a node report, it normalizes model names before storing them (accelerator.NormalizeGPUName): it removes a leading vendor word (NVIDIA, Ascend, Huawei, Hygon, AMD, Apple, or Kunlunxin, followed by whitespace), trims leading and trailing whitespace, and collapses internal whitespace. Both NVIDIA GeForce RTX 4090 and GeForce RTX 4090 are stored as GeForce RTX 4090; Ascend 910B4 is stored as 910B4. The inference catalog's name predicates, PD's same-model check, and the GPU list all read this value. The original name is retained only on the agent side (doctor, logs). Existing benchmark snapshots retain the names recorded at the time.

Only raw architecture facts are stored: NVIDIA stores compute capability; other vendors store the most specific identity that can be determined in arch_id. The architecture family (the API's arch_family) and NVIDIA's sm_XY (the API's arch_id) are calculated by accelerator.GPUArch at read time and are not stored in the database. The server does not read the arch_family field reported by the agent.

Vendorarch_id (Stored)Architecture Family (Derived, Lowercase)
NVIDIANot stored (sm_XY is derived from compute capability)maxwell 5.x, pascal 6.x, volta 7.0/7.2, turing 7.5, ampere 8.0–8.7, ada 8.9, hopper 9.x, blackwell 10.x–12.x
AscendChip model: 910B4, 910B2C, 910C, 310P3910b, 910c, 310p; empty for first-generation 910 and unrecognized models
HygonSKU: K100AI, BW, BW1000; stores the series K100 when only the series is reported; empty if unrecognizedk100, z100, bw, bw1000, bw1100
PPUZW810E (from the name)beethoven (the architecture column in ppu-smi)
AMDKFD gfx_target_version, e.g. gfx90a, gfx942, gfx1151; empty when unavailableNot derived yet
AppleSoC name: M1 Max, M4 Pro (the model from ioreg with Apple removed)Chip generation m1…m6
Kunlunxinkl3 (native product_architecture, lowercased)kl3

When the agent detects no accelerator cards (none of the vendor CLIs respond or are installed, and no vendor device files exist), the node's vendor is empty; it no longer reports NVIDIA by default.

Kunlunxin uses vendor kunlunxin and framework xre; CUDA compatibility versions are neither CUDA identity nor XRE versions. internal/accelerator owns capabilities, evaluated against candidate and selected cards. P800 supports whole-card TP, PP and engine DP, but not PD, partitions or sleep. It requires Linux amd64 and Docker/nerdctl, or a Kubernetes node (whole cards, device nodes through the agent's device plugin); runveil is not enabled. Incompatible cards produce structured placement refusals without preventing other candidates from being tried. Production single-GPU launch, gateway chat, stale-device cleanup and agent restart passed validation on 2026-11-10. Single-node multi-GPU moved past the loopback-interface failure but remains blocked by the test environment's XRE OAM/XLink initialization error; cross-node operation was not tested, so multi-GPU and cross-node acceptance is not claimed. On 2026-11-14 a Kubernetes node was validated on a real P800 under a temporary k3s: detection, the agent's device plugin handing the instance Pod only the selected card (/dev/xpuN and /dev/xpuctrl), single-card vLLM chat through the gateway, and cleanup after delete all passed; vLLM needs the template's --enforce-eager and related arguments, without which speculative_attention fails during CUDA graph capture (unrelated to the runtime).

Kunlun requires an explicit nonempty GPU selection. Unknown handwritten commands fail closed with LAUNCH_FEATURES_UNKNOWN / launch_features_unknown. At launch, backend-managed XPU/XDR device mappings and conflicting runtime options from templates or user parameters are removed before the current selection is injected; unrelated device mappings are preserved. Single-node multi-GPU collectives also require a non-loopback interface: launch assembly uses the real interface reported on the node's management network, and the agent derives BKCL_SOCKET_IFNAME. Missing interfaces refuse the launch before container creation; single-GPU launches are unaffected.

A card with invalid identity or ECC information becomes unschedulable without losing its peers; PCI inventory supplies missing cards. Runtime injection freshly verifies UUID, PCI BDF, minor and sysfs character-device numbers, automatically exposing only the selected /dev/xpuN and /dev/xpuctrl. The common process inventory reports compute occupancy; zero memory use is not proof of availability.

Single-node device injection does not require XDR or RDMA. Multi-node TP requires XDR on every rank's host (/dev/xdrdrv as a real character device), and a rank without it fails during device injection; /dev/infiniband, an unlimited memlock, IPC_LOCK and GLOO_SOCKET_IFNAME come from the host RDMA compute network. The Kunlun probe appends the selected XPUs, control device and XDR character device, derives the BKCL switches and BKCL_SOCKET_IFNAME, and neither enumerates HCAs nor duplicates options the network already resolved. The deployment environment must also provide BKCL_RDMA_NICS. When resolved resources or netdevs are missing, or a finite memlock is set, a multi-node launch is refused rather than topped up. The platform neither loads drivers nor changes host networking.

Container options follow the common administrator template contract. Scheduled device injection is not a security sandbox for arbitrary privileged options, images or host mounts. The agent injects container-logical XPU_VISIBLE_DEVICES, and CUDA_VISIBLE_DEVICES in the image entrypoint also uses container-logical indexes. Import suitable image and model specs explicitly; see Kunlun P800 deployment.

Was this page helpful?