Reference
Vendor Support Matrix
On this page
| Vendor/Framework | Detection | Full-GPU Deployment | dGPU | Topology/Notes |
|---|---|---|---|---|
| NVIDIA / CUDA | nvidia-smi, sysfs | Docker, runveil, nerdctl | HAMi libvgpu.so (x86_64, glibc ≥ 2.17) | Supports CUDA/NCCL topology; driver libraries are supplied by the host |
| Ascend / CANN | npu-smi, driver and CANN facts | Docker + Ascend runtime; runveil in some environments | vcann-rt; capability depends on host device-share permissions | Host and container CANN versions must not be conflated into one conclusion |
| AMD / ROCm | amdgpu sysfs, KFD | Linux amd64, whole-card deployment on Docker/nerdctl/runveil/Kubernetes; the upstream vLLM ROCm image has been tested for single-GPU BF16 text inference on MI325X (gfx942) and AMD395 (gfx1151), other combinations require per-profile validation | amd-dgpu memory quota (rocm-slicing-shim, user space; tested on MI300X/gfx942 under Docker and runveil and on MI325X under Kubernetes; Docker/nerdctl/runveil/Kubernetes) | Multi-node and PP run on the vLLM mp backend (RCCL reads the NCCL_* network variables; PP=2 and cross-node TP=2 tested on two MI325X hosts); vLLM sleep/wake (L1/L2) and PD (vllm-nixl, Ethernet UCX TCP) tested on MI300X; not a compatibility claim for all AMD GPUs or ROCm combinations |
| PPU / HGGC | ppu-smi | Depends on provider/runtime | ppu-slicing-shim memory quota and weighted QoS (Docker, Kubernetes) | Sustained 20/80 workloads have demonstrated throughput weighting under load, not hard SM partitioning |
| Hygon / DTK | hy-smi | Determined by the catalog spec and runtime | Native Z100 / K100_AI / BW vdev (experimental, x86_64, Docker or runveil/udocker) | Requires the optional native helper, host libraries and a per-GPU behavioral probe; other models remain full-GPU only |
| Apple / Metal | sysctl, ioreg (macOS 15+, Apple silicon) | host (the engine is a host process; the image is a darwin environment package) | None; a Mac runs multiple instances concurrently using unified memory | One unified-memory card; GPU memory is the Metal working set; engine sleep, multi-node operation, and the KV disk tier are unsupported; see ADR 0044 |
| Kunlunxin / XRE | xpu-smi XML, PCI/sysfs device identity | Linux amd64, Docker/nerdctl/Kubernetes, P800 PCIe / KL3, whole-card TP / PP / DP | Unsupported | TP, PP and engine DP; cross-node communication prerequisites apply; no PD, sleep or runveil; explicit image/model catalog specs |
The dGPU capability status in the table above is derived from the server's partition capability assessment:
ready: dGPU is available.unavailable: dGPU is unavailable, with areasonexplaining why, such as “Missing dGPU assets” or “dGPU probe failed”.
When dGPU is unavailable, the node can still use full-GPU deployment. The platform does not silently start instances that fail dGPU requirements with unlimited GPU memory instead. For tested Ascend combinations, CANN, and runtime details, see the Ascend Matrix.
Kubernetes nodes (see Connect a Kubernetes Cluster) support whole-card deployment for NVIDIA, Ascend, Hygon, PPU, AMD, and Kunlunxin; dGPU is available there for NVIDIA HAMi-core, Ascend vcann-rt, PPU ppu-slicing-shim, and AMD rocm-slicing-shim, while Hygon's native vdev remains unavailable on Kubernetes (whole-card deployment only).
dGPU Capability Comparison
| Backend | GPU memory limit | Compute semantics | Memory quota changes | Behavioral probe |
|---|---|---|---|---|
| NVIDIA HAMi-core | Hard memory limit | The platform does not expose a compute cap; CUDA graphs and cuBLAS bypass HAMi launch throttling | Supports safe live adjustment; the engine may still need a restart to use the new quota | Uses embedded asset validation and admission checks; exempt from behavioral probes |
| Ascend vcann-rt | Memory quota, subject to host device-share permissions | Depends on the vcann-rt and host combination; does not promise a fixed throughput ratio | Requires an instance restart | Must pass per GPU |
| PPU ppu-slicing-shim | Memory quota | Weighted QoS under contention, borrowing spare capacity when idle; not hard SM partitioning | Requires an instance restart | Must pass per GPU; Docker and Kubernetes |
| Hygon Z100 / K100_AI / BW native vdev (experimental) | Native vdev memory quota; one GPU per instance | CUs are allocated in proportion to memory; no independently configurable compute quota | Requires an instance restart, recreating the vdev with the new memory quota | Must pass per GPU; Docker or runveil/udocker on x86_64 |
| AMD rocm-slicing-shim (user space) | Hard memory limit interposed at the HIP runtime layer | The platform does not expose a compute cap; ROCm enforces compute quotas in hardware through HSA_CU_MASK, below what a preloaded library can see or verify | Requires an instance restart (the limit is read once, at HIP runtime init) | Exempt from behavioral probes (neither the host nor the lightweight probe image is guaranteed a ROCm user-space runtime to probe with); relies on embedded asset validation; Docker/nerdctl/runveil/Kubernetes |
For safe live adjustment conditions and reasons a restart is pending, see Adjust dGPU Quotas.
These capabilities do not imply that every inference image has been verified; availability still depends on the GPU's status / reason.
AMD full-GPU implementation boundary and validation status
The current AMD host integration in the agent and server is scoped to Linux amd64, Docker, nerdctl, Kubernetes or runveil (the udocker driver), and full-GPU deployment (single host or vLLM multi-node); Kubernetes supports both whole cards and dGPU memory partitioning. The PCI roster recognizes AMD functions only when bound to amdgpu. The agent then reads their amdgpu sysfs/KFD facts, checks inventory by PCI BDF and available unique IDs, and injects only the selected card's render node plus /dev/kfd. dGPU memory partitioning (amd-dgpu, libvrocm, memory only — no compute limit) layers an LD_PRELOAD and quota environment on top of this device injection; see "dGPU Capability Comparison" above. dGPU runs under Docker, nerdctl, Kubernetes and runveil (PRoot binds libvrocm.so and loads it through LD_PRELOAD alike). runveil binds the host's whole /dev and /sys, so HIP enumerates every card; the agent pins the selection with ROCR_VISIBLE_DEVICES=GPU-<unique_id> (ROCm's GPU UUID), and refuses a card without a unique_id under runveil. PRoot runs as the non-root runtime user with the device nodes' owning groups (such as render) added automatically; validated on MI300X. Data parallelism (DP) takes the same path as on NVIDIA. Sleep and PD reuse the vLLM protocol and the vllm-nixl mode; with RDMA the platform names the HCA but not the CUDA-specific UCX_TLS, letting UCX choose its ROCm transports, which is not validated on hardware. Multi-node and PP use the vLLM mp backend: like NVIDIA, RCCL reads NCCL_SOCKET_IFNAME and NCCL_IB_*, the agent fills in the interface from the node address, and IB is disabled without RDMA. This describes GPUWorks' implementation scope; it is not a compatibility claim for every AMD GPU, driver, kernel, or ROCm userspace combination.
The built-in image catalog lists gpuworks/gpuworks-runner:rocm7.2.3-vllm0.30.0-r1 (from the upstream vllm/vllm-openai-rocm, ROCm 7.2.3, vLLM 0.30.0). The built-in inference catalog has BF16 vLLM specs for Qwen3 0.6B–32B and 30B-A3B with arch_id: gfx942 (MI300X/MI325X), and for 0.6B–8B with arch_id: gfx1151 (Ryzen AI Max+ 395); pick a model and deploy, with no image or template import. Tested: MI325X single GPU, PP=2 and cross-node TP=2 on two MI325X hosts, and AMD395 single GPU; this shows short text inference and the deploy/stop flow only, not answer quality, sustained performance, long contexts, or image input. The SGLang ROCm entry remains an unpublished catalog: false candidate. UMA capacity uses the observed KFD/GTT facts and should not be generalized to other APUs or hosts. gpus.arch_id is an exact, case-insensitive match; missing or mismatched architecture facts do not pass. See the AMD Hardware Validation Guide and the ROCm image recipe.
AMD dGPU memory quota
The amd-dgpu backend uses libvrocm.so (rocm-slicing-shim, build/runtime-assets/amd/), which interposes the HIP runtime's allocating and capacity-query entry points. It links only libc, so one build serves every ROCm version. Injection layers on top of AMD's full-GPU device selection (render node + /dev/kfd): LD_PRELOAD points at the mounted /opt/gpuworks/partition/libvrocm.so, VROCM_DEVICE_MEMORY_LIMIT_<N> (MiB) is written for each selected card at its position N in the container's own visible-device list (not its physical ordinal), and VROCM_LEDGER_PATH names a per-instance ledger under the host's /dev/shm (/dev/shm/gpuworks-amd-<instance uuid>), unchanged across Docker, nerdctl and runveil. The container reaches that path under Docker, nerdctl and runveil through --ipc=host, sharing the host's /dev/shm directly. On Kubernetes alone the agent's own /dev/shm is its own pod's private tmpfs rather than the host's: the agent pre-creates an empty placeholder at that same path through the host root (/proc/1/root), mode 0o666 (not 0o777: the container may run as a UID with no entry on the host and must be able to write the file, but needs no execute bit), and hostPath-mounts that exact host path, read-write, into the instance pod at the same path. VROCM_LEDGER_PATH's value is unchanged by this; libvrocm.so opens the (already-existing) file as usual and unlinks it on a clean exit, and the agent's own cleanup removes it again through the host root as a backstop. The limit is read once, at HIP runtime init, so lowering it takes effect only after an instance restart.
There is no compute (SM/CU) quota: ROCm enforces compute quotas in hardware through HSA_CU_MASK, below anything a preloaded library can see or verify, so this backend is memory-only and never exposes a compute limit.
Unlike PPU/Ascend/Hygon, AMD's full-GPU device injection mounts no host ROCm user-space library at all (only /dev/kfd and the selected render node), and the lightweight probe image has no HIP runtime to call into either. This backend is therefore probe-exempt (SkipsBehaviorProbe): readiness rests on the embedded asset's checksum and the Docker/nerdctl/udocker/Kubernetes runtime constraint alone — the same approach as NVIDIA's HAMi-core, unlike PPU/Ascend/Hygon, which require a passing per-GPU probe.
PPU device identity and dGPU probes
PPU queries ppu-smi and verifies identities against device nodes, then injects two ordered lists: numeric device selectors in CUDA_VISIBLE_DEVICES for compute, and full UUIDs in GPUWORKS_PPU_DEVICE_UUIDS for the probe and HGML quota library. Docker/nerdctl selectors use the filtered inventory; udocker/runveil selectors use host indexes. Both lists preserve selection order. vLLM and SGLang share numeric visibility without engine-specific dispatch. HGML reuses UUID-order quota and ledger lookup, falling back to the older agent visibility variable only when the separate UUID variable is absent.
The embedded gpuworks/partition-probe:v4 verifies both the selected UUID and its memory quota. Missing, invalid, duplicate or unqueryable identities fail, as does a quota applied only to another card. Recreate existing workloads after upgrading the agent to activate the new selection and slicing libraries. Device visibility under runveil still does not provide kernel device isolation.
HGGC_LEDGER_PATH names a per-instance ledger under the host's /dev/shm (/dev/shm/gpuworks-ppu-<instance uuid>), reaching the instance pod the same way AMD's does (see "AMD dGPU memory quota" above): unchanged under Docker (--ipc=host), and pre-created through the host root and hostPath-mounted at the same path on Kubernetes. PPU dGPU partitioning currently supports Docker and Kubernetes; udocker remains unverified.
Hygon full-GPU driver libraries
Hygon full-GPU workloads receive the host /opt/hyhal through a structured mount shared by Docker, nerdctl and udocker/runveil. For example, DTK 26.04 PyTorch needs its librocm_smi64.so.2. Device injection fails early if the directory is absent. Containers keep the image's DTK and library search path; recreate workloads after upgrading the agent to apply the mount fix. The mount is declared read-only; PRoot binds use the host runtime account's permissions and do not provide kernel-enforced read-only isolation.
The selection variables HIP_VISIBLE_DEVICES and HYGON_VISIBLE_DEVICES use the numbering HIP sees inside the container. Docker and nerdctl pass only the selected cards' DRM nodes, and HIP numbers them 0..N-1 in the container, so these runtimes get container ordinals: host card 2 is 0 in a one-card container. udocker/runveil binds the host's whole /dev and /sys, HIP sees every card, and the host indexes are kept. Recreate workloads after upgrading the agent for the new numbering to apply.
On hosts whose cards cannot use P2P, Hygon vLLM's custom all-reduce may fail to start at TP > 1 (HIP error: invalid device pointer); add --disable-custom-all-reduce to the extra arguments.
Hygon Z100 / K100_AI / BW dGPU
Z100, K100_AI devices identified precisely as K100AI, and cards reporting the exact model BW use the hygon-dgpu backend; K100_AI and K100 AI are aliases of the same SKU. BW is not rewritten as BW1000, and a report containing only the K100 family does not enable dGPU. The agent reads the selected physical card's CU count through hy-smi virtual -d N --show-device-info, then assigns CUs in proportion to the memory quota, rounding up. Failed or incomplete queries reject vdev creation without falling back to a fixed CU count. Build and install the optional gpuworks-hygon-vdev-helper on the node (see build/runtime-assets/hygon-vdev-helper/README.md in the source tree), and provide the host DTK, HYHAL and DCGM libraries. DTK 26 environments may keep the RSMI library in HYHAL's lib directory. The agent itself remains a static binary. Docker and runveil/udocker on x86_64 are supported; nerdctl and other models do not receive this capability automatically.
Each dGPU instance can use only one supported Hygon GPU. The requested memory quota, rounded up to MiB, must be positive and strictly below the card's physical memory. The CU count is ceil(quota MiB × physical CU count / physical memory MiB); there is no separate compute quota or promise of proportional inference throughput. Quota changes require an instance restart, releasing the old vdev before creating one with the new quota.
The current compatibility profile lists DTK 26.04.x × x86_64, but the profile is not marked as fully verified on real hardware. Version information is advisory and cannot replace the per-GPU probe.
Container configuration uses /etc/vdev/docker/vdev<ID>.conf matching the native ID, exposing exactly one Docker configuration. Hardware checks on Z100 and K100_AI show that renaming a nonzero native ID to vdev0.conf prevents HIP initialization. The native filename has not been revalidated on older HYHAL 1.4 environments; they must still pass the per-GPU probe. The small Docker probe receives a private /var/log/hylog tmpfs and supplementary tinfo5/6 libraries under the SONAMEs actually installed on the host.
runveil/udocker binds an entire private /etc/vdev view for each instance, containing only that instance's native configuration and the Docker configuration with the same filename. Other IDs and the ownership ledger are not exposed through this view. HYHAL logs use a separate writable sticky 1777 directory for each instance. Both launchers reuse the same structured mounts. PRoot provides neither a separate cgroup nor kernel-enforced read-only mounts; workloads use the host runtime account's permissions, so this does not establish security isolation against hostile programs. The BW Notebook experiment verified that two HIP contexts with different quotas coexist in the same outer cgroup, retain their data, reject allocations over quota, and allow one context to continue after the other stops. Each node must still pass its own per-GPU probe.
The per-GPU probe must confirm that HIP reports the requested memory quota, permits a 1 MiB allocation, and rejects an allocation exceeding the quota by 1 MiB. dGPU scheduling requires a passing probe. Inference containers use their own DTK; host DTK and supplementary ABI libraries are used only by the small probe. The built-in Qwen3-0.6B bf16-vllm-hygon-z100 spec uses the SourceFind image tested on Z100; the DTK 26.04 vLLM spec also accepts cards reporting the exact model BW.
Non-root users can use a vdev created beforehand by root. The image must provide a passwd entry for the workload UID and a writable HYHAL log directory. The workload must be able to read the configuration snapshot and DTK/HYHAL libraries and access /dev/kfd, /dev/mkfd, and the selected DRM devices. A host DTK symlink into /root can still block a non-root user through directory permissions. The K100_AI HYHAL 1.5.0 CLI requires root to create and delete vdevs, so non-root workloads do not imply that the agent's entire vdev lifecycle can run without root. Inference containers do not need root to satisfy host lifecycle permissions.
Under a host lock, the helper creates a vdev and saves a configuration snapshot and owner token, preventing stale requests from releasing a later gpuworks allocation that reuses the same ID. The vdev is released only after its container has been removed. Failed cleanup retains the record and the instance stays removing; no replacement container starts for that instance. After an agent restart, orphaned records require manual inspection and are not scanned automatically. Other vdev managers that bypass the helper lock and tokens are outside this ownership protection.
GPU Models and Architectures
When the server receives a node report, it normalizes model names before storing them (accelerator.NormalizeGPUName): it removes a leading vendor word
(NVIDIA, Ascend, Huawei, Hygon, AMD, Apple, or Kunlunxin, followed by whitespace), trims leading and trailing whitespace, and collapses internal whitespace.
Both NVIDIA GeForce RTX 4090 and GeForce RTX 4090 are stored as GeForce RTX 4090; Ascend 910B4 is stored as 910B4.
The inference catalog's name predicates, PD's same-model check, and the GPU list all read this value. The original name is retained only on the agent side (doctor, logs).
Existing benchmark snapshots retain the names recorded at the time.
Only raw architecture facts are stored: NVIDIA stores compute capability; other vendors store the most specific identity that can be determined in arch_id.
The architecture family (the API's arch_family) and NVIDIA's sm_XY (the API's arch_id) are calculated by accelerator.GPUArch at read time
and are not stored in the database. The server does not read the arch_family field reported by the agent.
| Vendor | arch_id (Stored) | Architecture Family (Derived, Lowercase) |
|---|---|---|
| NVIDIA | Not stored (sm_XY is derived from compute capability) | maxwell 5.x, pascal 6.x, volta 7.0/7.2, turing 7.5, ampere 8.0–8.7, ada 8.9, hopper 9.x, blackwell 10.x–12.x |
| Ascend | Chip model: 910B4, 910B2C, 910C, 310P3 | 910b, 910c, 310p; empty for first-generation 910 and unrecognized models |
| Hygon | SKU: K100AI, BW, BW1000; stores the series K100 when only the series is reported; empty if unrecognized | k100, z100, bw, bw1000, bw1100 |
| PPU | ZW810E (from the name) | beethoven (the architecture column in ppu-smi) |
| AMD | KFD gfx_target_version, e.g. gfx90a, gfx942, gfx1151; empty when unavailable | Not derived yet |
| Apple | SoC name: M1 Max, M4 Pro (the model from ioreg with Apple removed) | Chip generation m1…m6 |
| Kunlunxin | kl3 (native product_architecture, lowercased) | kl3 |
When the agent detects no accelerator cards (none of the vendor CLIs respond or are installed, and no vendor device files exist), the node's vendor is empty; it no longer reports NVIDIA by default.
Kunlunxin uses vendor kunlunxin and framework xre; CUDA compatibility versions are neither CUDA identity nor XRE versions. internal/accelerator owns capabilities, evaluated against candidate and selected cards. P800 supports whole-card TP, PP and engine DP, but not PD, partitions or sleep. It requires Linux amd64 and Docker/nerdctl, or a Kubernetes node (whole cards, device nodes through the agent's device plugin); runveil is not enabled. Incompatible cards produce structured placement refusals without preventing other candidates from being tried. Production single-GPU launch, gateway chat, stale-device cleanup and agent restart passed validation on 2026-11-10. Single-node multi-GPU moved past the loopback-interface failure but remains blocked by the test environment's XRE OAM/XLink initialization error; cross-node operation was not tested, so multi-GPU and cross-node acceptance is not claimed. On 2026-11-14 a Kubernetes node was validated on a real P800 under a temporary k3s: detection, the agent's device plugin handing the instance Pod only the selected card (/dev/xpuN and /dev/xpuctrl), single-card vLLM chat through the gateway, and cleanup after delete all passed; vLLM needs the template's --enforce-eager and related arguments, without which speculative_attention fails during CUDA graph capture (unrelated to the runtime).
Kunlun requires an explicit nonempty GPU selection. Unknown handwritten commands fail closed with LAUNCH_FEATURES_UNKNOWN / launch_features_unknown. At launch, backend-managed XPU/XDR device mappings and conflicting runtime options from templates or user parameters are removed before the current selection is injected; unrelated device mappings are preserved. Single-node multi-GPU collectives also require a non-loopback interface: launch assembly uses the real interface reported on the node's management network, and the agent derives BKCL_SOCKET_IFNAME. Missing interfaces refuse the launch before container creation; single-GPU launches are unaffected.
A card with invalid identity or ECC information becomes unschedulable without losing its peers; PCI inventory supplies missing cards. Runtime injection freshly verifies UUID, PCI BDF, minor and sysfs character-device numbers, automatically exposing only the selected /dev/xpuN and /dev/xpuctrl. The common process inventory reports compute occupancy; zero memory use is not proof of availability.
Single-node device injection does not require XDR or RDMA. Multi-node TP requires XDR on every rank's host (/dev/xdrdrv as a real character device), and a rank without it fails during device injection; /dev/infiniband, an unlimited memlock, IPC_LOCK and GLOO_SOCKET_IFNAME come from the host RDMA compute network. The Kunlun probe appends the selected XPUs, control device and XDR character device, derives the BKCL switches and BKCL_SOCKET_IFNAME, and neither enumerates HCAs nor duplicates options the network already resolved. The deployment environment must also provide BKCL_RDMA_NICS. When resolved resources or netdevs are missing, or a finite memlock is set, a multi-node launch is refused rather than topped up. The platform neither loads drivers nor changes host networking.
Container options follow the common administrator template contract. Scheduled device injection is not a security sandbox for arbitrary privileged options, images or host mounts. The agent injects container-logical XPU_VISIBLE_DEVICES, and CUDA_VISIBLE_DEVICES in the image entrypoint also uses container-logical indexes. Import suitable image and model specs explicitly; see Kunlun P800 deployment.
Was this page helpful?