Skip to main content

User Glossary

This is a concise glossary of user-facing terms. For complete definitions and counterexamples, refer to CONTEXT.md in the root directory.

TermMeaning
NodeA host or container that runs gpuworks-agent, reports GPUs, and hosts instances
InstanceAn inference service process/container that a deployment runs on a node
Original modelThe stable identity of a model, used for aggregation and deduplication across engines (such as deepseek-ai/deepseek-v4)
SpecAn executable combination of engine, image, weights, GPU predicates (GPU matching conditions), and parameters
ScenarioA label describing a spec's intended use, such as low latency, balanced, or high throughput
PresenceThe state of an image/model on a node, such as absent, pulling, or present
AcquisitionThe process by which weights and images arrive on a node. When starting a replica, the node decides based on pull settings whether to use an existing local copy, copy within a sync group, pull, or download; the deployment does not choose the source
Peer syncDirect transfer of models or images from an online node with a complete copy, without relaying through the server; happens automatically within sync groups and can also be initiated manually
Pull settingsSource rules (image rules and model site rules), proxies, sync groups, and credentials that determine where nodes pull images and download models, which proxy they use, and which peers they can copy from
Site ruleA source rule whose match starts with http(s)://: it matches a model site's root (such as https://huggingface.co), and its alternative address can include a path. Rules are cumulative, fallback to the original source can be disabled, and "Used by the management node for model metadata" can be selected
Pull proxyAn ordered proxy entry in pull settings. A node uses the first entry whose label requirements it meets; the management node uses the first entry with "Used for management node outbound access" selected
Pull candidateA candidate registry reference for the same image or a candidate download address for the same model (alternative addresses from source rules come first, followed by the image's own replicas or the site itself). Candidates are tried in order until one succeeds; failed addresses enter a 30-minute cooldown
EvaluationThe umbrella concept for measuring a service; currently includes benchmarks
BenchmarkApplies a controlled load to an OpenAI-compatible endpoint and records performance metrics
DeviatedAn informational state indicating that you have overridden template fields; not an error
Quick/Expert modeTwo views of the same deployment page: Quick mode shows only preselected common fields (an allowlist-based form), while Expert mode provides full parameter editing
Weighted QoSBackends such as PPU allocate throughput according to quotas when busy; this does not mean hardware partitioning

Was this page helpful?