One address for models running on different machines
ZStack announced on November 4 that GPUWorks is open source. The software gives infrastructure teams a way to run AI models on their own accelerator servers and rented cloud machines, while applications send inference requests to a common OpenAI-compatible endpoint. The project is publicly available on GitHub under Apache 2.0. Bundled components retain their own licenses.
The project’s setup guide starts with a management server and a compute node. An operator selects a model and an inference template. GPUWorks estimates the memory needed and proposes where to run the workload. A team can review that placement before deploying, then issue an API token to applications that need to call the model.
Check the model and engine against the hardware
The project documents deployments across NVIDIA, AMD, Huawei Ascend, T-Head PPU, Hygon and Apple Silicon hardware. The options vary by platform. For example, the repository lists native inference engines for Apple Silicon but does not list the same multi-node or GPU-sharing features there. Its documentation describes Kubernetes and Slurm integration as future work.
For an evaluation, start with a model the team already serves and verify placement, memory use and request latency under its own traffic. The shared endpoint simplifies how applications reach deployed models, while hardware compatibility still depends on the model and inference engine underneath.
The license covers software, not operating costs
Public code and setup instructions make self-hosting an option. Teams still supply or rent the compute and check the separate terms for the models they deploy. The sources reviewed do not establish pricing for commercial support or country-by-country availability of cloud integrations. CGH reviewed the documentation and did not run a deployment or performance test.