3,000 Tokens/Sec Locally: Testing the ZGX Fury

Video thumbnail: 3,000 Tokens/Sec Locally: Testing the ZGX Fury
Oct 30, 202613m 58s video lengthSam Witteveen

The Signal

HP’s ZGX Fury G1N AI Station is a high-end enterprise system utilizing NVIDIA’s GB300 Blackwell Ultra architecture, designed to function as a shared on-premise infrastructure rather than a personal workstation. The unit differentiates itself through proprietary software, specific OS certifications, and tuned model optimizations, aiming to reduce cloud token costs and enhance data sovereignty for team-based AI workloads.

The Case

Hardware and Positioning

  • The ZGX Fury is a rack-mountable or desk-side appliance featuring 72-core Grace CPUs, 748 GB of shared memory with 252 GB of HBM, and 20 PFLOPS of FP4 compute.1:17
  • HP positions this as a team-scale tool, intended to support multiple concurrent users running agents, fine-tuning, and inference tasks simultaneously from a single local source.0:44
  • The hardware supports high-speed connectivity via two 400 Gb/s network ports, allowing for scalability if workloads exceed the capacity of one unit.1:44

Software and Optimization

  • HP provides the ZRT (HPZ runtime) toolkit to automate model deployment via Hugging Face, configure vLLM serving, and monitor concurrent throughput.3:41
  • The system ships with Ubuntu but claims to be the first GB300 station certified for Red Hat Enterprise Linux, a move intended to align the box with standard data-center production environments.5:37
  • A customized version of the Nemotron Super model, tuned by Infinity AI, is reportedly included to significantly increase concurrency and aggregate token throughput on the same hardware.6:11

Performance and Economics

  • The system demonstrates high performance in live benchmarks, such as maintaining 1,300 tokens per second for 16 concurrent users and processing over 500,000 tokens during 64-user stress tests.7:11
  • The core economic value proposition is the elimination of per-token API bills and external policy constraints, allowing organizations to maintain full control over model guardrails and sensitive data.9:06

The 1 Minute Signal Take

The ZGX Fury represents a move toward bringing data-center-grade compute directly into the office to bypass the cost and security trade-offs of cloud-based AI. While the performance benchmarks demonstrate significant capacity for shared team environments, the practical value of the system relies on whether your team’s specific workflows benefit from local orchestration over external APIs.

Pro Analysis

Why It Matters

The ZGX Fury represents a pivotal shift in the AI hardware landscape: the professionalization of the 'local box.' By shifting from individual GPU workstations to shared-team AI appliances, HP is directly challenging the cloud-dominant paradigm for enterprise inference.

Strategic Implications

This hardware forces a rethink of AI economics. If a team can serve 64 concurrent users on-premise, the 'per-token' pricing model of cloud providers becomes a liability rather than a necessity. Companies in regulated sectors like healthcare or cybersecurity gain the ability to enforce their own privacy guardrails without trusting external providers.

Evidence & Hype Audit

The content relies heavily on demonstrations rather than independent, third-party benchmarks. While the throughput figures (>3,000 tokens/sec) are impressive, they are presented in controlled environments using specific tuned models. The claim that this is the 'only' station allowed for such customization remains an HP-asserted marketing point rather than an independently verified fact.

Counterarguments

The primary risk is the 'locked-in' nature of the hardware-software stack. While HP's tooling provides convenience, it creates a dependency on HP's maintenance of the ZRT ecosystem. Furthermore, massive up-front capital expenditure (CapEx) for local hardware may be less efficient than cloud bursts for organizations with highly variable or unpredictable AI workloads.

Who Should Care

  • CTOs/Infrastructure Leads: Evaluating on-prem AI to control costs and data residency.
  • Security/Red-Team Leads: Looking to run sensitive experiments without cloud guardrails.
  • Enterprise AI Teams: Needing shared resources for concurrent inference and fine-tuning.

What To Do Next

  • Audit current monthly cloud token spending to determine the 'break-even' point for on-prem hardware.
  • Test your specific model weights against the 748 GB memory limit to ensure they fit within a single-node configuration.
  • Evaluate your team’s readiness for local operations, including power, cooling, and network infrastructure management.
  • Request a pilot unit to perform a multi-user concurrency test using your team’s production RAG workloads.
  • Verify if your existing software CI/CD pipelines can integrate with the ZRT toolset.
Time saved:10m 30s

Share this

Tags

Written by: 1 Minute Signal Editorial Team