Solana Justifies Use of Bare Metal for Validators

Solana emphasizes the importance of bare metal hardware for its validators, particularly as the network nears enhancements like the activation of 100 million compute units (CUs). This new cap represents a 66% increase over the current 60 million CU limit, enabling higher transaction capacities but also shifting the protocol's bandwidth bottleneck to the Turbine layer, which is responsible for propagating blocks across the network.

The upcoming increase in transaction capacity underscores the need for improved validator performance. "If shreds can’t fan out to thousands of nodes fast enough, the extra capacity is not helpful to the network," the blog states. To support the expected surge in demand, Solana plans to enable XDP (eXpress Data Path), a high-performance networking mode, by default for all clients.

While cloud-based solutions like AWS and GCP have the capacity to run validators, they often incur performance limitations due to abstractions that obscure direct access to hardware. As Solana's blog points out, the performance of these cloud services is not as competitive compared to bare metal setups. Running XDP effectively requires elevated capabilities and dedicated physical cores, as well as considerations for packet rates that may challenge a cloud's structural limitations.

Moreover, the blog highlights that when operators rely on cloud instances, they must engage in extensive tuning and operational adjustments that essentially negate the advantages of using cloud environments. Any virtualized setting that requires significant modifications to maintain performance results in a loss of the flexibility typically associated with cloud deployments.

For future Agave validators operating with XDP, the recommendation is clear: bare metal is preferred. This includes the use of high-clock CPUs, ECC memory, and fast NVMe storage, with connectivity provided by 10 to 25GbE to manage the expected growth in transactions. Solana suggests utilizing dedicated hardware to achieve optimal results as TPS (transactions per second) increases.

In summary, while cloud infrastructure has its merits, for high-performance validators aiming to utilize the forthcoming 100 million CU feature, Solana strongly recommends employing dedicated bare metal hardware to prevent potential bottlenecks and enhance network efficiencies.