Traffic Shaping for Workload Classification

Abstract

Note: this is a repost of a verification design I developed working as a consultant with the team at Lucid Computing and originally posted on the Lucid Labs Substack. Overview In this design brief we present “Traffic Shaping for Workload Classification” – a verification solution that enables a third party to verify that frontier training [1] is either not occurring in a declared compute facility, or is occurring, but at a cost multiple that would render a frontier training run of a new model that is 10x or more the size of current frontier models economically infeasible. Our design builds on existing traffic restriction and compartmentalization designs, which require reorganizing compute into size-restricted “pods” and introducing traffic shapers to throttle the external network traffic to the pod to a threshold per GPU which is sufficient for inference, but insufficient for frontier training. Additionally, our design requires the introduction of a “random router” for inference requests, which ensures that inference requests are routed randomly to pods. This adds resistance to decentralized training techniques like pipeline parallelism. Unlike network tap designs with recomputation

Research area