Diminishing Returns of Scaling

The phenomenon where performance gains grow sub-linearly as hardware count increases linearly in distributed inference or traini