Not specifically about ML, but a good paper about unnecessary complexity introduced by a premature 'scalabilitization':
The COST of a given platform for a given problem is the hardware configuration required before the platform outperforms a competent single-threaded implementation.
Or
“You can have a second computer once you’ve shown you know how to use the first one.” –Paul Barham
