My understanding, which is not complete, is China has done some amazing things optimizing training on slower chips.

Which is cool, but there are limits to the number of times you can do that.

At the end of the day, the little man has to flip the switch.