We've had basic heating, insulation, and lighting efficiency improvements. Also, population has moved from colder to warmer climates. You can probably count up your personal energy usage and see that it's reduced. LED lighting, CRT televisions obsolete, vehicle efficiency, and so on.
Meanwhile, people mostly use a couple devices with high energy efficiency that are used for basically everything, instead of twenty lower quality devices...
