I have to imagine that the vast majority of the CI time is spent rendering HTML and executing javascript on a wide variety of hardware and software combinations, and has nothing to do with compiling. Even the simplest possible support matrix would look like this:

MacOSX

===> Monterray, Big Sur, Catalina

===> x86, Aarch64

Windows

===> 10, 11

===> x64, Aarch64

===> Intel CPU $(flavors), AMD CPU $(flavors)

===> Nvidia GPU $(flavors), AMD GPU $(flavors), Intel GPU $(flavors)

And that's excluding Linux where the combinations get even crazier