An analogy is that computer science is like if universities had an "optical science" program. The professors in this field would be interested in cutting edge research on microscopes, cameras, and telescopes. The problem is that most of their students aren't actually interested in that but want to get a mainstream job as an optician helping people figure out what glasses they need.

But "optical science" actually is taught, in the physics and engineering programs.

    students aren't actually interested in that
    but want to get a mainstream job as an
    optician

The ophthalmologist is more specialized than the optometrist.

Eye surgery has replaced the need for glasses in cases.

Funny you should pick out the word "science" which Sussman in the video @4:20 says "more like science, you grab this piece of library, and you poke at it..."

He says "like science" but he doesn't say that it is like physics. Sounds more like highly risky field xenobiology with the grabbing and poking.

The empirical process of the scientific method is what came to mind as opposed to the more mathematical theory of computation would be my interpretation.

Sussman went on to write a textbook that investigates classical mechanics starting from the Lagrangian and proceeding by way of simulation experiments in Scheme. I think the distinction is not between science and non-science, but between the "analytical-synthetic" process he mentions and the approach of understanding a living system, hence the preponderance of ecosystem metaphors (Python has the better ecosystem than Java, etc). It might be interesting to think about a science of library ecosystems (NPM, etc) but mostly this probably boils down to "exposure to the bad or inapplicable decisions of others".