'Racism' is absolutely a caste system, since the fundamental point of a caste system is to proportion resource and opportunity between breeding pools, however devised those are. The differentiator is simply the terms used to draw these boundaries. Which matters because different terms, say religious terms vs medico-scientific terms, resource different graphs for further narrativization. Jewish folk from Europe know full well the difference between being narrativized according to religious lines and according to medico-scientific lines, neither even approaching something like true or right. Black folk in America have been hounded about their medico-scientific distinction from the get-go, right down to the resurgence of heritable IQ and multi-regional emergence theory today.