This is reminiscent of Parallella [1]/Epiphany (which unfortunately failed to gain traction). They didn't release any designs with core counts like that, but the entire point of the design was to enable large numbers of cores with in-core memory and predictable latency for accessing the memory of cores elsewhere in the grid.
Epiphany did have (slower) access to external memory too, but the challenge of these architectures is finding problems that requires too much branching to be able to take proper advantage of GPUs, yet so parallel that relatively low single-core performance is outweighed by throwing more cores at it and/or where the low power usage of the Epiphany would be worth it, and it seems they just didn't find enough of a market for it.
I have two of the kickstarter boards sitting around. Wish they'd gotten larger scale versions out.
Epiphany did have (slower) access to external memory too, but the challenge of these architectures is finding problems that requires too much branching to be able to take proper advantage of GPUs, yet so parallel that relatively low single-core performance is outweighed by throwing more cores at it and/or where the low power usage of the Epiphany would be worth it, and it seems they just didn't find enough of a market for it.
I have two of the kickstarter boards sitting around. Wish they'd gotten larger scale versions out.
[1] https://www.parallella.org/