Checkout https://en.wikipedia.org/wiki/Embedding#Topology_and_geometr...

Crudely put, embedding is the verb, metric space is the noun.

Wikipedia traces it back to this [1] 2002 paper, which talks about translated documents having an "embedding space" in each language. I guess you could interpret their paper as both English and French being embedded in some larger space, which would make it a valid use of the mathematical term.

From there, people apparently just rolled with it and used the term more broadly, as tends to happen with language.

1: https://proceedings.neurips.cc/paper/2002/file/d5e2fbef30a4e...

Checkout my comment next to yours. Embeddings in Geometry / in vector spaces are a very old topic.

Also, your interpretation of their use of the word embedding is exactly correct.