Watch: Live Blog: Hacking Gemini Embeddings
An experimental study reproducing the vec2vec research paper by attempting to translate and align Gemini and MxbAI embedding spaces using unsupervised methods.
Transcript
Can we translate text embeddings from one AI model to another without any paired data? A recent research paper claims they all share a universal geometry, meaning we can map them to each other, or even reverse-engineer them.
To test this, I set up an experiment comparing Google's Gemini embeddings with an open-source model called Mixedbread AI. In the first round, the models had different dimensions. Interestingly, translating from the higher-dimensional Gemini space to the lower-dimensional Mixedbread space was highly accurate. But going the other way, from low to high, completely failed.
For the next attempt, I used a technique called Matryoshka Representation Learning to make their dimensions equal. I trained a translation model to align the two spaces, but the mapping barely moved the needle. Document retrieval was no better than random guessing. Even when I scaled up to a much larger dataset, the translation quality remained very low.
While the theory of a universal embedding geometry is fascinating, these tests show that practical translation is incredibly difficult. The quality depends heavily on the direction of translation, the size of the dataset, and how we handle different dimensionalities.