Developer releases embedflow, a tool to swap embedding models without re-embedding a corpus

A developer posting to r/LocalLLaMA has released embedflow, a library that migrates a vector index from one embedding model to another without re-embedding the full document set. The post describes the method as simple: take K documents from the old index built with the source model, rerank them with the new model, and use that to bridge the two embedding spaces. When K is large enough, the author reports retrieval quality matching the target model's native retrieval. The author says determining K is the hard part of the process.

The problem embedflow targets is the backfill cost of an embedding upgrade. The post lays out the arithmetic: a corpus of 1 billion vectors built with model A cannot be served with model B until every document is re-embedded. On an H100, the author estimates that job at roughly 108 days using Qwen embed 8B at 106 documents per second. That figure is the author's own calculation for one model and one GPU, not a benchmark from a third party.

The author reports testing 63 migrations on corpora of up to 1 million documents. The best result cited was an upgrade from a Qwen 4B embedding model to an 8B model, where 50 documents were enough to match native retrieval quality. The post does not break down results for the other 62 migrations, and the excerpt does not state which models were tested beyond the Qwen pair or what retrieval metric was used to declare parity.

embedflow ships as a PyPI package installable with pip install embedflow, and the author says it works with Qdrant, pgvector, and FAISS. The GitHub repository is public at github.com/arnsri33/embedflow. The author is asking LocalLLaMA readers to try it in their own pipelines, which means the reported results have not been independently reproduced by anyone other than the author at the time of posting.

The claim that a small set of reranked documents can stand in for a full re-embedding is the part that needs scrutiny. Retrieval parity on the author's test set does not establish parity across domains, query distributions, or corpus sizes beyond 1 million documents. The post itself flags K selection as unresolved, which suggests the method's reliability depends on a tuning step the author has not fully automated.

What happens next depends on whether other users reproduce the 63-migration results and whether the K-selection problem gets a documented answer. Until then, the 108-day H100 figure and the 50-document parity result are single-source claims from the tool's author. Readers evaluating embedflow for production should treat the migration quality numbers as unverified and test K on their own query set before retiring the old index.