tomaarsen
They're so back 👀 @mixedbreadai released two absolutely tiny late-interaction models for search: 17M and 32M parameters (that's 0.017B and 0.032B for those stuck thinking in B's 😆) Sequence length: 32k Integrated with PyLate Beats ColBERT-v2 on BEIR License: Apache 2.0 🧵
Mixedbread
Models: mxbai-edge-colbert-v0-17m: https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-17m mxbai-edge-colbert-v0-32m: https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m Report: https://mixedbread.com/papers/small_colbert_report.pdf Blog post: https://mixedbread.com/blog/edge-v0