平均プールのコサイン類似度は長さ不変ではない:長さ不変な代替指標の理論と実証 Mean-Pooled Cosine Similarity is Not Length-Invariant: Theory and Cross-Domain Evidence for a Length-Invariant Alternative

Similarity computation over learned text embeddings underpins much of modern machine learning infrastructure, from retrieval and recommendation to RAG pipelines and clustering. This paper argues that one of the most common building blocks in that stack, computing cosine similarity over mean-pooled token embeddings, harbors a structural flaw that has largely gone unexamined.

The central claim is that mean pooling produces vectors whose norm and direction depend systematically on the length of the input sequence. As a consequence, the resulting cosine similarity is not invariant to length: when comparing a short query to a long document, or two passages of very different lengths, the score reflects not just semantic closeness but also a length-dependent artifact. The authors derive this behavior theoretically and propose an alternative similarity measure designed to be length-invariant by construction.

On the empirical side, the paper reports cross-domain experiments showing that the proposed measure reduces length-induced bias and yields more consistent rankings and decisions than the conventional mean-pooled cosine baseline. For tasks such as retrieval evaluation, near-duplicate detection, or semantic clustering, where length distributions vary across queries and corpora, this bias can directly distort headline metrics, so the practical implications are non-trivial.

It is worth situating the result in the broader embedding ecosystem. Sentence-BERT popularized mean pooling for sentence representations, and many widely deployed encoders, including the OpenAI and Cohere embedding APIs and open models such as E5 and BGE, are trained and evaluated under a mean-pool-plus-cosine regime (or a closely related CLS-token variant). Whenever the training distribution of sequence lengths differs from the deployment distribution, or when mixed-length corpora are indexed together, the bias identified here could plausibly surface in production systems.

The finding also connects to a longer-running discussion about the geometry of contrastively trained embeddings. Prior work has shown that such embeddings tend to occupy narrow cones on the unit sphere, that cosine and dot product can disagree in subtle ways, and that normalization choices interact with temperature during training. Length sensitivity can be seen as another axis of this geometric story: pooling more tokens changes the statistical properties of the resulting vector even before any similarity function is applied.

Some caveats are appropriate. The magnitude of the improvement from a length-invariant alternative will likely depend on the encoder family, the training objective, and the length distribution of the target workload. Encoders that already include length-aware normalization, or that are trained with hard negatives spanning diverse lengths, may exhibit smaller gaps. The proposed measure also needs to be compatible with approximate nearest neighbor indexes used at scale, which typically assume inner-product or cosine geometry; integration costs in vector databases such as FAISS, ScaNN, or pgvector are a practical consideration not fully resolved by a theoretical fix.

For practitioners building retrieval and RAG systems, the most actionable takeaway may be evaluation hygiene: explicitly stratify similarity benchmarks by sequence length, and check whether observed quality differences track content or simply length. If the paper's analysis generalizes, length-aware similarity could become a useful default rather than a niche adjustment, particularly as embedding models are pushed toward longer context windows where length variance grows.