Why does the dot product of multiple vectors measure similarity?

1. The Geometric Intuition: Alignment

At its core, the dot product measures angular coincidence. Given two vectors $\mathbf{u}$ and $\mathbf{v}$, the relationship is defined as:

$$\mathbf{u} \cdot \mathbf{v} = |\mathbf{u}| |\mathbf{v}| \cos(\theta)$$

• Positive Correlation: When $\theta < 90^\circ$, the vectors “agree” on their direction.
• Orthogonality (Zero): When $\theta = 90^\circ$, the vectors are independent. In feature space, this means knowing something about $\mathbf{u}$ tells you nothing about $\mathbf{v}$.
• Negative Correlation: When $\theta > 90^\circ$, the vectors actively oppose each other.

2. The Projection Perspective (Scalar Projection)

Think of the dot product as a “shadow” test. If we normalize $\mathbf{v}$ to a unit vector $\hat{\mathbf{v}}$, then $\mathbf{u} \cdot \hat{\mathbf{v}}$ is the scalar projection of $\mathbf{u}$ onto $\mathbf{v}$.

• It measures the magnitude of $\mathbf{u}$ that exists along the span of $\mathbf{v}$.
• In AI, if $\mathbf{v}$ represents a “concept” (like ‘Sentiment’ in an embedding), the dot product tells you how much of that concept is present in your input vector $\mathbf{u}$.

3. The Algebraic Perspective: Feature Matching

In an $n$-dimensional space (like Word2Vec or CLIP embeddings), the dot product is the sum of element-wise products:

$$\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^{n} u_i v_i$$

• Reinforcement: If both vectors have large positive values in the same dimension ($u_i, v_i > 0$), the product is large and positive, increasing similarity.
• Cancellation: If one is positive and the other is negative, they penalize the similarity score.
• Sparsity: If one vector has a 0 (no feature present), that dimension contributes nothing to the similarity, regardless of the other vector’s value.

4. Important Distinction: Magnitude vs. Angle

The raw dot product is sensitive to the magnitude (length) of the vectors.
* In Recommendation Systems, a high dot product might just mean a user interacts with a lot of content, not necessarily that their tastes are a perfect match.
* To measure “pure” similarity (direction only), we use Cosine Similarity, which is the dot product of the vectors after they have been scaled to a length of 1 (unit vectors).

$$\text{Cosine Similarity} = \frac{\mathbf{u} \cdot \mathbf{v}}{|\mathbf{u}| |\mathbf{v}|}$$

• This topic was modified 3 months ago by Wolf.
• This topic was modified 3 months ago by Wolf.
• This topic was modified 3 months ago by Wolf.