Multi-Head Attention Subspace Specialization
Concept 02 Demo
Head 1: Hardware Modifiers
Head 2: Verb & Actions
Head 3: Spatial Relations
Head 4: Object Attributes
Active Head Specialization
Hardware & Component Modifiers
Subspace Dimension d_k
64 dims (4 × 64 = 256)
Key Attended Token Pair
"robot" ➔ "swerve" (88%)