Theoretical Analysis of Positional Encodings in Transformer Models arxiv.org 33 points by PaulHoule 15 hours ago
semiinfinitely 11 hours ago Kinda disappointing that rope- the most common pe- is given about one sentence in this work and omitted from the analysis. gsf_emergency_2 9 hours ago Maybe it's because ropes by themselves do nothing for the model capacity?
gsf_emergency_2 9 hours ago Maybe it's because ropes by themselves do nothing for the model capacity?
Kinda disappointing that rope- the most common pe- is given about one sentence in this work and omitted from the analysis.
Maybe it's because ropes by themselves do nothing for the model capacity?