Facebook AI researchers are sharing an all-attention layer to simplify the Transformer model and an adaptive attention span method to make it more efficient. Even with a much simpler architecture, these methods match or improve state-of-the-art results. https://ai.facebook.com/blog/making-transformer-networks-simpler-and-more-efficient/ | AI at Meta