[Article] Shared functional specialization in transformer-based language models and the human brain.
Summary: Transformers are recently being compared to the brain. Usually, the internal representations (“embeddings”) are adopted for comparisons. However, the authors focused on “transformations” that integrate contextual information across words, and found that they are more layer-specific than the embeddings. It differs from existing research in that it focuses on transformations related to attention instead of embeddings, which has been one of our recent interests.