A Transformer neural network is a type of model introduced in the "Attention is All You Need" paper by Vaswani et al. (2017). Its core design principle is "attention", effectively weighting the significance of different inputs in a sequence. The Transformer consists of encoder and decoder parts. The encoder receives and processes the input sequentially, and the decoder transforms the encoded sequences into the final output. A key advantage of Transformers is parallelizability, which means they can process all tokens in the sequence at once, whereas models like RNNs need to process sequentially. This leads to faster training times. The most famous application of Transformer models is in natural language processing, such as OpenAI's Generative Pretrained Transformer (GPT) series and Google's Bidirectional Encoder Representations from Transformers (BERT) models. These models have broken multiple benchmarks and helped popularize Transformers across machine learning tasks.