Paper List

AI

Introductory:

• A visual proof that neural nets can compute any function

Transformers:

• Attention is All You Need
• An Introduction to Transformers
• Formal Algorithms for Transfomers

GPTs:

• Let's build GPT: from scratch, in code, spelled out.
• Language Models are Unsupervised Multitask Learners (GPT-2)
• Language Models are Few-Shot Learners (GPT-3)

Mixture of Experts (MoE):

• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Augmented LLMs:

• Augmented Language Models: a Survey
• Retrieval-Augmented Generation for Large Language Models: A Survey
• ReAct: Synergizing Reasoning and Acting in Language Models

Test-Time Compute:

• Let's Verify Step by Step

Model Deployment:

• Distributed Training of Deep Learning Models: A Taxonomic Perspective

Embedding Models:

• Text and Code Embeddings by Contrastive Pre-Training

Fine-Tuning:

• LoRA: Low-Rank Adaptation of Large Language Models

Evaluation:

• RAGAS: Automated Evaluation of Retrieval Augmented Generation

Distributed Systems

• Spanner: Google’s Globally-Distributed Database
• MapReduce: Simplified Data Processing on Large Clusters
• ZooKeeper: Wait-free coordination for Internet-scale systems