Connecting Attention in Transformers to Linear Regression

The attention mechanism actually performs similar computations to linear regression - probably a loose connection but worth noting down.

From the EM Algorithm to Predictive Coding

Notes about my understanding of predictive coding as a special case of the Expectation-Maximization algorithm

If you want to have a chat, contact me through the following ways:

• mufeng.tang@ndcn.ox.ac.uk
• (44) 07497865905
• MRC Brain Network Dynamics Unit
  University of Oxford, Mansfield Road, Oxford OX1 3TH