Mixer

1. Motivation

In this study, we demonstrate that while convolutions and attention can achieve satisfactory performance levels, neither convolution nor attention is necessary.

2. Contribution

• We introduce the MLP-Mixer architecture (referred to as “Mixer” for brevity), an innovative yet straightforward in both conceptual framework and technical implementation, designed exclusively to sidestep the use of convolutions or self-attention mechanisms.
• The Mixer's arch