1) Vectors can be expressed in various bases. This is obvious if you draw an arrow on a piece of paper and draw two different coordinate axes.
2) It's useful to talk about linear transformations of vectors.
3) Those linear transformations can be expressed as matrices, but doing this requires a basis choice.
4) Changing a matrix/vector from one representation to another involves a transformation (which is unitary), and that transformation has the same matrix in the two bases it connects.
5) Integration and derivatives are linear operations. The exponentials diagonalize derivatives.