1. A lot of problems can be expressed without a particular choice of basis. This is because integration and derivatives are linear operators.
2. E&M, linear mechanics problems (like the oscillator) and quantum are examples.
3. Matrices are representations of linear operators in particular bases.
4. Change of basis.
5. Greens functions are really simple if you think of things in this way.