I think the point is that otherwise for p,q distinct, the tangent spaces TpM and TqM on their own have no relation whatsoever. That shouldn't be the case though, there is some kind of smooth dependence (motivated by considering vector fields in R^n, for example). TM provides a way of giving this 'bundle' of spaces a smooth structure, which allows you to define a notion of smoothness for example.
