Imagine an m x n matrix A, as a transformation transforming a n-dimensional hypercube into an m-dimensional parallelepiped. (Or imagine spheres/hyperellipsoids e.t.c). A = U*S*V^t is the decomposition. U and V are orthogonal matrices (U*U^t=I_m, V*V^t = I_n) and S has the same size as A, but S is zero except for the diagonal.
So the values on the diagonal (the *singular values*) are all nonnegative and ordered from the largest to the smallest (by definition). They are almost like eigenvalues but in the direction of the vectors in U/V respectively. The number of nonzero singular values is o…