In mathematics, statistics, finance, computer science, particularly in machine learning and inverse problems, regularization is the process of adding information in order to solve an ill-posed problem or to prevent overfitting.Regularization applies to objective functions in ill-posed optimization problems. The regularization term, or penalty, imposes a cost on the optimization function for overfitting the function or to find an optimal solution. == Classification == Empirical learning of classifiers (from a finite data set) is always an underdetermined problem, because it attempts to inf...

Tikhonov regularization, named for Andrey Tikhonov, is a method of regularization of ill-posed problems. A special case of Tikhonov regularization, known as ridge regression, is particularly useful to mitigate the problem of multicollinearity in linear regression, which commonly occurs in models with large numbers of parameters. In general, the method provides improved efficiency in parameter estimation problems in exchange for a tolerable amount of bias (see bias–variance tradeoff).In the simplest case, the problem of a near-singular moment matrix (...