How to build an objective function in machine learning

Machine Learning and Artificial Intelligence can be generally described as using a combination of data, mathematics and software to get computers to make the best possible guesses (hence often also decisions) of what (“y”) will happen if “x” happens – very often better than humans do: this is done by effectively finding a function H which matches best an input vector (X) with an observation vector (Y), for a given collection of (X,Y) combinations:

Y = H(X)

For example, one can forecast the sales of ice creams (Y) for a given temperature (X).
The goal is to find the
most (appropriately) simple H which predicts Y using X as inputs for a given prediction accuracy, on a given data set.
The performance of H in matching X with Y is called the “Objective function”.

Usually, the quality of H is measured by two terms: the matching error (L) and the regularization (or “complexity control”) term (W):

Obj(H) = L(H) + W(H)

L is an estimate of the fitting error over the available data sets, and W is a measure of the complexity of H. For both terms, the lower the better.

Machine learning, in this context, consists in minimizing the objective function Obj(H) as the best potential compromise between fitting accuracy on past data sets and complexity.

This turns out – mathematically provable under general conditions, and empirically observed – to be the best way to predict the Y for future X’s.

avoid over fitting objective function machine learning

Other articles published on Basedig

Data Collection Services

Basedig provides a platform for data collection services from multiple data sources

DIY Market statistics in France

Statistics about the French DIY and construction market: main products sold, product prices and product sales volumes by product EAN, product type, distributors and brands.

Optimisation des stocks

Optimisation des stocks par le big data, l'intelligence artificielle et le machine learning