Check **Linear Regression** post here.

**1. Basis Function**

In mathematics, a basis function is an element of a particular basis for a function space. Every continuous function in the function space can be represented as a linear combination of basis functions, just as every vector in a vector space can be represented as a linear combination of basis vectors.

One trick you can use to adapt linear regression to nonlinear relationships between variables is to transform the data according to basis functions.

This is still a linear model, the linearity refers to the fact that the coefficients an
never multiply or divide each other.

What we have effectively done is taken
our one-dimensional x values and projected them into a higher dimension, so that a
linear fit can fit more complicated relationships between x and y

**Polynomial basis functions**

The

**PolynomialFeatures**transformer has converted one-dimensional array into a three dimensional (polynominal) array by taking the exponent of each value. The new higher dimensional data representation can then be plugged into a linear regression.import numpy as np import matplotlib.pyplot as plt from sklearn.linear_model import LinearRegression from sklearn.preprocessing import PolynomialFeatures x = np.array([1, 2, 3, 4, 5]) y = np.array([4, 2, 1, 3, 7]) plt.scatter(x, y); plt.show() X = x[:, np.newaxis] poly = PolynomialFeatures(degree=3, include_bias=False) X2 = poly.fit_transform(X) model = LinearRegression().fit(X2, y) yfit = model.predict(X2) plt.scatter(x, y) plt.plot(x, yfit); plt.show()

**Gaussian basis functions**

Other basis functions can be used. Instead of using a sum of polynomial try to use a sum of Gaussians.

import numpy as np import matplotlib.pyplot as plt from sklearn.linear_model import LinearRegression rng = np.random.RandomState(1) x = 10 * rng.rand(50) y = np.sin(x) + 0.1 * rng.randn(50) center_of_gauss = np.linspace(x.min(), x.max(), 20).astype(np.float) std = (center_of_gauss[1] - center_of_gauss[0])*0.7 def get_gauss(x): X = x[:, np.newaxis] gauss = np.exp(-0.5 * (((X-center_of_gauss)/1.41*std) ** 2)) return gauss model = LinearRegression().fit(get_gauss(x), y) xfit = np.linspace(0, 10, 1000) yfit = model.predict(get_gauss(xfit)) plt.scatter(x, y) plt.plot(xfit, yfit) plt.xlim(0, 10); plt.show()

## No comments:

## Post a Comment