numpy.linalg.lstsq#
Return the least-squares solution to a linear matrix equation.
Computes the vector x that approximately solves the equation a @ x = b . The equation may be under-, well-, or over-determined (i.e., the number of linearly independent rows of a can be less than, equal to, or greater than its number of linearly independent columns). If a is square and of full rank, then x (but for round-off error) is the “exact” solution of the equation. Else, x minimizes the Euclidean 2-norm \(||b — ax||\) . If there are multiple minimizing solutions, the one with the smallest 2-norm \(||x||\) is returned.
Parameters : a (M, N) array_like
b array_like
Ordinate or “dependent variable” values. If b is two-dimensional, the least-squares solution is calculated for each of the K columns of b.
rcond float, optional
Cut-off ratio for small singular values of a. For the purposes of rank determination, singular values are treated as zero if they are smaller than rcond times the largest singular value of a.
Changed in version 1.14.0: If not set, a FutureWarning is given. The previous default of -1 will use the machine precision as rcond parameter, the new default will use the machine precision times max(M, N). To silence the warning and use the new default, use rcond=None , to keep using the old behavior, use rcond=-1 .
Least-squares solution. If b is two-dimensional, the solutions are in the K columns of x.
residuals ndarray
Sums of squared residuals: Squared Euclidean 2-norm for each column in b — a @ x . If the rank of a is < N or M b is 1-dimensional, this is a (1,) shape array. Otherwise the shape is (K,).
s (min(M, N),) ndarray
If computation does not converge.
Similar function in SciPy.
If b is a matrix, then all array results are returned as matrices.
Fit a line, y = mx + c , through some noisy data-points:
>>> x = np.array([0, 1, 2, 3]) >>> y = np.array([-1, 0.2, 0.9, 2.1])
By examining the coefficients, we see that the line should have a gradient of roughly 1 and cut the y-axis at, more or less, -1.
We can rewrite the line equation as y = Ap , where A = [[x 1]] and p = [[m], [c]] . Now use lstsq to solve for p:
>>> A = np.vstack([x, np.ones(len(x))]).T >>> A array([[ 0., 1.], [ 1., 1.], [ 2., 1.], [ 3., 1.]])
>>> m, c = np.linalg.lstsq(A, y, rcond=None)[0] >>> m, c (1.0 -0.95) # may vary
Plot the data along with the fitted line:
>>> import matplotlib.pyplot as plt >>> _ = plt.plot(x, y, 'o', label='Original data', markersize=10) >>> _ = plt.plot(x, m*x + c, 'r', label='Fitted line') >>> _ = plt.legend() >>> plt.show()
numpy.linalg.lstsq#
Return the least-squares solution to a linear matrix equation.
Computes the vector x that approximately solves the equation a @ x = b . The equation may be under-, well-, or over-determined (i.e., the number of linearly independent rows of a can be less than, equal to, or greater than its number of linearly independent columns). If a is square and of full rank, then x (but for round-off error) is the “exact” solution of the equation. Else, x minimizes the Euclidean 2-norm \(||b — ax||\) . If there are multiple minimizing solutions, the one with the smallest 2-norm \(||x||\) is returned.
Parameters : a (M, N) array_like
b array_like
Ordinate or “dependent variable” values. If b is two-dimensional, the least-squares solution is calculated for each of the K columns of b.
rcond float, optional
Cut-off ratio for small singular values of a. For the purposes of rank determination, singular values are treated as zero if they are smaller than rcond times the largest singular value of a.
Changed in version 1.14.0: If not set, a FutureWarning is given. The previous default of -1 will use the machine precision as rcond parameter, the new default will use the machine precision times max(M, N). To silence the warning and use the new default, use rcond=None , to keep using the old behavior, use rcond=-1 .
Least-squares solution. If b is two-dimensional, the solutions are in the K columns of x.
residuals ndarray
Sums of squared residuals: Squared Euclidean 2-norm for each column in b — a @ x . If the rank of a is < N or M b is 1-dimensional, this is a (1,) shape array. Otherwise the shape is (K,).
s (min(M, N),) ndarray
If computation does not converge.
Similar function in SciPy.
If b is a matrix, then all array results are returned as matrices.
Fit a line, y = mx + c , through some noisy data-points:
>>> x = np.array([0, 1, 2, 3]) >>> y = np.array([-1, 0.2, 0.9, 2.1])
By examining the coefficients, we see that the line should have a gradient of roughly 1 and cut the y-axis at, more or less, -1.
We can rewrite the line equation as y = Ap , where A = [[x 1]] and p = [[m], [c]] . Now use lstsq to solve for p:
>>> A = np.vstack([x, np.ones(len(x))]).T >>> A array([[ 0., 1.], [ 1., 1.], [ 2., 1.], [ 3., 1.]])
>>> m, c = np.linalg.lstsq(A, y, rcond=None)[0] >>> m, c (1.0 -0.95) # may vary
Plot the data along with the fitted line:
>>> import matplotlib.pyplot as plt >>> _ = plt.plot(x, y, 'o', label='Original data', markersize=10) >>> _ = plt.plot(x, m*x + c, 'r', label='Fitted line') >>> _ = plt.legend() >>> plt.show()
NumPy Least Squares
Today, we are going to learn about the least squares in linear equations and how to implement the least square method for the best fit for the regression line in the provided datasets. But before that, let’s get the basic knowledge of NumPy. NumPy is one of the best mathematical packages of Python that provides services for multi-dimensional arrays and matrices along with a wide range of complex numerical operations that may be performed on these matrices/arrays.
One of the methods of Python lstsq() is used to find the regression line of the known linear equation ax=b that best fits with this equation. This means you have to determine the line which properly shows the relationship between the x and y points if your data indicates that there is one. The line between both points is known as a regression line when it is used to find the least square through this equation, ax=b.
Syntax:
Let’s start learning the implementing style of the linalg.lstsq() function. First, we write the library name that we use in Python which is “numpy”. Then, we concatenate the linalg() function and concatenate the lstsq() function. The linalg() function means linear algebra. It is always used with the lstsq() function because it is a linear algebraic expression. After this, we pass the arguments in the function brackets.
Parameters:
Let’s understand the parameters of the linalg.lstsq() function:
point1: It is the coefficient matrix.
point2: This matrix or array contains dependent variables.
rcond: The datatype of it is float. The rcond ratio serves as a cut-off for smaller singular values of point_1. If a singular value is less than the rcond times the biggest singular element of point_1, it is considered zero when determining the rank.
Return Value:
In return, we get the least square of known variable x in the equation ax=b.
Example 1:
Let’s start implementing our first example of a least square method of the Python library, NumPy. First, we need a Python compiler so that we can code in it. Open the compiler. You also need to install the NumPy library because we are using one of the functions of NumPy which is the lstsq() function. Then, you have to import the NumPy package in it. First, write the keyword “import” which tells the compiler that we are going to import the package. Then, we have to write the package name which we use in the function which is “numpy”. And then, we also write the alternate name of the NumPy “np” because many programmers use this approach. This is a good programming approach, and it saves time.
After importing the package, we start writing the actual line of code which we want to do. We print the messages first so that the user can easily understand what we are doing in the example using the print() statement. We create the one-dimensional array “A” using the array() function and then print it by calling the print() statement. Then, we create another one-dimensional array “B” using the array() function and print it using the print() function.
print ( «Implementation of Least Square Method in NumPy: » )
A = np. array ( [ 1 , 2 , 1 , 1 , 1 , 2 , 2 , 1 , 1 ] )
print ( » \n The Array A is: » , A )
B = np. array ( [ 4 , 3 , 5 , 4 , 2 , 3 , 6 , 3 , 2 ] )
print ( » \n The Array B is: » , B )
X = np. linalg . lstsq ( np. vstack ( [ A , np. ones ( len ( A ) ) ] ) . T , B , rcond = None ) [ 0 ]
print ( » \n The Least Square is: » , X )
After the creation of both points A and B, we implement the lstsq() function. But first, we use the vstack() function to stack the elements of “A”, sequence-wise. Then, we take the transpose of array “A”. Then, we pass the vstack() function as the first argument of the lstsq() function. The second argument is the “B” array and the third argument is “rcond” in which we set the value of rcond as “none”. Then, we store the whole function in another array named “x” which shows that it is the known variable linear equation, ax=b. After this, we display the results so we use the print() statement for this and pass the “x” array in it.
Example 2:
Now, let’s start implementing another example of NumPy least squares. We always import the library first which we use in the program which is NumPy. First, we write the keyword “import” to get the package in the program. We also write the package name which is “numpy” and then his alias, “np”. Then, we call the print() method so that we can display the retable message of the least squares for a better understanding of the user.
Then, we create the array name “x_axis” and store the array in it using the arange() function. Then, we print it using the print() method. Then, we create another array name “y_axis” and store the array in it which we created in the following illustration.
After creating both arrays, we implement the ones() method on the x_axis array and store it in another array named “array_a”. And then, we also print this array. We create another array named “arg_reg_line” and implement a linalg.lstsq() function on it. Then, we pass the parameters to this function so that we can get the least squares between two arrays or points. The first parameter is that we take the transpose of array_a. The second parameter is the second point which is the y_axis. Then, we have “rcond” which contains the “none” value. We then display the array using the print() method.
print ( «Implementation of linalg.lstsq() function: » )
print ( » \n The Value of x axis are: » , x_axis )
y_axis = [ 10.3 , 10.5 , 11 , 11.5 , 13.2 , 13.9 , 14 , 15.5 , 16.6 , 17 ]
print ( » \n The Value of y axis are: » , y_axis )
array_a = np. array ( [ x_axis , np. ones ( 10 ) ] )
print ( » \n The Array is: \n » , array_a )
arg_reg_line = np. linalg . lstsq ( array_a. T , y_axis , rcond = None ) [ 0 ]
print ( » \n The Parameters of Regrssion Line are: » , arg_reg_line )
reg_line = arg_reg_line [ 0 ] * x_axis + arg_reg_line [ 1 ]
import matplotlib. pyplot as plt
plt. plot ( x_axis , reg_line , ‘r-‘ )
plt. plot ( x_axis , y_axis , ‘o’ )
plt. title ( «Linear Regression Line» )
Here is the output of the previoulsy-implemented example:
We import another package of NumPy which is the “matplotlib” package which is used to plot the graph. Then, we plot the x_axis values and y_axis_values. Next, we set the title and labels of the graph. Lastly, we display the graph using the show() method.
Here is the desired graph of the given example:
Conclusion
In this article, we learned what is the least square and how we get the linalg.lstsq() of the unknown variable x using the linear equation ax=b. We used multiple functions of NumPy to find the least squares and implemented some examples with detailed explanations for a better understanding of the user.
About the author
Omar Farooq
Hello Readers, I am Omar and I have been writing technical articles from last decade. You can check out my writing pieces.