In this article, we’ll learn to implement Linear regression from scratch using Python. Linear regression is a basic and most commonly used type of predictive analysis. It is used to predict the value of a variable based on the value of another variable. The variable we want to predict is called the dependent variable. The variable we are
using to predict the dependent variable’s value is called the independent variable. The simplest form of the regression equation with one dependent and one independent variable. y = m * x + b where, In this article, we will implement the Linear Regression from scratch using only
Numpy. While there are many loss functions to implement, We will use the Mean Squared Error function as our loss function. A mean squared error function as the name suggests is the mean of squared sum of difference between true and predicted value. As the predicted value of y depends on the slope and constant, hence our goal is to find the values for slope and constant that minimize the loss function or in other words, minimize the difference between y predicted and true values. 2. Optimization AlgorithmOptimization algorithms are used to find the optimal set of parameters given a training dataset that minimizes the loss function, in our case we need to find the optimal value of slope (m) and constant (b). One such Algorithm is Gradient Descent. Gradient descent is by far the most popular optimization algorithm used in machine learning. Using gradient descent we iteratively calculate the gradients of the loss function with respect to the parameters and keep on updating the parameters till we reach the local minima. 3. Steps to Implement Gradient DescentLet’s understand how the gradient descent algorithm works behind the scenes. Step-1 Initializing the parameters Here, we need to initialize the values for our parameters.
Let’s keep We will also need a learning rate to determine the step size at each iteration while moving toward a minimum value of our loss function. Step -2 Calculate the Partial Derivatives with respect to parameters Here we partially differentiate our loss function with respect to the parameters we have. Gradient Of Slope And BiasStep – 3 Updating the parameters Now, we update the values of our parameters using the equations given below: The updated values for our parameters will be the values with which, each step minimizes our loss function and reduces the difference between the true and predicted values. Repeat the process to reach a point of local minima. 4. Implementing Linear Regression from Scratch in PythonNow that we have an idea about how Linear regression can be implemented using Gradient descent, let’s code it in Python. We will
define #Import required modules import numpy as np #Defining the class class LinearRegression: def __init__(self, x , y): self.data = x self.label = y self.m = 0 self.b = 0 self.n = len(x) def fit(self , epochs , lr): #Implementing Gradient Descent for i in range(epochs): y_pred = self.m * self.data + self.b #Calculating derivatives w.r.t Parameters D_m = (-2/self.n)*sum(self.data * (self.label - y_pred)) D_b = (-1/self.n)*sum(self.label-y_pred) #Updating Parameters self.m = self.m - lr * D_m self.c = self.b - lr * D_c def predict(self , inp): y_pred = self.m * inp + self.b return y_pred We create an instance of our The With the 4. Testing the Linear Regression ModelNow as we created our class let’s test in on the data. Learn more about how to split training and testing data sets. You can find the datasets and other resources used within this tutorial here. #importing Matplotlib for plotting import matplotlib.pyplot as plt #Loding the data df = pd.read_csv('data_LinearRegression.csv') #Preparing the data x = np.array(df.iloc[:,0]) y = np.array(df.iloc[:,1]) #Creating the class object regressor = LinearRegression(x,y) #Training the model with .fit method regressor.fit(1000 , 0.0001) # epochs-1000 , learning_rate - 0.0001 #Prediciting the values y_pred = regressor.predict(x) #Plotting the results plt.figure(figsize = (10,6)) plt.scatter(x,y , color = 'green') plt.plot(x , y_pred , color = 'k' , lw = 3) plt.xlabel('x' , size = 20) plt.ylabel('y', size = 20) plt.show()Prediction Linear Regression Works fine ! ConclusionThis article was all about how we can make a Linear Regression model from scratch using only Numpy. The goal of this tutorial was to give you a deeper sense of what Linear Regression actually is and how it works. Till we meet next time. Happy Learning! How do you implement gradient descent from scratch Python?To implement a gradient descent algorithm we need to follow 4 steps:. Randomly initialize the bias and the weight theta.. Calculate predicted value of y that is Y given the bias and the weight.. Calculate the cost function from predicted and actual values of Y.. Calculate gradient and the weights.. How do you implement gradient descent in linear regression in Python?Gradient Descent Overview. Downloading the dataset.. Installing the requirements.. Theory needed for coding gradient descent.. Visualising the dataset.. Preparing the data.. Implementing the gradient descent function.. Visualising the prediction line.. How do you implement a linear regression in python from scratch?Simple Linear Regression Using Python. import numpy as np. import pandas as pd. import matplotlib.pyplot as plt. ... . data = pd.read_csv('Salary_Data.csv') x = data['YearsExperience'] y = data['Salary'] ... . print(data.head()) YearsExperience Salary. 0 1.1 39343.0.. How do you calculate gradient descent in linear regression?Gradient Descent Algorithm
The algorithm starts with some value of m and c (usually starts with m=0, c=0). We calculate MSE (cost) at point m=0, c=0. Let say the MSE (cost) at m=0, c=0 is 100. Then we reduce the value of m and c by some amount (Learning Step).
|