Category: Machine Learning

Machine Learning – Multiple Linear Regression It is basically the extension of simple linear regression that predicts a response using two or more features. Mathematically we can explain it as follows − Consider a dataset having n observations, p features i.e. independent variables and y as one response i.e. dependent variable the regression line for p features can be calculated as follows − $$hleft ( x_{i} right )=b_{0}+b_{1}x_{i1}+b_{2}x_{i2}+cdot cdot cdot +b_{p}x_{ip}$$ Here,$hleft ( x_{i} right )$ is the predicted response value and $b_{0},b_{1},b_{2}….b_{p}$ are the regression coefficients. Multiple Linear Regression models always includes the errors in the data known as residual error which changes the calculation as follows − $$hleft ( x_{i} right )=b_{0}+b_{1}x_{i1}+b_{2}x_{i2}+cdot cdot cdot +b_{p}x_{ip}+e_{i}$$ We can also write the above equation as follows − $$y_{i}=hleft ( x_{i} right )+e_{i}:: or :: e_{i}=y_{i}-hleft ( x_{i} right )$$ Python Implementation To implement multiple linear regression in Python using Scikit-Learn, we can use the same LinearRegression class as in simple linear regression, but this time we need to provide multiple independent variables as input. Let”s consider the Boston Housing dataset from Scikit-Learn and implement multiple linear regression using it. Example from sklearn.datasets import load_boston from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error, r2_score import numpy as np import matplotlib.pyplot as plt # Load the Boston Housing dataset boston = load_boston() # Split the dataset into training and testing sets X_train, X_test, y_train, y_test = train_test_split(boston.data, boston.target, test_size=0.2, random_state=0) # Create a linear regression object lr_model = LinearRegression() # Fit the model on the training data lr_model.fit(X_train, y_train) # Make predictions on the test data y_pred = lr_model.predict(X_test) # Calculate the mean squared error mse = mean_squared_error(y_test, y_pred) # Calculate the coefficient of determination r2 = r2_score(y_test, y_pred) print(”Mean Squared Error:”, mse) print(”Coefficient of Determination:”, r2) # Plot the predicted values against the actual values plt.figure(figsize=(7.5, 3.5)) plt.scatter(y_test, y_pred) plt.xlabel(”Actual Values”) plt.ylabel(”Predicted Values”) # Add a regression line to the plot x = np.linspace(0, 50, 100) y = x plt.plot(x, y, color=”red”) # Show the plot plt.show() In this code, we first load the Boston Housing dataset using the load_boston() function from Scikit-Learn. We then split the dataset into training and testing sets using the train_test_split() function. Next, we create a LinearRegression object and fit it on the training data using the fit() method. We then make predictions on the test data using the predict() method and calculate the mean squared error and coefficient of determination using the mean_squared_error() and r2_score() functions, respectively. Finally, we plot the predicted values against the actual values using the scatter() function and add a regression line to the plot using the plot() function. We label the x-axis and y-axis using the xlabel() and ylabel() functions and display the plot using the show() function. Output When you execute the program, it will produce the following plot as the output and it will print the Mean Squared Error and the Coefficient of Determination on the terminal − Mean Squared Error: 33.44897999767653 Coefficient of Determination: 0.5892223849182507

Machine Learning – Simple Linear Regression Simple linear regression is a type of regression analysis in which a single independent variable (also known as a predictor variable) is used to predict the dependent variable. In other words, it models the linear relationship between the dependent variable and a single independent variable. Python Implementation Given below is an example that shows how to implement simple linear regression using the Pima-Indian-Diabetes dataset in Python. We will also plot the regression line. Data Preparation First, we need to import the Diabetes dataset from scikit-learn and split it into training and testing sets. We will use 80% of the data for training the model and the remaining 20% for testing. from sklearn.datasets import load_diabetes from sklearn.model_selection import train_test_split # Load the Diabetes dataset diabetes = load_diabetes() # Split the dataset into training and testing sets X_train, X_test, y_train, y_test = train_test_split(diabetes.data[:, 2], diabetes.target, test_size=0.2, random_state=0) # Reshape the input data X_train = X_train.reshape(-1, 1) X_test = X_test.reshape(-1, 1) Here, we are using the third feature (column) of the dataset, which represents the mean blood pressure, as our independent variable (predictor variable) and the target variable as our dependent variable (response variable). Model Training We will use scikit-learn”s LinearRegression class to train a simple linear regression model on the training data. The code for this is as follows − from sklearn.linear_model import LinearRegression # Create a linear regression object lr_model = LinearRegression() # Fit the model on the training data lr_model.fit(X_train, y_train) Here, X_train represents the input feature (mean blood pressure) of the training data and y_train represents the output variable (target variable). Model Testing Once the model is trained, we can use it to make predictions on the test data. The code for this is as follows − # Make predictions on the test data y_pred = lr_model.predict(X_test) Here, X_test represents the input feature of the test data and y_pred represents the predicted output variable (target variable). Model Evaluation We need to evaluate the performance of the model to determine its accuracy. We will use the mean squared error (MSE) and the coefficient of determination (R^2) as evaluation metrics. The code for this is as follows − from sklearn.metrics import mean_squared_error, r2_score # Calculate the mean squared error mse = mean_squared_error(y_test, y_pred) # Calculate the coefficient of determination r2 = r2_score(y_test, y_pred) print(”Mean Squared Error:”, mse) print(”Coefficient of Determination:”, r2) Here, y_test represents the actual output variable of the test data. Plotting the Regression Line We can also visualize the regression line to see how well it fits the data. The code for this is as follows − import matplotlib.pyplot as plt # Plot the training data plt.scatter(X_train, y_train, color=”gray”) # Plot the regression line plt.plot(X_train, lr_model.predict(X_train), color=”red”, linewidth=2) # Add axis labels plt.xlabel(”Mean Blood Pressure”) plt.ylabel(”Disease Progression”) # Show the plot plt.show() Here, we are using the scatter() function from the matplotlib library to plot the training data points and the plot() function to plot the regression line. The xlabel() and ylabel() functions are used to label the x-axis and y-axis of the plot, respectively. Finally, we use the show() function to display the plot. Complete Implementation Example The complete code for implementing simple linear regression in Python is as follows − from sklearn.datasets import load_diabetes from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error, r2_score import matplotlib.pyplot as plt # Load the Diabetes dataset diabetes = load_diabetes() # Split the dataset into training and testing sets X_train, X_test, y_train, y_test = train_test_split(diabetes.data[:, 2], diabetes.target, test_size=0.2, random_state=0) # Reshape the input data X_train = X_train.reshape(-1, 1) X_test = X_test.reshape(-1, 1) # Create a linear regression object lr_model = LinearRegression() # Fit the model on the training data lr_model.fit(X_train, y_train) # Make predictions on the test data y_pred = lr_model.predict(X_test) # Calculate the mean squared error mse = mean_squared_error(y_test, y_pred) # Calculate the coefficient of determination r2 = r2_score(y_test, y_pred) print(”Mean Squared Error:”, mse) print(”Coefficient of Determination:”, r2) # Plot the training data plt.figure(figsize=(7.5, 3.5)) plt.scatter(X_train, y_train, color=”gray”) # Plot the regression line plt.plot(X_train, lr_model.predict(X_train), color=”red”, linewidth=2) # Add axis labels plt.xlabel(”Mean Blood Pressure”) plt.ylabel(”Disease Progression”) # Show the plot plt.show() Output On executing this code, you will get the following plot as the output and it will also print the Mean Squared Error and the Coefficient of Determination on the terminal − Mean Squared Error: 4150.680189329983 Coefficient of Determination: 0.19057346847560164

Machine Learning – K-Nearest Neighbors (KNN) KNN is a supervised learning algorithm that can be used for both classification and regression problems. The main idea behind KNN is to find the k-nearest data points to a given test data point and use these nearest neighbors to make a prediction. The value of k is a hyperparameter that needs to be tuned, and it represents the number of neighbors to consider. For classification problems, the KNN algorithm assigns the test data point to the class that appears most frequently among the k-nearest neighbors. In other words, the class with the highest number of neighbors is the predicted class. For regression problems, the KNN algorithm assigns the test data point the average of the k-nearest neighbors” values. The distance metric used to measure the similarity between two data points is an essential factor that affects the KNN algorithm”s performance. The most commonly used distance metrics are Euclidean distance, Manhattan distance, and Minkowski distance. Working of KNN Algorithm The KNN algorithm can be summarized in the following steps − Load the data − The first step is to load the dataset into memory. This can be done using various libraries such as pandas or numpy. Split the data − The next step is to split the data into training and test sets. The training set is used to train the KNN algorithm, while the test set is used to evaluate its performance. Normalize the data − Before training the KNN algorithm, it is essential to normalize the data to ensure that each feature contributes equally to the distance metric calculation. Calculate distances − Once the data is normalized, the KNN algorithm calculates the distances between the test data point and each data point in the training set. Select k-nearest neighbors − The KNN algorithm selects the k-nearest neighbors based on the distances calculated in the previous step. Make a prediction − For classification problems, the KNN algorithm assigns the test data point to the class that appears most frequently among the k-nearest neighbors. For regression problems, the KNN algorithm assigns the test data point the average of the k-nearest neighbors” values. Evaluate performance − Finally, the KNN algorithm”s performance is evaluated using various metrics such as accuracy, precision, recall, and F1-score. Implementation in Python Now that we have discussed the KNN algorithm”s theory, let”s implement it in Python using scikit-learn. Scikit-learn is a popular library for Machine Learning in Python and provides various algorithms for classification and regression problems. We will use the Iris dataset, which is a popular dataset in Machine Learning and contains information about three different species of Iris flowers. The dataset has four features, including the sepal length, sepal width, petal length, and petal width, and a target variable, which is the species of the flower. To implement KNN in Python, we need to follow the steps mentioned earlier. Here”s the Python code for implementing KNN on the Iris dataset − Example # import libraries from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.neighbors import KNeighborsClassifier from sklearn.metrics import accuracy_score # load the Iris dataset iris = load_iris() #split the data into training and test sets X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.35, random_state=42) #normalize the data scaler = StandardScaler() X_train = scaler.fit_transform(X_train) X_test = scaler.transform(X_test) #initialize the KNN algorithm knn = KNeighborsClassifier(n_neighbors=5) #train the KNN algorithm knn.fit(X_train, y_train) #make predictions on the test set y_pred = knn.predict(X_test) #evaluate the performance of the KNN algorithm accuracy = accuracy_score(y_test, y_pred) print(“Accuracy: {:.2f}%”.format(accuracy*100)) Output When you execute this code, it will produce the following output − Accuracy: 98.11%

Machine Learning – Percentiles Percentiles are a statistical concept used in machine learning to describe the distribution of a dataset. A percentile is a measure that indicates the value below which a given percentage of observations in a group of observations falls. For example, the 25th percentile (also known as the first quartile) is the value below which 25% of the observations in the dataset fall, while the 75th percentile (also known as the third quartile) is the value below which 75% of the observations in the dataset fall. Percentiles can be used to summarize the distribution of a dataset and identify outliers. In machine learning, percentiles are often used in data preprocessing and exploratory data analysis to gain insights into the data. Python provides several libraries for calculating percentiles, including NumPy and Pandas. Calculating Percentiles using NumPy Below is an example of how to calculate percentiles using NumPy − Example import numpy as np data = np.array([1, 2, 3, 4, 5]) p25 = np.percentile(data, 25) p75 = np.percentile(data, 75) print(”25th percentile:”, p25) print(”75th percentile:”, p75) In this example, we create a sample dataset using NumPy and then calculate the 25th and 75th percentiles using the np.percentile() function. Output The output shows the values of the percentiles for the dataset. 25th percentile: 2.0 75th percentile: 4.0 Calculating Percentiles using Pandas Below is an example of how to calculate percentiles using Pandas − Example import pandas as pd data = pd.Series([1, 2, 3, 4, 5]) p25 = data.quantile(0.25) p75 = data.quantile(0.75) print(”25th percentile:”, p25) print(”75th percentile:”, p75) In this example, we create a Pandas series object and then calculate the 25th and 75th percentiles using the quantile() method of the series object. Output The output shows the values of the percentiles for the dataset. 25th percentile: 2.0 75th percentile: 4.0

Machine Learning – Bias and Variance Bias and variance are two important concepts in machine learning that describe the sources of error in a model”s predictions. Bias refers to the error that results from oversimplifying the underlying relationship between the input features and the output variable, while variance refers to the error that results from being too sensitive to fluctuations in the training data. In machine learning, we strive to minimize both bias and variance in order to build a model that can accurately predict on unseen data. A model with high bias may be too simplistic and underfit the training data, while a model with high variance may overfit the training data and fail to generalize to new data. Example Below is an implementation example in Python that illustrates how bias and variance can be analyzed using the Boston Housing dataset − import numpy as np import pandas as pd from sklearn.datasets import load_boston boston = load_boston() X = boston.data y = boston.target from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error lr = LinearRegression() lr.fit(X_train, y_train) train_preds = lr.predict(X_train) train_mse = mean_squared_error(y_train, train_preds) print(“Training MSE:”, train_mse) test_preds = lr.predict(X_test) test_mse = mean_squared_error(y_test, test_preds) print(“Testing MSE:”, test_mse) Output The output shows the training and testing mean squared errors (MSE) of the linear regression model. The training MSE is 21.64 and the testing MSE is 24.29, indicating that the model has a moderate level of bias and variance. Training MSE: 21.641412753226312 Testing MSE: 24.291119474973456 Reducing Bias and Variance To reduce bias, we can use more complex models that can capture non-linear relationships in the data. Example Let”s try a polynomial regression model − from sklearn.preprocessing import PolynomialFeatures poly = PolynomialFeatures(degree=2) X_train_poly = poly.fit_transform(X_train) X_test_poly = poly.transform(X_test) pr = LinearRegression() pr.fit(X_train_poly, y_train) train_preds = pr.predict(X_train_poly) train_mse = mean_squared_error(y_train, train_preds) print(“Training MSE:”, train_mse) test_preds = pr.predict(X_test_poly) test_mse = mean_squared_error(y_test, test_preds) print(“Testing MSE:”, test_mse) Output The output shows the training and testing MSE of the polynomial regression model with degree=2. The training MSE is 5.31 and the testing MSE is 14.18, indicating that the model has a lower bias but higher variance compared to the linear regression model. Training MSE: 5.31446956670908 Testing MSE: 14.183558207567042 Example To reduce variance, we can use regularization techniques such as ridge regression or lasso regression. In the following example, we will be using ridge regression − from sklearn.linear_model import Ridge ridge = Ridge(alpha=1) ridge.fit(X_train_poly, y_train) train_preds = ridge.predict(X_train_poly) train_mse = mean_squared_error(y_train, train_preds) print(“Training MSE:”, train_mse) test_preds = ridge.predict(X_test_poly) test_mse = mean_squared_error(y_test, test_preds) print(“Testing MSE:”, test_mse) Output The output shows the training and testing MSE of the ridge regression model with alpha=1. The training MSE is 9.03 and the testing MSE is 13.88 compared to the polynomial regression model, indicating that the model has a lower variance but slightly higher bias. Training MSE: 9.03220937860839 Testing MSE: 13.882093755326755 Example We can further tune the hyperparameter alpha to find the optimal balance between bias and variance. Let”s see an example − from sklearn.model_selection import GridSearchCV param_grid = {”alpha”: np.logspace(-3, 3, 7)} ridge_cv = GridSearchCV(Ridge(), param_grid, cv=5) ridge_cv.fit(X_train_poly, y_train) train_preds = ridge_cv.predict(X_train_poly) train_mse = mean_squared_error(y_train, train_preds) print(“Training MSE:”, train_mse) test_preds = ridge_cv.predict(X_test_poly) test_mse = mean_squared_error(y_test, test_preds) print(“Testing MSE:”, test_mse) Output The output shows the training and testing MSE of the ridge regression model with the optimal alpha value. Training MSE: 8.326082686584716 Testing MSE: 12.873907256619141 The training MSE is 8.32 and the testing MSE is 12.87, indicating that the model has a good balance between bias and variance.

Machine Learning – Regression Analysis Regression is a type of supervised learning algorithm in machine learning. The key objective of regression-based tasks is to predict output labels or responses which are continues numeric values, for the given input data. The output will be based on what the model has learned in training phase. Basically, regression models use the input data features (independent variables) and their corresponding continuous numeric output values (dependent or outcome variables) to learn specific association between inputs and corresponding outputs. Types of Regression Models Regression models are of following two types − Simple regression model − This is the most basic regression model in which predictions are formed from a single, univariate feature of the data. Multiple regression model − As name implies, in this regression model the predictions are formed from multiple features of the data. Building a Regressor in Python Regressor model in Python can be constructed just like we constructed the classifier. Scikit-learn, a Python library for machine learning can also be used to build a regressor in Python. In the following example, we will be building basic regression model that will fit a line to the data i.e. linear regressor. The necessary steps for building a regressor in Python are as follows − Step 1: Importing necessary python package For building a regressor using scikit-learn, we need to import it along with other necessary packages. We can import the by using following script − import numpy as np from sklearn import linear_model import sklearn.metrics as sm import matplotlib.pyplot as plt Step 2: Importing dataset After importing necessary package, we need a dataset to build regression prediction model. We can import it from sklearn dataset or can use other one as per our requirement. We are going to use our saved input data. We can import it with the help of following script − input = r”C:linear.txt” Next, we need to load this data. We are using np.loadtxt function to load it. input_data = np.loadtxt(input, delimiter=”,”) X, y = input_data[:, :-1], input_data[:, -1] Step 3: Organizing data into training & testing sets As we need to test our model on unseen data hence, we will divide our dataset into two parts: a training set and a test set. The following command will perform it − training_samples = int(0.6 * len(X)) testing_samples = len(X) – num_training X_train, y_train = X[:training_samples], y[:training_samples] X_test, y_test = X[training_samples:], y[training_samples:] Step 4: Model evaluation & prediction After dividing the data into training and testing we need to build the model. We will be using LineaRegression() function of Scikit-learn for this purpose. Following command will create a linear regressor object. reg_linear = linear_model.LinearRegression() Next, train this model with the training samples as follows − reg_linear.fit(X_train, y_train) Now, at last we need to do the prediction with the testing data. y_test_pred = reg_linear.predict(X_test) Step 5: Plot & visualization After prediction, we can plot and visualize it with the help of following script − plt.scatter(X_test, y_test, color = ”red”) plt.plot(X_test, y_test_pred, color = ”black”, linewidth = 2) plt.xticks(()) plt.yticks(()) plt.show() Output In the above output, we can see the regression line between the data points. Step 6: Performance computation We can also compute the performance of our regression model with the help of various performance metrics as follows. print(“Regressor model performance:”) print(“Mean absolute error(MAE) =”, round(sm.mean_absolute_error(y_test, y_test_pred), 2)) print(“Mean squared error(MSE) =”, round(sm.mean_squared_error(y_test, y_test_pred), 2)) print(“Median absolute error =”, round(sm.median_absolute_error(y_test, y_test_pred), 2)) print(“Explain variance score =”, round(sm.explained_variance_score(y_test, y_test_pred), 2)) print(“R2 score =”, round(sm.r2_score(y_test, y_test_pred), 2)) Output Regressor model performance: Mean absolute error(MAE) = 1.78 Mean squared error(MSE) = 3.89 Median absolute error = 2.01 Explain variance score = -0.09 R2 score = -0.09 Types of ML Regression Algorithms The most useful and popular ML regression algorithm is Linear regression algorithm which further divided into two types namely − Simple Linear Regression algorithm Multiple Linear Regression algorithm. We will discuss about it and implement it in Python in the next chapter. Applications The applications of ML regression algorithms are as follows − Forecasting or Predictive analysis − One of the important uses of regression is forecasting or predictive analysis. For example, we can forecast GDP, oil prices or in simple words the quantitative data that changes with the passage of time. Optimization − We can optimize business processes with the help of regression. For example, a store manager can create a statistical model to understand the peek time of coming of customers. Error correction − In business, taking correct decision is equally important as optimizing the business process. Regression can help us to take correct decision as well in correcting the already implemented decision. Economics − It is the most used tool in economics. We can use regression to predict supply, demand, consumption, inventory investment etc. Finance − A financial company is always interested in minimizing the risk portfolio and want to know the factors that affects the customers. All these can be predicted with the help of regression model.

Machine Learning – Scatter Matrix Plot Scatter Matrix Plot is a graphical representation of the relationship between multiple variables. It is a useful tool in machine learning for visualizing the correlation between features in a dataset. This plot is also known as a Pair Plot, and it is used to identify the correlation between two or more variables in a dataset. A Scatter Matrix Plot displays the scatter plot of each pair of features in a dataset. Each scatter plot represents the relationship between two variables. It is also possible to add a diagonal line to the plot that shows the distribution of each variable. Python Implementation of Scatter Matrix Plot Here, we will implement the Scatter Matrix Plot in Python. For our example given below, we will be using Sklearn”s Iris dataset. The Iris dataset is a classic dataset in machine learning. It contains four features: Sepal Length, Sepal Width, Petal Length, and Petal Width. The dataset has 150 samples, and each sample is labeled as one of three species: Setosa, Versicolor, or Virginica. We will use the Seaborn library to implement the Scatter Matrix Plot. Seaborn is a Python data visualization library that is built on top of the Matplotlib library. Example Below is the Python code to implement the Scatter Matrix Plot − import seaborn as sns import pandas as pd # load iris dataset iris = sns.load_dataset(”iris”) # create scatter matrix plot sns.pairplot(iris, hue=”species”) # show plot plt.show() In this code, we first import the necessary libraries, Seaborn and Pandas. Then, we load the Iris dataset using the sns.load_dataset() function. This function loads the Iris dataset from the Seaborn library. Next, we create the Scatter Matrix Plot using the sns.pairplot() function. The hue parameter is used to specify the column in the dataset that should be used for color encoding. In this case, we use the species column to color the points according to the species of each sample. Finally, we use the plt.show() function to display the plot. Output The output of this code will be a Scatter Matrix Plot that shows the scatter plots of each pair of features in the Iris dataset. Notice that each scatter plot is color-coded according to the species of each sample.

Machine Learning – Standard Deviation Standard deviation is a measure of the amount of variation or dispersion of a set of data values around their mean. In machine learning, it is an important statistical concept that is used to describe the spread or distribution of a dataset. Standard deviation is calculated as the square root of the variance, which is the average of the squared differences from the mean. The formula for calculating standard deviation is as follows − $$sigma =sqrt{left [Sigma left ( x-mu right )^{2}/N right ]}$$ Where − $sigma$is the standard deviation $Sigma$ is the sum of $x$ is the data point $mu$ is the mean of the dataset $N$ is the total number of data points In machine learning, standard deviation is used to understand the variability of a dataset and to detect outliers. For example, in finance, standard deviation is used to measure the volatility of stock prices. In image processing, standard deviation can be used to detect image noise. Types of Examples Example 1 In this example, we will be using the NumPy library to calculate the standard deviation − import numpy as np data = np.array([1, 2, 3, 4, 5, 6]) std_dev = np.std(data) print(”Standard deviation:”, std_dev) Output It will produce the following output − Standard deviation: 1.707825127659933 Example 2 Let”s see another example in which we will calculate the standard deviation of each column in Iris flower dataset using Python and Pandas library − import pandas as pd # load the iris dataset iris_df = pd.read_csv(”https://archive.ics.uci.edu/ml/machine-learningdatabases/iris/iris.data”, names=[”sepal length”, ”sepal width”, ”petal length”, ”petal width”, ”class”]) # calculate the standard deviation of each column std_devs = iris_df.std() # print the standard deviations print(”Standard deviations:”) print(std_devs) In this example, we load the Iris dataset from the UCI Machine Learning Repository using Pandas” read_csv() method. We then calculate the standard deviation of each column using the std() method of the Pandas dataframe. Finally, we print the standard deviations for each column. Output On executing the code, you will get the following output − Standard deviations: sepal length 0.828066 sepal width 0.433594 petal length 1.764420 petal width 0.763161 dtype: float64 This example demonstrates how standard deviation can be used to understand the variability of a dataset. In this case, we can see that the standard deviation of the ”petal length” column is much higher than that of the other columns, which suggests that this feature may be more variable and potentially more informative for classification tasks.

Machine Learning – Linear Regression Linear regression may be defined as the statistical model that analyzes the linear relationship between a dependent variable with given set of independent variables. Linear relationship between variables means that when the value of one or more independent variables will change (increase or decrease), the value of dependent variable will also change accordingly (increase or decrease). Mathematically the relationship can be represented with the help of following equation − $$Y=mX+b$$ Here, Y is the dependent variable we are trying to predict X is the dependent variable we are using to make predictions m is the slop of the regression line which represents the effect X has on Y. b is a constant, known as the Y-intercept. If X = 0, Y would be equal to b. Furthermore, the linear relationship can be positive or negative in nature as explained below − Positive Linear Relationship A linear relationship will be called positive if both independent and dependent variable increases. It can be understood with the help of following graph − Negative Linear Relationship A linear relationship will be called positive if independent increases and dependent variable decreases. It can be understood with the help of following graph − Linear regression is of two types, “simple linear regression” and “multiple linear regression”, which we are going to discuss in the next two chapters of this tutorial. Types of Linear Regression Linear regression is of the following two types − Assumptions The following are some assumptions about dataset that is made by Linear Regression model − Multi-collinearity − Linear regression model assumes that there is very little or no multi-collinearity in the data. Basically, multi-collinearity occurs when the independent variables or features have dependency in them. Auto-correlation − Another assumption Linear regression model assumes is that there is very little or no auto-correlation in the data. Basically, auto-correlation occurs when there is dependency between residual errors. Relationship between variables − Linear regression model assumes that the relationship between response and feature variables must be linear.

Machine Learning – Skewness and Kurtosis Skewness and kurtosis are two important measures of the shape of a probability distribution in machine learning. Skewness refers to the degree of asymmetry of a distribution. A distribution is said to be skewed if it is not symmetrical about its mean. Skewness can be positive, indicating that the tail of the distribution is longer on the right-hand side, or negative, indicating that the tail of the distribution is longer on the left-hand side. A skewness of zero indicates that the distribution is perfectly symmetrical. Kurtosis refers to the degree of peakedness of a distribution. A distribution with high kurtosis has a sharper peak and heavier tails than a normal distribution, while a distribution with low kurtosis has a flatter peak and lighter tails. Kurtosis can be positive, indicating a higher-than-normal peak, or negative, indicating a lower than normal peak. A kurtosis of zero indicates a normal distribution. Both skewness and kurtosis can have important implications for machine learning algorithms, as they can affect the assumptions of the models and the accuracy of the predictions. For example, a highly skewed distribution may require data transformation or the use of non-parametric methods, while a highly kurtotic distribution may require different statistical models or more robust estimation methods. Example In Python, the SciPy library provides functions for calculating skewness and kurtosis of a dataset. For example, the following code calculates the skewness and kurtosis of a dataset using the skew() and kurtosis() functions − import numpy as np from scipy.stats import skew, kurtosis # Generate a random dataset data = np.random.normal(0, 1, 1000) # Calculate the skewness and kurtosis of the dataset skewness = skew(data) kurtosis = kurtosis(data) # Print the results print(”Skewness:”, skewness) print(”Kurtosis:”, kurtosis) This code generates a random dataset of 1000 samples from a normal distribution with mean 0 and standard deviation 1. It then calculates the skewness and kurtosis of the dataset using the skew() and kurtosis() functions from the SciPy library. Finally, it prints the results to the console. Output On executing this code, you will get the following output − Skewness: -0.04119418903611285 Kurtosis: -0.1152250196054534 The resulting skewness and kurtosis values should be close to zero for a normal distribution.