Category: python Deep Learning

Python Deep Learning – Fundamentals ”; Previous Next In this chapter, we will look into the fundamentals of Python Deep Learning. Deep learning models/algorithms Let us now learn about the different deep learning models/ algorithms. Some of the popular models within deep learning are as follows − Convolutional neural networks Recurrent neural networks Deep belief networks Generative adversarial networks Auto-encoders and so on The inputs and outputs are represented as vectors or tensors. For example, a neural network may have the inputs where individual pixel RGB values in an image are represented as vectors. The layers of neurons that lie between the input layer and the output layer are called hidden layers. This is where most of the work happens when the neural net tries to solve problems. Taking a closer look at the hidden layers can reveal a lot about the features the network has learned to extract from the data. Different architectures of neural networks are formed by choosing which neurons to connect to the other neurons in the next layer. Pseudocode for calculating output Following is the pseudocode for calculating output of Forward-propagating Neural Network − # node[] := array of topologically sorted nodes # An edge from a to b means a is to the left of b # If the Neural Network has R inputs and S outputs, # then first R nodes are input nodes and last S nodes are output nodes. # incoming[x] := nodes connected to node x # weight[x] := weights of incoming edges to x For each neuron x, from left to right − if x <= R: do nothing # its an input node inputs[x] = [output[i] for i in incoming[x]] weighted_sum = dot_product(weights[x], inputs[x]) output[x] = Activation_function(weighted_sum) Print Page Previous Next Advertisements ”;

Python Deep Learning – Quick Guide ”; Previous Next Python Deep Learning – Introduction Deep structured learning or hierarchical learning or deep learning in short is part of the family of machine learning methods which are themselves a subset of the broader field of Artificial Intelligence. Deep learning is a class of machine learning algorithms that use several layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. Deep neural networks, deep belief networks and recurrent neural networks have been applied to fields such as computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, and bioinformatics where they produced results comparable to and in some cases better than human experts have. Deep Learning Algorithms and Networks − are based on the unsupervised learning of multiple levels of features or representations of the data. Higher-level features are derived from lower level features to form a hierarchical representation. use some form of gradient descent for training. Python Deep Learning – Environment In this chapter, we will learn about the environment set up for Python Deep Learning. We have to install the following software for making deep learning algorithms. Python 2.7+ Scipy with Numpy Matplotlib Theano Keras TensorFlow It is strongly recommend that Python, NumPy, SciPy, and Matplotlib are installed through the Anaconda distribution. It comes with all of those packages. We need to ensure that the different types of software are installed properly. Let us go to our command line program and type in the following command − $ python Python 3.6.3 |Anaconda custom (32-bit)| (default, Oct 13 2017, 14:21:34) [GCC 7.2.0] on linux Next, we can import the required libraries and print their versions − import numpy print numpy.__version__ Output 1.14.2 Installation of Theano, TensorFlow and Keras Before we begin with the installation of the packages − Theano, TensorFlow and Keras, we need to confirm if the pip is installed. The package management system in Anaconda is called the pip. To confirm the installation of pip, type the following in the command line − $ pip Once the installation of pip is confirmed, we can install TensorFlow and Keras by executing the following command − $pip install theano $pip install tensorflow $pip install keras Confirm the installation of Theano by executing the following line of code − $python –c “import theano: print (theano.__version__)” Output 1.0.1 Confirm the installation of Tensorflow by executing the following line of code − $python –c “import tensorflow: print tensorflow.__version__” Output 1.7.0 Confirm the installation of Keras by executing the following line of code − $python –c “import keras: print keras.__version__” Using TensorFlow backend Output 2.1.5 Python Deep Basic Machine Learning Artificial Intelligence (AI) is any code, algorithm or technique that enables a computer to mimic human cognitive behaviour or intelligence. Machine Learning (ML) is a subset of AI that uses statistical methods to enable machines to learn and improve with experience. Deep Learning is a subset of Machine Learning, which makes the computation of multi-layer neural networks feasible. Machine Learning is seen as shallow learning while Deep Learning is seen as hierarchical learning with abstraction. Machine learning deals with a wide range of concepts. The concepts are listed below − supervised unsupervised reinforcement learning linear regression cost functions overfitting under-fitting hyper-parameter, etc. In supervised learning, we learn to predict values from labelled data. One ML technique that helps here is classification, where target values are discrete values; for example,cats and dogs. Another technique in machine learning that could come of help is regression. Regression works onthe target values. The target values are continuous values; for example, the stock market data can be analysed using Regression. In unsupervised learning, we make inferences from the input data that is not labelled or structured. If we have a million medical records and we have to make sense of it, find the underlying structure, outliers or detect anomalies, we use clustering technique to divide data into broad clusters. Data sets are divided into training sets, testing sets, validation sets and so on. A breakthrough in 2012 brought the concept of Deep Learning into prominence. An algorithm classified 1 million images into 1000 categories successfully using 2 GPUs and latest technologies like Big Data. Relating Deep Learning and Traditional Machine Learning One of the major challenges encountered in traditional machine learning models is a process called feature extraction. The programmer needs to be specific and tell the computer the features to be looked out for. These features will help in making decisions. Entering raw data into the algorithm rarely works, so feature extraction is a critical part of the traditional machine learning workflow. This places a huge responsibility on the programmer, and the algorithm”s efficiency relies heavily on how inventive the programmer is. For complex problems such as object recognition or handwriting recognition, this is a huge issue. Deep learning, with the ability to learn multiple layers of representation, is one of the few methods that has help us with automatic feature extraction. The lower layers can be assumed to be performing automatic feature extraction, requiring little or no guidance from the programmer. Artificial Neural Networks The Artificial Neural Network, or just neural network for short, is not a new idea. It has been around for about 80 years. It was not until 2011, when Deep Neural Networks became popular with the use of new techniques, huge dataset availability, and powerful computers. A neural network mimics a neuron, which has dendrites, a nucleus, axon, and terminal axon. For a network, we need two neurons. These neurons transfer information via synapse between the dendrites of one and the terminal axon of another. A probable model of an artificial neuron looks like this − A neural network will look like as shown below − The circles are neurons or nodes, with their functions on the data and the lines/edges connecting them are the weights/information being passed along. Each column is a layer.

Computational Graphs ”; Previous Next Backpropagation is implemented in deep learning frameworks like Tensorflow, Torch, Theano, etc., by using computational graphs. More significantly, understanding back propagation on computational graphs combines several different algorithms and its variations such as backprop through time and backprop with shared weights. Once everything is converted into a computational graph, they are still the same algorithm − just back propagation on computational graphs. What is Computational Graph A computational graph is defined as a directed graph where the nodes correspond to mathematical operations. Computational graphs are a way of expressing and evaluating a mathematical expression. For example, here is a simple mathematical equation − $$p = x+y$$ We can draw a computational graph of the above equation as follows. The above computational graph has an addition node (node with “+” sign) with two input variables x and y and one output q. Let us take another example, slightly more complex. We have the following equation. $$g = left (x+y right ) ast z $$ The above equation is represented by the following computational graph. Computational Graphs and Backpropagation Computational graphs and backpropagation, both are important core concepts in deep learning for training neural networks. Forward Pass Forward pass is the procedure for evaluating the value of the mathematical expression represented by computational graphs. Doing forward pass means we are passing the value from variables in forward direction from the left (input) to the right where the output is. Let us consider an example by giving some value to all of the inputs. Suppose, the following values are given to all of the inputs. $$x=1, y=3, z=−3$$ By giving these values to the inputs, we can perform forward pass and get the following values for the outputs on each node. First, we use the value of x = 1 and y = 3, to get p = 4. Then we use p = 4 and z = -3 to get g = -12. We go from left to right, forwards. Objectives of Backward Pass In the backward pass, our intention is to compute the gradients for each input with respect to the final output. These gradients are essential for training the neural network using gradient descent. For example, we desire the following gradients. Desired gradients $$frac{partial x}{partial f}, frac{partial y}{partial f}, frac{partial z}{partial f}$$ Backward pass (backpropagation) We start the backward pass by finding the derivative of the final output with respect to the final output (itself!). Thus, it will result in the identity derivation and the value is equal to one. $$frac{partial g}{partial g} = 1$$ Our computational graph now looks as shown below − Next, we will do the backward pass through the “*” operation. We will calculate the gradients at p and z. Since g = p*z, we know that − $$frac{partial g}{partial z} = p$$ $$frac{partial g}{partial p} = z$$ We already know the values of z and p from the forward pass. Hence, we get − $$frac{partial g}{partial z} = p = 4$$ and $$frac{partial g}{partial p} = z = -3$$ We want to calculate the gradients at x and y − $$frac{partial g}{partial x}, frac{partial g}{partial y}$$ However, we want to do this efficiently (although x and g are only two hops away in this graph, imagine them being really far from each other). To calculate these values efficiently, we will use the chain rule of differentiation. From chain rule, we have − $$frac{partial g}{partial x}=frac{partial g}{partial p}ast frac{partial p}{partial x}$$ $$frac{partial g}{partial y}=frac{partial g}{partial p}ast frac{partial p}{partial y}$$ But we already know the dg/dp = -3, dp/dx and dp/dy are easy since p directly depends on x and y. We have − $$p=x+yRightarrow frac{partial x}{partial p} = 1, frac{partial y}{partial p} = 1$$ Hence, we get − $$frac{partial g} {partial f} = frac{partial g} {partial p}ast frac{partial p} {partial x} = left ( -3 right ).1 = -3$$ In addition, for the input y − $$frac{partial g} {partial y} = frac{partial g} {partial p}ast frac{partial p} {partial y} = left ( -3 right ).1 = -3$$ The main reason for doing this backwards is that when we had to calculate the gradient at x, we only used already computed values, and dq/dx (derivative of node output with respect to the same node”s input). We used local information to compute a global value. Steps for training a neural network Follow these steps to train a neural network − For data point x in dataset,we do forward pass with x as input, and calculate the cost c as output. We do backward pass starting at c, and calculate gradients for all nodes in the graph. This includes nodes that represent the neural network weights. We then update the weights by doing W = W – learning rate * gradients. We repeat this process until stop criteria is met. Print Page Previous Next Advertisements ”;

Python Deep Learning – Useful Resources ”; Previous Next The following resources contain additional information on Python Deep Learning. Please use them to get more in-depth knowledge on this. Useful Video Courses Computer Vision and Deep Learning in Python: Novice to Expert 107 Lectures 13.5 hours Abhilash Nelson More Detail Natural Language Processing with Deep Learning Course Most Popular 60 Lectures 2.5 hours Mike West More Detail Deep Learning And Neural Networks With Python By Spotle.ai Most Popular 24 Lectures 2.5 hours Spotle Learn More Detail Computer Vision Powered By Deep Learning, OpenCV And Python By Spotle.ai Best Seller 20 Lectures 2 hours Spotle Learn More Detail Performance Tuning Deep Learning in Python – A Masterclass 115 Lectures 4.5 hours Packt Publishing More Detail Deep Learning with Python for Image Classification 22 Lectures 1.5 hours Mazhar Hussain More Detail Print Page Previous Next Advertisements ”;

Discuss Python Deep Learning ”; Previous Next Python is a general-purpose high level programming language that is widely used in data science and for producing deep learning algorithms. This brief tutorial introduces Python and its libraries like Numpy, Scipy, Pandas, Matplotlib; frameworks like Theano, TensorFlow, Keras. The tutorial explains how the different libraries and frameworks can be applied to solve complex real world problems. Print Page Previous Next Advertisements ”;

Python Deep Learning – Implementations ”; Previous Next In this implementation of Deep learning, our objective is to predict the customer attrition or churning data for a certain bank – which customers are likely to leave this bank service. The Dataset used is relatively small and contains 10000 rows with 14 columns. We are using Anaconda distribution, and frameworks like Theano, TensorFlow and Keras. Keras is built on top of Tensorflow and Theano which function as its backends. # Artificial Neural Network # Installing Theano pip install –upgrade theano # Installing Tensorflow pip install –upgrade tensorflow # Installing Keras pip install –upgrade keras Step 1: Data preprocessing In[]: # Importing the libraries import numpy as np import matplotlib.pyplot as plt import pandas as pd # Importing the database dataset = pd.read_csv(”Churn_Modelling.csv”) Step 2 We create matrices of the features of dataset and the target variable, which is column 14, labeled as “Exited”. The initial look of data is as shown below − In[]: X = dataset.iloc[:, 3:13].values Y = dataset.iloc[:, 13].values X Output Step 3 Y Output array([1, 0, 1, …, 1, 1, 0], dtype = int64) Step 4 We make the analysis simpler by encoding string variables. We are using the ScikitLearn function ‘LabelEncoder’ to automatically encode the different labels in the columns with values between 0 to n_classes-1. from sklearn.preprocessing import LabelEncoder, OneHotEncoder labelencoder_X_1 = LabelEncoder() X[:,1] = labelencoder_X_1.fit_transform(X[:,1]) labelencoder_X_2 = LabelEncoder() X[:, 2] = labelencoder_X_2.fit_transform(X[:, 2]) X Output In the above output,country names are replaced by 0, 1 and 2; while male and female are replaced by 0 and 1. Step 5 Labelling Encoded Data We use the same ScikitLearn library and another function called the OneHotEncoder to just pass the column number creating a dummy variable. onehotencoder = OneHotEncoder(categorical features = [1]) X = onehotencoder.fit_transform(X).toarray() X = X[:, 1:] X Now, the first 2 columns represent the country and the 4th column represents the gender. Output We always divide our data into training and testing part; we train our model on training data and then we check the accuracy of a model on testing data which helps in evaluating the efficiency of model. Step 6 We are using ScikitLearn’s train_test_split function to split our data into training set and test set. We keep the train- to- test split ratio as 80:20. #Splitting the dataset into the Training set and the Test Set from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2) Some variables have values in thousands while some have values in tens or ones. We scale the data so that they are more representative. Step 7 In this code, we are fitting and transforming the training data using the StandardScaler function. We standardize our scaling so that we use the same fitted method to transform/scale test data. # Feature Scaling fromsklearn.preprocessing import StandardScaler sc = StandardScaler() X_train = sc.fit_transform(X_train) X_test = sc.transform(X_test) Output The data is now scaled properly. Finally, we are done with our data pre-processing. Now,we will start with our model. Step 8 We import the required Modules here. We need the Sequential module for initializing the neural network and the dense module to add the hidden layers. # Importing the Keras libraries and packages import keras from keras.models import Sequential from keras.layers import Dense Step 9 We will name the model as Classifier as our aim is to classify customer churn. Then we use the Sequential module for initialization. #Initializing Neural Network classifier = Sequential() Step 10 We add the hidden layers one by one using the dense function. In the code below, we will see many arguments. Our first parameter is output_dim. It is the number of nodes we add to this layer. init is the initialization of the Stochastic Gradient Decent. In a Neural Network we assign weights to each node. At initialization, weights should be near to zero and we randomly initialize weights using the uniform function. The input_dim parameter is needed only for first layer, as the model does not know the number of our input variables. Here the total number of input variables is 11. In the second layer, the model automatically knows the number of input variables from the first hidden layer. Execute the following line of code to addthe input layer and the first hidden layer − classifier.add(Dense(units = 6, kernel_initializer = ”uniform”, activation = ”relu”, input_dim = 11)) Execute the following line of code to add the second hidden layer − classifier.add(Dense(units = 6, kernel_initializer = ”uniform”, activation = ”relu”)) Execute the following line of code to add the output layer − classifier.add(Dense(units = 1, kernel_initializer = ”uniform”, activation = ”sigmoid”)) Step 11 Compiling the ANN We have added multiple layers to our classifier until now. We will now compile them using the compile method. Arguments added in final compilation control complete the neural network.So,we need to be careful in this step. Here is a brief explanation of the arguments. First argument is Optimizer.This is an algorithm used to find the optimal set of weights. This algorithm is called the Stochastic Gradient Descent (SGD). Here we are using one among several types, called the ‘Adam optimizer’. The SGD depends on loss, so our second parameter is loss. If our dependent variable is binary, we use logarithmic loss function called ‘binary_crossentropy’, and if our dependent variable has more than two categories in output, then we use ‘categorical_crossentropy’. We want to improve performance of our neural network based on accuracy, so we add metrics as accuracy. # Compiling Neural Network classifier.compile(optimizer = ”adam”, loss = ”binary_crossentropy”, metrics = [”accuracy”]) Step 12 A number of codes need to be executed in this step. Fitting the ANN to the Training Set We now train our model on the training data. We use the fit method to fit our model. We also optimize the weights to improve model efficiency. For this, we have to update the weights. Batch size is the number of observations after which we update the weights.

Python Deep Learning – Environment ”; Previous Next In this chapter, we will learn about the environment set up for Python Deep Learning. We have to install the following software for making deep learning algorithms. Python 2.7+ Scipy with Numpy Matplotlib Theano Keras TensorFlow It is strongly recommend that Python, NumPy, SciPy, and Matplotlib are installed through the Anaconda distribution. It comes with all of those packages. We need to ensure that the different types of software are installed properly. Let us go to our command line program and type in the following command − $ python Python 3.6.3 |Anaconda custom (32-bit)| (default, Oct 13 2017, 14:21:34) [GCC 7.2.0] on linux Next, we can import the required libraries and print their versions − import numpy print numpy.__version__ Output 1.14.2 Installation of Theano, TensorFlow and Keras Before we begin with the installation of the packages − Theano, TensorFlow and Keras, we need to confirm if the pip is installed. The package management system in Anaconda is called the pip. To confirm the installation of pip, type the following in the command line − $ pip Once the installation of pip is confirmed, we can install TensorFlow and Keras by executing the following command − $pip install theano $pip install tensorflow $pip install keras Confirm the installation of Theano by executing the following line of code − $python –c “import theano: print (theano.__version__)” Output 1.0.1 Confirm the installation of Tensorflow by executing the following line of code − $python –c “import tensorflow: print tensorflow.__version__” Output 1.7.0 Confirm the installation of Keras by executing the following line of code − $python –c “import keras: print keras.__version__” Using TensorFlow backend Output 2.1.5 Print Page Previous Next Advertisements ”;

Python Deep Learning – Applications ”; Previous Next Deep learning has produced good results for a few applications such as computer vision, language translation, image captioning, audio transcription, molecular biology, speech recognition, natural language processing, self-driving cars, brain tumour detection, real-time speech translation, music composition, automatic game playing and so on. Deep learning is the next big leap after machine learning with a more advanced implementation. Currently, it is heading towards becoming an industry standard bringing a strong promise of being a game changer when dealing with raw unstructured data. Deep learning is currently one of the best solution providers fora wide range of real-world problems. Developers are building AI programs that, instead of using previously given rules, learn from examples to solve complicated tasks. With deep learning being used by many data scientists, deeper neural networks are delivering results that are ever more accurate. The idea is to develop deep neural networks by increasing the number of training layers for each network; machine learns more about the data until it is as accurate as possible. Developers can use deep learning techniques to implement complex machine learning tasks, and train AI networks to have high levels of perceptual recognition. Deep learning finds its popularity in Computer vision. Here one of the tasks achieved is image classification where given input images are classified as cat, dog, etc. or as a class or label that best describe the image. We as humans learn how to do this task very early in our lives and have these skills of quickly recognizing patterns, generalizing from prior knowledge, and adapting to different image environments. Print Page Previous Next Advertisements ”;

Python Deep Learning Tutorial PDF Version Quick Guide Resources Job Search Discussion Python is a general-purpose high level programming language that is widely used in data science and for producing deep learning algorithms. This brief tutorial introduces Python and its libraries like Numpy, Scipy, Pandas, Matplotlib; frameworks like Theano, TensorFlow, Keras. The tutorial explains how the different libraries and frameworks can be applied to solve complex real world problems. Audience This tutorial has been prepared for professionals aspiring to learn the basics of Python and develop applications involving deep learning techniques such as convolutional neural nets, recurrent nets, back propagation, etc. Prerequisites Before you proceed with this tutorial, we assume that you have prior exposure to Python, Numpy, Pandas, Scipy, Matplotib, Windows, any Linux distribution, prior basic knowledge of Linear Algebra, Calculus, Statistics and basic machine learning techniques. Print Page Previous Next Advertisements ”;

Artificial Neural Networks ”; Previous Next The Artificial Neural Network, or just neural network for short, is not a new idea. It has been around for about 80 years. It was not until 2011, when Deep Neural Networks became popular with the use of new techniques, huge dataset availability, and powerful computers. A neural network mimics a neuron, which has dendrites, a nucleus, axon, and terminal axon. For a network, we need two neurons. These neurons transfer information via synapse between the dendrites of one and the terminal axon of another. A probable model of an artificial neuron looks like this − A neural network will look like as shown below − The circles are neurons or nodes, with their functions on the data and the lines/edges connecting them are the weights/information being passed along. Each column is a layer. The first layer of your data is the input layer. Then, all the layers between the input layer and the output layer are the hidden layers. If you have one or a few hidden layers, then you have a shallow neural network. If you have many hidden layers, then you have a deep neural network. In this model, you have input data, you weight it, and pass it through the function in the neuron that is called threshold function or activation function. Basically, it is the sum of all of the values after comparing it with a certain value. If you fire a signal, then the result is (1) out, or nothing is fired out, then (0). That is then weighted and passed along to the next neuron, and the same sort of function is run. We can have a sigmoid (s-shape) function as the activation function. As for the weights, they are just random to start, and they are unique per input into the node/neuron. In a typical “feed forward”, the most basic type of neural network, you have your information pass straight through the network you created, and you compare the output to what you hoped the output would have been using your sample data. From here, you need to adjust the weights to help you get your output to match your desired output. The act of sending data straight through a neural network is called a feed forward neural network. Our data goes from input, to the layers, in order, then to the output. When we go backwards and begin adjusting weights to minimize loss/cost, this is called back propagation. This is an optimization problem. With the neural network, in real practice, we have to deal with hundreds of thousands of variables, or millions, or more. The first solution was to use stochastic gradient descent as optimization method. Now, there are options like AdaGrad, Adam Optimizer and so on. Either way, this is a massive computational operation. That is why Neural Networks were mostly left on the shelf for over half a century. It was only very recently that we even had the power and architecture in our machines to even consider doing these operations, and the properly sized datasets to match. For simple classification tasks, the neural network is relatively close in performance to other simple algorithms like K Nearest Neighbors. The real utility of neural networks is realized when we have much larger data, and much more complex questions, both of which outperform other machine learning models. Print Page Previous Next Advertisements ”;