Category: keras

Keras Tutorial Job Search Keras is an open source deep learning framework for python. It has been developed by an artificial intelligence researcher at Google named Francois Chollet. Leading organizations like Google, Square, Netflix, Huawei and Uber are currently using Keras. This tutorial walks through the installation of Keras, basics of deep learning, Keras models, Keras layers, Keras modules and finally conclude with some real-time applications. Audience This tutorial is prepared for professionals who are aspiring to make a career in the field of deep learning and neural network framework. This tutorial is intended to make you comfortable in getting started with the Keras framework concepts. Prerequisites Before proceeding with the various types ofs concepts given in this tutorial, we assume that the readers have basic understanding of deep learning framework. In addition to this, it will be very helpful, if the readers have a sound knowledge of Python and Machine Learning.

Keras – Introduction Deep learning is one of the major subfield of machine learning framework. Machine learning is the study of design of algorithms, inspired from the model of human brain. Deep learning is becoming more popular in data science fields like robotics, artificial intelligence(AI), audio & video recognition and image recognition. Artificial neural network is the core of deep learning methodologies. Deep learning is supported by various libraries such as Theano, TensorFlow, Caffe, Mxnet etc., Keras is one of the most powerful and easy to use python library, which is built on top of popular deep learning libraries like TensorFlow, Theano, etc., for creating deep learning models. Overview of Keras Keras runs on top of open source machine libraries like TensorFlow, Theano or Cognitive Toolkit (CNTK). Theano is a python library used for fast numerical computation tasks. TensorFlow is the most famous symbolic math library used for creating neural networks and deep learning models. TensorFlow is very flexible and the primary benefit is distributed computing. CNTK is deep learning framework developed by Microsoft. It uses libraries such as Python, C#, C++ or standalone machine learning toolkits. Theano and TensorFlow are very powerful libraries but difficult to understand for creating neural networks. Keras is based on minimal structure that provides a clean and easy way to create deep learning models based on TensorFlow or Theano. Keras is designed to quickly define deep learning models. Well, Keras is an optimal choice for deep learning applications. Features Keras leverages various optimization techniques to make high level neural network API easier and more performant. It supports the following features − Consistent, simple and extensible API. Minimal structure – easy to achieve the result without any frills. It supports multiple platforms and backends. It is user friendly framework which runs on both CPU and GPU. Highly scalability of computation. Benefits Keras is highly powerful and dynamic framework and comes up with the following advantages − Larger community support. Easy to test. Keras neural networks are written in Python which makes things simpler. Keras supports both convolution and recurrent networks. Deep learning models are discrete components, so that, you can combine into many ways.

Keras – Installation This chapter explains about how to install Keras on your machine. Before moving to installation, let us go through the basic requirements of Keras. Prerequisites You must satisfy the following requirements − Any kind of OS (Windows, Linux or Mac) Python version 3.5 or higher. Python Keras is python based neural network library so python must be installed on your machine. If python is properly installed on your machine, then open your terminal and type python, you could see the response similar as specified below, Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)] on win32 Type “help”, “copyright”, “credits” or “license” for more information. >>> As of now the latest version is ‘3.7.2’. If Python is not installed, then visit the official python link – and download the latest version based on your OS and install it immediately on your system. Keras Installation Steps Keras installation is quite easy. Follow below steps to properly install Keras on your system. Step 1: Create virtual environment Virtualenv is used to manage Python packages for different projects. This will be helpful to avoid breaking the packages installed in the other environments. So, it is always recommended to use a virtual environment while developing Python applications. Linux/Mac OS Linux or mac OS users, go to your project root directory and type the below command to create virtual environment, python3 -m venv kerasenv After executing the above command, “kerasenv” directory is created with bin,lib and include folders in your installation location. Windows Windows user can use the below command, py -m venv keras Step 2: Activate the environment This step will configure python and pip executables in your shell path. Linux/Mac OS Now we have created a virtual environment named “kerasvenv”. Move to the folder and type the below command, $ cd kerasvenv kerasvenv $ source bin/activate Windows Windows users move inside the “kerasenv” folder and type the below command, .envScriptsactivate Step 3: Python libraries Keras depends on the following python libraries. Numpy Pandas Scikit-learn Matplotlib Scipy Seaborn Hopefully, you have installed all the above libraries on your system. If these libraries are not installed, then use the below command to install one by one. numpy pip install numpy you could see the following response, Collecting numpy Downloading https://files.pythonhosted.org/packages/cf/a4/d5387a74204542a60ad1baa84cd2d3353c330e59be8cf2d47c0b11d3cde8/ numpy-3.1.1-cp36-cp36m-macosx_10_6_intel. macosx_10_9_intel.macosx_10_9_x86_64. macosx_10_10_intel.macosx_10_10_x86_64.whl (14.4MB) |████████████████████████████████| 14.4MB 2.8MB/s pandas pip install pandas We could see the following response, Collecting pandas Downloading https://files.pythonhosted.org/packages/cf/a4/d5387a74204542a60ad1baa84cd2d3353c330e59be8cf2d47c0b11d3cde8/ pandas-3.1.1-cp36-cp36m-macosx_10_6_intel. macosx_10_9_intel.macosx_10_9_x86_64. macosx_10_10_intel.macosx_10_10_x86_64.whl (14.4MB) |████████████████████████████████| 14.4MB 2.8MB/s matplotlib pip install matplotlib We could see the following response, Collecting matplotlib Downloading https://files.pythonhosted.org/packages/cf/a4/d5387a74204542a60ad1baa84cd2d3353c330e59be8cf2d47c0b11d3cde8/ matplotlib-3.1.1-cp36-cp36m-macosx_10_6_intel. macosx_10_9_intel.macosx_10_9_x86_64. macosx_10_10_intel.macosx_10_10_x86_64.whl (14.4MB) |████████████████████████████████| 14.4MB 2.8MB/s scipy pip install scipy We could see the following response, Collecting scipy Downloading https://files.pythonhosted.org/packages/cf/a4/d5387a74204542a60ad1baa84cd2d3353c330e59be8cf2d47c0b11d3cde8 /scipy-3.1.1-cp36-cp36m-macosx_10_6_intel. macosx_10_9_intel.macosx_10_9_x86_64. macosx_10_10_intel.macosx_10_10_x86_64.whl (14.4MB) |████████████████████████████████| 14.4MB 2.8MB/s scikit-learn It is an open source machine learning library. It is used for classification, regression and clustering algorithms. Before moving to the installation, it requires the following − Python version 3.5 or higher NumPy version 1.11.0 or higher SciPy version 0.17.0 or higher joblib 0.11 or higher. Now, we install scikit-learn using the below command − pip install -U scikit-learn Seaborn Seaborn is an amazing library that allows you to easily visualize your data. Use the below command to install − pip install seaborn You could see the message similar as specified below − Collecting seaborn Downloading https://files.pythonhosted.org/packages/a8/76/220ba4420459d9c4c9c9587c6ce607bf56c25b3d3d2de62056efe482dadc /seaborn-0.9.0-py3-none-any.whl (208kB) 100% |████████████████████████████████| 215kB 4.0MB/s Requirement already satisfied: numpy> = 1.9.3 in ./lib/python3.7/site-packages (from seaborn) (1.17.0) Collecting pandas> = 0.15.2 (from seaborn) Downloading https://files.pythonhosted.org/packages/39/b7/441375a152f3f9929ff8bc2915218ff1a063a59d7137ae0546db616749f9/ pandas-0.25.0-cp37-cp37m-macosx_10_9_x86_64. macosx_10_10_x86_64.whl (10.1MB) 100% |████████████████████████████████| 10.1MB 1.8MB/s Requirement already satisfied: scipy>=0.14.0 in ./lib/python3.7/site-packages (from seaborn) (1.3.0) Collecting matplotlib> = 1.4.3 (from seaborn) Downloading https://files.pythonhosted.org/packages/c3/8b/af9e0984f 5c0df06d3fab0bf396eb09cbf05f8452de4e9502b182f59c33b/ matplotlib-3.1.1-cp37-cp37m-macosx_10_6_intel. macosx_10_9_intel.macosx_10_9_x86_64 .macosx_10_10_intel.macosx_10_10_x86_64.whl (14.4MB) 100% |████████████████████████████████| 14.4MB 1.4MB/s ……………………………….. ……………………………….. Successfully installed cycler-0.10.0 kiwisolver-1.1.0 matplotlib-3.1.1 pandas-0.25.0 pyparsing-2.4.2 python-dateutil-2.8.0 pytz-2019.2 seaborn-0.9.0 Keras Installation Using Python As of now, we have completed basic requirements for the installtion of Kera. Now, install the Keras using same procedure as specified below − pip install keras Quit virtual environment After finishing all your changes in your project, then simply run the below command to quit the environment − deactivate Anaconda Cloud We believe that you have installed anaconda cloud on your machine. If anaconda is not installed, then visit the official link, and choose download based on your OS. Create a new conda environment Launch anaconda prompt, this will open base Anaconda environment. Let us create a new conda environment. This process is similar to virtualenv. Type the below command in your conda terminal − conda create –name PythonCPU If you want, you can create and install modules using GPU also. In this tutorial, we follow CPU instructions. Activate conda environment To activate the environment, use the below command − activate PythonCPU Install spyder Spyder is an IDE for executing python applications. Let us install this IDE in our conda environment using the below command − conda install spyder Install python libraries We have already known the python libraries numpy, pandas, etc., needed for keras. You can install all the modules by using the below syntax − Syntax conda install -c anaconda <module-name> For example, you want to install pandas − conda install -c anaconda pandas Like the same method, try it yourself to install the remaining modules. Install Keras Now, everything looks good so you can start keras installation using the below command − conda install -c anaconda keras Launch spyder Finally, launch spyder in your conda terminal using the below command − spyder To ensure everything was installed correctly, import all the modules, it will add everything and if anything went wrong, you will get module not found error message.

Keras – Customized Layer Keras allows to create our own customized layer. Once a new layer is created, it can be used in any model without any restriction. Let us learn how to create new layer in this chapter. Keras provides a base layer class, Layer which can sub-classed to create our own customized layer. Let us create a simple layer which will find weight based on normal distribution and then do the basic computation of finding the summation of the product of input and its weight during training. Step 1: Import the necessary module First, let us import the necessary modules − from keras import backend as K from keras.layers import Layer Here, backend is used to access the dot function. Layer is the base class and we will be sub-classing it to create our layer Step 2: Define a layer class Let us create a new class, MyCustomLayer by sub-classing Layer class − class MyCustomLayer(Layer): … Step 3: Initialize the layer class Let us initialize our new class as specified below − def __init__(self, output_dim, **kwargs): self.output_dim = output_dim super(MyCustomLayer, self).__init__(**kwargs) Here, Line 2 sets the output dimension. Line 3 calls the base or super layer’s init function. Step 4: Implement build method build is the main method and its only purpose is to build the layer properly. It can do anything related to the inner working of the layer. Once the custom functionality is done, we can call the base class build function. Our custom build function is as follows − def build(self, input_shape): self.kernel = self.add_weight(name = ”kernel”, shape = (input_shape[1], self.output_dim), initializer = ”normal”, trainable = True) super(MyCustomLayer, self).build(input_shape) Here, Line 1 defines the build method with one argument, input_shape. Shape of the input data is referred by input_shape. Line 2 creates the weight corresponding to input shape and set it in the kernel. It is our custom functionality of the layer. It creates the weight using ‘normal’ initializer. Line 6 calls the base class, build method. Step 5: Implement call method call method does the exact working of the layer during training process. Our custom call method is as follows def call(self, input_data): return K.dot(input_data, self.kernel) Here, Line 1 defines the call method with one argument, input_data. input_data is the input data for our layer. Line 2 return the dot product of the input data, input_data and our layer’s kernel, self.kernel Step 6: Implement compute_output_shape method def compute_output_shape(self, input_shape): return (input_shape[0], self.output_dim) Here, Line 1 defines compute_output_shape method with one argument input_shape Line 2 computes the output shape using shape of input data and output dimension set while initializing the layer. Implementing the build, call and compute_output_shape completes the creating a customized layer. The final and complete code is as follows from keras import backend as K from keras.layers import Layer class MyCustomLayer(Layer): def __init__(self, output_dim, **kwargs): self.output_dim = output_dim super(MyCustomLayer, self).__init__(**kwargs) def build(self, input_shape): self.kernel = self.add_weight(name = ”kernel”, shape = (input_shape[1], self.output_dim), initializer = ”normal”, trainable = True) super(MyCustomLayer, self).build(input_shape) # Be sure to call this at the end def call(self, input_data): return K.dot(input_data, self.kernel) def compute_output_shape(self, input_shape): return (input_shape[0], self.output_dim) Using our customized layer Let us create a simple model using our customized layer as specified below − from keras.models import Sequential from keras.layers import Dense model = Sequential() model.add(MyCustomLayer(32, input_shape = (16,))) model.add(Dense(8, activation = ”softmax”)) model.summary() Here, Our MyCustomLayer is added to the model using 32 units and (16,) as input shape Running the application will print the model summary as below − Model: “sequential_1” _________________________________________________________________ Layer (type) Output Shape Param #================================================================ my_custom_layer_1 (MyCustomL (None, 32) 512 _________________________________________________________________ dense_1 (Dense) (None, 8) 264 ================================================================= Total params: 776 Trainable params: 776 Non-trainable params: 0 _________________________________________________________________

Keras – Deep learning Keras provides a complete framework to create any type of neural networks. Keras is innovative as well as very easy to learn. It supports simple neural network to very large and complex neural network model. Let us understand the architecture of Keras framework and how Keras helps in deep learning in this chapter. Architecture of Keras Keras API can be divided into three main categories − Model Layer Core Modules In Keras, every ANN is represented by Keras Models. In turn, every Keras Model is composition of Keras Layers and represents ANN layers like input, hidden layer, output layers, convolution layer, pooling layer, etc., Keras model and layer access Keras modules for activation function, loss function, regularization function, etc., Using Keras model, Keras Layer, and Keras modules, any ANN algorithm (CNN, RNN, etc.,) can be represented in a simple and efficient manner. The following diagram depicts the relationship between model, layer and core modules − Let us see the overview of Keras models, Keras layers and Keras modules. Model Keras Models are of two types as mentioned below − Sequential Model − Sequential model is basically a linear composition of Keras Layers. Sequential model is easy, minimal as well as has the ability to represent nearly all available neural networks. A simple sequential model is as follows − from keras.models import Sequential from keras.layers import Dense, Activation model = Sequential() model.add(Dense(512, activation = ”relu”, input_shape = (784,))) Where, Line 1 imports Sequential model from Keras models Line 2 imports Dense layer and Activation module Line 4 create a new sequential model using Sequential API Line 5 adds a dense layer (Dense API) with relu activation (using Activation module) function. Sequential model exposes Model class to create customized models as well. We can use sub-classing concept to create our own complex model. Functional API − Functional API is basically used to create complex models. Layer Each Keras layer in the Keras model represent the corresponding layer (input layer, hidden layer and output layer) in the actual proposed neural network model. Keras provides a lot of pre-build layers so that any complex neural network can be easily created. Some of the important Keras layers are specified below, Core Layers Convolution Layers Pooling Layers Recurrent Layers A simple python code to represent a neural network model using sequential model is as follows − from keras.models import Sequential from keras.layers import Dense, Activation, Dropout model = Sequential() model.add(Dense(512, activation = ”relu”, input_shape = (784,))) model.add(Dropout(0.2)) model.add(Dense(512, activation = ”relu”)) model.add(Dropout(0.2)) model.add(Dense(num_classes, activation = ”softmax”)) Where, Line 1 imports Sequential model from Keras models Line 2 imports Dense layer and Activation module Line 4 create a new sequential model using Sequential API Line 5 adds a dense layer (Dense API) with relu activation (using Activation module) function. Line 6 adds a dropout layer (Dropout API) to handle over-fitting. Line 7 adds another dense layer (Dense API) with relu activation (using Activation module) function. Line 8 adds another dropout layer (Dropout API) to handle over-fitting. Line 9 adds final dense layer (Dense API) with softmax activation (using Activation module) function. Keras also provides options to create our own customized layers. Customized layer can be created by sub-classing the Keras.Layer class and it is similar to sub-classing Keras models. Core Modules Keras also provides a lot of built-in neural network related functions to properly create the Keras model and Keras layers. Some of the function are as follows − Activations module − Activation function is an important concept in ANN and activation modules provides many activation function like softmax, relu, etc., Loss module − Loss module provides loss functions like mean_squared_error, mean_absolute_error, poisson, etc., Optimizer module − Optimizer module provides optimizer function like adam, sgd, etc., Regularizers − Regularizer module provides functions like L1 regularizer, L2 regularizer, etc., Let us learn Keras modules in detail in the upcoming chapter.

Keras – Time Series Prediction using LSTM RNN In this chapter, let us write a simple Long Short Term Memory (LSTM) based RNN to do sequence analysis. A sequence is a set of values where each value corresponds to a particular instance of time. Let us consider a simple example of reading a sentence. Reading and understanding a sentence involves reading the word in the given order and trying to understand each word and its meaning in the given context and finally understanding the sentence in a positive or negative sentiment. Here, the words are considered as values, and first value corresponds to first word, second value corresponds to second word, etc., and the order will be strictly maintained. Sequence Analysis is used frequently in natural language processing to find the sentiment analysis of the given text. Let us create a LSTM model to analyze the IMDB movie reviews and find its positive/negative sentiment. The model for the sequence analysis can be represented as below − The core features of the model are as follows − Input layer using Embedding layer with 128 features. First layer, Dense consists of 128 units with normal dropout and recurrent dropout set to 0.2. Output layer, Dense consists of 1 unit and ‘sigmoid’ activation function. Use binary_crossentropy as loss function. Use adam as Optimizer. Use accuracy as metrics. Use 32 as batch size. Use 15 as epochs. Use 80 as the maximum length of the word. Use 2000 as the maximum number of word in a given sentence. Step 1: Import the modules Let us import the necessary modules. from keras.preprocessing import sequence from keras.models import Sequential from keras.layers import Dense, Embedding from keras.layers import LSTM from keras.datasets import imdb Step 2: Load data Let us import the imdb dataset. (x_train, y_train), (x_test, y_test) = imdb.load_data(num_words = 2000) Here, imdb is a dataset provided by Keras. It represents a collection of movies and its reviews. num_words represent the maximum number of words in the review. Step 3: Process the data Let us change the dataset according to our model, so that it can be fed into our model. The data can be changed using the below code − x_train = sequence.pad_sequences(x_train, maxlen=80) x_test = sequence.pad_sequences(x_test, maxlen=80) Here, sequence.pad_sequences convert the list of input data with shape, (data) into 2D NumPy array of shape (data, timesteps). Basically, it adds timesteps concept into the given data. It generates the timesteps of length, maxlen. Step 4: Create the model Let us create the actual model. model = Sequential() model.add(Embedding(2000, 128)) model.add(LSTM(128, dropout = 0.2, recurrent_dropout = 0.2)) model.add(Dense(1, activation = ”sigmoid”)) Here, We have used Embedding layer as input layer and then added the LSTM layer. Finally, a Dense layer is used as output layer. Step 5: Compile the model Let us compile the model using selected loss function, optimizer and metrics. model.compile(loss = ”binary_crossentropy”, optimizer = ”adam”, metrics = [”accuracy”]) Step 6: Train the model LLet us train the model using fit() method. model.fit( x_train, y_train, batch_size = 32, epochs = 15, validation_data = (x_test, y_test) ) Executing the application will output the below information − Epoch 1/15 2019-09-24 01:19:01.151247: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not co mpiled to use: AVX2 25000/25000 [==============================] – 101s 4ms/step – loss: 0.4707 – acc: 0.7716 – val_loss: 0.3769 – val_acc: 0.8349 Epoch 2/15 25000/25000 [==============================] – 95s 4ms/step – loss: 0.3058 – acc: 0.8756 – val_loss: 0.3763 – val_acc: 0.8350 Epoch 3/15 25000/25000 [==============================] – 91s 4ms/step – loss: 0.2100 – acc: 0.9178 – val_loss: 0.5065 – val_acc: 0.8110 Epoch 4/15 25000/25000 [==============================] – 90s 4ms/step – loss: 0.1394 – acc: 0.9495 – val_loss: 0.6046 – val_acc: 0.8146 Epoch 5/15 25000/25000 [==============================] – 90s 4ms/step – loss: 0.0973 – acc: 0.9652 – val_loss: 0.5969 – val_acc: 0.8147 Epoch 6/15 25000/25000 [==============================] – 98s 4ms/step – loss: 0.0759 – acc: 0.9730 – val_loss: 0.6368 – val_acc: 0.8208 Epoch 7/15 25000/25000 [==============================] – 95s 4ms/step – loss: 0.0578 – acc: 0.9811 – val_loss: 0.6657 – val_acc: 0.8184 Epoch 8/15 25000/25000 [==============================] – 97s 4ms/step – loss: 0.0448 – acc: 0.9850 – val_loss: 0.7452 – val_acc: 0.8136 Epoch 9/15 25000/25000 [==============================] – 95s 4ms/step – loss: 0.0324 – acc: 0.9894 – val_loss: 0.7616 – val_acc: 0.8162Epoch 10/15 25000/25000 [==============================] – 100s 4ms/step – loss: 0.0247 – acc: 0.9922 – val_loss: 0.9654 – val_acc: 0.8148 Epoch 11/15 25000/25000 [==============================] – 99s 4ms/step – loss: 0.0169 – acc: 0.9946 – val_loss: 1.0013 – val_acc: 0.8104 Epoch 12/15 25000/25000 [==============================] – 90s 4ms/step – loss: 0.0154 – acc: 0.9948 – val_loss: 1.0316 – val_acc: 0.8100 Epoch 13/15 25000/25000 [==============================] – 89s 4ms/step – loss: 0.0113 – acc: 0.9963 – val_loss: 1.1138 – val_acc: 0.8108 Epoch 14/15 25000/25000 [==============================] – 89s 4ms/step – loss: 0.0106 – acc: 0.9971 – val_loss: 1.0538 – val_acc: 0.8102 Epoch 15/15 25000/25000 [==============================] – 89s 4ms/step – loss: 0.0090 – acc: 0.9972 – val_loss: 1.1453 – val_acc: 0.8129 25000/25000 [==============================] – 10s 390us/step Step 7 − Evaluate the model Let us evaluate the model using test data. score, acc = model.evaluate(x_test, y_test, batch_size = 32) print(”Test score:”, score) print(”Test accuracy:”, acc) Executing the above code will output the below information − Test score: 1.145306069601178 Test accuracy: 0.81292

Keras – Modules As we learned earlier, Keras modules contains pre-defined classes, functions and variables which are useful for deep learning algorithm. Let us learn the modules provided by Keras in this chapter. Available modules Let us first see the list of modules available in the Keras. Initializers − Provides a list of initializers function. We can learn it in details in Keras layer chapter. during model creation phase of machine learning. Regularizers − Provides a list of regularizers function. We can learn it in details in Keras Layers chapter. Constraints − Provides a list of constraints function. We can learn it in details in Keras Layers chapter. Activations − Provides a list of activator function. We can learn it in details in Keras Layers chapter. Losses − Provides a list of loss function. We can learn it in details in Model Training chapter. Metrics − Provides a list of metrics function. We can learn it in details in Model Training chapter. Optimizers − Provides a list of optimizer function. We can learn it in details in Model Training chapter. Callback − Provides a list of callback function. We can use it during the training process to print the intermediate data as well as to stop the training itself (EarlyStopping method) based on some condition. Text processing − Provides functions to convert text into NumPy array suitable for machine learning. We can use it in data preparation phase of machine learning. Image processing − Provides functions to convert images into NumPy array suitable for machine learning. We can use it in data preparation phase of machine learning. Sequence processing − Provides functions to generate time based data from the given input data. We can use it in data preparation phase of machine learning. Backend − Provides function of the backend library like TensorFlow and Theano. Utilities − Provides lot of utility function useful in deep learning. Let us see backend module and utils model in this chapter. backend module backend module is used for keras backend operations. By default, keras runs on top of TensorFlow backend. If you want, you can switch to other backends like Theano or CNTK. Defualt backend configuration is defined inside your root directory under .keras/keras.json file. Keras backend module can be imported using below code >>> from keras import backend as k If we are using default backend TensorFlow, then the below function returns TensorFlow based information as specified below − >>> k.backend() ”tensorflow” >>> k.epsilon() 1e-07 >>> k.image_data_format() ”channels_last” >>> k.floatx() ”float32” Let us understand some of the significant backend functions used for data analysis in brief − get_uid() It is the identifier for the default graph. It is defined below − >>> k.get_uid(prefix=””) 1 >>> k.get_uid(prefix=””) 2 reset_uids It is used resets the uid value. >>> k.reset_uids() Now, again execute the get_uid(). This will be reset and change again to 1. >>> k.get_uid(prefix=””) 1 placeholder It is used instantiates a placeholder tensor. Simple placeholder to hold 3-D shape is shown below − >>> data = k.placeholder(shape = (1,3,3)) >>> data <tf.Tensor ”Placeholder_9:0” shape = (1, 3, 3) dtype = float32> If you use int_shape(), it will show the shape. >>> k.int_shape(data) (1, 3, 3) dot It is used to multiply two tensors. Consider a and b are two tensors and c will be the outcome of multiply of ab. Assume a shape is (4,2) and b shape is (2,3). It is defined below, >>> a = k.placeholder(shape = (4,2)) >>> b = k.placeholder(shape = (2,3)) >>> c = k.dot(a,b) >>> c <tf.Tensor ”MatMul_3:0” shape = (4, 3) dtype = float32> >>> ones It is used to initialize all as one value. >>> res = k.ones(shape = (2,2)) #print the value >>> k.eval(res) array([[1., 1.], [1., 1.]], dtype = float32) batch_dot It is used to perform the product of two data in batches. Input dimension must be 2 or higher. It is shown below − >>> a_batch = k.ones(shape = (2,3)) >>> b_batch = k.ones(shape = (3,2)) >>> c_batch = k.batch_dot(a_batch,b_batch) >>> c_batch <tf.Tensor ”ExpandDims:0” shape = (2, 1) dtype = float32> variable It is used to initializes a variable. Let us perform simple transpose operation in this variable. >>> data = k.variable([[10,20,30,40],[50,60,70,80]]) #variable initialized here >>> result = k.transpose(data) >>> print(result) Tensor(“transpose_6:0″, shape = (4, 2), dtype = float32) >>> print(k.eval(result)) [[10. 50.] [20. 60.] [30. 70.] [40. 80.]] If you want to access from numpy − >>> data = np.array([[10,20,30,40],[50,60,70,80]]) >>> print(np.transpose(data)) [[10 50] [20 60] [30 70] [40 80]] >>> res = k.variable(value = data) >>> print(res) <tf.Variable ”Variable_7:0” shape = (2, 4) dtype = float32_ref> is_sparse(tensor) It is used to check whether the tensor is sparse or not. >>> a = k.placeholder((2, 2), sparse=True) >>> print(a) SparseTensor(indices = Tensor(“Placeholder_8:0”, shape = (?, 2), dtype = int64), values = Tensor(“Placeholder_7:0”, shape = (?,), dtype = float32), dense_shape = Tensor(“Const:0”, shape = (2,), dtype = int64)) >>> print(k.is_sparse(a)) True to_dense() It is used to converts sparse into dense. >>> b = k.to_dense(a) >>> print(b) Tensor(“SparseToDense:0″, shape = (2, 2), dtype = float32) >>> print(k.is_sparse(b)) False random_uniform_variable It is used to initialize using uniform distribution concept. k.random_uniform_variable(shape, mean, scale) Here, shape − denotes the rows and columns in the format of tuples. mean − mean of uniform distribution. scale − standard deviation of uniform distribution. Let us have a look at the below example usage − >>> a = k.random_uniform_variable(shape = (2, 3), low=0, high = 1) >>> b = k. random_uniform_variable(shape = (3,2), low = 0, high = 1) >>> c = k.dot(a, b) >>> k.int_shape(c) (2, 2) utils module utils provides useful utilities function for deep learning. Some of the methods provided by the utils module is as follows − HDF5Matrix It is used to represent the input data in HDF5 format. from keras.utils import HDF5Matrix data = HDF5Matrix(”data.hdf5”, ”data”) to_categorical It is used to convert class vector into binary class matrix. >>> from keras.utils import to_categorical >>> labels = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Keras – Overview of Deep learning Deep learning is an evolving subfield of machine learning. Deep learning involves analyzing the input in layer by layer manner, where each layer progressively extracts higher level information about the input. Let us take a simple scenario of analyzing an image. Let us assume that your input image is divided up into a rectangular grid of pixels. Now, the first layer abstracts the pixels. The second layer understands the edges in the image. The Next layer constructs nodes from the edges. Then, the next would find branches from the nodes. Finally, the output layer will detect the full object. Here, the feature extraction process goes from the output of one layer into the input of the next subsequent layer. By using this approach, we can process huge amount of features, which makes deep learning a very powerful tool. Deep learning algorithms are also useful for the analysis of unstructured data. Let us go through the basics of deep learning in this chapter. Artificial Neural Networks The most popular and primary approach of deep learning is using “Artificial neural network” (ANN). They are inspired from the model of human brain, which is the most complex organ of our body. The human brain is made up of more than 90 billion tiny cells called “Neurons”. Neurons are inter-connected through nerve fiber called “axons” and “Dendrites”. The main role of axon is to transmit information from one neuron to another to which it is connected. Similarly, the main role of dendrites is to receive the information being transmitted by the axons of another neuron to which it is connected. Each neuron processes a small information and then passes the result to another neuron and this process continues. This is the basic method used by our human brain to process huge about of information like speech, visual, etc., and extract useful information from it. Based on this model, the first Artificial Neural Network (ANN) was invented by psychologist Frank Rosenblatt, in the year of 1958. ANNs are made up of multiple nodes which is similar to neurons. Nodes are tightly interconnected and organized into different hidden layers. The input layer receives the input data and the data goes through one or more hidden layers sequentially and finally the output layer predict something useful about the input data. For example, the input may be an image and the output may be the thing identified in the image, say a “Cat”. A single neuron (called as perceptron in ANN) can be represented as below − Here, Multiple input along with weight represents dendrites. Sum of input along with activation function represents neurons. Sum actually means computed value of all inputs and activation function represent a function, which modify the Sum value into 0, 1 or 0 to 1. Actual output represent axon and the output will be received by neuron in next layer. Let us understand different types of artificial neural networks in this section. Multi-Layer Perceptron Multi-Layer perceptron is the simplest form of ANN. It consists of a single input layer, one or more hidden layer and finally an output layer. A layer consists of a collection of perceptron. Input layer is basically one or more features of the input data. Every hidden layer consists of one or more neurons and process certain aspect of the feature and send the processed information into the next hidden layer. The output layer process receives the data from last hidden layer and finally output the result. Convolutional Neural Network (CNN) Convolutional neural network is one of the most popular ANN. It is widely used in the fields of image and video recognition. It is based on the concept of convolution, a mathematical concept. It is almost similar to multi-layer perceptron except it contains series of convolution layer and pooling layer before the fully connected hidden neuron layer. It has three important layers − Convolution layer − It is the primary building block and perform computational tasks based on convolution function. Pooling layer − It is arranged next to convolution layer and is used to reduce the size of inputs by removing unnecessary information so computation can be performed faster. Fully connected layer − It is arranged to next to series of convolution and pooling layer and classify input into various categories. A simple CNN can be represented as below − Here, 2 series of Convolution and pooling layer is used and it receives and process the input (e.g. image). A single fully connected layer is used and it is used to output the data (e.g. classification of image) Recurrent Neural Network (RNN) Recurrent Neural Networks (RNN) are useful to address the flaw in other ANN models. Well, Most of the ANN doesn’t remember the steps from previous situations and learned to make decisions based on context in training. Meanwhile, RNN stores the past information and all its decisions are taken from what it has learnt from the past. This approach is mainly useful in image classification. Sometimes, we may need to look into the future to fix the past. In this case bidirectional RNN is helpful to learn from the past and predict the future. For example, we have handwritten samples in multiple inputs. Suppose, we have confusion in one input then we need to check again other inputs to recognize the correct context which takes the decision from the past. Workflow of ANN Let us first understand the different phases of deep learning and then, learn how Keras helps in the process of deep learning. Collect required data Deep learning requires lot of input data to successfully learn and predict the result. So, first collect as much data as possible. Analyze data Analyze the data and acquire a good understanding of the data. The better understanding of the data is required to select the correct ANN algorithm. Choose an algorithm (model) Choose an algorithm, which will best fit for the type of learning process (e.g image classification, text processing, etc.,)

Keras – Layers As learned earlier, Keras layers are the primary building block of Keras models. Each layer receives input information, do some computation and finally output the transformed information. The output of one layer will flow into the next layer as its input. Let us learn complete details about layers in this chapter. Introduction A Keras layer requires shape of the input (input_shape) to understand the structure of the input data, initializer to set the weight for each input and finally activators to transform the output to make it non-linear. In between, constraints restricts and specify the range in which the weight of input data to be generated and regularizer will try to optimize the layer (and the model) by dynamically applying the penalties on the weights during optimization process. To summarise, Keras layer requires below minimum details to create a complete layer. Shape of the input data Number of neurons / units in the layer Initializers Regularizers Constraints Activations Let us understand the basic concept in the next chapter. Before understanding the basic concept, let us create a simple Keras layer using Sequential model API to get the idea of how Keras model and layer works. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers from keras import regularizers from keras import constraints model = Sequential() model.add(Dense(32, input_shape=(16,), kernel_initializer = ”he_uniform”, kernel_regularizer = None, kernel_constraint = ”MaxNorm”, activation = ”relu”)) model.add(Dense(16, activation = ”relu”)) model.add(Dense(8)) where, Line 1-5 imports the necessary modules. Line 7 creates a new model using Sequential API. Line 9 creates a new Dense layer and add it into the model. Dense is an entry level layer provided by Keras, which accepts the number of neurons or units (32) as its required parameter. If the layer is first layer, then we need to provide Input Shape, (16,) as well. Otherwise, the output of the previous layer will be used as input of the next layer. All other parameters are optional. First parameter represents the number of units (neurons). input_shape represent the shape of input data. kernel_initializer represent initializer to be used. he_uniform function is set as value. kernel_regularizer represent regularizer to be used. None is set as value. kernel_constraint represent constraint to be used. MaxNorm function is set as value. activation represent activation to be used. relu function is set as value. Line 10 creates second Dense layer with 16 units and set relu as the activation function. Line 11 creates final Dense layer with 8 units. Basic Concept of Layers Let us understand the basic concept of layer as well as how Keras supports each concept. Input shape In machine learning, all type of input data like text, images or videos will be first converted into array of numbers and then feed into the algorithm. Input numbers may be single dimensional array, two dimensional array (matrix) or multi-dimensional array. We can specify the dimensional information using shape, a tuple of integers. For example, (4,2) represent matrix with four rows and two columns. >>> import numpy as np >>> shape = (4, 2) >>> input = np.zeros(shape) >>> print(input) [ [0. 0.] [0. 0.] [0. 0.] [0. 0.] ] >>> Similarly, (3,4,2) three dimensional matrix having three collections of 4×2 matrix (two rows and four columns). >>> import numpy as np >>> shape = (3, 4, 2) >>> input = np.zeros(shape) >>> print(input) [ [[0. 0.] [0. 0.] [0. 0.] [0. 0.]] [[0. 0.] [0. 0.] [0. 0.] [0. 0.]] [[0. 0.] [0. 0.] [0. 0.] [0. 0.]] ] >>> To create the first layer of the model (or input layer of the model), shape of the input data should be specified. Initializers In Machine Learning, weight will be assigned to all input data. Initializers module provides different functions to set these initial weight. Some of the Keras Initializer function are as follows − Zeros Generates 0 for all input data. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers my_init = initializers.Zeros() model = Sequential() model.add(Dense(512, activation = ”relu”, input_shape = (784,), kernel_initializer = my_init)) Where, kernel_initializer represent the initializer for kernel of the model. Ones Generates 1 for all input data. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers my_init = initializers.Ones() model.add(Dense(512, activation = ”relu”, input_shape = (784,), kernel_initializer = my_init)) Constant Generates a constant value (say, 5) specified by the user for all input data. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers my_init = initializers.Constant(value = 0) model.add( Dense(512, activation = ”relu”, input_shape = (784,), kernel_initializer = my_init) ) where, value represent the constant value RandomNormal Generates value using normal distribution of input data. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers my_init = initializers.RandomNormal(mean=0.0, stddev = 0.05, seed = None) model.add(Dense(512, activation = ”relu”, input_shape = (784,), kernel_initializer = my_init)) where, mean represent the mean of the random values to generate stddev represent the standard deviation of the random values to generate seed represent the values to generate random number RandomUniform Generates value using uniform distribution of input data. from keras import initializers my_init = initializers.RandomUniform(minval = -0.05, maxval = 0.05, seed = None) model.add(Dense(512, activation = ”relu”, input_shape = (784,), kernel_initializer = my_init)) where, minval represent the lower bound of the random values to generate maxval represent the upper bound of the random values to generate TruncatedNormal Generates value using truncated normal distribution of input data. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers my_init = initializers.TruncatedNormal(mean = 0.0, stddev = 0.05, seed = None model.add(Dense(512, activation = ”relu”, input_shape = (784,), kernel_initializer = my_init)) VarianceScaling Generates value based on the input shape and output shape of the layer along with the specified scale. from keras.models import Sequential from keras.layers import Activation, Dense from keras import initializers my_init = initializers.VarianceScaling( scale = 1.0, mode = ”fan_in”, distribution = ”normal”, seed = None) model.add(Dense(512, activation = ”relu”, input_shape = (784,),