Category: Machine Learning

Image Recognition using TensorFlow TensorFlow includes a special feature of image recognition and these images are stored in a specific folder. With relatively same images, it will be easy to implement this logic for security purposes. The folder structure of image recognition code implementation is as shown below − The dataset_image includes the related images, which need to be loaded. We will focus on image recognition with our logo defined in it. The images are loaded with “load_data.py” script, which helps in keeping a note on various image recognition modules within them. import pickle from sklearn.model_selection import train_test_split from scipy import misc import numpy as np import os label = os.listdir(“dataset_image”) label = label[1:] dataset = [] for image_label in label: images = os.listdir(“dataset_image/”+image_label) for image in images: img = misc.imread(“dataset_image/”+image_label+”/”+image) img = misc.imresize(img, (64, 64)) dataset.append((img,image_label)) X = [] Y = [] for input,image_label in dataset: X.append(input) Y.append(label.index(image_label)) X = np.array(X) Y = np.array(Y) X_train,y_train, = X,Y data_set = (X_train,y_train) save_label = open(“int_to_word_out.pickle”,”wb”) pickle.dump(label, save_label) save_label.close() The training of images helps in storing the recognizable patterns within specified folder. import numpy import matplotlib.pyplot as plt from keras.layers import Dropout from keras.layers import Flatten from keras.constraints import maxnorm from keras.optimizers import SGD from keras.layers import Conv2D from keras.layers.convolutional import MaxPooling2D from keras.utils import np_utils from keras import backend as K import load_data from keras.models import Sequential from keras.layers import Dense import keras K.set_image_dim_ordering(”tf”) # fix random seed for reproducibility seed = 7 numpy.random.seed(seed) # load data (X_train,y_train) = load_data.data_set # normalize inputs from 0-255 to 0.0-1.0 X_train = X_train.astype(”float32”) #X_test = X_test.astype(”float32”) X_train = X_train / 255.0 #X_test = X_test / 255.0 # one hot encode outputs y_train = np_utils.to_categorical(y_train) #y_test = np_utils.to_categorical(y_test) num_classes = y_train.shape[1] # Create the model model = Sequential() model.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), padding = ”same”, activation = ”relu”, kernel_constraint = maxnorm(3))) model.add(Dropout(0.2)) model.add(Conv2D(32, (3, 3), activation = ”relu”, padding = ”same”, kernel_constraint = maxnorm(3))) model.add(MaxPooling2D(pool_size = (2, 2))) model.add(Flatten()) model.add(Dense(512, activation = ”relu”, kernel_constraint = maxnorm(3))) model.add(Dropout(0.5)) model.add(Dense(num_classes, activation = ”softmax”)) # Compile model epochs = 10 lrate = 0.01 decay = lrate/epochs sgd = SGD(lr = lrate, momentum = 0.9, decay = decay, nesterov = False) model.compile(loss = ”categorical_crossentropy”, optimizer = sgd, metrics = [”accuracy”]) print(model.summary()) #callbacks = [keras.callbacks.EarlyStopping( monitor = ”val_loss”, min_delta = 0, patience = 0, verbose = 0, mode = ”auto”)] callbacks = [keras.callbacks.TensorBoard(log_dir=”./logs”, histogram_freq = 0, batch_size = 32, write_graph = True, write_grads = False, write_images = True, embeddings_freq = 0, embeddings_layer_names = None, embeddings_metadata = None)] # Fit the model model.fit(X_train, y_train, epochs = epochs, batch_size = 32,shuffle = True,callbacks = callbacks) # Final evaluation of the model scores = model.evaluate(X_train, y_train, verbose = 0) print(“Accuracy: %.2f%%” % (scores[1]*100)) # serialize model to JSONx model_json = model.to_json() with open(“model_face.json”, “w”) as json_file: json_file.write(model_json) # serialize weights to HDF5 model.save_weights(“model_face.h5”) print(“Saved model to disk”) The above line of code generates an output as shown below −

Recommendations for Neural Network Training In this chapter, we will understand the various aspects of neural network training which can be implemented using TensorFlow framework. Following are the ten recommendations, which can be evaluated − Back Propagation Back propagation is a simple method to compute partial derivatives, which includes the basic form of composition best suitable for neural nets. Stochastic Gradient Descent In stochastic gradient descent, a batch is the total number of examples, which a user uses to calculate the gradient in a single iteration. So far, it is assumed that the batch has been the entire data set. The best illustration is working at Google scale; data sets often contain billions or even hundreds of billions of examples. Learning Rate Decay Adapting the learning rate is one of the most important features of gradient descent optimization. This is crucial to TensorFlow implementation. Dropout Deep neural nets with a large number of parameters form powerful machine learning systems. However, over fitting is a serious problem in such networks. Max Pooling Max pooling is a sample-based discretization process. The object is to down-sample an input representation, which reduces the dimensionality with the required assumptions. Long Short Term Memory (LSTM) LSTM controls the decision on what inputs should be taken within the specified neuron. It includes the control on deciding what should be computed and what output should be generated.

TensorFlow Tutorial Job Search TensorFlow is an open source machine learning framework for all developers. It is used for implementing machine learning and deep learning applications. To develop and research on fascinating ideas on artificial intelligence, Google team created TensorFlow. TensorFlow is designed in Python programming language, hence it is considered an easy to understand framework. Audience This tutorial has been prepared for python developers who focus on research and development with various machine learning and deep learning algorithms. The aim of this tutorial is to describe all TensorFlow objects and methods. Prerequisites Before proceeding with this tutorial, you need to have a basic knowledge of any Python programming language. Knowledge of artificial intelligence concepts will be a plus point.

TensorFlow – Distributed Computing This chapter will focus on how to get started with distributed TensorFlow. The aim is to help developers understand the basic distributed TF concepts that are reoccurring, such as TF servers. We will use the Jupyter Notebook for evaluating distributed TensorFlow. The implementation of distributed computing with TensorFlow is mentioned below − Step 1 − Import the necessary modules mandatory for distributed computing − import tensorflow as tf Step 2 − Create a TensorFlow cluster with one node. Let this node be responsible for a job that that has name “worker” and that will operate one take at localhost:2222. cluster_spec = tf.train.ClusterSpec({”worker” : [”localhost:2222”]}) server = tf.train.Server(cluster_spec) server.target The above scripts generate the following output − ”grpc://localhost:2222” The server is currently running. Step 3 − The server configuration with respective session can be calculated by executing the following command − server.server_def The above command generates the following output − cluster { job { name: “worker” tasks { value: “localhost:2222” } } } job_name: “worker” protocol: “grpc” Step 4 − Launch a TensorFlow session with the execution engine being the server. Use TensorFlow to create a local server and use lsof to find out the location of the server. sess = tf.Session(target = server.target) server = tf.train.Server.create_local_server() Step 5 − View devices available in this session and close the respective session. devices = sess.list_devices() for d in devices: print(d.name) sess.close() The above command generates the following output − /job:worker/replica:0/task:0/device:CPU:0

TensorFlow – Convolutional Neural Networks After understanding machine-learning concepts, we can now shift our focus to deep learning concepts. Deep learning is a division of machine learning and is considered as a crucial step taken by researchers in recent decades. The examples of deep learning implementation include applications like image recognition and speech recognition. Following are the two important types of deep neural networks − Convolutional Neural Networks Recurrent Neural Networks In this chapter, we will focus on the CNN, Convolutional Neural Networks. Convolutional Neural Networks Convolutional Neural networks are designed to process data through multiple layers of arrays. This type of neural networks is used in applications like image recognition or face recognition. The primary difference between CNN and any other ordinary neural network is that CNN takes input as a two-dimensional array and operates directly on the images rather than focusing on feature extraction which other neural networks focus on. The dominant approach of CNN includes solutions for problems of recognition. Top companies like Google and Facebook have invested in research and development towards recognition projects to get activities done with greater speed. A convolutional neural network uses three basic ideas − Local respective fields Convolution Pooling Let us understand these ideas in detail. CNN utilizes spatial correlations that exist within the input data. Each concurrent layer of a neural network connects some input neurons. This specific region is called local receptive field. Local receptive field focusses on the hidden neurons. The hidden neurons process the input data inside the mentioned field not realizing the changes outside the specific boundary. Following is a diagram representation of generating local respective fields − If we observe the above representation, each connection learns a weight of the hidden neuron with an associated connection with movement from one layer to another. Here, individual neurons perform a shift from time to time. This process is called “convolution”. The mapping of connections from the input layer to the hidden feature map is defined as “shared weights” and bias included is called “shared bias”. CNN or convolutional neural networks use pooling layers, which are the layers, positioned immediately after CNN declaration. It takes the input from the user as a feature map that comes out of convolutional networks and prepares a condensed feature map. Pooling layers helps in creating layers with neurons of previous layers. TensorFlow Implementation of CNN In this section, we will learn about the TensorFlow implementation of CNN. The steps,which require the execution and proper dimension of the entire network, are as shown below − Step 1 − Include the necessary modules for TensorFlow and the data set modules, which are needed to compute the CNN model. import tensorflow as tf import numpy as np from tensorflow.examples.tutorials.mnist import input_data Step 2 − Declare a function called run_cnn(), which includes various parameters and optimization variables with declaration of data placeholders. These optimization variables will declare the training pattern. def run_cnn(): mnist = input_data.read_data_sets(“MNIST_data/”, one_hot = True) learning_rate = 0.0001 epochs = 10 batch_size = 50 Step 3 − In this step, we will declare the training data placeholders with input parameters – for 28 x 28 pixels = 784. This is the flattened image data that is drawn from mnist.train.nextbatch(). We can reshape the tensor according to our requirements. The first value (-1) tells function to dynamically shape that dimension based on the amount of data passed to it. The two middle dimensions are set to the image size (i.e. 28 x 28). x = tf.placeholder(tf.float32, [None, 784]) x_shaped = tf.reshape(x, [-1, 28, 28, 1]) y = tf.placeholder(tf.float32, [None, 10]) Step 4 − Now it is important to create some convolutional layers − layer1 = create_new_conv_layer(x_shaped, 1, 32, [5, 5], [2, 2], name = ”layer1”) layer2 = create_new_conv_layer(layer1, 32, 64, [5, 5], [2, 2], name = ”layer2”) Step 5 − Let us flatten the output ready for the fully connected output stage – after two layers of stride 2 pooling with the dimensions of 28 x 28, to dimension of 14 x 14 or minimum 7 x 7 x,y co-ordinates, but with 64 output channels. To create the fully connected with “dense” layer, the new shape needs to be [-1, 7 x 7 x 64]. We can set up some weights and bias values for this layer, then activate with ReLU. flattened = tf.reshape(layer2, [-1, 7 * 7 * 64]) wd1 = tf.Variable(tf.truncated_normal([7 * 7 * 64, 1000], stddev = 0.03), name = ”wd1”) bd1 = tf.Variable(tf.truncated_normal([1000], stddev = 0.01), name = ”bd1”) dense_layer1 = tf.matmul(flattened, wd1) + bd1 dense_layer1 = tf.nn.relu(dense_layer1) Step 6 − Another layer with specific softmax activations with the required optimizer defines the accuracy assessment, which makes the setup of initialization operator. wd2 = tf.Variable(tf.truncated_normal([1000, 10], stddev = 0.03), name = ”wd2”) bd2 = tf.Variable(tf.truncated_normal([10], stddev = 0.01), name = ”bd2”) dense_layer2 = tf.matmul(dense_layer1, wd2) + bd2 y_ = tf.nn.softmax(dense_layer2) cross_entropy = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits(logits = dense_layer2, labels = y)) optimiser = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cross_entropy) correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) init_op = tf.global_variables_initializer() Step 7 − We should set up recording variables. This adds up a summary to store the accuracy of data. tf.summary.scalar(”accuracy”, accuracy) merged = tf.summary.merge_all() writer = tf.summary.FileWriter(”E:TensorFlowProject”) with tf.Session() as sess: sess.run(init_op) total_batch = int(len(mnist.train.labels) / batch_size) for epoch in range(epochs): avg_cost = 0 for i in range(total_batch): batch_x, batch_y = mnist.train.next_batch(batch_size = batch_size) _, c = sess.run([optimiser, cross_entropy], feed_dict = { x:batch_x, y: batch_y}) avg_cost += c / total_batch test_acc = sess.run(accuracy, feed_dict = {x: mnist.test.images, y: mnist.test.labels}) summary = sess.run(merged, feed_dict = {x: mnist.test.images, y: mnist.test.labels}) writer.add_summary(summary, epoch) print(“nTraining complete!”) writer.add_graph(sess.graph) print(sess.run(accuracy, feed_dict = {x: mnist.test.images, y: mnist.test.labels})) def create_new_conv_layer( input_data, num_input_channels, num_filters,filter_shape, pool_shape, name): conv_filt_shape = [ filter_shape[0], filter_shape[1], num_input_channels, num_filters] weights = tf.Variable( tf.truncated_normal(conv_filt_shape, stddev = 0.03), name = name+”_W”) bias = tf.Variable(tf.truncated_normal([num_filters]), name = name+”_b”) #Out layer defines the output out_layer = tf.nn.conv2d(input_data, weights, [1, 1, 1, 1], padding = ”SAME”) out_layer += bias out_layer = tf.nn.relu(out_layer) ksize = [1, pool_shape[0], pool_shape[1], 1] strides =

Understanding Artificial Intelligence Artificial Intelligence includes the simulation process of human intelligence by machines and special computer systems. The examples of artificial intelligence include learning, reasoning and self-correction. Applications of AI include speech recognition, expert systems, and image recognition and machine vision. Machine learning is the branch of artificial intelligence, which deals with systems and algorithms that can learn any new data and data patterns. Let us focus on the Venn diagram mentioned below for understanding machine learning and deep learning concepts. Machine learning includes a section of machine learning and deep learning is a part of machine learning. The ability of program which follows machine learning concepts is to improve its performance of observed data. The main motive of data transformation is to improve its knowledge in order to achieve better results in the future, provide output closer to the desired output for that particular system. Machine learning includes “pattern recognition” which includes the ability to recognize the patterns in data. The patterns should be trained to show the output in desirable manner. Machine learning can be trained in two different ways − Supervised training Unsupervised training Supervised Learning Supervised learning or supervised training includes a procedure where the training set is given as input to the system wherein, each example is labeled with a desired output value. The training in this type is performed using minimization of a particular loss function, which represents the output error with respect to the desired output system. After completion of training, the accuracy of each model is measured with respect to disjoint examples from training set, also called the validation set. The best example to illustrate “Supervised learning” is with a bunch of photos given with information included in them. Here, the user can train a model to recognize new photos. Unsupervised Learning In unsupervised learning or unsupervised training, include training examples, which are not labeled by the system to which class they belong. The system looks for the data, which share common characteristics, and changes them based on internal knowledge features.This type of learning algorithms are basically used in clustering problems. The best example to illustrate “Unsupervised learning” is with a bunch of photos with no information included and user trains model with classification and clustering. This type of training algorithm works with assumptions as no information is given.

TensorFlow – Keras Keras is compact, easy to learn, high-level Python library run on top of TensorFlow framework. It is made with focus of understanding deep learning techniques, such as creating layers for neural networks maintaining the concepts of shapes and mathematical details. The creation of freamework can be of the following two types − Sequential API Functional API Consider the following eight steps to create deep learning model in Keras − Loading the data Preprocess the loaded data Definition of model Compiling the model Fit the specified model Evaluate it Make the required predictions Save the model We will use the Jupyter Notebook for execution and display of output as shown below − Step 1 − Loading the data and preprocessing the loaded data is implemented first to execute the deep learning model. import warnings warnings.filterwarnings(”ignore”) import numpy as np np.random.seed(123) # for reproducibility from keras.models import Sequential from keras.layers import Flatten, MaxPool2D, Conv2D, Dense, Reshape, Dropout from keras.utils import np_utils Using TensorFlow backend. from keras.datasets import mnist # Load pre-shuffled MNIST data into train and test sets (X_train, y_train), (X_test, y_test) = mnist.load_data() X_train = X_train.reshape(X_train.shape[0], 28, 28, 1) X_test = X_test.reshape(X_test.shape[0], 28, 28, 1) X_train = X_train.astype(”float32”) X_test = X_test.astype(”float32”) X_train /= 255 X_test /= 255 Y_train = np_utils.to_categorical(y_train, 10) Y_test = np_utils.to_categorical(y_test, 10) This step can be defined as “Import libraries and Modules” which means all the libraries and modules are imported as an initial step. Step 2 − In this step, we will define the model architecture − model = Sequential() model.add(Conv2D(32, 3, 3, activation = ”relu”, input_shape = (28,28,1))) model.add(Conv2D(32, 3, 3, activation = ”relu”)) model.add(MaxPool2D(pool_size = (2,2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(128, activation = ”relu”)) model.add(Dropout(0.5)) model.add(Dense(10, activation = ”softmax”)) Step 3 − Let us now compile the specified model − model.compile(loss = ”categorical_crossentropy”, optimizer = ”adam”, metrics = [”accuracy”]) Step 4 − We will now fit the model using training data − model.fit(X_train, Y_train, batch_size = 32, epochs = 10, verbose = 1) The output of iterations created is as follows − Epoch 1/10 60000/60000 [==============================] – 65s – loss: 0.2124 – acc: 0.9345 Epoch 2/10 60000/60000 [==============================] – 62s – loss: 0.0893 – acc: 0.9740 Epoch 3/10 60000/60000 [==============================] – 58s – loss: 0.0665 – acc: 0.9802 Epoch 4/10 60000/60000 [==============================] – 62s – loss: 0.0571 – acc: 0.9830 Epoch 5/10 60000/60000 [==============================] – 62s – loss: 0.0474 – acc: 0.9855 Epoch 6/10 60000/60000 [==============================] – 59s – loss: 0.0416 – acc: 0.9871 Epoch 7/10 60000/60000 [==============================] – 61s – loss: 0.0380 – acc: 0.9877 Epoch 8/10 60000/60000 [==============================] – 63s – loss: 0.0333 – acc: 0.9895 Epoch 9/10 60000/60000 [==============================] – 64s – loss: 0.0325 – acc: 0.9898 Epoch 10/10 60000/60000 [==============================] – 60s – loss: 0.0284 – acc: 0.9910