Category: Machine Learning

TensorFlow – Forming Graphs A partial differential equation (PDE) is a differential equation, which involves partial derivatives with unknown function of several independent variables. With reference to partial differential equations, we will focus on creating new graphs. Let us assume there is a pond with dimension 500*500 square − N = 500 Now, we will compute partial differential equation and form the respective graph using it. Consider the steps given below for computing graph. Step 1 − Import libraries for simulation. import tensorflow as tf import numpy as np import matplotlib.pyplot as plt Step 2 − Include functions for transformation of a 2D array into a convolution kernel and simplified 2D convolution operation. def make_kernel(a): a = np.asarray(a) a = a.reshape(list(a.shape) + [1,1]) return tf.constant(a, dtype=1) def simple_conv(x, k): “””A simplified 2D convolution operation””” x = tf.expand_dims(tf.expand_dims(x, 0), -1) y = tf.nn.depthwise_conv2d(x, k, [1, 1, 1, 1], padding = ”SAME”) return y[0, :, :, 0] def laplace(x): “””Compute the 2D laplacian of an array””” laplace_k = make_kernel([[0.5, 1.0, 0.5], [1.0, -6., 1.0], [0.5, 1.0, 0.5]]) return simple_conv(x, laplace_k) sess = tf.InteractiveSession() Step 3 − Include the number of iterations and compute the graph to display the records accordingly. N = 500 # Initial Conditions — some rain drops hit a pond # Set everything to zero u_init = np.zeros([N, N], dtype = np.float32) ut_init = np.zeros([N, N], dtype = np.float32) # Some rain drops hit a pond at random points for n in range(100): a,b = np.random.randint(0, N, 2) u_init[a,b] = np.random.uniform() plt.imshow(u_init) plt.show() # Parameters: # eps — time resolution # damping — wave damping eps = tf.placeholder(tf.float32, shape = ()) damping = tf.placeholder(tf.float32, shape = ()) # Create variables for simulation state U = tf.Variable(u_init) Ut = tf.Variable(ut_init) # Discretized PDE update rules U_ = U + eps * Ut Ut_ = Ut + eps * (laplace(U) – damping * Ut) # Operation to update the state step = tf.group(U.assign(U_), Ut.assign(Ut_)) # Initialize state to initial conditions tf.initialize_all_variables().run() # Run 1000 steps of PDE for i in range(1000): # Step simulation step.run({eps: 0.03, damping: 0.04}) # Visualize every 50 steps if i % 500 == 0: plt.imshow(U.eval()) plt.show() The graphs are plotted as shown below −

TensorFlow – CNN And RNN Difference In this chapter, we will focus on the difference between CNN and RNN − CNN RNN It is suitable for spatial data such as images. RNN is suitable for temporal data, also called sequential data. CNN is considered to be more powerful than RNN. RNN includes less feature compatibility when compared to CNN. This network takes fixed size inputs and generates fixed size outputs. RNN can handle arbitrary input/output lengths. CNN is a type of feed-forward artificial neural network with variations of multilayer perceptrons designed to use minimal amounts of preprocessing. RNN unlike feed forward neural networks – can use their internal memory to process arbitrary sequences of inputs. CNNs use connectivity pattern between the neurons. This is inspired by the organization of the animal visual cortex, whose individual neurons are arranged in such a way that they respond to overlapping regions tiling the visual field. Recurrent neural networks use time-series information – what a user spoke last will impact what he/she will speak next. CNNs are ideal for images and video processing. RNNs are ideal for text and speech analysis. Following illustration shows the schematic representation of CNN and RNN −

TensorFlow – XOR Implementation In this chapter, we will learn about the XOR implementation using TensorFlow. Before starting with XOR implementation in TensorFlow, let us see the XOR table values. This will help us understand encryption and decryption process. A B A XOR B 0 0 0 0 1 1 1 0 1 1 1 0 XOR Cipher encryption method is basically used to encrypt data which is hard to crack with brute force method, i.e., by generating random encryption keys which match the appropriate key. The concept of implementation with XOR Cipher is to define a XOR encryption key and then perform XOR operation of the characters in the specified string with this key, which a user tries to encrypt. Now we will focus on XOR implementation using TensorFlow, which is mentioned below − #Declaring necessary modules import tensorflow as tf import numpy as np “”” A simple numpy implementation of a XOR gate to understand the backpropagation algorithm “”” x = tf.placeholder(tf.float64,shape = [4,2],name = “x”) #declaring a place holder for input x y = tf.placeholder(tf.float64,shape = [4,1],name = “y”) #declaring a place holder for desired output y m = np.shape(x)[0]#number of training examples n = np.shape(x)[1]#number of features hidden_s = 2 #number of nodes in the hidden layer l_r = 1#learning rate initialization theta1 = tf.cast(tf.Variable(tf.random_normal([3,hidden_s]),name = “theta1”),tf.float64) theta2 = tf.cast(tf.Variable(tf.random_normal([hidden_s+1,1]),name = “theta2”),tf.float64) #conducting forward propagation a1 = tf.concat([np.c_[np.ones(x.shape[0])],x],1) #the weights of the first layer are multiplied by the input of the first layer z1 = tf.matmul(a1,theta1) #the input of the second layer is the output of the first layer, passed through the activation function and column of biases is added a2 = tf.concat([np.c_[np.ones(x.shape[0])],tf.sigmoid(z1)],1) #the input of the second layer is multiplied by the weights z3 = tf.matmul(a2,theta2) #the output is passed through the activation function to obtain the final probability h3 = tf.sigmoid(z3) cost_func = -tf.reduce_sum(y*tf.log(h3)+(1-y)*tf.log(1-h3),axis = 1) #built in tensorflow optimizer that conducts gradient descent using specified learning rate to obtain theta values optimiser = tf.train.GradientDescentOptimizer(learning_rate = l_r).minimize(cost_func) #setting required X and Y values to perform XOR operation X = [[0,0],[0,1],[1,0],[1,1]] Y = [[0],[1],[1],[0]] #initializing all variables, creating a session and running a tensorflow session init = tf.global_variables_initializer() sess = tf.Session() sess.run(init) #running gradient descent for each iteration and printing the hypothesis obtained using the updated theta values for i in range(100000): sess.run(optimiser, feed_dict = {x:X,y:Y})#setting place holder values using feed_dict if i%100==0: print(“Epoch:”,i) print(“Hyp:”,sess.run(h3,feed_dict = {x:X,y:Y})) The above line of code generates an output as shown in the screenshot below −

TensorFlow – Basics In this chapter, we will learn about the basics of TensorFlow. We will begin by understanding the data structure of tensor. Tensor Data Structure Tensors are used as the basic data structures in TensorFlow language. Tensors represent the connecting edges in any flow diagram called the Data Flow Graph. Tensors are defined as multidimensional array or list. Tensors are identified by the following three parameters − Rank Unit of dimensionality described within tensor is called rank. It identifies the number of dimensions of the tensor. A rank of a tensor can be described as the order or n-dimensions of a tensor defined. Shape The number of rows and columns together define the shape of Tensor. Type Type describes the data type assigned to Tensor’s elements. A user needs to consider the following activities for building a Tensor − Build an n-dimensional array Convert the n-dimensional array. Various Dimensions of TensorFlow TensorFlow includes various dimensions. The dimensions are described in brief below − One dimensional Tensor One dimensional tensor is a normal array structure which includes one set of values of the same data type. Declaration >>> import numpy as np >>> tensor_1d = np.array([1.3, 1, 4.0, 23.99]) >>> print tensor_1d The implementation with the output is shown in the screenshot below − The indexing of elements is same as Python lists. The first element starts with index of 0; to print the values through index, all you need to do is mention the index number. >>> print tensor_1d[0] 1.3 >>> print tensor_1d[2] 4.0 Two dimensional Tensors Sequence of arrays are used for creating “two dimensional tensors”. The creation of two-dimensional tensors is described below − Following is the complete syntax for creating two dimensional arrays − >>> import numpy as np >>> tensor_2d = np.array([(1,2,3,4),(4,5,6,7),(8,9,10,11),(12,13,14,15)]) >>> print(tensor_2d) [[ 1 2 3 4] [ 4 5 6 7] [ 8 9 10 11] [12 13 14 15]] >>> The specific elements of two dimensional tensors can be tracked with the help of row number and column number specified as index numbers. >>> tensor_2d[3][2] 14 Tensor Handling and Manipulations In this section, we will learn about Tensor Handling and Manipulations. To begin with, let us consider the following code − import tensorflow as tf import numpy as np matrix1 = np.array([(2,2,2),(2,2,2),(2,2,2)],dtype = ”int32”) matrix2 = np.array([(1,1,1),(1,1,1),(1,1,1)],dtype = ”int32”) print (matrix1) print (matrix2) matrix1 = tf.constant(matrix1) matrix2 = tf.constant(matrix2) matrix_product = tf.matmul(matrix1, matrix2) matrix_sum = tf.add(matrix1,matrix2) matrix_3 = np.array([(2,7,2),(1,4,2),(9,0,2)],dtype = ”float32”) print (matrix_3) matrix_det = tf.matrix_determinant(matrix_3) with tf.Session() as sess: result1 = sess.run(matrix_product) result2 = sess.run(matrix_sum) result3 = sess.run(matrix_det) print (result1) print (result2) print (result3) Output The above code will generate the following output − Explanation We have created multidimensional arrays in the above source code. Now, it is important to understand that we created graph and sessions, which manage the Tensors and generate the appropriate output. With the help of graph, we have the output specifying the mathematical calculations between Tensors.

TensorFlow – Single Layer Perceptron For understanding single layer perceptron, it is important to understand Artificial Neural Networks (ANN). Artificial neural networks is the information processing system the mechanism of which is inspired with the functionality of biological neural circuits. An artificial neural network possesses many processing units connected to each other. Following is the schematic representation of artificial neural network − The diagram shows that the hidden units communicate with the external layer. While the input and output units communicate only through the hidden layer of the network. The pattern of connection with nodes, the total number of layers and level of nodes between inputs and outputs with the number of neurons per layer define the architecture of a neural network. There are two types of architecture. These types focus on the functionality artificial neural networks as follows − Single Layer Perceptron Multi-Layer Perceptron Single Layer Perceptron Single layer perceptron is the first proposed neural model created. The content of the local memory of the neuron consists of a vector of weights. The computation of a single layer perceptron is performed over the calculation of sum of the input vector each with the value multiplied by corresponding element of vector of the weights. The value which is displayed in the output will be the input of an activation function. Let us focus on the implementation of single layer perceptron for an image classification problem using TensorFlow. The best example to illustrate the single layer perceptron is through representation of “Logistic Regression”. Now, let us consider the following basic steps of training logistic regression − The weights are initialized with random values at the beginning of the training. For each element of the training set, the error is calculated with the difference between desired output and the actual output. The error calculated is used to adjust the weights. The process is repeated until the error made on the entire training set is not less than the specified threshold, until the maximum number of iterations is reached. The complete code for evaluation of logistic regression is mentioned below − # Import MINST data from tensorflow.examples.tutorials.mnist import input_data mnist = input_data.read_data_sets(“/tmp/data/”, one_hot = True) import tensorflow as tf import matplotlib.pyplot as plt # Parameters learning_rate = 0.01 training_epochs = 25 batch_size = 100 display_step = 1 # tf Graph Input x = tf.placeholder(“float”, [None, 784]) # mnist data image of shape 28*28 = 784 y = tf.placeholder(“float”, [None, 10]) # 0-9 digits recognition => 10 classes # Create model # Set model weights W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) # Construct model activation = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax # Minimize error using cross entropy cross_entropy = y*tf.log(activation) cost = tf.reduce_mean (-tf.reduce_sum (cross_entropy,reduction_indices = 1)) optimizer = tf.train. GradientDescentOptimizer(learning_rate).minimize(cost) #Plot settings avg_set = [] epoch_set = [] # Initializing the variables init = tf.initialize_all_variables() # Launch the graph with tf.Session() as sess: sess.run(init) # Training cycle for epoch in range(training_epochs): avg_cost = 0. total_batch = int(mnist.train.num_examples/batch_size) # Loop over all batches for i in range(total_batch): batch_xs, batch_ys = mnist.train.next_batch(batch_size) # Fit training using batch data sess.run(optimizer, feed_dict = { x: batch_xs, y: batch_ys}) # Compute average loss avg_cost += sess.run(cost, feed_dict = { x: batch_xs, y: batch_ys})/total_batch # Display logs per epoch step if epoch % display_step == 0: print (“Epoch:”, ”%04d” % (epoch+1), “cost=”, “{:.9f}”.format(avg_cost)) avg_set.append(avg_cost) epoch_set.append(epoch+1) print (“Training phase finished”) plt.plot(epoch_set,avg_set, ”o”, label = ”Logistic Regression Training phase”) plt.ylabel(”cost”) plt.xlabel(”epoch”) plt.legend() plt.show() # Test model correct_prediction = tf.equal(tf.argmax(activation, 1), tf.argmax(y, 1)) # Calculate accuracy accuracy = tf.reduce_mean(tf.cast(correct_prediction, “float”)) print (“Model accuracy:”, accuracy.eval({x: mnist.test.images, y: mnist.test.labels})) Output The above code generates the following output − The logistic regression is considered as a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal or independent variables.

TensorFlow – Useful Resources The following resources contain additional information on TensorFlow. Please use them to get more in-depth knowledge on this. Useful Links on TensorFlow − Official Website of TensorFlow. − Wikipedia Reference for TensorFlow. Useful Books on Tensorflow To enlist your site on this page, please drop an email to [email protected]