Category: python Network Programming

Python – Custom HTTP Requests ”; Previous Next The Hypertext Transfer Protocol (HTTP) is a protocol used to enable communications between clients and servers. It works as a request-response protocol between a client and server. The requesting device is known as the client and the device that sends the response is known as server. The urllib is the traditional python library which is used in python programs to handle the http requests. But now there is urllib3 which does more than what urllib used to do. We import the urllib3 library to see how python can use it to make a http request and receive a response. We can customize the type of request by choosing the request method. Pip install urllib3 Example In the below example we use the PoolManager() object which takes care of the connection details of the http request. Next we use the request() object to make a http request with the POST method. Finally we also use the json library to print the received values in json format. import urllib3 import json http = urllib3.PoolManager() r = http.request( ”POST”, ”http://httpbin.org/post”, fields={”field”: ”value”}) print json.loads(r.data.decode(”utf-8”))[”form”] When we run the above program, we get the following output − {field”: value”} URL Using a Query We can also pass query parameters to build custom URLs. In the below example the request method uses the values in the query string to complete the URL which can be further used by another function in the python program. import requests query = {”q”: ”river”, ”order”: ”popular”, ”min_width”: ”800”, ”min_height”: ”600”} req = requests.get(”https://pixabay.com/en/photos/”, params=query) print(req.url) When we run the above program, we get the following output − https://pixabay.com/en/photos/?q=river&min_width=800&min_height=600&order=popular Print Page Previous Next Advertisements ”;

Python – Connection Re-use ”; Previous Next When a client makes a valid request to a server, a temporary connection is established between them to complete the sending and receiving process. But there are scenarios where the connection needs to be kept alive as there is need of automatic requests and responses between the programs which are communicating. Take for example an interactive webpage. After the webpage is loaded there is a need of submitting a form data or downloading further CSS and JavaScript components. The connection needs to be kept alive for faster performance and an unbroken communication between the client and the server. Python provides urllib3 module which had methods to take care of connection reuse between a client and a server. In the below example we create a connection and make multiple requests by passing different parameters with the GET request. We receive multiple responses but we also count the number of connection that has been used in the process. As we see the number of connection does not change implying the reuse of the connection. from urllib3 import HTTPConnectionPool pool = HTTPConnectionPool(”ajax.googleapis.com”, maxsize=1) r = pool.request(”GET”, ”/ajax/services/search/web”, fields={”q”: ”python”, ”v”: ”1.0”}) print ”Response Status:”, r.status # Header of the response print ”Header: ”,r.headers[”content-type”] # Content of the response print ”Python: ”,len(r.data) r = pool.request(”GET”, ”/ajax/services/search/web”, fields={”q”: ”php”, ”v”: ”1.0”}) # Content of the response print ”php: ”,len(r.data) print ”Number of Connections: ”,pool.num_connections print ”Number of requests: ”,pool.num_requests When we run the above program, we get the following output − Response Status: 200 Header: text/javascript; charset=utf-8 Python: 211 php: 211 Number of Connections: 1 Number of requests: 2 Print Page Previous Next Advertisements ”;

Python – Network Programming Python Network Programming is about using python as a programming language to handle computer networking requirements. For example, if we want to create and run a local web server or automatically download some files from a URL with a pattern. Audience This tutorial is designed for Computer Science graduates as well as Software Professionals who are willing to learn Network programming in simple and easy steps using Python as a programming language. Prerequisites Before proceeding with this tutorial, you should have a basic knowledge of writing code in Python programming language, using any python IDE and execution of Python programs. If you are completely new to python then please refer our Python tutorial to get a sound understanding of the language. Print Page Previous Next Advertisements ”;

Python – Network Environment ”; Previous Next Python 3 is available for Windows, Mac OS and most of the flavors of Linux operating system. Even though Python 2 is available for many other OSs, Python 3 support either has not been made available for them or has been dropped. Local Environment Setup Open a terminal window and type “python” to find out if it is already installed and which version is installed. Getting Python Windows platform Binaries of latest version of Python 3 (Python 3.5.1) are available on this download page The following different installation options are available. Windows x86-64 embeddable zip file Windows x86-64 executable installer Windows x86-64 web-based installer Windows x86 embeddable zip file Windows x86 executable installer Windows x86 web-based installer Note − In order to install Python 3.5.1, minimum OS requirements are Windows 7 with SP1. For versions 3.0 to 3.4.x Windows XP is acceptable. Linux platform Different flavors of Linux use different package managers for installation of new packages. On Ubuntu Linux, Python 3 is installed using the following command from the terminal. $sudo apt-get install python3-minimal Installation from source Download Gzipped source tarball from Python”s download URL − https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tgz Extract the tarball tar xvfz Python-3.5.1.tgz Configure and Install: cd Python-3.5.1 ./configure –prefix = /opt/python3.5.1 make sudo make install Mac OS Download Mac OS installers from this URL − https://www.python.org/downloads/mac-osx/ Mac OS X 64-bit/32-bit installer − python-3.5.1-macosx10.6.pkg Mac OS X 32-bit i386/PPC installer − python-3.5.1-macosx10.5.pkg Double click this package file and follow the wizard instructions to install. The most up-to-date and current source code, binaries, documentation, news, etc., is available on the official website of Python − Python Official Website − https://www.python.org/ You can download Python documentation from the following site. The documentation is available in HTML, PDF and PostScript formats. Python Documentation Website − www.python.org/doc/ Setting up PATH Programs and other executable files can be in many directories. Hence, the operating systems provide a search path that lists the directories that it searches for executables. The important features are − The path is stored in an environment variable, which is a named string maintained by the operating system. This variable contains information available to the command shell and other programs. The path variable is named as PATH in Unix or Path in Windows (Unix is case-sensitive; Windows is not). In Mac OS, the installer handles the path details. To invoke the Python interpreter from any particular directory, you must add the Python directory to your path. Setting Path at Unix/Linux To add the Python directory to the path for a particular session in Unix − In the csh shell − type setenv PATH “$PATH:/usr/local/bin/python3” and press Enter. In the bash shell (Linux) − type export PYTHONPATH=/usr/local/bin/python3.4 and press Enter. In the sh or ksh shell − type PATH=”$PATH:/usr/local/bin/python3″ and press Enter. Note − /usr/local/bin/python3 is the path of the Python directory. Setting Path at Windows To add the Python directory to the path for a particular session in Windows − At the command prompt − type path %path%;C:Python and press Enter. Note − C:Python is the path of the Python directory Running Python There are three different ways to start Python − Interactive Interpreter You can start Python from Unix, DOS, or any other system that provides you a command-line interpreter or shell window. Enter python the command line. Start coding right away in the interactive interpreter. $python # Unix/Linux or python% # Unix/Linux or C:>python # Windows/DOS Integrated Development Environment You can run Python from a Graphical User Interface (GUI) environment as well, if you have a GUI application on your system that supports Python. Unix − IDLE is the very first Unix IDE for Python. Windows − PythonWin is the first Windows interface for Python and is an IDE with a GUI. Macintosh − The Macintosh version of Python along with the IDLE IDE is available from the main website, downloadable as either MacBinary or BinHex”d files. If you are not able to set up the environment properly, then you can take the help of your system admin. Make sure the Python environment is properly set up and working perfectly fine. Note − All the examples given in subsequent chapters are executed with Python 3.4.1 version available on Windows 7 and Ubuntu Linux. We have already set up Python Programming environment online, so that you can execute all the available examples online while you are learning theory. Feel free to modify any example and execute it online. Print Page Previous Next Advertisements ”;

Python – DNS Look-up ”; Previous Next The IP addresses when translated to human readable formats or words become known as domain names. The translation of domain names to IP address is managed by the python module dnspython.This module also provides methods to find out CNAME and MX records. Finding ”A” Record In the below program we find the ip address for the domain using the dns.resolver method. Usually this mapping between IP address and domain name is also known as ”A” record. import dnspython as dns import dns.resolver result = dns.resolver.query(”tutorialspoint.com”, ”A”) for ipval in result: print(”IP”, ipval.to_text()) When we run the above program, we get the following output − (”IP”, u”94.130.81.180”) Finding CNAME Value A CNAME record also known as Canonical Name Record is a type of record in the Domain Name System (DNS) used to map a domain name as an alias for another domain. CNAME records always point to another domain name and never directly to an IP address. In the query method below we specify the CNAME parameter to get the CNAME value. import dnspython as dns import dns.resolver result = dns.resolver.query(”mail.google.com”, ”CNAME”) for cnameval in result: print ” cname target address:”, cnameval.target When we run the above program, we get the following output − cname target address: googlemail.l.google.com. Finding MX Record A MX record also called mail exchanger record is a resource record in the Domain Name System that specifies a mail server responsible for accepting email messages on behalf of a recipient”s domain. It also sets the preference value used to prioritizing mail delivery if multiple mail servers are available. Similar to above programs we can find the value for MX record using the ”MX” parameter in the query method. result = dns.resolver.query(”mail.google.com”, ”MX”) for exdata in result: print ” MX Record:”, exdata.exchange.text() When we run the above program, we get the following output − MX Record: ASPMX.L.GOOGLE.COM. MX Record: ALT1.ASPMX.L.GOOGLE.COM. MX Record: ALT2.ASPMX.L.GOOGLE.COM. The above is a sample output and not the exact one. Print Page Previous Next Advertisements ”;

Python – HTTP Requests ”; Previous Next The http or Hyper Text Transfer Protocol works on client server model. Usually the web browser is the client and the computer hosting the website is the server. IN python we use the requests module for creating the http requests. It is a very powerful module which can handle many aspects of http communication beyond the simple request and response data. It can handle authentication, compression/decompression, chunked requests etc. An HTTP client sends an HTTP request to a server in the form of a request message which includes following format: A Request-line Zero or more header (General|Request|Entity) fields followed by CRLF An empty line (i.e., a line with nothing preceding the CRLF) indicating the end of the header fields Optionally a message-body The following sections explain each of the entities used in an HTTP request message. Request-Line The Request-Line begins with a method token, followed by the Request-URI and the protocol version, and ending with CRLF. The elements are separated by space SP characters. Request-Line = Method SP Request-URI SP HTTP-Version CRLF Let”s discuss each of the parts mentioned in the Request-Line. Request Method The request method indicates the method to be performed on the resource identified by the given Request-URI. The method is case-sensitive and should always be mentioned in uppercase. The following table lists all the supported methods in HTTP/1.1. S.N. Method and Description 1 GET The GET method is used to retrieve information from the given server using a given URI. Requests using GET should only retrieve data and should have no other effect on the data. 2 HEAD Same as GET, but it transfers the status line and the header section only. 3 POST A POST request is used to send data to the server, for example, customer information, file upload, etc. using HTML forms. 4 PUT Replaces all the current representations of the target resource with the uploaded content. 5 DELETE Removes all the current representations of the target resource given by URI. 6 CONNECT Establishes a tunnel to the server identified by a given URI. 7 OPTIONS Describe the communication options for the target resource. 8 TRACE Performs a message loop back test along with the path to the target resource. Request-URI The Request-URI is a Uniform Resource Identifier and identifies the resource upon which to apply the request. Following are the most commonly used forms to specify an URI: Request-URI = “*” | absoluteURI | abs_path | authority   S.N. Method and Description 1 The asterisk * is used when an HTTP request does not apply to a particular resource, but to the server itself, and is only allowed when the method used does not necessarily apply to a resource. For example: OPTIONS * HTTP/1.1 2 The absoluteURI is used when an HTTP request is being made to a proxy. The proxy is requested to forward the request or service from a valid cache, and return the response. For example: GET http://www.w3.org/pub/WWW/TheProject.html HTTP/1.1 3 The most common form of Request-URI is that used to identify a resource on an origin server or gateway. For example, a client wishing to retrieve a resource directly from the origin server would create a TCP connection to port 80 of the host “www.w3.org” and send the following lines: GET /pub/WWW/TheProject.html HTTP/1.1 Host: www.w3.org Note that the absolute path cannot be empty; if none is present in the original URI, it MUST be given as “/” (the server root). Using Python requests We will use the module requests for learning about http request. pip install requests In the below example we see a case of simple GET request annd print out the result of the response. We choose to print only the first 300 characters. # How to make http request import requests as req r = req.get(”http://www.tutorialspoint.com/python/”) print(r.text)[0:300] When we run the above program, we get the following output − <!DOCTYPE html> <!–[if IE 8]><html class=”ie ie8″> <![endif]–> <!–[if IE 9]><html class=”ie ie9″> <![endif]–> <!–[if gt IE 9]><!–> <html> <!–<![endif]–> <head> <!– Basic –> <meta charset=”utf-8″> <title>Python Tutorial</title> <meta name=”description” content=”Python Tutorial Print Page Previous Next Advertisements ”;

Python – Internet Protocol ”; Previous Next The Internet Protocol is designed to implement a uniform system of addresses on all of the Internet-connected computers everywhere and to make it possible for packets to travel from one end of the Internet to the other. A program like the web browser should be able to connect to a host anywhere without ever knowing which maze of network devices each packet is traversing on its journey. There are various categories of internet protocols. Thes protocols are created to serve the needs of different types of data communication between different computers in the internet. Python has several modules to handle each of these communication scenarios. The methods and functions in these modules can do the simplest job of just validating a URL or also the complex job of handling the cookies and sessions. In this chapter we will look at the most prominent python modules used for internet protocols. Protocol Python Module Name Description HTTP urllib.request Opening the HTTP URL HTTP urllib.response Create a reponse object for a url request HTTP urllib.parse To break Uniform Resource Locator (URL) strings up in components like (addressing scheme, network location, path etc.), HTTP urllib.robotparser It finds out whether or not a particular user agent can fetch a URL on the Web site that published the robots.txt file. FTP ftplib implements the client side of the FTP protocol. You can use this to write Python programs that perform a variety of automated FTP jobs, such as mirroring other FTP servers. POP poplib This module defines a class, POP3, which encapsulates a connection to a POP3 server to read messages from a email server IMAP imaplib This module defines three classes, IMAP4, IMAP4_SSL and IMAP4_stream, which encapsulate a connection to an IMAP4 server to read emails. SMTP smtplib The smtplib module defines an SMTP client session object that can be used to send mail to any Internet machine with an SMTP listner deamon. Telnet telnet This module provides a Telnet class that implements the Telnet protocol to access a server thorugh teleent. Each of them is discussed in detail in the subsequent chapters. Print Page Previous Next Advertisements ”;

Python – IP Address ”; Previous Next IP Address (Internet Protocol) is a fundamental networking concept that provides address assignation capability in a network. The python module ipaddress is used extensively to validate and categorize IP address to IPV4 and IPV6 type. It can also be used to do comparison of the IP address values as well as IP address arithmetic for manipulating the ip addresses. Validate the IPV4 Address The ip_address function validates the IPV4 address. If the range of values is beyond 0 to 255, then it throws an error. print (ipaddress.ip_address(u”192.168.0.255”)) print (ipaddress.ip_address(u”192.168.0.256”)) When we run the above program, we get the following output − 192.168.0.255 ValueError: u”192.168.0.256” does not appear to be an IPv4 or IPv6 address Validate the IPV6 Address The ip_address function validates the IPV6 address. If the range of values is beyond 0 to ffff, then it throws an error. print (ipaddress.ip_address(u”FFFF:9999:2:FDE:257:0:2FAE:112D”)) #invalid IPV6 address print (ipaddress.ip_address(u”FFFF:10000:2:FDE:257:0:2FAE:112D”)) When we run the above program, we get the following output − ffff:9999:2:fde:257:0:2fae:112d ValueError: u”FFFF:10000:2:FDE:257:0:2FAE:112D” does not appear to be an IPv4 or IPv6 address Check the type of IP Address We can supply the IP address of various formats and the module will be able to recognize the valid formats. It will also indicate which category of IP address it is. print type(ipaddress.ip_address(u”192.168.0.255”)) print type(ipaddress.ip_address(u”2001:db8::”)) print ipaddress.ip_address(u”192.168.0.255”).reverse_pointer print ipaddress.ip_network(u”192.168.0.0/28”) When we run the above program, we get the following output − 255.0.168.192.in-addr.arpa 192.168.0.0/28 Comparison of IP Addresses We can make a logical comparison of the IP addresses finding out if they are equal or not. We can also compare if one IP address is greater than the other in its value. print (ipaddress.IPv4Address(u”192.168.0.2”) > ipaddress.IPv4Address(u”192.168.0.1”)) print (ipaddress.IPv4Address(u”192.168.0.2”) == ipaddress.IPv4Address(u”192.168.0.1”)) print (ipaddress.IPv4Address(u”192.168.0.2”) != ipaddress.IPv4Address(u”192.168.0.1”)) When we run the above program, we get the following output − True False True IP Addresses Arithmetic We can also apply arithmetic operations to manipulate IP addresses. We can add or subtract integers to an IP address. If after addition the value of the last octet goes beyond 255 then the previous octet gets incremented to accommodate the value. If the extra value can not be absorbed by any of the previous octet then a value error is raised. print (ipaddress.IPv4Address(u”192.168.0.2”)+1) print (ipaddress.IPv4Address(u”192.168.0.253”)-3) # Increases the previous octet by value 1. print (ipaddress.IPv4Address(u”192.168.10.253”)+3) # Throws Value error print (ipaddress.IPv4Address(u”255.255.255.255”)+1) When we run the above program, we get the following output − 192.168.0.3 192.168.0.250 192.168.11.0 AddressValueError: 4294967296 (>= 2**32) is not permitted as an IPv4 address Print Page Previous Next Advertisements ”;