Category: python Data Persistence

Python Data Persistence – Introduction ”; Previous Next Overview of Python – Data Persistence During the course of using any software application, user provides some data to be processed. The data may be input, using a standard input device (keyboard) or other devices such as disk file, scanner, camera, network cable, WiFi connection, etc. Data so received, is stored in computer’s main memory (RAM) in the form of various data structures such as, variables and objects until the application is running. Thereafter, memory contents from RAM are erased. However, more often than not, it is desired that the values of variables and/or objects be stored in such a manner, that it can be retrieved whenever required, instead of again inputting the same data. The word ‘persistence’ means “the continuance of an effect after its cause is removed”. The term data persistence means it continues to exist even after the application has ended. Thus, data stored in a non-volatile storage medium such as, a disk file is a persistent data storage. In this tutorial, we will explore various built-in and third party Python modules to store and retrieve data to/from various formats such as text file, CSV, JSON and XML files as well as relational and non-relational databases. Using Python’s built-in File object, it is possible to write string data to a disk file and read from it. Python’s standard library, provides modules to store and retrieve serialized data in various data structures such as JSON and XML. Python’s DB-API provides a standard way of interacting with relational databases. Other third party Python packages, present interfacing functionality with NOSQL databases such as MongoDB and Cassandra. This tutorial also introduces, ZODB database which is a persistence API for Python objects. Microsoft Excel format is a very popular data file format. In this tutorial, we will learn how to handle .xlsx file through Python. Print Page Previous Next Advertisements ”;

File Handling with os Module ”; Previous Next In addition to File object returned by open() function, file IO operations can also be performed using Python”s built-in library has os module that provides useful operating system dependent functions. These functions perform low level read/write operations on file. The open() function from os module is similar to the built-in open(). However, it doesn”t return a file object but a file descriptor, a unique integer corresponding to file opened. File descriptor”s values 0, 1 and 2 represent stdin, stdout, and stderr streams. Other files will be given incremental file descriptor from 2 onwards. As in case of open() built-in function, os.open() function also needs to specify file access mode. Following table lists various modes as defined in os module. Sr.No. Os Module & Description 1 os.O_RDONLY Open for reading only 2 os.O_WRONLY Open for writing only 3 os.O_RDWR Open for reading and writing 4 os.O_NONBLOCK Do not block on open 5 os.O_APPEND Append on each write 6 os.O_CREAT Create file if it does not exist 7 os.O_TRUNC Truncate size to 0 8 os.O_EXCL Error if create and file exists To open a new file for writing data in it, specify O_WRONLY as well as O_CREAT modes by inserting pipe (|) operator. The os.open() function returns a file descriptor. f=os.open(“test.dat”, os.O_WRONLY|os.O_CREAT) Note that, data is written to disk file in the form of byte string. Hence, a normal string is converted to byte string by using encode() function as earlier. data=”Hello World”.encode(”utf-8”) The write() function in os module accepts this byte string and file descriptor. os.write(f,data) Don’t forget to close the file using close() function. os.close(f) To read contents of a file using os.read() function, use following statements: f=os.open(“test.dat”, os.O_RDONLY) data=os.read(f,20) print (data.decode(”utf-8”)) Note that, the os.read() function needs file descriptor and number of bytes to be read (length of byte string). If you want to open a file for simultaneous read/write operations, use O_RDWR mode. Following table shows important file operation related functions in os module. Sr.No Functions & Description 1 os.close(fd) Close the file descriptor. 2 os.open(file, flags[, mode]) Open the file and set various flags according to flags and possibly its mode according to mode. 3 os.read(fd, n) Read at most n bytes from file descriptor fd. Return a string containing the bytes read. If the end of the file referred to by fd has been reached, an empty string is returned. 4 os.write(fd, str) Write the string str to file descriptor fd. Return the number of bytes actually written. Print Page Previous Next Advertisements ”;