Category: python Forensics

Python Forensics – Indicators of Compromise ”; Previous Next Indicators of Compromise (IOC) is defined as “pieces of forensic data, which includes data found in system log entries or files, that identify potentially malicious activity on a system or network.” By monitoring for IOC, organizations can detect attacks and act quickly to prevent such breaches from occurring or limit damages by stopping attacks in earlier stages. There are some use-cases, which allow querying the forensic artifacts such as − Looking for a specific file by MD5 Searching for a specific entity, which is actually stored in the memory Specific entry or set of entries, which is stored in Windows registry The combination of all the above provides better results in searching artifacts. As mentioned above, Windows registry gives a perfect platform in generating and maintaining IOC, which directly helps in computational forensics. Methodology Look for the locations in the file system and specifically for now into Windows registry. Search for the set of artifacts, which have been designed by forensic tools. Look for the signs of any adverse activities. Investigative Life Cycle Investigative Life Cycle follows IOC and it searches for specific entries in a registry. Stage 1: Initial Evidence − Evidence of the compromise is detected either on a host or on the network. The responders will investigate and identify the exact solution, which is a concrete forensic indicator. Stage 2: Create IOCs for Host & Network − Following the data collected, the IOC is created, which is easily possible with Windows registry. The flexibility of OpenIOC gives limitless number of permutations on how an Indicator can be crafted. Stage 3: Deploy IOCs in the Enterprise − Once the specified IOC has been created, the investigator will deploy these technologies with the help of API in Windows registers. Stage 4: Identification of Suspects − The deployment of IOC helps in the identification of suspects in a normal way. Even additional systems will be identified. Stage 5: Collect and Analyze Evidence − Evidences against the suspects are gathered and analyzed accordingly. Stage 6: Refine & Create New IOCs − The investigative team can create new IOCs based of their evidences and data found in the enterprise and additional intelligence, and continue to refine their cycle. The following illustration shows the phases of Investigative Life Cycle − Print Page Previous Next Advertisements ”;

Python Forensics – Useful Resources ”; Previous Next The following resources contain additional information on Python Forensics. Please use them to get more in-depth knowledge on this. Useful Video Courses Python Flask and SQLAlchemy ORM 22 Lectures 1.5 hours Jack Chan More Detail Python and Elixir Programming Bundle Course 81 Lectures 9.5 hours Pranjal Srivastava More Detail TKinter Course – Build Python GUI Apps 49 Lectures 4 hours John Elder More Detail A Beginner”s Guide to Python and Data Science 81 Lectures 8.5 hours Datai Team Academy More Detail Deploy Face Recognition Project With Python, Django, And Machine Learning Best Seller 93 Lectures 6.5 hours Srikanth Guskra More Detail Professional Python Web Development with Flask 80 Lectures 12 hours Stone River ELearning More Detail Print Page Previous Next Advertisements ”;

Python Forensics – Network Time Protocol ”; Previous Next The most widely used protocol for synchronizing time and which has been widely accepted as a practice is done through Network Time Protocol (NTP). NTP uses the User Datagram Protocol (UDP) which uses minimum time to communicate the packets between the server and the client who wish to synchronize with the given time source. Features of Network Time Protocol are as follows − The default server port is 123. This protocol consists of many accessible time servers synchronized to national laboratories. The NTP protocol standard is governed by the IETF and the Proposed Standard is RFC 5905, titled “Network Time Protocol Version 4: Protocol and Algorithms Specification” [NTP RFC] Operating systems, programs, and applications use NTP to synchronize time in a proper way. In this chapter, we will focus on the usage of NTP with Python, which is feasible from third-party Python Library ntplib. This library efficiently handles the heavy lifting, which compares the results to my local system clock. Installing the NTP Library The ntplib is available for download at https://pypi.python.org/pypi/ntplib/ as shown in the following figure. The library provides a simple interface to NTP servers with the help of methods that can translate NTP protocol fields. This helps access other key values such as leap seconds. The following Python program helps in understanding the usage of NTP. import ntplib import time NIST = ”nist1-macon.macon.ga.us” ntp = ntplib.NTPClient() ntpResponse = ntp.request(NIST) if (ntpResponse): now = time.time() diff = now-ntpResponse.tx_time print diff; The above program will produce the following output. The difference in time is calculated in the above program. These calculations help in forensic investigations. The network data obtained is fundamentally different than the analysis of data found on the hard drive. The difference in time zones or getting accurate time zones can help in gathering evidence for capturing the messages through this protocol. Print Page Previous Next Advertisements ”;

Python Forensics – Memory and Forensics ”; Previous Next In this chapter, we will focus on investigating the volatile memory with the help of Volatility, a Python-based forensics framework applicable on the following platforms: Android and Linux. Volatile Memory Volatile memory is a type of storage where the contents get erased when the system”s power is turned off or interrupted. RAM is the best example of a volatile memory. It means, if you were working on a document that has not been saved to a non-volatile memory, such as a hard drive, and the computer lost power, then all the data will be lost. In general, volatile memory forensics follow the same pattern as other forensic investigations − Selecting the target of the investigation Acquiring forensic data Forensic analysis The basic volatility plugins which are used for Android gathers RAM dump for analysis. Once the RAM dump is gathered for analysis, it is important to start hunting for malware in RAM. YARA Rules YARA is a popular tool which provides a robust language, is compatible with Perl-based Regular Expressions, and is used to examine the suspected files/directories and match strings. In this section, we will use YARA based on the pattern matching implementation and combine them with utility power. The complete process will be beneficial for forensic analysis. Example Consider the following code. This code helps in extracting the code. import operator import os import sys sys.path.insert(0, os.getcwd()) import plyara.interp as interp # Plyara is a script that lexes and parses a file consisting of one more Yara # rules into a python dictionary representation. if __name__ == ”__main__”: file_to_analyze = sys.argv[1] rulesDict = interp.parseString(open(file_to_analyze).read()) authors = {} imps = {} meta_keys = {} max_strings = [] max_string_len = 0 tags = {} rule_count = 0 for rule in rulesDict: rule_count += 1 # Imports if ”imports” in rule: for imp in rule[”imports”]: imp = imp.replace(”””,””) if imp in imps: imps[imp] += 1 else: imps[imp] = 1 # Tags if ”tags” in rule: for tag in rule[”tags”]: if tag in tags: tags[tag] += 1 else: tags[tag] = 1 # Metadata if ”metadata” in rule: for key in rule[”metadata”]: if key in meta_keys: meta_keys[key] += 1 else: meta_keys[key] = 1 if key in [”Author”, ”author”]: if rule[”metadata”][key] in authors: authors[rule[”metadata”][key]] += 1 else: authors[rule[”metadata”][key]] = 1 #Strings if ”strings” in rule: for strr in rule[”strings”]: if len(strr[”value”]) > max_string_len: max_string_len = len(strr[”value”]) max_strings = [(rule[”rule_name”], strr[”name”], strr[”value”])] elif len(strr[”value”]) == max_string_len: max_strings.append((rule[”rule_name”], strr[”key”], strr[”value”])) print(“nThe number of rules implemented” + str(rule_count)) ordered_meta_keys = sorted(meta_keys.items(), key = operator.itemgetter(1), reverse = True) ordered_authors = sorted(authors.items(), key = operator.itemgetter(1), reverse = True) ordered_imps = sorted(imps.items(), key = operator.itemgetter(1), reverse = True) ordered_tags = sorted(tags.items(), key = operator.itemgetter(1), reverse = True) The above code will produce the following output. The number of YARA rules implemented helps in giving a better picture of the suspected files. Indirectly, the list of suspected files help in gathering appropriate information for forensics. Following is the source code in github: https://github.com/radhikascs/Python_yara Print Page Previous Next Advertisements ”;

Discuss Python Forensics ”; Previous Next Python has built-in capabilities to support digital investigation and protect the integrity of evidence during an investigation. In this tutorial, we will explain the fundamental concepts of applying Python in computational (digital) forensics that includes extracting evidence, collecting basic data, and encryption of passwords as required. Print Page Previous Next Advertisements ”;

Python Forensics – Quick Guide ”; Previous Next Python Forensics – Introduction Python is a general-purpose programming language with easy, readable code that can be easily understood by both professional developers as well as novice programmers. Python comprises of many useful libraries that can be used with any stack framework. Many laboratories rely on Python to build basic models for predictions and to run experiments. It also helps to control critical operational systems. Python has built-in capabilities to support digital investigation and protect the integrity of evidence during an investigation. In this tutorial, we will explain the fundamental concepts of applying Python in digital or computation forensics. What is Computational Forensics? Computational Forensics is an emerging research domain. It deals with solving forensic problems using digital methods. It uses computational science to study digital evidence. Computation Forensics includes a broad range of subjects which has objects, substances, and processes investigated, mainly based on pattern evidence, such as toolmarks, fingerprints, shoeprints, documents etc., and also includes physiological and behavioral patterns, DNA, and digital evidence at crime scenes. The following diagram shows the broad range of subjects covered under Computational Forensics. Computational forensics is implemented with the help of some algorithms. These algorithms are used for signal and image processing, computer vision and graphics. It also includes data mining, machine learning, and robotics. Computational forensics involves diverse digital methods. The best solution to ease all digital methods in forensics is to use a general-purpose programming language like Python. Python Forensics – Installation of Python As we need Python for all the activities of computational forensics, let us move step by step and understand how to install it. Step 1 − Go to https://www.python.org/downloads/ and download the installation files of Python according to the Operating System you have on your system. Step 2 − After downloading the package/installer, click on the exe file to start the installation process. You will get to see the following screen after the installation is complete. Step 3 − The next step is to set the environment variables of Python in your system. Step 4 − Once the environment variables are set, type the command “python” on the command prompt to verify whether the installation was successful or not. If the installation was successful, then you will get the following output on the console. Python Forensics – Overview of Python The codes written in Python look quite similar to the codes written in other conventional programming languages such as C or Pascal. It is also said that the syntax of Python is heavily borrowed from C. This includes many of the Python keywords which are similar to C language. Python includes conditional and looping statements, which can be used to extract the data accurately for forensics. For flow control, it provides if/else, while, and a high-level for statement that loops over any “iterable” object. if a < b: max = b else: max = a The major area where Python differs from other programming languages is in its use of dynamic typing. It uses variable names that refer to objects. These variables need not be declared. Data Types Python includes a set of built-in data types such as strings, Boolean, numbers, etc. There are also immutable types, which means the values which cannot be changed during the execution. Python also has compound built-in data types that includes tuples which are immutable arrays, lists, and dictionaries which are hash tables. All of them are used in digital forensics to store values while gathering evidence. Third-party Modules and Packages Python supports groups of modules and/or packages which are also called third-party modules (related code grouped together in a single source file) used for organizing programs. Python includes an extensive standard library, which is one of the main reasons for its popularity in computational forensics. Life Cycle of Python Code At first, when you execute a Python code, the interpreter checks the code for syntax errors. If the interpreter discovers any syntax errors, then they are displayed immediately as error messages. If there are no syntax errors, then the code is compiled to produce a bytecode and sent to PVM (Python Virtual Machine). The PVM checks the bytecode for any runtime or logical errors. In case the PVM finds any runtime errors, then they are reported immediately as error messages. If the bytecode is error-free, then the code gets processed and you get its output. The following illustration shows in a graphical manner how the Python code is first interpreted to produce a bytecode and how the bytecode gets processed by the PVM to produce the output. Python Forensics – Basic Forensic Application For creating an application as per the Forensic guidelines, it is important to understand and follow its naming conventions and patterns. Naming Conventions During the development of Python forensics applications, the rules and conventions to be followed are described in the following table. Constants Uppercase with underscore separation HIGH_TEMPERATURE Local variable name Lowercase with bumpy caps (underscores are optional) currentTemperature Global variable name Prefix gl lowercase with bumpy caps (underscores are optional) gl_maximumRecordedTemperature Functions name Uppercase with bumpy caps (underscores optional) with active voice ConvertFarenheitToCentigrade(…) Object name Prefix ob_ lowercase with bumpy caps ob_myTempRecorder Module An underscore followed by lowercase with bumpy caps _tempRecorder Class names Prefix class_ then bumpy caps and keep brief class_TempSystem Let us take a scenario to understand the importance of naming conventions in Computational Forensics. Suppose we have a hashing algorithm that is normally used for encrypting data. The one-way hashing algorithm takes input as a stream of binary data; this could be a password, a file, binary data, or any digital data. The hashing algorithm then produces a message digest (md) with respect to the data received in the input. It is practically impossible to create a new binary input that will generate a given message digest. Even a single bit of the binary input data, if changed, will generate a unique message, which is different than the

Python Forensics – Multiprocessing Support ”; Previous Next Forensic specialists normally find it difficult to apply digital solutions to analyze the mountains of digital evidence in common crimes. Most digital investigation tools are single threaded and they can execute only one command at a time. In this chapter, we will focus on the multiprocessing capabilities of Python, which can relate to the common forensic challenges. Multiprocessing Multiprocessing is defined as the computer system”s ability to support more than one process. The operating systems that support multiprocessing enable several programs to run concurrently. There are various types of multiprocessing such as symmetric and asymmetric processing. The following diagram refers to a symmetric multiprocessing system which is usually followed in forensic investigation. Example The following code shows how different processes are listed internally in Python programming. import random import multiprocessing def list_append(count, id, out_list): #appends the count of number of processes which takes place at a time for i in range(count): out_list.append(random.random()) if __name__ == “__main__”: size = 999 procs = 2 # Create a list of jobs and then iterate through # the number of processes appending each process to # the job list jobs = [] for i in range(0, procs): out_list = list() #list of processes process1 = multiprocessing.Process( target = list_append, args = (size, i, out_list)) # appends the list of processes jobs.append(process) # Calculate the random number of processes for j in jobs: j.start() #initiate the process # After the processes have finished execution for j in jobs: j.join() print “List processing complete.” Here, the function list_append() helps in listing the set of processes in the system. Output Our code will produce the following output − Print Page Previous Next Advertisements ”;

Python Forensics in Linux ”; Previous Next The major concern of digital investigations is to secure important evidences or data with encryption or any other format. The basic example is storing the passwords. It is therefore necessary to understand the usage of Linux operating system for digital forensic implementation to secure these valuable data. Information for all the local users are mostly stored in the following two files − /etc/passwd etc/shadow The first one is mandatory, which stores all the passwords. The second file is optional and it stores information about the local users including the hashed passwords. Issues arise regarding the security issue of storing the password information in a file, which is readable by every user. Therefore, hashed passwords are stored in /etc/passwd, where the content is replaced by a special value “x“. The corresponding hashes have to be looked up in /etc/shadow. The settings in /etc/passwd may override the details in /etc/shadow. Both the text files in Linux include one entry per line and the entry consists of multiple fields, separated by colons. The format of /etc/passwd is as follows − Sr.No. Field Name & Description 1 Username This field consists of the attributes of human-readable format 2 Password hash It consists of the password in an encoded form according to the Posix crypt function If the hash password is saved as empty, then the corresponding user will not require any password to log into the system. If this field contains a value that cannot be generated by the hash algorithm, such as an exclamation mark, then the user cannot log on using a password. A user with a locked password can still log on using other authentication mechanisms, for example, SSH keys. As mentioned earlier, the special value “x” means that the password hash has to be found in the shadow file. The password hash includes the following − Encrypted salt − The encrypted salt helps maintain the screen locks, pins, and passwords. Numerical user ID − This field denotes the ID of the user. The Linux kernel assigns this user ID to the system. Numerical group ID − This field refers to the primary group of the user. Home directory − The new processes are started with a reference of this directory. Command shell − This optional field denotes the default shell that is to be started after a successful login to the system. Digital forensics include collecting the information which is relevant to tracking an evidence. Hence, the user ids are useful in maintaining the records. Using Python, all of this information can be automatically analyzed for the Indicators of Analysis, reconstructing the recent system activity. Tracking is simple and easy with the implementation of Linux Shell. Python Programming with Linux Example import sys import hashlib import getpass def main(argv): print ”nUser & Password Storage Program in Linux for forensic detection v.01n” if raw_input(”The file ” + sys.argv[1] + ” will be erased or overwrite if it exists .nDo you wish to continue (Y/n): ”) not in (”Y”,”y”) : sys.exit(”nChanges were not recordedn”) user_name = raw_input(”Please Enter a User Name: ”) password = hashlib.sha224(getpass.getpass(”Please Enter a Password:”)).hexdigest() # Passwords which are hashed try: file_conn = open(sys.argv[1],”w”) file_conn.write(user_name + ”n”) file_conn.write(password + ”n”) file_conn.close() except: sys.exit(”There was a problem writing the passwords to file!”) if __name__ == “__main__”: main(sys.argv[1:]) Output The password is stored in a hexadecimal format in pass_db.txt as shown in the following screenshot. The text files are saved for further use in computational forensics. Print Page Previous Next Advertisements ”;

Python Forensics – Network Forensics ”; Previous Next The scenario of modern network environments is such that investigating can be fraught due to a number of difficulties. This can happen whether you are responding to a breach support, investigating insider activities, performing assessments related to vulnerability, or validating a regulatory compliance. Concept of Network Programming The following definitions are used in network programming. Client − Client is a part of client-server architecture of network programming which runs on a personal computer and workstation. Server − The server is a part of client-server architecture that provides services to other computer programs in the same or other computers. WebSockets − WebSockets provide a protocol between the client and the server, which runs over a persistent TCP connection. Through this, bi-directional messages can be sent between the TCP socket connection (simultaneously). WebSockets come after many other technologies that allow the servers to send information to the client. Other than handshaking the Upgrade Header, WebSockets is independent from HTTP. These protocols are used to validate the information which is sent or received by the third party users. As encryption is one of the methods used for securing messages, it is also important to secure the channel through which the messages have been transferred. Consider the following Python program, which the client uses for handshaking. Example # client.py import socket # create a socket object s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # get local machine name host = socket.gethostname() port = 8080 # connection to hostname on the port. s.connect((host, port)) # Receive no more than 1024 bytes tm = s.recv(1024) print(“The client is waiting for connection”) s.close() Output It will produce the following output − The server accepting the request for communication channel will include the following script. # server.py import socket import time # create a socket object serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # get local machine name host = socket.gethostname() port = 8080 # bind to the port serversocket.bind((host, port)) # queue up to 5 requests serversocket.listen(5) while True: # establish a connection clientsocket,addr = serversocket.accept() print(“Got a connection from %s” % str(addr)) currentTime = time.ctime(time.time()) + “rn” clientsocket.send(currentTime.encode(”ascii”)) clientsocket.close() The client and server created with the help of Python programming listen to the host number. Initially, the client sends a request to the server with respect to data sent in the host number and the server accepts the request and sends a response immediately. This way, we can have a secure channel of communication. Print Page Previous Next Advertisements ”;

Python Forensics – Dshell and Scapy ”; Previous Next DShell Dshell is a Python-based network forensic analysis toolkit. This toolkit was developed by the US Army Research Laboratory. The release of this open source toolkit was in the year 2014. The major focus of this toolkit is to make forensic investigations with ease. The toolkit consists of large number of decoders which are listed in the following table. Sr.No. Decoder Name & Description 1 dns This is used to extract DNS related queries 2 reservedips Identifies the solutions for DNS problems 3 large-flows Listing of the netflows 4 rip-http It is used extract the files from the HTTP traffic 5 Protocols Used for identification of non-standard protocols The US Army Laboratory has maintained the clone repository in GitHub in the following link − https://github.com/USArmyResearchLab/Dshell The clone consists of a script install-ubuntu.py () used for installation of this toolkit. Once the installation is successful, it will automatically build the executables and dependencies that will be used later. The dependencies are as follows − dependencies = { “Crypto”: “crypto”, “dpkt”: “dpkt”, “IPy”: “ipy”, “pcap”: “pypcap” } This toolkit can be used against the pcap (packet capture) files, which is usually recorded during the incidents or during the alert. These pcap files is either created by libpcap on Linux platform or WinPcap on Windows platform. Scapy Scapy is a Python-based tool used to analyze and manipulate the network traffic. Following is the link for Scapy toolkit − http://www.secdev.org/projects/scapy/ This toolkit is used to analyze packet manipulation. It is very capable to decode packets of a wide number of protocols and capture them. Scapy differs from the Dshell toolkit by providing a detailed description to the investigator about network traffic. These descriptions have been recorded in real time. Scapy has the ability to plot using third-party tools or OS fingerprinting. Consider the following example. import scapy, GeoIP #Imports scapy and GeoIP toolkit from scapy import * geoIp = GeoIP.new(GeoIP.GEOIP_MEMORY_CACHE) #locates the Geo IP address def locatePackage(pkg): src = pkg.getlayer(IP).src #gets source IP address dst = pkg.getlayer(IP).dst #gets destination IP address srcCountry = geoIp.country_code_by_addr(src) #gets Country details of source dstCountry = geoIp.country_code_by_addr(dst) #gets country details of destination print src+”(“+srcCountry+”) >> “+dst+”(“+dstCountry+”)n” This script gives the detailed description of the country details in the network packet, who are communicating with each other. The above script will produce the following output. Print Page Previous Next Advertisements ”;