Category: Big Data & Analytics

Splunk – Basic Chart ”; Previous Next Splunk has great visualization features which shows a variety of charts. These charts are created from the results of a search query where appropriate functions are used to give numerical outputs. For example, if we look for the average file size in bytes from the data set named web_applications, we can see the result in the statistics tab as shown below − Creating Charts In order to create a basic chart, we first ensure that the data is visible in the statistics tab as shown above. Then we click on the Visualization tab to get the corresponding chart. The above data produces a pie chart by default as shown below. Changing the Chart Type We can change the chart type by selecting a different chart option from the chart name. Clicking on one of these options will produce the chart for that type of graph. Formatting a Chart The charts can also be formatted by using the Format option. This option allows to set the values for the axes, set the legends or show the data values in the chart. In the below example, we have chosen the horizontal chart and selected the option to show the data values as a Format option. Print Page Previous Next Advertisements ”;

Splunk – Lookups ”; Previous Next In the result of a search query, we sometimes get values which may not clearly convey the meaning of the field. For example, we may get a field which lists the value of product id as a numeric result. These numbers will not give us any idea of what kind of product it is. But if we list the product name along with the product id, that gives us a good report where we understand the meaning of the search result. Such linking of values of one field to a field with same name in another dataset using equal values from both the data sets is called a lookup process. The advantage is, we retrieve the related values from two different data sets. Steps to Create and Use Lookup File In order to successfully create a lookup field in a dataset, we need to follow the below steps − Create Lookup File We consider the dataset with host as web_application, and look at the productid field. This field is just a number, but we want product names to be reflected in our query result set. We create a lookup file with the following details. Here, we have kept the name of the first field as productid which is same as the field we are going to use from the dataset. productId,productdescription WC-SH-G04,Tablets DB-SG-G01,PCs DC-SG-G02,MobilePhones SC-MG-G10,Wearables WSC-MG-G10,Usb Light GT-SC-G01,Battery SF-BVS-G01,Hard Drive Add the Lookup File Next, we add the lookup file to Splunk environment by using the Settings screens as shown below − After selecting the Lookups, we are presented with a screen to create and configure lookup. We select lookup table files as shown below. We browse to select the file productidvals.csv as our lookup file to be uploaded and select search as our destination app. We also keep the same destination file name. On clicking the save button, the file gets saved to the Splunk repository as a lookup file. Create Lookup Definitions For a search query to be able to lookup values from the Lookup file we just uploaded above, we need to create a lookup definition. We do this by again going to Settings → Lookups → Lookup Definition → Add New . Next, we check the availability of the lookup definition we added by going to Settings → Lookups → Lookup Definition . Selecting Lookup Field Next, we need to select the lookup field for our search query. This is done my going to New search → All Fields . Then check the box for productid which will automatically add the productdescription field from the lookup file also. Using the Lookup Field Now we use the Lookup field in the search query as shown below. The visualization shows the result with productdescription field instead of productid. Print Page Previous Next Advertisements ”;

Splunk – Knowledge Management ”; Previous Next Splunk knowledge management is about maintenance of knowledge objects for a Splunk Enterprise implementation. Below are the main features of knowledge management − Ensure that knowledge objects are being shared and used by the right groups of people in the organization. Normalize event data by implementing knowledge object naming conventions and retiring duplicate or obsolete objects. Oversee strategies for improved search and pivot performance (report acceleration, data model acceleration, summary indexing, batch mode search). Build data models for Pivot users. Knowledge Object It is a Splunk object to get specific information about your data. When you create a knowledge object, you can keep it private or you can share it with other users. The examples of knowledge object are: saved searches, tags, field extractions, lookups, etc. Uses of Knowledge Objects On using the Splunk software, the knowledge objects are created and saved. But they may contain duplicate information, or they may not be used effectively by all the intended audience. To address such issues, we need to manage these objects. This is done by classifying them properly and then using proper permission management to handle them. Below are the uses and classification of various knowledge objects − Fields and field extractions Fields and field extractions is the first layer of Splunk software knowledge. The fields automatically extracted from the Splunk software from the IT data help bring meaning to the raw data. The manually extracted fields expand and improve upon this layer of meaning. Event types and transactions Use event types and transactions to group together interesting sets of similar events. Event types group together sets of events discovered through searches. Transactions are collections of conceptually-related events that span time. Lookups and workflow actions Lookups and workflow actions are categories of knowledge objects that extend the usefulness of your data in various ways. Field lookups enable you to add fields to your data from external data sources such as static tables (CSV files) or Python-based commands. Workflow actions enable interactions between fields in your data and other applications or web resources, such as a WHOIS lookup on a field containing an IP address. Tags and aliases Tags and aliases are used to manage and normalize sets of field information. You can use tags and aliases to group sets of related field values together, and to give extracted field tags that reflect different aspects of their identity. For example, you can group events from set of hosts in a particular location (such as a building or city) together by giving the same tag to each host. If you have two different sources using different field names to refer to same data, then you can normalize your data by using aliases (by aliasing clientip to ipaddress, for example). Data models Data models are representations of one or more datasets, and they drive the Pivot tool, enabling Pivot users to quickly generate useful tables, complex visualizations, and robust reports without needing to interact with the Splunk software search language. Data models are designed by knowledge managers who fully understand the format and semantics of their indexed data. A typical data model makes use of other knowledge object types. We will discuss some of the examples of these knowledge objects in the subsequent chapters. Print Page Previous Next Advertisements ”;

Splunk – Event Types ”; Previous Next In Splunk search, we can design our own events from a dataset based on certain criteria. For example, we search for only the events which have a http status code of 200. This event now can be saved as an event type with a user defined name as status200 and use this event name as part of future searches. In short, an event type represents a search that returns a specific type of event or a useful collection of events. Every event that can be returned by the search gets an association with that event type. Creating Event Type There are two ways to create an event type after we have decided the search criteria. One is to run a search and then save it as an Event Type. Another is to add a new Event Type from the settings tab. We will see both the ways of creating it in this section. Using a Search Consider the search for the events which have the criteria of successful http status value of 200 and the event type run on a Wednesday. After running the search query, we can choose Save As option to save the query as an Event Type. The next screen prompts to give a name for the Event Type, choose a Tag which is optional and then choose a colour with which the events will be highlighted. The priority option decides which event type will be displayed first in case two or more event types match the same event. Finally, we can see the Event Type has been created by going to the Settings → Event Types option. Using New Event Type The other option to create a new Event Type is to use the Settings → Event Types option as shown below where we can add a new Event Type − On clicking the button New Event Type we get the following screen to add the same query as in the previous section. Viewing the Event Type To view the event we just created above, we can write the below search query in the search box and we can see the resulting events along with the colour we have chosen for the event type. Using the Event Type We can use the Event type along with other queries. Here we specify some partial criteria from the Event Type and the result is a mix of events which shows the coloured and non-coloured events in the result. Print Page Previous Next Advertisements ”;