Category: Big Data & Analytics

Aug 10

Cognos – Creating a Chart

Cognos – Creating a Chart ”; Previous Next In Report Studio, you can create many chart types like column, bar, area, line charts or a custom chart that combines these chart types. In the Source tab, expand the query. Drag Revenue to the Measure (y-axis) drop zone. Drag Current year to the Series drop zone. Drag Order to the Categories (x-axis) drop zone. Drag the objects as shown in the above screenshot. Save the chart using the tool bar at the top. Save it to Public or My Folder as mentioned in the previous topic. Run the report to see the result in a chart format. You can also create a Repeater table or a map report in Cognos Report Studio. Print Page Previous Next Advertisements ”;

Aug 10

AVRO – Serialization

AVRO – Serialization ”; Previous Next Data is serialized for two objectives − For persistent storage To transport the data over network What is Serialization? Serialization is the process of translating data structures or objects state into binary or textual form to transport the data over network or to store on some persisten storage. Once the data is transported over network or retrieved from the persistent storage, it needs to be deserialized again. Serialization is termed as marshalling and deserialization is termed as unmarshalling. Serialization in Java Java provides a mechanism, called object serialization where an object can be represented as a sequence of bytes that includes the object”s data as well as information about the object”s type and the types of data stored in the object. After a serialized object is written into a file, it can be read from the file and deserialized. That is, the type information and bytes that represent the object and its data can be used to recreate the object in memory. ObjectInputStream and ObjectOutputStream classes are used to serialize and deserialize an object respectively in Java. Serialization in Hadoop Generally in distributed systems like Hadoop, the concept of serialization is used for Interprocess Communication and Persistent Storage. Interprocess Communication To establish the interprocess communication between the nodes connected in a network, RPC technique was used. RPC used internal serialization to convert the message into binary format before sending it to the remote node via network. At the other end the remote system deserializes the binary stream into the original message. The RPC serialization format is required to be as follows − Compact − To make the best use of network bandwidth, which is the most scarce resource in a data center. Fast − Since the communication between the nodes is crucial in distributed systems, the serialization and deserialization process should be quick, producing less overhead. Extensible − Protocols change over time to meet new requirements, so it should be straightforward to evolve the protocol in a controlled manner for clients and servers. Interoperable − The message format should support the nodes that are written in different languages. Persistent Storage Persistent Storage is a digital storage facility that does not lose its data with the loss of power supply. Files, folders, databases are the examples of persistent storage. Writable Interface This is the interface in Hadoop which provides methods for serialization and deserialization. The following table describes the methods − S.No. Methods and Description 1 void readFields(DataInput in) This method is used to deserialize the fields of the given object. 2 void write(DataOutput out) This method is used to serialize the fields of the given object. Writable Comparable Interface It is the combination of Writable and Comparable interfaces. This interface inherits Writable interface of Hadoop as well as Comparable interface of Java. Therefore it provides methods for data serialization, deserialization, and comparison. S.No. Methods and Description 1 int compareTo(class obj) This method compares current object with the given object obj. In addition to these classes, Hadoop supports a number of wrapper classes that implement WritableComparable interface. Each class wraps a Java primitive type. The class hierarchy of Hadoop serialization is given below − These classes are useful to serialize various types of data in Hadoop. For instance, let us consider the IntWritable class. Let us see how this class is used to serialize and deserialize the data in Hadoop. IntWritable Class This class implements Writable, Comparable, and WritableComparable interfaces. It wraps an integer data type in it. This class provides methods used to serialize and deserialize integer type of data. Constructors S.No. Summary 1 IntWritable() 2 IntWritable( int value) Methods S.No. Summary 1 int get() Using this method you can get the integer value present in the current object. 2 void readFields(DataInput in) This method is used to deserialize the data in the given DataInput object. 3 void set(int value) This method is used to set the value of the current IntWritable object. 4 void write(DataOutput out) This method is used to serialize the data in the current object to the given DataOutput object. Serializing the Data in Hadoop The procedure to serialize the integer type of data is discussed below. Instantiate IntWritable class by wrapping an integer value in it. Instantiate ByteArrayOutputStream class. Instantiate DataOutputStream class and pass the object of ByteArrayOutputStream class to it. Serialize the integer value in IntWritable object using write() method. This method needs an object of DataOutputStream class. The serialized data will be stored in the byte array object which is passed as parameter to the DataOutputStream class at the time of instantiation. Convert the data in the object to byte array. Example The following example shows how to serialize data of integer type in Hadoop − import java.io.ByteArrayOutputStream; import java.io.DataOutputStream; import java.io.IOException; import org.apache.hadoop.io.IntWritable; public class Serialization { public byte[] serialize() throws IOException{ //Instantiating the IntWritable object IntWritable intwritable = new IntWritable(12); //Instantiating ByteArrayOutputStream object ByteArrayOutputStream byteoutputStream = new ByteArrayOutputStream(); //Instantiating DataOutputStream object DataOutputStream dataOutputStream = new DataOutputStream(byteoutputStream); //Serializing the data intwritable.write(dataOutputStream); //storing the serialized object in bytearray byte[] byteArray = byteoutputStream.toByteArray(); //Closing the OutputStream dataOutputStream.close(); return(byteArray); } public static void main(String args[]) throws IOException{ Serialization serialization= new Serialization(); serialization.serialize(); System.out.println(); } } Deserializing the Data in Hadoop The procedure to deserialize the integer type of data is discussed below − Instantiate IntWritable class by wrapping an integer value in it. Instantiate ByteArrayOutputStream class. Instantiate DataOutputStream class and pass the object of ByteArrayOutputStream class to it. Deserialize the data in the object of DataInputStream using readFields() method of IntWritable class. The deserialized data will be stored in the object of IntWritable class. You can retrieve this data using get() method of this class. Example The following example shows how to deserialize the data of integer type in Hadoop − import java.io.ByteArrayInputStream; import java.io.DataInputStream; import org.apache.hadoop.io.IntWritable; public class Deserialization { public void deserialize(byte[]byteArray) throws Exception{ //Instantiating the IntWritable class IntWritable intwritable =new IntWritable(); //Instantiating ByteArrayInputStream object ByteArrayInputStream InputStream = new ByteArrayInputStream(byteArray); //Instantiating DataInputStream object

Aug 10

Cognos – Useful Resources

Cognos – Useful Resources ”; Previous Next The following resources contain additional information on Cognos. Please use them to get more in-depth knowledge on this. Python Programming Certification 2024 Most Popular  9 Courses     1 eBooks Tutorialspoint More Detail Artificial Intelligence and Machine Learning Certification 2024 Most Popular  7 Courses     1 eBooks Tutorialspoint More Detail Java Certification 2024 Best Seller  7 Courses     1 eBooks Tutorialspoint More Detail Print Page Previous Next Advertisements ”;

Aug 10

Cognos – Custom Calculations

Cognos – Custom Calculations ”; Previous Next You can add custom calculations to your report as per the business requirement. With the help of operators, different calculations can be added like if you want to add a new value salary*0.2 as a Bonus. To create Calculations in a Report − Select the item in the report. Click the insert calculation button and select the calculation to perform. Note − Calculations that are not applicable to the items you selected are greyed out. To change the order of the operands or the name of the calculated item added to the report, click Custom. The calculation appears as a new row or a column in your report. Drilling Drill up and drill down is used to perform analysis by moving between levels of information. Drill down is used to see more detailed information to lowest level and drill up is used to compare the results. To drill down or up in a single row or column, pause the pointer over the label text until the icon with the plus sign (+) and caret drill down drill up icon appears and the text is underlined, and then click. To drill down or up in both a row and column simultaneously, click on the value at the intersection of the row and the column, and then click again. Print Page Previous Next Advertisements ”;

Aug 10

Cognos – Save an Analysis

Cognos – Save an Analysis ”; Previous Next To save an analysis, you can click on the save button at the top as shown in the following screenshot. Enter a name of the analysis and location → then click OK. Print Page Previous Next Advertisements ”;

Aug 10

Cognos – Report Templates

Cognos – Report Templates ”; Previous Next In Report Studio, you can create different types of reports. They allow you to present the data in different formats like a list report can be used to show the customer information. The following reports can be created in Report Studio − List Report This report is used to show the data in detailed format. Data is shown in rows and columns and each column contains all the values of a data item. Quarter Order number Quantity Revenue Q4 101035 105 $4,200.00 101037 90 $8,470.80 101044 124 $11,479.92 101052 193 $15,952.42 101064 58 $5,458.96 101065 78 $7,341.36 101081 145 $5,800.00 101092 81 $7,623.72 101093 50 $4,706.00 101103 139 $5,560.00 Crosstab Like list report, a cross tab report also shows the data in row and columns, but the data is compact and not detailed. At the intersection points of rows and columns, you show the summarized data. Chart You can use the Report Studio to create many chart types, including column, bar, area, and line charts. You can also create custom charts that combine these chart types. Map You can also use maps in the Report Studio to present data for a particular region, country or a location. A map report consists of three parts − Region Layer Point Layer Display Layer Repeater Repeaters are used to add repeat items in a report while running the report. To add a Repeater, drag a repeater from the tool box to work area. Print Page Previous Next Advertisements ”;

Aug 10

Cognos – Filters

Cognos – Filters ”; Previous Next Filters are used to limit the data that you want in your report. You can apply one or more filters in a Cognos report and the report returns the data that meet the filter conditions. You can create various custom filters in a report as per the requirement. Select the column to filter by. Click the drop down list from the Filter button. Choose Create Custom Filter. The Filter Condition dialog displays. In the next window, define the filter’s parameters. Condition − click the list arrow to see your choices (Show or Don’t show the following values). Values − click the list arrow to see your choices. Keywords − allows you to search for specific values within the values list. Values List − shows the field values which you can use as filter values. You can select one or many. Use the arrow button to add multiple values. Select a value and click the right pointing arrow to move the value into the selected column. You can use the Ctrl key to add multiple values at tone time. Click OK when the filter is defined. Note − You can view filters in the Query Explorer page and not the page explorer. You can go to the query explorer and view the filters. Deleting a Filter A filter can be deleted by using the following steps − Go to the Query Explorer as shown in the above screenshot Click on Query and Locate the Detail Filters pane in the upper right side of the window as shown in above screenshot Select the filter that you want to delete and press the delete button You can also cut/copy a filter Print Page Previous Next Advertisements ”;

Aug 10

Big Data Analytics – Charts & Graphs

Big Data Analytics – Charts & Graphs ”; Previous Next The first approach to analyzing data is to visually analyze it. The objectives at doing this are normally finding relations between variables and univariate descriptions of the variables. We can divide these strategies as − Univariate analysis Multivariate analysis Univariate Graphical Methods Univariate is a statistical term. In practice, it means we want to analyze a variable independently from the rest of the data. The plots that allow to do this efficiently are − Box-Plots Box-Plots are normally used to compare distributions. It is a great way to visually inspect if there are differences between distributions. We can see if there are differences between the price of diamonds for different cut. # We will be using the ggplot2 library for plotting library(ggplot2) data(“diamonds”) # We will be using the diamonds dataset to analyze distributions of numeric variables head(diamonds) # carat cut color clarity depth table price x y z # 1 0.23 Ideal E SI2 61.5 55 326 3.95 3.98 2.43 # 2 0.21 Premium E SI1 59.8 61 326 3.89 3.84 2.31 # 3 0.23 Good E VS1 56.9 65 327 4.05 4.07 2.31 # 4 0.29 Premium I VS2 62.4 58 334 4.20 4.23 2.63 # 5 0.31 Good J SI2 63.3 58 335 4.34 4.35 2.75 # 6 0.24 Very Good J VVS2 62.8 57 336 3.94 3.96 2.48 ### Box-Plots p = ggplot(diamonds, aes(x = cut, y = price, fill = cut)) + geom_box-plot() + theme_bw() print(p) We can see in the plot there are differences in the distribution of diamonds price in different types of cut. Histograms source(”01_box_plots.R”) # We can plot histograms for each level of the cut factor variable using facet_grid p = ggplot(diamonds, aes(x = price, fill = cut)) + geom_histogram() + facet_grid(cut ~ .) + theme_bw() p # the previous plot doesn’t allow to visuallize correctly the data because of the differences in scale # we can turn this off using the scales argument of facet_grid p = ggplot(diamonds, aes(x = price, fill = cut)) + geom_histogram() + facet_grid(cut ~ ., scales = ”free”) + theme_bw() p png(”02_histogram_diamonds_cut.png”) print(p) dev.off() The output of the above code will be as follows − Multivariate Graphical Methods Multivariate graphical methods in exploratory data analysis have the objective of finding relationships among different variables. There are two ways to accomplish this that are commonly used: plotting a correlation matrix of numeric variables or simply plotting the raw data as a matrix of scatter plots. In order to demonstrate this, we will use the diamonds dataset. To follow the code, open the script bda/part2/charts/03_multivariate_analysis.R. library(ggplot2) data(diamonds) # Correlation matrix plots keep_vars = c(”carat”, ”depth”, ”price”, ”table”) df = diamonds[, keep_vars] # compute the correlation matrix M_cor = cor(df) # carat depth price table # carat 1.00000000 0.02822431 0.9215913 0.1816175 # depth 0.02822431 1.00000000 -0.0106474 -0.2957785 # price 0.92159130 -0.01064740 1.0000000 0.1271339 # table 0.18161755 -0.29577852 0.1271339 1.0000000 # plots heat-map(M_cor) The code will produce the following output − This is a summary, it tells us that there is a strong correlation between price and caret, and not much among the other variables. A correlation matrix can be useful when we have a large number of variables in which case plotting the raw data would not be practical. As mentioned, it is possible to show the raw data also − library(GGally) ggpairs(df) We can see in the plot that the results displayed in the heat-map are confirmed, there is a 0.922 correlation between the price and carat variables. It is possible to visualize this relationship in the price-carat scatterplot located in the (3, 1) index of the scatterplot matrix. Print Page Previous Next Advertisements ”;

Aug 10

Cognos – List Report

Cognos – List Report ”; Previous Next A list report that shows the data in rows and columns and each cell shows the data in the database or you can also add custom calculations in a list report. To create a new list report, go to New → Blank as shown in the following screenshot. When you select a list report, you get the following structure of the report in the Report Studio. You have to drag the objects from the package on the left side to the report structure. You can also edit the title of the report that will appear once you run the report. You can use different tools at the top for the report formatting. To save a report, click on the save button. To run a report, click on Run report. Once you save the report, you have an option to save it in the Public folder or My folder. When you click on the Run option, you can select different formats to run the report. Print Page Previous Next Advertisements ”;

Aug 10

Cognos – Home

Cognos Tutorial PDF Version Quick Guide Resources Job Search Discussion IBM Cognos Business intelligence is a web based reporting and analytic tool. It is used to perform data aggregation and create user friendly detailed reports. IBM Cognos provides a wide range of features and can be considered as an enterprise software to provide flexible reporting environment and can be used for large and medium enterprise. Cognos also provides you an option to export the report in XML or PDF format or you can view the reports in XML format. Audience IBM Cognos provides a wide range of features and can be considered as an enterprise software to provide flexible reporting environment and can be used for large and medium enterprises. It meets the needs of Power Users, Analysts, Business Managers and Company Executives. Power users and analysts want to create ad-hoc reports and can create multiple views of the same data. Business Executives want to see summarize data in dashboard styles, cross tabs and visualizations. Cognos allows both the options for all set of users. Prerequisites IBM Cognos Business Intelligence is an advanced topic. Even though the content has been prepared keeping in mind the requirements of a beginner, the reader should be familiar with the fundamentals of running and viewing reports or manage schedules, portal layouts, and other users” permissions before starting with this tutorial. Print Page Previous Next Advertisements ”;