Category: Big Data & Analytics

Statistics – Normal Distribution ”; Previous Next A normal distribution is an arrangement of a data set in which most values cluster in the middle of the range and the rest taper off symmetrically toward either extreme. Height is one simple example of something that follows a normal distribution pattern: Most people are of average height the numbers of people that are taller and shorter than average are fairly equal and a very small (and still roughly equivalent) number of people are either extremely tall or extremely short.Here”s an example of a normal distribution curve: A graphical representation of a normal distribution is sometimes called a bell curve because of its flared shape. The precise shape can vary according to the distribution of the population but the peak is always in the middle and the curve is always symmetrical. In a normal distribution the mean mode and median are all the same. Formula ${y = frac{1}{sqrt {2 pi}}e^{frac{-(x – mu)^2}{2 sigma}} }$ Where − ${mu}$ = Mean ${sigma}$ = Standard Deviation ${pi approx 3.14159}$ ${e approx 2.71828}$ Example Problem Statement: A survey of daily travel time had these results (in minutes): 26 33 65 28 34 55 25 44 50 36 26 37 43 62 35 38 45 32 28 34 The Mean is 38.8 minutes, and the Standard Deviation is 11.4 minutes. Convert the values to z – scores and prepare the Normal Distribution Graph. Solution: The formula for z-score that we have been using: ${z = frac{x – mu}{sigma} }$ Where − ${z}$ = the “z-score” (Standard Score) ${x}$ = the value to be standardized ${mu}$ = mean ${sigma}$ = the standard deviation To convert 26: First subtract the mean: 26-38.8 = -12.8, Then divide by the Standard Deviation: -12.8/11.4 = -1.12 So 26 is -1.12 Standard Deviation from the Mean Here are the first three conversions. Original Value Calculation Standard Score (z-score) 26 (26-38.8) / 11.4 = -1.12 33 (33-38.8) / 11.4 = -0.51 65 (65-38.8) / 11.4 = -2.30 … … … And here they graphically represent: Print Page Previous Next Advertisements ”; Tutorials Point is a leading Ed Tech company striving to provide the best learning material on technical and non-technical subjects. About us Company Our Team Careers Jobs Contact Us Terms of use Privacy Policy Refund Policy Cookies Policy FAQ”s Tutorials Point India Private Limited, Incor9 Building, Kavuri Hills, Madhapur, Hyderabad, Telangana – 500081, INDIA Tutorials Articles Jobs Courses Certifications Annual Membership Languages Python Tutorial Java Tutorial C++ Tutorial C Programming Tutorial C# Tutorial PHP Tutorial R Tutorial Go Tutorial Web Technologies HTML Tutorial CSS Tutorial JavaScript Tutorial ReactJS Tutorial Bootstrap Tutorial AngularJS Tutorial Node.js Tutorial TypeScript Tutorial Database SQL Tutorial MySQL Tutorial DBMS Tutorial MongoDB Tutorial SQLite Tutorial PL/SQL Tutorial PostgreSQL Tutorial Excel Tutorial Editors Online SQL Editor Online Html Editor Online Css Editor Online Javascript Editor Online Latext Editor Online TEX Editor Online Mathml Compiler Online Markdown Editor Trending Technologies Cloud Computing Tutorial Amazon Web Services Tutorial Microsoft Azure Tutorial Git Tutorial Ethical Hacking Tutorial Docker Tutorial Kubernetes Tutorial Compilers Online Java Compiler Online C Compiler Online C++ Compiler Online C# Compiler Online Php Compiler Online Matlab Compiler Online Bash Compiler Terminals Online Unix Terminal Online Python3 Terminal Online Php Terminal Online Nodejs Terminal Online R Terminal Online Numpy Terminal Online Octave Terminal Data Science & ML NLP Tutorial NumPy Tutorial Python Pandas Tutorial Machine Learning Tutorial Big Data Analytics Tutorial Cryptography Tutorial Power BI Tutorial Computer Science DSA Tutorial Spring Boot Tutorial SDLC Tutorial Unix Tutorial Operating System Tutorial Assembly Programming Tutorial Digital Circuits Tutorial Microprocessor Tutorial System Analysis and Design Tutorial Flutter Tutorial Top Certifications Business Analytics Certification Java & Spring Boot Advanced Certification Data Science Advanced Certification Advanced Certification In Cloud Computing And DevOps Advanced Certification In Business Analytics Artificial Intelligence And Machine Learning Certification DevOps Certification   Game Development Certification Front-End Developer Certification AWS Certification Training Python Programming Certification Generative AI Certification Microsoft Excel Certification Training Java Certification Cyber Security Certification Coding For Beginners Certification   JavaScript Certification Apache Spark Certification Advanced Python Certification Back-End Developer Certification Front-End Developer Certification Web Developer Advanced Certification Linux System Administrator Certification Automation Testing Certification Training © Copyright 2024. All Rights Reserved.

Statistics – Log Gamma Distribution ”; Previous Next Log Gamma Distribution is a probability density function with positive shape parameters $ {alpha, beta } $ and location parameter $ { mu } $. It is defined by following formula. Formula ${ f(x) = frac{e^{beta x}e^{frac{-e^x}{alpha}}}{ alpha^beta Gamma(beta)} \[7pt] , where -infty gt x lt infty }$ Where − ${alpha}$ = positive shape parameter. ${beta}$ = positive shape parameter. ${x}$ = random variable. Following diagram shows the probability density function with three different parameter combinations. Print Page Previous Next Advertisements ”;

Statistics – Probability Multiplicative Theorem ”; Previous Next For Independent Events The theorem states that the probability of the simultaneous occurrence of two events that are independent is given by the product of their individual probabilities. ${P(A and B) = P(A) times P(B) \[7pt] P (AB) = P(A) times P(B)}$ The theorem can he extended to three or more independent events also as ${P(A cap B cap C) = P(A) times P(B) times P(C) P (A,B and C) = P(A) times P(B) times P(C) }$ Example Problem Statement: A college has to appoint a lecturer who must be B.Com., MBA, and Ph. D, the probability of which is ${frac{1}{20}}$, ${frac{1}{25}}$, and ${frac{1}{40}}$ respectively. Find the probability of getting such a person to be appointed by the college. Solution: Probability of a person being a B.Com.P(A) =${frac{1}{20}}$ Probability of a person being a MBA P(B) = ${frac{1}{25}}$ Probability of a person being a Ph.D P(C) =${frac{1}{40}}$ Using multiplicative theorem for independent events ${ P (A,B and C) = P(A) times P(B) times P(C) \[7pt] = frac{1}{20} times frac{1}{25} times frac{1}{40} \[7pt] = .05 times .04 times .025 \[7pt] = .00005 }$ For Dependent Events (Conditional Probability) As defined earlier, dependent events are those were the occurrences or nonoccurrence of one event effects the outcome of next event. For such events the earlier stated multiplicative theorem is not applicable. The probability associated with such events is called as conditional probability and is given by P(A/B) = ${frac{P(AB)}{P(B)}}$ or ${frac{P(A cap B)}{P(B)}}$ Read P(A/B) as the probability of occurrence of event A when event B has already occurred. Similarly the conditional probability of B given A is P(B/A) = ${frac{P(AB)}{P(A)}}$ or ${frac{P(A cap B)}{P(A)}}$ Example Problem Statement: A coin is tossed 2 times. The toss resulted in one head and one tail. What is the probability that the first throw resulted in a tail? Solution: The sample space of a coin tossed two times is given as S = {HH, HT, TH, TT} Let Event A be the first throw resulting in a tail. Event B be that one tail and one head occurred. ${ P(A) = frac{P(TH,TT)}{P(HH,HT,TH,TT)} = frac{2}{4} =frac {1}{2} \[7pt] P(A cap B) = frac{P(TH)}{P(HH,HT,TH,TT)} =frac{1}{4} \[7pt] So P (A/B) = frac{P(A cap B)}{P(A)} \[7pt] = frac{frac{1}{4}}{frac{1}{2}} \[7pt] = frac{1}{2} = 0.5 }$ Print Page Previous Next Advertisements ”;

Statistics – Qualitative Data Vs Quantitative Data ”; Previous Next Qualitative Data Qualitative data is a set of information which can not be measured using numbers. It generally consist of words, subjective narratives. Result of an qualitative data analysis can come in form of highlighting key words, extracting information and concepts elaboration. For example, a study on parents perception about the current education system for their kids. The resulted information collected from them might be in narrative form and you need to deduce the analysis that they are satisfied, un-satisfied or need improvement in certain areas and so on. Strengh Better understanding – Qualitative data gives a better understanding of the perspectives and needs of participants. Provides Explaination – Qualitative data along with quantitative data can explain the result of the survey and can measure the correction of the quantitative data. Better Identification of behavior patterns – Qualitative data can provide detailed information which can prove itself useful in identification of behaviorial patterns. Weakness Lesser reachability – Being subjective in nature, small population is generally covered to represent the large population. Time Consuming – Qualitative data is time consuming as large data is to be understood. Possiblity of Bias – Being subjective analysis; evaluator bias is quite feasible. Quantitative Data Quantitative data is a set of numbers collected from a group of people and involves statistical analysis.For example if you conduct a satisfaction survey from participants and ask them to rate their experience on a scale of 1 to 5. You can collect the ratings and being numerical in nature, you will use statistical techniques to draw conclusions about participants satisfaction. Strengh Specific Quantitative data is clear and specific to the survey conducted. High ReliabilityIf collected properly, quantitative data is normally accurate and hence highly reliable. Easy communicationQuantitative data is easy to communicate and elaborate using charts, graphs etc. Existing supportMany large datasets may be already present that can be analyzed to check the relevance of the survey. Weakness Limited Options – Respondents are required to choose from limited options. High Complexity – Qualitative data may need complex procedures to get correct sample. Require Expertise – Analysis of qualitative data requires certain expertise in statistical analysis. Print Page Previous Next Advertisements ”;

Statistics – Process Capability (Cp) & Process Performance (Pp) ”; Previous Next Process Capability Process capability can be defined as a measurable property of a process relative to its specification. It is expressed as a process capability index ${C_p}$. The process capability index is used to check the variability of the output generated by the process and to compare the variablity with the product tolerance. ${C_p}$ is governed by following formula: Formula ${ C_p = min[frac{USL – mu}{3 times sigma}, frac{mu – LSL}{3 times sigma}] }$ Where − ${USL}$ = Upper Specification Limit. ${LSL}$ = Lower Specification Limit. ${mu}$ = estimated mean of the process. ${sigma}$ = estimated variability of the process, standard deviation. Higher the value of process capability index ${C_p}$, better is the process. Example Consider the case of a car and its parking garage. garage size states the specification limits and car defines the process output. Here process capability will tell the relatonship between car size, garage size and how far from middle of the garage you can parked the car. If car size is litter smaller than garage size then you can easily fit your car into it. If car size is very small compared to garage size then it can fit from any distance from center. In term of process of control, such process with little variation, allows to park car easily in garage and meets the customer”s requirement. Let”s see the above stated example in terms of process capability index ${C_p}$. ${C_p = frac{1}{2}}$ – garage size is smaller than car and can not accomodate your car. ${C_p = 1}$ – garage size is just sufficient for car and can accomodate your car only. ${C_p = 2}$ – garage size is two times than your car and can accomodate two cars at a time. ${C_p = 3}$ – garage size is three times than your car and can accomodate three cars at a time. Process Performance Process performance works to check the conformance of the sample generated using the process. It is expressed as a process performance index ${P_p}$. It checks whether it is meeting customer requirement or not. It varies from Process Capability in the fact that Process Performance is applicable to a particular batch of material. Sampling method may need to be quite substancial to support of the variation in the batch. Process Performance is only to be used when a process control cannot be evaluated. ${P_p}$ is governed by following formula: Formula ${ P_p = frac{USL – LSL}{6 times sigma} }$ Where − ${USL}$ = Upper Specification Limit. ${LSL}$ = Lower Specification Limit. ${sigma}$ = estimated variability of the process, standard deviation. Higher the value of process performance index ${P_p}$, better is the process. Print Page Previous Next Advertisements ”;

Statistics – Range Rule of Thumb ”; Previous Next The Range Rule of Thumb says that the range is about four times the standard deviation. The standard deviation is another measure of spread in statistics. It tells you how your data is clustered around the mean. Formula ${s approx frac{R}{4}}$ Where − ${s}$ = standard deviation. ${R}$ = Maximum – Minimum of a range. How the range rule works, we will look at the following example. Example Problem Statement: Given the following values: 12, 12, 14, 15, 16, 18, 18, 20, 20, and 25. Calculate standard deviation using range rule of thumb. Solution: These values have mean of 17. we first calculate the range of our data as 25 – 12 = 13, and then divide this number by four we have our estimate of the standard deviation as ${frac{13}{4} = 3.25}$. This number is relatively close to the true standard deviation, and good for a rough estimate. Print Page Previous Next Advertisements ”;