Category: Big Data & Analytics

Statistics – One Proportion Z Test ”; Previous Next The test statistic is a z-score (z) defined by the following equation. ${z = frac{(p – P)}{sigma}}$ where P is the hypothesized value of population proportion in the null hypothesis, p is the sample proportion, and ${sigma}$ is the standard deviation of the sampling distribution. Test Statistics is defined and given by the following function: Formula ${ z = frac {hat p -p_o}{sqrt{frac{p_o(1-p_o)}{n}}} }$ Where − ${z}$ = Test statistics ${n}$ = Sample size ${p_o}$ = Null hypothesized value ${hat p}$ = Observed proportion Example Problem Statement: A survey claims that 9 out of 10 doctors recommend aspirin for their patients with headaches. To test this claim, a random sample of 100 doctors is obtained. Of these 100 doctors, 82 indicate that they recommend aspirin. Is this claim accurate? Use alpha = 0.05. Solution: Define Null and Alternative Hypotheses ${ H_0;p = .90 \[7pt] H_0;p ne .90 }$ Here Alpha = 0.05. Using an alpha of 0.05 with a two-tailed test, we would expect our distribution to look something like this: Here we have 0.025 in each tail. Looking up 1 – 0.025 in our z-table, we find a critical value of 1.96. Thus, our decision rule for this two-tailed test is: If Z is less than -1.96, or greater than 1.96, reject the null hypothesis.Calculate Test Statistic: ${ z = frac {hat p -p_o}{sqrt{frac{p_o(1-p_o)}{n}}} \[7pt] hat p = .82 \[7pt] p_o = .90 \[7pt] n = 100 \[7pt] z_o = frac {.82 – .90}{sqrt{frac{ .90 (1- .90)}{100}}} \[7pt] = frac{-.08}{0.03} \[7pt] = -2.667 }$ As z = -2.667 Thus as result we should reject the null hypothesis and as conclusion, The claim that 9 out of 10 doctors recommend aspirin for their patients is not accurate, z = -2.667, p < 0.05. Print Page Previous Next Advertisements ”;

Statistics – Multinomial Distribution ”; Previous Next A multinomial experiment is a statistical experiment and it consists of n repeated trials. Each trial has a discrete number of possible outcomes. On any given trial, the probability that a particular outcome will occur is constant. Formula ${P_r = frac{n!}{(n_1!)(n_2!)…(n_x!)} {P_1}^{n_1}{P_2}^{n_2}…{P_x}^{n_x}}$ Where − ${n}$ = number of events ${n_1}$ = number of outcomes, event 1 ${n_2}$ = number of outcomes, event 2 ${n_x}$ = number of outcomes, event x ${P_1}$ = probability that event 1 happens ${P_2}$ = probability that event 2 happens ${P_x}$ = probability that event x happens Example Problem Statement: Three card players play a series of matches. The probability that player A will win any game is 20%, the probability that player B will win is 30%, and the probability player C will win is 50%. If they play 6 games, what is the probability that player A will win 1 game, player B will win 2 games, and player C will win 3? Solution: Given: ${n}$ = 12 (6 games total) ${n_1}$ = 1 (Player A wins) ${n_2}$ = 2 (Player B wins) ${n_3}$ = 3 (Player C wins) ${P_1}$ = 0.20 (probability that Player A wins) ${P_1}$ = 0.30 (probability that Player B wins) ${P_1}$ = 0.50 (probability that Player C wins) Putting the values into the formula, we get: ${ P_r = frac{n!}{(n_1!)(n_2!)…(n_x!)} {P_1}^{n_1}{P_2}^{n_2}…{P_x}^{n_x} , \[7pt] P_r(A=1, B=2, C=3)= frac{6!}{1!2!3!}(0.2^1)(0.3^2)(0.5^3) , \[7pt] = 0.135 }$ Print Page Previous Next Advertisements ”;

Statistics – Poisson Distribution ”; Previous Next Poisson conveyance is discrete likelihood dispersion and it is broadly use in measurable work. This conveyance was produced by a French Mathematician Dr. Simon Denis Poisson in 1837 and the dissemination is named after him. The Poisson circulation is utilized as a part of those circumstances where the happening”s likelihood of an occasion is little, i.e., the occasion once in a while happens. For instance, the likelihood of faulty things in an assembling organization is little, the likelihood of happening tremor in a year is little, the mischance”s likelihood on a street is little, and so forth. All these are cases of such occasions where the likelihood of event is little. Poisson distribution is defined and given by the following probability function: Formula ${P(X-x)} = {e^{-m}}.frac{m^x}{x!}$ Where − ${m}$ = Probability of success. ${P(X-x)}$ = Probability of x successes. Example Problem Statement: A producer of pins realized that on a normal 5% of his item is faulty. He offers pins in a parcel of 100 and insurances that not more than 4 pins will be flawed. What is the likelihood that a bundle will meet the ensured quality? [Given: ${e^{-m}} = 0.0067$] Solution: Let p = probability of a defective pin = 5% = $frac{5}{100}$. We are given: ${n} = 100, {p} = frac{5}{100} , \[7pt] Rightarrow {np} = 100 times frac{5}{100} = {5}$ The Poisson distribution is given as: ${P(X-x)} = {e^{-m}}.frac{m^x}{x!}$ Required probability = P [packet will meet the guarantee] = P [packet contains up to 4 defectives] = P (0) +P (1) +P (2) +P (3) +P (4) $ = {e^{-5}}.frac{5^0}{0!} + {e^{-5}}.frac{5^1}{1!} + {e^{-5}}.frac{5^2}{2!} + {e^{-5}}.frac{5^3}{3!} +{e^{-5}}.frac{5^4}{4!}, \[7pt] = {e^{-5}}[1+frac{5}{1}+frac{25}{2}+frac{125}{6}+frac{625}{24}] , \[7pt] = 0.0067 times 65.374 = 0.438$ Print Page Previous Next Advertisements ”;

Statistics – Power Calculator ”; Previous Next Whenever a hypothesis test is conducted, we need to ascertain that test is of high qualitity. One way to check the power or sensitivity of a test is to compute the probability of test that it can reject the null hypothesis correctly when an alternate hypothesis is correct. In other words, power of a test is the probability of accepting the alternate hypothesis when it is true, where alternative hypothesis detects an effect in the statistical test. $ {Power = P( reject H_0 | H_1 is true) } $ Power of a test is also test by checking the probability of Type I error($ { alpha } $) and of Type II error($ { beta } $) where Type I error represents the incorrect rejection of a valid null hypothesis whereas Type II error represents the incorrect retention of an invalid null hypothesis. Lesser the chances of Type I or Type II error, more is the power of statistical test. Example A survey has been conducted on students to check their IQ level. Suppose a random sample of 16 students is tested. The surveyor tests the null hypothesis that the IQ of student is 100 against the alternative hypothesis that the IQ of student is not 100, using a 0.05 level of significance and standard deviation of 16. What is the power of the hypothesis test if the true population mean were 116? Solution: As distribution of the test statistic under the null hypothesis follows a Student t-distribution. Here n is large, we can approximate the t-distribution by a normal distribution. As probability of committing Type I error($ { alpha } $) is 0.05 , we can reject the null hypothesis ${H_0}$ when the test statistic $ { T ge 1.645 } $. Let”s compute the value of sample mean using test statistics by following formula. $ {T = frac{ bar X – mu}{ frac{sigma}{sqrt mu}} \[7pt] implies bar X = mu + T(frac{sigma}{sqrt mu}) \[7pt] , = 100 + 1.645(frac{16}{sqrt {16}})\[7pt] , = 106.58 } $ Let”s compute the power of statistical test by following formula. $ {Power = P(bar X ge 106.58 where mu = 116 ) \[7pt] , = P( T ge -2.36) \[7pt] , = 1- P( T lt -2.36 ) \[7pt] , = 1 – 0.0091 \[7pt] , = 0.9909 } $ So we have a 99.09% chance of rejecting the null hypothesis ${H_0: mu = 100 } $ in favor of the alternative hypothesis $ {H_1: mu gt 100 } $ where unknown population mean is $ {mu = 116 } $. Print Page Previous Next Advertisements ”;

Statistics – Process Sigma ”; Previous Next Process sigma can be defined using following four steps: Measure opportunities, Measure defects, Calculate yield, Look-up process sigma. Formulae Used ${DPMO = frac{Total defect}{Total Opportunities} times 1000000}$ ${Defect (%) = frac{Total defect}{Total Opportunities} times 100}$ ${Yield (%) = 100 – Defect (%) }$ ${Process Sigma = 0.8406+sqrt{29.37}-2.221 times (log (DPMO)) }$ Where − ${Opportunities}$ = Lowest defect noticeable by customer. ${DPMO}$ = Defects per Million Opportunities. Example Problem Statement: In equipment organization hard plate produced is 10000 and the defects is 5. Discover the process sigma. Solution: Given: Opportunities = 10000 and Defects = 5. Substitute the given qualities in the recipe, Step 1: Compute DPMO $ {DPMO = frac{Total defect}{Total Opportunities} times 1000000 \[7pt] , = (10000/5) times 1000000 , \[7pt] , = 500}$ Step 2: Compute Defect(%) $ {Defect (%) = frac{Total defect}{Total Opportunities} times 100 \[7pt] , = frac{10000}{5} times 100 , \[7pt] , = 0.05}$ Step 3: Compute Yield(%) $ {Yield (%) = 100 – Defect (%) \[7pt] , = 100 – 0.05 , \[7pt] , = 99.95}$ Step 3: Compute Process Sigma $ {Process Sigma = 0.8406+sqrt{29.37}-2.221 times (log (DPMO)) \[7pt] , = 0.8406 + sqrt {29.37} – 2.221 times (log (DPMO)) , \[7pt] , = 0.8406+sqrt(29.37) – 2.221*(log (500)) , \[7pt] , = 4.79 }$ Print Page Previous Next Advertisements ”;

Statistics – Harmonic Resonance Frequency ”; Previous Next Harmonic Resonance Frequency represents a signal or wave whose frequency is an integral multiple of the frequency of a reference signal or wave. Formula ${ f = frac{1}{2 pi sqrt{LC}} } $ Where − ${f}$ = Harmonic resonance frequency. ${L}$ = inductance of the load. ${C}$ = capacitanc of the load. Example Calculate the harmonic resonance frequency of a power system with the capcitance 5F, Inductance 6H and frequency 200Hz. Solution: Here capacitance, C is 5F. Inductance, L is 6H. Frequency, f is 200Hz. Using harmonic resonance frequency formula, let”s compute the resonance frequency as: ${ f = frac{1}{2 pi sqrt{LC}} \[7pt] implies f = frac{1}{2 pi sqrt{6 times 5}} \[7pt] , = frac{1}{2 times 3.14 times sqrt{30}} \[7pt] , = frac{1}{ 6.28 times 5.4772 } \[7pt] , = frac{1}{ 34.3968 } \[7pt] , = 0.0291 }$ Thus harmonic resonance frequency is $ { 0.0291 }$. Print Page Previous Next Advertisements ”;

Statistics – Grand Mean ”; Previous Next When sample sizes are equal, in other words, there could be five values in each sample, or n values in each sample. The grand mean is the same as the mean of sample means. Formula ${X_{GM} = frac{sum x}{N}}$ Where − ${N}$ = Total number of sets. ${sum x}$ = sum of the mean of all sets. Example Problem Statement: Determine the mean of each group or set”s samples. Use the following data as a sample to determine the mean and grand mean. Jackson 1 6 7 10 4 Thomas 5 2 8 14 6 Garrard 8 2 9 12 7 Solution: Step 1: Compute all means $ {M_1 = frac{1+6+7+10+4}{5} = frac{28}{5} = 5.6 \[7pt] , M_2 = frac{5+2+8+14+6}{5} = frac{35}{5} = 7 \[7pt] , M_3 = frac{8+2+9+12+7}{5} = frac{38}{5} = 7.6 }$ Step 2: Divide the total by the number of groups to determine the grand mean. In the sample, there are three groups. $ {X_{GM} = frac{5.6+7+7.6}{3} = frac{20.2}{3} \[7pt] , = 6.73 }$ Print Page Previous Next Advertisements ”;

Statistics – Signal to Noise Ratio ”; Previous Next Sign to-commotion proportion (contracted SNR) is a measure utilized as a part of science and designing that analyzes the level of a coveted sign to the level of foundation clamor. It is characterized as the proportion of sign energy to the clamor power, regularly communicated in decibels. A proportion higher than 1:1 (more prominent than 0 dB) shows more flag than clamor. While SNR is regularly cited for electrical signs, it can be connected to any type of sign, (for example, isotope levels in an ice center or biochemical motioning between cells). Signal-to-noise ratio is defined as the ratio of the power of a signal (meaningful information) and the power of background noise (unwanted signal): ${SNR = frac{P_{signal}}{P_{noise}}}$ If the variance of the signal and noise are known, and the signal is zero: ${SNR = frac{sigma^2_{signal}}{sigma^2_{noise}}}$ If the signal and the noise are measured across the same impedance, then the SNR can be obtained by calculating the square of the amplitude ratio: ${SNR = frac{P_{signal}}{P_{noise}} = {(frac{A_{signal}}{A_{noise}})}^2} $ Where A is root mean square (RMS) amplitude (for example, RMS voltage). Decibels Because many signals have a very wide dynamic range, signals are often expressed using the logarithmic decibel scale. Based upon the definition of decibel, signal and noise may be expressed in decibels (dB) as ${P_{signal,dB} = 10log_{10}(P_{signal})} $ and ${P_{noise,dB} = 10log_{10}(P_{noise})} $ In a similar manner, SNR may be expressed in decibels as ${SNR_{dB} = 10log_{10}(SNR)} $ Using the definition of SNR ${SNR_{dB} = 10log_{10}(frac{P_{signal}}{P_{noise}})} $ Using the quotient rule for logarithms ${10log_{10}(frac{P_{signal}}{P_{noise}}) = 10log_{10}(P_{signal}) – 10log_{10}(P_{noise})} $ Substituting the definitions of SNR, signal, and noise in decibels into the above equation results in an important formula for calculating the signal to noise ratio in decibels, when the signal and noise are also in decibels: ${SNR_{dB} = P_{signal,dB} – P_{noise,dB}} $ In the above formula, P is measured in units of power, such as Watts or mill watts, and signal-to-noise ratio is a pure number. However, when the signal and noise are measured in Volts or Amperes, which are measures of amplitudes, they must be squared to be proportionate to power as shown below: ${SNR_{dB} = 10log_{10}[{(frac{A_{signal}}{A_{noise}})}^2] \[7pt] = 20log_{10}(frac{A_{signal}}{A_{noise}}) \[7pt] = A_{signal,dB} – A_{noise,dB}} $ Example Problem Statement: Compute the SNR of a 2.5 kHz sinusoid sampled at 48 kHz. Add white noise with standard deviation 0.001. Set the random number generator to the default settings for reproducible results. Solution: ${ F_i = 2500; F_s = 48e3; N = 1024; \[7pt] x = sin(2 times pi times frac{F_i}{F_s} times (1:N)) + 0.001 times randn(1,N); \[7pt] SNR = snr(x,Fs) \[7pt] SNR = 57.7103}$ Print Page Previous Next Advertisements ”;

Statistics – Hypothesis testing ”; Previous Next A statistical hypothesis is an assumption about a population which may or may not be true. Hypothesis testing is a set of formal procedures used by statisticians to either accept or reject statistical hypotheses. Statistical hypotheses are of two types: Null hypothesis, ${H_0}$ – represents a hypothesis of chance basis. Alternative hypothesis, ${H_a}$ – represents a hypothesis of observations which are influenced by some non-random cause. Example suppose we wanted to check whether a coin was fair and balanced. A null hypothesis might say, that half flips will be of head and half will of tails whereas alternative hypothesis might say that flips of head and tail may be very different. $ H_0: P = 0.5 \[7pt] H_a: P ne 0.5 $ For example if we flipped the coin 50 times, in which 40 Heads and 10 Tails results. Using result, we need to reject the null hypothesis and would conclude, based on the evidence, that the coin was probably not fair and balanced. Hypothesis Tests Following formal process is used by statistican to determine whether to reject a null hypothesis, based on sample data. This process is called hypothesis testing and is consists of following four steps: State the hypotheses – This step involves stating both null and alternative hypotheses. The hypotheses should be stated in such a way that they are mutually exclusive. If one is true then other must be false. Formulate an analysis plan – The analysis plan is to describe how to use the sample data to evaluate the null hypothesis. The evaluation process focuses around a single test statistic. Analyze sample data – Find the value of the test statistic (using properties like mean score, proportion, t statistic, z-score, etc.) stated in the analysis plan. Interpret results – Apply the decisions stated in the analysis plan. If the value of the test statistic is very unlikely based on the null hypothesis, then reject the null hypothesis. Print Page Previous Next Advertisements ”;

Statistics – Harmonic Mean ”; Previous Next What is Harmonic Mean? Harmonic Mean is also a mathematical average but is limited in its application. It is generally used to find average of variables that are expressed as a ratio of two different measuring units e. g. speed is measured in km/hr or miles/sec etc. Weighted Harmonic Mean Formula $H.M. = frac{W}{sum (frac{W}{X})}$ Where − ${H.M.}$ = Harmonic Mean ${W}$ = Weight ${X}$ = Variable value Example Problem Statement: Find the weighted H.M. of the items 4, 7,12,19,25 with weights 1, 2,1,1,1 respectively. Solution: ${X}$ ${W}$ $frac{W}{X}$ 4 1 0.2500 7 2 0.2857 12 1 0.0833 19 1 0.0526 25 1 0.0400   $sum W$ $sum frac{W}{X}$= 0.7116 Based on the above mentioned formula, Harmonic Mean $G.M.$ will be: $H.M. = frac{W}{sum (frac{W}{X})} \[7pt] , = frac{6}{0.7116} \[7pt] , = 8.4317 $ ∴ Weighted H.M = 8.4317 We”re going to discuss methods to compute the Harmonic Mean for three types of series: Individual Data Series Discrete Data Series Continuous Data Series Individual Data Series When data is given on individual basis. Following is an example of individual series: Items 5 10 20 30 40 50 60 70 Discrete Data Series When data is given alongwith their frequencies. Following is an example of discrete series: Items 5 10 20 30 40 50 60 70 Frequency 2 5 1 3 12 0 5 7 Continuous Data Series When data is given based on ranges alongwith their frequencies. Following is an example of continous series: Items 0-5 5-10 10-20 20-30 30-40 Frequency 2 5 1 3 12 Print Page Previous Next Advertisements ”;