Category: Big Data & Analytics

Statistics – Odd and Even Permutation ”; Previous Next Consider X as a finite set of at least two elements then permutations of X can be divided into two category of equal size: even permutation and odd permutation. Odd Permutation Odd permutation is a set of permutations obtained from odd number of two element swaps in a set. It is denoted by a permutation sumbol of -1. For a set of n numbers where n > 2, there are ${frac {n!}{2}}$ permutations possible. For example, for n = 1, 2, 3, 4, 5, …, the odd permutations possible are 0, 1, 3, 12, 60 and so on… Example Compute the odd permutation for the following set: {1,2,3,4}. Solution: Here n = 4, thus total no. of odd permutation possible are ${frac {4!}{2} = frac {24}{2} = 12}$. Following are the steps to generate odd permutations. Step 1: Swap two numbers one time. Following are the permutations obtainable: ${ { 2, 1, 3, 4 } \[7pt] { 1, 3, 2, 4 } \[7pt] { 1, 2, 4, 3 } \[7pt] { 3, 2, 1, 4 } \[7pt] { 4, 2, 3, 1 } \[7pt] { 1, 4, 3, 2 } }$ Step 2: Swap two numbers three times. Following are the permutations obtainable: ${ { 2, 3, 4, 1 } \[7pt] { 2, 4, 1, 3 } \[7pt] { 3, 1, 4, 2 } \[7pt] { 3, 4, 2, 1 } \[7pt] { 4, 1, 2, 3 } \[7pt] { 4, 3, 1, 2 } }$ Even Permutation Even permutation is a set of permutations obtained from even number of two element swaps in a set. It is denoted by a permutation sumbol of +1. For a set of n numbers where n > 2, there are ${frac {n!}{2}}$ permutations possible. For example, for n = 1, 2, 3, 4, 5, …, the even permutations possible are 0, 1, 3, 12, 60 and so on… Example Compute the even permutation for the following set: {1,2,3,4}. Solution: Here n = 4, thus total no. of even permutation possible are ${frac {4!}{2} = frac {24}{2} = 12}$. Following are the steps to generate even permutations. Step 1: Swap two numbers zero time. Following is the permutation obtainable: ${ { 1, 2, 3, 4 } }$ Step 2: Swap two numbers two times. Following are the permutations obtainable: ${ { 1, 3, 4, 2 } \[7pt] { 1, 4, 2, 3 } \[7pt] { 2, 1, 4, 3 } \[7pt] { 2, 3, 1, 4 } \[7pt] { 2, 4, 3, 1 } \[7pt] { 3, 1, 2, 4 } \[7pt] { 3, 2, 4, 1 } \[7pt] { 3, 4, 1, 2 } \[7pt] { 4, 1, 3, 2 } \[7pt] { 4, 2, 1, 3 } \[7pt] { 4, 3, 2, 1 } }$ Print Page Previous Next Advertisements ”;

Statistics – Required Sample Size ”; Previous Next A critical part of testing is the choice of the measure of test i.e. the quantity of units to be chosen from the populace for completing the exploration. There is no unequivocal answer or answer for characterizing the most suitable size. There are sure misguided judgments with respect to the span of test like the example ought to be 10% of the populace or the specimen size is relative to the extent of the universe. However as said before, these are just misguided judgments. How extensive a specimen ought to be is capacity of the variety in the populace parameters under study and the assessing exactness required by the specialist. The decision on optimum size of the sample can be approached from two angles viz. the subjective and mathematical. Subjective Approach to Determining Sample Size Mathematical Approach to Sample Size Determination Subjective Approach to Determining Sample Size The choice of the size of sample is affected by various factors discussed as below: The Nature of Population – The level of homogeneity or heterogeneity influences the extent of a specimen. On the off chance that the populace is homogeneous concerning the qualities of interest then even a little size of the specimen is adequate. However in the event that the populace is heterogeneous then a bigger example would be required to guarantee sufficient representativeness. Nature of Respondent – If the respondents are effortlessly accessible and available then required data can be got from a little example. On the off chance that, notwithstanding, the respondents are uncooperative and non-reaction is relied upon to be high then a bigger specimen is required. Nature of Study – A onetime study can be led utilizing a substantial example. If there should be an occurrence of examination studies which are of constant nature and are to be seriously completed, a little specimen is more suitable as it is anything but difficult to oversee and hold a little example over a long compass of time. Sampling Technique Used – An essential variable affecting the span of test is the examining system received. Firstly a non-likelihood system requires a bigger specimen than a likelihood strategy. Besides inside of likelihood testing, if straightforward irregular examining is utilized it requires a bigger example than if stratification is utilized, where a little specimen is adequate. Complexity of Tabulation – While settling on the specimen estimate the specialist ought to likewise consider the quantity of classifications and classes into which the discoveries are to be assembled and broke down. It has been seen that more the quantity of classifications that are to be produced the bigger is the example size. Since every class ought to be enough spoken to, a bigger specimen is required to give solid measures of the littlest classification. Availability of Resources – The assets and the time accessible to specialist impact the span of test. Examination is a period and cash escalated assignment, with exercises like readiness of instrument, contracting and preparing field staff, transportation costs and so forth taking up a considerable measure of assets. Subsequently if the scientist does not have enough time and supports accessible he will settle on a littler example. Degree of Precision and Accuracy Required – . It has turned out to be clear from our prior discourse that accuracy, which is measured by standard blunder, wills high just if S.E is less or the example size is substantial. Also to get a high level of precision a bigger specimen is required. Other then these subjective efforts, sample size can be determined mathematically also. Mathematical Approach to Sample Size Determination In the mathematical approach to sample size determination the precision of estimate required is stated first and then the sample size is worked out. The precision can be specified as ${pm}$ 1 of the true mean with 99% confidence level. This means that if the sample mean is 200, then the true value of the mean will be between 199 and 201. This level of precision is denoted by the term ”c” Sample Size determination for means. The confidence interval for the universe mean is given by ${bar x pm Zfrac{sigma_p}{sqrt N} or bar x pm e}$ Where − ${bar x}$ = Sample mean ${e}$ = Acceptable error ${Z}$ = Value of standard normal variate at a given confidence level ${sigma_p}$ = Standard deviation of the population ${n}$ = Size of the sample The acceptable error ”e” i.e. the difference between ${mu}$ and ${bar x}$ is given by ${Z.frac{sigma_p}{sqrt N}}$ Thus, Size of the sample is:> ${n = frac{Z^2{sigma_p}^2}{e^2}}$ Or In case the sample size is significant visa-a-vis the population size then above formula will be corrected by the finite population multiplier. ${n = frac{Z^2.N.{sigma_p}^2}{(N-1)e^2 + Z^2.{sigma_p}^2}}$ Where − ${N}$ = size of the population Sample Size Determination for Proportions The method for determining the sample size when estimating a proportion remains the same as the method for estimating the mean. The confidence interval for universe proportion ${hat p}$ is given by ${ p pm Z. sqrt{frac{p.q}{n}}}$ Where − ${p}$ = sample proportion ${q = (1 – p)}$ ${Z}$ = Value of standard normal variate for a sample proportion ${n}$ = Size of the sample Since ${ hat p}$ is to be estimated hence the value of p can be determined by taking the value of p = 0.5, an acceptable value, giving a conservative sample size. The other option is that the value of p is estimated either through a pilot study or on a personal judgement basis. Given the value of p, the acceptable error ”e” is given by ${ e= Z. sqrt{frac{p.q}{n}} \[7pt] e^2 = Z^2frac{p.q}{n} \[7pt] n = frac{Z^2.p.q}{e^2}}$ In case the population is finite then the above formula will be corrected by the finite population multiplier. ${n = frac{Z^2.p.q.N}{e^2(N-1) + Z^2.p.q}}$ Example Problem Statement: A shopping store is interested in estimating the proportion of households possessing the store Privilege Membership card. Previous studies have shown

Statistics – Student T Test ”; Previous Next T-test is small sample test. It was developed by William Gosset in 1908. He published this test under the pen name of “Student”. Therefore, it is known as Student”s t-test. For applying t-test, the value of t-statistic is computed. For this, the following formula is used: Formula ${t} = frac{Deviation from the population parameter}{Standard Error of the sample statistic}$ Where − ${t}$ = Test of Hypothesis. Test of Hypothesis about population Formula ${t} ={bar X – frac{mu}{S}.sqrt{n}} , \[7pt] , where {S} = sqrt{frac{sum{(X-bar X)}^2}{n-1}}$ Example Problem Statement: An irregular sample of 9 qualities from an ordinary populace demonstrated a mean of 41.5 inches and the entirety of square of deviation from this mean equivalent to 72 inches. Show whether the supposition of mean of 44.5 inches in the populace is reasonable.(For ${v}={8}, {t_.05}={2.776}$) Solution: ${bar x = 45.5}, {mu = 44.5}, {n=9}, {sum{(X-bar X)}^2 = 72} $ Let us take the null hypothesis that the population mean is 44.5. $ i.e. {H_0: mu = 44.5} and {H_1: mu ne 44.5} , \[7pt] {S} = sqrt{frac{sum{(X-bar X)}^2}{n-1}}, \[7pt] = sqrt{frac{72}{9-1}} = sqrt{frac{72}{8}} = sqrt{9} = {3}$ Applying t-test: $ {|t|} = {bar X – frac{mu}{S}.sqrt{n}} , \[7pt] {|t|} = frac{|41.5 – 44.5|}{3} times sqrt {9}, \[7pt] = {3}$ Degrees of freedom = $ {v = n-1 = 9-1 = 8 }$. For ${v = 8, t_{0.05}}$ for two tailed test = ${2.306}$. Since, the calculated value of $ {|t|}$ > the table value of $ {t}$, we reject the null hypothesis. We conclude that the population mean is not equal to 44.5. Print Page Previous Next Advertisements ”;

Statistics – Sampling methods ”; Previous Next Sampling methods are the ways to choose people from the population to be considered in a sample survey. Samples can be divided based on following criteria. Probability samples – In such samples, each population element has a known probability or chance of being chosen for the sample. Non-probability samples – In such samples, one can not be assured of having known probility of each population element. Probability sampling methods Probability sampling methods ensures that the sample choosen represent the population correctly and the survey conducted will be statistically valid. Following are the types of probability sampling methods: Simple random sampling. – This method refers to a method having following properties: The population have N objects. The sample have n objects. All possible samples of n objects have equal probability of occurence. One example of simple random sampling is lottery method. Assign each population element a unique number and place the numbers in bowl.Mix the numbers throughly. A blind-folded researcher is to select n numbers. Include those population element in the sample whose number has been selected. Stratified sampling – In this type of sampling method, population is divided into groups called strata based on certain common characteristic like geography. Then samples are selected from each group using simple random sampling method and then survey is conducted on people of those samples. Cluster sampling – In this type of sampling method, each population member is assigned to a unique group called cluster. A sample cluster is selected using simple random sampling method and then survey is conducted on people of that sample cluster. Multistage sampling – In such case, combination of different sampling methods at different stages. For example, at first stage, cluster sampling can be used to choose clusters from population and then sample random sampling can be used to choose elements from each cluster for the final sample. Systematic random sampling – In this type of sampling method, a list of every member of population is created and then first sample element is randomly selected from first k elements. Thereafter, every kth element is selected from the list. Non-probability sampling methods Non-probability sampling methods are convenient and cost-savvy. But they do not allow to estimate the extent to which sample statistics are likely to vary from population parameters. Whereas probability sampling methods allows that kind of analysis. Following are the types of non-probability sampling methods: Voluntary sample – In such sampling methods, interested people are asked to get involved in a voluntary survey. A good example of voluntary sample in on-line poll of a news show where viewers are asked to participate. In voluntary sample, viewers choose the sample, not the one who conducts survey. Convenience sample – In such sampling methods, surveyor picks people who are easily available to give their inputs. For example, a surveyer chooses a cinema hall to survey movie viewers. If the cinema hall was selected on the basis that it was easier to reach then it is a convenience sampling method. Print Page Previous Next Advertisements ”;

Statistics – Reliability Coefficient ”; Previous Next A measure of the accuracy of a test or measuring instrument obtained by measuring the same individuals twice and computing the correlation of the two sets of measures. Reliability Coefficient is defined and given by the following function: Formula ${Reliability Coefficient, RC = (frac{N}{(N-1)}) times (frac{(Total Variance – Sum of Variance)}{Total Variance})}$ Where − ${N}$ = Number of Tasks Example Problem Statement: An undertaking was experienced with three Persons (P) and they are assigned with three distinct Tasks (T). Discover the Reliability Coefficient? P0-T0 = 10 P1-T0 = 20 P0-T1 = 30 P1-T1 = 40 P0-T2 = 50 P1-T2 = 60 Solution: Given, Number of Students (P) = 3 Number of Tasks (N) = 3. To Find, Reliability Coefficient, follow the steps as following: Step 1 Give us a chance to first figure the average score of the persons and their tasks The average score of Task (T0) = 10 + 20/2 = 15 The average score of Task (T1) = 30 + 40/2 = 35 The average score of Task (T2) = 50 + 60/2 = 55 Step 2 Next, figure the variance for: Variance of P0-T0 and P1-T0: Variance = square (10-15) + square (20-15)/2 = 25 Variance of P0-T1 and P1-T1: Variance = square (30-35) + square (40-35)/2 = 25 Variance of P0-T2 and P1-T2: Variance = square (50-55) + square (50-55)/2 = 25 Step 3 Presently, figure the individual variance of P0-T0 and P1-T0, P0-T1 and P1-T1, P0-T2 and P1-T2. To ascertain the individual variance value, we ought to include all the above computed change values. Total of Individual Variance = 25+25+25=75 Step 4 Compute the Total change Variance= square ((P0-T0) – normal score of Person 0) = square (10-15) = 25 Variance= square ((P1-T0) – normal score of Person 0) = square (20-15) = 25 Variance= square ((P0-T1) – normal score of Person 1) = square (30-35) = 25 Variance= square ((P1-T1) – normal score of Person 1) = square (40-35) = 25 Variance= square ((P0-T2) – normal score of Person 2) = square (50-55) = 25 Variance= square ((P1-T2) – normal score of Person 2) = square (60-55) = 25 Now, include every one of the qualities and figure the aggregate change Total Variance= 25+25+25+25+25+25 = 150 Step 5 At last, substitute the qualities in the underneath offered equation to discover ${Reliability Coefficient, RC = (frac{N}{(N-1)}) times (frac{(Total Variance – Sum of Variance)}{Total Variance}) \[7pt] = frac{3}{(3-1)} times frac{(150-75)}{150} \[7pt] = 0.75 }$ Print Page Previous Next Advertisements ”;

Statistics – Shannon Wiener Diversity Index ”; Previous Next In the literature, the terms species richness and species diversity are sometimes used interchangeably. We suggest that at the very least, authors should define what they mean by either term. Of the many species diversity indices used in the literature, the Shannon Index is perhaps most commonly used. On some occasions it is called the Shannon-Wiener Index and on other occasions it is called the Shannon-Weaver Index. We suggest an explanation for this dual use of terms and in so doing we offer a tribute to the late Claude Shannon (who passed away on 24 February 2001). Shannon-Wiener Index is defined and given by the following function: ${ H = sum[(p_i) times ln(p_i)] }$ Where − ${p_i}$ = proportion of total sample represented by species ${i}$. Divide no. of individuals of species i by total number of samples. ${S}$ = number of species, = species richness ${H_{max} = ln(S)}$ = Maximum diversity possible ${E}$ = Evenness = ${frac{H}{H_{max}}}$ Example Problem Statement: The samples of 5 species are 60,10,25,1,4. Calculate the Shannon diversity index and Evenness for these sample values. Sample Values (S) = 60,10,25,1,4 number of species (N) = 5 First, let us calculate the sum of the given values. sum = (60+10+25+1+4) = 100 Species ${(i)}$ No. in sample ${p_i}$ ${ln(p_i)}$ ${p_i times ln(p_i)}$ Big bluestem 60 0.60 -0.51 -0.31 Partridge pea 10 0.10 -2.30 -0.23 Sumac 25 0.25 -1.39 -0.35 Sedge 1 0.01 -4.61 -0.05 Lespedeza 4 0.04 -3.22 -0.13 S = 5 Sum = 100     Sum = -1.07 ${H = 1.07 \[7pt] H_{max} = ln(S) = ln(5) = 1.61 \[7pt] E = frac{1.07}{1.61} = 0.66 \[7pt] Shannon diversity index(H) = 1.07 \[7pt] Evenness =0.66 }$ Print Page Previous Next Advertisements ”;