The calculator below implements the most known statistical test, namely, the Independent Samples t-test or Two samples t-test. t-test, also known as Student's t-test, after William Sealy Gosset. "Student" was his pen name.
The test deals with the null hypothesis such that the means of two populations are equal. To put it in other words, the difference we find between the means of the two samples should not significantly differ from zero.
Again, the test works only if certain assumptions are met. These are:
- That the two samples are independently and randomly drawn from the source population(s).
- That the scale of measurement for both samples has the properties of an equal-interval scale.
- That the source population(s) can be reasonably supposed to have a normal distribution.
- And, for this particular implementation of the test, that the variance of each population is the same
The calculator displays a level of confidence for both directional and non-directional tests. Let's say you get the result of 96%. Essentially this means that you have 96% confidence that the obtained difference shows something more than simple luck. The chance that you can get the obtained difference and the means of the two samples are the same is only 4%. This is the level of significance you calculate. Now, depending on your chosen level of significance, you can reject or fail to reject your null hypothesis.
To estimate the confidence we need to calculate t-value, and then lookup the inverse of CDF of Student's t-distribution with degrees of freedom. is the size of sample A and is the size of sample B.
To find t-value you start from calculating the mean and sum of squared deviations, or sum of squares for each sample.
Then you estimate the variance of the source population as
This estimation is called pooled variance, and it is a method for estimating the variance of several different populations when the mean of each population may be different. Still, one may assume that the variance of each population is the same.
Then you estimate the standard deviation of the sampling distribution of sample-mean differences (the "standard error" of ) as
Finally, you calculate t as
If you care to find more, you can read excellent explanations here, starting from Chapter 9.