Data Analysis

Successful data analysis is sometimes hampered due to a lack of normality in the sample data.  There are several reasons why sample data are not normally distributed with the most frequent being outliers.  The word outliers here is not being used in the statistical sense that an outlier test shows the point(s) as being outliers but rather just points that don’t fit with the rest of the data and cause the test of normality to have low p value.

We often lose sight in discussions when samples are found to be not normally distributed that it isn’t the sample that’s of interest, it’s the population.  We use sample statistics to infer population parameters. If the sample isn’t normally distributed, we need to ask whether the sample represents the population and it’s that the population isn’t normally distributed or is the population normally distributed and only the sample isn’t?  

Remember, when testing for normality 10% of samples drawn from a normal distribution will have a p value less than 0.1, 5% will have a p value less than 0.05, and 1% will have a p value less than 0.1.  That’s the definition of p value.

What is also important to remember is that different tests of normality are sensitive to different aspects of normality.  The Anderson Darling test is sensitive to a bimodal data or a separation in the data; while the Shapiro Wilks test is more sensitive to outliers.

Do you have a statistical project or problem you need help with? Call Statisitical Manufacting Solutions to get professional data analysis consulting services. We are located in Columbia, SC but serve clients all over the country. 

What Our Clients Are Saying