Data Transformations

Data Transformations

Purpose

Is to transform data that is not normally distributed into data that follows a normal distribution, thus allowing us to calculate basic statistics and valid probabilities related to the population (mean, standard deviation, Z values, etc.).

Reference:     Juran’s Quality Control Handbook - Ch. 23, P. 91-94

Terminology

A.   Mathematical transformations - Some of the commonly used

B.   Range of variable studied.

C.  Original distribution of the variable.

D.  Resulting distribution after applying a mathematical transformation.

( By CLT , the more data , the more likely normal your distribution will be, thus by collecting more data , the need of data transformation may not be required )

What will you do if your data failed normality test in Minitab.

If the data is not normally distributed (e.g. fails the normality test in Minitab) conduct the following steps:

1.   Examine the data to see if there is a nonstatistical explanation for the unusual distribution. It is wise to have a simple histogram analysis on the distribution to get the idea or pattern for the distribution under study . From histogram you can understand ,whether the collected data from various sources (similar machines or individuals performing the same process) has a different mean or standard deviation, thus resulted the combined output of the sources will have an unusual distribution such as a mixture of the individual distributions ; example binomial distribution. In this case, separate analyses could be made for each source (individual, machine, etc.).

2.   Analyze the data in terms of averages instead of individual values. In minitab, you can analyze by using Graphical Summary, it will give the output not only 95% CI for the mean of the distribution , but also 95% CI for the median of the distribution. If the 95% CI for mean and median are overlapping , we can consider the sample averages closely follow a normal distribution even if the population of individual values from which the sample averages came is not normally distributed.

3.   If steps 1 and 2 do not provide with reliable estimates, use the Weibull distribution. The resulting straight line can provide estimates of the probabilities for the population.

4.   If all above steps fail in providing reliable estimates, use one of the most common mathematical transformations as below: