Statistical Distributions Guide: The Shapes That Data Takes

Discrete Distributions

Discrete distributions assign probabilities to countable outcomes (integers: 0, 1, 2, 3...). Each possible value has a specific probability, and the sum of all probabilities equals one.

The binomial distribution models the number of successes in n independent trials, each with the same probability p of success. Flipping a coin 20 times and counting heads, testing 100 manufactured items for defects, or counting how many of 50 patients respond to treatment all follow binomial models. Parameters: n (number of trials) and p (probability of success). Mean = np, variance = np(1-p). As n grows large, the binomial approaches the normal distribution, which is why normal approximations work for large-sample proportion tests.

The Poisson distribution models the count of events occurring in a fixed interval of time or space when events happen independently at a constant average rate. Examples include the number of emails received per hour, radioactive decay events per second, or car accidents at an intersection per month. Single parameter: lambda (the average rate). Mean = lambda, variance = lambda. The Poisson is useful when events are rare relative to opportunities and independent of each other. When lambda is large (above 20), the Poisson closely approximates the normal distribution.

The geometric distribution models the number of trials needed to achieve the first success. How many times must you roll a die to get a 6? How many products must be inspected to find the first defect? Parameter: p (probability of success on each trial). Mean = 1/p. The negative binomial distribution generalizes this to the number of trials needed for a specified number of successes, making it useful for modeling overdispersed count data where the variance exceeds the mean (unlike the Poisson, where mean and variance are equal).

Continuous Distributions

Continuous distributions assign probabilities to intervals of values rather than to individual points, because the probability of any single exact value is zero for a continuous variable. Probabilities are calculated as areas under the probability density curve.

The normal (Gaussian) distribution is symmetric and bell-shaped, defined by mean (mu) and standard deviation (sigma). It arises whenever a measured quantity results from many small, independent additive effects. The central limit theorem ensures that sample means follow a normal distribution regardless of the underlying population shape, making normality the most important distribution in inferential statistics. The standard normal distribution (mean = 0, standard deviation = 1) serves as the reference for z-scores and probability calculations.

The t-distribution resembles the normal but has heavier tails, meaning more probability in extreme regions. It arises when estimating a population mean from a small sample because the sample standard deviation introduces additional uncertainty beyond what the normal distribution accounts for. Defined by degrees of freedom (df = n - 1 for a one-sample problem), the t-distribution becomes indistinguishable from the normal as df exceeds about 30. All t-tests and regression coefficient tests use this distribution.

The chi-square distribution is the distribution of a sum of squared standard normal variables. It is always positive and right-skewed, becoming less skewed as degrees of freedom increase. It appears in chi-square tests of categorical data, tests of variance, and as a building block of the F-distribution used in ANOVA. Defined by degrees of freedom, with mean = df and variance = 2*df.

The F-distribution is the ratio of two chi-square variables divided by their respective degrees of freedom. It is the basis for ANOVA F-tests and tests comparing two variances. Always positive and right-skewed, it is defined by two parameters (df of numerator and df of denominator). When the F-statistic is large, it indicates that between-group variance substantially exceeds within-group variance, suggesting that the group means differ more than random chance would produce.

The Weibull distribution generalizes the exponential by adding a shape parameter that models increasing or decreasing failure rates. When the shape parameter equals 1, the Weibull reduces to the exponential (constant failure rate). When it is greater than 1, the failure rate increases over time, modeling aging or wear. When less than 1, the failure rate decreases, modeling infant mortality in manufactured components. This flexibility makes the Weibull the standard distribution for reliability engineering and survival analysis.

Time-to-Event and Other Distributions

The exponential distribution models the time between events in a Poisson process (events occurring at a constant rate). It answers questions like how long until the next customer arrives, how long until a machine fails, or how long between earthquakes. Parameter: lambda (the rate). Mean = 1/lambda. It has the memoryless property: the probability of waiting another t units is the same regardless of how long you have already waited, which makes it appropriate for modeling lifetimes of components that do not deteriorate with age.

The uniform distribution assigns equal probability to all values in a specified range. Rolling a fair die produces a discrete uniform distribution. Random number generators produce continuous uniform distributions between 0 and 1. It represents maximum ignorance about where in a range a value will fall and serves as the starting point for generating random samples from other distributions.

The beta distribution is defined on the interval [0, 1] and models proportions, probabilities, and percentages. Its two shape parameters allow it to take on a wide variety of forms: symmetric, left-skewed, right-skewed, U-shaped, or uniform. In Bayesian statistics, the beta distribution is commonly used as a prior distribution for probability parameters because it is the conjugate prior for the binomial likelihood, meaning the posterior is also a beta distribution.

Choosing the Right Distribution

Selecting the appropriate distribution depends on your data type and the process generating it. Count data (0, 1, 2...) often follows Poisson or negative binomial distributions. Binary outcomes follow binomial. Continuous measurements that result from many additive factors follow the normal. Time-to-event data follows exponential, Weibull, or gamma distributions. Proportions follow beta distributions. Matching your data to the correct distribution ensures valid inference and accurate probability calculations.

Diagnostic tools help verify distributional assumptions. Q-Q plots compare the quantiles of your data to theoretical quantiles from a candidate distribution, with points falling along a straight line indicating good fit. Goodness-of-fit tests (chi-square, Kolmogorov-Smirnov, Anderson-Darling) formally test whether observed data could plausibly come from a specified distribution. Histograms provide visual evidence of shape, including skewness and the number of modes. When no standard distribution fits well, nonparametric methods avoid distributional assumptions entirely.

Remember that distribution choice is a modeling decision, not an absolute truth about the data. Real-world data rarely follows any theoretical distribution exactly. The goal is to find a distribution that captures the essential features of the data well enough to produce valid statistical inferences. When multiple distributions fit reasonably well, the choice between them may depend on interpretability, mathematical convenience, or established conventions in your field.

Relationships Between Distributions

Statistical distributions are interconnected through mathematical relationships that explain why certain distributions appear in specific tests. The chi-square distribution is the sum of squared standard normals, which is why chi-square tests involve squared differences. The F-distribution is the ratio of two chi-square distributions, which is why ANOVA uses F-tests to compare variances. The t-distribution is the ratio of a standard normal to the square root of a chi-square divided by its degrees of freedom, connecting it to both the normal and chi-square families.

Many distributions converge to the normal distribution under specific conditions. The binomial approaches normality when np and n(1-p) are both at least 5. The Poisson approaches normality when lambda exceeds about 20. The t-distribution approaches the standard normal as degrees of freedom increase. These convergence properties explain why the normal distribution is so central to statistics: it serves as the large-sample approximation for many other distributions, simplifying calculations and unifying seemingly different procedures.