Regression analysis

Generalized Linear Models

Linear regression predicts continuous outcomes, such as sales revenue. But how do we predict whether a customer will churn, or how many orders a store will receive? Generalized Linear Models (GLMs) extend the same basic framework to these different outcomes. By choosing an appropriate probability distribution and link function, we can model continuous values, probabilities or counts within one framework. This post builds on the distributions from Normal Distribution and Four Common Discrete Distributions and shows how they fit into regression.

The problem: linear regression does not fit every outcome

Ordinary linear regression models the conditional mean of an outcome as a linear combination of predictors. Under the classical assumptions, the errors have constant variance and are independent; normality is mainly relevant for exact small-sample statistical inference. This works well for continuous outcomes such as revenue, but many business questions break these assumptions:

Will this customer churn? The outcome is yes or no. A straight line can predict 1.3 or −0.2, which are not valid probabilities.
How many orders will we get tomorrow? The outcome is a count: 0, 1, 2 and so on. Count outcomes cannot be negative, and their variability often increases with their mean. Poisson regression assumes that the conditional mean and variance are equal.

You can force a straight line through such data, but you will get impossible predictions and misleading error estimates. We need a model that respects the nature of the outcome.

The big idea: three components

A GLM extends linear regression to handle different types of outcomes. It combines predictors in a linear equation, but adapts the model to the kind of data we are trying to predict.

Every GLM has three components:

Random component. Defines the probability distribution of the outcome, such as normal, binomial or Poisson.
Systematic component. Combines the predictors into a weighted sum, called the linear predictor η.
Link function. Connects the expected outcome to the linear predictor. We explore how this works in the sections below.

η = β₀ + β₁X₁ + … + βₚXₚ g( E[Y | X] ) = η η is the linear predictor, E[Y | X] is the expected outcome given the predictors and g is the link function.

GLMs are generally fitted using maximum likelihood estimation. For a Gaussian model with an identity link, this produces the same coefficient estimates as ordinary least squares.

Three link functions

Imagine NordHome Retail, an online home-goods retailer that wants to understand what drives its performance. It cares about three outcomes:

Order value in euros (continuous)
Whether a customer churns (yes/no)
Number of orders per day (count)

Each needs a different distribution and link function. The inverse link transforms the linear predictor η back into the expected outcome.

Model	Link function	Expected outcome
Linear regression	Identity	μ = η
Logistic regression	Logit	p = 1 / (1 + e^−η)
Poisson regression	Log	μ = e^η

Linear regression: identity link

The identity link leaves the linear predictor unchanged, so the weighted sum of the predictors is the prediction.

μ = η = β₀ + β₁X₁ + … + βₚXₚ μ is the expected outcome, for example the order value in euros, and η is the linear predictor.

A one-unit increase in X changes the predicted conditional mean by β, holding other predictors constant.

Chart 1 Identity link: the prediction can take any value. Illustrative example.

Example. NordHome wants to predict the expected order value, for example from the number of items in the basket. If a customer's linear predictor comes out as 60, the model predicts an order value of €60. If X is the number of items and β = 12, each extra item adds €12 to the predicted order value.

Logistic regression: logit link

For a yes/no outcome, logistic regression models the log-odds of the event rather than the probability directly. Applying the inverse link converts the linear predictor into a probability between 0 and 1.

log( p1 − p ) = η p = 1 / (1 + e^−η) p is the probability of the event, for example that a customer churns.

Exponentiating a coefficient, e^β, gives the odds ratio: the factor by which the odds change with a one-unit increase in a predictor, holding other predictors constant.

Chart 2 Logit link: the prediction always stays between 0 and 1. Illustrative example.

Example. Now NordHome wants to know which customers are likely to churn. If a customer's linear predictor is −1.39, the model gives p = 1 / (1 + e^1.39) ≈ 0.20, a 20% churn probability. If the coefficient for support tickets is 0.405, then e^0.405 ≈ 1.5, so each extra ticket multiplies the odds of churn by 1.5. A customer at 20% (odds of 0.25) moves to odds of 0.375, which is about 27%, not 30%.

Poisson regression: log link

For count outcomes, Poisson regression models the logarithm of the expected count. The inverse link gives a positive prediction.

log(μ) = η μ = e^η μ is the expected count, for example the number of orders per day.

Exponentiating a coefficient gives a multiplicative effect on the expected count: e^β is the factor by which the expected count changes with a one-unit increase in X, holding other predictors constant.

Chart 3 Log link: the prediction is always above 0 and rises faster as η grows. Illustrative example.

Example. Finally, NordHome wants to forecast daily orders. If the linear predictor for a given day is 5.01, the model predicts e^5.01 ≈ 150 orders. If the coefficient for a promotion day is 0.18, then e^0.18 ≈ 1.20: a promotion day has about 20% more expected orders, so 150 becomes roughly 180.

A note on assumptions

Poisson regression assumes that the conditional variance equals the conditional mean. If the data show substantial overdispersion, a negative binomial model may be more appropriate. If observations have different exposure periods, an offset can account for differences in exposure.

Business application

Back to NordHome: one framework covers three typical questions.

Predict sales (linear). What is the expected order value, given customer segment, discount and season? Each coefficient is an effect in euros.
Predict customer churn (logistic). Which customers are likely to stop buying, based on days since the last order, order frequency and support contacts? Each coefficient is an odds ratio.
Predict order counts (Poisson). How many orders will arrive per day, given weekday, promotion and weather? Each coefficient is a multiplicative effect on the expected count, which helps with staffing and stock.

In Python, statsmodels fits all three models with the same function. You only change the formula and the distribution family; the usual link function is set automatically for each family.

import numpy as np
import statsmodels.api as sm
import statsmodels.formula.api as smf

# 1. Order value (continuous): Gaussian family, identity link
sales_model = smf.glm("sales ~ discount + C(season)", data=df,
                      family=sm.families.Gaussian()).fit()

# 2. Churn (0/1): Binomial family, logit link
churn_model = smf.glm("churned ~ recency + frequency + support_tickets", data=df,
                      family=sm.families.Binomial()).fit()

# 3. Daily orders (count): Poisson family, log link
orders_model = smf.glm("orders ~ C(weekday) + promo_active", data=df,
                       family=sm.families.Poisson()).fit()

# Logistic regression: odds ratios
odds_ratios = np.exp(churn_model.params)

# Poisson regression: multiplicative effects on the expected count
count_effects = np.exp(orders_model.params)

print(odds_ratios)
print(count_effects)

A Gaussian GLM with an identity link gives the same result as ordinary least squares under the usual setup, but you do not need a GLM for ordinary linear regression. It is shown here only to make the three models comparable.

Reading the exponentiated coefficients

Exponentiated coefficients are interpreted relative to the model's link function. For logistic regression they are odds ratios. For a standard log-link Poisson model they are multiplicative effects on the expected count. They are not automatically causal effects.

How to explain it to stakeholders

“For illustration, imagine our churn model finds that each extra support ticket in the last 90 days multiplies a customer's odds of leaving by 1.5. These are odds, not probabilities: a customer who starts at a 20% churn risk moves to about 27% after one extra ticket, not to 30%.”

“For illustration, suppose we model daily orders as counts and find that a promotion day multiplies the expected number of orders by about 1.2. If we normally get 150 orders a day, we should plan for roughly 180 on a promotion day.”

“So the business question is: which customers should the service team contact first, and how much staff and stock do we need on promotion days?”

Key takeaway

Linear, logistic and Poisson regression are one mathematical framework, adapted to different outcome distributions through the choice of distribution and link function. Before you fit a model, ask what kind of outcome you are predicting: a continuous number, a yes/no, or a count. That answer tells you which GLM to use.

Four Common Discrete Distributions Bernoulli, binomial and Poisson: the distributions behind logistic and Poisson regression.
Normal Distribution The distribution behind ordinary linear regression.
Exploratory Data Analysis How to explore relationships in retail data before you model them.
Hypothesis Testing How to judge whether a coefficient is really different from zero.