Regression analysis

Simple Linear Regression: Finding the Relationship in Data

Start with the simplest question: can we use one variable to predict another? Simple linear regression answers it with a straight line. This post explains what the model predicts and what its coefficients mean, using an advertising example from NordHome Retail. OLS: How Does Linear Regression Find the Best-Fit Line? shows how the best line is found, and Generalized Linear Models extends the idea to other types of outcomes.

Start with a business question

NordHome's marketing manager has next month's budget in front of her. She asks: “If we spend €6k on advertising, how much will we sell?”

She has five past months of data:

Advertising spend (€k)	Sales (€k)
2	27
4	42
5	49
7	53
8	59

Plot these points and you see a pattern: more advertising goes with more sales, and the points sit roughly along a straight line. Linear regression finds the one straight line that fits them best, and then lets us use it to answer questions like hers.

Scatter plot of five months of NordHome data: advertising spend in thousand euros on the horizontal axis and sales in thousand euros on the vertical axis. The points are (2, 27), (4, 42), (5, 49), (7, 53) and (8, 59) and rise from left to right roughly along a straight line. A dashed vertical line marks the planned spend of 6 thousand euros. — Chart 1 NordHome's five past months: sales rise with advertising spend. The dashed line marks the €6k the manager is considering. Illustrative data.

Linear Regression Basics

Linear regression models a relationship that can be drawn as a line on a graph. The line shows how a dependent variable (y) changes as an independent variable (x) changes. It has two jobs: to predict (estimate sales for a spend we have not tried) and to explain (quantify how much sales tend to change when spend changes).

Dependent variable (y). The continuous outcome we want to estimate. Here, sales.
Independent variable (x). The predictor, or explanatory factor, we use to make the estimate. Here, advertising spend.

Key Terms and Examples

The model describes the line with one equation:

ŷ = b₀ + b₁x ŷ (“y-hat”) is the predicted value, not the observed one. b₀ is the intercept and b₁ is the slope.

For NordHome's data, the best-fitting line is:

ŷ = 20 + 5x x is advertising spend and ŷ is predicted sales, both in €k.

Slope (b₁ = 5). How much the dependent variable changes for each one-unit increase in the independent variable. Each additional €1k of advertising goes with about €5k more sales, on average. This is the number the manager cares about most.

Intercept (b₀ = 20). The predicted value of y when x is zero. Since our observations start at €2k, this is an extrapolation. The intercept is mathematically necessary, but we cannot conclude from these data that NordHome would generate €20k in baseline sales.

Fitted line and residuals. The fitted line minimises the total squared vertical distance between the points and the line. The distance for each point is its residual: actual − predicted. For the month with €7k spend, the line predicts 55 and the actual value was 53:

53 − 55 = −2 The residual for the €7k month: the actual value is 2 below the line.

But how does the model decide which line fits best? That is where the method of least squares comes in, and we will explore it in the next article.

Answering the manager's question

For advertising spend of €6k:

ŷ = 20 + 5 × 6 = 50 The expected sales are €50k.

Scatter plot of five months of advertising spend against sales with the fitted line. The line starts at 20 on the sales axis and rises to 60 at 8 thousand euros of spend. A yellow marker shows the prediction of 50 thousand euros of sales for 6 thousand euros of spend, and a short red segment shows the residual of minus 2 for the month with 7 thousand euros of spend. — Chart 2 The fitted line through NordHome's five months (blue dots). The yellow dot is the prediction for €6k, the red bracket is the residual of the €7k month, and the hollow dot is the intercept, outside the observed range. Illustrative data.

b₁ = Σ(x − x̄)(y − ȳ)Σ(x − x̄)² b₀ = ȳ − b₁x̄ Where the numbers come from: x̄ = 5.2 and ȳ = 46 are the averages, so b₁ = 114 / 22.8 = 5 and b₀ = 46 − 5 × 5.2 = 20. In practice, software computes this for you.

The same logic works elsewhere. A café could model cake sales from coffee sales: more coffees, more cakes, so the slope is positive. A retailer could model product sales from price: a higher price, fewer units sold, so the slope is negative.

Correlation and Interpretation

Correlation measures the strength and direction of a linear relationship, on a scale from −1 to +1. A positive correlation means the variables rise or fall together, as with advertising and sales. A negative correlation means one rises while the other falls, as with price and units sold. For NordHome's five months, the correlation is r ≈ 0.97: a very strong positive relationship.

r = Σ(x − x̄)(y − ȳ)√( Σ(x − x̄)² · Σ(y − ȳ)² ) For NordHome: r = 114 / √(22.8 × 604) ≈ 0.97. The correlation has no unit and does not depend on whether sales are measured in euros or thousands of euros.

Correlation does not imply causation. A relationship between two variables does not mean one causes the other. NordHome's sales could have risen because of advertising, or because of a seasonal peak that also pushed the advertising budget up. The model cannot tell these apart.

Two more limits to keep in mind:

Stay inside the data. Predictions are only trustworthy within the range of the data (€2k to €8k). Predicting for €50k is extrapolation.
An average, not a promise. Individual months will sit above or below the line.

Key takeaway

Linear regression fits a line ŷ = b₀ + b₁x to data. The slope tells you how much y changes per unit of x, and the intercept is the model's prediction when x = 0. The line lets you predict, as long as you stay within the data and remember that correlation does not imply causation.