Predict a numeric value and evaluate how close predictions are to observed targets.

Linear regression

Learning objectives

  • Identify a regression problem.
  • Fit a linear model with scikit-learn.
  • Calculate a baseline and mean absolute error.
  • Interpret predictions without claiming certainty.

Regression questions

Regression predicts a number, such as temperature, travel time, or energy use. A model learns a relationship between features and a numeric target.

Train a model

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split

X = [[1], [2], [3], [4], [5], [6]]
y = [2, 4, 5, 8, 10, 11]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=42
)

model = LinearRegression()
model.fit(X_train, y_train)
predictions = model.predict(X_test)

print(predictions)
print(mean_absolute_error(y_test, predictions))

Mean absolute error (MAE) is the average absolute distance between prediction and target. If the target is measured in hours, the MAE is also measured in hours.

Compare with a baseline

baseline_value = sum(y_train) / len(y_train)
baseline_predictions = [baseline_value] * len(y_test)

model_error = mean_absolute_error(y_test, predictions)
baseline_error = mean_absolute_error(y_test, baseline_predictions)

print(model_error, baseline_error)

The model is useful only if it improves on a reasonable baseline under the same test conditions.

Inspect the model

print(model.coef_)
print(model.intercept_)

The coefficient describes how the prediction changes for one unit of the feature within the learned relationship. It does not prove that the feature causes the target to change.

Practice

Add a second feature, such as practice_minutes and projects_completed. Compare the two-feature model with a one-feature model using the same split.

Common mistakes

  • Treating a small dataset as proof of a universal relationship.
  • Ignoring the units of the error.
  • Comparing models on different test examples.
  • Extrapolating far beyond the training range.

Checkpoint

What does an MAE of 3 mean when predicting minutes? Why is a baseline necessary before interpreting that number?

Next step

Continue to classification and logistic regression.