Classification and logistic regression

Learning objectives

• Distinguish classification from regression.
• Train a logistic-regression classifier.
• Separate probabilities from final class decisions.
• Evaluate a classifier on held-out data.

Classification questions

Classification predicts a category, such as safe or stop, spam or not spam, or one of several species. The labels are categories rather than measurements on a continuous scale.

Train a classifier

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X = [[5], [8], [12], [15], [20], [25], [30], [40]]
y = [1, 1, 1, 1, 0, 0, 0, 0]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y
)

model = LogisticRegression()
model.fit(X_train, y_train)

print(model.predict(X_test))
print(model.predict_proba(X_test).round(2))

predict() returns a class. predict_proba() returns a probability estimate for each class. A probability is not a guarantee.

Thresholds

The default decision threshold is commonly 0.5, but the right threshold depends on the cost of errors. If missing an obstacle is dangerous, a robot may need to favor catching possible obstacles even when that creates more false alarms.

Practice

Create a two-feature classifier for whether a robot should stop. Change the threshold in a small experiment and record how false positives and false negatives change.

Common mistakes

• Calling probabilities certainty.
• Choosing a threshold without considering consequences.
• Measuring only accuracy on an imbalanced dataset.
• Training and testing on duplicate or near-duplicate rows.

Checkpoint

Describe a false positive and a false negative for a spam filter. Which error is more harmful in your example, and why?

Next step

Continue to confusion matrices and evaluation metrics.