Classification Problem

• Precision

$$ \frac{TP}{TP+FP} $$

• Recall (Sensitivity): What % of positives are correctly identified?

  $$ \frac{TP}{TP+FN} $$

• Specificity: What % of negatives are correctly identified?

  $$ \frac{TN}{TN+FP} $$

• Accuracy

$$ \frac{TP+TN}{All \space Observations} $$

• F1 score

$$ \frac{2 \times Precision \times Recall}{Precision + Recall} $$

• Confusion Matrix

  | | Actual 1 | Actual 0 | | --- | --- | --- | | Predicted 1 | TP | FP | | Predicted 0 | FN | TN |

• ROC AUC (Receiver Operator Characteristics)

  Plot True Positive Rate vs False Positive Rate

  OR Sensitivity vs 1-Specificity

• PR AUC (Precision Recall)

  Plot Recall/Sensitivity vs Precision

  If % of zeros is high, we can use precision instead of false +ve rate

• Information value

$$ IV = \sum(\%Goods - \%Bads)\times ln(\frac{\%Good}{\%Bads}) $$

Regression Problem

1. Mean Absolute Error (MAE): $\frac{1}{n}\sum|y_i - \hat{y_i}|$
   • less sensitive to outliers compared to the mean squared error.
2. Mean Squared Error (MSE): $\frac{1}{n}\sum(y_i - \hat{y_i})^2$
   • gives higher weight to large errors and is sensitive to outliers.
3. Root Mean Squared Error (RMSE): $\sqrt{\frac{\sum(y_i - \hat{y_i})^2}{n}}$
   • is in the same unit as the target variable
   • provides a more interpretable measure of error compared to MSE.