Videos

• Bayes theorem, the geometry of changing beliefs
• The medical test paradox, and redesigning Bayes' rule

$$ \begin{align*} P(Hypothesis|Evidence) &=\frac{P(Evidence|Hypothesis)P(Hypothesis)}{P(Evidence)} \\ \\ &= \frac{P(E|H)P(H)}{P(E|H)P(H)+P(E|notH)P(not H)} \\ \end{align*}

$$

• P(H) is the prior probability
• P(H|E) is the posterior probability
• P(H|E) and P(E|H) are called likelihoods

Bayesian inference

Bayesian inference is a method of statistical inference that applies Bayes' theorem to update probability estimates for hypotheses based on new evidence or data. It provides a framework for making probabilistic predictions and updating beliefs in the presence of uncertainty. The core idea behind Bayesian inference is to use prior knowledge and combine it with observed data to obtain a posterior probability distribution, which represents the updated beliefs about the parameters of interest.

Naive Bayes

Video: Naive Bayes, Clearly Explained!!!

For spam filter example:

1. Find the P(Spam | Text) and P(Normal | Text). Label message as spam based on relative values of these probabilities
2. If Text = “Hello World”, then
   • P(Spam | Text) is proportional to P(Hello | Spam) x P(World | Spam) x P(Spam)
   • Similarly for P(Normal | Text)
   • Note that we ignoring the denominator which is P(Text) which is same for both
3. If a word never comes in spam or normal, then P(Word | Spam) would be zero. That would make the P(Spam | Text) = 0
   • To overcome this, we make sure that at least one count for each word for spam and normal message. This means that each word occurs at least once in both spam and normal message data

Why is Naive Bayes naive?