Probability & Linear Algebra

Probability

• Disjoint (mutually exclusive):

  • The outcome of a single coin toss cannot be a head and a tail

  • A student can’t both fail and pass a class

  • Sum of probabilities of two disjoint events may not add up to 1. Meaning there are more than two outcomes of the random process

  • P(A and B) = 0

• Complementary Events

  • Complementary events are two mutually exclusive events whose probabilities add up to 1

  • Coin toss

• Adding Probabilities

  • P(A or B) = P(A) + P(B) - P(A and B)

  • Disjoint:

    • P(A and B) = 0

    • P(A or B) = P(A) + P(B)

• Independence

  • Two processes are independent if knowing the outcome of one provides no useful information about the outcome of the other

  • P(A|B) = P(A)

• Conditional

  • P(A|B) = P(AandB)/A(B)

  • P(AB) = P(A|B)P(B)

  • P(AB|C) = P(B|AC)P(A|C) = P(B|C)P(A|C)

• Marginalization $p(x) = \sum_{y}{p(x,y)}$

• Law of Total Probability $p(x) = \sum_{y}{p(x|y)\cdotp(y)}$

  • if we have a partition of the sample space into mutually exclusive events, the probability of another event can be computed as the sum of its conditional probabilities over the partition.