Parametric Testing

Parametric tests: Assume that sample data is drawn from a population that can be characterized by distribution parameters and reduce data to sample statistics to estimate these population parameters

Z-Score

z-scores are usually used for normalization

$$z = \frac{x-\mu}{\sigma}$$

$$z = \frac{\bar{x} - \mu}{SEM}$$

Decisions can be wrong, we have type 1 and 2 error

Null Hypothesis: the treatment is not effective, difference between observed and actual is just chance

Alternative Hypothesis: the observed sample mean is not equal to the expected population mean because the treatment was effective and shifted the mean

The p-value does not tell you the probability that the null hypothesis is true. Instead, it tells you the probability of the observed data (or something more extreme) if the null hypothesis were true.

Degrees of Freedom

The number of independent pieces of information (numbers, measurements, data points) in a dataset that a parameter estimation is based on.

The higher the number of degrees of freedom, the more evidence it is based on, the more stable the parameter estimate is.

Why is it lost?

Imagine you have a set of n observations. If there are no constraints, all n observations can vary independently. However, when you impose a constraint—such as estimating a parameter from those observations—not all observations remain free to vary.

The term k typically represents the number of parameters estimated from the data.

T-Test

$$t=\frac{\bar{x}_1 - \bar{x}_2}{SEM}$$

For small sample size and unknown parameters

If we draw two samples, and the two sample means are too far from each other, it is unlikely that they came from the same underlying population: They probably came from 2 populations with different means

To meaningfully interpret a difference in the samples by comparing their means, we have to assume “Homogeneity of variance” - that variability within each sample is similar – only the means differ.

Several Versions of T-Test

which one to use depends on number of independent groups

2 independent groups we use between subject design

Expectation: Difference of (group) means is 0 if H0= true

1 independent group: t-test for dependent groups (or paired samples or correlated groups)

In some experiments, the same people are tested twice. This is called a “within-subjects-design”.

df = n-1

Welch T-Test

In most real use cases, variances won’t be homogeneous, that is different groups will not have the same variance

Big idea: The variance of the two samples is not pooled, but modeled separately