The Pennsylvania State University, Spring 2021 Stat 415-001, Hyebin Song

Introduction

Go to course main page

Introduction to Statistical Inference

Learning objectives

1. Understand the relationship between probability and statistics
2. Understand basic terminology in statistics

Statistics

Statistics is a data-driven science that concerns the extraction of useful information from the observed data in a principled way, accounting for uncertainty in the observed data.

Example 1

Goal: understand an average height of all PSU Students (40K)

Procedure:

1. Ask 100 PSU students their heights and record the responses
2. Compute an average of 100 height values
3. Claim that this computed average should be close to the average height of all PSU Students

Some definitions

Example 1 (Continued)

1. Population: collection of all heights from PSU students
2. Sample: $(X_1,\dots,X_{100})$. A random vector of size 100
3. Parameter of interest: $\mu = \frac{1}{4K}(5'8''+6'+\dots)$.
4. Estimator: $\bar{X} = \frac{1}{100}(X_1+\dots+X_{100})$.
5. Estimate: $\bar{x}= \frac{1}{100}(5'8''+4'11''+\dots)$.

Example 2

Goal: understand the probability of face "1" in a biased die

Procedure:

1. Toss the die 1000 times and record the outcomes.
2. Compute the average proportion of "1" among 1000 trials
3. Argue that the computed proportion is close to the true probability of outcome "1".

In this example,

1. population: probability of outcome 1,...,6 of the die.
2. parameter of interest: $\theta$ = probability of outcome 1
3. sample: a random vector of size 1000 $(X_1, \dots, X_{1000})$ where $X_i$: = ith toss outcome
4. estimator of $\theta$: a sample proportion $\widehat{\theta}(X_1,\dots,X_n) = \frac{1}{1000}\sum_{i=1}^{1000}1[X_i=1]$
5. estimate of $\theta$: a realized sample proportion $\widehat{\theta}(1,0,\dots,1)=\frac{1}{1000}(1+0+1+1 + \dots+1 )$

Relationship between probability and statistics

1. Given $P(H) = P(T) = 1/2$, what is the probability that the number of heads $\leq 10$ among 20 tosses?

   Model: $X_i \sim \textrm{Ber}(0.5$). $(X_1,\dots,X_{20})$ i.i.d.

   Predict data: $P(\sum_{i=1}^{20}X_i\leq 10)$

2. Given ($x_1=1, x_2= 0, \dots, x_{20}=1$), what is the true probability of head?

   Data: $(x_1,\dots,x_{20})$ where each $x_i \in\{0,1\}$.

   Infer a model:

   • Since each $x_i \in\{0,1\}$, assume $X_i \sim \textrm{Ber}(\theta)$, i.i.d.
   • Using $(x_1,\dots,x_{20})$, we make a guess about $\widehat{\theta}$, and argue $X_i \sim \textrm{Ber}(\theta)$ with $\theta \approx \widehat{\theta}$.

Parametric and distribution-free models

• Parametric models: assume that the distribution of each observation is known up to a parameter.

  Example: $X_i \sim \textrm{Ber}(\theta)$, $X_i \sim N(\mu,\sigma^2)$, ...

• Distribution-free models: do not make an assumption on the distribution of a sample.

Remark 1. The problem of inferring a model reduces to inferring a parameter value.

Remark 2. In this class, we will mostly focus on parametric models.

Overview of this course

Suppose we have an observed sample $(x_1,\dots,x_n)$ where each $x_i$ is assumed to be from a parametric distribution $P_\theta$ with an unknown parameter $\theta$.