Posterior analysis & computation

class: center, middle, inverse, title-slide

# Posterior analysis & computation
### Math 315: Bayesian Statistics

---

class: middle, clear

## 1. Summarizing the posterior

## 2. Basics of Bayesian computing

---

# Posterior analysis

.large[
> To a Bayesian, the best information one can ever have about `$\theta$` is to know the posterior density.
>
> — Christensen, et al; *Bayesian Ideas and Data Analysis*, p. 31
]

.center[
<img src="04-intervals-probs-grid_files/figure-html/unnamed-chunk-1-1.svg" width="75%" />
]

---

# Point estimates

.Large[
- Posterior mean 
- Posterior median 
- Posterior mode — i.e. *maximum a posteriori* (MAP) estimate
]

.center[
<img src="04-intervals-probs-grid_files/figure-html/unnamed-chunk-2-1.svg" width="80%" />
]

---

# Credible intervals

.center[
<img src="04-intervals-probs-grid_files/figure-html/ci1-1.svg" width="80%" />
]

```r
# q*() functions calculate quantiles from the specified distribution
c(lower = qbeta(0.055, 14, 4), upper = qbeta(1 - 0.055, 14, 4))
```

```
##     lower     upper 
## 0.6100397 0.9126956
```

---

# Credible intervals are not unique

.large[Here are three 89% credible intervals]

---

# Highest Posterior Density Interval (HPDI)

.center[

![](04-intervals-probs-grid_files/figure-html/hpdi-1.svg)
]

---
class: center
![](04-intervals-probs-grid_files/figure-html/ci1-1.svg)

![](04-intervals-probs-grid_files/figure-html/hpdi-1.svg)

---

# Testing a hypothesis

.large[
Suppose the researchers were interested in testing

.pull-left[

`$H_0: \theta \le 0.5$`

`$P(\theta \le 0.5 | Y = 14) = 0.004$`

![](04-intervals-probs-grid_files/figure-html/unnamed-chunk-4-1.svg)

]

.pull-right[

`$H_1: \theta > 0.5$`

`$P(\theta > 0.5 | Y = 14) = 0.996$`

![](04-intervals-probs-grid_files/figure-html/unnamed-chunk-5-1.svg)
]

]

---
class: inverse, middle

# Basics of Bayesian computing

---

# Design, redux (for the last time)

.large[
**Data:** N N N N B B N N N B N N N N N N N (14 Ns; 3 Bs)

**Data model:**

Some true proportion of guesses, `$\theta$`

Toss a coin with probability of heads, `$\theta$`

**Belief about `$\theta$`:**

Researchers believe that `$\theta = 2/3$` is most likely
]

---

## Triangle distribution

.large[
`$$f(\theta | a, b, c) = 
 \begin{cases}
 \frac{2(\theta -a)}{(b-a)(c-a)} & a \le \theta <c,\\
 \frac{2(b-\theta )}{(b-a)(b-c)} & c \le \theta <b,\\
 0 & \text{otherwise.}
 \end{cases}$$`
]

---

## Posterior derivation
.large[
.pull-left[
`$$\pi(\theta) = 
 \begin{cases}
 3\theta& 0 \le \theta <2/3,\\
 6(1-\theta) & 2/3 \le \theta < 1,\\
 0 & \text{otherwise.}
 \end{cases}$$`
 
`$f(Y | \theta) = \dbinom{n}{y} \theta^Y (1-\theta)^{n-Y}$`

`$$p(\theta | Y) \propto
 \begin{cases}
 \theta^{Y+1} (1-\theta)^{n-Y} & 0 \le \theta <2/3,\\
 \theta^{Y} (1-\theta)^{n-Y+1} & 2/3 \le \theta < 1,\\
 0 & \text{otherwise.}
 \end{cases}$$`
]

.pull-right[

]

.content-box-blue[
This is not a "known" distribution...
]

]

---

## Grid approximate posterior

.Large[
1. Choose a grid of values of `$\theta$` over an interval that covers the posterior density.

2. Compute likelihood `$\times$` prior on the grid.

3. Normalize by dividing each product by the sum of the products.
]

---

## Grid approximate posterior

.large[

```
## # A tibble: 1,001 x 5
## theta prior likelihood unstd.posterior posterior
## <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 0 0 0. 0. 0. 
## 2 0.001 0.003 6.78e-40 2.03e-42 3.02e-44
## 3 0.002 0.006 1.11e-35 6.64e-38 9.87e-40
## 4 0.003 0.009 3.22e-33 2.90e-35 4.31e-37
## 5 0.004 0.012 1.80e-31 2.16e-33 3.21e-35
## 6 0.005 0.015 4.09e-30 6.13e-32 9.11e-34
## 7 0.006 0.018 5.23e-29 9.42e-31 1.40e-32
## 8 0.007 0.021 4.52e-28 9.48e-30 1.41e-31
## 9 0.008 0.024 2.92e-27 7.01e-29 1.04e-30
## 10 0.009 0.027 1.51e-26 4.09e-28 6.07e-30
## # … with 991 more rows
```

.content-box-blue[We've approximated the PDF with a PMF...]
]

---

## Grid approximate posterior

.large[
.content-box-blue[
We've approximated the PDF with a PMF... with pretty good results!
]
]

---

## What do we want to do with our posterior?

.Large[
- Calculate point estimates

e.g. Calculate the posterior mean or MAP estimate

- Determine an interval with specified probability

e.g. Find an 97% credible interval for `$\theta$`

- Determine probability in a fixed interval

e.g. Find `$P(\theta > 0.75)$`
  
]

---

## How can we use the grid-approximate posterior?
.large[
.content-box-blue[
**Monte Carlo Sampling**

Draw a sample of size `$S$` from the posterior

`$$\theta^{(1)}, \theta^{(2)}, \ldots, \theta^{(S)} \sim p(\theta | Y)$$`

and use these samples to approximate the posterior
]
]

---

## Monte Carlo sampling

.large[
Now, use sample statistics to approximate posterior quantities
]

---

## Calculating `$P(\theta > 0.75)$`

.large[
Estimate probabilities by the proportion of samples, `$\theta^{(i)}$`, that fall in the interval.

```r
mean(posterior_sample > 0.75)
```

```
## [1] 0.55499
```
]

---

## Calculating a 97% credible interval

.content-box-blue[
**Equal-tailed CI (i.e.percentile interval)** 
Trim an equal proportion from each side of the sample
]

```r
# using the quantile function like in Math 275
quantile(posterior_sample, probs = c(0.015, 0.985))
```

```
##  1.5% 98.5% 
## 0.552 0.921
```

---

## Calculating a 97% HPDI

.content-box-blue[
**HPDI**: use the `HDInterval` package
]

```r
HDInterval::hdi(posterior_sample, credMass = 0.97)
```

```
## lower upper 
## 0.561 0.925 
## attr(,"credMass")
## [1] 0.97
```