```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

Goal of this worksheet is to understand what the questions are saying for Homework 3, but on a different example. 
See pages 5-9 of the textbook for the outline, while learning how to best answer these questions yourself.  

<!--------------->
<!-- Problem 1 -->
<!--------------->

### 1. Likelihood inference. 

* Write down the likelihood function $L(\theta|D)=\prod p_{\theta}(j)^{u_j}$.
* Write down the log-likelihood function $l(\theta|D)=\log L(\theta|D)$.
* The maximum likelihood estimate $\hat{\theta}$ is the maximizer of the log-likelihood function:
$$
    \hat{\theta} = \text{argmax}_{\theta \in \Theta} l(\theta|D).
$$
Unlike the MLE on hw3, this one is easier to derive. Use the formulation above and calculus (yes, take partial derivatives by hand) to derive the MLEs $\hat\pi_{ij},\hat\alpha_ij},\hat\beta_{jk}$ written in equation 1.1.4 in the book. 
  * For the observed data $u=(4,0,3,0,1,1,3,1)$, compute directly the value of the MLE $\hat p$ that is written on page 7 of the book. 
  * Note this was the same vector that appeared on Homework 1. 

<!--------------->
<!-- Problem 2 -->
<!--------------->

### 2. Evidence for or against the model fit for the two-step Markov chain.

* What is the probability of the observed data $u=(4,0,3,0,1,1,3,1)$?
* Propose a definition of the $p$-value for this data set, for the problem of testing the goodness of fit of the Markov chain model.
* What problem do you have to solve explicitly in order to compute this $p$-value? 
  * Explain how this is similar to and different from the setup of problem 2 on homework 3. 
