Solutions to Midterm 1 Extra Questions

Ch. 8, Extra Question 1

Consistency is a large sample property for an estimator. It says that, if we have a large sample, then our estimator should be close to the population quantity that we are trying to estimate.

An unbiased estimator is one that, if we could repeatedly draw samples of size \(n\) (\(n\) could be large or small here) and re-estimate the parameter of interest, then on average (across samples in this repeated sampling thought experiment), our estimate would be equal to the population parameter that we are trying to estimate.

Ch. 8, Extra Question 2

No, an unbiased estimator does not necessarily have to be consistent. Consider estimating \(\mathbb{E}[Y]\) by \(Y_1\) (i.e., using the value of \(Y\) for the first observation in the data). This estimate of \(\mathbb{E}[Y]\) is unbiased (since \(\mathbb{E}[Y_1] = \mathbb{E}[Y])\), but it is not consistent (because it does not even depend on the sample size at all).

Ch. 8, Extra Question 3

No, consistent estimators can be biased. Consider estimating \(\mathbb{E}[Y]\) by \(\bar{Y} + \frac{c}{n}\) where \(c\) is a constant. This is consistent because \(\bar{Y} \rightarrow \mathbb{E}[Y]\) as \(n \rightarrow \infty\) (by the law of large numbers) and \(\frac{c}{n} \rightarrow 0\) as \(n \rightarrow \infty\) (because it is a constant divided by a number that is going to infinity). However, notice that

\[ \begin{aligned} \mathbb{E}\left[\bar{Y} + \frac{c}{n} \right] &= \mathbb{E}[Y] + \frac{c}{n} \\ & \neq \mathbb{E}[Y] \end{aligned} \] so, for any fixed sample size \(n\), \(\bar{Y} + \frac{c}{n}\) is biased for \(\mathbb{E}[Y]\).

Ch. 8, Extra Question 4

Since \(n\) grows faster than \(\sqrt{n}\), \(n \left(\frac{1}{n}\sum_{i=1}^n (Y_i - \mathbb{E}[Y])\right)\) diverges (i.e., the absolute value goes to infinity as \(n \rightarrow \infty\))
Since \(n^{1/3}\) grows slower than \(\sqrt{n}\), \(n^{1/3} \left(\frac{1}{n}\sum_{i=1}^n (Y_i - \mathbb{E}[Y])\right)\) converges to 0 as \(n \rightarrow \infty\)

Solutions to Additional Questions

1. What would be the result from running the following code?
```
all( c(1,2,3,4,5) > 0)
```
  Answer: TRUE, this checks if all elements of the vector 1,2,3,4,5 are greater than 0. Since they all are, it returns TRUE
2. Consider the following function
```
a_function <- function(n) {
  out <- 0
  for (i in 1:n) {
    out <- out + i^2
  }
  out
}
```
  If you run the following code, what will it output?
```
a_function(5)
```
  Answer: It will output 55 which comes from adding up \(1+4+9+16+25\)
Suppose there are two random variables \(X\) and \(Y\).
1. If you know that \(X\) and \(Y\) are independent, do you know what their covariance is equal to? Explain. If yes, what is the covariance equal to?
  
  Answer: Yes, their covariance is 0. Independent random variables have 0 covariance.
2. If you know that \(\mathrm{cov}(X,Y)=0\), are \(X\) and \(Y\) independent? Explain.
  
  Answer: Not necessarily, random variables can have 0 covariance without being fully independent.
3. If you know that \(\mathrm{cov}(X,Y)=1\), are \(X\) and \(Y\) independent? Explain.
  
  Answer: \(X\) and \(Y\) are not independent in this case. If they were, their covariance would be equal to 0.
Suppose that \(X_1\) and \(X_2\) are two random variables such that \(\mathbb{E}[X_1] = 0\), \(\mathbb{E}[X_2] = 5\), \(\mathrm{var}(X_1) = 1\), \(\mathrm{var}(X_2) = 10\) and \(\mathrm{cov}(X_1,X_2) = -1\). Suppose that \(Y=X_1 + X_2\).
1. What is \(\mathbb{E}[Y]\)?
  
  Answer:
  
  \[\begin{align*} \mathbb{E}[Y] &= \mathbb{E}[X_1 + X_2] \\ &= \mathbb{E}[X_1] + \mathbb{E}[X_2] \\ &= 0 + 5 = 5 \end{align*}\] where the second equality holds because expectations can pass through sums
2. What is \(\mathrm{var}(Y)\)?
  
  Answer:
  
  \[\begin{align*} \mathrm{var}(Y) &= \mathrm{var}(X_1 + X_2) \\ &= \mathrm{var}(X_1) + \mathrm{var}(X_2) + 2\mathrm{cov}(X_1,X_2) \\ &= 1 + 10 + 2(-1) \\ &= 9 \end{align*}\]
Consider a random variable \(Y\) that is equal to a firm’s profits (in thousands of dollars) and another random variable \(X\) that is equal to firm’s number of employees. Suppose you know that \[\begin{align*} \mathbb{E}[Y|X=x] = 50 + 10x \end{align*}\]
1. Explain how to interpret \(\mathbb{E}[Y|X=x]\).
  
  Answer: This is the conditional expectation of \(Y\) given \(X\) takes the particular value \(x\). In the context of the problem, it is the mean profit’s of firms that have \(x\) number of employees. The value can change for different numbers of employees.
2. What is \(\mathbb{E}[Y|X=10]\)?
  
  Answer: \(\mathbb{E}[Y|X=10] = 50 + 10(10) = 150\)
3. Suppose that \(\mathrm{var}(Y) = 40\), \(\mathbb{E}[X]=30\), and \(\mathrm{var}(Y)=20\), calculate \(\mathbb{E}[Y]\).
  
  Answer:
  
  \[\begin{align*} \mathbb{E}[Y] &= \mathbb{E}[50 + 10 X] \\ &= 50 + 10 \mathbb{E}[X] \\ &= 50 + 10 (30) \\ &= 350 \end{align*}\]
Suppose that we have a random sample of \(n\) observations of \(X\) and \(Y\).
1. Suppose that you want to estimate the covariance between \(X\) and \(Y\) using the data that we have. Propose an estimator for the covariance. Hint: Try using the analogy principle and the expression \(\mathrm{cov}(X,Y) = \mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]\).
  
  Answer:
  
  \[\begin{align*} \widehat{\mathrm{cov}}(X,Y) = \frac{1}{n} \sum_{i=1}^n X_i Y_i - \left( \frac{1}{n} \sum_{i=1}^n X_i \right) \left( \frac{1}{n} \sum_{i=1}^n Y_i \right) \end{align*}\]
2. Alternatively, the definition of covariance is \(\mathrm{cov}(X,Y) = \mathbb{E}\left[ (X - \mathbb{E}[X])(Y-\mathbb{E}[Y]) \right]\). Propose an estimator for the covariance based on this expression. Would you expect this to give you the same estimate of the covariance as in part a?
  
  Answer: To conserve on notation, let \(\bar{X}\) and \(\bar{Y}\) denote the sample averages of \(X\) and \(Y\), respectively. Then,
  
  \[\begin{align*} \widehat{\mathrm{cov}}(X,Y) = \frac{1}{n} \sum_{i=1}^n (X_i - \bar{X})(Y_i - \bar{Y}) \end{align*}\]