Problem Set 7: Choosing an Estimator

Author

POL 211

Published

November 10, 2026

Due: Monday, November 16 by 11:59 PM. Email both your .qmd and your rendered .html to Max Welch (mwelch@ucdavis.edu).

TipReview Quant Camp as you go

This set leans on the R you learned at Quant Camp. When you get stuck on filtering, grouping, or plotting, go back to the Quant Camp materials. They’re the fastest way to get unstuck.

ImportantUsing AI

Use AI the way people used Google, R’s help pages, and Stack Overflow before AI: ask precise, limited questions to get unstuck on part of a question. Passing off AI-generated work wholesale as your own is not allowed. See the syllabus and the Department’s policy on generative AI.

Format: Write your answers in one Quarto document, saved in the sets/ folder of your course project. Start it with this header, so the HTML renders as one self-contained file:

---
title: "Problem Set 7"
author: "Your name"
format:
  html:
    embed-resources: true
---

Your document should read as a short report: brief prose explaining your reasoning, not just code and output.

Libraries you will need:

library(tidyverse)
library(juanr)
WarningMethods subfield

Parts marked Methods portfolio are required if you plan to choose methods as a subfield. They’re optional, and recommended, for everyone else.

Question 1: Four estimators of the average vote share

The elections data in juanr has one row for every US county. The variable per_dem_2020 is the share of the vote the Democratic candidate won in that county in 2020, from 0 to 1.

For this question, treat the 3,000-plus counties as the population. The estimand is \(\mu\), the average Democratic vote share across counties. A researcher who could only afford to collect results from a random sample of counties would have to estimate \(\mu\). Here we can see the whole population, so we know the truth and can judge how well each estimator does.

Start by pulling out the population, in percentage points:

counties = elections |>
  filter(!is.na(per_dem_2020)) |>
  mutate(dem = 100 * per_dem_2020) |>
  pull(dem)
  1. Compute \(\mu\), the population average. Then compute the population median. Make a histogram of counties and describe its shape. Why are the mean and the median different?

  2. Here are four estimators of \(\mu\), from a random sample of \(n\) counties \(Y_1, \ldots, Y_n\). They’re close cousins of the four from lecture:

    Estimator Rule
    1 The third county \(\hat{\mu}_1 = Y_3\)
    2 The sample median \(\hat{\mu}_2 = \text{median}(Y_1, \ldots, Y_n)\)
    3 The sample average \(\hat{\mu}_3 = \bar{Y} = (Y_1 + \cdots + Y_n) / n\)
    4 An “adjusted” average \(\hat{\mu}_4 = (Y_1 + \cdots + Y_n) / (n - 5)\)

    Write a function, four(y), that takes a sample of counties and returns a tibble with one row per estimator and its estimate (model it on four() in the week 7 class script, 07-code.R). Draw one random sample of 20 counties and apply your function to it. How close is each estimate to \(\mu\)?

  3. Rerun the study. Draw 10,000 samples of 20 counties, apply four() to each, and plot the four sampling distributions side by side, with a vertical line at \(\mu\) (as on the lecture slide “Rerun the study: each estimator has its own sampling distribution”). Describe what you see: where is each sampling distribution centered, and how spread out is it?

  4. Bias. Use your simulation to estimate the bias of each estimator at \(n = 20\): the average of its 10,000 estimates, minus \(\mu\). Which estimators seem likely to be unbiased? (Note: I use “seems” because we can’t know for sure via simulation).

  5. Consistency. Now repeat the simulation with samples of 20, 100, and 1,000 counties, and plot all twelve sampling distributions in a grid (as on the lecture slide “More data: do the estimators close in on the truth?”). Which estimators close in on \(\mu\) as \(n\) grows? Which don’t, and why? (Hint: the median does close in on something.)

WarningMethods portfolio
  1. Say one county’s vote share has variance \(\sigma^2\) and counties are drawn independently. Use week 5’s rules to find the variance of estimators 1, 3, and 4. Plug in \(n = 20\) and the population’s \(\sigma\) (compute it from counties, dividing by \(N\)), and compare with the SDs of your simulated estimates from (c).

Question 2: When a biased estimator wins

Two pollsters are estimating a candidate’s share of the vote in a district, and the true share is 50%. Each poll is one draw from its pollster’s sampling distribution.

  • Pollster A is unbiased. Its estimates have a standard deviation of 4 points.
  • Pollster B has a “house effect”: its estimates run 2 points too high on average. But it uses a better sampling method, and its estimates have a standard deviation of only 2 points.

Do this question by hand.

  1. Compute each pollster’s bias, variance, and mean squared error.

  2. Which pollster’s estimate typically lands closer to the truth? By about how much? Explain why a biased estimator can sometimes beat an unbiased one.

  3. Pollster A can afford a bigger sample. Increasing the sample size by a factor of \(k\) divides the variance by \(k\) and leaves the bias alone. How many times bigger must A’s sample be for A to beat B?

WarningMethods portfolio
  1. Prove that \(\text{MSE}(\hat{\theta}) = \text{Bias}(\hat{\theta})^2 + \text{Var}(\hat{\theta})\), where \(\text{MSE}(\hat{\theta}) = E\big[(\hat{\theta} - \theta)^2\big]\). (Hint: write \(\hat{\theta} - \theta = \big(\hat{\theta} - E[\hat{\theta}]\big) + \big(E[\hat{\theta}] - \theta\big)\), square it, and take expectations.)

Question 3: Why R’s var() divides by \(n - 1\)

In week 3, you computed a variance by averaging the squared misses from the average: divide by \(n\). R’s var() divides by \(n - 1\) instead. This question shows why: dividing by \(n\) is a biased estimator of the population variance, \(\sigma^2\).

Use the counties population from Question 1.

  1. Compute the population variance, \(\sigma^2\), dividing by \(N\). This is the estimand.

  2. Draw 2,000 samples of 5 counties. For each sample, compute two estimators of \(\sigma^2\): the average squared miss from the sample average dividing by \(n\), and R’s var(), which divides by \(n - 1\). What is the average of each estimator across the 2,000 samples? Which one is biased, and in which direction? By roughly what fraction is it off?

  3. Why too small? Do this part by hand. Say the true mean is \(\mu = 8\), and a sample of three counties has vote shares 2, 6, and 10.

    • Compute the sample average, \(\bar{Y}\).
    • Compute the sum of the squared misses from \(\bar{Y}\), and the sum of the squared misses from \(\mu\). Which is smaller?
    • Check that the gap between the two sums equals \(n(\bar{Y} - \mu)^2\).
    • In words: why do squared misses from \(\bar{Y}\) always add up to less than squared misses from \(\mu\)? (Hint: week 3 asked which guess makes the squared misses smallest.) Why does that make the divide-by-\(n\) estimator too small?
  4. Repeat (b) with samples of 100 counties. How biased is the divide-by-\(n\) estimator now? Is it consistent?