Selected only one pollster, to observe the variability of the time effect:
one_pollster <- polls_us_election_2016 %>%
filter(pollster == "Ipsos" & state == "U.S.") %>%
mutate(spread = rawpoll_clinton/100 - rawpoll_trump/100)
Calculation of the observed and theoretical standard error over all polls of this pollster:
se <- one_pollster %>%
summarize(empirical = sd(spread),
theoretical = 2*sqrt(mean(spread)*(1-mean(spread))/min(samplesize)))
se
| empirical | theoretical |
|---|---|
| <dbl> | <dbl> |
| 0.04025194 | 0.03256719 |
The distribution of the data does not follow a normal distribution:
one_pollster %>% ggplot(aes(spread)) +
geom_histogram(binwidth = 0.01, color = "black") +
theme(axis.text = element_text(size=14))
The variability is accounted by the time variation due to party conventions, on both sides.
Graphics of the trend across time for one pollster and several pollsters:
one_pollster %>%
filter(state == "U.S.") %>%
mutate(spread = rawpoll_clinton/100 - rawpoll_trump/100) %>%
ggplot(aes(enddate, spread)) +
geom_smooth(formula = y ~ x, method = "loess", span = 0.1) +
geom_point(aes(color = pollster), show.legend = FALSE, alpha = 0.6) +
theme(axis.text = element_text(size=14))
polls_us_election_2016 %>%
filter(state == "U.S." & enddate >= "2016-07-01") %>%
group_by(pollster) %>%
filter(n() >= 10) %>%
ungroup() %>%
mutate(spread = rawpoll_clinton/100 - rawpoll_trump/100) %>%
ggplot(aes(enddate, spread)) +
geom_smooth(formula = y ~ x, method = "loess", span = 0.1) +
geom_point(aes(color = pollster), show.legend = FALSE, alpha = 0.6) +
theme(axis.text = element_text(size=14))
This shows that there is an extra variable that needs to be modelled to represent the time variability. The sample of election can be rewritten as:
$Xi,j,t = d + b + hj + bt + f(t) + \varepsilon i,j,t$
$bt$: bias term of time effect
$f(t)$: trend of time effect