Sentiment Analysis on Holy Books

Now the analysis is conducted over the 4 holy books:

bible <- bible %>%
  select(book, text)

quran <- gutenberg_download(7440)
names(quran) <- c("book", "text")
quran$book <- "The Quran"

dhammapada <- gutenberg_download(35185)
names(dhammapada) <- c("book", "text")
dhammapada$book <- "Dhammapada"

mormon <- gutenberg_download(17)
names(mormon) <- c("book", "text")
mormon$book <- "The Book of Mormon"

Determining mirror for Project Gutenberg from http://www.gutenberg.org/robot/harvest

Using mirror http://aleph.gutenberg.org

A new tidy table is created listing all words from all books:

books_words <- rbind(bible, quran, dhammapada, mormon) %>%
  mutate(text = str_to_lower(text)) %>%
  mutate(text = str_replace_all(text, "'s", "")) %>%
  filter(text != "") %>%
  unnest_tokens(word, text)

Removing stop words, empty lines, digits and punctuation from the texts:

filtered_books_words <- books_words %>%
  filter(!word %in% c(stop_words$word, pattern)) %>%
  filter(word != "") %>%
  filter(!str_detect(word, "^\\d+$")) %>%
  filter(!str_detect(word, "[[:punct:]]"))

Here is shown the words that are repeated the most in all Holy Books:

filtered_books_words %>%
  count(word, sort = TRUE) %>%
  filter(n > 2300) %>%
  mutate(word = reorder(word, n))%>%
  ggplot(aes(n, word)) +
  geom_bar(stat = "identity", show.legend = FALSE) + 
  ylab("word") + xlab("# times repeated") +
  coord_flip() + 
  theme(axis.text.x = element_text(size=14, angle = 90, hjust = 1), axis.text.y = element_text(size=14))

The purpose is to compare sentiments between positive or negative.
Therefore we use now the bing lexicon for positive and negative index:

bing <- get_sentiments("bing") %>%
  select(word, sentiment)

Again there is a big difference in the size of the texts as follows:

filtered_books_words %>%
  group_by(book) %>%
  summarize(words = n()) %>%
  arrange(words)

A tibble: 4 × 2

book	words
<chr>	<int>
Dhammapada	6487
The Book of Mormon	79140
The Quran	143462
The Holy Bible	253791

We define again the odds ratio, so the size of the text does not affect the results:

books_sentiment <- books_words %>%
  left_join(bing, by = "word") %>%
  count(book, sentiment) %>%
  spread(sentiment, n) %>%
  group_by(book) %>%
  mutate(negative_odd = negative / (positive + `<NA>`),
         positive_odd = positive / (negative + `<NA>`),
         or = positive_odd / negative_odd)
books_sentiment

A grouped_df: 4 × 7

book	negative	positive	<NA>	negative_odd	positive_odd	or
<chr>	<int>	<int>	<int>	<dbl>	<dbl>	<dbl>
Dhammapada	784	944	14654	0.05026285	0.06114782	1.2165608
The Book of Mormon	6575	7011	269337	0.02379246	0.02541028	1.0679968
The Holy Bible	18868	17188	754357	0.02445483	0.02222898	0.9089811
The Quran	12315	11550	407344	0.02939885	0.02752235	0.9361710

Graphically it is possible to see that the odds ratios of negative and positive are quite similar in all books. The Dhammapada however has a slight more positive words:

ggplot(melt(books_sentiment[,c(1, 5, 6)]),
       aes(x=book, value, fill=variable,  width = 0.5)) +
  geom_bar(stat="identity", position = position_dodge(width=0.5)) +
  theme(axis.text = element_text(size=14))

Using book as id variables