Bayes Factors for differences

Previously, we emphasised that summaries and descriptions of data are used to answer research questions.

For example, we used a boxplot to compare the differences in the amount of weight participants lost when treated with either Functional Imagery Training or Motivational Interviewing (the funimagery dataset). This lets us answer the question: “which intervention was more effective in helping participants lose weight?”:

This boxplot shows the median and interquartile range for each treatment group. It’s immediately obvious that — in this sample — there is a difference between the two groups (3.8 kg, in fact), and that — in this sample — FIT helped people lose more weight than MI.

For many purposes this plot is enough.

The difference here is quite large and we can interpret this graph at face value. If the costs of these interventions were similar (i.e. if ‘all other things are equal’) we should choose FIT rather than MI.

However, researchers often want to quantify the evidence provided by the sample of data they collect. This is important in psychology, and other sciences, because the size of the effects we see are often quite small or subtle.

It might be the case that patterns we see are just a ‘fluke’. That is, they have come about by chance. Perhaps if we took another sample from the same population (treated more people with FIT and MI) then we would see no difference?

The Bayes Factor quantifies how confident we should be about any given pattern in our data by comparing two possibilities:

1. There really is a difference in how much weight people lost with FIT and MI (we call this H1)
2. There’s NO difference between FIT and MI; any difference we see is likely to be a fluke (H0)

Bayes Factors for correlations

We have also seen how a scatterplot describes the relationship between two columns of data:

From this plot the relationship seems quite clear, and the correlation between these columns is 0.79.

However, we don’t have that many data points, so the pattern might have happened by chance.

The Bayes Factor allows us to quantify how much evidence we have for this correlation. As before, we are comparing two different possibilities:

1. There is a correlation between engine_size and power
2. There is NO correlation (the pattern is just a fluke)

Running a t-test

To run a t-test and calculate the Bayes Factor we first need to load the BayesFactor package:

library(psydata)
library(BayesFactor)

And then use the ttestBF function.

ttestBF(formula = weight_lost_end_trt ~ intervention, data=funimagery)

Bayes factor analysis
--------------
[1] Alt., r=0.707 : 515647.5 ±0%

Against denominator:
  Null, mu1-mu2 = 0 
---
Bayes factor type: BFindepSample, JZS

The important part of the code here is where we write: weight_lost_end_trt ~ intervention. This tells R which column to use as the outcome and which to use as the group predictor. The tilde symbol, ~, just means “is predicted by”.

fitbf <- ttestBF(formula = weight_lost_end_trt ~ intervention, data=funimagery) %>%
  as_tibble() %>% pull(bf) %>% .[1]

The number 5.1564753^{5} is our Bayes Factor. This is very large, so we have strong evidence that the difference in means is real.

We explain how to interpret Bayes Factors in more detail in the section below, but large numbers are evidence for a difference.

A worked example

We want to check the relationship of two variables in the happy dataset, which is based on the Gallup World Happiness Report 2021:

• happiness (a national measure of average life satisfaction)
• perceptions_of_corruption (a national measure of how corrupt they think their society is)

The first thing to do is to plot the data, to check that the relationship between these variables is approximately linear (a straight line). Here I have used filter to select only the data from 2019.

happy %>% 
  filter(year==2019) %>% 
  ggplot(aes(perceptions_of_corruption, happiness)) + 
  geom_point() + 
  geom_smooth(method=lm, se=F)

Warning: Removed 8 rows containing non-finite outside the scale range (`stat_smooth()`).

Warning: Removed 8 rows containing missing values or values outside the scale range (`geom_point()`).

The plot suggests a fairly strong negative correlation, as you might expect.

We can check the correlation using select and correlate():

happy  %>% 
  filter(year == 2019) %>% 
  select(happiness, perceptions_of_corruption) %>% 
  correlate()

So - the question now is, how much evidence do we have that the correlation is ‘real’, and that this pattern isn’t just a fluke.

# first, make a copy of the happy data, including only data from 2019
happy.2019 <- happy %>% filter(year == 2019)

with(happy.2019,  correlationBF(happiness, perceptions_of_corruption))

Bayes factor analysis
--------------
[1] Alt., r=0.333 : 70351.73 ±0%

Against denominator:
  Null, rho = 0 
---
Bayes factor type: BFcorrelation, Jeffreys-beta*

Now, the Bayes Factor is 70352.

That means we can be very confident that the pattern we see is not simply due to chance.