Ways to increase statistical power

Michael Großbach
michael.grossbach@hmtm-hannover.de

Hochschule für Musik, Theater und Medien – Hannover

Institut für Musikphysiologie und Musiker-Medizin

2023-12-07

Sample sizes are always too small

“Increase your sample size!”

  • Lack of resources
    • Money
    • Staff
    • Time
  • Lack of subjects / small populations

Determinants of statistical power

  • The statistical power (i.e. probability) to detect an existing effect depends on
    • \(\alpha\)
    • \(\beta\)
    • The ‘true’ effect size
    • Sample size
    • Measurement error

But there’s more

Increase statistical power

by choosing an appropriate experiment design

Simple nested design

Code
subj_n <- 40  # total number of subjects
b0 <- 0       # intercept
b1 <- 0.5     # fixed effect of condition
u0s_sd <- 0.5 # random intercept SD for subjects
sigma_sd <- 2 # error SD
# set up data structure
nested_data <- function(sn, intercept, slope, varinterc, sigma) {
  add_random(subj = sn) %>%
  # add and recode categorical variables
  add_between("subj", cond = c("control", "test")) %>%
  add_recode("cond", "cond.t", control = 0, test = 1) %>%
  # add random effects 
  add_ranef("subj", u0s = u0s_sd) %>%
  add_ranef(sigma = sigma_sd) %>%
  # calculate DV
  mutate(dv = b0 + u0s + b1 * cond.t + sigma) ->
    nd
  return(nd)
}
nest_dat <- nested_data(subj_n, b0, b1, u0s, sigma_sd)
ABCDEFGHIJ0123456789
subj
<chr>
cond
<fct>
cond.t
<dbl>
u0s
<dbl>
sigma
<dbl>
subj01control0-0.166471.7310
subj02test10.174120.3845
subj03control0-0.07888-0.5047
subj04test1-0.142942.8256
subj05control0-0.338840.2356
subj06test1-0.001620.5628
subj07control0-0.31935-0.8816
subj08test10.28254-2.9658
subj09control00.171013.4184
subj10test11.31660-0.0825
Next
1234
Previous
1-10 of 40 rows | 1-5 of 6 columns

Difference between conditions/groups?


    Welch Two Sample t-test

data:  dv by cond
t = -0.7, df = 37, p-value = 0.5
alternative hypothesis: true difference in means between group control and group test is not equal to 0
95 percent confidence interval:
 -1.743  0.813
sample estimates:
mean in group control    mean in group test 
               -0.054                 0.411 

Power analysis by simulation

Control and Test groups

Simple crossed design

Code
subj_n <- 40  # total number of subjects
b0 <- 0       # intercept
b1 <- 0.5     # fixed effect of condition
u0s_sd <- 0.5   # random intercept SD for subjects
sigma_sd = 2 # error SD
# set up data structure
crossed_data <- function(sn, intercept, slope, varinterc, sigma) {
  add_random(subj = sn) %>%
    # add and recode categorical variables
    add_within("subj", cond = c("control", "test")) %>%    # <- CHANGED!!!
    add_recode("cond", "cond.t", control = 0, test = 1) %>%
    # add random effects 
    add_ranef("subj", u0s = u0s_sd) %>%
    add_ranef(sigma = sigma_sd) %>%
    # calculate DV
    mutate(dv = b0 + u0s + b1 * cond.t + sigma) ->
    cd
  return(cd)
}
cross_dat <- crossed_data(subj_n, b0, b1, u0s, sigma_sd)
ABCDEFGHIJ0123456789
subj
<chr>
cond
<fct>
cond.t
<dbl>
u0s
<dbl>
sigma
<dbl>
subj01control00.9684-1.2129
subj01test10.96843.5411
subj02control00.10631.6244
subj02test10.10630.3922
subj03control0-0.4836-0.1828
subj03test1-0.4836-2.5661
subj04control0-0.14820.2709
subj04test1-0.1482-0.6592
subj05control0-1.03570.3206
subj05test1-1.0357-0.5857
Next
123456
...
8
Previous
1-10 of 80 rows | 1-5 of 6 columns

Power analysis by simulation

Each Subj their own control

Crossed design with replicates

Code
subj_n <- 40  # total number of subjects
b0 <- 0       # intercept
b1 <- 0.5     # fixed effect of condition
u0s_sd <- 0.5 # random intercept SD for subjects
u1s_sd <- 0.2 # random slope for replicates        # <- NEW !!!
r_01s <- 0.7  # correlation between intercept & slope # <- NEW !!!
sigma_sd = 2 # error SD
# set up data structure
cross_repl_data <- function(sn, intercept, slope, varinterc, varslope, varcor, sigma) {
  add_random(subj = sn) %>%
    # add and recode categorical variables
    add_within("subj", 
               cond = c("control", "test"),
               replicate = 0:9) %>%  # <- NEW !!!
    add_recode("cond", "cond.t", control = 0, test = 1) %>%
    # add random effects 
    add_ranef("subj", 
              u0s = varinterc,
              u1s = varslope,                       # <- NEW !!!
              .cors = varcor) %>%                  # <- NEW !!!
    add_ranef(sigma = sigma) %>%
    # calculate DV
    mutate(dv = b0 + u0s + b1 * cond.t + u1s + sigma) ->
    cd
  return(cd)
}
cr_dat <- cross_repl_data(subj_n, b0, b1, u0s_sd, u1s_sd, r_01s, sigma_sd)
ABCDEFGHIJ0123456789
subj
<chr>
cond
<fct>
replicate
<int>
cond.t
<dbl>
u0s
<dbl>
subj01control00-0.6772
subj01control10-0.6772
subj01control20-0.6772
subj01control30-0.6772
subj01control40-0.6772
subj01control50-0.6772
subj01control60-0.6772
subj01control70-0.6772
subj01control80-0.6772
subj01control90-0.6772
Next
123456
...
80
Previous
1-10 of 800 rows | 1-5 of 8 columns

Power analysis by simulation

Each subject their own control; 10 trials per condition

Know your parameters

To be able to simulate realistically…

  • \(\alpha\), \(\beta\)
  • effect size estimate (literature, experience)
  • inter-individual variation (literature, experience)
  • inter-condition variation (literature, experience)
  • …

Takeaway message

The required sample size to achieve the desired statistical power can be manipulated by choosing a suitable experimental design!

This requires a lot of prior research!

IMMM 2023

Ways to increase statistical power Michael Großbach michael.grossbach@hmtm-hannover.de Hochschule für Musik, Theater und Medien – Hannover Institut für Musikphysiologie und Musiker-Medizin 2023-12-07

  1. Slides

  2. Tools

  3. Close
  • Ways to increase statistical power
  • Sample sizes are always too small
  • Determinants of statistical power
  • But there’s more
  • Simple nested design
  • Slide 6
  • Difference between conditions/groups?
  • Power analysis by simulation
  • Simple crossed design
  • Power analysis by simulation
  • Crossed design with replicates
  • Power analysis by simulation
  • Know your parameters
  • Takeaway message
  • f Fullscreen
  • s Speaker View
  • o Slide Overview
  • e PDF Export Mode
  • ? Keyboard Help