keywords jamovi, logistic models, generalized linear models, odd-ratios, ANOVA, ANCOVA, factorial designs, interactions
Draft version, mistakes may be around , GAMLj version ≥ 0.9.7
In this example we analyze a 3 X 2 X 2 design, with a dichotomous dependent variable. We employ logistic regression for modelling the dichotomous dependent variable.
We run the analyses with the GAMLj module in jamovi. One can follow the example by downloading the cvs file and open it in jamovi. Be sure to install GAMLj module from within jamovi library.
The data are from a SAS User’s Guide example, 51.2. There you can find similar analyses in SAS.
Here is the data description from SAS User’s Guide:
“The data set Neuralgia contains five variables: Treatment, Sex, Age, Duration, and Pain. The last variable, Pain, is the response variable. A specification of Pain=Yes indicates there was pain, and Pain=No indicates no pain. The variable Treatment is a categorical variable with three levels: A and B represent the two test treatments, and P represents the placebo treatment. The gender of the patients is given by the categorical variable Sex. The variable Age is the age of the patients, in years, when treatment began. The duration of complaint, in months, before the treatment began is given by the variable Duration.”
The cross-tab of frequencies of participants Treatment
by Sex
is in the next table (in jamovi
frequencies
-> Contingency tables
), and
these are the first independent variables we analyze.
The base-rate frequencies for Pain
(the dependent
variable) are in the table.
Age and Duration are continuous variables (disregard the little icon indicating the variable format you see in jamovi variables list, we’re are going to use them as continuous). Their descriptives are the following:
We want to understand how the pain reported by the patients is affected by the Treatment and if this relationship is moderated by gender. Furthermore, the influence of Age and Duration is of interest as well.
We first start with Treatment
and Sex
and
their interaction as predictors of Pain
. Then we include
also Age
and Duration
. In the terminology of
the general linear model, these two analyses correspond to an ANOVA and
an ANCOVA, respectively. Because the dependent variable is a dichotomous
variable, we employ a logistic model. In the logistic model there’s no
variance to be analyzed (is the deviance or likelihood that is
analyzed), thus we should intend the terminology used here as meaning
ANOVA-like and ANCOVA-like models.
The logistic model predicts the probability of being in one group
defined by the dependent variable over the probability of being in the
other group (the odd). Jamovi GAMLj predicts the probability of the
group indicated by the last level of the dependent variable, in this
example being Pain=Yes
. If you are comparing the present
results with the analyses in the SAS
User’s Guide, please keep in mind that SAS predicts
Pain=No
, so the results are equivalent but the effects go
in the opposite direction. One can change the direction of the effects
by re-coding the Pain
variable.
Recall that the predicted values of the models are the logit
(logarithm of the odd) of being in pain (Pain=Yes
) over not
being in pain (Pain=Yes
). Thus, the model coefficients are
expressed in the logit scale ( log(odd) ) . A clearer scale is
obtained by removing the logarithm from the logit, obtaining
coefficients expressed in the odd scale. This is the exp(B)
column of the results tables: exp(B)
is an odd-ratio. The
plots and the estimated marginal means are conveniently transformed in
the probability scale, so they can be interpreted directly as the
probability of being in pain (P(Pain=Yes)
).
Let’s start with predicting Pain
with Sex
and Treatment
. In GAMLj
generalized linear model
we select the
logistic model
, push the Pain
variable in the
Dependent Variable
field and Sex
and
Treatment
in Factors
.
As soon as we fix the variables, results are displayed in the tables, with the first table showing some info about the model.
Here we find the R-squared, that gives information about the goodness
of fit of the model (see technical details
for more info). Our ability to predict pain increases of 26% thanks to
the independent variables as compared to using the base-rate
probabilities of Pain
.
The omnibus Chi-Squared tests inform us on the main effects and
interactions statistical significance. We can see there that we obtained
a main effect of Treatment
and one of Sex
.
Thus, on average, the probability of being in pain changes across
treatments and it is different between women and men. There is clearly
no interaction (\(\chi^2\)=0, p=1).
To interpret the main effects, let’s ask for the plot in the
Plots
tab:
and see what we obtain:
The effect of Sex
is due to the fact that the
men
group is much more likely to report pain than the group
of women
. The effect of Treatment
is mainly
due to the higher pain report rate in the P treatment than in the other
two conditions. The treatments are working.
(you can skip this if you’re not interested in beautiful details and some fundamental laws of statistics)
We should stop for a while and think very broadly about these
results, because they have many things to teach us in general terms. We
can see that Sex
has a statistical significant effect on
Pain
, but in the plots the confidence intervals seem to
overlap quite a bit. One may be inclined to believe that it is about the
3 by 2 design weird error deviance being included in the SE computation,
but it is not. We can remove Treatment
and still find that
Sex
is significant but the confidence intervals of the two
means of gender overlap.
Some people may find this strange, because of the common belief that two overlapping confidence intervals should correspond to a non-significant difference. However, this belief is wrong.
Andrea Knezevic at the Cornell University and Prasanna Parasurama at Atipica explain this in a clear way. Here is a shorter version:
Take a z-test for simplicity. Given two means a and b, their significance depends on the inequality \({a-b} > 1.96 \cdot \sqrt{{SE_a}^2+{SE_b}^2}\). The confidence intervals of the two means are not-overlapping if \({a-b} > 1.96 \cdot ( \sqrt{{SE_a}^2}+\sqrt{{SE_b}^2})\). Thus, as \(a-b\) increases, the significance is reached first by the test with smaller right term of the inequality. By Jensen inequality, we know that \(\sqrt{{SE_a}^2+{SE_b}^2} < \sqrt{{SE_a}^2}+\sqrt{{SE_b}^2}\) therefore there will be some values of \(a-b\) where the z-test is significant but the means confidence intervals overlap. Our examples shows exactly this.
From the same reasoning we can derive that the opposite is not true: If the confidence intervals do not overlap, the z-test must be significant.
If one needs (and seldom one does in these cases), one can look at the model coefficients, the regression coefficients.
Skipping the intercept (recall that nobody interprets the intercept
:-) ), the first coefficient is Sex1
: B=0.896. The
exp(B)
is 2.449. This means that the odd of reporting
Pain=yes
over Pain=No
for men (see
Contrast
column) is 2.499 times larger than for the average
of the sample ((M,F)
). This comparison may sound awkward,
but the awkwardness can be fixed. Recall that GAMLj default coding for
factors is deviations
, meaning that factors are coded such
that each group (but the first) is compared with the average of the
sample. When only two groups are present, a better coding is
simple
, that “simply” compares the two groups. We can set
the coding in Factor Coding
.
The B and exp(B) parameters associated with Sex
have
changed, although the z-test and p-value did not. The odd-ratio
associated with Sex
is now exp(B)=6, and the contrast says
M-F
- This means that the odd of reporting the pain is 6
times larger for men than for women.
Indeed, if we ask for the estimated marginal means of Sex averaging across Treatment we see the probabilities of reporting pain for the two genders.
We can check that the probability of pain is .614 for men and .209 for women, which yields:
The other coefficients can be interpreted along the same line.
Now we add Duration
and Age
as covariates,
and we see how they influence Pain
- We go back to the
variables set-up and add them in the Covariates
field.
Results show that Duration
does not have a significant
effect on Pain
, whereas Age
does. We can check
the direction of the Age
effect by looking at the
parameters.
The effect of Age is positive, B=0.268, and the odd-ratio is larger then 1, exp(B)=1.30. The older one gets, the more pain one reports.
As usual, a picture is worth a thousand words.
The older one gets, the more pain one reports, and the critical age seems to be around 70 years, where chanches of reporting pain considerably increase.
As regards Treatment
we can state that
Treatment
has still an effect even when Age
and Duration
are kept constant.
The last interesting question is whether treatment has a different
effect depending on the patient’s age. To answer this question we should
simply add the interaction Treatment*Age
in the model and
plot the interaction plot. I find it easier to plot Age
on
the x-axis and show the probabilities of pain for different treatments.
Let’s do that.
As suspected, there is an interaction between Treatment
and Age
, so it means that the efficacy of the treatments
depends on the patient’s age. The plot will tell us how.
First, the placebo group reports high rates of pain independently of age (blue line). For younger people (age<70) the two treatments A and B are beneficial, because the probability of reporting pain is very low. When patients get older, treatments are not so effective any longer, because the probabilities increase and converge to high rates for all three groups.
This may very well conclude the analyses.