keywords jamovi, multinomial models, generalized linear models, post-hoc, moderated regression, interactions

In this example we study the relationships between a continuous independent variable, a categorical independent variable and their interaction on a categorical dependent variable.

We run the analyses with the GAMLj module in Jamovi. One can follow the example by downloading the cvs file and open it in jamovi. Be sure to install GAMLj module from within jamovi library.

The data are from a idre hsbdemo example. You can find similar analyses in pure R and a nice explanation of them at the UCLA idre web page.

The data set contains variables on 200 students. The outcome (dependent) variable is `prog`

, program type. There are three programs that students can choose: general program, vocational program and academic program. The predictor (independent) variables are social economic status, `ses`

, a three-level categorical variable and writing score, `write`

, a continuous variable UCLA idre web page.

The cross-tab of frequencies of participants combining social economical status and the outcome program is in the next table (in jamovi `frequencies`

-> `Contingency tables`

) .

The descriptive of the continuous independent variable `write`

are in the table.

We want to understand if choosing a particular program out of the three available (general program, vocational program and academic program) can be linked to the student ability to write (`write`

) and her/his social economical status. Because the two predictors can be correlated, we want (ultimately) to run a single model (multiple regression) such that the effect of each predictor is estimated while keeping constant the effect of the other, and a possible interaction can be assessed (moderated regression). We will run some preliminary models to warm up.

The dependent variable is a 3-level categorical variable, so we need a multinomial model. The aim of a multinomial model is straightfoward: Estimating how the probability of each category in the dependent variable varies as a function of the independent variable(s). In our example, we are going to estimate how the probability of choosing each program dependends on the ability to write (scores of `write`

) and whether this probability is different for the three levels of social economical status (groups of `ses`

).

The way the multinomial model does that is less straightforward ( *you can skip this if you are in a hurry* ): The dependent variable is decomposed in K-1 dummy variables (where K is the number of categories in the dependent variable) and a (sort of) logistic model is estimated for each dummy. Thus, if we pick a reference group for the dependent variable, say `academic program`

, the model estimates the influence of the independent variable(s) on the **logit** (log of odd) of choosing each program over the academic program. Having three programs, our analysis will estimate two (K-1) set of coefficients: the effect of the independent variables on the (log) odd of choosing `general`

program over choosing `academic`

, and the (log) odd of choosing `vocation`

program over choosing `academic`

. The exact information about the change in odd (rather than the logit) can be obtained by looking at the **odd ratios** (`exp(B)`

).

To be clearer, let’s consider the frequencies of the dependent variable:

The proabilities in the dependent variable are P(academic)=.525, P(general)=.225, P(vocation)=.25. To capture the “change” in probabilities and link it to the independent variable, the multinomial model starts with the odds: the general vs academic odd is P(general)/P(academic)=.225/.525=0.429. Thus, on average, choosing the general program is less than half as likely as choosing the academic program. The model estimates how this odd depends on the independent variable. The same goes for the vocation vs academic odd, P(vocation)/P(academic)=.25/.525=0.476. The model estimates how this odd is influenced by the independent variables. Remember that the B coefficients are expressed in the logit scale (log(odd)), the `exp(B)`

in the odd scale.

The overall test for each independent variable (Omnibus test Chi-squared) tests the null hypothesis that all the coefficients associated with an independent variable are zero, thus providing a “main effect” across all the dependent variable groups.

To simplify the interpretation, we can always look at the plots of the effects. In jamovi GAMLj the plots are on the probability scale, thus very easy to interpret: they show how the probability of each program changes for different levels of the independent variable.

The choice of the reference group is statisticaly immaterial, but can be adjusted for interpretational purposes. Here we use `academic`

because in jamovi GAMLj the default is to set the first group as the reference group: the `prog`

variable is a string variable, thus the groups are alphabetically ordered. If one needs to change the reference group, a different coding of the dependent variable groups can be used.

Let’s start with predicting `prog`

with the social economical status. In GAMLj `generalized linear model`

we select the `multinomial model`

, push the `prog`

variable in the `Dependent Variable`

field and `ses`

in `Factors`

.

As soon as we fix the variables, the results are there, with the first table showing some info about the model.

Here we can outline the R-squared, that gives information about the goodness of fit of the model (see technical details for more info). Our error of approximation of the data decreases of 4% thanks to the `ses`

variable. Put it in another way, our ability to predict `prog`

increases of 4% thanks to `ses`

over using only the observed probabilities.

The other information in the table helps to interpret the results. In particulat, the row `Direction`

is useful. It gives the definition of the logit that is used, including which is the reference group of the dependent variable. In the example, it indicates that the there are two logits, one is comparing `prog=general`

against `prog=academic`

, the other `prog=general`

against `prog=academic`

. Thus we know that all the independent variables positively related with the first logit are positively related with the odd of being in program general over the academic one, the independent variables positively related with the rescon logit are positively related with the odd of being in program vocation over the academic one.

The omnibus Chi-Squared test tests the null hypothesis that the probabilities of `prog`

choice are the same for all `ses`

groups. Based on the p-value, our results seem rare under the null hypothesis, so we can deem the effect of `ses`

as statistical significant. Let’s go straight to the interpretation of the results by visualizing the effects.

Ask for the plot in the `Plots`

panel:

and see what we obtain:

The effec of `ses`

is due to the fact that `high ses`

group is much more likely to choose the academic program (`prog`

=1) over the other two programs, while `low ses`

and `middle ses`

choose the three programs with more or less the same probability. The effect is not strong (recall R-squared=.04), but is at least visible in the plot.

An interesting note can be made for the omnibus test Chi-squared=16.8, p=.002. This test is equivalent to the standard Chi-squared one obtains by running a chi-squared test on the contingency table `prog X ses`

. In fact (in jamovi `frequencies`

-> `Contingency tables`

)

The standard chi-squared test is 16.6, but the `Likelihood ratio`

is 16.8. QED, the multinomial omnibus test for categorical independent variables is exactly the chi-squared test obtained on a cross-tabs, only estimated with the maximum likelihood method. Accordingly, one can say that the frequencies of the cross-tab `prog X ses`

are not independent.

If one needs (and seldom one does in these cases), one can look at the model coefficients, the regression coefficients.

Skipping the intercept (recall that nobody interprets the interaction :-) ), the first coefficient, `ses1`

is associated with the dependent variable contrast `general-academic`

as predicted by the contrast `low ses`

versus the average of the sample `(high, low , middle)`

. The `exp(B)`

is 1.938. This means that the odd of choosing `general`

over `academic`

for people of low social economical is 1.93 times higher than for the average person in the sample, and this effect is statistically significant (z=2.429, p=.001). Indeed, this is what the plot actualy shows.

The other coefficients can be interpreted along the same line.

Let’s include `write`

as independent variable and see the results.

Now we have two omnibus tests, indicating that the ability to write has a stistically significant effect on the probability of choosing a program, while keeping constant `ses`

. The analysis also confirms an effect of `ses`

, also when `write`

is kept constant. As before, we can interpret the results by looking at the probability plot.

The better one writes, the higher the probability of choosing the academic program, and the lower is the probability of choosing the vocational program. Choosing the general program does not depends on the writing skills.

Please notice in the plot that the independent variable `write`

it is centered to its mean. This is a default of jamovi GAMLj in order to avoid unexpected results when interactions and other complex effects are estimated. However, it is only a default setting, so we can change it as we’re pleased.

By going to `Covariates scoring`

tab, one can choose not to center the variable.

Obviously, the results are not different, but the plot x-axis looks nicer now.

A question we can ask is whether the effect of writing abilities may be different at different levels of social economical status, thus putting forward a moderation hypothesis.

After setting the `write`

covariate back to centered, we can go to the `Model`

tab and push the interaction term to the right field. We need this because GAMLj abides by an old rule of not estimating by default the iteraction between continuous and categorical variable (this default may change in future releases).

Results show very weak support for an interaction, because the Chi-square is low (3.46) and the p-value high (.484). This means that the probabilities profile along the `write`

scores are not substantially different across social economical groups. We can verify that by eyeballing the plot of probabilities broken down by `ses`

groups.

Got comments, issues or spotted a bug? Please open an issue on GAMLj at github“ or send me an email