Factors associated with the effectiveness of interventions to prevent obesity in children: a synthesis of evidence from 204 randomised trials

Mapping of outcome to common measurement scale

We chose zBMI as our primary outcome because, unlike BMI, it accounts for the age and sex of the child and, unlike percentile, it was reported in a large number of trials. For trials that only reported results in terms of unstandardised BMI or BMI percentiles, we mapped these values onto the zBMI scale using methods we developed previously.9 To map from BMI to zBMI, we used a sampling method, implemented with 10 000 samples. The relationship between BMI and zBMI depends on an individual’s age and sex. Assuming a lognormal distribution for BMI, a normal distribution for age and a binomial distribution for sex, we set parameters of these distributions according to information reported from the trial. We then sampled 10 000 individuals from these distributions, calculated their zBMI and used these values to determine mean zBMI. To obtain zBMI from BMI percentile, we employed an analytic method that, assuming a normal distribution for zBMI, uses standard integral results to evaluate the expectation and variance of BMI percentile. For details of these mapping methods, we refer readers to our paper.9 In addition, nine trials reported results as the proportion of overweight or obese individuals. As described in our Cochrane reviews, we used normality assumptions to estimate mean zBMI from these values.6 7

Statistical model

To analyse our data, we used a bespoke multi-level meta-regression model described in detail previously.10 The model includes indicator variables (as covariates) defined on three levels: trial, intervention arm and time point. It assumes additive effects of all indicators while allowing for interactions between or within any level. An intercept term is included to capture the effect, relative to control, of an intervention whose indicators are all equal to zero. The mathematical details of our model are provided in Section B of the online supplemental material.

Each observation in the data is a mean difference between an intervention and a reference arm. The model takes a different form depending on whether the reference arm is a control arm or another active intervention. Comparisons between active interventions provide information about the effect of characteristics that differ between the arms of the trial. Therefore, these trials do not inform estimates of the intercept, trial-level indicators or time-point-level indicators. This is discussed further in our methodological paper.10

To account for correlations due to multi-arm and multi-follow-up trials, we specify a within-trial covariance matrix that depends on the correlation coefficient between observations at different time points. Based on observations in the data, we chose a correlation of 0.8. For further details, refer to the online supplemental material (Section C).

In our primary model, we included random effects (RE) to capture variation in intervention effects between trials. We assumed equal between-trial heterogeneity variances across interventions and follow-up times. That is, we made the usual assumption from network meta-analysis that the variation in relative intervention effects between trials is the same for different intervention comparisons.12 We made the additional assumption that the variation in intervention effects between trials is the same at different follow-up times, essentially treating multi-follow-up trials in the same way as multi-arm trials.

We fitted our model in a Bayesian framework using Markov chain Monte Carlo (MCMC) methods implemented in JAGS.13 Unless otherwise stated, we assigned uninformative prior distributions to all parameters. For the heterogeneity parameter, this deviated from our protocol in which we proposed to use an informative prior. However, we were able to include more data than expected in our final analysis and had sufficient information to estimate heterogeneity from the data. To aid convergence, we centred all indicators (including interactions) about their mean. While this does not affect the interpretation of any estimated coefficients, it means the intercept represents the effect of indicators at their mean value. Details of the implementation of our analyses can be found in Section D of the online supplemental material.

In presenting the results, we provide the estimated coefficient and 95% credible interval (CI) associated with each indicator and interaction term, along with the probability that each coefficient is less than zero, P(< 0), and the probability that it is greater than zero, P(> 0). These probabilities were calculated as the proportion of parameter samples falling either side of zero during the MCMC (after convergence of the chain).

Selection of characteristics and interactions

To ensure maximum statistical power, while retaining the detail of our analytic framework, we aimed to reduce the number of indicators in our model through a three-step selection process. The first two steps involved the selection of indicators, while the latter focused on selecting interactions. First, we assessed the collinearity between the different indicators and inspected pairs with absolute correlations of at least 0.5. We either discarded one of the indicators or combined them into a new indicator encapsulating information from both. In a second step, we identified indicators that received identical responses (ones or zeros) more than 80% of the time. We discarded or redefined these indicators, ensuring that we retained any deemed to be particularly important by our stakeholders (see our Patient and public involvement statement).

To select interaction terms, we used a Bayesian stochastic search variable selection (SSVS) approach based on work by Efthimiou et al.14 At each iteration of the MCMC, the SSVS model uses information from the data to select interactions from a prespecified set. Essentially, the model ‘shrinks’ parameter estimates in order to avoid overfitting. Interactions for which there is the most evidence of an effect are selected most often. Given the number of indicators in our model, there were almost 300 possible pairwise interaction terms. As it was infeasible to investigate all these interactions, even within the SSVS framework, we instead focused on interactions with age and behaviour targeted (diet and/or physical activity) as these were the interactions deemed most interesting by our stakeholders (see below). We applied the SSVS model in a stepwise process, fitting models with the following interaction terms: (i) no interactions, (ii) interactions between age and all other indicators and (iii) interactions with the behaviour targeted and all other indicators. In our final model, we included interactions from (ii) and (iii) that were selected more than 50% of the time. We describe details of the SSVS models in Section E of the online supplemental material.

Secondary and sensitivity analyses

As a secondary analysis, we performed a fixed effects (FE) model by setting the heterogeneity to zero. Although this was not pre-specified in our analysis plan, FE meta-regression analyses have a valid interpretation in the presence of unexplained heterogeneity.15

In addition, we performed sensitivity analyses on the different outcome scales. We fitted the model for (non-mapped) BMI and zBMI observations separately (but not for BMI percentile due to the small number of data points). Because the conversion from BMI percentile to zBMI involves fewer assumptions than the conversion from BMI, we also performed our analysis with zBMI and (mapped) BMI percentile measurements only. In our analysis protocol, we specified that we would include an indicator for whether the outcome had been mapped. Since the purpose of this is fulfilled by the sensitivity analyses and to keep the number of variables to a minimum, we chose not to include this indicator in our model.

Finally, we performed sensitivity analyses using different values for the correlation coefficient between observations at different time points. Based on observations in the data (see Section C of the online supplemental material), we investigated correlations of 0.5 and 0.95.

Primary analysis (RE)

Figure 1 shows a forest plot of the results of our primary analysis (RE model). Intervention effects were measured as mean differences (MD) in change from baseline in zBMI. Since effective interventions lead to smaller increases (or larger decreases) in zBMI, coefficients of indicators that are less than zero indicate greater effectiveness. The estimate of the model intercept and its 95% CI was –0.037 (–0.053, –0.022). This represents the effect of an intervention with all indicators set to their mean value, indicating that the interventions were beneficial on average. The heterogeneity standard deviation (and its 95% CI) was estimated to be 0.080 (0.067, 0.093), which is relatively large compared with the typical values of the estimated coefficients. In the following, we present estimates of coefficients as differences in mean differences (DMD) and their 95% CI. For a particular indicator, the DMD represents the additional effect of an intervention with indicator value 1 compared with an intervention with indicator value 0, conditional on the two interventions sharing the same values of all other indicators. Therefore, a negative DMD indicates that an intervention with that indicator value equal to one is more beneficial than an otherwise identical intervention with that indicator equal to 0. Conversely, a positive DMD implies that setting that indicator value to 1 rather than 0 makes the intervention less beneficial with respect to an otherwise identical intervention.

Our results provide strong evidence for a greater beneficial effect of interventions targeting physical activity alone compared with diet alone (–0.227 (–0.362,–0.090), P(< 0) = 1). Conversely, we observe no evidence of differing effects between diet and physical activity compared with diet alone. There is some indication of (small) greater effects for interventions that use multiple strategies (–0.040 (–0.097, 0.016), P(< 0) = 0.92) and are fully integrated within the curriculum/every day habits (–0.028 (–0.065, 0.009), P(< 0) = 0.93). On the other hand, the results indicate that interventions are less beneficial if they involve a change to the physical environment (most commonly the food environment) (0.050 (0.017, 0.085), P(> 0) = 1) and possibly less beneficial if they have some home-based component (0.033 (–0.005, 0.072), P(> 0) = 0.95).

There is strong evidence that, overall, interventions are more effective for children aged 12–18 years than for those aged 5–11 years (–0.284 (–0.450,–0.125), P(< 0) = 1). The analysis also suggests that the average intervention may work better in higher-income countries (–0.049 (–0.111, 0.010), P(< 0) = 0.95). However, both age and income status of the country appear in multiple interactions, so the interpretation of these results is more complicated. We return to this below (Section "Combinations of indicators"). The results for follow-up time indicate that, compared with short-term, larger beneficial effects are seen at medium-term (–0.036 (–0.064,–0.008), P(< 0) = 0.99), potentially followed by long-term (–0.019 (–0.23, 0.014), P(< 0) = 0.87).

Interactions are described as synergistic if the combined effect of the two indicators is greater than the sum of each independently or as antagonistic if the combined effect is less than the sum of each independently. In figure 1, interactions less than zero indicate a synergistic effect of the two indicators, whereas interactions greater than zero indicate an antagonistic effect. We find antagonistic interactions between the older age group and (i) interventions targeting both diet and physical activity and (ii) interventions conducted in high-income countries. We also observe antagonistic interactions between interventions targeting physical activity alone and (i) those conducted in high-income countries and (ii) interventions with long duration. We find no evidence of other interactions with interventions that target both diet and physical activity. We discuss the interpretation of these interactions in Section "Combinations of indicators".

Secondary analysis (FE)

The results of our secondary analysis (FE model) are shown in figure 2. As expected, this analysis leads to more precise parameter estimates. In addition to the effects identified in the primary analysis, the FE model suggests beneficial effects for interventions that are based in a school, target diet and physical activity (compared with diet alone) and are high intensity. It also identifies additional, less beneficial effects for interventions that are considered fun, resonant and involve an educational component. The FE analysis provides stronger evidence (compared with RE) that the average effect of interventions is more beneficial in higher-income countries and that greater benefits are seen at long-term follow-up compared with short term. In addition, this model finds that high risk of bias is associated with larger beneficial effects.

In addition to the interactions identified by the RE analysis, the FE model identifies several other effects. For the older age group, we observe antagonistic interactions with interventions that are integrated into the curriculum, and, with lower certainty, synergistic effects with interventions delivered by someone resonant. For interventions targeting both diet and physical activity, the FE model finds strong evidence of antagonistic effects with school-based interventions and high-income countries, and weaker evidence for antagonistic effects with interventions that involve home-based activities or delivery. Conversely, we observe synergistic effects between dietary and physical activity interventions and those that are considered fun and those that involve an electronic component. For interventions that target physical activity alone, we find reasonable evidence of all interactions (probabilities ≥ 0.9). The strongest signals are for antagonistic effects with long duration, high-income countries and high risk of bias, and synergistic effects with interventions delivered in a community setting and those involving an individual-level component. We summarise differences between the results of the FE and RE models in online supplemental tables S3 and S4 (section J).

Combinations of indicators

By evaluating the model at every combination of indicator values, we found that it predicts the greatest effectiveness for interventions that are set in a school, have an element that is delivered individually, target physical activity alone, contain multiple strategies, are high intensity, of short duration, are delivered by someone resonant and aim to modify behaviour through participation. These findings are associated with the older age group, low-income countries, mixed (rather than low) SES and medium-term follow-up.

Since we found evidence of various interactions with age and income status, we re-evaluated the ‘best’ combination of indicator values specific to each age group in both high and non-high-income countries. Online supplemental table S5 (section K) summarises these results. For the younger age group, resonance is not associated with the largest beneficial effect, but instead, the model identifies interventions that are integrated into the curriculum or daily habits. The indicator combination which differs most from all others is for the younger age group in high-income countries. This is the subset associated with the smallest most beneficial effect. In contrast to the older age group and the younger age group in low-income countries, the results indicate that beneficial effects in this group are associated with interventions involving an electronic component, that target both diet and physical activity and that are fun.

For each subset (age and income status), the largest beneficial effect predicted by the model is for medium-term follow-up and mixed SES (rather than targeted at low SES). We re-evaluated the most beneficial indicator combination specific to low SES, long-term follow-up and short-term follow-up, respectively. In each subset, the model identified the same set of intervention indicators associated with the largest beneficial effect. In online supplemental table S5, we list the value of the model obtained in each scenario.

The results for the worst (least beneficial) combination of indicators are presented in online supplemental table S6. As expected, these indicators mostly take the opposite value to those in the best (most beneficial) combination. Indicator values that take the same value in both the best and worst combinations are highlighted; they reflect the presence of interactions involving this indicator.

As expected, many other combinations of indicators predict outcomes that are similar to the best and worst. This can be deduced from the effect estimates for individual indicators, many of which are very close to zero and are not involved in any interactions (eg, school, community, individually targeted, flexible, participation, social). Therefore, changing the value of these indicators has a minimal effect on the predicted mean difference.

Across all possible combinations, some predict outcomes that are less beneficial than a control (ie, positive mean differences). Consistent with the results for the best combination, the greatest benefits are associated with the older age group in non-high-income countries where over 99.9% of possible combinations predict beneficial effects. However, across the younger age group and the older age group in high-income countries, this is only 61%.

The investigation of the best and worst 1% of combinations provides some further insights. In non-high-income countries (regardless of age group), 100% of the best combinations involve physical activity alone. In the older age group (regardless of income status of the country), this is 94%. The best combinations in these groups also tend to be of short duration (96%). However, for the younger age group in high-income countries, we find different signals. For example, the best combinations in this group tend to favour diet and physical activity (75%), while physical activity alone appears in 99.9% of the worst combinations. Across all age groups and countries, the best combinations rarely involved changes to the physical environment (11%) or any home-based activity (18%), whereas both these indicators were prevalent in the worst combinations (96% and 87%, respectively).

Model without interactions

The parameter estimates for the RE model without interactions (fitted in the first stage of the SSVS procedure) are shown in online supplemental figure S3 (section I.1.1). Most estimates are similar to those in the primary analysis. However, without interactions, the strong beneficial effects of physical activity alone, high-income status of country and the older age group disappear. As expected, these are the variables involved in the strongest interactions found in the primary model. This highlights that the effects of these variables depend on the values of the others. For example, the model without interactions provides no evidence for an effect of physical activity (compared with diet) on average. However, the primary analysis and the tests in the previous section show that this differs depending on the age and income status of the trial populations. Online supplemental table S5 shows that physical activity alone is more beneficial in low-income countries (regardless of age) and in the older age group (regardless of income status) but is less beneficial for younger children in high-income countries. Similarly, the effect of age group depends on the income status of the country. In online supplemental table S5, we observe that the difference in predicted outcome between age groups is larger in non-high-income countries compared with high income countries.

Sensitivity analyses

Sensitivity analyses with different outcome scales analysed separately, that is, excluding any mapped data, are reported in full in online supplemental figures S9–S11 (section L). Results for BMI only are similar to those of the primary analysis, while the zBMI only results are largely uncertain, with wider credible intervals. We summarise the main differences between the results on different scales in online supplemental tables S7 and S8. For BMI alone, effects are greater in high-income countries, while for zBMI alone there is a greater beneficial effect in low-income countries; this could be due in part to the small amount of data available from low-income countries. For the interaction terms, we observe no conflicting evidence between the analyses on separate outcome scales. The only interaction that appears with strong evidence in one of the separate outcome analyses that does not appear in the primary analysis is a synergistic interaction between interventions with an electronic component and interventions targeting both diet and physical activity in the analysis of BMI only. In sensitivity analyses of the main RE analysis assuming different correlations between observations at different time points, we observed negligible differences from the primary analysis across all estimated effects (online supplemental figures S12 and S13 (Section M)).