Published on June 14, 2011 by

Heterogeneity and HIV prevention trials

Randomised controlled trials (RCTs) are used to evaluate HIV prevention methods conducted among populations with a heterogeneous risk of HIV infection among individuals. The effects of heterogeneity on these RCTs were explored in a recent publication (1) and here the key findings of that study are summarised.

Designs in HIV prevention trials: parallel versus crossover

HIV-prevention trials use a parallel design which, in most of cases, consists of randomising individuals in either an intervention or control group for the follow-up period. This design was thought to be appropriate in the context of binary endpoints such as HIV-infection. However, the heterogeneity in the risk of infection may be high when RCTs are conducted among participants recruited from the general population. As a consequence, if the intervention is effective, individuals at higher risk of infection are lost in the control group at a higher rate than those in the intervention group. This can result in a dilution of the effect of the intervention over time.

An alternative to the parallel design is the crossover design. Under this design, individuals are followed-up for at least two periods and outcome is measured at the end of each. There the endpoints, which can be censored data (2), are assessed for each individual under treatment. In the case of the infection with HIV, the endpoint is subject to censoring and the corresponding alternating treatment design is a ‘crossover design with single failure-time endpoint’ which will be named ‘crossover design’ here.

The use of the crossover design has some ethical advantages. It was first used in the treatment of chronic granulomatous disease (3) for these ethical advantages. In the case of HIV infection for example, the incidence is the same in both study arms. The potential benefits and risks are therefore better distributed between all participants than in the parallel design. The use of the crossover design can also make the recruitment easier and reduce loss to follow-up. In the case where the treatment involves unequal participation time and effort for volunteers, the crossover can justify the same financial allowance. The use of the crossover design may also be considered when a RCT is needed to convince decision makers to promote a prevention method. Moreover, even when the parallel design and crossover design can both provide a good estimate of the true effect, the crossover design achieves a better variance (4).

How often is the crossover design used in HIV-prevention trials? A bibliographic search in MEDLINE, EMBASE, Web of Science, and the Cochrane databases was performed. Papers published up to December 2008 that included “HIV”, “Randomised controlled trial”, and “Africa” as keywords or text in the title or the abstract were registered (1). It came out that all the identified studies used the parallel design, despite the fact that the crossover design has some potential advantages. Most of the studies (60%) were conducted in several sites with randomisation and statistical analysis stratified according to site. 27% of the studies found were conducted among female sex workers. It appeared that the heterogeneity in risk within study sites was neither accounted for in the sample size calculation nor discussed as potential limitation of the findings. Only two studies mentioned this issue. In the microbicide trial published in 2008 (5), the authors questioned whether they should have used two cohorts of women: one with a high coital frequency and one with a low coital frequency. In the diaphragm trial published in 2007 (6), the authors suggested that recruitment could have been organized to select a population with a higher risk of infection and higher compliance, which would have reduced heterogeneity (1). A call for the use of the crossover design in the field of anti-HIV microbicide was recently made (4). A new design of crossover trial was presented and it was argued that there is place for crossover trial designs with absorbing endpoints such as HIV infection.

Parallel design or crossover design: what do theoretical calculations suggest?

The theoretical calculations (1) showed that when the study population was truly homogeneous, the parallel design and the crossover design were equivalent and yielded an unbiased estimation of the effect of the intervention. Four main results were obtained from these theoretical calculations: a) both the parallel design and the crossover design estimate effects overestimated the true effect of the intervention, leading to an underestimation of the true impact of the intervention. This bias towards the null is smaller when using the crossover design, increases with mean HIV incidence in the study population, and increases with the duration of follow-up; b) the variance is generally smaller when using the crossover design, which was also observed in another study (4); c) the statistical power in both designs is smaller than what can be expected when it is assumed that the population is homogeneous, with the statistical power of the crossover design being similar to or higher than that of the parallel design; and d) when using the crossover design, the average HIV incidence rate over the entire follow-up period is the same between both randomisation groups, regardless of the true effect of the intervention.

Parallel design or crossover design: what do simulations suggest?

Numerical results for the comparison of the parallel and the crossover designs were assessed through simulations. One study simulated a RCT having two follow up periods of one-year duration each (1). Many scenarios were designed and the parameter values were chose to be similar to those observed in sub-Saharan Africa. For example, the incidence of HIV and the effect of the intervention were assumed to be constant, equal to 0.035 per year and 0.4 respectively. The effects where then estimated using parallel and crossover design respectively. Results from these simulations revealed that: a) both designs gave unbiased estimates of the effect when the population was assumed to be homogeneous with a power of 0.85; b) the effect of the intervention was overestimated, varying from 0.46 to 0.66, when using the parallel design in a heterogeneous population. The bias increased with the heterogeneity; c) the estimated effects given by the crossover design were close to the true effect, ranging from 0.40 to 0.41; and d) the statistical power obtained for a heterogeneous population decreased with the heterogeneity, from 0.68 to 0.22 when using the parallel design and from 0.83 to 0.79 when using the crossover design.

Crossover design should be considered when intervention effects are rapidly reversible

In a population with a heterogeneous risk of HIV infection, those with a high risk become infected with HIV on average sooner than those with a lower risk of infection. The average risk therefore decreases over time in each group. In the case of an effective effect of the intervention, this diminution is faster in the intervention group and leads to an underestimation of the true effect of the intervention. This is an example of the noncollapsibility of the hazard ratio and its built-in selection bias, induced by an attrition process over time due to some high baseline rates (7).

When the treatment is alternated, those who stop receiving the intervention become more at risk compared to those starting the intervention. This compensates to some extent the effect described above. As a result, the crossover design yields a better estimation of the true effect. Finally, the crossover design provides an estimation of the true effect which is less diluted and has a lower variance in some cases (4). Thus the statistical power of the crossover design is higher than that of the parallel design.

The above observations lead to the conclusion that the heterogeneity in individual risk of HIV infection is an underestimated problem which should be taken into account when designing and interpreting RCTs that test prevention methods of HIV heterosexual acquisition in adult sub-Saharan African populations with a high HIV incidence. When the effects of tested interventions are rapidly reversible, the use of the crossover design should be considered.