Published on June 13, 2012 by

Measuring the performance of ART programmes in Sub-Saharan Africa

Why assess the performance of ART programmes?

Development assistance for health allocated to HIV/AIDS has dramatically increased from US$ 0.8 billion in 2000 to $US 5.1 billion in 2007 (1). Specifically, with the creations of the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) and the United States’ President’s Emergency Plan for AIDS Relief (PEPFAR), antiretroviral therapy (ART) scale-up has been remarkable. Among low- and middle-income countries, 400,000 people were receiving ART in 2003, when the “3 by 5” initiative was launched; this increased to about 5,000,000 by the end of 2009 (2). Yet, in 2009, 33.3 million people were HIV-positive worldwide and 2.6 million new infections had occurred (3). The HIV epidemic is becoming financially unsustainable: the aids2031 working group estimates that an annual US$ 35 billion is needed in developing countries by 2031 to address the epidemic, if today’s status quo is maintained (4).

It is therefore essential to assess the effectiveness and cost-effectiveness of different HIV treatment programmes and models so that the existing limited resources can be allocated optimally. Data are gathered periodically to measure the retention of patients on ART after treatment enrolment at the national level (2,5), though this data collection is often complicated by patients lost to follow-up (LTFU) (5,6). In 2008, the mean retention rates on ART at 12 and 24 months were respectively 75.2% and 66.8% in Sub-Saharan Africa (5). However, little work so far has assessed the performance (effectiveness and cost-effectiveness) of ART programs differing by the kind of providers and subsequently identified the “good” versus (vs.) “bad” performers.

Identifying the determinants of good performance for ART programmes, if possible both at the patient- and provider-level, is essential (7): decision-makers will then be able to potentially improve ART delivery in countries.

Measuring performance

In order to understand performance of ART programmes, one must identify and understand the complex landscape of ART providers and patients in Sub-Saharan Africa (7-9). The attributes characterizing this landscape are attributes of care at the ART facility level which are important determinants for the patient adherence (7). These attributes are: the facility type (e.g. rural or urban facility, large hospital or HIV-dedicated clinic); the management type (e.g. government- or privately-managed); the distance to the facility (for the patient); the size and characteristics of the facility personnel (e.g. if there are physicians, nurses, community health workers); the presence of counselling services to patients; the type of antiretroviral drugs delivered (about 10% of sub-Saharan African ART facilities are providing tenofovir-based antiretroviral drugs (2)); the drug regimen (about 2% of ART patients in low- and middle-income countries received second-line regimens in 2009 (2)); etc.

Moreover, some patient attributes must be considered. These attributes are for example: sex, age, CD4 count and tuberculosis (TB) diagnosis at ART initiation, and pregnancy status for women. In sub-Saharan Africa, 65% of ART patients are women (10), the mean age at ART initiation is 33 years for women and 38 years for men (10), the median CD4 count at ART initiation is about 110 cells/mm3 (10), and about 30% of female ART patients are pregnant (2). About 10% of ART patients are diagnosed with TB at ART initiation (11).

Methodologically, measuring survival of ART patients is rendered difficult by the issue of LTFU patients (5,6). In a large number of Sub-Saharan African HIV treatment programmes, a substantial number of ART patients can be lost to follow-up, whose status (dead or alive) is subsequently unknown. Therefore, for a given ART programme, counting solely the deaths within the programme cohort leads to overestimating programme survival, whereas assuming that all LTFU patients died leads to underestimating programme survival. Several correction methods including mortality among LTFU patients in survival estimates have been proposed (12-14). A recent study (14) gathered data from Sub-Saharan African ART cohorts, and including mortality among both patients still on ART and patients LTFU produced corrected estimates of survival on ART. Precisely, the study used mathematical modelling and data on retention, mortality and loss to follow-up, in order to produce corrected survival curves for ART patients up to five years after ART initiation. The estimated life-years gained for five patient-years were about 2 years (14): counting solely the deaths within the programme cohort led to overestimating programme survival of +15%, whereas assuming that all LTFU patients died led to underestimating programme survival of -15% (14). Faster declines in corrected survival were estimated as CD4 count at ART initiation went from > to < 100 cells/mm3 (14). Accounting for mortality among LTFU patients substantially corrects the estimated effectiveness of ART programmes i.e. the life-years gained for patients on ART, compared with the commonly used measurements of retention and survival within ART programmes; it provides a critical tool for the evaluation of HIV treatment programmes’ performance.

Also, assessing costs of ART programmes is difficult as a substantial variation has been observed in cost estimates between different ART programmes. For example, one study (15) reported costs of PEPFAR-supported clinics providing free HIV treatment in several Sub-Saharan African countries, and found a median annual cost of US$ 880 per patient. Newly initiated ART patients cost about 20% more than longer-term patients, and per patient costs decreased as sites matured after scale-up (15). Treatment costs varied between countries and were sensitive to changes in the antiretroviral drug regimens and the packages of services delivered (15).

Improving performance measurement

The attributes of ART programmes listed above, complemented by analytical methods for the estimation of patient survival in those ART programmes, can well inform the selection of a sample of facilities for data collection in Sub-Saharan African countries, also including the selection of a representative sample of patients. A first step would be to use the data collected by the International epidemiological databases to evaluate AIDS (IeDEA,, and to try to identify the determinants of ART programmes performance (e.g. effectiveness and cost-effectiveness). One would then be able to potentially flag the characteristics of “good” vs. “bad” performance, and to draw meaningful conclusions with regard to the determinants of the cost per year of life lived on ART. A second step would be to augment the list of facilities’ attributes and patients’ characteristics exposed here. Additional determinants, e.g. the number of supervision visits in a given programme and specific leadership features in a given programme, could well be significant. In fact, leadership and non-tangible random effects may well be the most important determinants of good performance.