The introduction and scale-up of new tools for the diagnosis of tuberculosis (TB) has the potential to make a huge difference to the lives of millions of people – often those living in poverty. To realise these benefits and make the best decisions, policy makers need answers to many questions about which new tools to implement and where in the diagnostic algorithm to apply them cost effectively (1) (Fig. 1). The decisions can be difficult, particularly in those countries which have most to gain from the technology. Why is this?:
- New diagnostic tools for TB are often expensive to implement and use.
- The tools and contexts are developing, so what is most effective today may not be so tomorrow.
- Health system, patient, and longer term transmission impacts are uncertain.
- There are competing demands on scarce health system resources.
Here we explore virtual implementation as a tool to predict the health system, patient, and community impacts of alternative diagnostics and diagnostic algorithms for TB, in order to facilitate context specific decisions on scale-up. Virtual implementation is an approach that can model the impacts of implementation of a new diagnostics by taking data from the context being considered alongside data from contexts where the new technology has been implemented (probably as a trial). Through linked operational and transmission modelling components, the approach projects the effects on patients, the health system, and the community in the context being considered.
Fig. 1. Questions on impacts that the Virtual Implementation needs to address (1)
A linked modelling approach to understand both short and longer term impacts
The computer modelling methodology used for the virtual implementation is Discrete Event Simulation (DES). This technique is:
- Flexible – allowing many diagnostic options and contexts to be modelled.
- Visual – engaging policy makers in the modelling and validation (Fig 2).
- Detailed – taking account of the complex interactions that affect outcomes.
- Powerful – enabling the rapid simulation of 10 years of diagnosis across a country.
- Output rich – making outcome data readily available for further analysis.
Fig 2. Example DES screen view for diagnostic laboratory (Using WITNESS software)
In order to enable longer term impacts caused by changes in disease transmission to be included, the DES can be linked to a dynamic epidemiological model to project TB incidence, prevalence, and mortality. Some of the DES outputs become inputs into the dynamic epidemiological model (e.g. diagnostic default%), and some of the outputs of the dynamic epidemiological model become inputs into the DES, as illustrated in Fig 3 (2).
Fig 3. Linked Discrete Event Simulation and Dynamic Epidemiological Components (2)
This virtual implementation approach has been validated and tested using data from Tanzania and could be applied to other more centralised contexts such as those in to South Africa.
Virtual implementation has been used to assess four combinations of different diagnostic options for TB diagnosis in diagnostic centres in Tanzania. These options are shown in Table 1.
Table 1 – TB Diagnostic Options for a diagnostic centre in Tanzania
|Option||Name||Primary Diagnostic Tool||Drug Sensitivity Testing||Treatment monitoring|
|Base case||ZN||Ziehl Neelsen microscopy – 2 sputum samples||DST in Central TB Reference Lab (CTRL)||ZN Microscopy|
|A||LED||LED Fluorescence microscopy – 2 samples||DST in CTRL||LED Fluorescence Microscopy|
|B||Xpert Full||Xpert MTB/RIF – 1 sample||Xpert MTB/RIF in Diagnostic Centre & DST in CTRL||LED Fluorescence Microscopy|
|C||Xpert Partial||Xpert MTB/RIF for known HIV+ suspects only- 1 sample LED microscopy for other cases – 2 samples||Xpert MTB/RIF in Diagnostic Centre & DST in CTRL||LED Fluorescence Microscopy>|
Results across many outcome variables were compared, including impacts effecting patients and the health system – see Table 2.
Table 2 – Example results from virtual implementation a diagnostic district in Tanzania
|Performance||Difference to Base case|
|Base Case ZN||A|
|Mean Time to treatment (days)||24.8 (24.7-24.9)||22.5 (22.4-22.6) **||15.0 (14.9–15.1) **||18.6 (18.5-18.7) **||-9%||-39%||-25%|
|Mean No. of visits/ patient||6.0 (5.9-6.1)||4.5 (4.4-4.6) **||3.7 (3.6-3.7) **||3.8 (3.7-3.8) **||-25%||-38%||-37%|
|Test +ve TB case /yr||562 (545-578)||670 (648-691) **||1060 (1041-1079) **||898 (882-915) **||+108||+499||+337|
|Test -ve TB case /yr||446 (434-458)||355 (343-367) **||53 (48-57) **||188 (178-197) **||-91||-393||-258|
|Total TB treatment cases /yr||1008 (988-1028)||1025 (1002-1048)||1113 (1093-1133) **||1086 (1068-1103) **||1.7%||10.5%||7.7%|
|MDR-TB Cases /yr||3.7 (2.4-5.0)||3.5 (2.2-4.8)||6.6 (5.0-8.2) **||4.6 (3.1-6.1)||-0.2||+2.9||+0.9|
|Samples /yr (1,000’s)||14.3||14.2||9.1||12.1||-1%||-36%||-15%|
|% Initial default||15.7% (15.5-15.9)||13.9% (13.6-14.2) **||10.7% (10.5-11.0) **||10.7% (10.5-10.9) **||-1.8%||-5.0%||-5.0%|
|Treatment||*Patients cured excludes estimated false positives who receive treatment but had no TB|
|Patients Cured p.a.* (95% C.I.)||842 (827-858)||884 (866-902) **||975 (955-995) **||933 (917-948) **||5.0%||15.8%||10.8%|
|False + Rate for TB||14.3%||11.5%||8.7%||9.3%||-2.8%||-5.6%||-5.0%|
|No. of lab staff Utilization||2 79%||2 48%||2 18%||2 37%||0 -31%||0 -61%||0 -42%|
|Costs in US $’s||+Investment costs have been discounted over 5 years at 5% per year|
|Incremental running cost/yr||95,226||95,259||142,908||124,934||+33||+46,683||+29,708|
|Incl. investment+ costs/yr||95,226||95,690||148,419||128,555||+464||+53,194||+33,330|
95% confidence limits for the means in brackets
** – significantly different from base case at 95% level
The results demonstrate that useful projections of the effects on the health system, running costs, and patient outcomes of alternative TB diagnostic strategies can be produced. In this resource constrained setting, the models estimate a 5% increase in TB cures could be delivered at very low investment by the implementation of LED fluorescence microscopy. With increased funding of $46,800 per annum and an investment of $34,700 the benefits in patients cured would rise to around 16%. These benefits principally accrue from earlier case detection for smear negatives, a reduced diagnostic default rate, and a reduction in false positive diagnosis. The increase in the overall number of patients started on TB treatment is small. This is because, rather than identifying many new cases, Xpert MTB/RIF brings forward many cases that are currently diagnosed as a result of clinical judgement following a negative smear test. Implementation of Xpert MTB/RIF just for HIV+ suspects would require a lower initial investment and reduced ongoing costs compared to full roll-out. However, the estimated increase in TB cures would be down to 11%.
In order to understand and compare the cost effectiveness, benefits and financial sustainability of each intervention, the outputs from the model have been used to calculate the benefits measured in terms of Disability Adjusted Life Years (DALY) averted (3). Fig. 4 demonstrates how this analysis might be used when comparing options for implementation. The benefits, in terms of DALY’s averted, is represented by the size of the circle and the financial sustainability by the incremental health system costs (horizontal axis). The incremental cost effectiveness ratio (ICER) has been calculated using the incremental health system costs divided by incremental DALY’s averted (vertical axis).
Fig. 4 – Cost effectiveness and sustainability for options in an example diagnostic centre in Tanzania
In this example all interventions would be considered cost effective if the threshold for cost effectiveness is set above $80 per DALY averted. This figure is well below the Gross Domestic Product (GDP) per capita for Tanzania which is often used as a benchmark for cost effectiveness. However cost effectiveness is not the same as financially sustainable. In the example below if $40,000 per year was considered the maximum sustainable incremental expenditure then full roll-out of Xpert MTB/RIF (Option B) in this location would not be sustainable, whereas partial implementation of Xpert MTB/RIF for HIV positive individuals seeking diagnosis (option C) would fall below the $40,000 per year cut-off. This option would deliver an estimated 843 DALY’s averted per year which is more than double what implementing LED fluorescence microscopy would achieve, but substantially less than full roll-out of Xpert MTB/RIF (Option B). So if the higher expense of full roll-out is sustainable ($53,300 per year) then full roll-out of Xpert MTB/RIF would be the preferred option.
In conclusion, virtual implementation provides information to help policy makers understand context-specific impacts of new TB diagnostic tools. The approach enables cost effectiveness and sustainability analysis to be completed which can assists policy makers in decisions and identifying priorities. The approach has been successfully applied and is now being used to assist policy makers in Tanzania to guide national TB diagnostic strategies and prioritise which diagnostics should be implemented in which districts (4). The approach can also be applied to other more centralised contexts and is currently being explored as a tool to assist in important decisions concerning multi-drug resistant TB (MDR-TB) diagnostic tools in Brazil, South Africa, and Russia.
Acknowledgements: The research is part of the TREAT TB initiative funded by USAID and led by the International Union Against Tuberculosis and Lung Disease (The Union). This document has been produced thanks to a grant from USAID. The contents of this document are the sole responsibility of the authors and can under no circumstances be regarded as reflecting the positions of The Union, nor those of its Donors.