A systematic literature review was conducted to identify all published articles describing economic evaluation of telemedicine use. It is clear from the literature that telemedicine encompasses a wide range of different technologies and services used in a variety of medical settings. In this review telemedicine refers to the use of information and communication technologies in direct patient care, i.e. it covers only aspects in which the patient is directly involved and the patient and provider are separated by distance. Services that did not require encounters between patients and providers, such as radiology and pathology were therefore excluded. Technologies of interest were audio and visual (i.e. videoconferencing) or just visual (still-image telemedicine) or audio and data transfer (monitoring) excluding telephone calls without any transfer of medical data, traditional e-mails, and information and support sites on the Internet. This restriction has been imposed to make the review process more manageable and corresponds to a previous review 3.

Only articles written in English from 1990 to 2007 were included. These must have been published in peer-reviewed journals. This excluded books, HTA reports and unpublished materials. The articles included were full economic evaluations, i.e. they undertook a comparative analysis of both costs as resource use and outcomes in form of non-resource consequences of at least two alternatives, (or one if it was a cost-benefit study). The non-resource consequences typically refer to the effect that alternative interventions have on peoples' health status. The articles included described either applied studies, i.e. they generated primary data, or studies in which secondary data was modelled. Cost analyses where the consequences of two interventions were assumed to be identical across the alternatives were excluded. In this review, evaluations using outcome measures such as successful telemedicine consultations, travels avoided, hospitalisations avoided and other avoided costs have been categorised as cost analyses and excluded.

A literature search was undertaken using the British National Health Service Economic Evaluation Database (NHS EED), PubMed, PsychInfo, Embase, CHINAL, Ovid Nursing, British Nursing Index, ISI-Web of Knowledge, EconLit and Telemedicine Information Exchange. The two main journals in the telemedicine field, 'Journal of Telemedicine and Telecare' and 'Telemedicine Journal and E-health', in addition to the reference lists in the retrieved articles and existing reviews, have been manually searched.

The search strategy included the Thesaurus terms "telemedicine" (including all subheadings) and "videoconferencing" (as a single term) and various other relevant text words. This search strategy was combined with Thesaurus "Costs and Cost-analyses" (including all subheadings). The latter is one hierarchical level above "Cost-benefit analysis" in the sub-tree, has a high sensitivity and has been recommended as a search strategy to identify economic evaluations 1516. When medical subheading terms were not available, the term was searched as a keyword. The search strategy in the non-medical database included text searches using "telemedicine", "telehealth", "remote consultations", "videoconferencing", "teleconsultations" and "telecare", while "cost", "cost analysis", "cost-effectiveness" and "economics" were included in the Telemedicine Information Exchange search. As a supplement, free-text searches in the medical databases with the search terms mentioned above were also conducted to include the more recent economic evaluations.

Selection of relevant publications was based on information found in the abstracts. Full-text articles were retrieved when the abstract indicated analyses of both costs and non-resource consequences. Full-text was also retrieved for closer inspection if the abstract did not provide clear indication of the content. All abstracts and full-text articles were read by the author.

Data were extracted from the full-text articles based on international benchmarks on conducting and reporting of economic evaluations 717. Such recommendations often have forms as checklists and these identify key parameters that one may expect to find in well-executed studies. The main parameters used to assess the articles reviewed in this paper were clarity of study objective, adequacy of comparison, choice of study perspective and design, transparency in the measurements and valuation of costs and outcomes, reporting of data sources, handling of uncertainty, and presentation of results. These evaluation criteria have been chosen because they reflect the main challenges in telemedicine evaluations reported in the literature 218. In addition, details on the intervention type, medical field, type of analysis, and key findings have been collected (see Additional file 1). The complete data table is obtainable from the author on request.

The literature search identified 779 abstracts which were reviewed for relevance. For 89 of these, full-text articles were retrieved. After review, 33 articles were found to meet the inclusion criteria, i.e. they measured both costs and consequences of telemedicine.

Additional file 1 provides an overview of the studies included in this review. The articles covered several different medical specialities ranging from dermatology and cardiology to psychiatry. The use of telemedicine was most common in diabetes care (n = 6) and cardiology (n = 6). Almost half of the studies evaluated telemedicine in home care (n = 15), the rest evaluated services provided either in secondary care or between primary and secondary care institutions. The services provided used a variety of different technologies ranging from store-and-forward telemedicine (n = 16) to real-time videoconferencing (n = 7) and a combination of the two (n = 10) (see Additional file 1 for details). One third of the studies (n = 11) were published in the two main telemedicine journals: 'Journal of Telemedicine and Telecare' and 'Telemedicine and e-Health', while the others were published in a variety of medical and health research journals.

A clear description of the study objective together with information on the comparators is vital for assessing the quality of an economic evaluation. This will provide the reader with information both on the aim of the evaluation and on the decision-making context. In the majority of the studies (n = 28) the objectives were stated in a clear and unambiguous manner and mainly addressed choice of techniques as well as cost-effective resource allocations. One study stated clearly that the main objective was to test the feasibility of the technology 19. All but six studies 202122232425 provided a clear description of the comparators and sufficient information on the study groups (who received what). Almost all studies (n = 32) compared telemedicine to at least one alternative without telemedicine. This was referred to as a "usual care or current alternative" in some kind of on-site encounter. This is a relevant and appropriate comparator as it reflects the conventional approach to patient management. One study compared two different telemedicine techniques which appeared appropriate in their context 26. In six studies telemedicine was analysed as a complement to traditional home care 212728293031, while in another three, it was unclear if telemedicine was provided in addition to or instead of usual care 232425.

The majority of the studies reviewed were trial-based or observational and included randomised controlled trials (n = 13), non-randomised trials (n = 2), case control studies with the controls receiving standard care (n = 3), before and after studies (n = 3), case series (n = 1) and crossover trials with one group (n = 2) (see Additional file 1 for details). One study used two groups, but no information on study design was provided 25, one study used matched controls 32 and one study compared before and after results from different patients, which the authors acknowledged as a limitation 33.

Six of the evaluations were decision-modelling studies using secondary data to analyse costs and consequences (see Additional file 1). Five of the studies involved a literature search to obtain the primary studies 3435363738, while one study used results from one single study to derive the effectiveness evidence 39. None of the studies provided information on the search strategy, the inclusion criteria or the method used to extract relevant data (one study reported a MedLine search 37). The number of primary studies ranged from four to 14. Three studies did not provide information about the model used 353739, two used decision-tree modelling 3638 and one used a Markov model 34. The modelling studies analysed telemedicine as a means to support diabetes care 3438, pre-hospital care 3537, dermatology 39 and paediatric ophthalmology 36.

Two studies reported use of a societal perspective 3540, eight reported a health provider perspective 2733343637383941 and two reported a combination of health provider and patient perspectives 4243. One of the studies which reported a societal perspective only included direct health care costs 40 while the other also included costs borne outside the health care system (police, lawsuits and insurance) 35. Two thirds of the studies (n = 21) did not explicitly report the study perspective.

The costs of telemedicine can be divided into two broad categories: health care costs and non-health care costs. Direct health care costs refer to the physical health resources required to produce a specific health service or programme. Non-health care costs are those outside the health care sector, for example time costs such as production loss or lost leisure time, travel costs and costs associated with childcare. All studies in this review measured direct health care costs. These included costs related to investment, installation, call costs, personnel costs, and other health care costs. In addition, travel costs for health care personnel were included if a visiting service or home care was the alternative to telemedicine (n = 8) 2728293132343944. Fewer than half of the studies (n = 11) estimated private travel costs and even fewer estimated time costs as production losses (n = 8) (see Additional file 1 for details). For the majority of the studies (n = 22) the patient did not travel or the health provider (for example federal agencies in the US) paid for travel (n = 3) 293439. Five studies provided no 45 or very limited information 24303237 on the costs included, and in almost half of the evaluations (n = 14) the costing method was unclear. Fifteen studies provided details on resources consumed in physical units and reported prices or unit costs separately from quantities. Five of these reported only certain costs and quantities separately.

The studies reviewed appeared to have used some combination of an 'ingredients' approach and an 'activity' based approach to costing. The former is a costing method where every cost item is broken down into its underlying components, while the latter does not and refers to, for example, the costs of a hospital bed-day without any further information. An 'ingredients' approach was used to calculate the cost of providing the telemedicine service or the intervention, while 'activity' based costing was mainly used to calculate hospital costs and the costs of home care visits. This combination of 'ingredients' and 'activity' approaches is common in economic analyses in health care in general 46. Three studies used an 'activity' approach only 242737.

In one third of the studies (n = 10) some of the cost items were calculated using charges or reimbursement fees 19222425273334414748. These were mainly hospital in-patient costs, costs of outpatient visits and costs of home care visits. One study estimated the hospital costs as a cost-to-charge ratio 33, which is a method that calculates costs using prices adjusted for excess profit and is mainly used in the US 6. For the majority of the studies (n = 18) it was not possible to determine how hospital costs had been valued. One study reported the use of an anecdotal costing method and estimated hypothetical hospital savings 49. The costs of providing the telemedicine service were calculated using market prices and average staff wages. Time costs for patients and relatives were valued using different shadow prices ranging from actual loss of earnings 50, to estimates of production loss using the friction method 51. One study estimated lost earnings, lost overtime and lost unpaid work 43.

Tariffs or user fees do not necessarily reflect the true opportunity costs of resource use. Only two studies addressed the potential limitation of using charges to estimate the costs of an intervention 2733. None of the studies assessed telemedicine costing in relation to production capacity or cost sharing. Only three studies reported calculation of marginal costs 214150.

Additional file 1 shows that there is considerable variation in the measures used to assess the non-resource consequences in economic evaluations of telemedicine. The effectiveness measures presented in the studies ranged from impacts on process to final outcomes. The measures varied from diagnostic accuracy, blood glucose levels, anxiety and depression levels, physical capacity and health-related quality of life (HRQL) to life-years gained (LYG) and quality-adjusted life-years (QALYs).

Examples of process measures presented in the studies include diagnostic accuracy and agreement in reading of transmitted data (n = 5), medication compliance (n = 2), time spent on diabetes care for both patient and professionals (n = 1), number of days to independent pouch change after abdominal surgery and colostomy (n = 1) and length of stay (n = 3) (see Additional file 1 for details). Examples of disease-specific surrogate measures used were blood glucose levels, hypoglycaemic events, percentage reduction in wound size and body mass index. These are all measures that can be related to future health status. Seven studies used generic health-related quality of life measures (SF-36 and EQ-5D), of which two studies also used disease- or group-specific QoL-instruments 4252. Other generic outcome measures employed in the reviewed studies were return to activities of daily living (n = 3), mortality in the form of death avoided or life-years gained (n = 5) and QALYs gained (n = 4). One study measured disability-adjusted life-years gained 22. The majority of the studies (n = 24) used multiple outcome measures to evaluate effectiveness. Of the trial-based studies, three used secondary data to measure effectiveness, i.e. the effectiveness data was reported in different articles 234350.

The studies reviewed varied with regard to whether they reported their data sources or not. Four of the studies did not report data sources for either costs or consequences 19253249. All the remaining studies reported the sources for the effectiveness data (n = 29). These were, for example, questionnaires, existing literature, patient chart records, case notes, and other clinical databases. The sources for the cost estimates were explicitly reported in most of the studies (n = 25). These were hospital accounting systems, project diaries, patient charts, case notes and official databases. Four of the studies reporting data sources for the effectiveness data did not report sources for the cost data 30394547. The modelling studies used existing literature, official statistics, and databases; and some included expert and author opinions (see Additional file 1 for details).

Fewer than half of the studies (n = 13) 22233134353637383940425153 included sensitivity analysis to assess the robustness of the findings to variability in parameters or model inputs. One-way analyses were performed in nine studies 222336373839425153, multi-way sensitivity analysis was performed in one study 35 and both types of analyses were performed in three studies 344043. The majority of the studies (n = 23) used some kind of statistical analyses to calculate confidence intervals around point estimates for the outcome measures. Three employed non-parametric bootstrapping to estimate confidence intervals around mean costs 424351. In four studies costs were treated deterministically with no need to account for the variability in the cost estimates 19213239. Of the total studies, less than one third (n = 10) applied sensitivity analysis, together with appropriate statistics to address the variability in both the effectiveness measure and the costs (where these were not treated deterministically).

Costs and consequences were summarised in a cost per effect measure in less than one third of the studies (n = 9) 223435373839404551. Incremental cost-effectiveness ratios were calculated in five studies 3438394551. Five studies were CUAs while the rest were CEAs. Of the latter, four reported use of cost-minimisation techniques (CMA), and 19 took disaggregated forms and employed a cost-consequence framework (CCA) (see Additional file 1 for details). The latter lists all the different outcomes together with the costs instead of using a one-dimensional outcome measure. The costs in these studies were calculated as total costs, annual costs or unit costs. The latter were most frequently presented as a cost per patient or a cost per visit/session (n = 15).

It is important to consider whether the costs and consequences of interventions and their alternatives can be adapted from one context to another. Only six studies explicitly addressed the issue of generalisability to other settings 213435383940. Another two studies included extensive sensitivity analyses 4351, which improved the external validity. Limitations are however acknowledged in the majority of the studies (n = 18) with statements that aspects such as a retrospective design, too small sample size, the use of charges and fees, a high drop-out rate and wide confidence interval might limit the validity of the results.

In some settings, telemedicine services are replacing traditional in-person encounters between patients and health care personnel. In these situations disease- or case-specific measures are sufficient to estimate the relative effectiveness of telemedicine versus conventional approaches to patient management. These measurements can even be at an ordinal level. If specific outcome measures show equal or better patient outcomes than usual care, then the next step is to assess the differences in costs using standard costing techniques. This approach to telemedicine evaluation will however limit generalisability and make it impossible to compare or synthesise results from evaluations with different disease-specific outcome measures.

Telemedicine can also be used to provide a completely new service alongside traditional care such as monitoring of chronic conditions for patients living at home. In these situations telemedicine is provided in addition to traditional home care and could potentially improve patients' health. For example, if investing in telemedicine costs more and is more effective, the decision-maker would need information on how much more beneficial it is for the costs involved. To be able to compare this with other services and programmes generic health status measurements such as QALYs or LYG are required.

Consistency in effectiveness measures has important implications for the usefulness of cost effectiveness results to decision-making 5. This review found outcome measures ranging from diagnostic accuracy, blood glucose levels, and quality of life measures to QALYs. Diagnostic accuracy was frequently used together with surrogate measures. The former will ensure quality of the transmitted information. The latter is related to future health and can be viewed as inputs into a health production function 54. If the objective of using telemedicine technologies in diabetes care is to reduce and stabilise blood glucose levels, it seems appropriate for the endpoint to measure blood glucose levels. On the other hand, it can be difficult to interpret cost-effectiveness in terms of a specific cost per reduction in blood glucose level. Another example is if the decision is whether to invest in telemedicine in dermatology or not, the consequence measure employed can be intermediate and case-specific, i.e. related to skin problems. While these examples are acceptable for assessing technical efficiency, (i.e. how to produce a given level of health outcome at least cost) this will not help in deciding how to allocate resources across programmes. In these situations, generic health measures are required to allow for comparison between the two such as LYG or QALYs. Few studies used such generic health measures. One reason for this may be that these are not sensitive enough to detect small changes in health outcomes which telemedicine services most likely produce. Another reason may be that most economic evaluations have been undertaken to justify decisions within clinical areas and to support reimbursement and payment systems, and not as a basis for broader decision-making.

In the economic evaluation of telemedicine, the distinction between costs incurred within or outside the health care system seems particularly relevant. One argument for promoting telemedicine is that the technology has potential to generate cost savings that are mainly outside the health care system. In many health care systems, it is the patients who pay the travel costs that telemedicine eliminates, while the time savings are often valued as production changes. The stakeholders bearing the costs may differ from those experiencing the benefits, which in most cases are patients and employers. To include non-health care costs however requires the measurement of opportunity costs to society and not just the opportunity costs as health benefits forgone. These savings are excluded if one adopts a health provider perspective. It is therefore important to be clear about the viewpoint chosen for the analysis and how this affects the results. This will also help the reader judge whether all the relevant costs are included. Few of the reviewed studies did explicitly report their study perspective. It was however possible to infer the viewpoint in most studies. Not reporting the perspective explicitly is a common failing in economic evaluations in general, even if the costs included are clearly presented 5.

The costs included in the reviewed studies were mainly direct health care costs and direct non-health costs. The former referred to the costs associated with providing the alternatives while the latter were time costs and travel costs. In some studies however, it was impossible to tell the type and magnitude of the costs items included. More than half of the studies provided limited or unclear cost information and did not present unit cost data alongside resource data. This could make it difficult for the reader to assess whether all appropriate costs were included and to judge whether the cost results could be adapted to their own setting. It also appeared that in some studies the total cost per visit differed from the cost per patient. Another challenge is the inclusion and valuation of time costs. Very few studies provided any reasons for including time costs. For example, one study gave no rationale for including time costs for patients or their families and friends 21. Another study included lost work and lost overtime in addition to lost unpaid time 43. A third study included time costs for the patients in a sensitivity analysis 39. Presenting non-health care costs separately from health care costs in a sensitivity analysis may seem more appropriate, implying that the former may have less direct impact in decisions. In practice, patients may already be off work because of their health condition, leaving the actual production loss unchanged. There is however, a low level of agreement in the literature about whether to include productivity changes or not 75556.

A clear study objective and relevant comparators are important, since economic evaluation is concerned with measuring the marginal costs and benefits relative to their highest valued alternative use 55. In some analyses it was unclear whether telemedicine was provided in addition to or as a substitute for traditional health care. In one of the studies, it appeared that telemedicine visits were provided in addition to regular home visits without the costs of the latter being accounted for in the telemedicine alternative 31. To compare the cost per visit for telemedicine and home visits will be misleading if these costs differ from the total cost per patient. On the other hand, in a real clinical situation it seems unlikely that telemedicine could be a complete substitute for in-person encounters; some combination of the two will probably be required. Similarly, analysing two different telemedicine alternatives without comparison to a current alternative will not produce relevant information if the objective is to decide whether to invest in or reimburse telemedicine services.

To calculate the cost of, for example, an outpatient consultation or a hospital bed-day is often time consuming and costly. Since most evaluations operate within budget constraints, it is easier to use readily available cost data such as charges or tariffs. Even if these are often used to calculate hospital costs 6 in most health care systems they are only financial parameters with no relation to actual resource consumption. In health systems with both tariffs and lump sum financing, tariffs may cover only part of the total cost of an activity. Hospitals are often multi-product organisations with a high degree of cross-subsidies and tariffs may therefore only be vehicles for funding activities. Only a few studies acknowledge that using charges could limit the validity of their findings.

One of the main challenges in all economic evaluations is to balance the need for internal validity against the ability to generalise results to other settings. One third of the studies reviewed were evaluations alongside randomised controlled trials (RCT). RCT is a design with stringent criteria for selecting participants and a strict compliance to the study protocol, minimising potential bias. The RCT may however not be a suitable design for evaluating telemedicine, which is highly sensitive to local conditions. For instance randomisation will not ensure that the intervention is separated from the context 2, i.e. that the distance between the sites and other local costs are equally distributed in two groups. It is also common in telemedicine research to allow for self-selection. A naturalistic design or decision modelling might be more appropriate. The modelling studies in this review provided few details about both the model employed and the criteria used to ensure the validity of the primary studies, making it difficult to assess the robustness of the estimate measures. On the other hand, extensive sensitivity analyses improved the external validity of these studies.

Whether to use trial-based data or modelling studies should be seen in relation to the objective of the study and the viewpoint of those who are expected to use the results. Some researchers argue that trial-based studies are superior, while others view them as complementary rather than alternative approaches 4. Decisions about whether to invest in telemedicine should ideally be based on a synthesis of all available data and not just on a single trial. In telemedicine research, however, there are few studies of good quality available in the different medical fields 214, which limits the use of decision modelling.

Researchers should comment directly on the transferability of their findings 57. Some studies did acknowledge limitations, but these were mainly related to internal validity such as a low sample size and wide confidence intervals. It is also important to address situation-specific features that could influence the results, such as local treatment procedures or organisational structures. Costs and outcomes of interventions are always associated with some degree of uncertainty. Uncertainty may be due to sampling variation in estimates of both costs and outcomes and it may also be related to the economic model and the evaluation process 58. In addition to highly different contexts and local settings, other parameters such as perspective, assumptions regarding cost and outcome identification, measurements and valuation may affect the results. Over half of the studies did not include sensitivity analysis. This limits the usefulness of the cost-effectiveness data as a basis for health system decision-making.

Presenting the cost and outcomes disaggregated in cost-consequence analyses (CCA) has been suggested as a useful systematic framework in evaluating telemedicine 59. More than half of the reviewed studies presented their results using this framework. CCA lists all the different outcomes together with the costs instead of using a unidimensional outcome measure. This partly overcomes the challenge of deciding on a common outcome measure in different telemedicine settings, but severely limits generalisability. Multidimensional presentation of the consequences can also be difficult to interpret, especially if the different outcome dimensions move in different directions 54. Another challenge is how to value, for example, a two-week reduction in waiting time if the consequences are presented in a descriptive manner. Other typical benefits claimed for telemedicine that are difficult to value would include improved quality of service, speed of service and transfer of skills. More research is needed to address these aspects. Hailey (2005) argues that the one of the immediate needs in economic evaluation of telemedicine is to improve clarity in the reporting of the economic findings 18.