Email updates

Keep up to date with the latest news and content from BMC Medical Ethics and BioMed Central.

Open Access Research article

The activity of French Research Ethics Committees and characteristics of biomedical research protocols involving humans: a retrospective cohort study

Evelyne Decullier1, Véronique Lhéritier2 and François Chapuis3*

Author Affiliations

1 UF de méthodologie en recherche clinique, Département d'Information Médicale des Hospices Civils de Lyon, Lyon, France

2 Hôpital Edouard Herriot, Lyon, France

3 French National Conference of Research Ethics Committees, Lyon, France

For all author emails, please log on.

BMC Medical Ethics 2005, 6:9  doi:10.1186/1472-6939-6-9

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6939/6/9


Received:21 March 2005
Accepted:17 October 2005
Published:17 October 2005

© 2005 Decullier et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Clinical trials throughout the world must be evaluated by research ethics committees. No one has yet attempted to clearly quantify at the national level the activity of ethics committees and describe the characteristics of the protocols submitted. The objectives of this study were to describe 1) the workload and the activity of Research Ethics Committees in France, and 2) the characteristics of protocols approved on a nation-wide basis.

Methods

Retrospective cohort of 976 protocols approved by a representative sample of 25/48 of French Research Ethics Committees in 1994. Protocols characteristics (design, study size, investigator), number of revisions requested by the ethics committee before approval, time to approval and number of amendments after approval were collected for each protocol by trained research assistant using the committee's files and archives.

Results

Thirty-one percent of protocols were approved with no modifications requested in 16 days (95% CI: 14–17). The number of revisions requested by the committee, and amendments submitted by the investigator was on average respectively 39 (95% CI: 25–53) and 37 (95% CI: 27–46), per committee and per year. When revisions were requested, the main reasons were related to information to the patient (28%) and consent modalities (18%). Drugs were the object of research in 68% of the protocols examined. The majority of the research was national (80%) with a predominance of single-centre studies. Workload per protocol has been estimated at twelve and half hours on average for administrative support and at eleven and half hours for expertise.

Conclusion

The estimated workload justifies specific and independent administrative and financial support for Research Ethics Committees.

Background

Clinical trials in biomedical research throughout the world must be evaluated by research ethics committees. No one has yet attempted to clearly quantify the activity of ethics committees and describe the characteristics of the protocols submitted, and only five studies[1-4] have dealt with this subject, two in England and the others in the USA, Australia and Spain. However, they each focused on only one or two research ethics committees, and no knowledge is available on a national or even a regional level. In addition, four of these studies[2-4] concentrated on the fate of protocols; some characteristics were gathered but they were presented only for protocols whose investigator was not lost to follow-up. None of these studies evaluated research ethics committees' workload, but one study[1] described the activity in terms of number of meetings, approvals and reasons of queries.

France was one of the first country to affirm these principles through the Huriet-Sérusclat Act of 1988 [5], which launched administrative and financially independent research ethics committees (REC) in 1991, called CCPPRBs (committees for the protection of human beings involved in biomedical research). Each protocol involving human beings in France has to be assessed by one of these committees.

A European harmonization was needed since a long time and was decided in 2001 [6]. All European governments were asked to change national laws according to this directive. Research ethics committees will be created where they do not exist currently and it is of upmost importance to anticipate activity, workload, financial and administrative support needed by these committees.

We describe one year of activity in a sample comprising half of the French RECs and the full characteristics of the protocols approved in these RECs during this year.

Methods

All biomedical research protocols approved by a representative random sample of every other French RECs the participating committees between 01 January 1994 and 31 December 1994 were included. This year was chosen in order to allow studies to be completed and so to collect data on all amendments, information and serious adverse effects.

Definitions

The definitions used in this study are drawn from French law on the subject (table 1).

Table 1. Definitions

Data collection

Ethics committee activity and protocol characteristics were gathered into four thematic groups. The first two comprised committee activities: (i) the process of approval (modifications requested before approval, approval date, etc.), (ii) the modifications requested after approval and the information transmitted. The other two comprised protocol characteristics: (iii) the legal and administrative characteristics of protocols (investigator, sponsor, etc.) and (iv) the proposed scientific characteristics (duration, sample size, etc.). Research assistants in charge of data collection were trained in order to ensure homogeneous results. The questionnaires were completed using the committee's archives, and were then sent to the coordinating centre. An identification number was given to each protocol in order to ensure anonymity of the investigator.

Assessment of activity and expertise workload

Activity

Each formal step was studied, i.e., the number of protocols approved, revisions requested by the ethics committee and amendments submitted by the investigator were studied. The number of revisions, amendments and total exchanges (defined as the sum of revisions and amendments) were calculated per approved protocol.

Administrative and expertise load

At the REC level, each protocol is registered and assessed for approval. A list of the different tasks was established by the Lyon B REC members and administrative staff and submitted to a national panel of REC members (table 2). The time required to accomplish each task was estimated based on individual experience and checked on site. For this study, the estimated times devoted to administrative work and expert assessment were then accepted by consensus among administrative staff and REC members respectively. The workload is estimated from a protocol perspective, i.e. each protocol is valued. We did not assess neither fix activities, such as or training of the members or structure management (accounting, legal, insurances, structure organisation,...), nor the committee meetings.

Table 2. Development of REC activities

Statistics

As this was mainly a descriptive study, we presented frequency distributions calculated with SAS software®. A Kaplan-Meier survival analysis [7] was performed with SPSS software® to establish probability of approval curves based on cumulative hazard function in order to study the time between the submission of the protocol and final approval (in days). A log-rank test [7] was then computed to compare protocols with direct approval, protocols with minor modifications and protocols with revisions.

A multiple correspondence analysis [8] was also performed on protocol characteristics. To determine the number of axes we used the scree test [9] and the total inertia was computed with Benzecri formulae [10]. To explain the meaning of each axis, modalities of variables were cumulated until 80% of inertia was explained. The individual coordinates were then used to obtain a hierarchical clustering with the Ward minimum variance method [11]. The number of clusters was chosen using the cubic clustering criterion [12].

To compare number of revisions, amendments and time per protocol between the clusters an ANOVA was performed [13].

Results

A total of 25/48 RECs throughout France participated in this national study. There were 1143 declared protocols approved by these committees during the year 1994. One hundred sixty-seven had to be excluded because the inclusion criteria were not fulfilled (Table 3). Thus 976 protocols were eligible (mean per committee, 39; median, 37; range, 17–81).

Table 3. Reasons for non-inclusion (167 protocols out of 1143)

REC activity

Approval

Only 31% of protocols were approved with no request for modifications. Minor or major modifications were requested for the remaining protocols.

Time from submission of application to approval

Because of missing registration and/or approval dates, seven protocols were dropped from the study, and the analysis was performed on 969 studies. The median time to approval for protocols with direct approval, with minor changes requested, and request for major changes (revisions) was, respectively, 16 days (95% confidence interval: 14–17), 27 days (24–30) and 48 days (43–52). The comparison between the three groups resulted in a significant log-rank test (figure 1).

thumbnailFigure 1. Probability of approval from time of submission, according to the initial decision of Research Ethics Committees.

Number of modifications studied

On average, the number of revisions required was 39 per committee and per year (median, 30; range, 4–132, 95% CI, 25–53) and the number of amendments was 37 (median, 28; range, 3–91, 95% CI, 27–46). Thus the average number of exchanges (revisions and amendments) was 76 (median, 60; range, 19–212, 95% CI, 55–96) per year and per committee, ranging from 13 exchanges for phase I studies on drugs, through 19 for studies not investigating drugs and to 45 for studies on drugs other than phase 1.

Revisions (major changes)

Revisions were requested for 555 (57%) protocols, but only 181 (19%) required a second revision or more. A revision could contain one or more points to be modified by the investigator. In total 1438 points for modification were cited (Table 4). Per protocol, the average number of reasons per revision was 2.01 (the same reasons could have required more than one revision). On average, one reason (or less) was mentioned in 45% of revisions, between one and two reasons in 25%, between two and three 15%, and more than three reasons in 15% (data not shown).

Table 4. 1211 main reasons for revision before approval (out of a total of 1438)

The main reasons for revisions were information to the patient (28% of responses), consent modalities (18%), inclusion criteria (8%), scientific factors (7%) and legal and administrative requirements (6%). This global ranking of reasons remained the same when considering the sub-group of protocols with only one revision and the sub-group of protocols with two revisions.

The ranking of reasons for protocols with three or more revisions showed that legal and administrative points, which were quite easy to solve, became less important, falling from the 4th position (for protocols with one or two revisions) to the 9th position. However, requests for more specific methodological and statistical information, which were more complex to solve, were more frequently cited, rising from the 11th rank (for protocols with one or two revisions) to 7th rank.

Amendments (after approval)

Over half of the investigators submitted no amendments (57%), although few protocols were not implemented and therefore could not have led to amendments. For the 416 protocols requiring at least one amendment, 875 reasons were mentioned. An average of 1.2 reasons per amendment was cited. A single reason was cited in 52% of cases.

The main reasons for which protocols had to be modified after approval were a change in the number of subjects to be included (18%), a modification of inclusion criteria, a modification of the timetable and changes in examinations and treatments (Table 5). Regarding the legal and administrative points cited, 43% concerned the update of the list of investigators (data not shown). The number of amendments submitted did not modify the ranking of these reasons.

Table 5. 657 main reasons for amendment after approval (out of a total of 875)

Additional information

As information on the progress of research is not mandatory, 81% of investigators did not send any further information regarding their study. When they did it was mainly to warn of the end (premature or normal) of the study (30%), to declare adverse side effects (25%) and to inform the committees of the intermediate or final study results (18%).

RECs' administrative and expertise workload

The administrative staff panel estimated that administrative tasks (registration, postal service, copying, classification, letters, etc.) required 4.5 hours per initial protocol recording, 4.5 hours per revision and 3.5 hours per amendment (table 2). The experts panel estimated that the expertise process required 6 hours per protocol, 3.25 hours per revision and 2.5 hours per amendment. This was confirmed during the test on-site.

These figures show, per year and per month respectively, an estimated administrative workload of 480.50/40 hours and an estimated expertise workload of 453.25/38 hours per REC (based on observed mean annual activity per committee: 39 protocols, 39 revisions and 37 amendments on average per REC and per year).

Protocol characteristics

Table 6 summarizes the main legal and administrative characteristics of protocols and Table 7 the main scientific and technical characteristics.

Table 6. Administrative characteristics of approved protocols

Table 7. Technical characteristics of approved protocols

Legal and administrative characteristics

Sponsors were mainly pharmaceutical firms (64%). The research setting was a tertiary teaching hospital in 55% of cases; another 15% of protocols were conducted simultaneously in a tertiary teaching hospital and another type of institution. The item "other" included other combinations of these settings and also private offices, etc.

Technical characteristics

Drug evaluation was the object of research in 68% of cases. For the research topic, the item "other" was cited in 8% of cases, usually for studies on genetics or vaccines, i.e., other types of clinical trials. It was also pointed out that participating RECs encountered problems in reporting expected duration since 26% of the answers on questionnaires neglected to mention this information.

Reasons for revisions varied according to investigators' status (p < 0.0001; revisions on scientific prerequisite were more frequent when investigators were neither professor, nor assistant) and to place of research (p = 0.005; revisions on inclusion criteria were more frequent when research was conducted in phase I units) and reasons for amendments according to sponsor (p = 0.01; amendements on sample size were more frequent when the sponsor was a public hospital), place of research (p = 0.03; amendments on treatment was more frequent for research conducted in phase I units), study phase (p = 0.04; amendements concerned more often treatment for phase I studies), study scope (p = 0.002; sample size was more often modified when the study was national and multicentre) and study duration (p = 0.001; amendments were more often about treatment when study duration was less than 2 months).

Multiple correspondence analysis

the scree test resulted in the selection of two axes explaining 40% of the inertia. The modalities most contributing to the construction of each axis were represented on figure 2. The first axis (dashed line) showed that study size differentiated protocols the most. On one side were phase I studies, protocols studying nutrition, cosmetics, protocols on physiology, without direct benefit to the patient, of short duration, single-centre studies, and those with fewer than 20 patients expected (small studies). On the other side were phase III studies, those with direct benefit to the patient, more than 200 patients to be included, national multicentre or international studies, and those with a planned intermediate analysis (large studies).

thumbnailFigure 2. Graphical representation of multiple correspondence analysis on protocols characteristics*. *graphical representation of the modalities which contributes the most to the construction of axes 1 and 2.

The second axis separated descriptive studies sponsored by the public sector from experimental studies on drugs, mostly sponsored by private pharmaceutical firms.

Hierarchical clustering using Ward's minimum variance

This analysis yielded four clusters explaining 75% of the variance. The first cluster included protocols on drug testing (98% in this cluster vs. 68% in the global population), with direct individual benefit, and phase II and phase IV studies (respectively, 35% and 22% vs. 18% and 11%). Cluster 2 grouped phase I drugs trials (89% vs. 17%), without direct individual benefit (97% vs. 34%), in a single centre (99% vs. 47%), lasting less than 2 months and sponsored by pharmaceutical firms. Cluster 3 also concerned drugs, but protocols were phase III studies (64% vs. 23%), with direct individual benefit, multicentre and international (42% vs. 16%) and with a randomized design (87% vs. 52%). Finally, cluster 4 grouped protocols not evaluating drugs (89% vs. 32%), the studies were descriptive (32% vs. 15%) and the sponsor was in the public sector (58% vs. 27%).

Activity and workload in relation with protocol characteristics

We have linked the different clusters to the number of revisions and amendments (table 8). The mean number of revisions is homogeneous through clusters with a minimum of 0.94 for cluster I and a maximum of 1.08 for cluster III. Number of amendments varied more: from 0.46 for cluster II to 2.14 for cluster III.

Table 8. Tendendy in characteristics of 976 protocols according to clusters defined by a hierarchical analysis

The mean time spent per protocol (workload) was estimated in each cluster, cluster III protocols are the most time-consuming.

Time spent by each cluster was obtained by multiplying number of protocols by mean time per protocol. At a national level 31% of committees' time is spent for cluster IV protocols, 28% for cluster III, 27% for cluster I and 13% for cluster II.

Discussion

On average, each REC studied 39 protocols, 39 revisions and 37 amendments per year, representing an annual workload of 934 hours (480.50 hours for administrative tasks and 453.25 for expertise tasks, on a monthly basis respectively 40 and 38 hours) including neither committee meetings nor training of the members since there is too much variability across committees. Most protocols evaluated drugs (68%), were experimental (85%) and were monocentric (47%).

The main advantage of our study is to have collected exhaustive information for half of French RECs over a full year (whether or not the principal investigator was lost to follow-up). Moreover, homogeneous data collection was guaranteed by the initial training followed by research assistants. The choice of the year 1994 was based on initial studies conducted by the French Ministry of Health [14-16] showing that on average six years were needed to reach publication, and that some studies were still ongoing eight years later. This choice enabled us to gather all the information during the whole protocol's duration. To our knowledge our study is the first to have evaluated the workload and the activity of RECs for each protocol through the collection of the number of revisions, amendments and additional information. The combination between 1994's activity data and 2004's workload assessment is justified since committees' missions have remained the same and there is no reason why the registration or the expert evaluation should take more or less time in 1994 compared to 2004. A 2001 French Senate report [17] showed similarities across years in the number of protocols evaluated by French RECs (total number of protocol, type of research,...). Moreover, this would make more sense to use the current procedure and workload (2004) rather than the old ones (1994). To our mind, this evaluation was closest to the reality, since a retrospective cohort was mandatory to evaluate characteristics and fate of biomedical protocols, whereas retrospective data collection was not possible to assess workload.

Moreover the French law is broader than EC 2001/20 European Directive [6], since it aims to protect human beings in all biomedical research (drugs, biomedical devices, physiology, vaccines,...), and therefore the protocols included in our study are not only on drugs evaluation.

The major difficulties encountered concerned definitions. Although the law clearly defines what are direct approvals and revisions, some RECs evocated direct approval with revision and postponed approval without revision. However this issue is not related to our study methods but to a difficulty in understanding and putting into effect the French law. Despite their training, research assistants had difficulties to retrieve information on study design and to categorise it. Some data were often missing in the protocols, such as the expected duration.

The four articles previously published collected information on only one or two ethics committees[1-4]. These studies did not describe the general situation at a country level and they provided information only when the investigator was not lost for follow-up. Only one study gave information on RECs' activity [1], namely some information on the number of new applications, correspondence, decisions, and the number of meetings needed to obtain approval were given.

One of our major results is that 46% of revisions concerned patient information and consent.

The above article showed that when studies were conditionally approved, deferred or rejected, most queries also concerned the patient information leaflet (85%). But only protocols with five or more centres were assessed; consequently these protocols were not representative of all biomedical research on human beings.

A French report [17] also gave such information, but the aim was more to point out problems of putting the law into effect in France.

Almost all protocols anticipate recruitment in terms of sample size rather than in terms of study duration. We think that the first point shows compliance with good clinical practices [18], and the latter expresses the need for a better training of prime investigators for clinical research study management and anticipation.

The workload theme in terms of numbers of protocols, number of revisions and number of amendments is very relevant since French RECs' members work for free during or after their workday, and at least administrative and financial support of the REC structure is obviously needed in order to guarantee independence in each country. The results of our study are fully supported by the recommendations of the Ad Hoc advisory group on the operation of NHS Research Ethics Committees [19], asking for independent opinion, managed operating system, time recognition and protection, as well as finances.

Across countries, RECs may not have the same structure (frequency of meetings, number of people involved in the committee), the same remit (only studies with intervention on human or also studies on physiology, for instance) and the same functioning since the number of protocols reported in the above article was very heterogeneous in the different countries. When looked at in terms of protocols approved per year and per committee, clear differences appeared: 110 protocols approved in the multicentre REC of London [1], 158 protocols in Barcelona [4], 180 protocols in Oxford [2], 94 protocols in Sydney [3] and on average 39 protocols in France.

It would be of great importance to launch a similar study in more recent years and in other settings at country level to see if things have changed, and if so in which way. Studies in different European countries will allow to collect information which would be very useful to ease harmonisation. It would also be interesting to launch a prospective recording of all tasks and time needed on a random sample of committees.

Conclusion

Up to now, one study described the activity of one REC specialised in multicentre trials and three other studies were carried out on the fate of protocols approved by one committee but nothing was known at a nation level.

Our study showed that 976 protocols were approved in one year by half of the French RECs and that median approval time ranges from 16 days (if no modification) to 48 days. Moreover, most protocols were carried out in France only (79%) for drugs evaluation (68%)

Revisions before approval relates first and foremost to patient information and consent modalities (46%). For a protocol evaluation (first evaluation, revisions and amendments), scientific and administrative workload varied on average from 21.3 hours up to 31.7 hours according to protocols' characteristics.

Competing interests

The author(s) declare that they have no competing interest.

Authors' contributions

ED coordinated the study, managed the data, performed the statistical analysis and drafted the manuscript.

VL participated in the design of the study, coordinated the study and managed the data.

FC designed, submitted and coordinated the study, interpreted data, and helped to draft the manuscript.

Acknowledgements

We thank Pr Yves Matillon and Pr Christian Hervé for their advice on drafting the protocol, Pr François Lemaire from the French National Clinical research Program (PHRC), Pr Terry Stacey, Central Office for Research Ethics Committees UK, for his advices on this paper, Marie-Pierre Rochette and Françoise Leclet, administrative staff of REC Lyon, Patricia Darnand and William Banga, members of the logistic staff; and the chairpersons and members of the participating French RECs: Alsace, Aulnay, Auvergne, Bordeaux, Boulogne, Brest, Dijon, Loire, Lyon A, Lyon B, Lyon C, Marseille1, Marseille2, Montpellier, Nice, Normandie, Paris-Bicetre, Paris-Creteil, Paris-Hotel dieu, Paris-Necker, Paris-Versailles, Poitou, Toulouse1, Toulouse2, Tours. Our views are not necessarily theirs.

Grant: French Ministry of Health (Programme Hospitalier de Recherche Clinique 1998 -065) and Hospices Civils de Lyon. French Ministry of Research and Higher Education and Claude Bernard University.

References

  1. Boyce M: Observational study of 353 applications to London multicentre research ethics committee 1997-2000.

    Bmj 2002, 325:1081. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  2. Easterbrook PJ, Matthews DR: Fate of research studies.

    J R Soc Med 1992, 85:71-76. PubMed Abstract OpenURL

  3. Stern JM, Simes RJ: Publication bias: evidence of delayed publication in a cohort study of clinical research projects.

    Bmj 1997, 315:640-645. PubMed Abstract | Publisher Full Text OpenURL

  4. Pich J, Carne X, Arnaiz JA, Gomez B, Trilla A, Rodes J: Role of a research ethics committee in follow-up and publication of results.

    Lancet 2003, 361:1015-1016. PubMed Abstract | Publisher Full Text OpenURL

  5. Loi 88-1138 du 20 Décembre 1988 dite Loi Huriet-Sérusclat. Loi relative à la protection des personnes qui se prêtent à des recherches biomédicales [http:/ / www.legifrance.gouv.fr/ WAspad/ UnDocument?base=LEGI&nod=SPEAJXXXXX XOOOAAXXXXXXAA] webcite

  6. Directive 2001/20/EC of the European Parliament and of the Council of 4 april 2001 on the approximation of the laws, regulations and administrative provisions of the Member States relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use , Official journal L121, 01/05/2001; 2001:34-44.

  7. Kaplan-Meier: Non parametric estimation from incomplete observations.

    J Am Stat Assoc 1958, 53:457-481. OpenURL

  8. Benzécri JP: L'analyse des données. 2, L'analyse des correspondances : introduction, théorie, applications diverses notamment à l'analyse des questionnaires, programmes de calcul. Paris, Dunod; 1981.

  9. Cattel RB: The scree test for the number of factors.

    Mult Behavioral Research 1966, 1:p245-276. OpenURL

  10. Benzécri JP: Sur le calcul des taux d'inertie dans l'analyse d'un questionnaire.

    Cahiers de l'Analyse des Données 1979, 3:p377-378. OpenURL

  11. Ward JH: Hierarchical grouping to optimize an objective function.

    J Am Stat Assoc 1963, 58:236-244. OpenURL

  12. Cubic Clustering Criterion SAS technical report A-108 edition. , SAS institute Inc., Cary, North Carolina; 1983:56.

  13. Zar JH: Biostatistical analysis. Fourth edition. , Pearson Higher Education; 1998.

  14. Pico F: Analyse à 6 ans de 168 projets financés dans le cadre du premier Programme Hospitalier de Recherche Clinique (PHRC 93). In Thèse de médecine. Université Paris XI, ; 1999. OpenURL

  15. Duffet JP: Rapport d'évaluation du Programme Hospitalier de Recherche Clinique 1994. , Ministère de l'emploi et de la solidarité- Ministère délégué à la santé; 2001.

  16. Duffet JP: Rapport d'évaluation du Programme Hospitalier de Recherche Clinique 1995. , Ministère de la santé, de la famille et des personnes handicapées; 2003.

  17. Huriet C: Rapport d'information fait au nom de la commission des affaires sociales sur le fonctionnement des comités consultatifs de protection des personnes dans la recherche biomédicale. , Sénat; 2001.

  18. Guideline for Good Clinical Practice [http://www.ich.org/MediaServer.jser?@_ID=482&@_MODE=GLB] webcite

    , International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use; 1996.

  19. Report of the Ad Hoc advisory group on the operation of NHS Research Ethics Committees [http://www.dh.gov.uk/assetRoot/04/11/24/17/04112417.pdf] webcite

    , Department of Health; 2005.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6939/6/9/prepub