Deborah R. Barnbaum
Kent State University, Kent, OH
An Institutional Review Board (IRB) must review and approve all research projects using human beings before researchers are permitted to begin their projects. In their recent article on the ethics of clinical trials, Miller and Brody echoed Emanuel, Wendler, and Grady in stating that "independent review" by a committee such as an IRB is a necessary ethical requirement for all clinical research.1, 2 Despite the widespread endorsement of IRB approval, the Office for Human Research Protection’s (OHRP) new Quality Improvement Program cites "the failure to 1) obtain prospective IRB approval" as the first of five problems the new Program aims to address.3 In cases in which researchers fail to obtain prospective IRB approval, one of two outcomes may ensue.
First, the researchers may never obtain IRB approval. Not surprisingly, there are few data on the frequency of researchers’ failure to obtain IRB review. Matot, Pizov, and Sprung’s one-year retrospective study of critical care research in seven leading medical journals found that 24% of published studies do not contain any evidence of IRB review (including approval of informed consent procedures).4 While the publication of these studies did not cite evidence of IRB review, the studies may have been conducted with an IRB’s approval, with the published results failing to cite this fact.
Merz cites a second possible outcome: "Either the authors failed to document in the manuscript that the study was submitted and approved by the appropriate IRB(s), or (the researchers) failed to submit the research protocol to their institutional IRB(s)."5 Perhaps IRB approval was not sought prior to data collection. A failure to obtain prospective IRB review may result in researchers petitioning for ex post facto IRB approval — IRB approval after the data have been collected. Is there a means by which IRBs can best consider projects that did not receive prospective approval, but for which researchers are requesting ex post facto approval?
When the researcher has failed to properly protect the interests of the participants, or the risks of performing the research outweighed the benefits, the answer to this question is clear-cut: if the project would not have passed muster with the IRB before the data were collected, it clearly should not pass muster with the IRB after the data are collected.6 But imagine the best-case scenario, in which every protection that the IRB would have hoped for was in place. What should be done in this circumstance? This situation is more challenging, because the participants were protected, and the benefits did outweigh the risks. The spirit of the IRB’s mandate was upheld.
A qualification — failure to obtain prospective IRB approval, and the question as to whether the IRB should then grant ex post facto IRB approval, is a question that is less likely to emerge in high-risk clinical studies, in which life-threatening adverse events are possible. The investigator who petitioned for ex post facto IRB approval for a phase I clinical study would soon be out of a job. Instead, ex post facto IRB review is more likely to become an issue in cases in which the investigator either was not aware that IRB purview extended to the research, or the investigator was not at all aware of IRB regulations. Social science research that the investigator didn’t realize was in fact "human research," or education research in which the investigator was unaware of IRB requirements, or comparatively low-risk research undertaken by novice PIs (including graduate students, or even undergraduates) are the types of research that are most likely to be considered for ex post facto review.7
Is There a Regulatory Context for Ex Post Facto Review?
Many granting agencies require that IRB review be documented prior to awarding of funds.8 §46.101(a)2 states that even the research projects that are "neither conducted nor supported by a Federal Department or Agency" but are nonetheless subject to the Federal Regulations "must be reviewed and approved" by an IRB. "Must be reviewed and approved" may imply that the review already should have taken place prior to data collection. However, the common rule does not explicitly require that this review take place prior to data collection. "IRB approval," as defined at §46.102(g), "means the determination of the IRB that the research has been reviewed and may be conducted at an institution within the constraints set forth by the IRB and by other institutional and Federal requirements." While this definition is not explicit as to when IRB approval must be obtained, the spirit of the common rule requires that IRB approval take place prior to data collection. Since the research "may be conducted" only "within the constraints set forth by the IRB," the assumption is that the IRB has reviewed the research protocol prior to commencement of the research. However, as OHRP’s Quality Improvement Program indicates, failure to obtain these "prospective" approvals continues to be a problem, which raises a question: isn’t an ex post facto approval better than no approval at all? Why couldn’t an IRB grant ex post facto approval, especially given a best-case scenario of the responsible researcher/s, adequate subject protections, and a favorable risk/benefit ratio?9
There are two practical hurdles, and one theoretical mistake, in granting ex post facto approval. The first practical hurdle is IRBs’ reluctance to set a bad precedent in granting ex post facto approval. The second of the practical hurdles is that IRBs have no way of verifying the integrity of the consent process for projects that are granted ex post facto approval. While both of these considerations present difficulties to IRBs, they do not, in themselves, constitute unsurpassable obstacles for an IRB. More significantly, there is a theoretical mistake in applying the assessment of risk/benefit ratios when conducting an ex post facto review. This theoretical mistake is the most significant barrier to granting approval after the research has been completed, because it demonstrates that one of the federally mandated standards for assessing research protocols is inappropriate for ex post facto reviews. Since such reviews cannot be accommodated using current IRB standards, we must either abandon one of the cornerstones of IRB review, or we deny the permissiblity of ex post facto review. I advocate the latter of these two options.
Practical Hurdle #1: Setting a Bad Precedent
A first consideration is concern about the negative precedent set when ex post facto review is undertaken. When word leaks out that such a review was granted to one researcher, the argument goes, other researchers will soon follow suit. Soon enough, the IRB will be inundated with requests for ex post facto reviews. Almost certainly, some of these requests will not mirror the best-case scenarios in which subjects were best protected, and for which the IRB would not have requested any changes. Furthermore, researchers may fail to submit protocols to the IRB prior to data collection, anticipating that ex post facto review is a viable option.
The concern here is founded on a slippery-slope argument: once the door is opened at a given institution to ex post facto review, it will be impossible to close the door. Slippery slope arguments, however, are fallacious: there is no guarantee that one ex post facto review will open the institutional floodgates. Additionally, the IRB should be savvy enough to distinguish the best-case scenario research project from one in which participants were not offered sufficient protections, or in which risks were not adequately minimized. The problem posed by a slippery slope is that we don’t know where to "draw the line." But IRBs are given clear direction as to where the line should be drawn, and should readily be able to distinguish between research projects that adequately protect participants and those that do not. The researcher who knowingly does not submit a protocol on time, counting on an ex post facto approval, is taking a chance in assuming that the protocol in question is in fact one of the best-case scenario protocols. This is a chance that many researchers may not be willing to take.
The rule that requires researchers to obtain IRB approval prior to data collection is in place because we assume that following this rule will result in the best consequences for all parties concerned. This is simple rule utilitarian reasoning But locating rules that result in the best consequences for all parties concerned is only a first step: For every rule, there may be exceptions, which may further the best consequences. The rule that demands that researchers obtain approval prior to data collection is a rule that will result in the best consequences. Granting approval to best-case scenario research projects may be a legitimate exception to that rule, because granting approval in this one case may in fact result in the best outcome for all parties concerned. While the IRB may be reluctant to set a bad precedent, the IRB should also be aware that the rules are in place for a reason, and if those reasons are best satisfied by breaking the rules, then breaking the rules is what is called for. This is not to say that there are not pragmatic reasons for an IRB to resist ex post facto approval. But these institutional concerns, such as a perception that the IRB is unfair in granting ex post facto approval to one researcher, and not to another, are not ultimately about participant protection.
Practical Hurdle #2: Were the Participants Adequately Informed?
A second concern is that many assurances about participant protection must be taken as a matter of faith when an ex post facto review is performed by the IRB. Receipt of signed consent forms after data have been collected may provide hard evidence that some steps have been taken to satisfy legal requirements for informed consent. However, signed consent documents are merely one step in the consenting process. Not surprisingly, more vulnerable populations — populations for whom greater assurances are needed to demonstrate competency, voluntariness, and informedness as a condition of their participation — are the very populations for whom assurances that they were well-informed are most difficult to determine after the fact. Cognitively proficient, healthy, literate, non-institutionalized, autonomous adults can be expected to get more out of a signed consent form than children, persons with cognitive disabilities that may affect decision making, persons with illnesses that may affect the voluntary nature of their participation, incarcerated individuals, etc. Even though the discussion of ex post facto IRB review is best restricted to social science, education, and similar studies, participants in such research are not necessarily autonomous individuals. Some participants in this type of research may require special protections, including more thorough documentation of informed consent guarantees. The greater the need for continued documentation of informed consent, the less likely that evidence of this need will be fulfilled in an ex post facto review.
Despite these difficulties, best-case scenario research projects may not encounter these obstacles. Perhaps such a project would include only the most autonomous participants, for whom a signed consent form is strong, albeit not sufficient, evidence that informed consent had been obtained. While questions about informed consent may persist in some studies, for other studies the signed consent forms, along with assurances from responsible and experienced researchers, may be all that the IRB would have required. In such a case, the IRB would not see a failure in the consenting process, even in an ex post facto review. In this case, there may be no real hurdle in offering IRB approval, on the basis of inadequate subject protection. However, one significant concern remains when IRBs undertake ex post facto reviews.
A Theoretical Hurdle: Assessing the Risk/Benefit Ratio
A final problem remains for IRBs that are asked to perform ex post facto review. The problem is a theoretical one, which calls into question the conceptual framework in which IRBs evaluate research protocols. In particular, the reviews in question present a challenge to the evaluation of the risk/benefit ratio, an evaluation that all IRBs are required to undertake.
IRBs are charged to make two assessments regarding the risks and benefits inherent in any research project. First, IRBs must determine that the risks of the research are minimized. If there are alternative means by which the same data could be collected, the researchers must take care to choose the method that will minimize risks to the participants. This is not to say that the research project must be risk-free. Rather, care must be taken to expose participants to the lowest level of risk that is compatible with the data collection. Second, IRBs must determine that the risks inherent in the protocol are outweighed by the benefits of performing the research. It is possible that the risks (usually borne by the participants) accrue to a different population than those who experience the benefits (sometimes the participants, but also the researchers, the scientific community as a whole, cohorts or future cohorts of the participants). However, the fact that the risks accrue to one population while the benefits accrue to another does not alter the fact that the risks must be outweighed by the benefits. Any project in which the risks outweigh the benefits should be deemed by the IRB to be unethical. In summary, protocols should have 1) the fewest risks possible, compatible with collection of the data, and 2) a favorable risk/ benefit ratio.
The challenge of