So You Want to Start a Reading Center?

A researcher in an academic reading center offers a recipe.

RONALD P. DANIS, MD

A common question asked of us by individuals and institutions all over the world is whether we can help them to start a reading center? In return, this request prompts clarifying questions, such as: What kind of reading center? Why do you want to do this?

Interestingly, the answers are often not very clear. It is not unusual for aspiring investigators to have several goals simultaneously. There may be a desire to provide centralized assessment for intramural research projects. Local diabetic retinopathy screening might have the potential to become a significant community contribution. The clinic might have a new image management software system that could be used for centralized image interpretation for clinical care.

It is natural to assume that, because these activities have some elements in common — namely a centralized data repository and an image management platform — a reading center might be developed and expanded to serve clinical trials and larger research projects some day.

The bad news is that the unsexy methodology behind building a functional reading center consists of Policy and Procedure (P&P). For those with a passion for new imaging technologies, new instrumentation, clinical discovery, and patient care, P&P and its sibling, Quality Assurance (QA), are onerous and tedious. Just as writing a protocol for a clinical trial forces one to think through the details of design, operational issues, and analysis, developing a new reading center requires the discipline to consider the many organizational elements required.

The process may shed light on important new issues not previously considered and might mitigate the regulatory, legal, and other potential risks of clinical trial conduct. If done well, P&P is a way of preserving the institutional knowledge of the organization, and it can be important for training new staff as the center grows.

THE RECIPE FOR SUCCESS

Building a reading center is like a baking a cake: one must follow a recipe. P&P is the recipe that the reading center staff must follow. Any funding agency will demand this, and QA requires it. Without following a recipe, there may be a mess to clean up later.

There are many kinds of reading centers with overlapping organizational infrastructure elements, but each type of reading center has special requirements regarding work flow. In the field of clinical retinal imaging, for instance, there are dedicated telemedicine reading centers, epidemiological reading centers, and clinical trial reading centers.

Each of these centers is guided by different regulations, has different goals for image analysis and distinct reading protocols, and has different requirements for evaluation turnaround time and data reporting. Mixing these functions together in the same organization may not necessarily conserve resources and might add significant operational complexity.

Telemedicine has a specialized work flow because of the need for rapid turnaround and reporting of the evaluation to the clinic and/or the patient. The turnaround time and the method of clinical reporting may need to be individualized for each clinical location and patient population. Reading of images for telemedicine usually does not require the same level of detail as that used for clinical research, but the need for high-quality image capture and efficient image management is paramount. As with any clinical care institution, computing security and patient privacy issues are of supreme importance and must be addressed in the organizational charter and procedures.

READING CENTERS IN CLINICAL TRIALS

Clinical trial reading centers play a crucial role in translational research and help to bring new devices, drugs, and therapeutic modalities into clinical care. A reading center is a core laboratory providing unbiased, independent image analyses to assist a sponsor (which may be an individual investigator, the National Eye Institute or other granting organization, or a pharmaceutical company) to conduct a clinical trial.

Clinical research reading centers are all about the standardization of data capture and reading assessments (the terms grading, evaluation, and image analysis are more or less synonymous, but the FDA has come to prefer the term reading, which is used here) to maximize the likelihood of capturing meaningful changes in outcomes and classifications of disease severity. Reading centers require trained individuals (graders or readers) to interpret images and/or edit automated software analysis by rigid protocols with testable reproducibility.

The opportunity to introduce cutting-edge technology into clinical research is very rewarding, and many organizations have gotten their start because of a particular expertise or technological innovation. From an industry sponsor’s standpoint, the main outcomes must be well validated (by time and experience) and accepted by the regulatory agencies (eg, the FDA).

Many industry sponsors are reluctant to support exploratory endpoints in their studies because all of the data collected can and very well might be reviewed by the regulatory agencies. What if the analysis results of the new endpoint are equivocal or even go in an unfavorable direction? Such results may complicate the interpretation of the trial results.

There is a certain risk that industry takes in collecting more information than is necessary. Funding the research into the validation of new imaging endpoints is usually (although not always) funded through mechanisms other than as part of trials for product registration.

Reading centers supporting drug or device development must operate under the rules of regulatory authorities, which impose significantly more P&P documentation and compliance requirements, adding time and expense to the evaluation process.

Of course, not all clinical trials are for drug or device registration, and reading centers also support natural history studies and epidemiological studies. Many funding agencies (eg, the US National Institutes of Health) have standards for the conduct of large clinical trials that are not very different from those of regulatory agencies governing drug and device development. While patient privacy and computing security are always critical concerns, the level of documentation and software testing is generally less than that required for patient care or drug registration. These features make the process less costly.

In the US, all clinical research is performed under the auspices of the Office for Human Research Protections (www.hhs.gov/ohrp/) and the Office of Research Integrity (http://ori.dhhs.gov/), which have their own sets of regulations and the authority to investigate noncompliance regardless of the involvement of other regulatory authorities.

BAKING THE CAKE

Do you still want to start a reading center? Perhaps you want to establish a reading center in the early phase, with only a few small projects and not for submission to the FDA or for large long-term studies. Investigators might want to evaluate the images from in-house projects for cost savings. Maybe there is little funding to support the effort initially, mostly the passion or the will to make it happen.

Sometimes, to get started, an investigator will attempt to piggyback on his/her department’s computing infrastructure. Perhaps there is a study coordinator that might join the effort part-time, and a little bit of network support and programming help is available.

The investigator knows what a Standard Operating Procedure (SOP) is and has a good idea of what the institutional review board will require to approve the projects. Once the organization has a track record of good performance, it may attract additional projects and grow into something self-sustainable one day. How do we bake this cake?

The documentation required for a reading center is to some extent commensurate with the size of the organization and size of the projects. A tiny organization does not need a slate of dozens of SOPs that cover, for instance, the minutiae of how to install antivirus software.

However, there are some essential ingredients in the recipe. For instance, there can be no compromise on patient privacy. An organization collecting clinical data must find a way to mask and protect patient identifiers from the standpoint of clinical and research personnel, computer security, and hard copy paper or reports.

Moreover, the steps for image handling and analysis must be written down in detail. Any personnel doing the actual reading must be trained in the reading procedure. This process requires a collection of example images to review for training purposes. Ideally, there should be a set of data that all of the readers analyze to demonstrate consistency.

It is a good plan to have a sign-off step to verify that the reader has been trained and has passed the test. The method for the recording of the analysis may vary from a custom database to spreadsheets to paper forms.

The process for quality control of the reading by testing the reproducibility of the evaluations should be part of the essential formula. Data handling will vary according to the needs of the project, but it should be clearly specified to all parties involved. Finally, centers must always ensure that the records are backed up and safe.

CENTER SIZE AND AUDITING

Large centers that read for industry and large federally funded multicenter studies must be extremely disciplined. Running a tight ship in a large reading center means directing staff to perform the functions of administration, project management, software validation, QA, and document writing. SOPs must be comprehensive and detailed and must reflect what staff are actually doing in practice. Industry sponsors frequently will audit reading centers prior to the awarding of a contract, as well as during the contracting period, to ensure that there are no corners cut in P&P and compliance.

A team of industry sponsor auditors may take days to go through the documentation, discussing the various policies, and procedures to determine whether they accurately reflect the actual conduct of work, and they will tour the workspace.

Ultimately, the sponsors are responsible for the quality and compliance of the data brought to the FDA, and they have an obligation to ensure that subcontractors follow all the rules. The FDA may arrive with little or no advance notification of its audits; thus, organizations engaged in clinical research for regulatory submission must always be on their toes.

Good auditors with significant experience may have great suggestions for improvement in efficiency while maintaining compliance (eg, the user testing for software validation can be done more quickly and easily using a different method). They may act like partners with the reading center to help everything to run smoothly with the sponsor’s study.

Conversely, sometimes auditors have expensive and time consuming requirements that nevertheless must be addressed (ie, waterless fire suppression is necessary in the computer server room, broken locks on cabinets must be replaced, several SOPs might be due for revision, etc.).

Legal, financial, and business efforts grow with the size of the projects and the organization. The expense involved in maintaining this infrastructure certainly adds to the cost of reading images or reports. Epidemiological studies and small projects do not require as much regulatory “baggage” and therefore are less costly for reading.

STANDARD IMAGING/TESTING PROTOCOL FOR CLINICAL SITES

All core clinical laboratories have similar work flows, whether for medical imaging or blood chemistry. SOPs describe the steps of the work flow and include rules for managing exceptions and disruptions to normal processes. Depending on the project and reading center, some of these steps may be omitted, and there may be additional important steps unique to a given project, but the following must at least be considered.

• Procedures for clinical sites to anonymize, label, and submit images

• Certification of technicians and/or instrumentation at the sites

• Acknowledgement to the site of the receipt of images by the reading center

• Quality assurance to ensure that identifiers are removed, study codes are correct, dates and visits are congruent, and there is completeness of the imaging submission, as well as rules for how to manage deviations and communications with clinical study sites

• A process for getting the images ready for reading (ie, uploading, file conversion, display requirements, etc.)

• Procedures for reading the images and procedures for the quality control of each project

• Reading work flow (ie, double reading with adjudication, longitudinal evaluation, reading with reference to other image types)

• Data collection forms (custom-built databases or others)

• Data storage, backup, protection, and export

Leveraging computing technology to make these processes efficient and to monitor work flow is highly desirable. However, image management can be challenging if there are multiple types of instruments involved in data collection. Small projects may rely on one make and model of imaging instrument to facilitate the storage, display, and analysis environment for reading.

Reading centers of projects that accept images from multiple types of optical coherence tomography instrumentation have the challenge of either analyzing the macular volume scans in the commercially available software from each make of OCT machine separately or developing a custom software solution.

Unlike medical imaging for radiology, in which compliance with international standards for file formats and metadata is commonplace among different imaging machines and technology, the field of ophthalmic imaging has been slow to adopt standard imaging formats, although international standards have been developed and published, such as the DICOM (Digital Imaging Communication and Management) standards, published at www.nema.org.

This lack of standardization leads to fragmented data collection and analysis, resulting in difficulty in tracking data through the reading center. Some imaging contract research organizations have developed innovative solutions to manage these diverse proprietary image formats, but the best solution for research in the field (as well as for clinical care) would be for manufacturers to comply with the published standards.

CONCLUSION

Do you still want to start a reading center? This article has touched upon the importance of focusing the goals of the reading center, the central roles of work flow, documentation, and compliance, and some computing issues to consider.

It is an enormous benefit to an investigator or institution’s research program to play a prominent role in multicenter trials. To have access to a large number of high-quality images and accompanying standardized analysis data can lead to very rewarding research beyond the scope of the original study.

Reading centers tend to be centers of innovation because they are involved with new technology and are interested in the validation of new structural endpoints for clinical care and research. These centers are fundamentally about collaboration on projects with other investigators at clinical sites and with clinical trial leadership. Investigators affiliated with reading centers may meet and work with cutting-edge scientists. It can be a fascinating enterprise. Having the chance to view cool retina images is the icing on the cake. RP