CTTI’s Metrics Framework defines what is valuable to measure across the clinical trials enterprise to assess both incremental and transformational change. Our aims for the Metrics Framework are to offer benchmarks for assessing collective progress as the enterprise moves toward 2030. The Metrics Framework consists of 57 individual metrics that all link to key concepts in CTTI’s Transforming Trials 2030 vision.
Read how our vision statement served as the foundation for the Metrics Framework.
Methodologies
CTTI’s Metrics Framework is produced from executive leadership perspectives at CTTI and across the clinical trials enterprise. The idea for these metrics emerged shortly after CTTI introduced the Transforming Trials 2030 vision as a critical tool for measuring that vision. Metrics were refined through a series of concept elicitation interviews with trial leaders and expert surveys to assess the value and accuracy of all individual metrics. CTTI’s Metrics Framework is supported with continual input from the Measuring Trials Transformation Project Team.
CTTI is exploring several methodologies to obtain clinical trials data and analyze individual metrics. We are amplifying our use of CTTI’s Aggregate Analysis of ClinicalTrials.gov (AACT) to investigate select metrics from the ClinicalTrials.gov registry. Our approaches include cross-sectional analysis of trials data and novel approaches for analyzing original trial documents reported to ClinicalTrials.gov (Protocols, Informed Consent Forms, Study Analysis Plans, Study Results, and Protocol Amendments). Our exploration of trial documents includes using lexicon and semantic based approaches and use of large language models (LLM) to uncover contextual metrics indications in trial documents.
Preliminary results will be published here as they become available.
Domains
Patient-Centered and Easily Accessible
- Population Awareness of Clinical Trials
- Accessibility of Clinical Trials
- Patient Input into Trials
- Representativeness of Trial Populations
- Participant Experience in Clinical Trials
Fully Integrated into health processes
- Healthcare system readiness for trials
- Learning healthcare system
- Consent in care setting
- Maximum use of EHR
Designed with a Quality Approach
- Meaning
- Fitness for purpose
- Stakeholder Engagement
- Innovation
- Infrastructure
Maximally Leverage All Available Data
- Interoperability (non-trial data)
- Privacy and Security
Improve Population Health
- Efficiency
- Actionable Learning
- Comprehensiveness
Individual Metrics
Term Search
Domains:
Accessibility of Clinical Trials
Near Term
Sponsors attempt to make it easier for people to get into trials.
Metric: % of trials that assess and attempt to mitigate common enrollment burdens prior to trial initiation beyond what’s routinely done today
Mid Term
Trial participation is not overly burdensome.
Metric: % of trials that successfully mitigate common burdens (e.g., provide full financial reimbursement, have appointment time and location flexibility) beyond what’s routinely done today
Long Term
Clinical trials are available to all relevant individuals who want to participate.
Metric: Within a given disease population and a given country, the % of patients who want to participate in a trial and are able to
Actionable learning
Near Term
Information related to trial conduct and results is easily accessible.
Metric: % of trials with results published on ClinicalTrials.gov within a year of trial’s end
Mid Term
Clinical trial evidence is leading to changes in recommended care.
Metric: % of clinical trials that are cited for updated guidelines for clinical care
Long Term
Clinical trial evidence is leading to practice change that improves public health.
Metric: Increase in overall survival rates and in relevant sub-populations (e.g., historically underrepresented populations, disease areas)
Comprehensiveness
Near Term
There are a greater number of trials that are adequately powered.
Metric: Number of adequately powered clinical trials per year
Mid Term
Clinical trials are focused on population need.
Metric: Number and % of trials focused on diseases that are most impactful for population health (e.g., areas with high morbidity/mortality)
Long Term
The clinical trials enterprise addresses the health needs of all individuals.
Metric: % of recommendations in clinical guidance documents across all disease areas that are supported by evidence from high quality clinical trials
Consent in care setting
Near Term
Trials enable consent in routine care settings.
Metric: % of trials in which consent into research is automated or embedded into clinical workflows
Mid Term
Informed consent for research participation is integrated into routine care.
Metric: % of clinical trial participants consenting into research via routine practice
Long Term
Informed consent for research participation is high quality and enmeshed in routine practice settings.
Metric: % of clinical trials with high quality consent processes and procedures enmeshed in routine practice
Efficiency
Near Term
The clinical trials enterprise generates sound evidence for new medical products.
Metric: Number of new FDA approvals (per year)
Mid Term
The clinical trials enterprise is getting better at generating evidence for new medical products.
Metric: Average length (in months) of Phase I-III clinical trials per new intervention
Long Term
Providers are rapidly deploying new products in the appropriate populations.
Metric: % of providers who are deploying new products within one year of approval
Fitness for purpose
Near Term
Trials are successful at answering their primary research question.
Metric: % of all trials that conclusively answer (positive or negative) their primary research question at the level of statistical significance defined in protocol
Mid Term
Successful trials are only as complex as they need to be to answer their primary research question.
Metric: % of trial protocols that clearly state the data elements/design aspects that are critical for answering their primary research question
Long Term
The clinical trials enterprise is getting better at addressing questions more efficiently.
Metric: Average cost per question answered in a clinical trial
Healthcare system readiness for trials
Near Term
Healthcare delivery sites are participating in trials.
Metric: # of major healthcare delivery systems that are involved in running trials as a recognized, measured part of their care delivery
Mid Term
Clinical trials are seamlessly integrated into routine care settings.
Metric: % of clinical trials that have key trial activities embedded into routine care or point of care in some way (e.g., informed consent, randomization, etc.)
Long Term
The CTE fully leverages routine care site visits or point of care interactions for trial activities.
Metric: % of clinical trial data derived from routine care or point of care interactions
Infrastructure
Near Term
Trials re-use site infrastructure from previous research.
Metric: % of trials that leverage common site infrastructure (e.g., training across sites and sponsors, site qualifications)
Mid Term
Trials re-use design infrastructure from previous research.
Metric: % of trials that leverage common trial design infrastructure, such as: master protocol, single IRB, non-proprietary network, database structure, data monitoring committee
Long Term
Individual trials contribute openly to the broader clinical trials infrastructure.
Metric: % of trials from which non proprietary infrastructure are made available to other trials
Innovation
Near Term
Trials deploy new methods.
Metric: % of trials employing innovative design or analytic techniques
Mid Term
Trial sponsors exhibit organization-level commitment to clinical trial design innovation.
Metric: % of sponsors investing in innovation (e.g., via a stated trial innovations role or function, partnerships to test new methods, outsourcing to innovative vendors, etc.)
Long Term
The clinical trial enterprise is fostering innovation in trial design and conduct, even at the risk of failure.
Metric: Number of unique innovations being tested within a given year across the clinical trials enterprise
Interoperability (non-trial data)
Near Term
Data formats, standards, and technology are making health and consumer data usable in trials.
Metric: % of trials that use non-trial data (e.g., eligibility, patient identification)
Mid Term
Data formats, standards, and technology are making health and consumer data easy to use in trials.
Metric: % of trials that electronically pull any data from non-trial data sources in an automated way
Long Term
All trials that would benefit seamlessly pull in external data electronically.
Metric: Average % of clinical trial data elements that are pulled directly from non-trial data sources
Learning healthcare system
Near Term
Healthcare delivery systems have a plan for integration of new knowledge from clinical trials.
Metric: # of major healthcare delivery systems with any SOPs for adding new information from clinical trials into care plans
Mid Term
Healthcare delivery systems comprehensively integrate new knowledge from clinical trials.
Metric: # of major healthcare delivery systems with internal knowledge sharing mechanisms for adding new information form clinical trials into care plans (e.g., Grand Rounds, advisory boards, available across all departments or therapeutic areas)
Long Term
New information derived from clinical trials is rapidly deployed in patient care.
Metric: % of relevant providers who are acting on new clinical trials evidence in their field within one year of integration into SOPs
Maximum use of EHR
Near Term
Clinical trials use basic EHR data.
Metric: % of trials that use EHR data (e.g., eligibility, patient identification)
Mid Term
Clinical trial data systems can make direct calls to EHRs to pull in data.
Metric: % of trials that directly, electronically pull any data from EHRs for patient identification and eligibility
Long Term
Clinical trial data are collected via the electronic health record.
Metric: Average % of clinical trial data elements that are pulled directly from EHRs
Meaning
Near Term
All trials have a clear purpose.
Metric: % of trial protocols whose rationale clearly articulates the contribution of the research
Mid Term
Trials address important questions.
Metric: % of trials whose primary research question was informed by key stakeholders
Long Term
Trials contribute valuable evidence for patient care, research, or both.
Metric: % of trials whose findings have a clinically meaningful impact on patient care and/or provide a meaningful building block for research
Participant Experience in Clinical Trials
Near Term
Sponsors make some effort to assess participant experience in trials.
Metric: % of trials that have actual participants provide feedback on trial experience through any instrument (e.g., SPFQ) for this purpose during or after the trial
Mid Term
Trial participants have positive experiences.
Metric: % of trials that have actual participants provide net positive feedback (mean and range) on trial experience through any instrument for this purpose during or after the trial
Long Term
Clinical trial experiences are good enough to increase demand for them.
Metric: Net promoter score (or similar) across a representative set of all trial participants at +2 years from trial conclusion
Patient Input into Trials
Near Term
Sponsors obtain some patient feedback on trial design.
Metric: % of trials that have patients with relevant conditions provide feedback on trial design through any process specific for this purpose (general or disease-specific) before finalizing the protocol
Mid Term
Sponsors include patients in a substantive way in trial design.
Metric: % of trials where design insights from patients were implemented (source: PFMD) OR patients have a meaningful role in trial governance
Long Term
Patients are full partners in clinical trials.
Metric: % of research programs where patients are partners in the design, conduct and reporting of trials
Population Awareness of Clinical Trials
Near Term
The general public seeks out clinical trials.
Metric: % of population using any set of search tools to find clinical trials
Mid Term
Sponsors are making an effort to inform the broadest eligible population.
Metric: % of trials that have a mechanism to reach and inform potential participants beyond site-based enrollment (e.g., direct letters, deep collaboration with patient groups, etc.)
Long Term
Everyone who could participate in a trial knows about it.
Metric: % of population informed about trial eligibility as a part of routine care and community outreach
Privacy and Security
Mid Term
Common standards exist to address privacy and security in an easily implementable and transparent manner.
Metric: Availability of consensus standards governing privacy and security of multiple data types in clinical trials
Long Term
Common standards addressing privacy and security are creating more opportunities for use of data.
Metric: % of trialists that report that the common privacy and security standards enable them to use data easily and transparently
Representativeness of Trial Populations
Near Term
Sponsors make an effort to recruit representative populations into trials.
Metric: % of trials that have mechanisms to track enrollment of historically underrepresented groups during the trial recruitment phase in a way that allows for remediation
Mid Term
Sponsors are achieving representativeness in trials.
Metric: % of research programs that have total trial populations that mirror disease population, within each country (e.g., assessed via Participation to Prevalence Ratio)
Long Term
Clinical trials are a representative enterprise.
Metric: The extent to which participants across all trials in a given country reflect that country’s overall population (e.g., assessed via Participation to Prevalence Ratio)
Stakeholder Engagement
Near Term
Sponsors are aware of the stakeholders whose input could strengthen trial design.
Metric: % of trial protocols that identify key stakeholders for input on design
Mid Term
Sponsors are including stakeholders in a substantive way in trial design.
Metric: % of trials in which design insights from key stakeholders were implemented or key stakeholders have a meaningful role in trial governance
Long Term
Relevant stakeholders are adequately engaged in trial design.
Metric: % of investigators and coordinators, patients and participants, clinician users and their health systems, regulators, and payers that report that they are appropriately engaged in design of relevant trials