More than a decade ago, I was alarmed by the high number of ethics scandals involving pharmaceutical companies documented in the media and the scholarly literature and hashed out in court cases and settlements. It was hard to know what to make of these reports. Were these ethics failures those of a few rogue companies or employees? Had the underlying issues been resolved? Or, did they constitute genuine widespread problems and risks for patients? I found myself wondering, are drug companies trustworthy, patient-centered, and socially responsible?
The general public certainly did not hold pharmaceutical companies in high regard, a condition that has only deteriorated since. The industry is perceived as the least socially responsible sector in health care. In some polls, drug companies rank just ahead of tobacco companies and behind Wall Street in perceived honesty, ethics and trustworthiness. Nine in 10 Americans think drug companies put profits before people; only 20 years ago, these firms were among the most esteemed and respected. (Frankly, large institutions in general are facing growing distrust, from regulators like the FDA to the media and political systems. Notwithstanding, the pharmaceutical sector is disproportionately distrusted.)
Determining whether drug companies are trustworthy and fixing any ethics problems is an urgent public health matter. Access to medicines that are safe and effective is an important determinant of health. When you think about new medicines, you have to think about the pharma industry, which sponsors most of the clinical research for new medicines in the U.S. We cannot afford for such a major player in health care to be rife with problems or distrusted. When patients don’t trust drug companies, they may be less likely to participate in clinical trials, take prescribed medicines and get vaccinated. For the sake of people’s health, we have to get this right.
This was the start of a journey that ultimately led to the Good Pharma Scorecard (GPS), a ranking of pharmaceutical companies on how well they implement good ethical practices. The GPS helps bridge asymmetries of information about the ethical performance of the industry. It also recognizes best practices and catalyzes reform where needed.
Since my colleagues and I at Bioethics International, Yale School of Medicine and Stanford University put out the first GPS in 2015, it has had a measurable positive effect on drug companies’ practices. Across all metrics, the industry has improved year after year, which suggests a steady improvement in ethical practices. Moreover, many large companies immediately improve practices, every year, upon receiving our recommendations on how to tighten their procedures, bringing them up to higher ethical standards. Many also use their GPS results in their annual reports, which itself creates an incentive to improve year over year. Health-care investors, especially those concerned with social responsibility, have begun to use the GPS in their Environmental-Social-Governance analyses. It has been hopeful to see many multi-national large companies changing nimbly and quickly, and tracking their progress year after year.
To date, BEI has assessed thousands of clinical trials that enrolled more than a million participants around the world. We used this data to publish GPS rankings in 2015 and 2017. In this issue we are presenting the results for 2019, the third round of data we gathered. But first, I want to explain how we arrived at the Scorecard as a vehicle for encouraging pharma to follow good ethical practices.
Where to Start?
The journey began by first mapping the chief ethics concerns about drug companies. Some companies were accused of hiding unfavorable data to make drugs appear safer and more efficacious. There were concerns, backed up by research, that doctors, formulary decision-makers, the Food and Drug Administration, patient groups, and medical educators were in the pocket of industry. Cases of companies bribing doctors with envelopes of cash were making the news. Experts accused drug companies of wasting money on marketing rather than researching and developing new drugs. Drugs were becoming increasingly unaffordable.
The next question was, should we tackle drug prices or marketing practices? I wanted to break off a piece of the problem that was not only important for public health but also ripe for reform.
We decided to tackle two issues: how well drug companies make the results of clinical trials available to the public and the medical profession (transparency), and how well they share the underlying data gathered during their trials (data sharing). In other words, whether companies are honest, truthful and transparent about the safety and efficacy of new medicines and vaccines. My colleagues and I reviewed medicines and vaccines approved by the FDA in 2012 for HIV, tuberculosis, breast cancer, rare diseases, pediatric meningitis and many other conditions.
We were not reassured by what we found. Only one in five drugs had public results for all the clinical trials supporting its FDA approval, when assessed a year after approval. For half of reviewed drugs on the market, the results of at least one phase 2 or 3 trial were unavailable to the public. (Phase 1 trials test a drug’s safety, and phase 2 and 3 trials test its efficacy.) Overall, only a median of 65 percent of clinical trial results supporting FDA approval of each reviewed drug were publicly available a year after approval.
Improving transparency, we realized, was critical for science, patient care and public trust. Doctors need valid and objective clinical trial information to prescribe the right drug for the right patient at the right time. Patients, as partners in their own care, need it, too, to make informed treatment decisions. If trial results are hidden, the whole ethics of a trial come into question, and so does the quality of our medical evidence. (To be fair, this is a problem in research generally, not just industry-funded research.)
Why We Chose a Scorecard
Once we had decided to address transparency and data sharing, we needed to determine the best way to help improve industry practices. Passing new laws takes a long time; plus, we had already seen that laws were not being adequately monitored and enforced. Instead, we looked for a tool that could help us catalyze the change we felt was needed across the industry. We wanted to give drug companies an incentive to adopt better practices and help us track and communicate their progress over time.
I noticed that almost every industry has some type of accreditation, certification or rating, and that these programs are associated with improved quality and firm performance. We learned that health care was one of the first to adopt such programs. In 1910, Ernest Codman, a surgeon at Massachusetts General Hospital and a faculty member of Harvard Medical School, and his colleagues developed a set of hospital standards related to patient safety. When he conducted on-site inspections of 692 hospitals, he found that only 13 percent met the standards. This finding was so embarrassing to the health-care establishment that the list of failing hospitals was burned to keep the press from finding out.
Within five years, however, most hospitals were meeting or exceeding the standards—a prime illustration of the old saw, “what gets measured, gets done.” The potential to earn society’s trust, the threat to a company’s reputation, the knowledge that other organizations are performing better, opportunities to learn and refine beliefs about previously unobserved qualities—all these factors make rankings an effective way to prompt companies to change. Restaurant rankings, for instance, led to significant improvements in cleanliness and, in turn, a reduction in hospitalizations for foodborne illnesses. Consumers are also more likely to patronize an institution with the best social responsibility reputation, when quality, service and price are uniform.
As I mulled the merits of ratings and rankings, I started noticing them everywhere. My mother wouldn’t buy a loaf of bread unless it had the American Heart Association’s heart-check trademark on the packaging. My father was trying to obtain ISO certification for his company. Was Mom’s AHA-certified bread really healthier? Would Dad’s company truly improve its practices for the sake of certification? I reviewed 75 rating programs (including car safety ratings and fair trade food labels) to understand their promises and pitfalls. I also reviewed the scholarly literature for evidence of efficacy of these programs, or lack thereof, and best practices. There seemed to be enough evidence to support building a ranking system to measure and improve the ethical performance of pharmaceutical companies. The Susan G. Komen Foundation offered seed funding and Harvard University a fellowship grant to run a pilot test.
I enlisted a multi-stakeholder team of advisors through Bioethics International, a nonprofit I founded in 2005 to help improve ethics and make health care more patient-centric. We developed standards and a scoring system for evaluating each pharmaceutical company’s ethics, based on a rigorous process that included reviewing recommendations from expert bodies and multi-stakeholder consultation.
Because we wanted our ranking to have an immediate impact, we built in a 30-day amendment window to give companies an opportunity to raise their scores. By the end of the 30 days, if a company could demonstrate that it had improved its policies and practices, BEI would recalculate its score based on this new information and publish a pre and post score. A company would be included in a particular year’s ranking only if it had a novel drug approved that year. After successfully piloting the GPS with the Edmond J. Safra Center for Ethics at Harvard University, Arnold Ventures generously awarded us a large grant to scale the GPS from a pilot to an industry-wide annual global ranking system.
The GPS transparency rankings look at three tiers of standards. The baseline is whether a company is following current laws in reporting. (Being legally compliant is no small thing: too many companies fall short of this standard.) The middle tier asks if a company is disclosing the protocols and results of all the clinical trials conducted with patients to gain FDA approval of their drugs. The third tier asks if the company makes all data about the patients who participate in a trial available to researchers who request it (while observing patient privacy and consent).
In the latest GPS, only two companies scored 100 percent on all three measures: Roche and Novo Nordisk. Both companies disclosed the results for all of the trials conducted in patients that support FDA approval of their drugs. Further down the list, Allergan and Valeant Pharmaceuticals jumped up in the rankings compared to their performance in the 2017 rankings. (Valeant has been renamed Bausch Health.) Gilead Sciences and Johnson & Johnson (J&J) fell in the rankings from 2017. A company can fall in the rankings for many reasons. Sometimes its product will rely on older studies that date to a time when standards of transparency were less rigorous. Sometimes it will acquire a smaller company that hasn’t been adequately reporting trial results. Anecdotally, executives have shared privately that, as a result of the GPS evaluations, their companies now stipulate as a condition of an acquisition or merger that all transparency problems are fixed ahead of time.
Drawing on recommendations from several expert bodies, including the Institute of Medicine (now known as the National Academy of Medicine), and consultations with patients, industry, academics, regulators and others, the data-sharing rankings evaluate whether companies provide others access to analysis-ready participant-level data sets and clinical study reports (CSR) for trials, within six months of FDA approval or 18 months of a trial’s completion. (CSRs contain a study’s protocol, case record, and statistical analysis plan, among other information.) We also evaluate whether companies transparently disclose how many data requests they receive and how each request is handled (i.e., granted or denied). In 2019, four companies achieved top scores in the GPS data-sharing ranking: J&J, Novartis, Novo Nordisk and Roche.
J&J was one of the first companies to commit to sharing patient-level data and helped develop a platform called the Yale University Open Data Access (YODA) Project. GlaxoSmithKline was also an early adopter, having helped create infrastructure to facilitate widespread data sharing among the industry (ClinicalStudyRequest.com); it was not included in the 2019 rankings because it didn’t have any new medicines approved that year but scored a 100 percent on data sharing in the 2015 rankings. Roche, another consistent performer, has kept pace with evolving data-sharing standards, as has Novo Nordisk, which initially took a unique approach to data sharing: It shared data directly on its website, allowing researchers to download them without any intermediaries or other hoops.
Novartis did not initially get a top score. It was deficient in one respect: it had made a public commitment to share data after drug approval, but only if a trial’s results were published. Since results from many trials are never published, this could create a loophole of sorts and a gap in data. Novartis used the GPS 30-day amendment window to close this gap and commit to sharing data for approved medicines regardless of whether a paper is published on a clinical trial—joining its peers in committing to high ethical standards in research transparency.
The Scorecard’s Impact
The pandemic has underscored the importance of data sharing in maintaining public trust in drug development and for public health. In the race to develop COVID-19 therapies and vaccines, drug companies have been inconsistent in their data-sharing practices, and regulators have been criticized for some of their decisions. This lack of transparency and the deterioration in public trust could make people reluctant to get vaccinated against coronavirus. Data sharing may help build public confidence in approved vaccines, as surveys show a majority of Americans would trust scientific research findings more if data were publicly shared. Given that there are no legal obligations to publish or share patient-level trial data with the public or health-care providers, we must rely on voluntary commitments by companies and other research sponsors. The GPS can help us track, improve and communicate progress on data sharing.
We believe we can continue to have a beneficial impact on the industry and patient care. We are working to expand the scope of the GPS into new areas and to new companies, to address issues that are important for patients, such as drug pricing and equity and inclusion in research, to name a few. The next rankings include, for the first time, companies of all sizes. The hope is that as ethical practices of pharma improve, as measured by this objective method, and progress is communicated to the public, the industry will earn back the respect of the people who depend on its products. People’s lives depend on it.
Jennifer E. Miller, PhD is an Assistant Professor at Yale University School of Medicine, Founder of Bioethics International, and Director of the Good Pharma Scorecard.
For more information, please visit bioethicsinternational.org.