Sacubitril - Valsartan, a "New Threshold of Hope" or Hype for Heart Failure?

Summer must be over, because the next new blockbuster drug appears to be here.  At the end of August, the media heralded the results of a study of a new product for congestive heart failure (CHF) from Novartis.

Novartis' New "Game Changer"

The New York Times article was fairly restrained, simply saying it

has shown a striking efficacy in prolonging the lives of people with heart failure

The Reuters article's headline said

New Novartis drug may upend heart failure treatment

In Forbes, Matthew Herper called it a

Game Changer

The accompanying editorial in the New England Journal of Medicine (1) suggested the drug

may well represent a new threshold of hope for patients with heart failure

On the heels of the debate about the high price of Sovaldi, Gilead's drug similarly touted as a game changer for the also common disease, hepatitis C (look here), almost immediately the concerns about cost followed.  Another Reuters article quoted the past president of the European Society of Cardiology,

We are entering a new era of treatment and, of course, it will cost a lot of money, which is a problem

This blog was late to the party for Sovaldi, and did not get to examine the evidence used to support its ostensibly wondrous properties until a while after it was published, and after the drug was approved.  Therefore, I thought we should examine the new evidence about Novartis' new drug now, before the drug is reviewed for marketing approval.

Congestive Heart Failure

First, let me give a very informal review of congestive heart failure (CHF).  CHF is a syndrome that is the final common pathway for many patients with heart disease.  It can be commonly caused by prolonged hypertension, coronary artery disease (often after one or more myocardial infarctions, or heart attacks), cardiac valve disease, or various problems with the heart muscle (cardiomyopathies).  In CHF, the heart cannot pump sufficiently to meet the body's needs.  The body responds in various ways, not all advantageous.  There are two main types of heart failure.  In one, systolic dysfunction, the heart's pump function, measured by its ejection fraction, is directly reduced.  In the other, sometimes called diastolic dysfunction, the heart becomes stiff, and hence it may not fill with blood adequately.  In either case, a common response is accumulation of fluid.  Fluid can accumulate in the lungs, causing breathlessness.  CHF patients often are fatigued, breathless, find it hard to do physical activity, and may have swelling of the legs or abdomen (edema).

Starting more than 20 years ago, a series of landmark clinical trials found that various types of medication could be used to modify the often disadvantageous compensatory mechanisms evoked by CHF with systolic dysfunction.  Controlled trials showed these medications could reduce symptoms, reduce hospitalizations, and prolong life.  These medications included those that affected the renin-angiotensin system, used by the body to control blood flow and pressure, including angiotensin converting enzyme inhibitors (ACEIs), and angiotensin receptor blockers (ARBs).  Also found to be efficacious were beta-blockers and mineralocorticoid blockers. At this point, all these medications are available in cheap, generic form in the US.

Review of the New Study of Sacubitril and Neprilysin Inhibtion

So let us examine the article just published online that reports a trial of Novartis' new drug.(2)

The new Novartis medication is sacubitril, which inhibits an enzyme called neprilysin, which degrades various active compounds in the body.  Inhibition of the enzyme and the resultant increase in the level of these compounds seems to favorably affect the disadvantageous compensation mechanisms found in CHF.

A previous effort to develop a neprilysin inhibitor found that it produced an unacceptable risk of a serious side-effect, angioedema, when combined with an ACE inhibitor.  So the current study involved another neprilysin inhibitor, sacubitril, combined with a angiotensin receptor blocker, valsartan, sold by Novartis under the trade name Diovan.

The trial had many good features.  It was large, including 8442 patients randomized.  It followed patients for longer than the short-term, a median of 27 months.  Most importantly, it assessed real clinical outcomes that might be important to patients (that is, "patient-centered outcomes,") including death, acute hospitalization, and a measure of health status and physical function, the Kansas City Cardiomyopathy Questionnaire (KCCQ).  

So far, so good.  However, a close read revealed some major and some minor problems.
  
Low Dose of Comparator 

This is a bit complex so bear with me.  

The study assessed a combination of sacubitril and valsartan versus a well-known ACE inhibitor, enalapril.  The target dose of the sacubitril and valsartan, rather confusingly called LCZ696, was 160 mg of valsartan twice daily, for a total of 320 mg/ day of valsartan.  The target dose of enalapril was 10 mg twice daily, for a total of 20 mg.

Note that both ACEIs like enalapril and ARBs like valsartan are considered appropriate first line treatment of CHF with systolic dysfunction.  In the most recent ACCF/ AHA guideline for heart failure, the maximum suggested dose of enalapril is 10-20 mg twice a day, and of valsartan, 160 mg twice a day.  Previous studies have suggested that pushing these doses as high as patients can tolerate improves outcomes.  

So the current study compared a targeted maximum dose of valsartan (plus sacubitril) to a less than maximum dose of enalapril.  The article did not explain why the study did not call for a maximum dose of enalapril, or indeed why valsartan plus sacubitril was not compared to valsartan alone. 

Thus the advantages of valsartan plus sacubitril found in the study could be entirely due to the superiority of maximum doses of valsartan versus moderate doses of enalapril.  I do not believe the study design can allow conclusions to be drawn about the efficacy of sacubitril.

 Note that this problem was noted in an online comment on the article by Dr Martin Nitschke.   

Choosing to compare the new drug to a less than maximally efficacious dose of the comparator medicine could make the new drug appear to be more efficacious than it actually is.  This apparent major design problem has been listed as one of the ways commercially sponsored studies may be manipulated to make the sponsors' products appear better.  The particular tactic of conducting a trial "against a treatment known to be inferior" was listed in a 2005 article on clinical research manipulation by Richard Smith.(3)


Active Run-In Period

Before patients were enrolled in the trial, they went through two "run-in" periods.  In the first, they were switched from their previous doses of ACEIs or ARBs to enalapril, up to 10 mg twice daily.  Patients unable to tolerate this were not randomized.  

Then the patients were switched from enalapril to valsartan 160 mg plus sacubitril up to .  Again, if they failed to tolerate this, they were also not randomized.  

The article noted that substantial patients were disqualified during these active run-in periods.  During the enalapril period, of 10,513 patients, 1102 were dropped from the study, including 591 (5.6%) due to an adverse effect, 66 (0.6%) due to an abnormal test results, and 49 (0.5%) who died.  During the valsartan plus sacubitril period, of 9419 patients, 977 were dropped, including 547 (5.8%) for adverse effects, 58 (0.6%) for abnormal test results, and 47 (0.5%) who died.  

There are major problems with such active run-in periods.  First, by eliminating patients who had adverse effects from the main results, they make the adverse effect rate appear smaller.  

Second, they raise questions about the patient population to whom the study results apply.  The study results only clearly apply to patients who are known to be able to tolerate valsartan and sacubitril.  In practice, should the drug be approved, at that time the only such patients would be those who already participated in trials.  (Of course, the results only apply to patients known to be able to tolerate enalapril.  This drug, however, has been on the market for years, and is widely used for problems, like hypertension, other than CHF.  So there are already quite a few patients known to tolerate it.)

The problems with interpreting studies that included active run-in periods were clearly described in an article by Pablos-Mendez et al in 1998.(4) 

Failure to Consider Adverse Effects Specific to Sacubitril

Valsartan and enalapril are now old drugs.  Their adverse effects are well known.

Sacubitril is a new drug in a new class.  It inhibits an enzyme that in turn affects the metabolism of many substances.  Thus, it is possible it has widespread effects, and may have multiple, and possibly unusual side-effects.

Yet the current study did not apparently include any efforts to anticipate and attempt to assess such side effects.  This problem was pointed out in a comment by Dr Ivan Boyadzhiev.

Failure to consider specific adverse effects may mean these effects are overlooked.  Thus, it is possible that the new drug has adverse effects that go beyond what are addressed in the article. 

Large Numbers of Sites and Quality Control

The study was done in 1043 centers in 47 countries.  The list of investigators took up six pages of the supplementary documentation.  Thus the average number of patients recruited per center was less than nine.

It seems improbable that a study involving so many investigators and centers, most of whom must have devoted little of their time and effort to this particular study, would have adequate quality control.  One indicator that quality control may have been a problem was that although the study inclusion criteria included patients with class II-IV heart failure, about 5% of study patients had class I heart failure.

The article and the supplementary material did not discuss study coordination or quality control.  It is not possible to tell whether any such problems with quality control may have distorted the study's results.

Lack of Generalizability in the Patient Population

The complete list of exclusion criteria, only appearing in the supplementary material, was extensive.  Patients with many common problems were supposed to be excluded, and the definition of the some exclusion criteria were vague and subjective.  These problems included coronary or carotid artery disease "likely to require" surgical or percutaneous intervention within 6 months, "history of severe pulmonary disease," "any surgical or medical condition which might significantly alter the absorption, distribution, metabolism or excretion of study drugs," including inflammatory bowel disease, duodenal or gastric ulcers, evidence of liver disease, and "presence of any other disease with a life expectancy of less than 5  years."

CHF patients are commonly elderly and have other medical problems.  Thus the study's results may not generalize to many patients with CHF in real-life. 

 Summary

It would certainly be nice to have a new medicine to improve the management of chronic CHF.  Sacubitril may have promise to do so.  Unfortunately, the clinical trial that just generated much favorable publicity ahead of regulatory consideration of this drug had major, possibly fatal flaws.  Use of a probably non-comparable and inferior "comparator" medication made it impossible to tell whether sacubitril actually is efficacious.  Use of an active run-in period, and failure to consider whether this new drug may have unexpected side-effects made it likely that the study underestimated its adverse effects.  Study results may not generalize to many real-life patients.  There are reasons to be concerned about whether the quality of study implementation was adequate.

Therefore, all the enthusiasm about this drug may be premature, and does not appear to be evidence-based.  That clinical research sponsored by organizations that sell health care goods and services may be manipulated to make the sponsors' products look better than they really are is now an old story.  We have seen multiple instances in which drugs and devices turned out to be less efficacious and/or more dangerous than originally advertised.  Excess enthusiasm about such new innovations may drive up costs, and worse, hurt patients.  Physicians, other health care professionals, and those concerned about health policy ought to be much more skeptical about every new instance of a purportedly wondrous innovation. 

 Evidence-based medicine rigorously applied suggests that individual health care and health policy decisions should be driven by the best available evidence, mostly from clinical research, about the benefits and harms of tests, treatments, programs, and so on, in the context of what outcomes matter to patients.  The skepticism EBM should engender lead to health care that is more about patients and their outcomes, and less about ideology, hype, and hucksterism.

ADDENDUM (3 September, 2014) - See also post by Dr Vinay Prasad on the CardioExchange site.  He drew similar conclusions about the choice of comparator, and the active run-in period, and added a new concern about the early stopping of the trial.  The comments include a - ahem - vigorous exchange between Dr Prasad and Dr Milton Packer, the Principal Investigator of the trial.  IMHO, Dr Packer did not completely understand or appreciate the points made by Dr Prasad.  Also, see our later blog post on this exchange here.

ADDENDUM (9 September, 2014) - See also a post by Gary Schwitzer on the HealthNewsReview.org blog that summarizes other critical online comments about the PARADIGM - HF study.  In particular, Dr Richard Lehman on his Journal Review blog on the British Medical Journal website wrote this scathing paragraph:

 And now, alas, to the much-vaunted PARADIGM-HF trial. This has been hyped as the biggest breakthrough in heart failure for at least 20 years.  A fixed dose of enalapril was compared with a substance called LCZ696. This turns out to have been a mixture of valsartan and sacubitril, a neutral endopeptidase inhibitor, and most participants ended up receiving a daily dose of 320mg of valsartan, versus 20mg of enalapril. A total of 711 patients (17.0%) receiving LCZ696 and 835 patients (19.8%) receiving enalapril died after a median follow-up of 27 months, at which point the trial was stopped prematurely. So just what are we looking at here? How can we distinguish the effect of the sacubitril from the effect of the high dose of valsartan? Well, we can’t. And this trial is a perfect example of everything that is wrong with heart failure trials. The mean age of the 8442 patients was 63.8, nearly  80% were male, and they were selected by reduced ejection fraction in 1043 centres across 47 countries. A logistic nightmare, but a great way for Novartis to spread influence. And Novartis then collected, managed and analyzed all the data itself. As I have said, a submaximal dose of one drug was compared with a maximal dose of another plus an extra ingredient. There was a run-in period, in which patients who were intolerant to the new treatment (12%) dropped out. Adding bias to bias, the trial was terminated prematurely. One of the primary end-points was hospitalization, which was ably demolished in a JAMA article I pointed out last week. And yes, there was a mortality benefit, but the number-needed-to-treat was about 35 to prevent one death in 2 and half years, in a population far removed from the elderly co-morbid patients we see in real life. As I draw to the end of a clinical career in which I’ve tried to help people with heart failure, I stand bemused. This is just how things have been done for the last 30 years, and it’s not good enough. At the very least, Novartis must make its full data set available for independent analysis. And before sacubitril is licensed, we need a properly designed trial, say between valsartan 160mg b.d. alone and valsartan plus sacubitril, in a typical population with heart failure. It will be very interesting to see what the FDA, the EMA and NICE decide.

 
References
1.  Jessup M. Neprilysin inhibition - a novel therapy for heart failure.  N Engl J Med 2014;  DOI: 10.1056/NEJMe1409898.  Link here.
2.  McMurray JJV, Packer M, Desai AS et al.  Angiotensin - neprilysin inhibition versus enalapril in heart failure.  N Engl J Med 2014; DOI: 10.1056/NEJMoa1409077  Link here.
3.  Smith R. Medical journals are an extension of them marketing arms of pharmaceutical companies.  PLoS Medicine 2005;  DOI: 10.1371/journal.pmed.0020138.  Link here.
4.  Pablos-Mendez A, Barr G, Shea S. Run-in periods in randomized trials: implications for the application of results in clinical practice.  JAMA 1998;  279(3): 222-225. doi:10.1001/jama.279.3.222.  Link here.

Desperate, Vulnerable Research Subjects, Cost-Cutting Contract Research Organizations and Threats to the Integrity of Clinical Research

Introduction - Clinical Research Done by Contract Research Organizations

Dr Carl Elliott seems to be one of the few people willing to investigate how modern medical research may threaten vulnerable research subjects.  His book, White Coat, Black Hat, opened with a chapter on vulnerable "guinea pigs," people willing to be clinical research subjects for money.  Such people may be desperate for money, and further may be homeless, and have psychiatric problems, including psychosis or drug or alcohol problems.  Dr Elliott just wrote another important article on the plight of vulnerable research subjects. As Dr Elliott wrote,

Most people think of pharmaceutical research as a highly technical activity that takes place in world-class medical centers. The reality is somewhat different.

This is apparent in a grainy video that I watched a few years ago. It had apparently been recorded on a cell phone, and the camerawork started off wobbly. A tanned man wearing sunglasses and a necklace appeared and was introduced as Dr. Johnny Edrozo, a psychiatric researcher. His shirt was unbuttoned partway down his chest. 'The latest stimulant coming out of the market is Vyvanse, which is a Dexedrine preparation,' Edrozo told the interviewer, pausing occasionally to chew gum. For reasons that were not explained, the interview took place in a parked car.

This was my introduction to South Coast Clinical Trials, a chain of private research sites in Southern California that specializes in testing psychiatric drugs. Pharmaceutical companies now typically outsource clinical studies to contract research organizations like South Coast, which run trials faster and at lower cost than universities do. Their job is simply to follow the instructions of their sponsors.

This formula is working: The contract research industry has grown steadily since the early 1990s and may now generate over $100 billion in annual income, according to the Tufts Center for the Study of Drug Development. At the top of the heap are corporations like Quintiles, which has 28,000 employees and operates in about 100 countries. At the other end are private physicians and small companies like South Coast, which are often based in strip malls or suburban office parks.

We first wrote about contract research organizations in 2005 based on a Bloomberg report that noted research conducted in a crumbling physical plant, on poor, often drug-addicted patients who "barely read" informed consent documents, bad record keeping, supervision by poorly trained or unlicensed clinicians.  It appears the problems have only gotten worse.

Vulnerable, Desperate Research Subjects


Dr Elliott made it clear that many of the research subjects enrolled by companies like South Coast are vulnerable. He noted that the companies purposefully recruit subjects from rooming houses and homeless shelters. Here is a description of the sorts of pitches they use.

'I was tired, I was hungry, and half an hour earlier the police had treated us like crap,' Burns said. 'And this woman is saying, ‘Imagine, in 40 days you’ll have $4,000!’ The recruiter made testing drugs sound like a vacation in a five-star hotel, Burns said. 'It’s like a resort selling time shares. They talk about all the benefits first, and it sounds great, but then you start to ask: What do I have to do?' 

Dr Elliott emphasized that such research studies can endanger subjects, but that poor, homeless, drug or alcohol addicted, or psychotic people offered thousands of dollars for participation are not likely to worry about the danger. He also suggested that monitoring and protection of such subjects may not be the most intense.

Obviously, this may be very bad for such research subjects. This is why it has long been considered unethical to recruit such vulnerable subjects in such dubious circumstances. But in this day and age of "greed is good," it may be all to easy to dismiss the risks as contingent on life styles that were already full of "bad choices."

Threats to the Validity of Clinical Research

Recruiting vulnerable patients, especially the poor, homeless, drug or alcohol addicted, or psychotic, into clinical research has dangers for patients not in those trials, and for health care professionals.

Remember that the principles of evidence-based medicine suggest that health care professionals should base decisions for patients on the best clinical research evidence. The quality of these decisions determine the quality of patient care and strongly affect patient outcomes. .

Yet if an increasing proportion of clinical research is being done on vulnerable, that is poor, homeless, drug or alcohol addicted or psychotic people, the validity of that research may be badly compromised.

Consider this narrative from Dr Elliott's article,

I walked round the corner to a shelter, where I talked to an elderly white man. 'I’d say the majority of guys here take advantage of that,' he told me, 'because they get a lot of money and they’re broke as hell.'

So, for example, to qualify for a study of drug addiction treatment,

I mentioned a recruitment flyer I’d seen outside the shelter asking for subjects with 'cocaine dependency.' George nodded. He told me that a lot of people start taking drugs just so they can qualify for those studies.

'You take that s*** two days before to get it into your blood.' He mentioned that he had recently screened for a trial at a research site running addiction studies. 'There were people in the waiting room high as a kite,' he said. 'They were incoherent.'

Also,

The main ethical issues here, of course, are the competence and judgment of the prospective subjects. 'When you say ‘money,’ everything else goes out the window,' said Hanif Jackson, a former program supervisor at the Ridge Avenue shelter in Philadelphia, which recently closed down. I heard the same thing from Harvey Bass, a chaplain who has worked at the Sunday Breakfast Rescue Mission shelter for 15 years. He said drug study recruiters often park outside the shelter and approach residents on the sidewalk. Although Bass didn’t think it was his place to warn residents away from the studies, it was clear that he was not exactly a fan. 'These guys have no job, no home, and a habit, he said. 'You have people at their lowest state, and they’ll say yes to anything.' 

As someone who has reviewed innumerable reports of clinical trials and other clinical research studies for journal clubs, journals, and grant review committees, I know that on paper clinical research studies have elaborate and specific criteria to include and exclude patients, and require patients to undergo detailed study protocols. However, can one really expect patients who are so desperate for money that "they'll say yes to anything" to admit to conditions which would exclude them from a trial? Can one really expect that they will faithfully adhere to research protocols that might require, for example, complete abstinence from alcohol? Can one really expect accurate answers on surveys meant to define their clinical outcomes?  There are numerous threats to the validity of trails conducted on poor, vulnerable, often psychotic or drug or alcohol addicted patients.

Cost-Cutting CROs and Shoddy Research

However, it is unlikely that contract research organizations who recruit such patients and run such studies are really up to implementing these complex, detailed, exacting protocols. Consider this narrative by Dr Elliott,

I visited a research unit at Lourdes Medical Center of Burlington County in Willingboro, New Jersey. The unit was operated by CRI Worldwide, the same company that Burns told me he had spoken with. (The company is now known as CRI Lifetree.) Its focus was on inpatient Phase I trials, which often involve gradually increasing the dose of a drug until subjects begin to feel toxic side effects. Some Phase I studies also require painful or unpleasant invasive procedures. For these reasons, the payment to subjects in Phase I trials is usually much higher than it would be for an outpatient study.

My first thought about the CRI unit: Its appearance did not exactly suggest clinical excellence. Most of the furniture looked as if it had been rescued from a Salvation Army store. Homemade notices with titles such as 'Smoke breaks' and 'Money requests' hung on the walls. No studies seemed to be going on, but a few people were wandering around or watching television, presumably waiting to be screened or assessed.. 

It was in this unit that a patient named Walter Jorden died. Dr Elliott narrated the events that took place before his death. Note that,

according to Jeffrey Fierstein, a cardiologist retained as an expert witness by Jorden’s family, the physicians involved deviated from expected standards of care by not more seriously considering the possibility that Jorden was having a heart attack. In Fierstein’s opinion, they not only ignored classic signs of a heart attack, but also neglected to perform an EKG and missed the opportunity to give Jorden the clot-busting drugs that might have saved his life. As Fierstein points out, 'My understanding is that it [the emergency department] was around the corner; it was right there.'

In addition, a companion article by Peter Aldhous  suggested that CROs and the drug, device and biotechnology companies that employ them are willing to allow physicians with questionable backgrounds to run clinical research. He found that a not insignificant minority of physicians listed as in charge of trials in a US Food and Drug Administration (FDA) database had records of dubious conduct, including various sanctions by state medical boards. He focused on one physician who had run trials since the 1980s, starting with Lovelace Scientific Resources. His record included one-year probation for drug issues, a diagnostic omission described by the medical board as "grossly negligent," then a three-year probation for that and other clinical care problems, then a license suspension, then finally license cancellation. Throughout all the years involved he continued to run clinical trials. In general, Mr Aldhous wrote,

My trawl netted dozens of doctors selected to work on clinical trials over the past five years who had previously been censured by state medical boards. Thousands of doctors are hired each year to test experimental drugs, making this a small minority. But most doctors have clean records, so companies should have few problems finding recruits without red flags against their name. 

His conclusion was,

After spending months in the world of clinical trials, I’m left with an impression of a system that has evolved beyond the FDA’s ability to manage it. I’ve also been struck by the profound disconnect between the disciplinary system that governs everyday medicine, and the separate regulations meant to protect clinical trial volunteers.

Speed and efficiency seem to be what matters to industry, and at times these factors appear to trump concerns about the doctors who run trials. 'The whole thing is profit driven,' says Michael Carome at Public Citizen, a consumer advocacy organization in Washington, D.C. 'You can see where corners might be cut, looking the other way when there might be concerns about an investigator.'

Some experts argue that the FDA’s entire rulebook for clinical trials, with its talk of things like 'institutional' review boards, reflects the academic past of clinical research—not today’s industrial juggernaut of for-profit clinical trials firms and for-hire review boards, which oversee a workforce of doctors drawn from regular medical practice. 'They are regulations for a world that doesn’t exist anymore,' says Elizabeth Woeckner, president of Citizens for Responsible Care and Research, which campaigns for the safety of medical research volunteers.

Again, this is similar to results of previous journalistic investigations of contract research organizations (some examples are here), going back to the Bloomberg article we discussed in 2005, and what Dr Elliott wrote in White Coat, Black Hat.  Thus we need to be extremely skeptical of the validity of clinical research implemented by contract research organizations, especially when the research subjects are vulnerable. 

Summary - Another Set of Threats to the Integrity of the Clinical Research Base

So given the push to do research rapidly at the lowest cost, the lack of supervision and regulation by the FDA, the hiring of physicians with problematic backgrounds, the willingness to take vulnerable patients desperately motivated by money, can we trust that the nice, clean, detailed descriptions of clinical trials implemented by contract research organizations presented in research articles and trial registries have anything to do with the reality of what went on? If not, what then should we make of the validity of the results of such trials?

This is compounded by our inability to tell who actually implemented any given trial.  While articles in most major clinical journals will list what companies sponsored trials, and whether the official paper authors had financial relationships with these companies, the articles do not describe who actually implemented the trials, or disclose whether contract research organizations were involved.

So the first obvious reform would be to require trial reports to disclose involvement of contract research organizations, and perhaps to list the personnel who actually were most responsible for implementing studies in addition to listed authors.

We already have discussed repeatedly how clinical research sponsored by organizations with interests in the outcomes favoring their products and service may be manipulated, and may be suppressed if even such manipulation fails to produce desired results.  We usually have assumed, however, that the published trial reports are generally accurate in their descriptions of trial implementation.  If they are not accurate, particularly because vulnerable subjects may say or do anything to stay in trials that pay them, and sometimes because of poorly qualified or impaired physicians running trials, this adds another layer of questions about the validity of commercially sponsored research.  This is yet another reason to ask whether we need to take research on human subjects meant to evaluate commercial products or services out of the hands of the companies that make those products and provide those services. 

Sovaldi, a Quantum Leap... Backwards to the Days Before Randomized Controlled Trials?

The Sovaldi (sofosbuvir - Gilead) media circus is continuing.  The New York Times just reported that sales of the new drug for hepatitis C were about $3.5 billion for the last quarter, which should intensify the kerfuffle over its US price ($1000 per pill, $84,000 for a 12 week course of medication).

Meanwhile, reports of its wondrous properties continue to appear in medical journals.

The latest was announced this way in Bloomberg,

Gilead Science Inc Solvadi, controversial because of its price, helps cure hepatitis C in people with HIV, according to researchers who say the drug has the potential to limit a top cause of death in these patients.

In a study of 223 HIV-infected patients, Solvadi combined with ribavirin cleared the most common US strain of hepatitis C in 76 percent of newly treated patients over 24 weeks. Only seven participants stopped treatment because of side effects and there were no adverse effects on HIV treatment, according to a report in the Journal of the American Medical Association.

Not unexpectedly, the study investigators were thrilled,

This is 'the first clinical trial to demonstrate that we can cure hepatitis C in patients with HIV co-infection without the use of interferon,' said Mark Sulkowski, the study author and medical director of the John Hopkins Infectious Disease Center for Viral Hepatitis in Baltimore. 'It represents a transformative step in our approach to this therapeutic area.'

Furthermore, as noted in a Medscape report,

In an accompanying editorial,(1) Michael Saag, MD, also of the University of Alabama at Birmingham, said the finding is a 'quantum leap' forward for people coping with both viruses.

What Clinical Trial?

I have posted previously about the controversy over the high price of Sovaldi.  I noted that most of the arguments, even from people most incensed about paying $1000 per pill, assumed that the drug was nearly miraculous, e.g., "a triumph," or "a revolution" in medicine.  Yet my reading suggested that the published evidence from clinical studies of the drug was in fact weak.  Critical reviews of the evidence from groups in the US, the UK, and Germany (look here, here, and here) also questioned the evidence backing Sovaldi.  Yet doubts about the drug's benefits versus harms have yet to influence the public discussion, which is focused on whether $1000 a pill is too much to charge for a miracle drug.

So I thought that "quantum leap" demonstrated in the latest trial(2) would bear further scrutiny.

The most important feature of the "trial" is how its design is described,

In this multicenter, open-label, nonrandomized, uncontrolled phase 3 trial, all patients received 400 mg of sofosbuvir (Gilead Sciences) administered orally once daily along with ribavirin (Ribasphere, Kadmon) administered orally twice daily, with doses determined according to body weight....

Let me make very clear, in this "trial," all patients got the same therapy.  There was no comparison group.  The study description is therefore inherently self-contradictory.  In fact, this study was a case-series, not a clinical trial.  It had no control group.

The classic Users Guides to the Medical Literature suggests that the first question a critical reviewer should ask about a study to assess a treatment is "was the assignment of patients to treatment randomized?"(3) The rationale was that studies of patients for whom clinicians made treatment decisions could easily be confounded by the reasons clinicians used to make these decisions.  Clinicians are trained to individual treatment decisions, and thus patients prescribed a treatment are likely to greatly differ from those not assigned that treatment.  The ways they differ could have as much, or more effect on their outcomes than the treatments themselves.  Although complex statistical adjustment methods could be used to try to compensate for these differences, it is rarely possible to be assured that all important differences have been taken into account. 

However, this still assumes a comparison between patients getting different treatments, or treatment versus no treatment.  In the case-series of sofosbuvir, there were no patients getting alternative treatments.  Thus the study itself could not be used to make comparative assessments of sofosbuvir.  

Of course, one could imply comparisons with the results of other studies.  However, such comparisons would not only be susceptible to confounding by reasons for treatment decisions as above, but to bias due to  differences between the current case series and previous studies, e.g., in characteristics of patients recruited, details of treatments given, study settings, and changes in patients, diseases, and treatments over time.   

Thus, a very basic rule in critical review is thou shalt not draw any conclusions about therapy from case-series unless the results are truly extraordinary, i.e., patients clearly doomed to die suddenly become completely well.  There were no such extraordinary results in the case-series of patients with HIV treated with Sovaldi for hepatitis C. 

The latest study of Sovaldi for patients with HIV and hepatitis C is virtually useless to assess whether Sovaldi might be the preferred therapy for such patients.   

Comparable Results?

Yet in an accompanying editorial(1), Dr Michael S Saag asserted that this non-trial 

demonstrates the efficacy of the all-oral regiment of the directly acting agent sofosbuvir plus ribavirin in patients with HCV who are coninfected with HIV

To demonstrate efficacy means to show that the treatment caused good effects.  But without a control group, it is not possible to assess causation.

Also,

The PHOTON-1 trial demonstrates that the all-oral combination of sofosbuvir plus ribavirin yielded results comparable with those of standard pegylated-interferon-ribavirin-vased regimens for each of the genotypic groups, thereby demonstrating that an all-oral combination could achieve similar outcomes as those derived from an injectable therapy.

However, this case-series could not assess comparability, because it included no comparison.


There Goes the Neighborhood - Throwing Out Skeptical Evaluation in the Rush to Promote "Innovation"

For years, the randomized, and usually double-blind, controlled trial has been considered the scientific cornerstone of the evaluation of new treatments.  Yet in the rush to bring new "innovative" treatments to market, presaging often tremendous revenues for their manufacturers, suddenly the randomized controlled trial does not seem so necessary.  In the US, the Food and Drug Administration (FDA) no longer seems to require randomized controlled trials to demonstrate efficacy of new drugs deemed to be "breakthroughs."   As noted by Steinbrook and Redberg in the July, 2014, issue of JAMA Internal Medicine(4)

these drugs [for hepatitis C] were approved quickly based on the surrogate endpoint of sustained viral response rate; they were studied in limited populations without long-term follow-up.  Under the US Food and Drug Administration's breakthrough therapy designation, the marketing for sofosbuvir did not include requirements for randomized clinical trials.

Yet without such trials, how could one ever determine if the drug is really so efficacious as to constitute a "breakthrough?"  


As we noted earlier, the two real randomized controlled trials of sofosbuvir did not assess its effects long term, even though it is a drug meant to treat a chronic disease, and particularly did not assess whether it affect clinical outcomes, that is, reduce rates of cirrhosis, liver failure, or liver cancer.  In fact, there has never been a controlled trial meant to assess whether any treatment of hepatitis C affects these outcomes.  Furthermore, while the first author of the case-series above, and many others have claimed the Sovaldi cures hepatitis C, it has never been shown that SVR equals cure.  As Gluud et al wrote in a letter to Lancet,(5)

The claim of cure rests solely on sustained virological response (SVR).  Since some patients who achieve SVRs still go on to develop end-stage liver disease and might die from the interventions, this concept is not correct and might even be wrong in principle.

However, despite all the words written, the concernthat Sovaldi (and possibly other new drugs for hepatitis C) might not be wondrous quantum leaps, because there is really no good evidence to that effect, has remained anechoic.  Not only does the evidence about the benefits and harms of this new drug need to be part of the discussion whether its price is excessive, there needs to be a larger discussion of whether in our rush for "innovation," we will throw out the baby of scientifically valid assessment of new therapies with the bathwater.

Finally,  the Sovaldi case is a signal example of how our health care system is awash in marketing hype and public relations buzz that has swamped rational skeptical thinking about logic and evidence.  That marketing and PR is ever enriching managers while it will send the rest of us, health care professionals included, to the poor house.  And all the money we spend will not buy us the promised miracles and triumphs.

As we have said until blue in the face, true health care reform would bring some skeptical thinking and regard for evidence and logic into the health policy discussion.  

 References

1. Saag MS. Quantum leaps, microeconomics, and the treatment of patients with hepatitis C and HIV coinfection. JAMA 2014; 312: 347. doi:10.1001/jama.2014.7735.  Link here.

2. Sulkowski MS et al.  Sofosbuvir and ribavirin for hepatitis C for patients with HIV coninfection.  JAMA 2014; 312: 353.  doi:10.1001/jama.2014.7734.  Link here.
3. Guyatt GH, Sackett DL, Cook DJ et al. Users' Guides to the Medical Literature: II. How to Use an Article About Therapy or Prevention A. Are the Results of the Study Valid? JAMA. 1993; 270(21): 2598-2601. doi:10.1001/jama.1993.03510210084032. Link here.
4.  Steinbrook R, Redberg R. The high price of the new hepatitis c virus drugs.  JAMA Int Med 2014; 174: 1172.  Link here. 
5.  Gluud C, Koretz R, Gurusamy K. Hepatitis C: a new direction, but an old story? Lancet 2014; 383: 2122-2123.  Link here.

GlaxoSmithKline's Marketing Firm Promised "Clinical Trials Could be Your Solution" to Poor Graduate Students

We have frequently raised concerns about the increasing domination of clinical research, particularly clinical trials, by those with vested in interests in the research producing particular results.  In particular, drug, biotechnology and device companies often sponsor trials meant to evaluate their own products.  Often it appears that commercial trial sponsors manipulate various aspects of research design, implementation, analysis, and dissemination to increase the likelihood of a result favorable to their interests.  Furthermore, when even such manipulation fails to produce the desired result, particular studies may simply be suppressed, that is, hidden.

Clinical Trial Participation as "Your Solution" to Graduate Students' Money Woes

A recent story, first told on the In-Pharma Technologist web site, gave an example of how trial recruitment could be gamed so as to produce more compliant study subjects. Since that web-site does not allow copy and pasting of even a few sentences of its content, I will quote, instead, from a subsequent article in the Guardian.

GlaxoSmithKline  the pharmaceuticals multinational, has apologised after being accused of playing on the hardship of unpaid interns to recruit them to take part in clinical trials.

A marketing firm working for the FTSE 100 company sought to place a blog on careers advice websites boasting that involvement in drug trials could help graduates to finance their way through unpaid work placements.

In a proposed 'guest blog' for the website Graduate Fog, an employee at TouchPoint Digital, working on behalf of GSK, wrote: 'Clinical trials could be your solution.'

The "guest blog" proposed that many graduate students "would prefer an immediate income to tide you over for the coming months."  So,

Clinical trials could be your solution. Depending on the length of the study, you could earn up to £2,000 per trial for up to 4 trials a year, plus reasonable travel expenses.

This was not a one-off,

 Touchpoint provided examples of similar articles previously published on websites targeted at students. In an email to the founder of Graduate Fog, Tanya de Grunwald, it added: 'Your readers would also benefit if there is a small link at the end of the article to the GSK website, which is the biggest pharmaceutical company in the UK, so that if they want to find out more or get answers to more specific questions they can do so.'

Excessive Inducement

The story was picked up by Ed Silverman at the PharmaLot blog in the US, who pointed out how TouchPoint Digital appeared to be supplying an excessive inducement to vulnerable subjects,

Tanya de Grunwald, who runs the Graduate Fog website in the U.K., objected to the tone of the overture. 'Some of my readers are feeling very low, vulnerable and desperate for money,' she wrote us in an e-mail after openly questioning the recruiting tactic in a note to readers on her website last week. 'Many are unemployed. Others are in low-paid graduate jobs or doing unpaid internships.'

'Suggesting that participating clinical trials is a great way to earn easy cash is not just crass – I think it’s irresponsible. Surely, the more desperate someone is for money, the more likely they are to be dazzled by the financial benefits and less careful about weighing up the risks.'

A long time ago, that is, from 2006 - 2008, we published a series of posts about how contract research organizations running clinical research projects for commercial health care firms often preyed on financially and otherwise vulnerable research subjects by offering what would appear to such people to be dazzling remuneration.  For example, we discussed the trials by SFBC International (later PharmaNet Development Group, and now apparently Inventiv Health Clinical)  in Miami that enrolled immigrants, often undocumented, under questionable circumstances and in Montreal that resulted in the transmission of active tuberculosis (see post here and links backward); and the trial by Parexel International in London that put most of its subjects in intensive care (see post here, with links backward). In 2008, as we discussed here, two articles questioning the ethics of research done under the auspices of CRO appeared in two major medical journals.  Excessively zealous recruiting designed to tempt vulnerable subjects with money appears unethical, and may be dangerous for these subjects.

How Excessive Inducement Could Lead to Invalid Trial Results

Not only is such a practice apparently unethical and dangerous, it could endanger patients outside the research projects in question by producing invalid research.  Subjects who could be dazzled by such inducements might avoid disclosing conditions which would otherwise exclude them from particular research projects, or participate in multiple projects without sufficient or any wash out periods in between, meaning that one treatment's effects could confound another.  (Note that the blog post proposed by the marketer above suggested that subjects could participate in multiple trials during one year.)  Furthermore, subjects desperate to complete studies and earn their fees might avoid reporting adverse effects to avoid the risk of premature termination of their participation, or avoid protesting questionable actions by the researchers.   

But now in the modern era of the Internet and social media, overzealous recruitment using monetary lures is still going on.  As per PharmaLot,

One expert believes the flap is not surprising, because drug makers are under pressure to maintain a steady stream of clinical trial recruits.

'Middle class folks just sit back and wait until the drug reaches the market. Poor folks are the guinea pigs in an economy that is more and more uneven, and uncertain by the day,' says Roberto Abadie, a senior researcher at the Social Networks Research Group at John Jay College, City University of New York, and author of ‘The Professional Guinea Pig: Big Pharma and the Risky World of Human Subjects.'

A Belated Response from GSK

Apparently only after the story broke did GSK notice there was a problem.  Again, per PharmaLot,

A Glaxo spokeswoman blamed the outside marketing firm for the episode. In an e-mail, she wrote us that 'we would agree that the tone used in the ads… is wrong. It trivializes the role of clinical trials in developing new medicines and the part that volunteers play in that process. This isn’t acceptable and we’re looking into what happened.'

Whether GSK was properly supervising the marketing firm, and whether GSK had provided incentives that might have lead to overzealous recruitment were not addressed.

Summary

This story is yet another reminder that health care professionals, health policy makers, and the public at large should be extremely skeptical of clinical research sponsored (and controlled) by those with vested interests in the research results turning out a certain way, particularly clinical trials run by drug, biotechnology, or biotechnology companies meant to assess those companies' own products. However, note that very rarely do published clinical trial reports provide enough detail about subject recruitment to determine whether overzealous recruitment and undue inducements may have lead to enrollment of subjects who should have been excluded, under reporting of adverse effects, etc.

In my humble opinion, we need to reassess current policies that allows organizations with such obvious conflicts of interest to run experiments on human beings (which is what clinical trials are.)

By the way, this story is also a reminder that most big drug, device and biotechnology firms seem to be associated again and again with dodgy clinical research.   For example, see what we have previously written about GSK here.

Fourth Time is the (Anti)Charm? - UK NICE Highlights "Uncertainties in the Evidence Base" About Sovaldi

As we have discussed, (here, here and here), while anger continues to build about the $1000/ pill price sought by Gilead for its new antiviral drug for hepatitis C, Sovaldi (sofosbuvir), almost all public discussion still treats the drug as miraculous.  However, my reading of some key trials, and reviews by three groups of evidence-based medicine experts, suggested that the evidence supporting the drug is actually weak and unclear, and hardly suggests it is miraculous.

NICE Weighs In

Now, as first noted by the indomitable Ed Silverman in his revived PharmaLot blog, the National Institute for Health and Care Excellence in the UK is also skeptical.   

the U.K.’s National Institute for Health and Care Excellence, otherwise known as NICE, has declined to endorse the use of Sovaldi, at least for now, until Gilead supplies further evidence of the medication works in certain subgroups of patients. In announcing the move, NICE officials wrote there are 'substantial uncertainties' in the evidence from the drug maker. In a statement issued about a draft guidance, NICE wrote that the agency 'is minded not to recommend' that the U.K.’s National Health Service cover the cost of Sovaldi,

The actual statement said,

 The available evidence shows that sofosbuvir is an effective treatment for chronic hepatitis C in certain patients. However, evidence is lacking for some subgroups of patients with chronic hepatitis C, and there are also substantial uncertainties in the evidence base presented by the manufacturer. The Committee has therefore requested further information from the manufacturer before it can decide whether sofosbuvir is a cost-effective use of NHS resources.

In the US, the only other news source that covered the NICE statement was Bloomberg.

The Problems with the Evidence

As we discussed in our previous posts, the problems with the evidence underlying Sovaldi include:
- Lack of a randomized controlled trial comparing Sovaldi to the previously most recommended treatment regimen
- The only trial trial to compare an antiviral regimen containing Sovaldi to one without it  (Sovaldi plus ribavirn versus peg-interferon plus ribavirin) used a lower dose of ribavirin in the comparator regimen, seemingly handicapping it; had a highly selected patient population whose results would be unlikely to generalize to many patients in the real world; had issues with randomization of patients; only assessed short-term "sustained" virologic response, but not any clinical outcomes; even so, did not show that the sofosbuvir containing regimen produced a better SVR than did the comparator; and appeared to show that the sofosbuvir containing regimen produced more severe adverse effects, and perhaps a higher death rate than the comparator regimen
-  So far, the other published trials included one versus placebo, and multiple trials that compared only sofosbuvir containing regimens to each other, and hence were effectively just case series of patients receiving sofosbuvir.  These case-series all had highly selected patient populations results from whom would be unlikely to generalize to real world patients.

The severe problems with the evidence have now also been noted by
-  the German Institute for Quality and Efficiency in Healthcare (IQWiG)
-  the US Institute for Clinical and Economic Review (look here)
-  the US Center for Evidence-Based Policy (look here)
-  the UK National Institute for Health and Care Excellence (NICE)

Summary - Why Does the Evidence, or Lack Thereof, Remain Anechoic?

Yet none of these reviews have gotten any significant attention in the US media or medical and health care literature, and the idea that Sovaldi has hardly been proven to be a miracle drug, or even better than older drugs for hepatitis C has not informed the US debate and the US outrage about its price.  Examples of the most recent outrage include this in Forbes,

A cure for hepatitis C is within reach for 170 million people around the world — thanks to the charitable efforts of poor and sick Americans who are picking up the tab by paying outrageous prices for their own treatment. It’s like Robin Hood in reverse.

Also,

It also took a complete lack of self-awareness — and unmitigated gall — to price Sovaldi the way Gilead has.

Furthermore, Democrats in the US House of Representatives are calling for an investigation (which their Republican "colleagues" will doubtless block), according to Bloomberg,

The company 'did not provide a compelling justification for the high price they are charging for most patients,' Waxman and DeGette wrote 

If the price is outrageous for a miracle drug, it would be even more outrageous for a drug that has not been proven to be better than previously available treatments.

The fact that skepticism about all the hype for Sovaldi has hardly touched the public discussion in the US is a prime example that the anechoic effect lives.  (The only skepticism from an expert could [probably only] be found again in PharmaLot, and came from one of the authors of the Center for Evidence Based Policy report [see above]). 

'For most patients with hepatitis C, they have time to make those decisions,' Valerie King, one of the physicians at the center who worked on the Sovaldi review, tells OregonLive. 'I’m certainly not saying that this is a bad drug. I’m just saying that we don’t know that it is a good drug.' [UPDATE: King later called us to say that 'the research hasn't been extensive enough or transparent enough to tell us it is a good drug or bad drug, or has limited application.' This was based solely on assessing clinical research, not cost issue.]

We have endlessly discussed the anechoic effect, that information and ideas that challenge the powers that be in health care, and particularly that challenge the ability of health care leaders and well-connected insiders to personally profit, often to a tremendous extent - the CEO of Gilead, John Martin, was listed by FiercePharma as the eighth highest paid biotech CEO in 2013, with total compensation of $15.45 million, before sales of Sovaldi really increased -  are considered recent unpleasantness that are just not to be discussed.  However, without open honest discussion of truths, however inconvenient or unpleasant, and the ideas that they suggest, health care will continue to degenerate into a plutocratically run, often corrupt swamp with ever increasing costs, and ever worsening access and quality, causing increasing suffering of patients and worsening of public health.

A good place to start true health care reform might be honest discussion of the evidence about sofosbuvir.   

The Smarming of Health Care - How to Spin a Minimally Efficacious Variant on an Old Drug as New and Wonderful

A recent announcement of the approval of a new drug showed the latest ways pharmaceutical companies can spin products as new and effective when in reality they are barely either. 

Announcing a Drug Approval with a Twist

United Therapeutics just got US Food and Drug Administration (FDA) approval for their new oral prostacyclin agent for the treatment of pulmonary arterial hypertension. The basics, per Bloomberg, were:

United Therapeutics Corp. (UTHR)’s gained U.S. marketing approval for the oral version of its treatment for high blood pressure in the arteries that supply the lungs.

The Food and Drug Administration cleared orenitram, an extended-release tablet, for the treatment of pulmonary arterial hypertension to improve exercise capacity in some types of patients, the Silver Spring, Maryland-based company said today in a statement.

The pill’s active ingredient, treprostinil, is the same as in the company’s approved injection and inhalation solution for PAH, a life-threatening disorder, United Therapeutics said. The FDA had rejected the oral version of the therapy in March and in October>

'This approval marks the first time that the FDA has approved an orally administered prostacyclin analogue for any disease -– and our fifth approval from the FDA for treatment of PAH -- supporting our mission of providing a wider choice of PAH therapies for physicians and patients,' Roger Jeffs, the company’s president and chief operating officer, said in the statement. 'We are grateful for the FDA’s thorough review and will continue to build clinical support for the use of Orenitram.'

But the triumphant announcement by the company CEO had some unusual twists.  As reported on National Public Radio's (NPRs) Shots blog,

'When I started the company 17 years ago, the only option for treating pulmonary hypertension was intravenous ,' Martine Rothblatt, CEO of United Therapeutics told investors in a celebratory conference call Monday. 'My young daughter looked at me and said, 'Can't there be a pill?''

Now there is

So, 

'What it has resulted in is something that as the parent of a lovely young lady with pulmonary hypertension, I cannot even put into words, the emotional meaning that it has to have a pill form of prostacyclin,' Rothblatt said.

It appears to be such a warm and happy story.  A mother is so concerned about her daughter's serious illness that she founds a company to develop treatments for it.  Now many years later they have come up with something groundbreaking.

How could anyone not shed a tear?  How could anyone not be happy about this triumph of commercial drug development.

However, while it may seem snarky to point this out, drug development is usually about issues other than "emotional meaning," and this discussion of emotional meaning took place during the public relations/ marketing blitz put on after the approval of the marketing of a new drug, so maybe closer examination, and some skepticism are warranted.


How Well Does Orenitram Work?

The quote above from the CEO was about "emotional meaning," not efficacy or effectiveness.  In fact, the blog post by Mr Hensley hinted at and a review of the one relevant published article strongly suggested that the drug really had minimal efficacy, and possibly benefits that failed to outweigh its harms.

Mr Hensley wrote,

 The FDA approved Orenitram on the basis of a study that gauged its safety and effectiveness as a standalone treatment. The drug helped people walk a little farther in a timed walking test. The most common side effects seen in the study were headache, nausea and diarrhea.

On my review of the evidence....

A single randomized controlled trial(1) showed that patients given treprostinil could walk on average 26 meters (less than 100 feet) more in 6 minutes after 12 weeks of treatment than they could at the beginning of the treatment, while those given placebo were only able to walk about as far as they did before.  The treated patients also had some improvement in their shortness of breath.  On the other hand, they were substantially more likely to experience adverse effects, particularly nausea, vomiting, diarrhea, flushing, jaw pain, or pain in the extremities. 

Furthermore, the study lasted only 12 weeks, and provided no information about outcomes after that.  Treated patients were not spared overall worsening of their clinical situation compared to untreated patients.  Treated patients were no less likely to be hospitalized or to die.

Various aspects of its design may increase doubts about its validity, including emphasis on a "modified intention-to-treat population" which was not clearly defined, a relatively high rate of premature study drug discontinuation, and a complex statistical analysis scheme which seemed unnecessary to assess the main results of a randomized controlled trial.

Finally, the generalizability of the study was limited to patients already optimally treated with conventional therapy, and perhaps by ist predominantly Asian female population.

So the only apparently positive trial of trepronsinil did not provide good evidence that its benefits outweighed its harms.  At best, some patients might find that the modest increases in exercise capacity and decreases in dyspnea it may produce may be worth nausea, vomiting, diarrhea, flushing, or jaw or extremity pain.  

Note that treprostinil is an oral version of a type of compound called a prostacyclin.  As noted above, United Therapeutics markets other forms of prostacyclin.  However, the latest (2005) Cochrane Collaboration systematic review of this class of drugs for pulmonary arterial hypertension only found evidence that the drugs helped when added to first-line drugs to improve symptoms for the short-term, no more than 12 weeks.  It found no evidence that prostacyclin prolongs survival, or lessens complications of the illness(2) 

So having an oral prostacyclin treatment for PAH may have emotional meaning, but the pill, while likely more convenient than an injectable prostacyclin, does not seem to have sufficient efficacy to outweigh its harms for many patients. 

What Does Orenitram Cost, and Why?

But United Therapeutics may have been able to make one aspect of this drug superlative, - its price.  Per Scott Hensley in the Shots blog,

Orenitram won't be available for sale for about six months, the company said, and its exact price hasn't been determined either. But it will be very expensive, about $150,000 a year, according to an estimate by Dr. Mark Schoenebaum, an industry analyst with ISI.

That would be about three times the median US family income.  But the CEO shrugged concerns about the price off,

Rothblatt acknowledged that the cost of the drugs is high but manageable, because pulmonary hypertension remains pretty rare, though the exact prevalence is hard to pin down.

One explanation for that amazingly high price comes from a post on Forbes by Matthew Herper,

She commercialized [prostacyclin] ..., and was able to get investors interested because she realized there was no maximum price insurers would pay for an effective drug. United Therapeutics became one of biotech’s great success stories.

A lack of an upper limit on pricing has other effects of course.  According to Morningstar, the company's sky high pricing enabled a net profit margin over 33% in 2012-13.   According to the United Therapeutics 2013 proxy statement, CEO Martine Rothblatt's total compensation is $7,958,328.  Three other executives got more than $4.5 million.

So I imagine that marketing a drug that may produce so much revenue may have emotional meaning for company executives.  On the other hand, in this era of high insurance deductibles and co-payments it may have considerable negative emotional meaning for patients.

Some Off-Key Notes


The recent publicity about the new oral prostacyclin did not dwell on an issue that appeared earlier in a December, 2013, press release from United Therapeutics about its other prostacyclin products,

 United Therapeutics Corporation (NASDAQ: UTHR) announced today that it has received a subpoena from the Office of the Inspector General of the Department of Health and Human Services reflecting an investigation by the United States Department of Justice, principally represented by the United States Attorney's Office in Baltimore. The subpoena requests documents regarding Remodulin® (treprostinil) Injection, Tyvaso® (treprostinil) Inhalation Solution and Adcirca® (tadalafil) Tablets, including the company's marketing practices relating to these products.

Instead, as we discussed above, the CEO of the company that will market this very expensive but not very efficacious drug went out of her way to bring up emotional issues about her own daughter's illness.  Certainly, anyone can believe the CEO was motivated to develop prostacyclins for PAH by concerns about her daughter's own illness.  However, focusing on such emotional issues may also distract from concerns about other aspects of the company's marketing.

So maybe it is not a surprise that she decided to name the new drug in a personal way,...  But she did not name it for her daughter.  Again per Scott Hensley,

 Now there is: Orenitram. The name comes from 'Martine Ro,' as in Martine Rothblatt, backward.

Instead, Martine Rothblatt names her company's new drug for.... herself. 

So what is this very emotional and personal drug marketing/ public relations campaign really about?


On Smarm in Health Care

So it appears that United Therapeutics and its CEO used what may now be called smarm to promote their vested interests and divert attention from uncomfortable facts. 

In a now widely read and commented upon article in Gawker, Tom Scocca asserted that smarm has become "an omnipresent, unnamed cultural force."  Some aspects of smarm, as discussed in the article,

Smarm is a kind of performance—an assumption of the forms of seriousness, of virtue, of constructiveness, without the substance.

 it expresses one agenda, while actually pursuing a different one. It is a kind of moral and ethical misdirection. Its genuine purposes lie beneath the greased-over surface.

 Smarm, whether political or literary, insists that the audience accept the priors it has been given. Debate begins where the important parts of the debate have ended.

So the smarmy element in the public relations about Orenitram was the misdirection created by invoking the CEO's daughter's illness and its emotional ramifications.  By focusing on an emotional issue which apparently only the callous could dismiss, the issues of the drug's minimal efficacy, frequent side effects and very high price were eclipsed.   Yet they, not the emotional connection between a particular parent and chile should be salient to patients, doctors, regulators, and the general public.

 (The issues of the investigation of the company's marketing of other prostacyclin products, and the egotism implied by the drug's name were certainly eclipsed as well.)

Presumably by raising these truths which might be unpleasant to United Therapeutics executives, I am now guilty of "snark."  As Scocca pointed out, one aspect of smarm is to exile reasonable criticism and skepticism as "snark,"

What is snark reacting to?

It is reacting to smarm. 

So snarky as it may be, I believe that the major considerations for patients and physicians when considering a therapy - like Orenitram - should be the benefits versus harms of the the therapy for the individual patient, and the strength of the evidence supporting both.  The more marketers, public relations functionaries, and executive try to distract from that, and the more smarm they employ to do so, the more skepticism is required. 

References

1.  Jing ZC, Parikh K, Pulido T et al.Efficacy and safety of oral treprostinil monotherapy for the treatment of pulmonary arterial hypertension: a randomized, controlled trial.  Circulation 2013; 127: 624-633. Link here.

2. Paramothayan NS, Lasserson TJ, Wells A et al.   Prostacyclin for pulmonary hypertension in adults. Cochrane Review 2005; Link here.

How Manipulated Clinical Evidence Could Distort Guidelines - the Case of Statins for Primary Prevention

The new American College of Cardiology (ACC)/ American Heart Association (AHA) guidelines on the primary prevention of cardiovascular disease, which we discussed here, continue to generate controversy. 

Articles in the media and online-first in medical journals underscored some of the issues we discussed before.  Jeanne Lenzer, in a news article in the British Medical Journal, found that the chair of the guideline panel had important past conflicts of interest that were not previously disclosed.(1)  The chair denied their significance (look here).   Guideline panel members continued to justify their efforts, but in my humble opinion, raised no new evidence or logic to support it (look here  and here)

Bigger Questions about the Validity of the Clinical Evidence

However, several new articles suggested the need for a deeper look at one particular aspect of this case, the validity of the evidence from clinical research about the benefits and harms of statins for primary prevention.

JAMA published a summary of the Cochrane review that provided a basis for the guideline developers' confidence in the worth of statin therapy in primary prevention.(2)

results suggest that the benefits of statin therapy outweigh serious life-threatening hazards.

However, almost as an aside, it noted,

Some trials included participants with CVD, but rather than exclude these trials, we included trials that contained 10% or fewer participants with documented CVD.

Primary prevention in this case is defined as prevention for patients without existing cardiovascular disease.  There is evidence that statins may well have benefits that outweigh harms when given to patients with known cardiovascular disease, particularly documented coronary artery disease (CAD).  Mixing such patients in any numbers into "primary prevention trials" would likely exaggerate the benefits of statins.  Yet such not quite primary prevention studies were included in a systematic review of primary prevention. 

In addition, a commentary by one of the guideline developers defending the group's work also underscored the fact that many of the supposedly primary prevention trials they used as evidence were not pure primary prevention trials.(3)

Notably, the 2013 cholesterol guideline cut points were derived from the placebo rates for myocardial infarction, stroke, and cardiovascular disease death observed in the 3 exclusively primary prevention statin trials, Air Force/Texas Coronary Atherosclerosis Prevention Study (AFCAPS/TexCAPS), Management of Elevated Cholesterol in the Primary Prevention Group of Adult Japanese (MEGA) study, and the Justification for the Use of Statins in Primary Prevention: an Intervention Trial Evaluating Rosuvastatin (JUPITER).[4-6]

The wording again suggests that all the other trials used as evidence about primary prevention were NOT "exclusively primary prevention studies," and hence, as I noted above, may have been biased so that they would likely exaggerate the apparent benefits of statins in primary prevention.

This suggested to me that the systematic review that provided a basis for the guidelines' aggressive recommendations about statins for primary prevention, and the trials on which it was based needed further skeptical, critical reviews.

The Cochrane Review: Was Evidence about Statin Benefits vs Harms Manipulated?

We have frequently discussed the manipulation of clinical research.  By that we have meant design, implementation, analysis or dissemination of research in ways likely to further vested interests.  In particular, when drug, biotechnology and device companies sponsor and control clinical research on their own products, they may set up the research in ways likely to make their products look better than they actually are.  

We looked through the most recent Cochrane review of statins for primary prevention(7) with a skeptical eye out for suggestions that the trials reviewed could have been so manipulated.  We found quite a bit.

First, we found that not all trials reported on the "hard" outcomes that one needs to consider when evaluating benefit vs harm of statins in primary prevention.

Data on all-cause mortality were provided in 11 trials.[of 19, and hence missing in 8].

And,

 Twelve trials provided data on adverse events.[and hence 7 did not.]

 Also, data on specific adverse events was often not reported: myalgia and rhabdomyolysis were reported in only 9/19 trials; new onset diabetes mellitus in only 2/18; hemmorhagic stroke in only 2/19; abnormal liver tests in only 10/19; kidney dysfunction in only 4/19; arthritis in only 2/19, and by implication, cognitive dysfunction in 0/19.  

  Failure to look for all the possible bad outcomes of treatment could obviously bias the study in the direction of minimizing the harms of the treatment.

A substantial number of trials either failed to report on crucial aspects of their methods, or admitted to flaws that could have induced important biases.: 3/19 did not described randomization methods; 4/19 did not use double-blinding; 6/19 did not use intention to treat analysis; 7/19 did not report their drop-out rates.

So it is very surprising to me that the authors concluded,

In general there was low risk of bias ... though all trials were either fully or partially funded by pharmaceutical companies.

In my humble opinion, the Cochrane review showed many trials that had flaws could have biased their outcomes, and hence the outcomes of the overall review.  Some of the flaws clearly could have lead to biases that would have made statins look more efficacious, or less dangerous than they might actually be.  I do not understand the conclusion that the risk of bias was slow, and the lack of discussion about the direction the bias could have taken.

 Review of the "3 Exclusively Primary Prevention Statin Trials"

Given the Cochrane review's apparent lack of skepticism about methodologic problems in the industry funded statin prevention trials, I endeavored to take a closer look at the three trials that Dr Robinson held out as the real primary prevention trials.  Instances of manipulation, as we defined it above, for each trial are described below

AFCAPS/ TexCAPS(4)

Narrow Patient Population - This study excluded many patient for whom the statins were not contraindicated or warned against: uncontrolled hypertension; type 1 or type 2 diabetes mellitus on insulin or with a HgBA1C at least 10%; and body weight more than 50% "desirable limit for height."  (Based on the official contraindications and warnings for commonly used statins, e.g., see contraindications for Lipitor here, active liver disease, pregnancy for likely to become pregnant, nursing mothers, hypersensitivity to the medicine; and warnings: use of cyclosprine or strong CYP3A4 inhibitors, uncontrolled hypothyroidism, renal impairment.)  Thus the results may not generalize to many patients who would otherwise be considered statin candidates.  By excluding such patients, the results may bias the study towards minimizing the probabilities of harms that might occur were statins used on a wide population for primary prevention.

Unknown Randomization and Allocation Concealment Procedures - According to the Cochrane Review, the study report did not explain how randomization or allocation concealment were accomplished.  

Early Termination/ Multiple Comparisons - The study was terminated early based on an early look at the number of outcome events.  Two such early or interim analyses were planned.  Taking multiple looks at the data over time raises a multiple comparisons problem, and may lead to exaggerating the benefits of the treatment.(8). Furthermore, stopping early decreases the sample size and hence the power to find adverse effects of treatment.

Implausible Dropout Rate, Missing Data - According to the Cochrane Review, the study reported no dropouts.  This seems somewhat improbable, suggesting skepticism about the accuracy and completeness of the data collection.  On the other hand, a study chronology suggests that of 6605 patients who started the study, 6540 had data on complete endpoint status, suggesting missing data.  Since dropouts and missing data may be due to different reasons in different arms of the study, they threaten the validity of data about benefits and harms.  

Adverse Effects not Reported - The study provided no data about development of diabetes, hemmorhagic stroke, kidney dysfunction, arthritis, or cognitive dysfunction, suggesting incomplete data about harms, and hence bias towards minimizing harms.    

MEGA (5)

Narrow Patient Population -  [The patient population was not described in the main report, but only in an earlier methods article.](9)  The study excluded patients with congenital or rheumatic heart disease; chronic atrial fibrillation, current diagnosis of malignancy; poorly controlled hypertension or diabetes mellitus; current use of oral or parenteral corticosteroids; and other conditions at the discretion of the physician.  These exclusions seem unrelated to the contraindications or warnings on the stain label.  Again, such a narrow patient population reduces the generalizability of the study results, and may bias the study to minimizing the harms of statins.

Only Single Blind - This was an open-label study, so patients and physicians knew who got statins and who got placebo.  Such knowledge could have biased patient management, including how diligently particular diagnoses and outcomes were pursued, and biased data collected from patients or physicians.

Adverse Effects Not Reported - The study provided no data about diabetes, hemmorhagic stroke, kidney dysfunction, arthritis, or cognitive dysfunction, again suggesting bias towards minimizing harms.  

JUPITER(6)

Narrow, Unusual Patient Population - The study was limited to patients without hyperlipidemia but with an increased C-reactive protein.  Thus it is not clear that its results would generalize to a more typically defined primary prevention population.  The study excluded patients receiving post-menopausal hormone-replacement; with diabetes; uncontrolled hyertension; cancer other than non-melanoma skin cancer within 5 years; recent history of drug or alcohol abuse; "another medical condition that might compromise safety or the successful completion of the study;" also patients with "inflammatory conditions such as severe arthritis, lupus, or inflammatory bowel disease...;" and also "patients taking immunosuppressant agents such as cyclosporine, tacrolimus, azathioprine, or long-term oral glucocorticoids."  Again, this narrow patient population would reduce generalizability and bias towards minimizing harms.

Early Termination/ Multiple Comparisons - This study was terminated early after an early look at the data.  Allowing for multiple looks at the data may exaggerate efficacy.  

Implausible Dropout Rate - According to the Cochrane Review, the study had no dropouts.  This value seems implausible, again suggesting data collection problems.  

Adverse Effects Not Reported - The study provided no data about hemmorhagic stroke, arthritis, or cognitive dysfunction, again suggesting bias towards minimizing harms.  [Revised December 9, 2013 - see comment below by Marilyn Mann.]

Summary

Aspects of the continuing controversy over the new ACC/ AHA guidelines for statins in the primary prevention of cardiovascular disease hinted that the clinical trials which provided the evidentiary basis for the guidelines, and for the use of statins in primary prevention in general, was more flawed than is widely appreciated.  The latest Cochrane Collaboration review of this data acknowledged multiple, important flaws affecting most of the studies.  Our more detailed review of the three studies held out as the purest found additional flaws.  Many of these flaws seemed likely to bias the studies towards exaggerating the efficacy and/or minimizing the harms of statins in primary prevention.  Since all these trials were funded, and presumably influenced by pharmaceutical companies that make statins, these flaws seem to be examples of manipulation of the clinical evidence.  Rather than being the result of simple mistakes, or inevitable trade-offs, they seem to be study features intended the support the vested interests of the study sponsors.

It is not clear why the Cochrane review did not temper its conclusions based on the flaws in the studies, and particularly by the possibility that these flaws represented study manipulation.  

The multiple flaws, possibly due to study manipulation, in the clinical evidence about statins in primary prevention suggest that we should be extremely skeptical about whether the benefits of such treatment outweighs its harms, and hence about whether the recommendations in the latest guidelines to give statins to all patients predicted (but perhaps not accurately) to be at even slightly elevated risk are warranted.

The flaws in multiple large studies of a very common clinical problem, and their effects on systematic reviews and clinical practice guidelines suggest that suppression and manipulation of research are rife in medicine and health care, presumably fueled by the pervasive web of conflicts of interest that spans health care.  We need extreme skepticism about the integrity of clinical research, especially research sponsored by those whose products and services are being studied, and who thus have vested interests in the research turning out to make their products and services look good.  

The good news is that we may not have to look too far to find ways to improve the trustworthiness of guidelines and the soundness of medical decision making.  Implementation of the Institute of Medicine's recommendations on reducing conflicts of interest (look here), and developing trustworthy guidelines (look here) might lead to the development of sound guidelines in the future.  

Furthermore, while endless discoveries of manipulated and suppressed research may have lead some evidence-based medicine advocates to despair, our latest exercise suggests that the principles of evidence-based medicine, unflinchingly applied, could really do good.  Review of the three statin studies above based on standard principles of critical review readily spotted the multiple signs of manipulation.  The problem with the Cochrane review was not that it missed these signs.  Rather, the reviewers for some reason noted most of them, but then did not react.  If systematic reviews were done with sufficient skepticism about the possibility of manipulation of clinical research, and were willing to call out when the emperor seemed short on fabric, then a lot of mischief could be avoided.  


References

1.  Lenzer J. Majority of panelists on controversial new cholesterol guideline have current or recent ties to drug manufacturers.  Brit Med J 2013.  Link here. 

2.  Taylor FC, Huffman M, Shah E. Statin therapy for primary prevention of cardiovascular disease.  JAMA 2013.  Link here.  

3.  Robinson JG.  Accumulating evidence for statins in primary prevention.  JAMA 2013;  Link here.

4.    Downs  JR, Clearfield  M, Weis  S,  et al; for the AFCAPS/TexCAPS Research Group.  Primary prevention of acute coronary events with lovastatin in men and women with average cholesterol levels: results of AFCAPS/TexCAPS. JAMA. 1998;279(20):1615-1622. Link here. 
5.  Nakamura  H, Arakawa  K, Itakura  H,  et al; MEGA Study Group.  Primary prevention of cardiovascular disease with pravastatin in Japan (MEGA study): a prospective randomised controlled trial. Lancet. 2006;368(9542):1155-1163. Link here.

6.  Ridker  PM, Danielson  E, Fonseca  FA,  et al; JUPITER Study Group.  Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein. N Engl J Med. 2008;359(21):2195-2207. Link here. 

7.  Taylor F,  Huffman MD, Macedo AF et al.  Statins for the prevention of cardiovascular disease. Cochrane Library 2013.  Link here.

8. Mueller PS, Montori VM, Bassler D et al.  Ethical issues in stopping randomized trials early because of apparent benefit.  Ann Intern Med 2007; 146: 878-881.  Link here.  

9.  Management of Elevated Cholesterol in the Primary Prevention Group of the Adult Japanese (MEGA) Study Group.  Design and baseline characteristics of a study of primary prevention of coronary events with pravastatin among Japanese with mildly elevated cholesterol levels.  Circ J  2004; 68: 860-867.  Link here.