Twitter and Prana

NIR has been asked on a few occasions why we do not utilize the social media potential offered by Twitter. The current contretemps around the Prana results in Huntington’s beautifully illustrate why we do not: Twitter is an effective venue for social media, true, but it also appears to be a direct route for communications that bypass the frontal lobes.

The Twitter micro-flame war that broke out between Prana’s scientific founder Rudy Tanzi, and Street.com‘s Adam Feuerstein after the release of the Phase II results for PBT2, illustrates this. Feuerstein has published the Twitter exchange on his Street.com thread, from which we quote a tweet from each:

Tanzi: <<adamfeuerstein seems set on seeing PBT2 fail in his biased blogs. PBT2 SIGNIFICANTLY improved cognition in pre-specified HD patients!>>

Feuerstein: <<RudyTanzi please save the spin for your analysts and gullible investors. PBT2 failed HD. Disgraceful you’d argue otherwise.>>

Both of these public statements are incorrect, both made by people who should know better. Tanzi’s statement regarding the improvement in “cognition” overstates the case: When it comes to the gold standard of p=.05, only one of the cognition measures utilized reached that level, and only one other measure (of functional capacity, not cognition) reached the lesser standard of a pronounced trend (Prana did not cite the p value, p=.10 is commonly utilized, p=.20 less often). We have previously published our opinion that the pattern of results from the trial suggest a preliminary signal–but a trial this size and duration proves nothing.

Feuerstein’s statement that “PBT2 failed HD” is also untrue. Meeting one (or with a lesser statistical criterion, two) endpoints on the efficacy testing is a mixed result. There is no universally accepted standard for  Phase II that decrees how many, or which, endpoints have to be met, at what p value, to declare success or failure. His willingness to make this a binary event that has been outright failed is just as arbitrary as Tanzi’s declaration of success.

It is understandable that both Feuerstein and Tanzi ended up on polarized ends of this Twitter fest. Tanzi’s scientific legacy is very much intertwined with the mechanism being tested here, regardless of his financial stake in Prana. Feuerstein has made his reputation partly on his willingness to confront pharma companies on their all-too-frequent willingness to cross the boundary from spin to deception.

We have to acknowledge that we have bias that could creep into our analysis of the situation, albeit on both sides: While lacking a financial stake in Prana, we very much want to see progress in addressing neurodegeneration. NIR also has its own history of challenging the practices of data-fracking and deceptive spin, and it feels a bit disorienting to be questioning this challenge.

In this situation, some restraint on both sides of the argument would seem appropriate, albeit somewhat too late. As we discussed in an earlier post, a trial this size rarely, if ever, proves anything.  We believe that there is reason to believe that there is a signal here, but one can not reasonably assume that it will be replicated and broadened across outcome measures in a larger, longer trial. By the traditional rubric of clinical trial progression, PBT2 should be going into PhIIb, to see if this signal emerges over a twelve or eighteen month treatment period, and a higher dose would be worth consideration.

Going into Phase III at this point is not a decision predicated upon science, it reflects the pressure to perform that is endemic to the pharma industry, impacting companies both large and small (e.g. Merck has gone into Phase III for Alzheimer’s with a BACE inhibitor that has never had efficacy testing done, the ‘Phase II’ component involved safety and biomarker assessment). NIR has previously written that “Phase IIb is not optional.” That continues to be the case, and making exceptions based on economics is a very high-risk bet. Saying that a product is ready for Phase III does not necessarily make it so. Saying anything on Twitter guarantees nothing more than brevity, and constitutes a threat to both credibility and civility.

This entry was posted in Big Pharma, BioFollies, Biotech, Muddled Media and tagged , , , . Bookmark the permalink.

One Response to Twitter and Prana

  1. Michael Gold, MD says:

    It seems some of the principles of the scientific method have been forgotten along the way.
    The parties to this debate seem to have forgotten that the scientific method, specially when it uses frequentist statistics, means that no single experiment can prove a hypothesis whereas a single study can refute or falsify it. The data from this study do not falsify the hypothesis to the point where we would all be comfortable abandoning it and at the same time do not provide consistent evidence in support of the hypothesis.

    We believe that the theory of natural selectionis correct because data from a vast number of independent observations and experiments have been consisten with predictions of the hypothesis and because killer experiments designed to falsify the hypothesis have failed.

    Debates about the meaning of a p-value are meaningless in this context and illustrate how we continue to torture exploratory studies beyond their capacity to inform the clinical research community. Is it any wonder we are losing credibility with patients?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s