Something May Be Working At Theranos, But You Don't Know What It Is

If there is working technology at Theranos, the embattled startup that promised to revolutionize blood tests by replacing needles in veins with finger pricks at a lower price with faster speed, you wouldn't be able to tell. To explain why, let's start with a paradox: right now the company’s best defense is a study it attacked as “flawed and inaccurate” and not fit for publication.

In March, researchers from the Mount Sinai School of Medicine published a study of 60 patients in the Journal of Clinical Investigation that compared results on Theranos’ proprietary tests, done with mere finger pricks, with those done using traditional blood draws by Quest and LabCorp . The conclusion was that Theranos was off by enough to matter, particularly with cholesterol tests.

Since then, Theranos has nullified test results for thousands of patients. Now the Mount Sinai study is at least evidence that these tests gave results that were close to being right--that the company's vaunted technology, for which it has never published any evidence, might be something more than vaporware.

What technology, though? This is the Gordian knot of Theranos. The company has been so loath to make disclosures that not only is it a mystery whether its tests for things like cholesterol, blood sugar and potassium levels are accurate, but it’s also impossible to keep straight which supposed technology is used for what. Or as Geoffrey Baird, a pathology professor at the University of Washington, puts it: “I’m incredibly confused by what these people are doing. No lab is run like this.”

Look at the Mount Sinai study. The scientists involved made sure all the tests were taken by finger stick. The Wall Street Journal reported that Theranos had voided all tests run on its Edison machines in 2014 and 2015. So were the Mount Sinai results voided or nullified? No.

Theranos statement:

Earlier this year, we undertook a comprehensive review process of our lab and determined we needed to evolve our quality standards. In addition to hiring a new lab director, we nullified prior results for finger-stick tests on our proprietary technology. Results for less than .8% of the 60 patients who participated in the Journal of Clinical Evaluation study were nullified.

So Theranos did not nullify all finger-stick tests on its proprietary technology. This could be a good sign, that some of Theranos' proprietary technology works! But the WSJ also has reported that some of Theranos' finger stick tests were conducted by diluting samples and running them on standard machines. "I don’t even think that’s possible," Elizabeth Holmes, Theranos' chief executive and founder, said at the WSJ's D Live conference.

I asked Theranos’ spokeswoman if this meant some of the tests were not conducted on Theranos technology. She responded:

No, we are stating the all of these tests were nullified due to the quality standards at the lab at the time. It's important to understand our decision to do this was not because of inaccuracies of the tests but because the lab didn't meet our standards of quality that are now in place under our new lab director and team in Newark.

In other words, Theranos claims that its technology works and all the problems were the result of procedural problems at one of its labs, in Newark, Calif. So I asked: Can you tell me whether these finger stick tests were run on standard test machines made by other companies, on Theranos' proprietary machines, known as Edison, or on a mix of the two?

Theranos’ response:

What I can tell you is that Theranos has multiple proprietary platforms for processing finger-stick samples, including but not limited to the Edison.

This is what it is like to talk to Theranos about its technical specs. Last year, the Food and Drug Administration told Theranos that its nanotainers, the containers that are used to hold blood in its testing systems, were an unapproved medical device. When I asked Holmes about that before the Forbes Healthcare Summit in December, she told me there were multiple nanotainers, and, onstage, spoke about how this meant Theranos could still use its proprietary technologies. “That has just to do with the tubes,” she said. “It did not have to do with the use of our device or the use of any of our tests which we can still run in our laboratory as lab-developed tests.”

And this slipperiness is a big part of what infuriates many medical experts about Theranos. What they want is a clear explanation of what Theranos does and data to prove that it works. This does not mean disclosing every trade secret the company has. But it does mean clearly explaining what device is being tested and what the proof is that it works.

“Based on what they say, I can conclude absolutely nothing, because they can say whatever they want,” says Eleftherios P. Diamandis, Head of Clinical Biochemistry, Mount Sinai Hospital and University Health Network in Toronto. “They can say, ‘I’ve got something in the pipeline that is so good that it can put Quest in bankruptcy.’ I can say that, too. But what is it worth? What I can say for sure is that what they say and what happens are two different things. The best way to find out is that they put this new technology through the scrutiny of peer review by experts who will say whether these new technologies are good, bad or indifferent.”

Holmes is going to be presenting some information at the annual meeting of the American Association for Clinical Chemistry in August, but it still won’t be a normal scientific presentation that is peer-reviewed ahead of time. Holmes has promised to publish data in a medical journal, but so far it hasn’t happened.

And the Mount Sinai paper is one more example of things not working out the way Theranos said they would. The company said the paper was so flawed as to be invalid. Eric Schadt, the senior author of that paper, says that his team went back and forth for a long time with Theranos and the editors of the journal. In the end, they decided on a single correction–a slight change to the dates when the tests were conducted–with no other alteration of the paper’s conclusions.