Predictive Models in Biomedical Science

In an attempt to justify funding for their research, scientists often say the following: “We need to study chimpanzees (or monkeys) because we are both primates.” To the first approximation there is truth in the claim. But does mere commonality justify expecting another species to predict human response? See if the following make sense.


Our research on mice should be funded because humans and mice are both mammals.


Our research on fish should be funded because humans and fish are both vertebrates.


Our research on stars should be funded because we are both composed of quarks.


From Animal Models in Light of Evolution:


Earlier in this book, when discussing the theoretical motivations behind predictive animal modeling, we drew attention to two important claims made by the modelers in defense of the predictive prospects of their science. The first claim was that at the basal levels in the biological hierarchy of organization (e.g., the biochemical, intracellular and cellular levels) there were profound similarities relevant to the use of animals as predictive models (there are obviously similarities between mice and men, say, that are not relevant to the prediction question—humans and mice are both composed of atoms consisting of protons, neutrons and electron; protons and neutrons are made of more basic things such as quarks, and so on).


Perhaps when the focus is on the relevant basal similarities, from a predictive point of view, mice and men are the same animal dressed up differently. This claim, we have argued, both on theoretical grounds (evolutionary biology and developmental genetics) and empirical grounds, is false. For instance, as we have seen in the geneticists’ response to the ancient metaphysical puzzle of unity in diversity, there are enormous genetic similarities between mice and men, but the devil’s differences lie in the details of the regulation of genes in the pathways, circuits and networks to which they belong. It is here that hopes for prediction at the basal levels of the biological hierarchy are confounded.


Clearly, sharing traits does not make a model predictive for disease and drug response. The shared traits must be causally relevant for the response in question in order for the model to be relevant.


As I have said many times, prediction has a very specific meaning in science in general and in medical science in particular. Allow me here to convey what prediction does not mean:


Gives us a general idea.


Points us in the right direction.


Makes us think.


Lets us know if we are on the right track.


Occasionally getting it right.


We tested twenty-five species and one ended up giving the same results as humans therefore it was predictive.


For any given drug, an animal can be found in retrospect that mimicked human response therefore the animal model per se is predictive.


The above is like saying astrology is predictive because it gives me a general idea about how to live my life, it points me in the right direction, makes me think, and let’s me know if I am on the right track. None of that qualifies as prediction.


The reason the prediction issue is important is that society has been led to believe that animal-based research and testing does in fact predict human response to drugs and disease. Researchers themselves say: “The research we are doing is necessary for medical science to advance so society can see new drugs and other cures.” But when pressed about prediction they say: “We are doing basic science research so no one should expect clinically relevant results. Physicists and chemists do basic research and no one yells at them about not producing clinically relevant results. Basic research is research into interesting puzzles and it may or may not lead to cures.” There can be no doubt that animal-based research investigates interesting puzzles and results in many publications. But does this justify all the money that goes to such research? Consider what Boat said in 2010:


As Dorsey et al (1) point out, broader measures are needed to adequately judge return on the substantial biomedical research investment. Ultimately, biomedical research productivity must be assessed against individual and population health. (2)


I am not alone is saying animal models are not predictive. The executive director for cancer research at Merck Research Laboratory 1997: “The fundamental problem in drug discovery for cancer is that the model systems are not predictive at all” (3). Chabner and Roberts: “Fewer than 10% of new drugs entering clinical trials in the period from 1970 to 1990 achieved FDA approval for marketing, and animal models seemed unreliable in predicting clinical success . . .” (4).


Kola and Landis wrote in Nature Reviews Drug Discovery:


The major causes of attrition in the clinic in 2000 were lack of efficacy (accounting for approximately 30% of failures) and safety (toxicology and clinical safety accounting for a further approximately 30%). The lack of efficacy might be contributing more significantly to therapeutic areas in which animal models of efficacy are notoriously unpredictive (5)


Eugene C. Butcher of the Department of Pathology, Stanford University stated in Nature Reviews Drug Discovery 2005:


Although a number of underlying problems with the current paradigm have been highlighted, here we focus on two that seem particularly crucial to the rate of innovation. First, target validation (independent of an inhibitory drug) could be fruitless. Current mouse-genetics-focused methods of target validation cannot reliably predict human biology; and even if a model is predictive of human target biology, target biology cannot reliably predict drug biology. One instance of this is the failure of antagonists of the neurokinin 1 (NK1) receptor as analgesics, but a recent review reminds us, with many examples, that mice are not men. (6)


(For more on prediction and animal models see Animal Models in Light of Evolution. If you are not scientifically inclined but still want to learn more please try FAQs About the Use of Animals in Science: A handbook for the scientifically perplexed.) )


References


1. E. R. Dorsey et al., JAMA 303, 137 (Jan 13, 2010).


2. T. F. Boat, JAMA 303, 170 (Jan 13, 2010).


3. T. Gura, Science 278, 1041 (Nov 7, 1997).


4. B. A. Chabner, T. G. Roberts, Jr., Nat Rev Cancer 5, 65 (Jan, 2005).


5. I. Kola, J. Landis, Nat Rev Drug Discov 3, 711 (Aug, 2004).


6. E. C. Butcher, Nat Rev Drug Discov 4, 461 (Jun, 2005).