Our columnist explores why many “experts” base their conclusions on a singular experience—and shouldn’t.

I met someone at a party the other day, and he asked me about a problem that his horse had. Before I could respond, the partygoer standing next to me (drink in hand) jumped in and told the person what the diagnosis was, what he needed to do and what the horse’s problem was going to be in recovery.

The slightly inebriated “expert” then looked to me for affirmation. Now that the complicated things had been explained, I evidently could fill in the details.

My response was that perhaps his friend was right, but the problem in the explanation was obvious. He waited to hear what the problem was while the “expert” left to find more wine.

Now that I had a clear field, I explained: “You’re N=1.”

I got puzzled looks all around. N is the number of patients in your experiment. So my friend who asked the question and the local “expert” most likely have one horse in their knowledge base on this particular problem. Most veterinarians who practice sport horse medicine will have hundreds, if not thousands, of N in their data or knowledge base. So whom do you want to believe?

Oddly enough, the answer is complicated by human nature.

You want to believe your experiences, or those of your close associates.

A famous historian pointed out that we’re simply the sum of our previous experiences, so if it hasn’t happened to us it doesn’t exist in our worldview. This is the argument for a liberal (not leftist, just widespread) education. It teaches us that many things have happened outside of our experiences, and it teaches us a rational way to evaluate that information.

The other complicating factor is that horse sports are filled with rugged individualists who consider riding and caring for horses an art form. Some are loath to apply science to the individual truths that they can only find on the back of a horse.

I agree that many elements of riding and understanding the horse’s mind are, and will always be, an art form. Science can contribute to these artistic aspects of horsemanship, but modern equine medicine is definitively a science.

So horse sports, like many fields of human endeavor, have their fair share of self-taught “experts” who are happy to share their N=1 of wisdom but don’t really understand many of the details.

Just The Evidence

Veterinary medicine also has its fair share of rugged individualists. We have to struggle to remain objective in what’s very much a relationship-based industry. One way that this is done is called evidence-based medicine. This simply says, “when you make a statement or a treatment regime, try to base it on science instead of what you think the answer is.”

For example, the other day a referring veterinarian asked how many cases of a new regenerative therapy procedure we’d done. My impression was that we’d done several hundred. I queried our database for the facts and found, surprisingly, that we’d done slightly more than 100.

Impressions and single-case stories will quickly lead you down the primrose path, and while they sound great, they should be avoided. If you’re actually practicing evidenced-based medicine you’ll not only stay current on the veterinary articles in your area of interest, but will also research a question when it’s asked of you.

Sometimes the results will surprise you. But no matter; what you’re attempting to avoid is the N=1 syndrome.

In this Google age of iPads and “Crackberries,” many clients come well researched and prepared from the Internet. I don’t mind this and actually encourage them to research, but I often have to patiently explain that Googling and peer-reviewed scientific literature are not the same thing.

Peer-reviewed means a paper has been reviewed by a veterinarian or a group. They’ve commented on its review of the current literature, its scientific merit and whether the authors considered all of the options available to them.

If you Google “navicular disease treatment,” you’ll find numerous hits that list “convincing testimonials of the use of gallium.”

Sound good? The only problem is that there’s little scientific basis for gallium, and claims of success in treatment are questionable! But it looks good on Google.

So be careful where and how you research these questions, and always ask your veterinarian about whether there’s any new research or peer-reviewed literature on the subject. Sometimes you have to prove your point via the literature.

I’ve had a good-natured debate with a local colleague about the treatment of “kissing spines” in horses’ backs for some years. He believed one treatment worked better, and I believed another worked better. But all we each had was our opinion.

That’s not good enough for evidence-based medicine. So our clinic went through all of our cases using our database and drafted a paper using multiple treatments for backs. We got the paper back with the reviewers’ statements that we’d used too many multiple treatments, and it was confusing in its complexity. They were right.

This year we rewrote the paper, keeping it much simpler and separating the treatments out. We talked about the previous work we had done on X-raying equine backs as well. The paper was accepted and will be published and presented at this year’s American Association Of Equine Practitioners meeting.

So next time we have a discussion about my opinion on treating backs, I can reference my cases and our peer-reviewed paper, not just my opinion. N=279.

The Power Of N

Even in the most controversial examination in equine veterinary medicine, the pre-purchase exam, objectivity has arrived.

The pre-purchase is the most complex and detailed exam a horse ever receives. The veterinarian must assess the soundness status, the neurological status as well as heart, lungs and eyes. The veterinarian must delve into history, treatment, behavior and previous treatments.

Then the horse undergoes a detailed musculoskeletal examination and usually a radiographic examination as well. All parts of this exam are debated on a regular basis. What do these changes on the radiograph mean? Will they have significance later?

The experienced veterinarian will use the ever-increasing power of N to help determine what the findings mean to an individual buyer. Does a small fragment in the coffin joint have significance to the horse in the future? Your veterinarian can consult the veterinary literature. He can tell you of his last experiences with such cases. But be sure to ask what his N is in this case.

If he works in a group practice he may well have the answer at his fingertips. He’ll often have a diagnosis database that he can mine for similar cases and outcomes.

We have a PPE database that we’ve followed for some years now, and, interestingly enough, it tracks either a little above or a little below 50 percent. So about one out of every two horses that presents for a PPE will have a positive outcome and a new owner.

Some trainers complain that they’ve had several horses fail. Either that means the horses were poorly prepared or lame, or he just had bad luck with a close nail or poor X-rays.

The other day, I asked a trainer what her experiences were in bringing a horse in for PPE. She said, “Great!” She’d brought four upper-level hunter/jumpers in, and all four passed. She obviously was bringing in sound, well-prepared horses that will go through the examination easily.

Occasionally, there’s a surprise—perhaps a mild heart murmur, an oddity in the eye examination. The buyer and particularly the seller and veterinarian must work together to answer the question as expeditiously as possible.

As a wise man once said, “A man’s character shows its flaws or strength, not when things are easy but when they are difficult.”

This is the difficult time of the PPE, and the seller must help answer or resolve the questions as best he can.

So challenges in the PPE have been going on for some time, what’s new? One of the areas of change in the PPE and lameness practice has been gait analysis systems that can help the experienced veterinarian document, locate and judge the magnitude of the lameness.

These systems can also compare a horse’s gaits from time to time. So when a veterinarian encounters a subtle lameness on the PPE, he can now document, show a degree change from a block or treatment and compare it to the next time when the horse returns for a recheck.

This system changes the lameness evaluation: A horse now goes home with a definitive diagnosis that another veterinarian can follow up on rather than a nebulous diagnosis.

Gait analysis systems have been around for years but have finally evolved to where they’re small enough, quick enough to put on, and accurate enough to be of major value.

They help the experienced lameness veterinarian objectify what can otherwise be a somewhat subjective exam. They also help veterinarians evaluate a horse’s stride for training purposes. You can expect to see them used increasingly in the future. They’re another way to increase the power of N and move away from the N=1 syndrome.

So the “expert” with the drink in hand has returned. The friend with the question is now fortified with more knowledge and a more discerning eye as to where the information is coming from and the scope of the N sample.

The “expert” is shocked to find his dissertation picked apart by the friend and beats a hasty retreat back to the bar when asked that devastating question, “How many of these have you seen?”

A. Kent Allen received his veterinary degree from the University of Missouri in 1979. His practice currently focuses on top-level sports medicine, lameness and diagnostic imaging at Virginia Equine Imaging in Middleburg, Va., and he’s certified in equine locomotor pathology. He’s chairman of the U.S. Equestrian Federation Veterinary and Drug and Medications Committees and serves on the Board of Directors. He’s the contact veterinarian for the Fédération Equestre Internationale and the USEF in the United States and answers medication questions for veterinarians and competitors around the nation and the world. He’s also the vice chairman of the FEI Veterinary Committee. He began contributing to Between Rounds in 2009.