How do we evaluate treatments?

I’ve been thinking a lot about the post I made earlier today on the Marshall Protocol and I decided to remove any material discussing my feelings about it, for now. I do plan to post on that (and on other things like energy healing), but I think that I need to step back first and ask the question — what are the metrics by which we can evaluate different approaches to healing in general, and approaches to healing lyme disease in particular. I need to do a much better job of elucidating the underlying facts and intuitions that lead me to judge whether a particular healing system is right for me. Also, if I am to decide what approach to take next, it would help a lot if I could compare and contrast two different approaches along dimensions that they share in common. So I’m going to start this series of posts by talking about some of the things that I think are important to evaluating something. I’m brainstorming here, and I hope some of you will add comments with your own thoughts about what helped you decide on a treatment. So here goes:

  • Frequency data (in approximate order of how much I might trust the evidence)
    • Is there a clinical trial showing that this treatment works? Is it replicated? How well designed is it (e.g., is there a control group; is it double blind; are there numerous participants or just a few; are the participants substantially similar to me or very different in ways that might matter)
    • Is there a large number of doctors and/or patients who have used this approach successfully? How is success defined?
  • Process Data
    • How does the treatment work?
    • How well is its underlying mechanism understood?
    • Is there agreement in the scientific community about its efficacy? Is there agreement among some other trustworthy community about its efficacy?
    • How well does the mechanism of the treatment match what is known about Lyme disease — is it reasonable to believe that it would be effective in treating Lyme?
  • Experience Data
    • Is there a long history of use (e.g., is this a form of medicine that has been used effectively for centuries but perhaps has not be studied in depth in the West)? Is there any evidence that it has helped individuals with Lyme disease? If the healing approach treats diseases (as opposed to symptoms), how long is its history specific to Lyme disease?
    • Personal experience — do I know something about my body or this treatment that makes me think it would (or would not) be effective?
    • Is there a doctor or other expert I trust who uses it and says it works? How much experience does that doctor have with this treatment? What evidence do they have that it works and how well do I trust that evidence? What biases do they have? What is their level of training?
    • Is there a patient (or patients) whom I know and trust who say it works? How broad is their experience with this treatment? What evidence do they have that it works and how well do I trust that evidence? What biases and/or knowledge do they have?
  • Safety
    • What are the potential short term side effects, if any?
    • What are the potential long term side effects, if any?
    • Could this make me susceptible to some other chronic condition?
  • Intuition & Emotion
    • Can I believe in it?
    • Does it feel like the right thing to do?
  • Practical Issues
    • For how long might I need to do this? My whole life or just until I’m better?
    • How long is it expected to take for this to work?
    • What are the costs in money and time? Does insurance cover this or would it be out of pocket?
    • Can I reasonably integrate this into my life
    • How else would it impact my life and am I willing to make those sacrifices (if that’s what they are?)
    • Accessibility — how hard is it to find the necessary pills, machines, or other things needed to complete the treatment
    • Can I find someone who knows how to apply it and is supportive of my using it? If not, is this something that an unskilled practitioner (myself) can effectively do?
    • Does it work with my current treatment or does it require a wholesale switch with the risks that may entail.

Please add to this list if you have ideas or just tell us what your own experience is or has been in choosing a treatment…

7 thoughts on “How do we evaluate treatments?

  1. Perhaps in addition to clinical trials in the US it may be useful to look at trials of similar substances in Europe. In general, it seems that European clinical research is largely ignored or understated, so when there are “new” studies here on drugs that have been studied for literally decades in the former Soviet bloc or western Europe, one tends to wonder whether the omission has any legitimate basis, or if it’s just skepticism of other work because it hasn’t met FDA’s and CDC’s narrow standards for admission.

    It seems that there’s been a number of both prescription and over-the-counter drugs (I’m thinking of weight loss and acne removal types of things, the kinds they advertise for a few weeks or months, and then they just disappear without a trace) that were pulled off the market after some terrible side effects started appearing, but, at the same time, there has been this incredible, recurring claim made by my (Eastern European) family members along the lines of, “This ingredient has been illegal in Europe and/or the former USSR for decades. How did they ever approve this? Didn’t they do long-term side effects studies? How is it possible that people could buy this and ruin their health when a good part of the world was pretty sure that’s what would happen?” Also, it’s interesting that there are several drugs (namely, various kinds of potent topical creams or patches; literally the same drug in the same packaging – look identical, at least) that are available by prescription only overseas, but over-the-counter here. Observations like that are at least a little worrisome, and have, to me, caused great distrust of pharmaceuticals.

    Seems that these could all be potentially useful things to consider when evaluating new treatments and medications.

  2. What has been most helpful for me in deciding which sort of complementary therapies I would pursue (for treating my Chronic Migraines) is consulting with my headache specialist. Last fall, after a series of really terrible reactions to traditional medications, I wanted to take a medication holiday and try some other things in the meantime. My headache specialist referred me to an internist with a degree in herbalism; she practices “holistic medicine”, meaning traditional and complementary. She balances the two with the individual patient’s needs in mind.

    For me, that meant an overhaul of supplements I was already taking, including specific brand and dosage recommendations. She also had me retry acupuncture and start massage therapy. Most recently, she’s added Nia dance (a very freeform type of dance meant to express emotion and find joy in movement) and visualization.

    Unfortunately, there just isn’t that much reliable data on complementary therapies. The few studies that have been done for specific conditions are usually very small in scope and inherently biased (by which I mean, not placebo-controlled or double-blind). And, at least with Chronic Migraine, there is no standard treatment protocol because every case is so different. (There are over 100 different medications used to prevent Migraines, but not a single one of them was initially developed for Migraine.)

    So, all of that said, I think your list is a great starting point for evaluating the use of particular treatments. If you can find a knowledgeable and open-minded doctor, that will go a long way in the efficacy of your treatment. Some of the treatments out there really are of the “snake oil” variety, but some are useful and effective in certain cases.

    Best of luck to you on this journey, and I look forward to reading what you discover.

    Be well,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s