Scott Miller on Why Most Therapists Are Just Average (and How We Can Improve)

Scott Miller: Sure. As old-fashioned as it sounds, I’m interested in the truth—what it is that really matters in the effectiveness of treatment. Early on in my career, I learned and promoted and helped develop a very specific model of treatment, solution-focused therapy. We had some researchers come in near the end of my tenure at the Family Therapy Center in Milwaukee who found that, while what we were doing was effective, it wasn’t any more effective than anything else. Now, for somebody who had been running around claiming that doing solution-focused work would make you more effective in a shorter period of time, that was a huge shock.

It was at that point that I started to cast about looking for an alternate explanation for the findings, which concluded that virtually everything clinicians did, however it was named, seemed to work despite the differences. That led back to the Common Factors—the theory that there are components shared by the various psychotherapy methodologies and that those shared components account more for positive therapy outcomes than any components that are unique to an approach. It was something that one of my college professors, Mike Lambert, had talked about, but that I had dismissed as not very sexy or interesting. I thought, how could that possibly be true?

It was at that time that I ran into a couple of people that I worked with for some time, Mark Hubble and Barry Duncan, and we had written several books about this. If you read Escape from Babel, which we coauthored, the argument wasn’t that Common Factors were a way of doing therapy, but rather a frame for people—therapists speaking different languages—to share and meet with each other. They were a common ground.

But by 1999, it was very clear to me that Common Factors were being turned into a model by folks, including members of our own team, and viewed as a way to do therapy. But you can’t do a Common Factors model of therapy—it’s illogical. The Common Factors are based on all models. This caused a large amount of consternation and difficulty, numerous discussions, and eventually I suggested to the team that the way therapists work didn’t make much of a difference.

What was critical was whether it worked with a particular client and a particular therapist at a particular time. Mike Lambert was already moving in this direction and said, “Let’s just measure them. Let’s find out. Who cares what model you use? Let’s make sure that the client is engaged by it and that it’s helping them.” So we began measuring, and what became clear very quickly was that some therapists were better at it than others.

So, since about 2004, Mark Hubble and others at the International Center for Clinical Excellence (ICCE) have been researching the practice patterns of top performing therapists. It’s not that I don’t believe, and in fact know, that the Common Factors are what accounts for effective psychotherapy. It’s just that an explanation is not the same as a strategy for effecting change. And the Common Factors can never be used as such. All models are equivalent. Pick one that appeals to you and your client.

SM: Well, by definition, you can’t do a Common Factors model because then it’s a specific factor. I’m not saying the Common Factors don’t matter—what I’m saying is that they are a therapeutic dead end. They will not help you do therapy. You still have to have a method for doing the therapy, and the Common Factors are not a method. Why? All treatment approaches return equal efficacy when the data is aggregated and methods compared in a randomized controlled trial. So you still need some kind of way to operationalize the Common Factors.

Since we have 400 or so different models of therapy, why invent a new one? It seems to be because in our field, each person has to have it their own way. The promise of a new model is a siren song in our profession that we have a hard time not turning our ship towards. What I say is, pick one of the 400 that appeals to you and then measure and see: Does your client like it, too? If not, then it’s time for you to change, not your client.

SM: The first one is knowing your baseline. You can’t get any better at an activity until you actually know how good you are at it now. We therapists think we know, but it turns out that data indicates that we generally, as a group, inflate our effectiveness by as much as 65%. So you really have to know just how effective you are in the aggregate. That means you’re going to have to use some kind of outcome tool to measure the effectiveness of your work with clients over time.

The second step is to get deliberate feedback. So once you know how effective you are, then it’s time to get some coaching, get some feedback, and you can do that in two ways. Number one, you can use the very same measures that you used to determine your effectiveness to get feedback from your clients on a case-by-case basis. Meaning that you can actually see when you’re helping and when you’re not, and use that to alter the course of the services provided to that individual client.

The second kind of feedback to get is from somebody whose work you admire, who has a slightly broader skill base than you do, and have them look at your work and comment specifically about those particular cases where your work falls short. In other words, you begin to look for patterns in your data about when it is you’re not particularly helpful to people, and seek out somebody who can provide you with coaching. It’s like in golf, once you know what your handicap is you can hire a coach who can look at your game and make fine tweaks. It’s not about revamping your whole style, or about learning an entirely new method of treatment, but pushing your skills and abilities to the next level of performance.

The third piece is deliberate practice. The key word in that expression is “deliberate.” All of us practice. We go to work. But it turns out the number of hours spent on a job is not a good predictor. In fact, it’s a poor predictor of treatment effectiveness. So what you have to do is identify the edge of your current realm of reliable performance. In other words, where’s the next spot where you don’t do your work quite as well? And then develop a plan, acquire the skills, practice those skills and then put them into place. Then measure again to see, have you made any improvement?

I can’t take credit for coming up with these three steps. We’ve simply borrowed them lock, stock, and barrel from the performance literature, and in particular, Anders Ericsson’s work, which has been applied in fields like the training of pilots, chess masters, computer programmers, surgeons, etc. If we have any sort of claim to fame, it’s that we’ve begun applying these to psychotherapy for the first time.

SM: Well Ericsson notes that the search for genetic factors responsible for the performance of eminent individuals has been surprisingly unsuccessful. In sports we often think, “Oh, there must be some genetic component involved here,” or “he just has the gift of music.” But it turns out that virtually everyone that researchers looked at where the “gift” is implied, even with Mozart—he had been playing the piano for 17 years before he wrote anything that was unique, which happened at about age 21. He’d been playing since he was 4. His father had been doing music scales with him since he was in the crib. So once you remove the practice component, you just don’t find any evidence for genetic factors—with very few exceptions.

For example, in boxing it appears that people with a slightly longer reach have a slight advantage. But we also know that if baseball pitchers don’t start pitching at a particular age, their arms will not make the adjustment required to throw the ball as fast and accurately as professional pitchers do.

There was another study that looked at social skills. You often will hear, in addition to the genetic claims, that, “Good therapists just have great social skills.” Well, they’ve measured that. It turns out not to be the case, and the reason is that these kinds of ideas are too high or general a level of abstraction. The real difference between the best and the rest is that they possess more deep, domain-specific knowledge. They have a highly contextualized knowledge base that is much thicker than average performers, and much more accessible to them and responsive to contextual clues.

SM: Well the classic one—and I say it to make fun of it—is suicide contracting. Or the suicide prevention interview. Somebody comes in and says, “I’m going to commit suicide.” And we respond with, “Do you have a plan? Have you ever attempted this before?” Blah, blah, blah. That’s decontextualized knowledge. You could ask those questions to a stick.

What a top performer does is ask those questions very differently, nuanced by the client’s presentation, in ways that the rest of us can’t see. Because of their more complex and well-organized knowledge, they can actually see patterns in what clients present that the rest of us would miss and respond to in a much more generic fashion. Is this making sense?

SM: So the real question is how to help clinicians develop that highly contextualized knowledge. Because once you have it, not only can you retrieve that knowledge at the appropriate moment, but it turns out you can make unique combinations and use them in novel ways that would never occur to the rest of us, or would only occur to the rest of us by chance.

SM: We know that following a treatment manual doesn’t result in better outcomes and it doesn’t decrease variability among clinicians using the same manual. So you still get a spread of outcomes, even when everybody is doing the same treatment.

At the same time, I think it’s critical that therapists learn a way of working, and, in the beginning at least, they hew to that approach. Why? Well, if you begin to introduce variation in your performance early on, you will not have the same ability to extend your performance in the future.

Let me give you an example. The first time I had a guitar lesson, I was taking classical guitar with this really interesting teacher. We spent the entire first lesson on how he wanted me to hold the neck of the guitar with my left hand—and I’m right handed. He said, “If you try to vary your hand grip from the outset, you’ll never have the same reach and ability to vary reliably when you need to in the future. So start with a common foundation, and then when we need to introduce variations later, we will.” My sense is that therapists instead begin in a highly complex, nuanced way and introduce variations into their style randomly and without much thought.

SM: And to vary your work in ways that allow you to measure the impact of your variation against what you usually do. This is the key. Otherwise, what you have is a bag of tricks. You can do them all, but there’s no cohesiveness to it, and you can’t explain why you vary at certain times rather than others.

SM: Absolutely not. In fact, I would suggest grabbing a manual and going to a place where they are teaching a specific approach that will allow you to practice and also watch others in a two-way mirror. Once you have that foundation down, you can introduce your own variations.

SM: No, it doesn’t. We know that not only in therapy, but in a variety of activities. If you think about it, you’ll understand why. While you’re doing your work, you don’t have time enough to correct your mistakes thoughtfully. So what we found, which I think is quite shocking, is that the difference between the best and the rest is what they do before they meet a client and after they’ve met them, not what they’re doing when they’re with them.  Let me give you an example from a field that is similar—figure skating. If you watch a championship figure skater perform a gold medal winning performance, you can describe what they did, but it won’t tell you how to do it yourself. Do you follow me?

SM: In order to be able to accomplish that performance, that figure skater must do something before they go on the ice, and after they leave the ice. It’s that time that leads to superior performance. You can go out and try to turn triple axels during the performances as much as you want. That experience will not make you better. You have to plan, practice, perform, and then reflect. Most of us don’t see all of the effort that goes into that great performance. We just appreciate how good it is.

SM: Well, number one, we’re all already making these mistakes. And the ones that I’m referring to are generally small and not fatal. So your performance doesn’t improve by isolating gross mistakes, or gross skills. Your performance improves when your usual skills begin to break down—meaning they don’t deliver—and remembering those, thinking about them after the session, and making a plan for what to do instead. That’s where improvement takes place.

When I hear people mention this kind of objection, I think they’re thinking that the errors are far grosser than what I’m talking about. Once therapists assess their baseline, most are going to find out—to their, perhaps, surprise—that they’re average in terms of their outcome, or slightly less than average. So if we’re average, then it’s not about bringing your game up to the average level. It’s about extending it to the next. That requires a focus on small process errors.

Let me give you another example. We have a pianist come and perform at one of our conferences. She is eight years old and she is really unbelievably able as a concert pianist. She plays a very difficult piece. I ask her if she made any mistakes. She says, “Of course, I made a lot.” I tell her I didn’t hear any, to which she says, “Well, that’s because you’re no good at this.”

I then say, “What do you mean? And what do you do about your mistakes?”

She says, “Look. I made lots of mistakes, but you cannot get better at playing the piano while you’re performing.” This is an 8-year-old.

I say, “So what do you do?”

She says, “Well, I hear these small errors. I remember them. My coach in the audience remembers them, and then that’s what I isolate for periods of practice between performances.”

SM: Most of us are average.

SM: And 50% of us are below average, right? So very few people do it, and this is the real mystery of expertise and excellence. Why do some go this extra mile? There’s no financial pay-off. I think this will change in the future, but at the present time, you don’t get paid one dime more if you’re average, crappy, or really good. The fees are set by the service provided.

SM: I think that we’re seeing movement in that direction. I think that our field will become like other fields, where outcome of the process is what leads to payment, rather than the delivery of it.

SM: That’s a component of practicing. A graduate student that I’ve been working with, Darryl Chow, who just finished his PhD at University of Perth in Australia, did his dissertation on this topic and found that the best performers spend significantly more time reading books and articles. We also know that the best performers spend more time reviewing basic therapeutic texts.

Therapists are often in search of the variation from their performance that will allow them to reach an individual client they’re struggling with. Top performers not only do that, but they’re also constantly going back to basics to make sure they’ve provided those. They spend time reading basic books that may be hugely boring but are nonetheless really helpful. Gerard Eagin’s The Skilled Helper, Corey Hammond’s book on therapeutic communication—these basic texts that remind us of things that we often forget in the flurry of cases we see every week.

SM: We don’t know about workshops. I’m cynical about them, simply because they’re not set up in a way that respects any principles of the last 30 years of research on human learning. Six hours, chosen by the person who needs the continuing education, and there’s no testing of skills, acquisition of skills, no awareness of particular deficits in practice. Greg Neimeyer has done a fair bit of research on this and he finds no evidence that our current CE standards lead to improved performance. None.

SM: It depends, but I like the sound of it. Not a scrimmage, where you do a whole game, but rather drilling people in very specific small skill sets again and again. That aligns with the principles of Ericsson’s researchers.

If you’re an experienced professional, your motivation for going to a CE event can be really varied. I know for me, I’m often just grateful to have a day off and hang out with friends. The particular content of the workshop, I’m ashamed to admit, is less important. The incentives are just all wrong.

SM: I don’t think our field incentivizes that kind of stuff. In fact, you can be punished.

SM: Drop-out can be both a good and a bad thing. For example, our current system incentivizes therapists to have a butt in the seat every available, billable hour. What that means is that therapists may be incentivized—we have some data about this, too—to keep clients, whether they are changing or not. That’s what I mean when I say that the incentives are all screwed up. There are, every once in a while, motivated people like yourself who say, “Wait a second. There has to be something beyond this.” But that requires a degree of reflection that may be difficult for most of us, especially if we are well defended. For these folks, people drop out because they are in denial about their own problems, not because of anything they, themselves, might be doing.

You put those things together and it can be a fatal combination. We need to take a step back as payers for services and as consumers of services and think about the incentives in our current system. I know this sounds terribly economic, but I think it’s important for our field.

SM: Yes it would. Especially in the beginning, when you have identified a particular area or weakness in your skill set that you may need some help with. In essence, you’re spending more time swimming in it while reflecting, which is the key part.

SM: Well there’s the stop-start strategies that Darryl Chow has been talking about. And Chris Hall is doing a study at UNC that we’re involved with, where therapists will watch short segments of a video and then they have to respond in the moment in a way that is maximally empathic, collaborative, and non-distancing. So they’re training therapists to develop a certain degree of proficiency with fairly straightforward clients.

Then you begin to vary the emotional context, or the physical context, in which the service is delivered. So now the client’s not just saying, “Hey, I feel sad.” They’re threatening to drop out or to commit suicide. More difficult and challenging things. And then simply spending time outside of the office planning and discussing individual particular cases with peers or consultants is another strategy.

In Darryl Chow’s research, which I think is the most exciting stuff, he found that within the first eight years of practice, therapists with the best outcomes spend approximately seven times more hours than the bottom two-thirds of clinicians engaged in these kinds of activities. Seven times.

SM: The good news is, now that we know this, we can start this process earlier. The bad news is, if you’ve been at this for awhile, it becomes impossible to catch up with the best. We just age out. We can’t do it. The key to this is really starting early and investing a little bit at a time. It’s sort of like how you’re advised to save for your retirement. Not in the last five years. Not in the first five years, but a little bit every year.

SM: Well, in essence, we’re violating John Wooden’s primary rule, which is, we are allowing students to scrimmage before they drill. And I have to tell you, all students want to scrimmage, but what you need to do more of, before and during, is drilling. The kind of drilling that I think your colleague was talking about. Or you go back to, “Here’s how we hold the guitar.” And we play very simple songs and then we begin varying the drill with greater degrees of complexity once easier tasks are managed.

SM: I’d want to see that kind of mastery. Let me give you an example. Do you want the pilot to be proficient at flying in fair weather, as demonstrated on the simulator, before they fly a plane?

SM: You want them to be prepared for all the complications: “Wait a minute, it’s raining,” “Wait a minute, you’ve got problems with your rudder.” These are complex skills and, yes, we can teach people to manage them as one-offs, but then they never integrate it into a coherent package that makes it easier to retrieve from memory later on when they need that skill. If it’s viewed as a one-off—“With the anxiety client, I did this”—it’s not integrated into an organized structure for retrieval later on.

SM: Or, better yet, somebody who has measured results, like yourself. All I need is an average pilot. I don’t need the best pilot in the world, because most of the time there’s not huge challenges. If you can document your results, and if you’re checking in with me, we’re going to catch most of the errors anyway. And then I want a therapist who has a professional development plan, that’s working on the aggregation of small improvements over a long period of time.

SM: I don’t buy it. Personally, I just don’t see that stuff and I would offer a very different explanation for it. Let me give you an example.

We know that each time there is a deterioration in scores, the probability of client drop-out goes up, whether or not the therapist thinks that it’s a good sign that the client is “getting in touch with reality and finally admitting their issues,” or had inflated how they really were doing for the first visit. So the key task here is not to say, “There must be another measure,” but to figure out what skills are required for me to get a higher score.

SM: Now you see why I think our field is forever chasing its tail. Because instead of becoming fully connected to our performance, we are constantly looking for the trick that will make us great. It’s like a singer looking for the song that will make them famous rather than learning how to sing. We’re forever going to workshops, and the level of the workshops are often so basic even when they’ve claimed to be advanced. The truth is, you can’t do an advanced workshop on psychotherapy for 100 people. You can’t do it. The content is too abstract and too general. You need to see a clinician’s performance and fine-tune it. So therapists go around and around, constantly picking up these techniques that they use in an unreliable fashion, and their outcomes don’t improve, but their confidence does.

SM: I think that once you’ve achieved a level of proficiency, the only hope for improvement is to get feedback on your specific deficits. And yours will be different from mine.

SM: Not alone—with an expert eye reviewing small segments. Otherwise the flood of information from video will have you second-guessing yourself, which can actually interrupt the way you work in an unhelpful way.

SM: I’m not averse to it, but I think it’s a little bit like a GPS—it can correct your moves in the moment, but you become GPS-dependent and you don’t learn the territory. What’s required in learning is reflection. If you don’t reflect, you can’t learn. As my uncle used to say, “You got to study that thang.”

I actually had great opportunities with live supervision when I was at the Family Therapy Center and got corrected in the moment by two really masterful clinicians. But I also think that what really made a difference was sitting behind a mirror, without any financial worries, watching endless hours of psychotherapy being done, and then talking about it afterwards. “This was said. What could you have said? How come we said this? What do you need to do?” It was a heavenly experience and as a result, I came away with a very highly nuanced and contextualized way of delivering that particular model.

And today, when I’m doing my Scott Miller way of working and I notice that a particular client wasn’t engaged or interested at a particular moment, I think, “What could I have said differently?” It’s at that small micro level that improved outcome is likely to be found. As opposed to just gross generic level.

People go to workshops and say, “I’ve had some traumatized clients. Maybe I’ll learn that EMDR thing.”

“Really?” I think. “Do you know how effective you are in working with these clients already?”

“No, I don’t.”

“What makes you think you need to do EMDR?”

“Well, it just seems so interesting.”

And I think, “Oh, you’re doomed.” Not that there’s anything wrong with EMDR, but I have to tell you, I watched Francine Shapiro do it and it looks a lot different than some other people I’ve seen doing it.

SM: It’s looking for a trick rather than thinking through, what else could I have said? What else could I have done that I already know how to do? Or getting a little bit of tweaking from a trusted mentor.

SM: Yes. I think that there are some very real barriers that we need to address, but yes, I do.

SM: I like this stuff. I’m fascinated by it and I’m very hopeful about the direction we’re going research-wise, so thank you for giving me the opportunity.