Concordia professor: Measuring client progress through feedback is necessary
In May 2016, 26-year-old John Chayka was named general manager of the National Hockey League’s Arizona Coyotes. Chayka’s hiring was not only surprising because of his young age, but also because he was the first analytics-driven executive to lead a hockey organization. He has never played professional hockey, and was a recent business administration graduate from Western University’s Ivey Business School.
Since the successful use of performance metrics in the early 2000s by Major League Baseball’s Oakland Athletics, many sports have followed suit, hiring stats geniuses in their 20s and 30s to manage their teams. Politics, like sports, have also embraced the use of metrics for polling and recruitment. According to clinical psychologist Tony Rousmaniere, big data could also transform mental health treatment, “if only psychologists would stop ignoring it.”
In an essay published last April in The Atlantic, Rousmaniere reminded readers that therapists don’t have instruments of measurement as other doctors do, like stethoscopes or lab tests.
Instead, therapists are the instruments themselves—they are the ones who measure how much their client’s mental health has improved. According to Adam Radomsky, a Concordia professor and the university’s research chair in anxiety and related disorders, this is a real problem in mental health care.
“There are guidelines [from the Ordre des psychologues du Québec (OPQ)] saying that you should use evidence-based approaches, but there are no evidence-based police out there to come and make sure you’re doing something that’s been shown to work,” Radomsky told The Concordian. The OPQ is the professional body responsible for licensing psychologists in the province.
What Radomsky described as “evidence-based approaches” and what Rousmaniere calls feedback-informed treatment, or FIT, in his article, are types of feedback that inform therapists about the progress their clients are making. According to Rousmaniere, “perhaps no field faces higher barriers of incorporating performance feedback than psychotherapy.”
Clients often feel vulnerable when meeting a therapist, Radomsky explained, so they might not talk openly about the state of their mental health, even if it’s deteriorating.
“Many clients are more willing to report worsening symptoms to a computer—even if they know that their therapists will see the results—than disappointing their therapist face-to-face,” Rousmaniere wrote in The Atlantic.
Radomsky said evidence-based psychological therapy can refer to two different things. The first is the use of a treatment that’s been shown to work, “that’s been studied extensively [and] it has met that threshold,” he explained. The other “is that you use evidence to track the progress of your work with clients or patients.”
The Concordia professor—who has a small number of clients in addition to teaching—added that he “absolutely would not” be able to work without client feedback.
According to Rousmaniere, nearly 50 feedback systems for therapists have been developed over the last 20 years.
Radomsky explained that many of the clients he has seen and supervised fill out one or “a very small number of questionnaires” every week or each time they come in for a therapy session.
“These are often standardized questionnaires, validated through scientific studies, so we know what they’re measuring and we know how well they measure them,” Radomsky explained. “Then we track that over time to make sure that things are moving in the [right] direction.”
One system developed by Brigham Young University researcher Michael Lambert involves a 45-question online survey conducted before each appointment. If the clients appear to be at risk, Rousmaniere explained in The Atlantic, “their therapists are sent alerts that are colour-coded for different concerns. Red for risk of dropout or deterioration, yellow for less-than-expected progress.”
Rousmaniere said his “anecdotal impression is that use of FIT today remains disappointingly low among therapists.” According to Radomsky, using an evidence-approach is “very uncommon for some, and absolutely required for others.”
It can often depend on the psychologist’s training, Radomsky said. Older approaches to psychotherapy “weren’t really subjected to scientific tests in the same way some of the newer approaches are.” One of the newer and most commonly used approaches by psychologist right now, according to Radomsky, is cognitive behaviour therapy.
“I think that those of us that have been trained in the newer approaches, [like] cognitive behaviour therapy and other similar approaches, a part of what you do, is you track the progress of your work,” he added. “I think some people would never use it, some people would always use it. I’m not sure how many people are in between.”
Educational process
Radomsky said clients should ask their therapists about their approach and whether there is evidence to show that it works. However, he admitted it can be uncomfortable to ask these questions. “If the answers seem strange or cryptic or vague, find another therapist,” he advised. “A good therapist is happy to answer these questions.”
The Concordia professor tells his clients that, “if after about eight weeks, we’re not starting to see some improvements, I might need to fire myself.”
“I refuse to be an unhelpful therapist,” Radomsky said. “It doesn’t mean that the problems will be all gone in eight weeks, but what it does mean is that we should be tracking the progress of the work.”
According to Radomsky, the biggest push towards using evidence-based approaches comes from training programs like those in Concordia’s psychology department.
“I think it’s sometimes harder for people who’ve been doing things in a particular way for a very long time to change,” Radomsky said. “I think they should, but at the moment, there isn’t a way to force them to do that.”
Five years ago, the OPQ, which could not be reached for comment before publication, started requiring practicing psychologists to take courses or attend conferences to keep their training up to date, Radomsky told The Concordian.
“All psychologists providing therapy in the province are required to show that they are continuing to learn and train on an ongoing basis,” he explained.
The research chair said he believes the increasing use of evidence-based approaches is an ongoing process, and it needs to be more common.
“What is the alternative? Opinion doesn’t cut it.”
Photo by Kirubel Mehari
