Why Feedback Must Lead To Action

Jeannie Walters 0:00

We're collecting more feedback than ever before. But what happens next? In this bonus episode of Experience Action, I'm bringing together two conversations I had from the Qualtrics X4 event in Seattle. I was able to sit down with a few professionals that I think you'll really enjoy hearing. Now, one of these conversations is really focused about turning feedback into action, especially in healthcare. And then the next conversation is all about how trust, AI, and human oversight will shape what comes next. I hope you enjoy listening to these conversations just as much as I enjoyed having them.

Kerry Kuluski 0:45

I'm Kerry Kuluski. I hold a research chair in patient and family centered care at the Institute for Better Health at Trillium Health Partners. And the Institute for Better Health is an embedded research unit within a multi-site hospital in Mississauga, Ontario, Canada. I'm also a professor at the University of Toronto and focus a lot on patient experience in my research.

How They Boosted Survey Responses

Adam Gdyczynski 1:02

I'm Adam Gdyczynski, a senior corporate lead of patient management experience and planning at Trillium Health Partners. We are one of the largest hospitals across Canada. We are also building one of the largest hospitals over the next eight years. So we're not only expanding our current portfolio, but we are currently very busy on our day-to-day operations even.

Jeannie Walters 1:18

One of the things that we've been talking about here at X4 in general is really, first of all, how do we get more and better results? We all have been collecting feedback for a long time. We've got all these signals. Now we're in the age where we're kind of like, that's great. We're getting a lot of stuff. How do we make the most of it? And we also want to make sure we're reaching as many people as possible. We're not just reaching one segment at a time. You all have a pretty incredible story with this. And one of the things that I read and heard was that you had a 880% increase in survey results. Is that accurate to say? That's amazing. Congratulations. And what did that increase do for both, you know, what insights you could find and how you act on them, but then also kind of operationally. That's a lot. So how did you balance all that?

Handling Volume Without Burning Out Staff

Adam Gdyczynski 2:08

The big increase for us happened when we moved to Qualtrics just three years ago. We moved away from call center and paper-based surveys. We moved away from long 60 to 80 question surveys down to about 15 questions short form. That really helped. The transition to Qualtrics helped because we finally were able to reach people through email. We put a QR code on our epic after visit summaries. We noticed certain areas like our NICU and mental health, we weren't doing great on electronic response rates. So we were providing iPads to patients as they were walking out the door. We were able to, for the first time, look at uh not only gender-based data, gender identity data, but also background and culture data. So we were able for the first time to see of those people in our diverse community who was actually responding to us and how did that actually coincide with our census? And the numbers were actually incredibly close. So we weren't just getting from one group over the other. It was actually really close to the people who lived in our Mississauga area. And that really helps us now as we plan moving forward that we can say, hey, we are hearing from different backgrounds. This is what somebody is saying with this background, this is what somebody is saying about this background. As we continue to build our new hospital, we need to think about equitable care. We need to figure out how to work with marginalized communities and we get to see what they're actually saying.

Kerry Kuluski 3:17

So I've been doing for several decades lots of qualitative interviews with patients and families. And the number one thing that they say is, we're so tired of sharing our story because we're not confident that anything's gonna be done with it.

Jeannie Walters 3:28

Yeah.

Kerry Kuluski 3:29

So when we got that huge increase in responses, I thought, wow, this is great. We have this huge repository of data. But I'm like, oh my gosh, how are we gonna manage the volume of responses we're getting? How do we actually filter this, share it with clinical teams, knowing that they're already burnt out, but to really work with them to then action it for quality improvement. So that was a big, like it was great to have data, but we also had like a huge responsibility to do something with it. So one really great engagement strategy and kudos to Adam for actioning this, is we ask patients not only where we can do better, we ask what's working well.

Jeannie Walters 4:04

I love that.

Designing The Whole Patient Journey

Kerry Kuluski 4:05

And so the positive data is used as an engagement strategy to get clinicians and frontline staff to actually look at the data. And once they see what's going well, it gives them a boost, but then they also say, okay, well, we there must be things that we can do a bit better. And so then it opens their eyes to be a little bit the first they're aware that there's data, but then they actually go and look at what's working well, where can we improve? And we've noticed that some clinical programs and units are really starting to put their heads together to say, okay, so what can we do to actually action a change? Initially, we thought we could do something sort of universal, top-down, that everyone should do this particular action, but that top-down doesn't work. So when when just given the data to teams, we're starting to see that some of them are naturally starting to look at the data and action the data. And then others that are curious are going, oh, what's this unit our program doing? They're more likely to engage in that change behavior. But we also accept there's going to be some programs and units that just aren't really, they're just kind of ignoring the data. So it's kind of accepting that everyone's at different stages and we could really start slowly. But when it's ground up, it's really empowering for staff and I think more sustainable than doing like a top-down action.

Jeannie Walters 5:13

When you are kind of introducing this feedback and some of the things that people have told you, yeah, you could do this a little better. When you're interacting with practitioners and physicians and office staff and all the people who make things happen, how do you help them understand that the experience is more than maybe what they're thinking of it? Because I think what fascinates me about healthcare patient experience is patients don't always talk about the interaction with the physician. They talk about it was hard to make an appointment. It was, you know, all these different things. So what do you do with that feedback? How do you handle that?

The Trust Question As AI Grows

Kerry Kuluski 5:50

Yeah. Well, a big focus that we have on our research unit is integrating the disparate parts of our healthcare system because we're, you know, based in Canada and all over the world, everything is really disconnected. So it's giving us an opportunity to highlight and patients say to us, we want you to think about this whole journey, not just this moment that we've had on this clinical unit. Well, we can still learn about what happened in your last encounter. It really is a connected experience.

Jeannie Walters 6:15

And as we think about that connected, holistic experience, it raises an important question. How do we handle all of that data responsibly, especially as AI becomes a bigger part of the picture?

Assaf Keren 6:31

My name is Assaf Keren. I'm the Chief Security Officer in Qualtrics.

Jeannie Walters 6:35

One of the things we're seeing is that customers are aware that we're entering kind of this new phase of leveraging AI, leveraging data, leveraging all these things. And there's some hesitation from customers. They're saying, you know, help me understand how are we going to do this responsibly? How are you going to do this in a way that is secure? So I'm curious, what would you say to those customers? What are you saying to people about that level of security?

Trust Basics And Radical Transparency

Assaf Keren 7:03

There is a lot of wish and there is a lot of expectation for personalization and for better processes and for automation and people want to see not necessarily they don't necessarily say artificial intelligence, but they want to see themselves being served and they're excited about being served in a more personal way with the data that brands and and companies have on them, but at the same time, they are very worried for their data and the sharing of their data. And when I talk about this, I like to say it's almost binary in the sense of we have brands that we trust where we implicitly share endless amounts of data with. And there are brands that we don't trust that we share zero data with. And the flip between being a trusted brand to being an untrusted brand can be very, very quick. And the buildup from being an untrusted brand to being a trusted brand is a very long process. There are two main drivers for trust building that I think are generic across the board. And there is one that is specific to the world of AI that we're stepping into. So um the two main drivers are being built into the basics. Understanding that and I'm a security person, so I think about this from the position and point of view of security. But the trust is bigger than security. Trust about being there, it's about being reliable, it's about being resilient, it's about securing things, it's about understanding the customer where you need to understand the customer and support them in their time of need. There is a lot more to trust than just security. I think in a lot of places, trust is about really being good at the basic things that companies need to do. And sadly, there are a lot of companies that are trying to shortcut. Those shortcuts create an incident. Like I tell teams internally as well, we are one incident away from being losing, losing trust to our customers, and that's why we need to be really good at the basics, especially all of the data that we collect. The second is being transparent about saying this is what we're doing, this is what we're not doing, these are the things that that we hold our we will hold us up to. And I think the best brands, when I think about brands that manage to be resilient through tough times, again, think about security breaches, all kinds of things, are the brands that were transparent and shared what they know and didn't try to shrug it aside. Are the brands that said, hey, we had a breach.

Jeannie Walters 10:07

Right.

Governing AI With Guardrails

Human Oversight And Confidence Levels

Assaf Keren 10:07

This is what happened, this is what we're doing, this is how we're covering this and how we're making our customers whole. When you think about AI, we're in this interesting space right now where inherently you're in a third-party risk management situation. There is no regulation, there is no law around what is best practice, and the best practice because this is moving so fast, best practices are changing on a day-by-day basis. Doing it in a way that is governed, doing it in a way that is managed, and governed is more inventory. Know who's using it, know what they're using it for, manage it. This is the data it should have access, build guardrails, build um uh data access in the right way, and it goes back to the best in the basics. If you do the basics really well in security and other places, then the AI that you bring on top is going to be integrated well into that versus being a plug-up on the top.

Jeannie Walters 11:15

How do we right now today leverage human leadership, human oversight to make sure that these remarkable tools that are in the beginning are doing really great? How do we make sure that they don't vear off course?

Acting With Intention And Final Thanks

Assaf Keren 11:30

There still needs to be a level of human oversight. We're not yet in the place where the technology is there. What I'm doing is taking the data back into the instruction set and say, hey, this is where you fail on several areas. Let's improve it. So over time, we'll gradually get to a higher level of accuracy. For security in politics, for example, we started working with a company that is doing security operations. So incident triage. And what we're doing is initially we're running humans and the agent side by side, and we're we're letting both of them make decisions. The agent moves faster but makes more mistakes. But then we take back the feedback and we slowly build the context and the understanding of our systems to make sure that it's better. Now, at some point in time, there will be a shift in which people will do less work, the agent will do more work. And that's how we're building our customer experience. One of the big things that we're building with customer experience agents right now is just our ability to build a context that is unmatched in other places. We we pre-build that context for the brand. And I think that that's part of the beauty of what we're building at Qualtrics. There still will need to be a person in the loop in a lot of cases. The beauty and the magic will happen when we'll be able to identify when that per there is a high-risk, low confidence answer and we need to bring a person in versus where there is a low-risk, high confidence answer, we can let the agents just run on its own. And I think that's where the equilibrium will be built. You need to solve an outcome. And if you solve an outcome that is for your customers, and you do it with AI because that's the best tool for the job, you're going to get a lot more back.

Jeannie Walters 13:20

This is kind of coming around full circle because in the beginning we talked about how easy it is to lose trust quickly and how you need to build trust slowly. And I think you just touched on that again. You need to build the context kind of slowly and with intention.

Assaf Keren 13:36

Yeah.

Jeannie Walters 13:36

Um, you can't just turn on the light switch and hope for the best. I know in today's world, you are probably in the midst of collecting feedback, acting on it, and exploring what's possible with AI. And the opportunity is there. It's clear. It's not just about listening. What we need to do is act with intention. We earn trust every step of the way along the customer journey and frankly in any relationship that we have. So a big thank you to my guests, Adam, Kerry, and Assaf. And thanks to you for joining me on this episode. We'll see you next time.

Jeannie Walters 14:15

Thanks for listening to Experience Action brought to you by Experience Investigators. If you're ready to turn insights into action, join our CXI Membership. That's our community for Customer Experience Investigators just like you. It's where CX leaders get the tools, support, and inspiration to move from ideas to true impact. And don't miss my new book, Experience Is Everything: Making Every Moment Count in the Age of Customer Expectations. It's available now for pre-order. Learn more and reserve your copy at experienceinvestigators.com. Until next time, keep asking questions, keep improving, and keep leading with experience.