How can an organization evaluate what their customers think about them, and why does that matter? We discuss how a survey could help to win awards and give your business a boost.

Jim: 0:00

Hi and welcome back. In fact, it is a welcome back to Chris Robinson. Who's come to talk to us about evaluation, Chris. Welcome back. Nice to see you again.

Chris: 0:09

Thank you very much. It's great to be back. Thanks to.

Jim: 0:12

Now you were saying that you were helping companies with their awards, but also you've got another part of your business, which is evaluation. What problem you're solving Chris we're business owners with this other business that you've got.

Chris: 0:24

Well based the part that does awards beginning in 2006. And what happened is we found that to win awards, you had to do surveys and research and you couldn't win an award just with the lovely purple prose you had to back up your claims with evidence may better, quick way to get evidence of happy customers, happy staff, happy suppliers, happy community, then a survey and a well-designed survey won awards. So we found that. Nothing won awards like a survey, but we also found when, especially working with L and D and customer contact. Teams nothing made them do a survey, which they'd be meaning to do for ages, but suddenly they had to it that it, because they wanted an award. So the two things worked really well, but because we started up just before the recession, we found that a lot of people didn't want to be seen to hire a marketing agency. And so we set up a new brand, a new legal entity called Boost Evaluation limited. And what was great is it actually. Grew legs and you know, there's 22 people in Boost, and a chunk of them work on surveys and research and a chunk of them, right. Award entry. So my two things marry up, but ,I want to talk to you with my evaluation hat on today in particular.

Jim: 1:38

Love to hear that. So tell us, how does a company go about doing evaluation so that, so it's meaningful because there's no point in having an evaluation that just says we're great that doesn't manifest itself as a, as a press release or anything. So how do you get it? So that it's well balanced and also somehow giving insight into what could be better because that's also another function. Isn't it of evaluation.

Chris: 2:02

Yeah, absolutely. And it's very interesting you phrase it like that because often because of our heritage in awards, people would say, I want to win an award. So how do we prove that we're amazing and you do a survey and if you cherry pick the best news and strip out the not so good news and, and pick out all the really glowing testimonials, then you can package it up and without. You know, distorting the truth. You just focus in on the good news. And often what we found is that people loved the fact that you put all the good news in, into a narrative, whether it's a case study or an award entry, and it felt great, but what they often forgot to do. Is what I think is more important within evaluation and why this is relevance on a, on a podcast that focuses on marketing and PR is the fact that you need to listen and that will improve your strategy, improve your future and make some important decisions. And that's often forgotten because they're so busy reflecting on how awesome the past was. They forgot to inform their future. So what I was the one I was listening to a fantastic podcast and I was walking the dog and I thought, you know, this is something I'd love to share. We, as a business family, entire awards industry was rubbish at doing this. So we ran a survey, got 330 people who enter awards to tell the awards industry collectively. What they wanted because the whole industry was doing totally different thing, guessing firing at the hip. And we got a bunch of awards organizers on a call and we said, this is what people want. And for them, it was an epiphany. And for us too, because we're in this industry and, and it made me think, God, if we hadn't have done that, no one would have done it. And the whole industry would continue firing from the hip and we're doing it with we're working with a recruitment company, but certainly one to enter an awards. And I was. And, and they stopped entering awards and that focusing over energy on a customer survey, which is that she got to go out and we're working with some training departments who are all about, we have had to reinvent training for the last year. What did we do? Well, what do people want to go back to, but what they want to stick with now, we've done it. Do they like digital learning? And you know, a lot of companies have had to reinvent themselves and come up with entirely new propositions and they need to learn from the process. And not just go back to the status quo and, and find it. But I actually find that the feature is probably different, but in what way, and we've seen that. You need to reinvent yourself, but how, and you, and, you know, I had fees, I had a light bulb experience and I've watched up as habit and I wanted to share through your fantastic podcast. It encouraged other people to do the same thing.

Jim: 4:47

Chris. Thank you for the compliments. And I was listening to a podcast about the daily brewer podcast and newsletter, and they were talking about digital metrics and how people are looking at all the insights from Google ad words and all the trackability. But that's only given them historical data and that they are even suggesting get on the phone to a sample of customers and talk to people. So can you just share with us Chris evaluation? How

Chris: 5:19

do you do it? Well, telephone is, is one mechanism. And sometimes people default to that. It's very difficult to shell a phone call or get through to someone it's quite an intrusive mechanism. It's often easier to begin with a digital survey there's laser platform serving my kids the most popular one. There's plenty of others. There's more expensive ones like Qualtrics. Is there some. Various small needs providers as well. But survey monkey is a good starting point. And often you start with additional survey, but you can invite people at the end to say yes, Can we contact you by phone? And so it's less of a, an intrusion. It's also it's as a mechanism, it is very effective, but I would always encourage you to put in an incentive. So a big part of the, how to answer your question, there needs to, to get as many responses as possible. So incentivizing a survey. We'll get you a higher response rate, but it also gets you a balanced response cause they're responding because they want the incentive. Whereas if you don't put in an incentive, you end up with what we call the Marmite effect, where people respond because they have a strong view either. They love you or they hate you. And that's more likely to make them want to tell you that view. So you get this imbalanced response. So digital surveys. Yes. But forums panel conversations are great as well. Zoom meetings are brilliant for that individual telephone calls, but in terms of how always bear in mind, you've got two aspects. You've got qualitative, which is a story of your anecdotes, adding color to your research. And you have a quantitative too, which is where the survey you've got to have numbers to. So a bit of both, a bit of numbers and a bit of color added to that through. Qual and quant. There's also two other bits of jargon I'd like to share with you. One is our protective measurements and the other is subjective measurement. Now subjective measurement are questions that you can only measure through a conversation through a survey. And so obviously if you're going to make a phone call and a survey, you need to focus on them. If you have a discussion with your colleagues about begin the whole research exercise in terms of your question of how right at the front of this is a hypothesis discussion, what is it we believe to be true? What is it? We don't know. What is it? We know exists, but we don't know how much or how little and you just brainstorm all the questions you want to answer the hypothesis you want tested. And then you come up with a list of questions and those are the things that you split into. This is a subjective question. It's about feeling and emotion and, and better sense of the future, their votes, that experiences the past visa sense things are subjective. Stick them in a survey or a phone call when you get objects, if data, these are things that defy interpretation and that's what you were describing there. These are data that you can get from Google analytics or from, you know, You can get objective, which is somewhere between the two, like Trustpilot scores, but it's a reflection of the past and they're just tracking data points. So when you're coming up with your evaluation strategy, start off with a discussion about what are we trying to prove? What are we testing? What are our hypotheses? What did we not know that we need them to vote on? Come up with that list? Divided into qualitative and quantitative and subjective and objective. So it's four different bits of jargon, but it is a process what you must never do in the, how is just dive in and just write a survey and check it out because you'll come back to it. I think God, I wish I asked a different question or I should have phrased it differently or it shouldn't have been an agree to disagree. Like out question, it should have been a vote or a scale. So that's the process. Okay.

Jim: 9:08

I love the idea that you have to have a hypothesis to kind of prove or not prove. First rather than just generally going out. What about simply the duration? Is that kind of some sort of magic number or of either questions or duration?

Chris: 9:23

That's a very good point. And and yes, dropout is a major problem. Yeah. Bain and co published a book called the ultimate question and their philosophy is that it should be one question. And that question is the most popular question. And everyone's seen it a million times. It's on a scale of no, it's 10. Would you recommend us to your friends? And so that's called the net promoter score. It's the most popular question out there. And it's one question and the whole philosophy is you just ask one question, however, I think that's overdoing it a bit. If we've done a facilities management did a massive exercise. Huge huge survey. It was, they were going to think as a thought leadership off the back of it. So it was getting them to say whether they've done it or haven't done it. If they haven't done it, then they got questions about what I thought were the barriers and the excitement and the ROI. If I have done it, they reflected on what works and what didn't work. There's a massive complicated this multipath survey. And yes, it was, we would time it. And we'd say that's a 10 minute exercise on average. So make sure you set expectations. This is a 10 minute survey, but in return. You'll be in with the council winning an iPad. So you've got a, a bit of a what's in it for me. People will give you a lot more time if there's a possible iPad or instead of a small chance, the big prize, you have a high chance of a small price. So you can have, we've got 10 30 pound Amazon vouchers, or even smaller prices. Everyone who answers gets a free cup of coffee through Bates.

Jim: 10:54

The other aspect or strategy seems to be, to have multiple surveys a bit like a funnel. So have you experienced in having maybe sort of a short survey where there, would you be willing to answer more questions? What about that, Chris, this sort of ongoing survey where you're funneling people into becoming almost sort of a shortlist focus group, is that energy that you've experienced with as well? Yeah, there's actually two pubs, which surveying and companies often do both. If you're in business to business, which I think a lot of your listeners are, then you tend to fall into the once a year or once every half year staff serving exercise where you.

Chris: 11:35

If all your customers at one instance in time, and that is a different approach to the other strategy, which is the continuous customer monitoring. If you work in a call center and you got one of your massive call centers in Dublin, for example you want to track daily by call center agent by department. The net promoter score on a call by call basis. So you do, but those classic text messages that you get and that's, and there, you would do your one or two instant questions that the vital questions. And then you'd say, can I ask you a few more? And you'd get like a 90% drop off rate, but you've captured the critical 10%. And those are two approaches and they both work, but it's, it's goes back to the, what are we trying to prove question at the start?

Jim: 12:23

You've also raised an issue there of the sample size. I read somewhere that Ms. Sample sizes, as smallest 30 respondents can give you an indication of what the overall universe is going to think. And I noticed this, like, for example, some of the American election surveys were kind of a hundred people. Is there some experience of what creates. Enough of a sample size to make it valid.

Chris: 12:54

There are mathematical models you can use. It's one side, you put in the variables and it tells you the degree of accuracy off the back of it and, or the required sample size to hit your targets degree of accuracy. And now that very often in B2B, you have a small universe, you have 50 odd people. So how do you make that more accurate? The trick that we recommend is having a 10 point scale because we find that statistics, but whatever surveying techniques you use, whether you're serving 10,000 people or 10 people, you'll find that you get a wobble of this statistic. You keep refreshing the survey and you're monitoring it and it wobbles on what was and what wasn't and what will become smaller, smaller, smaller, over time. Until it becomes rocks study and no matter how many more responses you get, it's really not going to change much. If you have a binary question, would you recommend it to a friend? Yes or no? The wobble lasts a lot longer because one person can have a big influence on it. If you have a 10 point scale, the wobble settles down a lot faster. So if you're doing a smaller group using a 10 point scale like net promoter score, but using it throughout the maths suggests you get it stabilized as much quicker in terms of the number of people. It depends on your deep dive, into the data. You might be that you, you are looking into splitting the data into new customers, long-term customers and sort of. Slice and dice for data like that. In which case you've got to look at what is the smallest group. And I would suggest that a statistic isn't sound unless you get at least 10 responses. But if I was doing a survey in PR principles, a magazine or a newspaper, wouldn't be interested in it if you wanted to publish it. And I said it was 500 response in terms of your customer survey. If you get an 80% response rate, It doesn't matter how big your customer survey is. That is representative. If you've only got 50 customers and 40 of them took part in the survey, That's the truth. So it, I broached your question a little bit then, but there are techniques to make it more accurate and equally it depends on the context. Yeah,

Jim: 15:03

What about the survey with your suppliers, for example, and your partners and also your staff, because in speak PR, which about three different audience groups, critic, your external customers, but also your staff and your suppliers, your partners can make a big difference. What's your experience of sort of evaluation there

Chris: 15:25

yeah. Interesting. Employee surveys. There are a lot of debates. One is about anonymity. With dealing with a Swiss telecoms company and they are quite upfront and say, it's not an anonymous survey. We know exactly who you are. And that's just our culture and that's just how it is. Whereas a lot of organizations, particularly organizations where there might be a union involved anonymity is essential. Also. If like we were doing a compliance related survey, you know, asking people about their behaviors to do with compliance, we're doing that with one client. You really don't want to. Give the person, the impression, but it'll get back to their boss that they've admitted that they're likely to discuss the company the at a dinner party. So you're asking sense of questions and our name is a big thing there. And staff surveys. You want to give anonymity with customers, surveys, you, you want to know who says what? And You don't need to worry about anonymity that much, you don't have to say, don't worry. It's completely anonymous. Say what you like. So they're different in that respect. And we've customers. I would also encourage people to have a box saying, if we can quote you tick this box here, because they might give you a fantastic testimonial, which he desperately wants on your homepage. And rather than go back to them and say, can I have a sign off? If they just tick a box saying. Can we quote you, you can actually harvest loads of snappy, spontaneous quotes. And I mean, you're a PR professional, you know how hard it is to get a testimonial out of people. And then when you do it's, it feels like a PR testimonial, whereas sometimes you'll dream snappy, but sort of funky soundbite. You might get on a film poster. You're more likely to get out of the survey where people are actually being really brief. It's like, Oh, wonderful company. Love working with you. Oh yeah. I don't mind you quoting me. And so it's very different, but staff surveys, I would encourage you be anonymous, truly anonymous. That's why you know, this is partly why we're kept in business, because we can say we're using a third party. Your, your responses will not be shared with anyone within the business. But yeah, anonymity is a debate it's worth having make decisions, because if you do that on empty, you lose the ability to send out reminders, accurately, and also to attribute problems. And there is a gray area. Sometimes you want to know which department they're from, but you just don't want to know their name so that you can go back to the manager and say, you got bit of an issue, man.

Jim: 17:57

Chris, what sort of costs are we looking at? Can you give us some idea, even just, what does it cost for MailChimp?

Chris: 18:03

Well seven monkey bar are free licenses so that you can just sign up. It'll be peppered with adverts and very limited in the number of people you can send it to. But equally there's a whole grading scale of how much he wants to pay. So if you're happy to pay two, 300 pounds, you can get. I think it's a nice, decent license and lose the branding and have lots of questions you did. It's an American system. So you do have to adhere to American CAN-SPAM laws and you can't use the mailing system to spam thousands of people. But Southern monkey is a great option. Very, very cheap there's loads of equivalents out there. And yeah, but in terms of hiring a consultancy to do it, you know, we just charged consultants day rates. And so, you know, Sometimes you can knock out a survey for 500 quid, but then on the other side of the spectrum is often a five figure fee for a two year long evaluation strategy. So it, it is a bit of a piece of string.

Jim: 18:57

Great. Well, it sounds like the measure is whatever you have as your original hypothesis, that sort of creates the, the scale of the survey. Chris, if he wants to find out about you, where can they find you?

Chris: 19:08

Our website is boost-evaluation.co.uk. And it's services are listed there

Jim: 19:15

Thank you so much, Chris Robinson for joining me today on the speak PR podcast.

Chris: 19:18

Lovely talking to you, Jim. Thanks for having me back.

Jim: 19:21

thanks for listening to this episode. And so we wish you the best of health, a sustainable business, or a profitable one. And if you're in business, just listen and do some evaluation of your customers, but I also encourage of your team and your partners so that you can get some guidance into what they need from you going forward as we get out of lockdown. Thanks so much for listening to this episode of speak. Yeah.