Arm Me with Harmony: How Changing Your AI's Ad Voice Can Improve Sales

[00:00:01] It is Friday, May 3rd. Today, a special deep dive marketing science episode. You don't

[00:00:07] have to spend long on TikTok or really any video platform these days to run into videos

[00:00:12] selling things using an AI generated voice. On TikTok there are a few commonly used voices,

[00:00:19] but I guess that the marketers making these ads won't spend a lot of time thinking

[00:00:24] about what voice to use. But is it possible that the perceived gender of the AI voice you

[00:00:32] choose could be one of the most important factors in whether you'll get a sale? Or if you market

[00:00:38] a charity, whether you should use one of those trembling voices to evoke empathy, if that

[00:00:44] even works. All of that is what Fotis F.T. Mehl

[00:00:47] set out to find out. He is a research associate at the University of St. Gallen

[00:00:51] in Switzerland and he joins me there now. Hello.

[00:00:54] Hi, Tod. Thank you for having me.

[00:00:56] Not at all. So we're going to be coming back I think to a term used in your paper quite

[00:01:01] frequently. So I want to get that defined first. You refer to the differences in

[00:01:05] AI voices as the vocal tract length. What are we actually talking about here?

[00:01:11] Yeah, that's a really good and common question that I often get. So by vocal tract length

[00:01:17] we are referring to the anatomical structure between the vocal folds and the lips. And

[00:01:24] many people tend to confuse it with pitch but just to clear this already from the beginning,

[00:01:30] when we speak the vocal tract length is filtering the voice and is a longer vocal tract length

[00:01:38] is actually promoting more the lower frequency of the voice so you end up having a more

[00:01:44] deep voice and a shorter vocal tract length promoting a more high, shrill voice. So even

[00:01:52] though pitch is one frequency that we have our voice consists of many different frequencies

[00:01:57] and that's what the vocal tract is doing is filtering all the other frequencies except

[00:02:02] the pitch. Gotcha. All right, let's play some examples

[00:02:05] of this in an AI context so that people know what we're talking about. This is

[00:02:09] the baseline default voice. Especially reasonable traveling Shisan. This is the one tweaked to

[00:02:16] masculine. Especially reasonable traveling Shisan. And this is the one tweaked to feminine.

[00:02:23] Especially reasonable traveling Shisan. Very subtle although to me that last one

[00:02:26] sounds still sounds masculine but maybe maybe effeminate or something but they're very

[00:02:30] subtle differences. Exactly and I think we should make clear that it's already it's

[00:02:36] continuum of masculinity femininity within the same biological sex if you could may say

[00:02:42] for an AI agent but we use a male voice and we manipulate this one vocal parameter

[00:02:49] the vocal tract length in the same male voice. Gotcha. So we mean gender as a continuum

[00:02:56] and masculinity femininity as a continuum within the male sex.

[00:02:59] Okay, those two definitions out of the way what did you set out to study?

[00:03:03] Our main research question was how if we change this one distinct parameter in the voice of

[00:03:11] an AI, how can we influence the human perceptions and also what practical implications this

[00:03:18] could have. And in the beginning we wanted to see what mental image is what people

[00:03:23] envision when they hear this voice so we run a study and we see that people perceive

[00:03:31] the longer vocal tract length voices let's say the more deep voice as an entity they envision

[00:03:40] an entity behind the AI that is a more big that has a higher physicality and by physicality we

[00:03:47] mean the weight and the height so we ask people how do you envision this entity in terms of

[00:03:51] height and weight and they have to choose. Then we also asked them about the perceived

[00:03:56] masculinity of the AI agent and again they tend to perceive more masculine the entities with a

[00:04:02] deeper voice and more feminine the entity with the more shrill, more high voice compared to

[00:04:11] the baseline condition and then we asked them can you match the voice with certain products?

[00:04:18] We asked in the beginning for some food products we also went into the cars we asked

[00:04:22] about car products and we saw that people tend to match more the deeper voices with a more masculine

[00:04:30] product for example we use the beef burger as the masculine food product and an example of a

[00:04:37] masculine food product and a vegan burger as an example of the feminine food product.

[00:04:43] And we saw that this matching between the voice and the product has also downstream

[00:04:48] consequences in advertising so we run a YouTube study and we actually saw that people tend to

[00:04:54] click more on the link when they have a matching between the voice and the product that is being

[00:05:04] advertised. And also this increased the overall advertising effectiveness of the study we run

[00:05:11] the study with cost per thousand impression and we see this costs are calculated by by

[00:05:19] Google ads and we saw that the advertisement campaign when there was matching between the voice

[00:05:25] and the product was cheaper and had a higher click-through rate. Sure and likely cheaper

[00:05:30] because of the higher click-through rate I would imagine the ad quality is better. So does

[00:05:35] this I mean this sort of confirms what my gut would have told me ahead of time right that

[00:05:39] in the other study that you did was around cars you know so for the male voice which is I'll just

[00:05:44] use that terminology for the male voice you paired it with trucks and for the female voice you paired

[00:05:50] it with a smart vehicle and a couple of others. It sounds like you're confirming what I think

[00:05:55] most of us assumed that use a male voice for male products a female AI voice for female products.

[00:06:02] Right there is I would say there are two differences that they are a bit crucial

[00:06:08] in our research in comparison to other research that are in advertising and masculine feminine

[00:06:13] products and all this gender discussion and the difference is that it's always a male voice

[00:06:21] it's just more on the feminine side or more on the masculine side and at the same time is an

[00:06:27] AI voice people can can realize that this is not a human voice but still they tend to make

[00:06:33] these associations and these are the two critical components that I believe differentiate our research

[00:06:39] compared to the to other research where they take a male voice and a female voice because

[00:06:45] a male and a female voice is not only different in terms of frequencies and vocal tract length it

[00:06:51] has another a bunch of other vocal features that are different but in our specific voice

[00:06:58] there is one one vocal parameter that is changing and this is what's driving the effect I see so

[00:07:04] so you were testing slightly different pitches of the male AI voice and yes and again just it's

[00:07:14] not exactly the pitch it's this formant frequencies that I don't want to go too much into

[00:07:18] technicality because the pitch will actually remain the same so we change the other frequencies

[00:07:23] of the voice a voice consists of many different frequencies the pitch was the same the speech rate

[00:07:28] was the same the loudness was the same what changed was these other frequencies in the voice that make

[00:07:33] that give this timber and this color of voice that is more deep and expresses this physical this

[00:07:40] enhanced physicality when the the vocal tract is longer and this more expressed femininity

[00:07:47] in the voice when the vocal tract is shorter right I'm just going to play those two again

[00:07:51] just in case people sort of lost track of it here's number one especially reasonable traveling

[00:07:55] shisan and number two especially reasonable traveling shisan I wonder sort of what other

[00:08:00] effects I mean certainly you you measured cpm you were measured click through did you measure or

[00:08:05] do you have a sense of how this might affect some of the other levers that marketers like to

[00:08:09] pull brand perception purchase likelihood things like that I think it uh it's um it can't position

[00:08:16] a product or a brand into a specific category let's say again and not only this voice I would I

[00:08:23] would like to also expand it to other voices we know that products and brats have certain personalities

[00:08:30] and when they are being advertised or generally they're being promoted with a certain

[00:08:35] persona behind they also tend to map their personalities into into the voice characteristics

[00:08:42] and that's what we also try to do we we found from prior literature this all stereotypically

[00:08:48] masculine food products and we also pre-test some other masculine cars versus feminine cars

[00:08:54] and we we try to map the characteristics of the products into the characteristics of the voice

[00:09:01] and I'm very even though we used only products in our research I'm pretty sure that this can

[00:09:06] apply also to brands that if you want to position your brand as a more in our case as a more masculine

[00:09:13] voice use a longer vocal tract length or you can also adjust it to to the audience the target

[00:09:19] audience that you have another example could be and I mean that's off the top of my head

[00:09:25] that maybe you want to target let's say more elderly people then make the voice a bit slower

[00:09:31] or you want to promote some exciting activities make the voice more exciting something about

[00:09:37] the environment I would say make the voice more warm and that's that's the whole idea behind the

[00:09:43] this line of research that we need to take into consideration the specific vocal features

[00:09:50] and we have to uncover the power that the voice has because the voice is a rich source

[00:09:56] of information we the moment we met probably you made some implicit associations about

[00:10:03] my personality or about my physicality even though we've never met and I think that's normal and

[00:10:08] people are hardwired to do those kind of associations as soon as we met I immediately

[00:10:14] thought that you were a patient person because we spent 10 minutes trying to fix my

[00:10:17] microphone so that was my first impression your paper was done your study rather was

[00:10:23] done in English using English phrases and words for testing do you think we'd see the same effect

[00:10:28] in other languages in other cultures I wouldn't be surprised if we could find cultural differences

[00:10:33] because there are different communication norms that exist between cultures however I would say

[00:10:41] that maybe the part where the vocal tract length links to people envisioning more

[00:10:49] bigger entities like physically enhanced I would say that this part I would expect personally to be the same

[00:10:57] because there are some evolutionary evolutionary bases there with people we also animals and humans

[00:11:05] know when to expect a danger and they when they hear like a roar of a lion they

[00:11:12] we are hardwired to know that this is something big coming this is something

[00:11:16] dangerous coming and I would expect maybe I would expect the second part when we go from

[00:11:22] physicality to perceive masculinity to change across cultures but yeah this is what I would expect

[00:11:29] that we didn't run it across cultures right and I know it's different my wife is a scientist so

[00:11:35] I know that asking scientists for their gut feel is sometimes a little foreign because you

[00:11:39] you prefer to use proven experiments over gut feel and I think the world is a better

[00:11:43] place overall for it when we first stated my wife had gone out to my then girlfriend had gone out to

[00:11:49] a movie and I asked her whether she liked the movie and she said by what criteria

[00:11:54] so yeah I knew I was dating someone different that's right yeah exactly without the ones like

[00:12:00] you who work tirelessly to keep things running everything would suddenly stop hospitals factories

[00:12:06] schools and power plants they all depend on you no matter the weather emergency or time of day

[00:12:12] you're the ones who get it done at Granger we're here for you with professional great industrial

[00:12:17] supplies count on real-time product availability and fast delivery call click Granger.com or just

[00:12:23] out by Granger for the ones who get it done you did another paper recently that tested

[00:12:32] a trembling voice as the narrator for a charity ad and just so that people

[00:12:38] know what I'm talking about here this is a regular read the reality is that a hungry and sick child

[00:12:43] need you right now and this is the trembling version the reality is that a hungry and sick child need

[00:12:49] you right now and this is for a charity and that sounds like that sounds like one of those SPCA

[00:12:55] type you know Sarah McLaughlin sad music kind of things it was another youtube test you measured

[00:12:59] click through is you had four versions one with no trembling one where the tremble was

[00:13:03] only in the information part of the ad one where the tremble was only in the final pitch what we

[00:13:09] might call the call to action and one where the trembling voice was present throat the entire ad

[00:13:14] which one did best when the trembling was either partially or the whole message was with

[00:13:23] with a trembling voice the advertising effectiveness was again higher better compared

[00:13:31] to the baseline again lower costs higher click through rates and that shows that because we

[00:13:37] we also took a dive on the on the mechanism on why why is this happening and we saw that

[00:13:43] people when they listen to this trembling voice they tend to perceive these entities as more

[00:13:50] vulnerable and that tends to increase their empathic concern and and that also where we

[00:13:57] saw in another study not in the advertising study one before that also increased their

[00:14:02] willingness to donate in this charity organization we have to also mention that the message that

[00:14:07] is communicated has a negative balance so that matches also the the emotionality of the message

[00:14:13] much matches the content and and that was actually a very surprising result to see that

[00:14:21] it not only changes these perceptions where you know you have the intuitive feeling that

[00:14:25] okay this will change people's perceptions about the ai entity but it actually has an effect on

[00:14:31] people's emotional state the emotional state of the of the person is changing and that makes them

[00:14:37] click on the link it's so uh i hope you won't take offense to this but it's so manipulative

[00:14:43] isn't it like that the voice yeah it's such a you know it's in a way it almost you know

[00:14:51] when i heard those first um when you sent them into us i thought there's no way people

[00:14:55] would fall for this and it is and actually the i make a bigger point in my paper about it and

[00:15:02] what i mentioned is that imagine that we know from already uh from years ago research happened

[00:15:10] years ago that people tend to treat different devices and robots and computers as social actors

[00:15:17] we know that but it was also i agree like just by adding a bit of trembling in the ai voice

[00:15:23] you just get more of the attention and elicit some emotions that's indeed surprising

[00:15:28] and there is an ethical aspect that i mentioned in in in the paper that imagine having an ai

[00:15:35] working that can work 24 seven can communicate in every language and and also uses this um

[00:15:43] as an example this trembling voice this is something that that is a bit concerning

[00:15:49] and needs some type of new regulations because um and it's not only with a trembling voice now we have

[00:15:55] all these cloned voices where people can can make a clone voice out of a few clips of someone's voice

[00:16:03] so i think there is there should be uh some extra attention to do what the what the voice

[00:16:12] is capable of and what the all this speech synthesis algorithms of today are capable of

[00:16:18] and we should really pay attention to create some new regulations about the the voices we

[00:16:22] should be transparent in my opinion when something like that is being used there should be explained

[00:16:27] why it's being used like that because in my opinion it would be okay to raise some funds for a

[00:16:31] pro-social event but it wouldn't be okay to uh to do some social manipulation in uh uh yeah via

[00:16:41] this voice and you know i mean of course that's that's the direction that this will

[00:16:45] probably go uh you know you could expect to see political ads where if the database tells them

[00:16:50] that that particular potential voter responds well emotionally to happy voices then the candidate's

[00:16:56] voice would be tweaked happy if they would react to someone who sounds fearful they might add a

[00:17:01] trembling a tremble to it it's you know both exciting as marketers and terrifying as people

[00:17:06] it could also have other applications you know like i don't know you call in a customer

[00:17:10] support center and then you have to encounter this trembling voice probably you won't do all

[00:17:16] the complaints you have probably you would be more mild towards this vulnerable person

[00:17:20] and uh and i think it should be given a bit more attention to to the specific

[00:17:25] uh features that are communicated out there you had co-authors on your papers who were they

[00:17:30] um in this the first the vocal tract length paper i have three co-authors my supervisor

[00:17:37] christian hildebrand my co-supervisor manuel de bellis and um former postdoc at my lab

[00:17:44] currently working in at in the industry uh william hampton and in the second paper it's only me

[00:17:50] and my supervisor and with my uh production coordinator right in saying in briefing me that

[00:17:54] you just received your phd yeah a few months ago i received my phd and actually at the moment

[00:18:00] i'm looking for some research and behavioral science and marketing positions in industry i decided to

[00:18:08] to live academia um and i'm looking for for my new uh role in industry this time how can

[00:18:16] read people reach you if they wanted to and they can reach me through my linkedin it's 40 70

[00:18:22] me if they managed to to spell correctly the uh this greek surname and uh i would be very happy

[00:18:30] if they read my emails and my papers and they're in in my papers there is my email so i would be

[00:18:36] happy to receive also questions comments or any other feedback that people have regarding my papers

[00:18:43] we will put your linkedin and a link to the papers in our show notes dr f the meal thank you

[00:18:48] so much for your time thank you very much fodus f the meal is the co-author of two papers the first

[00:18:53] one is published in the journal of interactive marketing it's called the power of ai generated

[00:18:58] voices how digital voice tracked length shapes product congruency and ad performance the second

[00:19:05] one in the i triple e transactions on effective computing journal that one is called empathy by

[00:19:10] design the influence of trembling ai voices on pro social behavior all right that'll do it for

[00:19:18] the week don't forget to follow us on social media you can find all of our links at today in

[00:19:22] digital dot com slash social or you can tap the link at the top of the show notes today in digital

[00:19:27] marketing is produced by engage q digital on the traditional territories of the su neymik first

[00:19:32] nation on vancouver island our production coordinator is sarah gild our theme is by mark

[00:19:36] bluvus ad coordination by red circle i'm todd maffin have a restful weekend and i will see you on