A lot of marketers still have questions about usability testing. Is it necessary? Is it useful? What is the purpose? Is it difficult to implement? If it’s your case, here are five facts about usability testing that might interest you… and convince you that usability testing is relevant! Each is based on recent data from eMarketer (detailed sources below). Read more
Last December, we used mobile eye tracking to identify good practices to grab consumers’ attention in an SAQ store. We worked with SAQ and OPTECH to see what grabs clients’ attention in the aisles and how product positioning impacts choice. Our study also had an online portion where we asked consumers to select the same products on SAQ’s online store. In this post, we will focus on online shopping experience based on results from the Web portion of our study. Our objective: find what grabs consumers’ attention online and how online and in-store experiences compare. Read more
In 2016, we worked with Houghton Mifflin Harcourt (HMH), a global learning company, to conduct usability tests in Texas, USA. HMH specializes in educational content, services and cutting edge technology solutions for children. The company serves more than 50 million students in over 150 countries. For this project, we adapted our usual approach of usability testing. Our first challenge was working with very specific users, children. The second was that tests took place in Texas, far from our Canadian facilities. Ultimately, we analyzed the recordings in Montreal to provide actionable recommendations while HMH designed, moderated and recorded the tests in Texas. We combined our expertise in user experience with HMH’s expertise in providing quality educational tools to help children achieve great results. Here is how the project went and what we found out… Read more
Brandy to add a twist to an eggnog recipe, vodka to create a festive punch, champagne to welcome 2017… We rarely run out of options for Holiday toasts. We recently collaborated with SAQ and OPTECH, a centre for applied research and technology in optics/photonics, to study in-store consumer behavior. We used mobile eye tracking to measure where consumers were looking while shopping at the SAQ to plan a Holiday party. Our objective: find out what grabs clients’ attention in the aisles and how product positioning impact their product choice. Read more
Usability testing, also known as UX testing, is one of the best tools to get your user’s pulse. The basics of usability testing are simple: we sit a user in front of a website and ask him or her to complete specific tasks. Meanwhile, a moderator collects impressions and comments. Obviously, usability testing is a little more complicated than that in practice. Here is a summary of the five steps that make a great usability test. Read more
We test websites because it’s still the best way to directly collect users’ point of view. It’s often the only way organizations can see how people actually use their website. For that alone, usability testing is essential, even in its simplest form.
Website testing, also known as usability testing, keeps getting better and better. Modern methodology and technology make usability testing more objective and precise than ever. We can now measure attention, attraction and arousal to better understand how users interact with a website. We can also track something we haven’t discussed much in our blog until now: cognitive load. Read more
First, let’s define the two methodologies:
Card sorting: research methodology used to better understand how users group concepts by having them sort, categorize and group cards on which are written information describing an interface’s contents.
Tree testing: research methodology used to test a given architecture by having users complete tasks through navigation in a menu structure
Now, let’s move on to 6 simple principles that will help aspiring usability experts better understand those two very useful methodologies. Read more
The internet or the World Wide Web was a huge breakthrough in the late 80s and early 90s; connecting computers changed the way we live. A few years later, connecting people became possible thanks to the cloud and mobile devices. This broadened our horizons and revolutionized our world. Today, something even greater is happening. We are connecting EVERYTHING, and this is what the Internet of things (IoT) is all about. We now use the technologies of the previous stages — the Internet, cloud, and mobile — and combine them with sensor technology in order to connect anything, anywhere in the world. Analysts predict that the IoT will consist of about 26 billion interconnected devices by 2020. Compare this to 7.3 billion units for PCs, smartphones and tablets combined together. What is more, the technology involved is accessible to all.
So there is a huge potential for business big and small. Read more
What happens when we combine traditional research and new technology to measure advertising effectiveness? To put this to the test, we worked with Leger to evaluate video ads with actual consumers. Specifically, our study monitored respondents’ facial expressions while they viewed video ads. This was a logical next step to our previous study measuring emotional reaction to four Christmas ads in 2015.
Video ads from Toyota, RE/MAX, Kijiji, Banque nationale, St-Hubert, Proprio Direct, Desjardins, Honda, Nutrilait and Aldo were shown to a sample of 200 consumers recruited through the Legerweb panel. Ads were chosen to provide an overview of advertising effectiveness in multiple industries. Most are 15-second ads (or-30 second for Aldo), short enough to be shown before online content.
Ads from St-Hubert, Desjardins and Nutrilait are the only ones to generate a globally positive emotional reaction. Reaction to Toyota, Honda and RE/MAX ads was almost neutral while ads from Banque nationale, Proprio Direct, Aldo and Kijiji caused negative emotions. By taking a closer look at our three winning ads, you’ll notice they have common characteristics.
St-Hubert’s ad “Sugar Sammy aime la variété” (Sugar Sammy likes variety) generated the most positive emotional reaction and ranked first in our survey. Respondents liked this ad the most and thought it would be the most enjoyable pre-roll ad (imposed before viewing online content). It also had the most favorable impact on their attitude towards the brand and was the most likely to be shared on social media.
Emotional summaries show that St-Hubert’s ad generated the highest proportion of joy. However, emotional journey, which relates to the positive or negative emotional variation (valence between -1 and 1 where -1: 100% negative expression, 1: 100% positive expression and 0: neutral expression), remains the most powerful predictor of ad appreciation. We noticed a positive rise of the emotional state in the last nine seconds starting when Sugar Sammy’s mother enters the room yelling. Emotion reaches a positive peak in the last five seconds of the video when butter chicken is mentioned.
Nutrilait’s ad “Boire à meme la bouteille de lait” (Drinking from the milk container) follows St-Hubert closely. It ranked second for ad appreciation, attitude as a pre-roll ad (pleasant or unpleasant), impact on the opinion toward the brand (favorable or not) and intention to share on social media platforms.
As in St-Hubert’s ad, there is a positive rise of the emotional state between 6 and 14 seconds starting when the little girl screams. Here, emotion drops in the last second of the video when Nutrilait’s slogan is presented. Such was not the case for St-Hubert.
Globally, Desjardins’ ad “Offre verte – Maison” (Green Program – Home) generated a more positive emotional reaction than Nutrilait (average valence of 0,035 vs. 0,022), but did not perform as well in the survey. RE/MAX and Proprio Direct outranked it for impact on the opinion toward the brand and intention to share on social media. It ranked third behind St-Hubert and Nutrilait for appreciation and perception as a pre-roll ad.
Despite a proportion of joy similar to St-Hubert (15% vs. 17%), positive rise of emotional state is not as intense for Desjardins as it is for St-Hubert and Nutrilait. There is a light rise between 5 and 10 seconds before stabilization in the last five seconds. In fact, emotion starts rising when the green roof is mentioned and stabilizes after the black roof is mentioned to conclude the joke. The video ends with a stable and positive emotional state while Desjardins’s Green Program benefits are presented.
We were not surprised to find a positive correlation between average emotional valence and appreciation of a given ad. Emotional journey is the most revealing indicator of effectiveness. Our statistical analysis show that emotional valence during the last ad segment is almost perfectly correlated with the ad’s appreciation, intent to share on social media and perception as a pre-roll ad. I told you previously that our three winning ads had something in common. Well, they all show positive variation of the emotional state in the last segment of the ad.
Ending on a positive AND rising emotional state is what distinguishes St-Hubert’s ad from the others. During the last five seconds, St-Hubert had an average valence of 0,181, compared to 0,087 for Nutrilait and 0,083 for Desjardins. We thus learned that the more positive the emotions, especially in the last few seconds of the ad, the more it may be appreciated, shared and accepted when shown before online content.
After analysis, we defined five best practices to create effective video ads.
You would like to know more about this research or emotions measurement? Contact alexander.tellier@imarklab.com and we will be happy to provide the full report and discuss the results! To test and challenge our research methods, we studied various industries and came up with very interesting insights. Now, imagine all the possibilities if we apply this technology to your brand!
***
In 2017, digital ad spending share will surpass television for the first time in the United States (eMarketer, 2016). Video ads are more present than ever with the popularity of pre-roll advertising. You probably already experienced this form of advertising where you are “forced” to watch a short video ad (usually less than a minute) to access different forms of online content such as a YouTube video. The purpose of our research was to integrate facial analysis in measuring the appreciation of a video ad. Specifically, we wanted to measure emotions objectively. By combining Leger’s expertise in conducting surveys with imarklab’s experience with new technologies, we had everything in hand to measure advertising effectiveness.
An online survey was conducted by imarklab and Leger from April 4th to 11th 2016 with a sample of 200 consumers 18 years or older. The sample was split into two groups that each viewed five of the ten ads. Results were weighted based on demographic profile to allow comparison between the groups. During the viewing of each ad, respondents’ facial expressions were captured with a webcam to be analyzed by FaceReader software. After viewing, a series of questions were asked to measure the appreciation of the ad, impact on the opinion toward the brand, intention to share on social media platforms and perception as a pre-roll ad showed before accessing different forms of content on the Internet (e.g., YouTube video).
To learn more about FaceReader software and the theory behind facial expression analysis, see: “Les expressions du visage comme source d’information” (in French).
At imarklab and Leger, research is our core business. As passionate researchers, we like to continually investigate our own research methods. We have recently been interested in emotions measurement through the FaceReader software. This technology is used in academic research by the Tech3Lab of HEC Montreal. In our case, we use this tool to track users’ emotional reactions when browsing a Web or mobile interface. We used it in a political context this fall to analyze the facial expressions of the federal parties’ leaders. This time, we studied emotional reaction to advertisement. Read more